Michael Nelson - Statistics in Nutrition and Dietetics

There are no simple answers to these questions. We can conclude that any system that we use to evaluate a hypothesis must take into account the possibility that there may be times when our hypothesis appears to be false when in fact it is true (and conversely, that it may appear to be true when in fact it is false). These potentially contradictory results may arise because of sampling variations (every sample drawn from the population will be different from the next, and because of sampling variation, not every set of observations will necessarily support a true hypothesis), and because our measurements can never be 100% accurate.

A finite amount of experimentation can disprove an exact hypothesis.

It is easier to disprove something than prove it. If we can devise a hypothesis which is the negation of what we believe to be true (rather than its opposite), and then disprove it, we could reasonably conclude that our hypothesis was true (that what we observe, for the moment, seems to be consistent with what we believe).

This negation of the hypothesis is called the ‘null’ hypothesis. The ability to refute the null hypothesis lies at the heart of our ability to develop knowledge. A good null hypothesis, therefore, is one which can be tested and refuted. If I can refute (disprove) my null hypothesis, then I will accept my hypothesis.

A theory which is not refutable by any conceivable event is non‐scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice. [1, p. 36]

Let us take an example. Suppose we want to know whether giving a mixture of three anti‐oxidant vitamins (β‐carotene, vitamin C, and vitamin E) will improve walking distance in patients with Peripheral Artery Disease (PAD), an atherosclerotic disease of the lower limbs. The hypothesis (which we denote by the symbol H 1) would be:

H 1: Giving anti‐oxidant vitamins A, C, and E as a dietary supplement will improve walking distance in patients with PAD. 8

The null hypothesis (denoted by the symbol H 0) would be:

H 0: Giving anti‐oxidant vitamins A, C, and E as a dietary supplement will not improve walking distance in patients with PAD.

H 0is the negation of H 1, suggesting that the vitamin supplements will make no difference. It is not the opposite, which would state that giving supplements reduces walking distance.

It is easier, in statistical terms, to set about disproving H 0. If we can show that H 0is probably not true, there is a reasonable chance that our hypothesis is true. Box 1.2summarizes the necessary steps. The statistical basis for taking this apparently convoluted approach will become apparent in Chapter 5.

1 Formulate the Hypothesis (H1)

2 Formulate the Null Hypothesis (H0)

3 Try to disprove the Null Hypothesis

Some studies are observational rather than experimental in nature. The purpose of these studies is often to help in the generation of hypotheses by looking in the data for relationships between different subgroups and between variables. Once the relationships have been described, it may be necessary to set up a new study which is designed to test a specific hypothesis and to establish causal relationships between the variables relating to exposure and outcome. For example, Ancel Keys observed that there was a strong association between the average amount of saturated fat consumed in a country and the rate of death from coronary heart disease: the more saturated fat consumed, the higher the death rate. Numerous studies were carried out subsequently to test the hypothesis that saturated fat causes heart disease. Some repeated Ancel Keys’ original design comparing values between countries, but with better use of the available data, including more countries. Other studies compared changes in saturated fat consumption over time with changes in coronary heart disease mortality. Yet other studies looked at the relationship between saturated fat consumption and risk of heart disease in individuals. Not all of the studies came to the same conclusions or supported the hypothesis. It took some time to understand why that was the case.

The Dodecahedron: ‘If you hadn’t done this one properly, you might have gone the wrong way’.

When designing an experiment, you should do it in such a way that allows the null hypothesis to be disproved. The key is to introduceand protecta random element in the design.

Consider some research options for the study to test whether anti‐oxidant vitamins supplements improve walking distance in peripheral arterial disease (PAD). In the sequence below, each of the designs has a weakness, which can be improved upon by introducing and protecting further elements of randomization.

1 Choose the first 100 patients with PAD coming into the clinic, give them the treatment, observe the outcome.Patients may naturally improve with time, without any intervention at all. Alternatively, there may be aplacebo effect (patients show improvement simply as a result of having taken part in the study because they believe they are taking something that is beneficial and alter their behaviour accordingly), even if the treatment itself is ineffective.

This is a weak observational study. Introduce a control group which receives a placebo.

1 Allocate the first 50 patients to the treatment group, the next 50 to the placebo group.If the two groups differ by age, sex, or disease severity, this could account for apparent differences in improvement between the groups.

This is a weak experimental study. Introduce matching.

1 Match patients in pairs for age, sex, disease severity; assign the first patient in each pair to receive treatment, the second patient to receive a placebo.

The person assigning patients may have a subconscious preference for putting one patient first in each pair. Does the patient know which treatment they are getting?

This is a weak placebo‐controlled intervention trial. Introduce randomization and blinding.

1 Allocate patients to treatment or placebo randomly within pairs. Make sure that the researcher does not know which patient is to receive which treatment (the researcher is then said to be ‘blind’ to the allocation of treatment). Make sure that the patient does not know which treatment they are receiving (keep the patient ‘blind’ as well). This makes the study ‘double blind’.

2 Conduct a placebo‐controlled randomized double‐blind intervention trial.

‘Randomization properly carried out is the key to success.’

(Sir Ronald Fisher)

Intervention studies of this type are often regarded as the most robust for testing a hypothesis. But sometimes randomized controlled trials are not ethical, especially if the exposure may be harmful (e.g. smoking, or increasing someone’s saturated fatty acid intake), or it is not possible to blind either the subjects or the researchers because the treatment being given is so obviously different from the placebo. In these cases, it is possible to mimic intervention studies in samples that are measured at baseline and then followed up over months or years. These types of studies raise issues about how to deal with factors that cannot be controlled for but which might affect the outcome (e.g. in a study looking at the impact of diet on risk of cardiovascular disease, the influence of smoking and diabetes) and changes in the general environment (e.g. indoor smoking bans) that have the potential to affect all the subjects in the study. Complex statistical analysis can be used to cope with some of these design issues. The ability to test the hypothesis in a robust way remains.