Michael Nelson - Statistics in Nutrition and Dietetics

Karl Popper puts it this way:

We can learn from our mistakes. The way in which knowledge progresses, and especially our scientific knowledge, is by unjustified (and unjustifiable) anticipations, by guesses, by tentative solutions to our problems, by conjectures. These conjectures are controlled by criticism; that is, by attempted refutations, which include severely critical tests. Criticism of our conjectures is of decisive importance: by bringing out our mistakes it makes us understand the difficulties of the problems which we are trying to solve. This is how we become better acquainted with our problem, and able to propose more mature solutions: the very refutation of a theory – that is, of any serious tentative solution to our problem – is always a step forward that takes us nearer to the truth. And this is how we can learn from our mistakes.

From ‘Conjectures and Refutations. The Growth of Scientific Knowledge’ [1].

This is a very compassionate view of human scientific endeavour. It recognizes that even the simplest of measurements is likely to be flawed, and that it is only as we refine and improve our ability to make measurements that we will be able to develop laws which more closely approximate the truth. It also emphasizes a notion which the atomic physicist Heisenberg formulated in his Uncertainty Principle. The Uncertainty Principle states in general terms that as we stop a process to measure it, we change its characteristics. This is allied to the other argument which states that the observer interacts with the measurement process. Heisenberg was talking in terms of subatomic particles, but the same problem applies when measuring diet, or blood pressure, or even more subjective things like pain or well‐being. Asking someone to reflect on how they feel, and the interaction between the person doing the measuring and the subject, has the potential to change the subject’s behaviour and responses. This is contrary to Newton’s idea that measurement, if carried out properly, could be entirely objective. It helps to explain why the discovery of the ‘truth’ is a process under continual refinement and not something which can be achieved ‘if only we could get the measurements right’.

Consider the question: ‘What do you understand if someone says that something has been proven “scientifically”?’ While we might like to apply to the demonstration of scientific proof words like ‘objective’, ‘valid’, ‘reliable’, ‘measured’, ‘true’, and so on, the common meanings of these words are very difficult to establish in relation to scientific investigation. For all practical purposes, it is impossible to ‘prove’ that something is ‘true’ . This does not allow you to go out and rob a bank, for example. An inability on the part of the prosecution to establish the precise moment of the robbery, for instance, would not excuse your action in a court of law. We cope in the world by recognizing that there is a substantial amount of inexactitude, or ‘error’, in all that we do and consider. This does not mean we accept a shop‐keeper giving us the wrong change, or worry that a train timetable gives us information only to the nearest minute when the train might arrive before (or more commonly after) the advertised time. There are lots of ‘gross’ measurements that are good enough for us to plan our days without having to worry about the microdetail of all that we see and do.

For example, it is very difficult to describe an individual’s ‘usual’ intake of vitamin C, or to relate that person’s intake of vitamin C to risk of stroke. On the other hand, if we can accumulate sufficient evidence from many observations to show that increasing levels of usual vitamin C intake are associated with reduced risk of stroke (allowing for measurement error in assessing vitamin C intake and the diagnosis of particular types of stroke), it helps us to understand that we can work with imprecise observations and laws which are not immutable. Moreover, it is important (for the sake of the growth of scientific knowledge) that any belief which we hold is formulated in a statement in such a way as to make it possible to test whether or not that statement is true . Statements which convey great certainty about the world but which cannot be tested will do nothing to improve our scientific understanding of the world. The purpose of this book, therefore, is to learn how to design studies which allow beliefs to be tested, and how to cope with the imprecision and variation inherent in all measurements when both collecting and analyzing data.

In science, we rely on logic to interpret our observations. Our aim is usually to draw a conclusion about the ‘truth’ according to how ‘strong’ we think our evidence is. The type of observations we choose to collect, and how we collect them, is the focus of research design : ‘How are we going to collect the information we need to test our belief?’ The decision about whether evidence is ‘strong’ or ‘weak’ is the province of statistics : ‘Is there good evidence that our ideas are correct?’ As Sherlock Holmes put it, ‘It is a capital mistake to theorise before one has data’. 3

There are two types of logic commonly applied to experience.

The aim with inductive logic is to infer a general law from particular instances: arguing from the particular to the general. This type of logic is good for generating new ideas about what we think might be true. It is less good for testing ideas about what we think is true.

Case studies provide a single example of what is believed to be true. The example is so compelling by itself that it is used to infer that the particular instance described may be generally true. For example:

A dietitian treating a severely underweight teenage girl worked with a psychotherapist and the girl’s family to create an understanding of both the physiological and psychological basis and consequences of the disordered eating, resulting in a return to normal weight within six months. The approach contained unique elements not previously combined, and could be expected to have widespread benefit for similar patients.

A case study can be interesting and provide a powerful example. But it provides very limited evidence of the general truth of the observation.

Descriptive studies bring together evidence from a number of related observations that demonstrate repeatability in the evidence. For example:

In four old peoples' homes, improved dining environments using baffles to reduce noise interference and allowing more time for staff to take orders and serve meals resulted in improved nutritional status among residents after one year.

This type of cross‐sectional evidence from numerous homes is better than evidence from a single home or a case study.

The generalizable conclusion, however, depends on a number of factors that might also need to be taken into account: what was the turnover among residents – did new residents have better nutritional status when they arrived, were they younger with better appetites, did they have better hearing so that they could understand more clearly what options were available on the menu for that day, etc.? One of the difficulties with descriptive studies is that we may not always be comparing like with like. We would have to collect information to demonstrate that apart from differences in noise levels and serving times, there were no other differences which could account for the change in nutritional status. We would also want to know if the circumstances in the four selected care homes were generalizable to other care homes with a similar population of residents.