Iain K. Crombie - The Pocket Guide to Critical Appraisal

The initial interrogation should provide a provisional map of the paper into which information about the sources of bias and the value of the study findings can be fitted. This process requires an in‐depth interrogation using detailed questions which probe the quality of the study. These questions are described in Chapters 5– 11. Before reading these chapters, it may be helpful to review Chapter 3‘Identifying the Research Design’ and Chapter 4‘Interpreting the Results’.

The first step in critical appraisal is to identify the research design used in a study so the appropriate appraisal checklist can be selected. This chapter provides an outline of the common research designs to aid their identification. It does not provide a full description of each method, but instead gives sufficient detail to confirm which design was used.

Surveys are often used to estimate how common something is: how many people have high blood pressure, or how many suffer from chronic pain. They can also investigate associations between factors: is it more common in men or women; does the frequency of high blood pressure vary with age? Surveys can study whole populations; for example, to establish the proportion of people who currently smoke cigarettes. Or they can investigate specific groups; for example, they could explore the health beliefs of pregnant women, or the frequency of loneliness in persons aged between 65 and 90 years.

Surveys take samples from a target group or population. The idea behind this is that a well‐taken sample contains almost as much information as would come from studying the whole population. In principle, surveys obtain a complete list of the group or population of interest from which a sample of individuals is selected for further study. The selection is carried out randomly (not haphazardly), such that each individual has an equal chance of being chosen. In practice, a complete list may not be available and alternative approaches such as cluster sampling can be used (these more complicated designs are not described here). The important points are that selection of individuals uses random sampling and that the sample obtained is representative of a target population.

Surveys usually do not have a separate control or comparison group, so studies that have them are not surveys. However, in the analysis of surveys one subgroup in the sample may be compared against another (e.g. men versus women, or old versus young). Comparisons are being made, but there is no sense in which one group is acting as a control to another group. All the individuals have been selected at the same time and an internal comparison is made.

Use of the term survey in a paper should identify the method, but sometimes the term is used for what is really a cohort study. Cross‐sectional is a helpful term because it is seldom used with any other research method. Prevalence , the frequency of something in a sample, also suggests that a study is a survey. The terms sample and random sample are unhelpful because they often appear in the description of the other research designs. The terms simple , cluster , or systematic can be used with the word sampling to describe different ways of drawing a sample, e.g. cluster sampling . These terms are seldom used with the other research designs. The phrase stratified sampling is also used in surveys, but the word stratified can also be used in randomised controlled trials (RCTs).

Cohort studies are used to find out what happens to study participants over time. It is the method of choice for studying disease prognosis, or for investigating the consequences of exposure to potentially harmful agents. For example, studies could investigate how long patients with acute low back pain take to recover, or how many people who smoke subsequently develop lung cancer. Whatever the topic, a group of individuals is identified and followed up to see what events befall them. Most commonly the aim is to determine whether exposure to a potentially noxious substance leads to an increased risk of a disease. Sometimes the interest is in whether potentially beneficial substances, such as dietary vitamins or fish oils, reduce the risk of disease.

The defining characteristic of cohort studies is the element of time: in cohort studies time flows forwards. A set of individuals is identified at one point in time and followed up to a later time to ascertain what has happened. These studies are called prospective cohort studies. When studying the impact of some exposure, cohort studies often have a comparison or control group. The controls are usually identified at around the same time and followed for the same length of time. This type of cohort study is called a concurrent cohort study .

Many cohort studies do not have a control group; instead they can make internal comparisons. For example, a cohort could recruit a large sample from the general population and identify who smoked and who did not. The whole sample would be followed up for several years to determine whether lung cancer occurs more frequently in smokers than in non‐smokers.

Some cohort studies identify the group from some time in the past and follow them up to the present. These studies might at first appear to be looking backwards in time, but they are not. Historical records are used, and time flows forwards from the point at which the individuals are identified. This type of design is usually called a retrospective cohort study . (Note the term historical cohort study is used when the exposed group is recruited from one period and the comparison group comes from an earlier period.)

The terms prospective and longitudinal suggest a cohort study, although these terms are also used in clinical trials. The term retrospective is used for one type of cohort study, but is also used with case–control studies.

Case–control studies ask what makes a group of individuals different from a control or comparison group. Often the study group will have a disease, which the control group do not have. For example, the cases could be women with breast cancer and the controls could be women of similar age who do not have the disease. The study compares the characteristics of the cases and controls to identify the factors from the past that might have caused the disease. The control group is often selected to be similar to the cases in factors such as age, gender, and life circumstances. This type of study can also explore why some people behave in a particular way. For example, a study could explore why some patients miss clinic appointments, by comparing those who fail to attend with those who do attend.

Case–control studies look backwards in time from some event (e.g. the diagnosis of breast cancer or failing to attend a clinic) to try to identify factors in the past that might explain why that event occurred. The direction of time is crucial for distinguishing between cohort studies and case–control studies: cohort studies look forwards; case–control studies backwards.

The use of a control group is not a defining characteristic, since RCTs always have one, and cohort studies often do.