Individual Participant Data Meta-Analysis

There may be sufficient detail in the trial dataset to check whether participants who deviated from intended interventions did so for pre‐specified or otherwise rational reasons. For example, if the data indicate that a participant had experienced an adverse event, this might explain why treatment was stopped early, and would also need to be considered in any analysis of adverse outcomes. It may also be possible to assess whether deviations from planned treatment are similar, and for comparable reasons across treatment groups (more so than with aggregate data).

If a treatment is a major procedure, such as surgery, or particularly toxic, then those delivering treatments and participants will usually be aware of the assigned treatment. Provided that outcomes are objectively measured or ‘hard’, such as mortality, this is unlikely to introduce bias. However, a carer might inadvertently or otherwise deliver a treatment or measure a more subjective outcome, such as an adverse effect, differently if they are aware of which treatment a participant received. Similarly, if a participant is aware of their assigned treatment, it might influence a patient‐reported outcome, such as pain or quality of life. Therefore, in this scenario, there is a potential for bias in this domain, which cannot be alleviated by the collection of IPD. However, the contact with trial teams, that is intrinsic to collaborative IPD meta‐analyses, can provide useful clarification of the methods used to blind participants, carers or outcome assessors, to help determine whether these are appropriate, and therefore allow the risk of bias to be judged with more accuracy.

Figure 4.10Days of the week participants were allocated to treatment and control groups in a trial included in (a) an IPD meta‐analysis of pre‐operative chemotherapy for non‐small cell lung cancer, 88and (b) an IPD meta‐analysis of post‐operative radiotherapy for non‐small cell lung cancer. 107

Source: (a) Based on NSCLC Meta-Analysis Collaborative Group. Preoperative chemotherapy for non-small cell lung cancer: a systematic review and meta-analysis of individual participant data. Lancet 2014;383:1561–71. (b) Based on PORT Meta-analysis Trialists Group. Postoperative radiotherapy in non-small-cell lung cancer: systematic review and meta-analysis of individual patient data from nine randomised controlled trials. The Lancet 1998;352(9124):257–63.

If participants drop out or are actively excluded from the analysis of a trial in substantial numbers and/or disproportionately by group, this could lead to quite considerable imbalances between intervention and control groups. More importantly, it could lead to incomplete outcome data and potentially attrition bias. For example, an examination of 14 cancer IPD meta‐analyses, incorporating 133 trials, found that between 0% and 38% of randomised participants were excluded from the original trial survival analyses, with the largest proportion often being excluded from the treatment groups ( Figure 4.11). In one of these projects examining the effects of chemotherapy for soft tissue sarcoma, 112the evidence for a benefit of chemotherapy on survival was stronger when it was based just on participants included in the original trial analyses (hazard ratio = 0.85; 95% CI: 0.72 to 1.00, p = 0.06), compared to when all possible participants were included (hazard ratio = 0.90, 95% CI: 0.77 to 1.04, p = 0.16).

Figure 4.11Percentage of participants excluded from the original analyses of trials included in 14 IPD meta‐analysis projects in the cancer field. 112

Source: Jayne Tierney, reproduced with permission from Tierney and Stewart. 112

Therefore, it is important to check that data on all, or as many as possible, participants recruited to a trial are included in an IPD meta‐analysis (assuming that all the individuals meet the IPD meta‐analysis inclusion criteria for population and setting). If good records are maintained for a trial, it is possible to recover data on participants who were excluded from the original trial analyses, as part of the IPD collection process, and incorporate them into the meta‐analysis. For example, in the same 14 cancer meta‐analyses described previously, when IPD were collected, approximately 1,800 participants who had been excluded from the original trial analyses were re‐instated, and without them most meta‐analysis results would have been biased towards the research intervention, albeit to a small degree in most cases. 112

A major advantage of IPD meta‐analysis is the ability to include all outcomes of relevance to the meta‐analysis, irrespective of whether they have been published or not, thereby overcoming the potential biases associated with differential reporting of outcomes, 113and providing a more balanced view of benefits and harms. For example, in a systematic review of laparoscopic versus open surgery for the repair of inguinal hernia, based on the available published aggregate data from three trials, the risk of persistent pain was found to be significantly greater with laparoscopic repair (odds ratio = 2·03, 95% CI: 1·03 to 4·01). 114However, when IPD were collected, data were available for a further 17 trials that had not published results for this outcome, and the combined meta‐analysis results showed that the risk of persistent pain was actually lower with laparoscopic repair (odds ratio = 0·54, 95% CI: 0·46 to 0·64). Recognising that some outcomes measured in trials may not be reported, it is always worth checking trial protocols, registry entries and with trial investigators to firmly establish which outcomes can be made available when IPD are provided. 115

Direct access to IPD does not usually allow the assessment of whether the measurement or ascertainment of outcomes differed between intervention groups. However, contact with trial investigators can provide useful clarification of the methods used, to help determine whether these are appropriate, and therefore allow the risk of bias associated with, for example, differential outcome assessments to be judged with more certainty. If the IPD are sufficiently detailed, outcomes may be defined more appropriately or consistently across trials. For example, a new standardised composite outcome might be constructed from a series of component variables.

For time‐to‐event outcomes, such as survival or time to relief of symptoms, bias can occur if certain participants are followed‐up for a longer duration than others, such that event rates appear higher. Therefore, for these outcomes, it is also desirable to check that follow‐up is balanced by treatment group. This can be achieved by selecting those trial participants who are event‐free, then using censoring as the event and the date of censoring as the time‐to‐event in a ‘reverse Kaplan‐Meier’ analysis. In one trial included in an IPD meta‐analysis of adjuvant chemotherapy for soft tissue sarcoma, 102all participants who were still alive had been followed for death for a minimum of about nine years, and subsequently to the same degree in both treatment groups ( Figure 4.12(a)). In another smaller trial from the same meta‐analysis, although participants were followed for a long time, there was an imbalance by group ( Figure 4.12(b)), which became less of an issue when the trial investigator provided updated IPD with extended follow‐up ( Figure 4.12(c)).