Individual Participant Data Meta-Analysis

In general, having access to IPD also supports more flexible and sophisticated analyses than are possible with only existing aggregate data. IPD are vital for a thorough investigation of participant‐level associations, for example to identify treatment effect modifiers ( Chapter 7). 7,9,43For instance, an IPD meta‐analysis project by the Early Breast Cancer Trialists Group, which combined IPD from 37,000 women in 55 randomised trials, established that the drug tamoxifen works better in the subgroup of breast cancer patients who are classed as oestrogen receptor positive. 15

With IPD, there is no need to rely on, or be restricted by, the original trial methods of analysis. For example, the IPD meta‐analysis research team could opt for alternative effect measures (e.g. hazard ratios rather than odds ratios) or assumptions (e.g. non‐proportional rather than proportional hazards), as appropriate, and consider a broader set of outcomes than originally reported. Collecting IPD also allows continuous variables to be analysed on their continuous scale; potential non‐linear relationships to be examined; and the analysis of outcomes, covariates (e.g. prognostic factors) and time‐points that were recorded, but not originally analysed by trial investigators.

As most IPD meta‐analysis projects are collaborative endeavours, direct contact with trial investigators can help to identify trials that may not be easily identifiable via other forms of searching, 7,9,43and to clarify the eligibility of potentially relevant trials. Trial investigators can also provide extra information leading to more reliable risk of bias assessments than are achievable from trial reports ( Section 4.6), 48and if potential biases or errors are identified, they may be able to supply additional data to resolve or minimise these ( Section 4.5.4). 7,9,43Bringing together a group of international and multi‐disciplinary collaborators can also facilitate wider discussion and interpretation of results, and aid dissemination of key findings ( Chapter 10).

The advantages shown in Table 2.1focus on the synthesis of randomised trials to evaluate treatment effects, but IPD meta‐analyses have further advantages for non‐efficacy questions. In particular, they are pivotal in guiding and tailoring diagnostic strategies ( Chapter 15), identifying risk and prognostic factors ( Chapter 16), and individualising risk prediction ( Chapter 17) to guide healthcare policy and practice. 32

The processes for collecting, checking and analysing IPD are more involved and complex than for conventional aggregate data reviews of the same topic, and therefore, more time and resources are required ( Chapter 3). 7,9,43Thus, before embarking on an IPD project, careful consideration about whether it is an appropriate course of action is needed ( Section 2.6). 47,49

As most IPD projects rely on collaboration with the teams responsible for the included trials, negotiating and maintaining relationships with investigators from different countries, settings and disciplines can take considerable time, effort, diplomacy and careful management ( Section 3.2.1). 43,44In an era in which the value of clinical data sharing is more widely appreciated, persuading trial investigators of the value of participating has perhaps become easier. That said, IPD meta‐analysis research teams and those providing IPD are now faced with additional tasks, such as seeking ethical or other institutional approval for the exchange of IPD ( Section 3.10), as well the development of detailed data‐sharing agreements ( Section 3.11).

Over the years, advances in database and statistical software and electronic communication have greatly reduced the burden of labour required for the data exchange, management and analysis aspects of IPD projects. 43Even so, these remain the most time‐consuming and resource‐intensive phases ( Section 3.8), and require skills and expertise beyond those needed for a conventional aggregate data review ( Section 3.5). This is emphasised by a growing number of methodological articles relating to IPD meta‐analysis projects, covering issues such as data checking and harmonisation, 7,28statistical methodology, 33,50–57examining potential biases, 46,58dealing with unavailable IPD, 59reporting, 60and statistical software development, 61amongst others.

There are many empirical comparisons of results produced by IPD meta‐analyses with results based on corresponding analyses of published aggregate data. An early example, in advanced ovarian cancer, found that results based on published aggregate data suggest a 7.5% absolute improvement in the percentage of women surviving at 30 months with platinum‐based chemotherapy, whereas the IPD meta‐analysis suggests a 2.5% improvement in the percentage surviving. 8This disparity, which could have led to different clinical conclusions, seemed to be driven by the IPD meta‐analysis project including more trials, participants and follow‐up, as well as including all of the events in a time‐to‐event analysis, rather than calculating a risk ratio from events observed at a fixed time‐point.

In contrast, in a different example, there was no clear evidence of an effect of ovarian ablation on survival of women with early breast cancer, based on the published aggregate data, but a 10% absolute increase in the percentage surviving at 15 years based on IPD. 62In this case, the IPD for the included trials incorporated much greater follow‐up, leading to a near doubling of events. This, and a more appropriate time‐to‐event analysis, were likely to be the key drivers for the discrepancy with the aggregate data findings. Many other comparisons of results from IPD and aggregate data meta‐analysis projects have been carried out, not only in cancer and cardiovascular disease where IPD meta‐analysis first gained traction, but also in other healthcare areas such as infectious diseases, neurology, nephrology and critical care. The differences shown between IPD and aggregate data findings are variable and seem context specific (e.g. depending on the research question; Sections 2.2.1and 2.6.1). 63

A large systematic review that brought together published comparisons of treatment effects from IPD and aggregate data meta‐analyses found that many pairs of IPD and aggregate data analyses agreed in terms of the statistical significance of the overall results for the main outcomes. However, the disagreement observed in 20% of cases could have led to different clinical conclusions. 63The discrepancies did not seem to be clearly associated with variation in the number of trials, number of participants or length of follow‐up. 63Importantly, discrepancies are likely to be more pronounced when going beyond overall treatment effects, which is often a key aim of an IPD meta‐analysis project, such as when examining treatment‐covariate interactions at the participant level ( Chapter 7). 33

Evidence from a large cohort of systematic reviews of the effects of cancer therapies on survival showed that, on average, meta‐analysis results for the overall treatment effect derived from published aggregate data (based on hazard ratios) were slightly more in favour of the research treatment than those from IPD. 47Although most results were similar between aggregate data and IPD meta‐analyses, those discrepancies that did occur were often substantial. 47Importantly, results from aggregate data were most likely to agree with those from IPD when the number of participants or events (absolute information size) and the proportion of participants or events available from the aggregate data relative to the IPD (relative information size) were large. This emphasises that assessing the amount of information provided by the available aggregate data, and what the obtainable IPD might add for a particular research question, is an important step in determining when IPD will bring the greatest value ( Section 2.6.3).