Individual Participant Data Meta-Analysis

Based on practical experience 7,43and the empirical evidence summarised in Section 2.5, 47,63any decision about undertaking an IPD meta‐analysis project should be based on the nature of the specific research question ( Section 2.6.1), the completeness and uniformity of the available aggregate data ( Section 2.6.2), the information size ( Section 2.6.3), and the data and analyses required to address the research question reliably ( Section 2.6.4). A checklist of questions to aid in this decision‐making is provided in Table 2.2, and is explained in more detail in the following subsections. The focus is on the evaluation of treatment effects from randomised trials. If the answer to one or more of the questions is “yes”, then an IPD project is likely to add considerable value compared to a conventional systematic review and meta‐analysis of aggregate data.

In certain circumstances, the rationale for obtaining and analysing IPD is immediately very strong, such as when the research question is focused on participant‐level relationships to inform more stratified or personalised approaches to treatment. In particular, if the desire is to evaluate whether particular types of participants benefit more or less from an intervention than others ( Chapter 7), IPD will almost always be needed to obtain a reliable assessment of treatment‐covariate interactions. 33This is also true for situations where participant‐level diagnosis, prognosis and prediction are of interest ( Chapters 15to 17). Another powerful motivation for seeking IPD is in controversial areas, where independent scrutiny of the trial IPD may improve credibility and increase transparency, which may be sufficient justification in itself. 20,65On the other hand, if it is clear that a conventional aggregate data meta‐analysis could provide reliable (albeit less detailed) results, then an IPD project may be of less value, or may not be considered a priority. For example, if the sole research objective is to investigate the overall effect of a treatment, then it may be reasonable to rely on aggregate data from existing trials (reported or otherwise available) that summarise these effects ( Section 2.6.2).

Table 2.2Signalling questions to help decide when aggregate data are insufficient for meta‐analysis and IPD are needed, focusing on the evaluation of treatment effects from randomised trials.

Source: Jayne Tierney, adapting the figure presented by Tudur Smith et al., 64with permission.

It will not always be possible to complete an IPD meta‐analysis project in a suitably timely manner for the question of interest, because, for example, trial investigators are focused on completion of their individual trials, trial data are embargoed for a period, or it will take too long to set up data‐sharing agreements. Therefore, if a therapeutic area is moving very quickly, there is an urgent policy need, or results are required to inform an ongoing trial of the same treatment(s), a prospective aggregate data meta‐analysis, 66, 67perhaps as part of a living systematic review, 67, 68may be more suitable for delivering results in the shortest time frame. However, ultimately the merits of any aggregate data synthesis need to be balanced against the benefits that a good‐quality IPD project could bring, and for an important question, the extra time needed for IPD meta‐analysis may be justified to better inform decision‐making in the longer term. Instead, researchers might choose to complete a conventional aggregate data meta‐analysis first, with the intention of conducting an IPD meta‐analysis project in a subsequent stage, if more reliable and nuanced results are needed.

Before deciding whether to collect IPD for meta‐analysis, an important step is to assess the completeness and uniformity of the available aggregate data, either based on a pre‐existing systematic review that addresses a similar research question, or by conducting a scoping or systematic review of existing trials of interest. If outcomes and participant‐level covariates have been collected in the eligible trials, but are not (adequately) described in the associated trial reports (such as side effects of treatment, multiple time‐points or continuous values of prognostic factors), this can be rectified by the collection and analysis of IPD. Even if all the outcomes, participant‐level covariates and, if relevant, interactions required for the analyses have been reported, if they are not defined consistently across trials it can be difficult to include or combine their results in aggregate data meta‐analysis in a meaningful way. At best, this could lead to findings that are difficult to interpret, and at worst, that are unreliable. If this is a cause for concern, IPD might be sought to allow standardisation of the variables in readiness for analysis ( Section 4.5).