XVI.Add.II.3.3. Pharmacoepidemiological methods

Outcomes of RMM may investigated with non-interventional methods that measure how medicinal products are prescribed, dispensed, or used over time, by means of electronic health records, medical chart abstraction or claims data (see XVI.Add.II.2). Detecting changes in adverse reaction reporting, despite known limitations, may contribute to this investigation (see XVI.Add.II.2.7. ).

Since RMM are generally implemented in the entire target population, the identification of a control group may not always be possible and the comparison against suitable reference values should be considered (see GVP Module XVI).

For marketed medicinal products, quantitative measures (see GVP Module XVI) should be estimated in the same study population before and after the RMM intervention, with preintervention information acting as a surrogate control (i.e. quasi-experimental designs). However, in absence of pre-intervention information (e.g. for medicinal products with RMM at the time of initial marketing authorisation), any effect of the RMM estimated at (a) time point(s) after implementation can only be evaluated against a predefined reference value (i.e. literature review, historical data, expected frequency in general population, outcome frequency in pre-authorisation clinical trials) taking into account all possible limitations (32) (see GVP Module XVI). The selection of a reference value should be justified.

Whilst appropriate to describe the population for understanding generalisability of observed outcomes, simple descriptive approaches do not determine whether statistically significant changes have occurred (3,33).

XVI.Add.II.3.3.1. Single time point cross-sectional study

The guidance on cross-sectional study designs in GVP Module VIII applies. Cross-sectional studies can only measure an association between exposure and outcome at a single point in time. Therefore, the method is commonly used to monitor indicators of RMM implementation and to complement other studies investigating e.g. patterns of medicines use.

XVI.Add.II.3.3.2. Before-after cross-sectional study

A before-after cross-sectional study is defined as an evaluation at one point in time before and one point in time after the RMM dissemination to healthcare systems (accounting for the implementation timeframe). Including a control can strengthen this design (3); however, careful consideration should be given to whether a suitable control can be identified, e.g. healthcare professionals not targeted by the RMM to control for general prescribing trends. When uncontrolled, baseline trends are ignored, potentially leading to RMM outcomes being estimated incorrectly.

When a RMM is put in place at the time of initial marketing authorisation, the comparison of an outcome frequency indicator obtained after the RMM against a predefined reference value would be acceptable (see GVP Module XVI).

XVI.Add.II.3.3.3. Before-after time series analysis

Time series analysis has commonly been used to evaluate the effectiveness of RMM and should be considered whenever feasible as one of the more robust approaches (3). A time series analysis spanning the date of RMM dissemination to healthcare systems (e.g. interrupted segmented regression analysis) accounts for secular trends and can provide statistical evidence about whether observed changes are significant.

Time series analysis is well suited to study changes in outcomes that are expected to occur relatively quickly following RMM, such as changing prescribing rates. Time series analysis can be used to estimate the immediate change in outcome after the RMM, the change in trend in the outcome over time compared to before, and the effects at specific time points following the RMM. The Cochrane Effective Practice and Organisation of Care (EPOC) Resources for Review Authors on Interrupted Time Series (ITS) Analysis10 provides further information on the utility of time series regression (34).

Time series analysis requires that enough data points are collected before and after the RMM. The power to undertake a time series analysis depends upon the sample size, the effect size, the prevalence of exposure, the number of data points and their balance before and after the intervention time period (35). Long time periods may also be affected by changes in trends unrelated to the RMM that can violate model assumptions and introduce confounding when evaluating RMM.

Like the before-after cross-sectional design, including a control can strengthen this design by minimising potential confounding.

Factors such as autocorrelation, seasonality and non-stationarity should be checked when conducting time series analysis and may require more complicated modelling approaches if detected or considered likely to occur (36). Interventions associated with major immediate changes (e.g. product withdrawals) may be evaluated without regression modelling, but they risk producing spurious results when the changes are more subtle or multiple confounders are present (3).

Time series analysis also requires that the time point of RMM dissemination to healthcare systems (accounting for the implementation timeframe) is known prior to the analysis. When this is not the case (e.g. during a phased roll out of a regulatory action), more complex modelling techniques and data-driven time series approaches such as Joinpoint regression analysis could be considered (37). There are literature examples of time series analysis using a control (38), estimating effects twelve months after the intervention (33), dealing with autocorrelation and seasonality (39), and using Joinpoint regression (40).