Table 3 Methods that can be used to inform the choice of the target difference

Methods that inform what is an important difference

Anchor: The outcome of interest can be “anchored” by using either a patient’s or health professional’s judgement to define what an important difference is. This may be achieved by comparing a patient’s health before and after treatment and then linking this change to participants who showed improvement/deterioration using a more familiar outcome (for which either patients or health professionals more readily agree on what amount of change constitutes an important difference).. Contrasts between patients (e.g., individuals with varying severity of a disease) can also be used to determine a meaningful difference.

Distribution: Approaches that determine a value based upon distributional variation. A common approach is to use a value that is larger than the inherent imprecision in the measurement and therefore likely to represent a minimal level needed for a noticeable difference.

Health economic: Approaches that use principles of economic evaluation. These compare cost with health outcomes, and define a threshold value for the cost of a unit of health effect that a decision-maker is willing to pay, to estimate the overall incremental net benefit of one treatment versus the comparator. A study can be powered to exclude a zero incremental net benefit at a desired statistical significance and power. A radically different approach is a (Bayesian) decision-theoretic value of information analysis which compares the added value with the added cost of the marginal observation, thus avoiding the need to specify a target difference.

Standardised effect size: The magnitude of the effect on a standardised scale defines the value of the difference. For a continuous outcome, the standardised difference (most commonly expressed as Cohen’s d “effect size”, the mean difference dividing by the standard deviation) can be used. Cohen’s cutoffs of 0.2, 0.5, and 0.8 for small, medium, and large effects, respectively, are often used. Thus a “medium” effect corresponds simply to a change in the outcome of 0.5 SDs. When measuring a binary or survival (time-to-event) outcome alternative metrics (e.g., an odds, risk, or hazard ratio) can be utilised in a similar manner, though no widely recognised cut-points exist. Cohen’s cut-points approximate odds ratios of 1.44, 2.48, and 4.27, respectively. Corresponding risk ratio values vary according to the control group event proportion.

Methods that inform what is a realistic difference

Pilot study: A pilot (or preliminary) study may be carried out where there is little evidence, or even experience, to guide expectations and determine an appropriate target difference for the trial. In a similar manner, a Phase 2 study could be used to inform a Phase 3 study though this would need to take account of methodological differences (e.g. inclusion criteria and outcomes) that should be reflected in specification of the target difference.

Methods that inform what is an important and/or a realistic difference

Opinion-seeking: The target difference can be based on opinions elicited from health professionals, patients, or others. Possible approaches include forming a panel of experts, surveying the membership of a professional or patient body, or interviewing individuals. This elicitation process can be explicitly framed within a trial context.

Review of evidence base: The target difference can be derived from current evidence on the research question. Ideally, this would be from a systematic review or meta-analysis of RCTs. In the absence of randomised evidence, evidence from observational studies could be used in a similar manner.