Comparison of anticipated and actual control group outcome in randomised trials in paediatric oncology provides evidence that non-randomised studies are biased in favour of the novel treatment
© Moroz et al; licensee BioMed Central Ltd. 2011
Published: 13 December 2011
Historically controlled studies (HCSs) compare data from two or more separately conducted studies – the new treatment arm, usually prospectively collected, is compared to a control arm of retrospective data. HCSs are frequently undertaken in paediatric oncology (PO), where there is a widespread belief that RCTs cannot be performed in rare diseases. This is despite the potential biases in HCSs being well known – e.g. other factors change over time.
To compare the outcome of the control group of RCTs in PO with that anticipated in the sample size calculation. The assumption being that, had an HCS been carried out instead, the control arm data in the historical control study would have likely been that utilized in the RCT’s sample size calculation.
A search for published PO RCTs was conducted using the Cochrane Central database from March 2011 to database inception. Search terms were “randomi?ed” plus the disease name in all fields. Only papers reporting sample size parameters and observed estimates were included. Data were extracted and compared on anticipated and observed outcomes in the control arms (and experimental arms).
To date 45 RCTs have been included from 16 tumour sites: 36 superiority trials, 9 equivalence/non-inferiority; 12 with dichotomous primary endpoints, 33 time-to-event; mean recruited number of patients was 231 (range: 50 to 2619). In 33 trials the control group did better than anticipated (in 13 cases >10% absolute difference), in 11 trials the control group did worse (by >10% in four); outcome was the same as anticipated in one trial. The median absolute difference between control groups’ observed outcome and anticipated outcome was 5% (range: -21% to +35%); mean difference was also 5%. In superiority trials, the median difference was 6%; it was 4% in equivalence/non-inferiority trials. In trials with a dichotomous endpoint, the median difference was 7%; it was 5% in trials with a time-to-event endpoint (8 out of 9 equivalence/non-inferiority trials had time-to-event endpoint). The median observed difference between the experimental and control groups was 0% (range: -16% to 21%); the median difference between the observed experimental arm outcome and anticipated control group outcome was 6% (range: -20% to 45%).
Since the control group (i.e. standard treatment arm) in RCTs did better than anticipated, non-randomised HSCs that use similar assumptions for outcome with standard treatment will overestimate the benefit of new treatments, potentially leading to children with cancer being given inappropriate therapy.
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.