- Open Access
Representativeness of the participants in the smoking Cessation in Pregnancy Incentives Trial (CPIT): a cross-sectional study
Trials volume 17, Article number: 426 (2016)
The limited representativeness of trial samples may restrict external validity. The aim of this study was to ascertain the representativeness of the population enrolled in the Cessation in Pregnancy Incentives Trial (CPIT), a therapeutic exploratory study to examine the effectiveness of financial incentives for smoking cessation during pregnancy.
CPIT participants (n = 492) were compared with all self-reported smokers at maternity booking who did not participate in the trial (n = 1982). Both groups were drawn from the National Health Service (NHS) Greater Glasgow and Clyde area over a 1-year trial enrolment period. Variables used for comparison were age, area-based deprivation index, body mass index, gestation, and carbon monoxide (CO) breath test level. Chi-square and Mann-Whitney U tests were used to compare groups.
From January to December 2012, 2474/13,945 (17.7 %) women, who booked for maternity care, self-reported as current smokers (at least one cigarette in the last week). Seven hundred and fifty-two were ineligible for trial participation because of a CO breath test level of less than 7 parts per million (ppm) used as a biochemical cut-off to corroborate self-report of current smoking. At telephone consent 301 could not be contacted, 11 had miscarried, 16 did not give consent and 3 opted out after randomisation, leaving 492 participants for analysis. There were no differences in demographic or clinical characteristics between trial participants, and self-reported smokers not enrolled in the trial in terms of CO breath test (as a measure of smoking level for those with a CO level of 7 ppm or higher), material deprivation (using an area-based measure), maternal age and maternal body mass index. Gestation at booking was statistically significantly lower for participants.
To ensure that all trial participants were smokers, biochemical validation excluded self-reported smokers with a CO level of less than 7 ppm from taking part in the trial, which excluded 30 % of self-reported smokers who were ‘lighter’ smokers. The efficacy of financial incentives would not have been likely to decrease if ‘lighter’ smokers had been included in the trial population. Trial participants were slightly earlier in their pregnancy at maternity booking, but this difference would not clinically affect the provision of financial incentives if provided routinely. Overall, the trial population was representative of all self-reported smokers with regard to available routinely collected data. Appropriate comparison of trial and target populations, with detailed reporting of exclusion criteria would contribute to the understanding of the wider applicability of trial results.
Current Controlled Trials ISRCTN87508788. Registered/Assigned on 1 September 2011.
The fundamental evidence for decision-making in patient care has in the past been based on clinical experience . More recently, clinicians and decision-makers have come to accept that rigorous research is required in addition to experience  in order to tackle growing population health challenges. Current intervention protocols and clinical guidelines combine high-quality research from individual patient treatment  with clinical experience. Randomised controlled trials (RCTs) with reliable internal validity are considered the ‘gold standard’  to inform decision-making for both individual patients and population health [5–7]. Guidelines to help clinicians make quick but accountable decisions should be guided by accurate and reliable research findings from RCTs  to minimise bias. However, trial findings must benefit the target group with similar therapeutic needs (external validity) . Clinicians [8–10] have questioned the effectiveness and reliability of applying trial evidence to target populations outside a trial setting. This may explain delayed use of trial evidence in clinical practice [10, 11]. In the USA, a large body of research has been reviewed showing that ethnic minority groups are under-represented in trials . Much of this review focussed on reasons why ethnic minority groups are more difficult to access and are less likely to agree to take part and be retained in research studies. There are many plausible reasons, but in general trials have insufficient representation from ethnic minority groups for clinicians to feel confident that the trial results should lead to change in treatment for those groups.
Trial selection criteria [6, 13, 14] and overall methodology to ensure internal validity may threaten representativeness. For instance, patients with comorbid conditions are often excluded from trials. In subsequent clinical practice, beneficial effects may not be realised perhaps due to side effects associated with a common comorbid condition that is highly represented in the target population. It is important, therefore, to compare trial participants with the target population with regard to demographic, clinical and other variables. Such comparison will help to uncover possible bias associated with nonrepresentative trial populations and allow a judgement to be made about the likely generalisability of trial results [4, 15]. Models of reasons for taking part and not taking part are important  to try to understand how better to run research projects that are representative and, therefore, produce results applicable to all groups in the target population. However, this present study did not attempt to understand the reasons behind lack of representativeness, but was limited to establishing representativeness or not, using limited criteria available from both the trial population and the target population from which the trial population was drawn.
The Cessation in Pregnancy Incentives Trial (CPIT) [17, 18] was a therapeutic exploratory phase II trial to examine the efficacy and acceptability of using financial incentives to help pregnant smokers to make use of routine Stop Smoking Services (SSS) and or quit smoking during pregnancy. Women were eligible if they were self-reported smokers with an exhaled carbon monoxide level of at least 7 parts per million (ppm) (to biochemically demonstrate current smoking), aged 16 years or older (for consent), less than 24 weeks pregnant (to allow intervention lasting 12–16 weeks by SSS), resident in NHS Greater Glasgow and Clyde (to allow access for research nurses to collect biochemical verification measures of smoking cessation), and able to understand and speak English (for telephone consent). This study used 492 trial participants of 2494 (20 %) women who self-reported as current smokers (at least one cigarette in the last week) at their first maternity booking contact appointment – the target population. The trial showed the intervention to be effective  and cost-effective ; however, there remains a valid question: ‘Is the intervention likely to be effective if applied to the whole target population?’
Comparisons were, therefore, made between self-reported pregnant smokers not included in the trial or without analysable data (group B; Fig. 1) and trial participants enrolled in CPIT with analysable data (group A; Fig. 1) [17, 18]. Together these groups make up the target population for the intervention. The findings will help policy-makers to determine the representativeness of the trial population and the likely generalisability of financial incentives for smoking cessation during pregnancy.
The design of this representativeness study was cross-sectional, comparing the trial population with self-reported pregnant smokers not included in the trial in terms of available routinely collected demographic and clinical data. The setting was maternity booking in early pregnancy in NHS Greater Glasgow and Clyde in the West of Scotland where all pregnant women are routinely asked about their smoking status and a CO breath test is administered as an objective proxy measure of smoking status – about 97 % of pregnant women at first maternity booking in Greater Glasgow and Clyde undergo this test . Client details, including self-reported smoking status and CO level are forwarded using an opt-out approach from the maternity booking appointment to the NHS SSS as recommended in national guidelines . The SSS then telephones all pregnant smokers to discuss smoking and attempting to quit. During this contact, verbal permission was obtained from eligible potential trial participants to pass their contact details to the trial team. Ineligible pregnant smokers were those: with a CO breath test level less than 7 ppm or where no CO breath test level was available; where no contact was possible by the SSS; who were under 16 years of age; who were at 24 full weeks gestation or more at maternity booking (to allow SSS support to take place prior to expected delivery); who were not approached by SSS about the trial; who did not speak English; and who refused permission to pass contact details to the research team. The trial team attempted to contact pregnant smokers to discuss trial participation and a few who were contacted were no longer pregnant. The remaining patients were asked for verbal consent to take part in the trial. Those who agreed to take part were enrolled into the trial.
Group A are trial participants (n = 492). Group B are self-reported smokers at maternity booking who were not included in the trial when analysed (n = 1982).
Variables used to compare group A with group B
Variables from the two population groups were gathered from routinely collected maternity service data: CO breath test level, height, weight, postcode, age and gestation. Gestation was recorded as estimated gestation (in weeks calculated from recall of last menstrual period) at the date of referral to the SSS and ranged from 4 to 40 weeks. Body mass index (BMI) was derived from maternal height and weight using the formula;
Finally, material deprivation was measured using the Scottish Index of Multiple Deprivation (SIMD)  with postcode as proxy. ‘SIMD ranks small areas (called datazones) from most deprived (ranked 1) to least deprived (ranked 6505) using 38 indicators from seven domains; income, employment, health, education, skills and training, housing, geographic access and crime’ . In order to compare the CO levels (and, therefore, heaviness of smoking ) between those enrolled in the trial and those who chose or could not be enrolled in the trial, comparison between groups A and B was made after excluding those with a CO level less than 7 ppm. The aim was to show that apart from the exclusion criterion of a CO level less than 7 ppm the groups were similar in heaviness of smoking.
Statistical analyses were performed with Stata 12.10 . Descriptive statistics (percentages, medians and interquartile ranges) were derived for all variables. Univariate statistical comparison used Pearson X 2 test for linear trend and Mann-Whitney U tests to ascertain possible group differences between group B – self-reported smokers not included in the trial and group A – the trial participants. We used multivariable logistic regression to control for the possible effects of confounding among our variables. Nonlinearities for continuous variables were tested using polynomial terms.
Study setting and sample characteristics
Of the women who booked for maternity care, 2474/13,945 (17.7 %) were self-reported smokers, 172 (7.0 %) were excluded from all analyses because their CO level was missing or unknown, 752 (30.4 %) were excluded because their CO level was below 7 ppm, and 727 (29.4 %) did not meet the other inclusion criteria (Fig. 1). After checking routine records, 5 (<1 %) had miscarried and, therefore, contact was not attempted, and for 301 (12 %) contact was unsuccessful. Out of the 517 women contacted, consent for enrolment in the trial was not given by 16 women, 6 had miscarried, and the remaining 495 were enrolled. Three participants withdrew, refusing permission for their data to be used, leaving 492 trial participants’ data for analysis (Fig. 1) as group A. Those not included in the trial, 1982 (2474 − 492), made up the comparison group B.
Comparison of groups
The median CO breath test result for both self-reported smokers at maternity booking not included in the trial (excluding those with a CO level less than 7 ppm) and the trial participants was 12 ppm (Table 1). A statistically significant difference (P < 0.001) was seen for gestation at booking with those in the trial booking at slightly earlier gestation (median 12.9 weeks compared with 13.3 weeks for those not included in the trial).
Table 2 shows the odds ratios for a multivariable logistic regression of trial status on gestation, age category, deprivation category and BMI category. All variables with the exception of estimated gestation are not statistically significant. The effect of estimated gestation is that of increasing the odds into the trial. However, the square term shows that at higher gestational ages the odds of being in the trial declines significantly, P = 0.007.
Comparing trial participants and self-reported smokers not included in the trial
Overall, there was no evidence from available routinely collected demographic or clinical characteristics that trial participants differed in a way that would make a difference to implementation or effectiveness of the incentives intervention. This suggests that the trial sample was representative of the whole target population of self-reported smokers identified at maternity booking in NHS Greater Glasgow and Clyde (Table 1). This is similar to randomised controlled trial studies in cannabis dependence  and also in smoking and chewing tobacco cessation trials [26, 27] none of which were among pregnant women. However, other studies [28, 29] found their trial populations to be nonrepresentative in terms of demographic and clinical characteristics. Evidence from systematic reviews [8, 30] indicates that pharmaceutical trials often exclude the majority of patients based on age, comorbidity, multiple drug use and other unexplained reasons, resulting in nonrepresentative samples. Our results indicate that the CPIT population was a representative sample of all self-reported smokers identified at first maternity booking contact visit (the target population) in NHS Greater Glasgow and Clyde Health Board area.
An exhaled CO level of less than 7 ppm excluded 30 % of self-reported smokers. The aim was to make sure that all participants were smokers rather than nonsmokers masquerading as smokers to try to receive incentive payments. A previous study in the same geographical area , before the incentives trial, showed that 36 % of self-reported smokers at maternity booking (with no advantage gained by calling themselves smokers) had a CO level of less than 7 ppm. It therefore seems likely that self-reported smokers who were excluded because their CO breath test gave a reading less than 7 ppm were true smokers but light smokers . Light smokers are more likely to successfully quit with effective support . Therefore, excluding these women is not likely to have overestimated the true effectiveness of financial incentives if offered to all self-reported smokers identified at maternity booking.
Gestation was significantly different with self-reported smokers who were not included in the trial, having a higher gestation at maternity booking of 13.3 weeks compared to trial participants’ 12.9 weeks. However, this difference is small and likely to be skewed by the few women who book late during pregnancy as women with greater than 24 weeks gestation at maternity booking were also excluded. This difference in gestation between groups A and B would not be large enough to affect implementation of financial incentives if they were rolled out across the health board area.
Achieving representativeness and external validity alongside internal validity is a difficult task especially in clinical trials. This study, using data from a novel trial for smoking cessation in pregnancy, showed that routinely collected data from maternity booking can provide large general target and trial population groups for adequate comparison, essential when assessing representativeness. Data items available for this study were by no means exhaustive, but those available show that participants were of similar age, material deprivation status, heaviness of smoking addiction and gestation to those women who did not take part from the target population. Therefore, it seems likely that an effective smoking cessation intervention demonstrated among trial participants would be generalisable to the whole of the target population of pregnant smokers identified at the first maternity booking visit.
Limitations of the study
Consideration of other important covariates with regard to representativeness of the trial population which were not available – such as marital status, education, partner smoking status, employment, symptoms of psychological distress, and others known to influence smoking during pregnancy – may have produced different results.
Reasons why the study population might not be representative such as ‘program benefits’ and ‘barriers to participation’  were not considered but are of great importance when beginning to design a public health intervention strategy and through the stages of testing the intervention – pilot, definitive trial and implementation.
Implications for practice
To help assess applicability and to enable clinicians to replicate intervention strategies to improve health, researchers can make use of available routinely collected data in intervention studies in a relatively inexpensive way. This would allow assessment of the representativeness of the study population compared with the target population from which it was drawn and for whom the intervention is planned. Detailed reporting of reasons for exclusions and presentation of results by age, sex, socioeconomic status and other important characteristics, such as ethnic group, could help clinicians and policy-makers to take more informed decisions on whether to implement interventions.
Enrolment and retention or not in public health intervention trials should be modelled carefully at every stage of the research process from design through pilot and definitive trial stages to the end of implementation . In this way, barriers to participation and retention in trials which often mirror barriers to implementation can be understood and removed if possible at an early stage. If barriers cannot be removed or the program benefits cannot be effectively enhanced to overcome barriers, then public health interventions can be abandoned or redesigned before large amounts of public money are spent on interventions that will not be effective for the target population.
This study shows that the trial population who took part in the Cessation in Pregnancy Incentives Trial (CPIT) were representative of the target population of pregnant smokers identified at maternity booking in the trial catchment area in relation to important demographic and clinical variables. This finding supports the view that the effectiveness of the intervention examined in the trial would be generalisable if offered to all pregnant smokers identified in the trial area at maternity booking and perhaps more widely. To underpin guidelines relevant to the majority of the population that call for quality, efficient, effective and holistic health care, trial participant representativeness to the target population should be examined alongside internal validity in the ranking of evidence.
Body mass index
Cessation in Pregnancy Incentives Trial
National Health Service
Parts per million
Randomised controlled trial
NHS Stop Smoking Service
Travers J, Marsh S, Williams M, Weatherall M, Caldwell B, Shirtcliffe P, et al. External validity of randomised controlled trials in asthma: to whom do the results of the trials apply? Thorax. 2007;62:219–23.
Herland K, Akselsen JP, Skjønsberg OH, Bjermer L. How representative are clinical study patients with asthma or COPD for a larger ‘real life’ population of patients with obstructive lung disease? Respir Med. 2005;99:11–9.
Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L. Randomized controlled trials: do they have external validity for patients with multiple comorbidities? Ann Fam Med. 2006;4:104–8.
Petersen MK, Andersen KV, Andersen NT, Soballe K. ‘To whom do the results of this trial apply?’: external validity of a randomized controlled trial involving 130 patients scheduled for primary total hip replacement. Acta Orthop. 2007;78:12–8.
Antes G. The evidence base of clinical practice guidelines, health technology assessments and patient information as a basis for clinical decision-making. Z Arztl Fortbild Qualitatssich. 2004;98:180–4.
Le Strat Y, Rehm J, Le Foll B. How generalisable to community samples are clinical trial results for treatment of nicotine dependence: a comparison of common eligibility criteria with respondents of a large representative general population survey. Tob Control. 2011;20:338–43.
Seale JP, Gebski VJ, Keech AC. Generalising the results of trials to clinical practice. Med J Aust. 2004;181:558–60.
Jones R, Jones RO, McCowan C, Montgomery AA, Fahey T. The external validity of published randomized controlled trials in primary care. BMC Fam Pract. 2009;10:5.
Okuda M, Hasin DS, Olfson M, Khan SS, Nunes EV, Montoya I, et al. Generalizability of clinical trials for cannabis dependence to community samples. Drug Alcohol Depend. 2010;111:177–81.
Dawes M. Co-morbidity: we need a guideline for each patient not a guideline for each disease. Fam Pract. 2010;27:1–2.
Flather M, Delahunty N, Collinson J. Generalizing results of randomized trials to clinical practice: reliability and cautions. Clin Trials. 2006;3:508–12.
Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants. Annu Rev Public Health. 2006;27:1–28.
Zimmerman M, Chelminski I, Posternak MA. Generalizability of antidepressant efficacy trials: differences between depressed psychiatric outpatients who would or would not qualify for an efficacy trial. Am J Psychiatry. 2005;162:1370–2.
Hoertel N, Le Strat Y, De Maricourt P, Limosin F, Dubertret C. Are subjects in treatment trials of panic disorder representative of patients in routine clinical practice? Results from a national sample. J Affect Disord. 2013;146:383–9.
Cole SR, Stuart EA. Generalizing evidence from randomized clinical trials to target populations The ACTG 320 Trial. Am J Epidemiol. 2010;172:107–15.
Spoth R, Redmond C. Parent motivation to enroll in parenting skills programs: A model of family context and health belief predictors. J Fam Psych. 1995;9:294–310.
Tappin DM, Bauld L, Tannahill C, de Caestecker L, Radley A, McConnachie A, et al. The Cessation in Pregnancy Incentives Trial (CPIT): study protocol for a randomised controlled trial. Trials. 2012;13:113.
Tappin D, Bauld L, Purves D, Boyd K, Sinclair L, MacAskill S, et al. Financial incentives for smoking cessation in pregnancy: randomised controlled trial. BMJ. 2015;350:h134.
Boyd KA, Briggs AH, Bauld L, Sinclair L, Tappin D. Are financial incentives cost-effective to support smoking cessation during pregnancy? Addiction. 2016;111(2):360–70. doi:10.1111/add.13160.
Director of Public Health, Greater Glasgow and Clyde NHS Board. Update on progress with recommendations from the Director of Public Health report: keeping health in mind. Board paper 12/52. 2014. http://library.nhsggc.org.uk/mediaAssets/Board%20Papers/12-52.pdf page 6. Accessed 18 Feb 2015
National Institute of Health and Care Excellence: quitting smoking in pregnancy and follow childbirth (PH26).2010. https://www.nice.org.uk/guidance/PH26. Accessed 18 Feb 2015
Scottish Index of Multiple Deprivation 2012. http://simd.scotland.gov.uk/publication-2012 Accessed 18 Feb 2015
Deveci SE, Deveci F, Acik Y, Ozan T. The Measurement of exhaled carbon monoxide in healthy smokers and non-smokers. Respir Med. 2004;98:551–6.
Stata C. Stata Statistical Software Release 7.0: Programming: Stata Corporation, 2001.
Frewen AR, Baillie AJ, Montebello ME. Are cannabis users who participate in a randomized clinical trial different from other treatment seekers? J Subst Abuse Treat. 2009;36:339–44.
Graham AL, Papandonatos GD, DePue JD, DePue JD, Pinto BM, Borrelli B, et al. Lifetime characteristics of participants and non-participants in a smoking cessation trial: implications for external validity and public health impact. Ann Behav Med. 2008;35:295–307.
Howard-Pitney B, Fortmann SP, Killen JD. Generalizability of findings from a chewing tobacco cessation clinical trial. Nicotine Tob Res. 2001;3:347–52.
Moore DA, Goodall RL, Ives NJ, Hooker M, Gazzard BG, Easterbrook PJ. How generalizable are the results of large randomized controlled trials of antiretroviral therapy? HIV Med. 2001;1:149–54.
Licht RW, Gouliaev G, Vestergaard P, Frydenberg M. Generalisability of results from randomised drug trials. A trial on antimanic treatment. Br J Psychiatry. 1997;170:264–7.
Van Spall HGC, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297:1233–40.
Usmani ZC, Craig P, Shipton D, Tappin D. Comparison of CO breath testing and women’s self-reporting of smoking behaviour for identifying smoking during pregnancy. Subst Abuse Treat Pr. 2008;3:4.
Kotz D, Fidler J, West R. Very low rate and light smokers: smoking patterns and cessation-related behaviour in England, 2006–11. Addiction. 2012;107(5):995–1002. doi:10.1111/j.1360-0443.2011.03739.x.
Mrs Lydia A. Yamoah: Principal of Midwifery and Health Care Assistants Training College, Mampong Ashanti. Ghana. She contributed critically to the drafting, fine tuning and proofreading the draft. Commonwealth Commission UK for providing financial support to undertake the Master’s degree program and the thesis project.
We would like to acknowledge the help and support of NHS Greater Glasgow and Clyde R&D department, in particular Dr Roma Armstrong and Brenda Colvin, without whose help this trial would not have been possible.
This study was funded by a grant from the Chief Scientist Office Scottish Government CZH/4/594, Glasgow Centre for Population Health, NHS Greater Glasgow and Clyde Endowments, the Royal Samaritan Endowment Fund and Glasgow Children’s Hospital Charity.
Availability of data and materials
Raw data from this trial may be made available on request by Professor Tappin.
BB conceived the study, was involved in designing the study, data analyses and drafting of the article. LB was involved in designing the study and drafting the article. LS was involved in data acquisition and cleaning, designing the study, and drafting and proofreading the article. DM was involved in data cleaning, analyses and proofreading the article. WS was involved in designing the study, and drafting and proofreading the article. DT conceived the study, was involved in designing the study, and drafting and proofreading the article. All authors read and approved the final manuscript.
1. Barnabas Bessing (field epidemiologist): WHO International STOP Consultant.
Has extensive midwifery/nursing teaching and public health working experience. Undertook this work as a Master’s thesis, and was involved in design, data analyses, and writing the paper.
2. Linda Bauld (Co-principal Investigator): Professor of Health Policy, University of Stirling and UK Centre for Tobacco and Alcohol Studies, has extensive experience of quantitative work related to smoking cessation. She supported the trial on a day-to-day basis and was involved in writing the paper.
3. Lesley Sinclair (Trial Manager): has experience of data management and the management of clinical trials. She ran the trial on a day-to-day basis, assisted with study design, and data acquisition and cleaning for analyses.
4. Daniel F Mackay: Reader in Public Health, has extensive experience in quantitative research work and data analyses. He assisted in the organisation, analyses of the data and was involved in the supervision of the project work.
5. William Spence: University Teacher in Public Health, has extensive experience in qualitative work and teaching. He was fully involved in the project design, supervision, and in writing the paper.
6. David Tappin (Co-Principal Investigator): Professor of Clinical Trials for Children is based at Glasgow University. He coordinated and managed the overall running of the trial, fully supervised this project and was closely involved in data analyses and in writing the paper.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The Cessation in Pregnancy Incentives Trial (CPIT) was approved by the NHS West of Scotland Research Ethics Committee 2 (11/AL/0204) and a substantial amendment granted by the NHS West of Scotland Research Ethics Subcommittee 4 allowed anonymous use of routine data on nontrial participants without direct patient consent. The Caldicott guardian also granted approval for BB to use patient-level data without direct patient identifiers. Trial participants in CPIT gave individual consent.
About this article
Cite this article
Bessing, B., Bauld, L., Sinclair, L. et al. Representativeness of the participants in the smoking Cessation in Pregnancy Incentives Trial (CPIT): a cross-sectional study. Trials 17, 426 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-016-1552-5
- Clinical markers
- Cross-sectional studies
- Data collection
- Treatment effectiveness