Skip to main content

A systematic review of the efficiency of recruitment to stroke rehabilitation randomised controlled trials

Abstract

Introduction

Randomised controlled trials (RCTs) that fail to meet their recruitment target risk increasing research waste. Acute stroke RCTs experience notable recruitment issues. The efficiency of recruitment to stroke rehabilitation RCTs has not been explored.

Aims and objectives

To explore recruitment efficiency and the trial features associated with efficient recruitment to stroke rehabilitation RCTs.

Methods

A systematic review of stroke rehabilitation RCTs published between 2005 and 2015 identified in a search of the Cochrane Stroke Group (CSG) Trials Register from 35 electronic databases (e.g. Medline, CINAHL; EMBASE), clinical trial registers, and hand-searching. Inclusion criteria are stroke rehabilitation intervention, delivered by a member of the rehabilitation team, and clinically relevant environment. We extracted data on recruitment efficiency and trial features.

Results

We screened 12,939 titles, 1270 abstracts and 788 full texts, before extracting data from 512 included RCTs (n = 28,804 stroke survivor participants). This is the largest systematic review of recruitment to date. A third of stroke survivors screened consented to participate (median 34% (IQR 14–61), on average sites recruited 1.5 participants per site per month (IQR 0.71–3.22), and one in twenty (6% (IQR 0–13) dropped out during the RCT. Almost half (48%) of those screened in the community were recruited compared to hospital settings (27%). Similarly, almost half (47%) those screened at least 6 months after stroke participated, compared to 23% of stroke survivors screened within a month of stroke. When one recruiter screened multiple sites, a median of one stroke survivor was recruited every 2 months compared to more than two per month when there was a dedicated recruiter per site. RCT recruitment was significantly faster per site, with fewer dropouts, for trials conducted in Asia (almost three stroke survivors monthly; 2% dropout) compared to European trials (approximately one stroke survivor monthly; 7% dropout).

Conclusions

One third of stroke survivors screened were randomised to rehabilitation RCTs at a rate of between one and two per month, per site. One in twenty did not complete the trial. Our findings will inform recruitment plans of future stroke rehabilitation RCTs. Limited reporting of recruitment details restricted the subgroup analysis performed.

Trial registration

Prospective Register of Systematic Reviews, registration number CRD42016033067.

Peer Review reports

Background

An estimated £132 billion of research funding is wasted each year [1] and recruitment issues are thought to be one of the key contributors to research waste [2]. Successfully meeting recruitment targets is vital to ensure statistically significant results are correctly identified (protect against type 1 and type 2 errors) [3,4,5] and to ensure accurate interpretation of statistical effect sizes (how effective an intervention has been) [6,7,8,9]. Randomised controlled trials (RCTs) experience a range of recruitment inefficiencies, including falling to achieve recruitment targets, exceeding the planned timeframe, requiring recruitment extensions, and early termination [3, 5, 10,11,12,13,14]. Between 1994 and 2002 less than one-third of UK trials funded by Health Technology Assessment (HTA) or Medical Research Council (MRC) met their recruitment targets [10]. In an update, just over half the trials published between 2002 and 2008 met recruitment targets, though the majority recruited at least 80% of their original target [15]. Trials that fail to retain their desired sample throughout the duration of the study also contribute to research waste [16, 17], impacting on the validity, reliability, and generalisability of results [7, 18,19,20].

RCT reports may lack important recruitment details [21,22,23] which hinders learning from past recruitment experiences amongst similar trial populations or recruitment contexts. Consolidated Standards of Reporting Trials (CONSORT) provides a checklist for reporting standards and a flow diagram to illustrate the movement of participants through an RCT [21, 22]. Despite this, a review of six major journals found that, although 79% of the included trials reported a CONSORT flow diagram, one-third of these were incomplete and 40% did not include the numbers screened for trial eligibility [22].

Recruitment of stroke survivors to clinical trials is challenging [13, 14, 24]. In the UK more than 100,000 people experience a stroke each year [25, 26] and there are currently 1.2 million people living with stroke-related impairments [27,28,29]. Stroke rehabilitation aims to maximise recovery, enabling stroke survivors to regain their confidence, independence, and quality of life [30]. In order to ensure stroke survivors receive the best treatment available, RCTs assess the effectiveness of rehabilitation interventions [31]. A systematic review and meta-analysis of 114 large scale (> 300 participants) acute (within 1 month after stroke) pharmacological stroke trials published between 1990 and 2004 reported a mean recruitment rate of 0.79 participants per site per month [13]. An update of this study revealed a median recruitment rate of 0.41 participants per site per month for trials published between 2010 and 2014 [24]. Despite notable difficulties recruiting stroke survivors, and limited improvement over the past 27 years [13, 14], little research has focused on recruitment of stroke survivors to rehabilitation RCTs.

A recent James Lind Alliance priority setting partnership looking at priorities for recruitment research highlighted that improving future recruitment predictions is a key priority [32]. The lack of a recruitment evidence base leads trialists to rely on past experience when anticipating or forecasting recruitment to their new RCTs [33]. Trialists tend to base their predictions on studies with positive recruitment experiences rather than trials that experienced recruitment challenges [34]. A recruitment evidence base would reduce trialists’ reliance on past experiences when planning future stroke rehabilitation RCTs [33].

Aims and objectives

The aim of our study was to examine recruitment to stroke rehabilitation RCTs published from 2005 to 2015. Specifically, we explored the recruitment efficiency of the RCTs, determined whether specific trial features impacted upon recruitment efficiency, and explored the reporting standards of the included RCTs.

Methods

Protocol registration and ethics

Our systematic review protocol was registered with the international Prospective Register of Systematic Reviews (CRD42016033067). No ethical approval was required. Data extracted were securely stored on a password-protected computer and was fully anonymised at the point of data extraction. This systematic review was designed in accordance with the Preferred Reporting Items and Systematic Reviews and Meta-Analysis (PRISMA) reporting guidelines where applicable.

Inclusion criteria

  • Published RCTs (described as RCTs in the paper) that compared a rehabilitation intervention with a control condition (usual care, active control, or an attention control) or another treatment

  • Interventions delivered by multi-disciplinary stroke team members (physiotherapist, occupational therapist, speech and language therapist, physician, nurse, or psychologist)

  • Participants were stroke survivors (including the control group)

  • Interventions were delivered in a stroke rehabilitation location (hospital, outpatient clinic, rehabilitation ward, home, community, nursing home, or support group)

Identification of studies

We manually hand searched the entire Cochrane Stroke Group (CSG) specialised trials register. We systematically searched for all RCTs published between January 2005 and December 2015. We applied no language restrictions. The CSG trials register contains trials identified from 37 major electronic databases (including MEDLINE, EMBASE, CINAHL, EBSCO, AMED, EMBASE classic, PUBMED, PSYCBITE, PSYCHINFO, and CENTRAL). The register also contains more than 25 clinical trial registers and hand searches of approximately 300 stroke-specific conferences, 150 neurology conferences, 40 neurosurgery conferences, 220 rehabilitation conferences, 60 geriatric conferences, and 96 books. The search strategy used by CSG for MEDLINE has been provided as an example (Additional file 1: Supplement A). For complete details of the search strategies see http://www.dcn.ed.ac.uk/csrg/entity/searchmethods.pdf.

Outcomes

Recruitment efficiency was the primary outcome and measured in terms of the:

  1. 1)

    Randomisation rate (the number of participants randomised as a percentage of the total number of participants screened for eligibility)

  2. 2)

    Recruitment rate (number of participants randomized by the time spent recruiting in months by site)

  3. 3)

    Dropout (the number of participants that failed to complete the trial as a percentage of the number randomised)

Adherence to the CONSORT reporting standards [35] were evaluated in relation to each trial’s report of the number of participants randomised, location and settings, baseline demographics, dropouts, the period of recruitment and follow-up, source of funding, and sample size calculations. A three-tier system was used to classify the reporting (fully reported, partially reported, and not reported).

Selection of studies, data extraction and management

All abstracts and full texts were independently screened for eligibility by two reviewers (KMcG and JMcG). Using a pro-forma developed and piloted for this review, data extraction was carried out independently, with reference to the full text and in correspondence with the trialists where possible. Discrepancies were discussed and resolved, with discussion with a third reviewer where required. CONSORT items relating to recruitment were extracted and categorised as “reported”, “partially reported” or “not reported”. Where data extraction items were unavailable from the published article, trialists were contacted via email. Where a reply was not received within 3 weeks, the original email was resent (Additional file 1: Supplement B) . All data were transferred to SPSS for analysis. We extracted data on trial features (Table 1) and relevant CONSORT items (Table 2).

Table 1 Recruitment and trial characteristics extracted from included stroke rehabilitation RCTs
Table 2 CONSORT checklist recruitment data items for RCTs

Included studies were described in the publications as RCTs. No evaluation of the quality of the randomisation process or other standard risk of bias assessments were undertaken since the purpose of this study was not to consider the effectiveness of a specific intervention [11, 14, 15, 24, 36,37,38,39]. Instead, our systematic review adopted an inclusive approach to the RCTs identified allowing a comprehensive evaluation of recruitment to stroke rehabilitation RCTs.

Data analysis plan

Randomisation rate, recruitment rate and dropout were stratified into trial and recruitment characteristics to allow for subgroup analysis using Kruskal–Wallis tests [40]. Where appropriate, Mann–Whitney U tests were used for post hoc analysis in order to explore the significant effect highlighted by the Kruskal–Wallis tests. Trial and recruitment characteristics were only used for subgroup analysis if group sizes were considered balanced [7, 40, 41]. Basic power calculations were referred to, to determine appropriate subgroup analysis (comparing two groups each of N subjects leads to the requirement N > [4/“effect”]2 for the detection of a difference at p = 0.05 and 80% power, where “effect” is in standard deviations [42]). Bonferroni corrections were manually applied to all post hoc analyses (providing corrected alpha values) in order to control for the effect of multiple comparison testing [43]. The accepted alpha value (0.05) was divided by the number of comparison groups. CONSORT reporting items were evaluated using descriptive statistics and displayed within a table and line graph.

Distribution of data

Kolmogorov–Smirnov (K-S) analysis was used to statistically test the outcome variables against a normally distributed bell curve [44]. The K-S highlighted non-normative distribution for all three dependent variables (< 0.001). We used non-parametric Kruskal–Wallis tests. Recruitment and trial characteristics were analysed in independent groups, typically with three or more variables. Three of the trial features (publication date, ethics approval, and type of intervention) had two grouping levels and therefore non-parametric Mann–Whitney U tests were adopted.

Results

Of 12,939 titles identified (Fig. 1), 1270 abstracts and 788 full texts were reviewed. We extracted data from 512 stroke rehabilitation RCTs reflecting the randomisation of 28,804 stroke survivors. A range of trial characteristics (Table 1) were used for the all subsequent analyses. A number of trial features were excluded from subgroup analysis because of insufficient data and imbalances across the groups (Table 3), leading to highly uneven group sizes which risked the production of misleading results [40, 41].

Fig. 1
figure 1

PRISMA diagram showing the flow of systematic identification, screening, inclusion and exclusions of records identified

Table 3 Trial and recruitment characteristics for the included 512 stroke rehabilitation RCTs

Recruitment efficiency

The median randomisation rate for stroke rehabilitation trials was 34% (RCTs 321, IQR 47%, range 2% to 100%). The median recruitment rate was 1.5 participants per site per month (RCTs 242, IQR 2.51, range 0.08 to 40). The median dropout rate was 6% (RCTs 414, IQR 13%, range 0% to 83%) Table 4.

Randomisation rate was significantly affected by stroke survivors’ living context, their stage of rehabilitation, the trial’s recruitment strategy, and the number of trial recruiters (Table 5). Post hoc analyses were performed for each of the significant effects. Mann–Whitney U tests indicated that recruitment from the community had a significantly higher randomisation rate (Mdn = 0.48) than recruitment from a rehabilitation or stroke-specific environment (Mdn = 0.27; U = 4210.50; p = 0.003). Screening community-dwelling stroke survivors had a significantly higher randomisation rate (Mdn = 0.49) than screening admissions (Mdn = 0.22; U = 725; p = 0.001). There was a significant difference between recruitment during acute (Mdn = 0.23) and chronic stages after stroke (Mdn = 0.47; U = 2723.5; p = 0.001). No other post hoc comparisons were significant. The trial features that did not have a significant effect on recruitment efficiency were RCT publication date, trial size, type of intervention, funding support, ethical approval, target impairment, control condition, recruiter’s profession(s), number of recruiters per site and continent of recruitment (Additional file 1: Supplement C).

Table 4 Trial recruitment details and primary recruitment efficiency outcomes (randomisation rate, recruitment rate and dropout) for included stroke rehabilitation RCTs
Table 5 Trial and recruitment characteristics that significantly affected randomisation rate for included stroke rehabilitation RCTs

Recruitment rate was significantly affected by trial size, targeted impairment, continent of recruitment, and recruiters per site (Table 6). Post hoc analyses were performed to further explore potential effects. Mann–Whitney U tests indicated a significantly slower recruitment rate for the trials with 21 or less participants (Mdn = 0.83) when compared to 35–60 (Mdn = 2.45; U = 940.5; p < 0.001). There was a significantly faster recruitment rate within RCTs based in Asia (Mdn = 2.68) compared to European RCTs (Mdn = 1.28; U = 1969; p > 0.001) and North American RCTs (Mdn = 1.35; U = 706.5; p < 0.001). RCTs conducted in Asia recruited at least one more patient per site per month than either European or North American RCTs.

Table 6 Trial and recruitment characteristics that significantly affected recruitment rate for included stroke rehabilitation RCTs

Recruitment rate was significantly slower when recruiters had more than one site (or were not full-time at the site; Mdn = 0.54) when compared to one recruiter per site (Mdn = 2.14; U = 152.5; p = 0.001), when compared to between one and two recruiters per site (Mdn = 1.5; U = 174.5; p = 0.008), or when compared to more than two recruiters per site (Mdn = 1.94; U = 104.5; p < 0.001). There was a significantly faster recruitment rate for interventions targeting gains in overall disability (Mdn = 2.16) compared to trials which targeted improvements in arm function (Mdn = 1.34; U = 2071; p = 0.001), cognition or vision (Mdn = 0.95; U = 764.5; p = 0.006). There was also a significantly faster recruitment rate for interventions targeting leg function (Mdn = 1.84) when compared to trials that targeted gains in arm function (Mdn = 1.34; U = 1728; p = 0.01), cognition or vision (Mdn = 2.16; U = 635; p = 0.006). No other significant comparisons were indicated. The trial features that did not have a significant effect were publication date, living context, type of intervention, funding support, ethical approval, stage of stroke rehab, control condition, recruitment strategy, and recruiters per site (Additional file 1: Supplement C).

Dropout rate was significantly affected by publication date, trial size, continent of recruitment, and recruitment strategy (Table 7). Post hoc analysis was then used to further explore effects. Mann–Whitney U test indicated a significantly higher dropout rate for RCTs published in or after 2010 (Mdn = 0.09) when compared to RCTs published in 2009 or before (Mdn = 0.03; U = 17,390; p = 0.001). There was a significantly lower dropout rate for RCTs with 21 or fewer participants (Mdn = 0) when compared to RCTs with sample sizes of between 22 to 34 participants (Mdn = 0.07; U = 3532; p = 0.001), between 35 to 60 participants (Mdn = 0.08; U = 2999; p < 0.001), between 60 to 99 participants (Mdn = 0.08; U = 1554; p < 0.001), and trials with at least 100 participants (Mdn = 0.008; U = 1350; p < 0.001). RCTs conducted in Asia reported significantly lower dropout rates (Mdn = 0.02) when compared to RCTs conducted in Europe (Mdn = 0.07; U = 6694; p = 0.002) or North America (Mdn = 0.08; U = 3747; p = 0.005). RCTs that screened inpatients experienced significantly lower dropout rates (Mdn = 0.03) than trials that screened hospital admissions (Mdn = 0.09; U = 2788; p = 0.015). The trial features that were found to have no significant effect included stroke survivor living context, type of intervention, funding support, ethical approval, targeted impairment, stage of stroke rehabilitation, control condition, profession of recruiter, recruiters per site (Additional file 1: Supplement C). All non-significant post-hoc test results have been provided (Additional file 1: Supplement D).

Table 7 Trial and recruitment characteristics that significantly affected dropout for included stroke rehabilitation RCTs

Consort reporting standards

The CONSORT items related to recruitment (CONSORT diagram, inclusion and exclusion criteria, who enrolled participants, numbers randomised, dropouts, source of funding, location and settings, data defining the period of recruitment and follow up, sample size calculations, baseline demographics) that were extracted from the 512 RCTs included in the systematic review are presented (Table 8 and Fig. 2). The percentage of RCTs fully reporting the CONSORT diagram improved from 32% to 81% between 2005 and 2010. The percentage of trial reports that did not include a CONSORT diagram fell steadily from 64% to 11% over the same period (Fig. 2).

Table 8 Number and percentage of RCTs that included each CONSORT reporting item associated with recruitment
Fig. 2
figure 2

Reporting, partial reporting and not reporting CONSORT diagram in published reports of stroke rehabilitation RCTs published between 2005 and 2015

Discussion

We explored the recruitment efficiency of stroke rehabilitation RCTs over a 10-year period. We found that one-third of stroke survivors screened were randomised into the trial, between one and two were recruited per site per month, and one in twenty stroke survivors randomised did not complete the trial. Stroke survivors were recruited most efficiently from i) the community, ii) utilising two or three recruiters per site and iii) within the chronic stage of recovery (more than 6 months after stroke). The slowest recruitment rate was experienced by i) the smallest RCTs (less than 21 participants), ii) RCTs conducted in Europe or North America and iii) RCTs using one recruiter to cover multiple sites. The lowest reported dropout was experienced by RCTs i) published before 2009, ii) conducted in Asia and iii) with the smallest sample sizes (less than 21 participants).

Our systematic review illustrates that stroke rehabilitation RCTs have intensive randomisation rates and relatively slow recruitment [3,4,5, 10, 13, 14]. Interestingly, the recruitment rate we observed for stroke rehabilitation RCTs was faster than the rate experienced by acute stroke RCTs [13, 14]. Stroke rehabilitation RCTs may have more freedom to recruit stroke survivors from across different contexts because of the multiple locations and environments in which rehabilitation takes place. Stroke survivors who are no longer in an acute setting may be more able to attend and commit to clinical trials. However, although it may be more efficient to recruit chronic stroke survivors living at home, stroke rehabilitation improvements are known to be greatest within the early stages following stroke onset. Thus, recruitment plans should be balanced with the trial objectives.

CONSORT reporting standards

Almost all included RCTs reported inclusion and exclusion criteria, numbers randomised, location and setting of trial, baseline demographics, and dropout. Less than a fifth reported the staff members involved with enrolment and less than half reported the recruitment duration. A priori sample size calculations were reported in less than a third of included RCTs and one-third of RCTs did not report their funders. CONSORT diagrams were fully reported by less than half of stroke rehabilitation RCTs, reflecting an improvement in trial reporting over the past 10 years [22]. This improvement came after the first major CONSORT update in 2010 [35] and is likely to reflect changes in editorial requirements from journals.

Faster recruitment and less dropout for RCTs conducted in Asia

RCTs conducted in Asia recruited one more patient per site per month and experienced 5% less dropout when compared to RCTs conducted in Europe or America. Cultural differences may have contributed to this effect, with participants in Asian countries being motivated more by societal collectivism when compared to western individualistic societies [45,46,47,48]. Collectivist societies tend to place importance on social identity and societal benefit [47]. In comparison, individualistic ideologies place more importance on self-importance and self-gain [48]. Stroke survivors based in Asian countries may be more willing to take part in RCTs because they find value in the potential societal benefits. Another potential influencing factor is the difference in medical staff–patient relationships [49,50,51,52]. Southeast Asia tends to have a more authoritarian approach to this interaction, with medical staff directing patients with little feedback [49, 53]. Furthermore, recruitment for RCTs conducted in Asia may be considered unethical by western standards, potentially affecting recruitment rate by reducing the need for some safeguards associated with modern research governance [54,55,56,57]. Of 9488 Chinese RCTs published between 2013 and 2016 across 12 nursing journals, informed consent was reported in only 51.8% of RCTs, and written consent in only 27.4%. Potentially, ethical processes for Asian trials could be improved by targeted research governance initiatives, leading to more trials using informed consent.

Recruitment from the community

For stroke rehabilitation RCTs community-based trials appear to have the most efficient recruitment. During the chronic stage of recovery stroke survivors may be in a better position to be approached regarding trial participation because the immediate psychological and physical implications for the stroke survivor have subsided [25, 26] and the most intensive period of standard rehabilitation is likely to have been completed [25, 26]. By recruiting chronic stroke survivors within the community there may be more time to form relationships and this approach could increase trust during the recruitment conversation [58, 59]. Recruitment forecasting should carefully consider the location of recruitment and the stage of rehabilitation when allocating recruitment time.

Recruitment staffing levels

Having one recruiter covering multiple sites who has a high work load may affect the ability to identify, screen, and approach potentially eligible participants for recruitment [10]. This is potentially contributed to by under-resourcing during trial planning [33], affecting the amount of time the clinical staff have to recruit [60]. However, the evidence did not support the notion that more recruiters led to more efficient recruitment. This may be caused by the lack of personal responsibility for a site’s recruitment target and accountability for recruitment failures. This may be caused by diffusion of responsibility [61,62,63] where individuals do not feel personally responsible for a task owing to the belief that someone else will do it, and this is effected by how competent an individual feels in their ability to complete a task [64].

Strengths of our systematic review

We adopted rigorous methods to search, screen, extract, and analyse all included stroke rehabilitation RCTs. The comprehensive nature of our search strategy supported the inclusion of all stroke rehabilitation RCTs published between 2005 and 2015. Two researchers independently screened all identified trials to ensure that all decisions were independently checked. Our systematic review included 512 stroke rehabilitation RCTs, making it the largest systematic review of trial recruitment conducted to date. Our large sample size ensured that our analysis had the statistical power to confidently protect against type 1 and type 2 errors and supported subgroup analysis of the trial features that could be important for recruitment efficiency. In total, additional information was provided by 177 authors, substantially increasing the availability of data for subgroup analysis.

Limitations of the systematic review

RCTs were included if the author described randomization methods. Inclusion of RCTS without quality appraisal was intentional in order to include and evaluate stroke rehabilitation RCTs regardless of quality. This allowed for the exploration of recruitment to all stroke rehabilitation RCTs, creating a well-rounded picture of the body of research. The ability to include RCTs not published in English could have contributed to the investigation of more trial features that impact recruitment efficiency. Unfortunately, the lack of reporting of recruitment details by RCTs governed some of the subgroup analysis that could be conducted. Some trial features that could have provided an insight into what makes for successful recruitment may have been missed. We did not extract individual participant level data on stroke severity (NIHSS for example) as our analysis was based only on group summary data. It is possible that stroke profile factors (such as severity of stroke, ability to consent, communication impairment) are important factors for trial participation at an individual participant level but these were not examined in our review.

Implications

The output from this study provides an evidence base for stroke rehabilitation trialists planning future RCTs. This evidence base could allow for more efficient recruitment to stroke rehabilitation RCTs through more accurate recruitment forecasting and knowledge of what trial features appear to facilitate efficient recruitment. We hope in future to observe a reduction in a) the number of trials that do not reach their target sample size, b) the production of inaccurate and unusable outcomes and c) the number of costly extensions and trial terminations. Better recruitment to stroke rehabilitation trials would lead to more accurate assessments of experimental treatment effects [3,4,5, 65], contributing to a more robust evidence base for stroke rehabilitation clinicians and better rehabilitation for stroke survivors.

Conclusions

Stroke rehabilitation RCTs experience recruitment inefficiencies including intensive screening rates and slow recruitment. RCTs recruiting stroke survivors living at home several months after their stroke had the highest level of recruitment efficiency. One recruiter based at a single site was the most efficient approach while a single recruiter covering multiple sites was the least efficient. Trials conducted in Asia experience significantly faster recruitment and fewer dropouts. Our findings will assist recruitment planning of future stroke rehabilitation RCTs contributing to a reduction in research waste amongst future stroke rehabilitation trials.

Availability of data and materials

The extracted data used and analysed during the current study are available from the corresponding author upon request.

Abbreviations

BMJ:

British Medical Journal

CONSORT:

Consolidation of Standards for Reporting Trials

CSG:

Cochrane Stroke Group

HTA:

Health Technology Assessment

JAMA:

Journal of the American Medical Association

MRC:

Medical Research Council

RCT:

Randomised controlled trial

SD:

Standard deviation

References

  1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

    Article  PubMed  Google Scholar 

  2. Salman RA-S, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85.

    Article  PubMed Central  Google Scholar 

  3. Foy R, Parry J, Duggan A, Delaney B, Wilson S. Lewin-van den Broek N, et al. How evidence based are recruitment strategies to randomized controlled trials in primary care? Experience from seven studies. Fam Pract. 2003;20(1):83–92.

    Article  CAS  PubMed  Google Scholar 

  4. Haidich A-B, Ioannidis JP. Determinants of patient recruitment in a multicenter clinical trials group: trends, seasonality and the effect of large studies. BMC Med Res Methodol. 2001;1(1):1–11.

    Article  Google Scholar 

  5. Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrøm M, Johansen M, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2):e002360.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365–76.

    Article  CAS  PubMed  Google Scholar 

  7. Moore R, Gavaghan D, Tramer M, Collins S, McQuay H. Size is everything–large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects. Pain. 1998;78(3):209–16.

    Article  CAS  PubMed  Google Scholar 

  8. Turner RM, Bird SM, Higgins JP. The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews. PLoS One. 2013;8(3):e59202.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Thorlund K, Imberger G, Walsh M, Chu R, Gluud C, Wetterslev J, et al. The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis—a simulation study. PLoS One. 2011;6(10):e25491.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9–16.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Hadidi N, Buckwalter K, Lindquist R, Rangen C. Lessons learned in recruitment and retention of stroke survivors. J Neurosci Nurs. 2012;44(2):105–10.

    Article  PubMed  Google Scholar 

  12. Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrøm M, Taskila T, et al. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2010;4(4):MR000013.

    Google Scholar 

  13. Elkins JS, Khatabi T, Fung L, Rootenberg J, Johnston SC. Recruiting subjects for acute stroke trials a meta-analysis. Stroke. 2006;37(1):123–8.

    Article  PubMed  Google Scholar 

  14. Feldman WB, Kim AS, Chiong W. Trends in recruitment rates for acute stroke trials, 1990–2014. Stroke. 2017;48(3):799–801.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Sully BG, Julious SA, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14(1):166–9.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Brueton VC, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2013;12:MR000032.

    Google Scholar 

  17. Brueton V, Tierney J, Stenning S, Meredith S, Harding S, Nazareth I, et al. Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis. BMJ Open. 2014;4(2):e003821.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Fewtrell MS, Kennedy K, Singhal A, Martin RM, Ness A, Hadders-Algra M, et al. How much loss to follow-up is acceptable in long-term randomised trials and prospective studies? Arch Dis Child. 2008;93(6):458–61.

    Article  PubMed  Google Scholar 

  19. Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet. 2002;359(9308):781–5.

    Article  PubMed  Google Scholar 

  20. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. The CONSORT group. CONSORT transparent reporting of trials 2016 Available from: http://www.consort-statement.org/.

    Google Scholar 

  22. Toerien M, Brookes ST, Metcalfe C, De Salis I, Tomlin Z, Peters TJ, et al. A review of reporting of participant recruitment and retention in RCTs in six major journals. Trials. 2009;10(1):52.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Hopewell S, Dutton S, Yu L-M, Chan A-W, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010;340:c723.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Feldman WB, Kim AS, Josephson SA, Lowenstein DH, Chiong W. Effect of waivers of consent on recruitment in acute stroke trials a systematic review. Neurology. 2016;86(16):1543–51.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Scottish Stroke Care Audit. Scottish stroke improvement programme, 2018 national report. 2018.

  26. Royal College of Physicians. Sentinel Stroke National Audit Programme (SSNAP), national clinical audit annual results portfolio. 2017.

  27. Scarborough P, Peto V, Bhatnagar P, Kaur A, Leal J, Luengo-Fernandez R, et al. Stroke statistics. Oxford: British Heart Foundation & Stroke Association; 2009.

    Google Scholar 

  28. Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, et al. Global and regional burden of stroke during 1990–2010: findings from the Global Burden of Disease Study 2010. Lancet. 2014;383(9913):245–55.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Rudd AG, Wolfe CD. Age and ethnic disparities in incidence of stroke over time. Stroke. 2013;44(12):3298–304.

    Article  PubMed  Google Scholar 

  30. World Health Organization. World report on disability, chapter 4 rehabilitation. 2011.

  31. Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet. 2011;377(9778):1693–702.

    Article  PubMed  Google Scholar 

  32. Healy P, Galvin S, Williamson PR, Treweek S, Whiting C, Maeso B, et al. Identifying trial recruitment uncertainties using a James Lind Alliance Priority Setting Partnership–the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials. 2018;19(1):147.

    Article  PubMed  PubMed Central  Google Scholar 

  33. White D, Hind D. Projection of participant recruitment to primary care research: a qualitative study. Trials. 2015;16(1):473.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185(4157):1124–31.

    Article  CAS  PubMed  Google Scholar 

  35. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux P, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–e37.

    Article  PubMed  Google Scholar 

  36. Haidich A-B, Ioannidis JP. Determinants of patient recruitment in a multicenter clinical trials group: trends, seasonality and the effect of large studies. BMC Med Res Methodol. 2001;1(1):4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. McGill K, Brady MC, Sackley C, Godwin J. Efficiency of recruitment to stroke rehabilitation randomised controlled trials: secondary analysis of recruitment data. In: Feinberg J, Ashton C, Hirst A, Pennell C, McCulloch P, editors. Trials: Meeting abstracts from the 4th International Clinical Trials Methodology Conference (ICTMC) and the 38th Annual Meeting of the Society for Clinical Trials 2017. p. 200.

  38. McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Howard L, de Salis I, Tomlin Z, Thornicroft G, Donovan J. Why is recruitment to trials difficult? An investigation into recruitment difficulties in an RCT of supported employment in patients with severe mental illness. Contemp Clin Trials. 2009;30(1):40–6.

    Article  PubMed  PubMed Central  Google Scholar 

  40. MacFarland TW, Yates JM. Kruskal–Wallis H-test for oneway analysis of variance (ANOVA) by ranks. In: MacFarland TW, Yates JM editors Introduction to nonparametric statistics for the biological sciences using R. Springer: Berlin; 2016. p. 177–211.

    Chapter  Google Scholar 

  41. Bland JM, Altman DG. Analysis of continuous data from small samples. BMJ. 2009;338:a3166.

    Article  PubMed  Google Scholar 

  42. Armitage P, Berry G, Matthews JNS. Statistical methods in medical research. Hoboken: Wiley; 2008.

    Google Scholar 

  43. Keppel G, Wickens T. Simultaneous comparisons and the control of type I errors. Design and analysis: A researcher’s handbook 4th ed. Upper Saddle River: Pearson Prentice Hall; 2004. p. 111–30.

    Google Scholar 

  44. Massey FJ Jr. The Kolmogorov-Smirnov test for goodness of fit. J Am Stat Assoc. 1951;46(253):68–78.

    Article  Google Scholar 

  45. Greif A. Cultural beliefs and the organization of society: a historical and theoretical reflection on collectivist and individualist societies. J Polit Econ. 1994;102(5):912–50.

    Article  Google Scholar 

  46. Ralston DA, Holt DH, Terpstra RH, Kai-Cheng Y. The impact of national culture and economic ideology on managerial work values: a study of the United States, Russia, Japan, and China. J Int Bus Stud. 1997;29:177–207.

    Article  Google Scholar 

  47. Triandis HC. Individualism & collectivism. Boulder: Westview Press; 1995.

    Google Scholar 

  48. Oyserman D, Coon HM, Kemmelmeier M. Rethinking individualism and collectivism: evaluation of theoretical assumptions and meta-analyses. Physchol Bull. 2002;128(1):3–72.

    Article  Google Scholar 

  49. Claramita M, Nugraheni MD, van Dalen J, van der Vleuten C. Doctor–patient communication in Southeast Asia: a different culture? Adv Health Sci Educ. 2013;18(1):15–31.

    Article  Google Scholar 

  50. Cong Y. Doctor-family-patient relationship: the Chinese paradigm of informed consent. J Med Philos. 2004;29(2):149–78.

    Article  PubMed  Google Scholar 

  51. Hou X, Xiao L. An analysis of the changing doctor-patient relationship in China. J Int Bioethique. 2012;23(2):83–94.

    Article  PubMed  Google Scholar 

  52. Kaba R, Sooriakumaran P. The evolution of the doctor-patient relationship. Int J Surg. 2007;5(1):57–65.

    Article  CAS  PubMed  Google Scholar 

  53. Getz KA. Informed consent process. Appl Clin Trials. 2002;11(11):30–6.

    Google Scholar 

  54. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19(2):159–66.

    Article  CAS  PubMed  Google Scholar 

  55. Pittler M, Abbot N, Harkness E, Ernst E. Location bias in controlled clinical trials of complementary/alternative therapies. J Clin Epidemiol. 2000;53(5):485–9.

    Article  CAS  PubMed  Google Scholar 

  56. Moher D, Fortin P, Jadad AR, Jüni P, Klassen T, Le Lorier J, et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet. 1996;347(8998):363–6.

    Article  CAS  PubMed  Google Scholar 

  57. Wu Y, Howarth M, Chunlan Z, Xue J, Jiexia O, Xiaojin L. Reporting of ethical approval and informed consent in clinical trials in 12 nursing journals in China between 2013 and 2016. Nurs Ethics. 2017:1–11.

  58. Kong DT, Dirks KT, Ferrin DL. Interpersonal trust within negotiations: meta-analytic evidence, critical contingencies, and directions for future research. Acad Manag J. 2014;57(5):1235–55.

    Article  Google Scholar 

  59. George S, Duran N, Norris K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders. Am J Public Health. 2014;104(2):e16–31.

    Article  PubMed  PubMed Central  Google Scholar 

  60. French C, Stavropoulou C. Specialist nurses’ perceptions of inviting patients to participate in clinical research studies: a qualitative descriptive study of barriers and facilitators. BMC Med Res Methodol. 2016;16(1):96.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Garcia SM, Weaver K, Moskowitz GB, Darley JM. Crowded minds: the implicit bystander effect. J Pers Soc Psychol. 2002;83(4):843–53.

    Article  PubMed  Google Scholar 

  62. Fischer P, Krueger JI, Greitemeyer T, Vogrincic C, Kastenmüller A, Frey D, et al. The bystander-effect: a meta-analytic review on bystander intervention in dangerous and non-dangerous emergencies. Psychol Bull. 2011;137(4):517–37.

    Article  PubMed  Google Scholar 

  63. Darley JM, Latane B. Bystander intervention in emergencies: diffusion of responsibility. J Pers Soc Psychol. 1968;8(4):377–83.

    Article  CAS  PubMed  Google Scholar 

  64. Cramer RE, Mcmaster MR, Bartell PA, Dragna M. Subject competence and minimization of the bystander effect. J Appl Soc Psychol. 1988;18(13):1133–48.

    Article  Google Scholar 

  65. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We acknowledge all trialists who shared additional recruitment information for this project, stroke survivors and their families who took part in the RCTs, and Dr. Pauline Campbell and the CSG for their assistance with identifying studies for inclusion.

Funding

This study was funded as a part of a PhD studentship at Glasgow Caledonian University, and was undertaken within the Nursing Midwifery and Allied Health professions Research Unit, which is funded by the Chief Science Office as part of the Scottish Government’s Health and Social Care directorate.

Author information

Authors and Affiliations

Authors

Contributions

KMcG, MB, CS and JG were responsible for the original concept and design of the study. KMcG and JMcG carried out all screening and data extraction. KMcG conducted all analysis with assistance from JG. KMcG prepared the draft manuscript. MB, CS and JG provided critical revisions of the document. All authors have approved the final manuscript.

Corresponding author

Correspondence to Kris McGill.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Supplment A: example cochrane group search strategy for MEDLINE B: Note on author contact C: Non-sig effects of trial features D: All non-sig post-hoc results.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McGill, K., Sackley, C.M., Godwin, J. et al. A systematic review of the efficiency of recruitment to stroke rehabilitation randomised controlled trials. Trials 21, 68 (2020). https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-019-3991-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-019-3991-2

Keywords