Aspects of study design | Assumptions/considerations | Intended | Realized | Lessons learned |
---|---|---|---|---|
Cluster randomized (stepped wedge) controlled trial design | The design promotes hospitals to participate because all participating hospitals will receive the intervention (which was shown effective in previous studies) | Participation of eight units in three (locations of) different hospitals, all starting with a (control) pre-HELP period | Two of the eight units were excluded from participation because of problems with the inclusion of patients during the pre-Hospital Elder Life Program (pre-HELP) period | Emphasize that the study starts before implementation of the intervention and the equal importance of accurate data collection both before and after the intervention is implemented |
The start of the trial was delayed. It took more time than expected to make all necessary arrangements, because not all hospitals were familiar with the regulations for performing a scientific study in the clinic | ||||
Allow for enough time to explain and familiarize participating hospitals with rules and regulations of scientific studies and make timely arrangements accordingly | ||||
Because of the delay, the time frame available to complete our research project was no longer sufficient to stick to the original scheme. Therefore, we decided to start the intervention in period 1 in the first two units to make up for the delay | A stepped wedge, randomized trial takes more time than a regular cluster randomized trial. Be sure the time frame available is comfortably sufficient, even if difficulties occur | |||
Individual recruitment and informed consent of patients is not necessary when using routinely collected data for a quality improvement trial | Analyses of the effects according to the intention-to-treat principle in all eligible patients admitted to the participating units using data from the electronic patient record systems to reduce the risk of bias | The Medical Ethical Review Board required informed consent of every patient included and thereby made individual recruitment inevitable | When using a novel and/or rarely used study design, consider the need to discuss the assumptions underlying the design with the members of the Medical Ethical Review Board before submitting the research proposal | |
Be sure to have a timely decision of the Medical Ethical Review Board to be able to adapt procedures to the need for individual recruitment | ||||
The study design enables a gradual recruitment of volunteers, implementation of the intervention and inclusion of sufficient numbers of patients in each period in each unit | Recruit and educate volunteers in four cycles, to enable two new units to start the intervention every 3 months | The recruitment and education of volunteers went accordingly to plan | ||
In the months before starting the intervention, nurses on the participating units include patients in the pre-HELP condition applying the same criteria to be used during the HELP period. Each unit had to include both pre-HELP and HELP patients to be compared and to avoid empty cells in the stepped wedge matrix | Several units had difficulties including patients in the pre-HELP condition. Two units were excluded from the study after a couple of months in which no patients were included. The ultimate result is the presence of empty wedges in the stepped wedge matrix | Strongly emphasize the importance of including patient during the preintervention period in every participating unit | ||
Inclusion of eligible patients | No need for informed consent | We intended to study the effects of a safe, noninvasive quality improvement intervention | The Medical Ethical Review Board decided informed consent was required. Inclusion of patients in our study severely lagged behind the expected numbers. The requirement of informed consent is likely to have diminished the number of patients willing to participate | Be sure to have a timely decision of the Medical Ethical Review Board to be able to adapt procedures |
The Ethical Committee permitted the use of a delayed informed consent procedure to improve the participation rate in a later stage. | ||||
No need for individual recruitment of patients | By applying the intervention to all patients at risk and using routinely collected data to monitor the effects, results of the intention-to-treat analyses are maximally generalizable | The need for informed consent and, therefore, individual recruitment in our nonblinded study, made the inclusion more prone to selection bias and the results less generalizable. | Reconsider the use of a stepped wedge, cluster randomized trial when there is a need for individual consent | |
The need for individual inclusion of the eligible patients made the analyses of the effects according to the intention-to-treat principle on cluster level to reduce the risk of bias impossible | ||||
Applying the same inclusion criteria in the control condition and in the intervention condition will create a balanced study population without empty cells in the stepped wedge matrix | Similar (numbers of) patients will be included in the wedges in the control period as compared to the intervention period | Comparing the baseline characteristics of patients included in the control versus the intervention period showed that patients included in the HELP condition seem slightly healthier. This may imply selective inclusion favoring positive effects of the HELP | Especially when the recruitment is done by different persons over the units, education on using the same inclusion criteria when considering patients for inclusion in both the control and the intervention condition is crucial | |
Each unit will include both pre-HELP and HELP patients to be compared and no empty cells will occur in the stepped wedge matrix | We failed to include a similar number of participating patients in the control condition versus the intervention condition. This caused the dropout of two units, empty cells in the stepped wedge matrix and an unequal distribution of patients in the control and the intervention groups. These phenomena would partly be concealed in a regular cluster randomized trial | Constant monitoring and motivational activities should be executed to enhance inclusion of patients, especially during the control period | ||
Use of routinely collected data | Helps to avoid selection bias (especially in the absence of the need to individually recruit patients) and enhances external validity because of the optimal reflection of usual situation | Routinely collected data from electronic patient record systems would be consecutively made available over the time frames of the stepped wedge design. | Data from the electronic patient records were not accessible during the study, but only provided after the study period was finished | Check the availability, consistency and quality of routinely collected data well before the study and monitor data quality throughout the study |
According to the study protocol, nurse practitioners (NPs) were to confirm the diagnosis of delirium using the Confusion Assessment Method (CAM) if the results of the routinely collected data on the Delirium Observation Screening Scale (DOSS) were positive | When receiving the data, a lot of data on the outcome appeared to be missing. As a consequence, an important share of the patients included in the study could not be included in the analyses, causing more empty cells in the stepped wedge matrix | |||
Data crucial for the assessment of the effectiveness of the HELP were considered to be present in all patients eligible for the study (who all were at increased risk for delirium) | Assure timely accessibility of the data to be able to do early checks on the quality and completeness of the data during the study period |