- Open Access
- Open Peer Review
Trial outcomes and information for clinical decision-making: a comparative study of opinions of health professionals
Trials volume 17, Article number: 344 (2016)
Trials are robust sources of data for clinical practice; however, trial outcomes may not reflect what is important to communicate for decision-making. The study compared clinicians’ views of outcomes to include in a core outcome set for colorectal cancer (CRC) surgery, with what clinicians considered important information for clinical practice (core information).
Potential outcome/information domains were identified through systematic literature reviews, reviews of hospital information leaflets and interviews with patients. These were organized into six categories and used to design a questionnaire survey that asked surgeons and nurses from a sample of CRC centers to rate the importance of each domain as an outcome or as information on a nine-point Likert scale. Respondents were re-surveyed (round 2) following group feedback (Delphi methods). Comparisons were made by calculating the difference in mean scores between the outcomes and information domains, and paired t tests were used to explore the difference between mean scores of the six outcome/information categories.
Data sources identified 1216 outcome/information items for CRC surgery that informed a 94-item questionnaire. First-round questionnaires were returned from 63/81 (78 %) of centers. Clinicians rated 76/94 (84 %) domains of higher importance to measure in trials than information to communicate to patients in round 1. This was reduced to 24/47 (51 %) in round 2. The greatest difference was evident in domains regarding survival, which was rated much more highly as a trial outcome than an important piece of information for decision-making (difference in mean 2.3, 95 % CI 1.9 to 2.8, p <0.0001). Specific complications and quality-of-life domains were rated similarly (difference in mean 0.18, 95 % CI −0.1 to 0.4, p = 0.2 and difference in mean 0.2, 95 % CI −0.1 to 0.5, p = 0.2, respectively).
Whilst clinicians want to measure key outcomes in trials, they rate these as less important to communicate in decision-making with patients. This discrepancy needs to be explored and addressed to maximize the impact of trials on clinical practice.
Clinical trials are a robust source of data to inform clinical decision-making. There are, however, some well described weaknesses that may undermine the interpretation of trial results. Heterogeneous measurement and reporting of outcomes, for example, can make cross-study comparisons difficult, hinder meta-analysis and lead to outcome-reporting bias. Such problems can be minimized through the establishment of a core outcome set (COS): a minimum set of outcomes that key stakeholders agree are to be measured in all trials in a particular field . Many COSs have now been developed, and the benefits have been embraced internationally by research funders , regulatory bodies [2, 3] and journal editors . Theoretically, these outcomes represent the most important aspects of an intervention and, therefore, may represent the most important issues to discuss with patients.
Communicating with patients is, however, challenging. High-quality patient-centered information is a priority for health care providers in many countries and settings [5–7], and patients generally prefer more rather than less information , although it is difficult to know how much is enough for an individual. One solution is to discuss very detailed information, but this may be counterproductive by overwhelming patients and reducing their understanding . Patient-led communication, where discussions are guided by the individual, is helpful but patients may lack sufficient baseline knowledge to ask important questions. One method for addressing the balance between over- and under-disclosure of information is to develop a core information set (CIS) for a specific treatment. Core information represents baseline information, determined by patients and clinicians, that is necessary to stimulate further patient-centered communication [10–12]. Core information sets are now being developed in several clinical areas [11, 13].
It is hypothesized that core outcomes for trials will correspond to core information for communication in decision-making. The rationale for this is that if an outcome is considered essential to measure in a trial, then it provides information of sufficient value to use in clinical practice. Similarly, if information is necessary to communicate to patients then high-quality data supporting that information is best obtained from a well-designed and well-conducted randomized trial. This has not been previously explored. The aim of this study is to compare clinicians’ views of outcomes to include in a COS with what they consider important information for clinical practice, using colorectal cancer (CRC) surgery as an example.
This substudy was embedded within a wider program of research to develop core outcome and information sets for CRC surgery. The scope of the COS was to agree on a minimum standard of outcomes for clinical effectiveness trials (rather than trials of treatment efficacy) of all surgical interventions for cancer of the colon and rectum. The scope of the CIS was to agree on a minimum standard of information of importance to patients and clinicians to discuss for informed consent for CRC surgery. Informed consent for recruitment to trials was not considered. Informed consent is defined by Beauchamp and Childress’ model of “autonomous authorization,” which includes the discussion of what surgeons consider to be “important” information.
The development of the core outcome and information sets was conducted according to established guidelines . Phase 1: a long list of outcomes that could be measured in colorectal cancer trials, and information to communicate for consent, was identified and outcomes were categorized into domains. Phase 2: domains were operationalized into a questionnaire that was used to survey stakeholders’ views on the importance of each domain using Delphi methods. Phase 3: clinicians’ views on important outcomes for trials were compared with clinicians’ and patient’s views on important information for consent. Appropriate ethics regulatory approval was granted (National Research Ethics Service number 10/H0102/82, reviewed by South West 4 Research Ethics Committee).
Phase 1– Domain generation
Colorectal cancer surgical outcomes and information were identified from three sources: (1) systematic review of clinical- and patient-reported outcome literature [14, 15], (2) interviews with patients, and (3) analysis of written patient information leaflets used for colorectal surgery in UK hospitals. Duplicates were removed and a long list of outcomes/information was created. Similar outcomes/information were categorized into domains by the following process. The long list was read and re-read to facilitate immersion in these data. Categorization was conducted on an iterative item-by-item basis using the principles of constant comparison, whereby each new item was compared with those before it and placed in what was considered an appropriate category. Categories were changed, or items re-categorized appropriately, as new items were considered. After independent categorization, results were compared and discrepancies highlighted. Patient-reported outcomes were grouped into domains (e.g., ability to walk and activity levels were grouped within the physical function domain) and verified by two researchers and a patient representative . Items from patient information leaflets were independently categorized by two surgeons. Discrepancies were resolved through discussion with the study lead. Overlapping domains between data sources were condensed producing a final list of domains.
The final domains were operationalized into questionnaire items using lay language with the medical terminology included in parentheses (Fig. 1). Domains were grouped into six categories: CRC-specific complications, general complications, operation and hospital stay, survival/recurrence, symptoms, and quality of life. The questionnaire was piloted by patients for face validity, understanding and acceptability and modified as a result of this feedback.
Phase 2 – Delphi process
The questionnaire developed in phase 1 was sent to CRC surgeons and clinical nurse specialists (round 1). Oncologists were excluded as chemo/radiotherapy was outside the scope of the COS/CIS. Surgeons and nurses were identified from UK National Health Service hospital trusts that routinely performed surgical resection of CRC and participated in the UK National Bowel Cancer Audit. Non-probabilistic purposive sampling was conducted to ensure center variation based upon geographical region (Northern England, the Midlands, South West England, South East England, and Wales), and caseload volume per annum as determined by number of major resections in 2012. Demographic details were collected including age, sex and seniority (years on the UK specialist register).
Questionnaires asked clinicians to rate the importance of (1) measuring the domains in clinical trials, and (2) discussing the domains as part of informed consent. A nine-point Likert scale was used, where 1 was “not essential,” and 9 “absolutely essential” outcome/information. Data from round 1 were collected and entered into an electronic database. Domains that were both rated between 7 and 9 (defined as “high importance”) by over 50 % of respondents and between 1 and 3 (defined as “low importance”) by less than 15 % were retained for round 2. The remainder were discarded. In round 2, participants were provided with feedback from round 1 in the form of their previous score for each domain and a mean score from their stakeholder group. In addition, some participants were also provided with feedback from both stakeholder groups separately (details of this exploratory substudy will be described separately). Participants were then asked to rescore each domain on the same nine-point Likert scale. Separate written informed consent was obtained from all participants.
Phase 3 – Comparison of clinicians’ views of outcomes and information
Clinicians’ scores from rounds 1 and 2 were collated and the mean score calculated for each outcome and information domain. The difference between these means were then calculated and the domains ranked accordingly. Paired t tests were used to compare mean outcome and information scores of the six domain categories and as an overall score.
There are no agreed methods to set the sample size for Delphi surveys. Therefore, an opportunistic approach was used with the aim of obtaining approximately 100 respondents for the survey.
Phase 1 – Domain generation
Review of all data sources identified 1216 outcomes of CRC surgery that were grouped into 100 domains. A total of six were considered only relevant to information, not trials (for example, “the risk of family members developing bowel cancer”) and were excluded from this analysis.
Phase 2 – Delphi process
A total of 81 CRC centers were sampled, of which 63 (78 %) responded including 90 surgeons and 8 clinical nurse specialists (Table 1). The centers represented all geographical regions of England and Wales, and caseload averaged 117 major resections per year (range 38 to 275). A total of 47 domains met the criteria to be retained for round 2. The response rate in round 2 was 80 % (78/98).
Phase 3 – Comparison of clinicians’ views of outcomes and information
In the first survey, clinicians considered most domains (76/94; 81 %) of greater importance to measure as trial outcomes than to communicate to patients (Fig. 2). The greatest difference was with the number of lymph nodes harvested during the operation (difference in mean 4.1, 95 % CI 3.5 to 4.7), unplanned re-admission to hospital (difference in mean 3.1, 95 % CI 2.5 to 3.6) and survival (difference in mean 2.8, 95 % CI 2.2 to 3.4). Of the 18 domains that were considered more important to communicate to patients than to measure in trials, rates of superficial surgical site infection (difference in mean −0.9, 95 % CI −1.1 to −0.5) and the need for a stoma (difference in mean −0.7, 95 % CI −0.9 to −0.4) showed the greatest difference. Paired t tests demonstrated that generic complications, the operation and hospital stay, survival/recurrence and symptom domain categories were all scored significantly higher as outcomes in trials rather than information for clinical practice (Table 4). The overall effect was that clinicians rated outcomes higher than information (difference in mean 0.7, 95 % CI 0.4 to 0.9, p <0.0001).
Results of the second survey were moderated compared to round 1. Only 24/47 (51 %) outcome domains were rated higher than the corresponding information domains. The greatest difference was still evident in the number of lymph nodes harvested during the operation (difference in mean 3.4, 95 % CI 2.8 to 4.0), survival (difference in mean 2.5, 95 % CI 1.9 to 3.0) and unplanned re-admission to hospital (difference in mean 2.4, 95 % CI 1.9 to 3.0). Paired t tests demonstrated that outcomes regarding the operation and hospital stay, and survival/recurrence remained more important than information on the same topics. In this round, clinicians thought CRC-specific complications were more important as information to discuss with patients then outcomes of trials, but the difference was small (difference in mean −0.3, 95 % CI −0.5 to −0.1, p <0001). The overall effect favored outcomes rather than information (Table 4).
This study demonstrated that clinicians consistently rated several health outcome domains as more important to measure in trials than to use as information provision for clinical decision-making with patients. The effect was most pronounced in the first survey round (which does not show views of others) where significant differences were observed regarding outcomes relating to the operation and hospital stay, and survival/recurrence. In the second survey (where views of patients and other health professionals are presented for the participant to re-score the domains) the effect was similar although attenuated. It is possible that health professionals’ really value trial outcomes differently to those of importance for use for decision-making. In which case, it is important to consider whether trial outcomes should be more representative of information that is valued. Findings also suggest that clinicians’ value outcomes that they find difficult to discuss with patients, in which case training in communication and shared decision-making is required. These discrepancies are important to explore further to maximize the impact of trials on clinical practice.
It has not been possible to identify previous studies that have directly compared clinician’s views on outcomes for trials and views on information for clinical practice. A recent study, however, has developed a CIS for oesophageal cancer surgery . Detailed comparisons with this work were not possible because only limited data are presented in the publication; however, clinicians’ views appear to be similar to those found here. Operation-specific complications, such as anastomotic leak, and peri-operative mortality were rated highly, and long-term survival information did not feature in the top 20 issues of importance to clinicians. The reasons for these findings are not clear. Multiple previous studies have shown that cancer patients have detailed and broad information needs, especially regarding issues of prognosis . A survey of 2300 cancer patients in the UK, for example, demonstrated that over 95 % wanted to know survival statistics . Clinicians’ views appear discordant with this evidence. It may be that clinicians consider the information upsetting for patients or difficult to understand. Perhaps clinicians value the data generated from randomized trials to make treatment recommendations, but do not consider it important to discuss with patients.
It has been widely recognized that patients and professionals differ with regards to what outcomes are considered important to measure in clinical trials. Core outcome set development was first pioneered in the field of rheumatology, where the OMERACT (Outcome Measures in Rheumatology) initiative started using consensus methodology to enable professionals to agree on which outcomes to measure in trials. This was initially a closed process, but it was opened up to patients after 6 years. Patients identified three outcome domains that were not considered important by professionals (fatigue, sleep disturbance and arthritic “flairs”) and these were subsequently included in the core set. Although not explicit, it is likely that patients were interested in discussing these trial outcomes in clinical consultations. This is, however, the first study to directly measure the internal discrepancies between what clinicians want to measure in trials and what they want to communicate to patients.
Methods used in this study followed guidelines established by the Core Outcome Measures in Effectiveness Trials (COMET) initiative to develop a COS, but there are some weaknesses. The validity of including specific outcome/information domains in the questionnaires can be criticized; however, this has no impact on the comparative importance that clinicians and patients place on such outcomes/information. Rather, it represents a limitation of the development of the COS. Another limitation is that the findings may represent a failure in methodology and, in particular, the way the questionnaire was presented may have biased responses. For example, participants were asked to rate domains as outcomes before rating them as information for decision-making. This may have induced the prioritization of the former over the latter. Similarly, the order of the domains within the questionnaire may have had an impact on responses. Domains regarding quality of life came at the end of the questionnaire, and thus may have been seen as of lower priority. This, however, should not have influenced the comparative scores of individual domains. Descriptive statistics were used to summarize domain scoring, and multiple hypothesis testing was avoided. Caution should be exercised, however, when comparing individual domain scores because of issues of multiplicity.
Further research is needed to understand the implications of this study. Understanding why clinicians rate these outcome and information domains differently is critical. Qualitative interviews with clinicians and triallists would provide a framework for exploring these issues. It is also important to understand the impact of integrating patients’ views of outcome and information into core set development. It may be that patients’ views mitigate some of the discrepancies evident in this study, but others may become apparent. Research is then needed to identify methods to harmonize the measurement of outcomes in trials and their use in clinical practice.
This study demonstrates that whilst clinicians want to measure key outcomes in trials, they rate these as less important communication in decision-making with patients. This discrepancy needs to be explored and addressed to maximize the impact of trials on clinical practice.
CI, confidence interval; CIS, core information set; COS, core outcome set; CRC, colorectal cancer
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
US Department of Health and Human Services, Food and Drug Administration. Guidance for Industry Clinical Development Programs for Drugs, Devices, and Biological Products for the Treatment of Rheumatoid Arthritis (RA). 1999. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM071579.pdf.
The European Agency for the Evaluation of Medicinal Products. Guideline on clinical investigation of medicinal products other than NSAIDs for treatment of rheumatoid arthritis. London; 2003. http://www.emea.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003439.pdf.
Khan K. The CROWN Initiative: journal editors invite researchers to develop core outcomes in women’s health. Midwifery. 2014;30:1147–8.
Epstein RM, Street RL. Patient-centered communication in cancer care: promoting healing and reducing suffering. In: Patient-centered communication in cancer care: promoting healing and reducing suffering (Editor ed.^eds.), vol. NIH Publication No. 07-6225. City: National Cancer Institute; 2007.
Department of Health. The NHS constitution for England. London; 2009.
Australian Commission on Safety and Quality in Health Care. Patient centred care: improving quality and safety through partnerships with patients and consumers. Sidney: ACSQHC; 2011.
Rutten LJ, Arora NK, Bakos AD, Aziz N, Rowland J. Information needs and sources of information among cancer patients: a systematic review of research (1980–2003). Patient Educ Couns. 2005;57:250–61.
Leydon GM, Boulton M, Moynihan C, Jones A, Mossman J, Boudini M, McPherson K. Cancer patients’ information needs and information seeking behaviour: in depth interview study. BMJ. 2000;320:909–13.
Beauchamp TL, Childress JF. Respect for autonomy. In: Principles of biomedical ethics. 5th ed. Oxford: Oxford University Press; 2001. p. 57–112.
Blazeby JM, Macefield R, Blencowe NS, Jacobs M, McNair AG, Sprangers M, Brookes ST, Research, Consensus Groups of the Core Outcomes and iNformation SiSSOC, et al. Core information set for oesophageal cancer surgery. Br J Surg. 2015;102:936–43.
Main BG, Davies L, McNair A, Blazeby JM. Bringing informed consent back to patients. BMJ. 2015. http://0-blogs.bmj.com.brum.beds.ac.uk/bmj/2014/08/05/barry-main-et-al-bringing-informed-consent-back-to-patients/.
Developing a core information set for consent to oral cancer surgery. http://www.hra.nhs.uk/news/research-summaries/developing-a-core-information-set-for-consent-to-oral-cancer-surgery/. Accessed 22 July 2016.
Whistance RN, Forsythe RO, McNair AG, Brookes ST, Avery KN, Pullyblank AM, Sylvester PA, Jayne DG, Jones JE, Brown J, et al. A systematic review of outcome reporting in colorectal cancer surgery. Colorectal Dis. 2013;15:e548–60.
McNair A, Whistance RN, Forsythe RO, Rees J, Jones JE, Pullyblank AM, Avery K, Brookes ST, Thomas MG, Sylvester PA, et al. Synthesis and summary of patient-reported outcome measures (PROMs) to inform the development of a core outcome set in colorectal cancer surgery. Colorectal Dis. 2015.
Macefield RC, Jacobs M, Korfage IJ, Nicklin J, Whistance RN, Brookes ST, Sprangers MA, Blazeby JM. Developing core outcomes sets: methods for identifying and including patient-reported outcomes (PROs). Trials. 2014;15:49.
Jenkins V, Fallowfield L, Saul J. Information needs of patients with cancer: results from a large study in UK cancer centres. Br J Cancer. 2001;84:48–51.
We would like to thank Claudette Blake for her administrative support throughout the whole project.
This work was supported by the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomized controlled Trials In Invasive procedures – MR/K025643/1). RNW was supported by an NIHR doctoral research fellowship.
Availability of data and materials
AGKM, STB, RNW, KNLA and JMB contributed to the study conception, design, data acquisition, analysis, interpretation and drafting the manuscript. ROF, JR, JEJ, GS, AMP, MGT, PAS, AR, AO, DM, RK, DGJ, RH, SJD, MGC, MC and JB contributed to the data acquisition, analysis, and interpretation. RM contributed to the study design, data acquisition, analysis, interpretation and drafting the manuscript. RHux contributed to the study design, data acquisition, analysis and interpretation. All authors read and approved the final manuscript.
The authors declare that they have no competing interests. This report is independent research arising from a Clinical Fellowship supported by the National Institute for Health Research. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health. The authors have no competing interest to declare.
Ethics approval and consent to participate
This study was approved by the South West 4 Research Ethics Committee (number 10/H0102/82) and all participants provided written informed consent.
About this article
Cite this article
McNair, A.G.K., Brookes, S.T., Whistance, R.N. et al. Trial outcomes and information for clinical decision-making: a comparative study of opinions of health professionals. Trials 17, 344 (2016) doi:10.1186/s13063-016-1492-0
- Core Outcome
- Information Domain
- Clinical Nurse Specialist
- Patient Information Leaflet
- Colorectal Cancer Trial