Skip to content

Advertisement

  • Study protocol
  • Open Access
  • Open Peer Review

Standardised Outcomes in Nephrology—Polycystic Kidney Disease (SONG-PKD): study protocol for establishing a core outcome set in polycystic kidney disease

  • 1, 2, 3Email authorView ORCID ID profile,
  • 4,
  • 5,
  • 6, 7,
  • 8,
  • 9,
  • 10,
  • 11,
  • 12,
  • 13, 14,
  • 15,
  • 16,
  • 17,
  • 18,
  • 19,
  • 20,
  • 20,
  • 6, 7,
  • 6, 7,
  • 6, 7,
  • 6, 7,
  • 6, 7,
  • 6, 7, 21 and
  • 6, 7
Trials201718:560

https://doi.org/10.1186/s13063-017-2298-4

  • Received: 27 July 2017
  • Accepted: 27 October 2017
  • Published:
Open Peer Review reports

Abstract

Background

Autosomal dominant polycystic kidney disease (ADPKD) is the most common potentially life threatening inherited kidney disease and is responsible for 5–10% of cases of end-stage kidney disease (ESKD). Cystic kidneys may enlarge up to 20 times the weight of a normal kidney due to the growth of renal cysts, and patients with ADPKD have an increased risk of morbidity, premature mortality, and other life-time complications including renal and hepatic cyst and urinary tract infection, intracranial aneurysm, diverticulosis, and kidney pain which impair quality of life. Despite some therapeutic advances and the growing number of clinical trials in ADPKD, the outcomes that are relevant to patients and clinicians, such as symptoms and quality of life, are infrequently and inconsistently reported. This potentially limits the contribution of trials to inform evidence-based decision-making. The Standardised Outcomes in Nephrology—Polycystic Kidney Disease (SONG-PKD) project aims to establish a consensus-based set of core outcomes for trials in PKD (with an initial focus on ADPKD but inclusive of all stages) that patients and health professionals identify as critically important.

Methods

The five phases of SONG-PKD are: a systematic review to identify outcomes that have been reported in existing PKD trials; focus groups with nominal group technique with patients and caregivers to identify, rank, and describe reasons for their choices; qualitative stakeholder interviews with health professionals to elicit individual values and perspectives on outcomes for trials involving patients with PKD; an international three-round Delphi survey with all stakeholder groups (including patients, caregivers, healthcare providers, policy makers, researchers, and industry) to gain consensus on critically important core outcome domains; and a consensus workshop to review and establish a set of core outcome domains and measures for trials in PKD.

Discussion

The SONG-PKD core outcome set is aimed at improving the consistency and completeness of outcome reporting across ADPKD trials, leading to improvements in the reliability and relevance of trial-based evidence to inform decisions about treatment and ultimately improve the care and outcomes for people with ADPKD.

Keywords

  • Core outcome set
  • Outcomes research
  • Patient-centred outcomes clinical trials
  • Chronic kidney disease
  • Autosomal dominant polycystic kidney disease

Background

Autosomal dominant polycystic kidney disease (ADPKD) is the most common inherited kidney disorder and is responsible for 5–10% of end-stage kidney disease (ESKD) requiring renal replacement therapy [13]. ADPKD is life-threatening and irreversible, and affects an estimated 12.5 million people worldwide [4] occurring equally in men and women, without ethnic disparities [5, 6]. Gene defects which lead to the development of ADPKD cause disruption in renal tubular epithelial differentiation leading to the formation of multiple cysts, and their expansion due to ongoing cellular proliferation and fluid secretion [7, 8]. The cysts progressively grow, enlarging kidneys up to 20 times bigger than normal size [9], displacing and destroying normal kidney tissue culminating in fibrosis and ultimately kidney failure [7, 8]. The rate of decline in kidney function is highly variable amongst patients (i.e. annual rate of decline in creatinine clearance 1.71 to 5.8 mL/min/1.73 m2), with greater reported rate of reduction in those who progress to ESKD [10, 11]. Up to 70% of patients with ADPKD progress to ESKD by the age of 65 years [12], and the annual cost of renal replacement therapy (RRT) provision for ADPKD patients in Europe alone approximates to 1.5 billion Euros [13]. Patients with ADPKD are also at high risk of complications including debilitating kidney pain, hepatic cysts, infection, intracranial aneurysms, and cardiovascular disease that may severely impair quality of life and well-being.

The past decade has witnessed some therapeutic advances, with treatments targeting proliferation of cysts [14] and cyst growth [15]. Despite the growing number of trials in ADPKD, the outcomes that are relevant to patients and clinicians, such as symptoms and quality of life, are infrequently and inconsistently reported [16]. Typically, patients are not included in the selection of outcomes and this may cause a mismatch in priorities between patients and clinicians [17]. For example, patients with ADPKD identified the psychosocial impact of diagnosis and the progression to ESKD as important outcomes, but these were not identified by health professionals on the guideline working group [17]. Pain, financial impact, stress, and anxiety are important concerns amongst patients but are inconsistently reported in trials. Although selected larger sized trials have reported quality of life-related outcomes, such as pain [15, 18], most trials have focused their attention on reporting biochemical and imaging parameters such as kidney function (e.g. estimated glomerular filtration rate (eGFR), serum creatinine concentration, doubling of serum creatinine or novel biomarkers) [14, 1922] and kidney structure (e.g. kidney and cyst volumes) [14, 21, 22]. There are new initiatives currently being developed to improve research quality (e.g. PKD Outcomes Consortium, https://c-path.org/programs/pkd/) and tools to better capture ADPKD-specific health-related outcomes (e.g. ADPKD-Impact Scale) [23].

Other problems with outcome reporting include variability in the outcomes measured and reported among trials, and the potential outcome reporting bias. The heterogeneity of outcomes can jeopardise the ability to compare and combine trial results and reliably estimate relative effectiveness [24]. In a systematic review on the prevention of progression of chronic kidney disease (CKD) in patients with ADPKD, changes in kidney function were reported using serum creatinine (12 trials), GFR (13 trials), doubling of creatinine (4 trials), or need for RRT (2 trials) [16]. Only one or two studies were able to be included in each meta-analysis per outcome because of variability in the definitions adopted [16]. Without a core outcomes set routinely reported in all trials, selective reporting of outcomes found to favour the intervention may occur, leading to an overestimation of the true effect of the intervention, or failure to report adverse events which can expose patients to unrecognised risks of harm [2527].

The increasing recognition of inefficiency and waste in medical research, attributed to problems with outcome selection [28], has given rise to numerous initiatives across medical specialties worldwide to develop core outcome sets—defined as an agreed minimum set of standardised outcomes that should be measured and reported in all trials on a specific clinical topic [29, 30]. This does not preclude trialists adding other outcomes specific to the trial population and intervention. The first of these initiatives was the Outcome Measures in Rheumatology (OMERACT), which was formed in 1992 to identify core outcomes in rheumatology through a consensus process involving healthcare providers, policy makers, patients, and their caregivers [3133]. The implementation of OMERACT core outcomes has improved the reporting and relevance of outcomes in rheumatology trials [3335]. More recently, the Core Outcome Measures in Effectiveness Trials (COMET) initiative was formed in 2010 to facilitate the development and implementation of core outcome sets [36, 37].

In nephrology, the Standardized Outcomes in Nephrology (SONG; www.songinitiative.org) initiative was formed in 2015 to establish core outcomes across the full spectrum of CKD. Ongoing work is focussed on haemodialysis (SONG-HD) [3840], kidney transplantation (SONG-Tx) [41], peritoneal dialysis (SONG-PD), and paediatric CKD (SONG-Kids) [42]. The SONG-Polycystic Kidney Disease (SONG-PKD) project aims to establish a core outcome set for trials and other types of clinical research involving patients with ADPKD. The initial focus of SONG-PKD will be on ADPKD as it is the most common form of PKD and presents differently to autosomal recessive polycystic kidney disease (ARPKD). The SONG-PKD Steering Group was convened in June 2017 and is comprised of a multidisciplinary team of health professionals and patients with PKD. The aims of SONG-PKD are to: describe the scope and consistency of outcomes reported in trials in ADPKD; identify and prioritise outcomes that are important to patients with ADPKD and their caregivers (including the reasons for their choices); generate insights on health professionals’ perspectives on outcomes in ADPKD; develop a consensus-based set of core outcomes of critical importance to all stakeholder groups including patients/caregivers and health professionals; and establish a set of core outcome domains to be reported in all trials in patients with ADPKD.

Methods/design

The SONG-PKD project will follow the SONG methodology [43], which has been adapted from the OMERACT and COMET initiatives [31, 36]. The five phases will include: a systematic review, focus groups with nominal group technique, semi-structured interviews with stakeholders, an international Delphi survey, and a consensus workshop (Fig. 1).
Fig. 1
Fig. 1

SONG-PKD study flowchart. PKD polycystic kidney disease

Phase 1: Systematic review of outcome domains reported in PKD trials

We will conduct a systematic review to identify and assess the scope and consistency of outcome domains and outcome measures reported in randomised controlled trials (RCTs) of interventions for adults with ADPKD.

Study search strategy

A comprehensive search of MEDLINE, Embase, the Cochrane Kidney and Transplant Specialized Register, Australian and New Zealand Clinical Trials Registry (ANZCTR; www.anzctr.org.au), EU clinical trials register (www.clinicaltrialsregister.eu) and ClinicalTrials.gov will be conducted to identify all RCTs that enrolled patients with ADPKD aged 18 years or older. We will not apply date or language restrictions.

All RCTs published in peer-reviewed journals and trial protocols registered in ClinicalTrials.gov, ANZCTR, and EU clinical trials register will be included (up to 2 March 2017). Abstracts and conference reports will not be included as they do not provide a complete and reliable source of all the outcomes reported and measured in trials.

Types of interventions

Any intervention used to treat and manage adult patients with ADPKD will be included. These may include pharmacological, surgical, lifestyle, psychosocial, and health service interventions.

Types of participants

Adult patients with ADPKD aged 18 years or older will be eligible. Studies that exclude patients with ADPKD, or only enrol children (aged 18 years or below) with ADPKD will not be included. Given the focus on ADPKD, trials in which patients with ADPKD comprise less than half the study population across all arms will be excluded.

Eligibility of studies

All records retrieved from the electronic searches will be independently assessed by two reviewers (BS and YC). The full texts of all potentially relevant RCTs will be independently assessed by the two reviewers (BS and YC). Any disagreement on the eligibility of studies will be resolved through discussion with a third reviewer (AT).

Data extraction

Characteristics from all included trials will be extracted by one reviewer (BS) and will include the following: first author, publication date, country in which the trial was conducted, participant characteristics (age, sex), trial duration, name and type of intervention (e.g. pharmacological, psychosocial, lifestyle), and primary or secondary outcomes as reported in the trial (including definitions, measurement instruments, thresholds, measurement time points or time frames, changes in level or percentage, scores) [3744]. Outcomes reported in the registration for each trial published will be evaluated in order to assess the selective reporting of outcomes. Two reviewers will cross check the data extraction (BS and YC).

Data analysis and presentation

The reviewer (BS) will group similar outcomes into outcome domains which will be classified as surrogate, clinical, or patient-reported. A surrogate endpoint or outcome is a biochemical or imaging marker used to substitute for a clinical outcome (e.g. total kidney volume) [4547]. A clinical outcome is a medical outcome that is determined or diagnosed by the clinician (e.g. infection) [40, 48, 49]. Patient-reported outcomes are reported directly from patients regarding how they function or feel in relation to a health condition and its therapy, without interpretation by a healthcare professional or anyone else (e.g. pain). The domains will be reviewed and discussed by the SONG-PKD Steering Group. The frequency of reporting across trials for each outcome domain will be ascertained. The primary outcome, if specified, will be identified and analysed. The number of different outcomes (including outcome measures and measurement time points) and the number of trials that assessed each specific outcome will also be assessed. Statistical analyses will be performed using R version 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria).

Phase 2: Nominal group techniques with patients and caregivers

Patients with ADPKD and their caregivers (including family members) will identify and rank outcomes that they consider are important to include in trials and will discuss reasons for their choices. The nominal group technique is highly recommended as a transparent, equitable, and systematic approach [5052] to generate ideas and consensus on priorities in health, including outcomes in CKD [53, 54], and allows each participant to raise their views and suggestions without direct rejection or criticism from others in the group [52].

Participants and recruitment

Patients with ADPKD aged 18 years and older and their caregivers will be eligible to participate. We will aim to convene a minimum of 20 nominal groups (involving 8–12 participants per session, estimated total n = 200). The final number of groups will depend on when data saturation, defined as the point when few or no new outcomes or issues are emerging, is reached [5456].

Participants will be recruited initially from participating centres across Australia (Westmead Hospital, Princess Alexandra Hospital, Monash Medical Centre), and will aim to recruit sites in other countries upon funding securement.

We will use a purposive sampling strategy to achieve maximum diversity in demographics (age, sex, socioeconomic status, ethnicity, location, and educational attainment) and clinical characteristics (stages of chronic kidney disease; i.e. patients not on RRT, CKD stage 1–5; patients on haemodialysis or peritoneal dialysis, CKD stage 5D; and patients with a kidney transplant, CKD stage 5 T; time since diagnosis, comorbidities, and complications). Informed consent will be obtained from all participants. Each participant will be reimbursed in cash (A$50) to help support their transport to attend the session.

Data collection

Each focus/nominal group will be 2 h in duration and take place in a centrally located venue external to the hospital in order to encourage open discussion and to minimise censoring of conversation among participants due to feeling disempowered in a clinical setting. The question guide will be developed based on those previously used to elicit patient-prioritised outcomes in kidney transplantation and haemodialysis [53, 54], and with input from the SONG-PKD Steering Group and investigators. Each group will cover the following:

Welcome and introduction (10 min): The facilitator will explain the aims of the study, define what outcomes are in the context of clinical trials, and ask participants to introduce themselves.

Focus group discussion (40 min): Participants will be asked to discuss their experiences of living with ADPKD, including perceived benefits, harms, and complications related to the disease and treatment.

Nominal group technique (70 min): Each participant will be asked to suggest one or two outcomes they consider are the most important to be reported in trials in ADPKD. The facilitator (YC, BS, AT, TG) will write all suggested outcomes on the flipchart/board and ask the group to provide clarification as required or to discuss their reasons for their suggestion. Once the group has generated the outcomes, the facilitator will add to the flipchart/board outcomes identified from the systematic review (Phase 1) and in previous nominal groups. The list of outcomes will be discussed to ensure that all members understand the meaning of each outcome. A copy of the outcomes will be printed for participants to individually rank all the outcomes in the order of perceived importance, from 1 (most important) to X (least important). The facilitator will ask participants to read out their top three and note these on the flipchart/board. Similarities and differences in ranking will be discussed among the group.

All discussions during the session will be audiotaped and transcribed verbatim, and a note-taker will record the contextual details around the discussion.

Data analysis

Quantitative analysis

A measure of importance (i.e. importance score) of each outcome, based on the rankings attributed in the focus/nominal groups, will be used to prioritise the outcomes. The calculation of this measure is described as follows.

The distribution of the ranking for each outcome is obtained by calculating the probability of each rank for each outcome. Using mathematical notation, this is written as P(O j in rank i), i.e. the probability of the outcome Oj being assigned the rank i. Thus, for each outcome we obtain the probability of being ranked in first place, in second place, and so on. By the total law of probabilities, these probabilities can be decomposed as:
$$ {\displaystyle \begin{array}{l}P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\;i\right)\kern0.5em =\\ {}\kern9.5em =\kern0.5em P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\;i\kern0.5em \left|{O}_j\kern0.5em is\kern0.5em nominated\right.\right)\kern0.5em \times \kern0.5em P\left({O}_j\kern0.5em is\kern0.5em nominated\right)\\ {}\kern9.5em +\kern0.5em P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\;i\kern0.5em \left|{O}_j\kern0.5em not\kern0.5em nominated\right.\right)\kern0.5em \times \kern0.5em P\left({O}_j\kern0.5em not\kern0.5em nominated\right)\end{array}} $$
where “nominated” means that the outcome was considered (and given a rank) by the participant. We will assume that the P(O j in rank i |O j not nominated) is 0. The reasoning for this is that if the participant did not mention the outcome O j , then the probability of any rank is 0. Therefore, the expression above simplifies to
$$ P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\;i\right)\kern0.5em =\kern0.5em P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\;i\kern0.5em \left|{O}_j\kern0.5em is\kern0.5em nominated\right.\right)\kern0.5em \times \kern0.5em P\left({O}_j\kern0.5em is\kern0.5em nominated\right) $$
From this expression, we can observe that the probability has two components: 1) the importance given to the outcome by the ranking; and 2) the consistency of being nominated by the participants. We then use these probabilities and compute the weighted sum of the inverted ranking \( \left(\frac{1}{i}\right) \) to obtain the importance score (IS)
$$ IS\kern0.5em =\kern0.5em \sum \limits_{i=1}^{\begin{array}{l}\kern1.5em nr\kern0.5em of\\ {} outcomes\end{array}}P\left({O}_j\kern0.5em in\kern0.5em \mathit{\operatorname{rank}}\kern0.5em i\right)\kern0.5em \times \kern0.5em \frac{1}{i} $$

The importance score can be interpreted as a summary measure of importance of the outcome that incorporates the consistency of being nominated and the rankings given by the participants. The reason for inverting the ranks is to give more weight to top ranks and less to lower ranks. Higher values of the score identify outcomes that are more valued by the participants. The standard errors for the importance score can be obtained through bootstrapping. This measure has a similar motivation to the Expected Reciprocal Rank Evaluation Metric that was proposed in a different context [57].

The importance scores will also be calculated separately for patients and their caregivers/family, and these results will be compared using a t test with a statistical significance level of p < 0.05. The analysis will be conducted using the software package Stata/SE version 14.0 (StataCorp., College Station, TX) and the R version 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria).

Qualitative analysis

The transcripts will be imported into HyperRESEARCH (ResearchWare Inc., www.researchware.com, version 3.7.2) software to facilitate qualitative data analysis. The transcripts will be reviewed line by line to identify concepts, and similar concepts will be grouped into themes that reflect the reasons for identifying and ranking the outcomes. The preliminary findings will be discussed among the research team to ensure that themes reflect the full breadth and depth of the data.

Phase 3: Stakeholder interviews

Semi-structured interviews will be conducted with health professionals to capture the range and depth of individual values, beliefs, and attitudes towards outcomes in PKD [56, 58]. We will use the Consolidated Criteria for Reporting Qualitative Health Research (COREQ) [59] to report this study.

Participants and recruitment

Health professionals (nephrologists, hepatologists, surgeons (i.e. urologists, transplant surgeons), geneticists, nurses, and allied health professionals (i.e. psychologists, social workers, genetic counsellors, dietitians)) who have expertise, experience, and interest in ADPKD will be eligible to participate in an interview. A minimum of 50 participants will be recruited worldwide through the networks of the Steering Group and investigators. We will apply a purposive sampling strategy to ensure a broad spectrum of perspectives by maximising variability in demographics, professional role, and experience. We will also specifically identify key informants who have experience in research (trials) and roles in policy making. Recruitment will continue until data saturation has been achieved. All participants will provide informed consent prior to the interview [55, 56, 60].

Data collection

Results from the systematic review (Phase 1) and nominal group techniques (Phase 2) will inform the design of the interview guide. We will conduct face-to-face interviews unless video-conferencing or telephone interviews are preferred by the participants, or when an in-person interview cannot be feasibly organised. Participants will be asked to reflect and discuss their perspectives on: 1) their role and experiences in providing care for patients with ADPKD, 2) aspects of treatment or care that are challenging, 3) shared-decision making in the context of ADPKD, 4) outcomes they consider to be critical or relevant to include in PKD trials and their reasons, 5) the results obtained from the nominal group technique study (Phase 2), and 6) views on the development and implementation of core outcomes in PKD trials. Each interview will take approximately 40 min, and will be audio-recorded and transcribed.

Data analysis

We will extract a list of outcomes suggested by the participants. Thematic analysis as described in Phase 2 of the project will be used to identify themes that reflect the attitudes, beliefs, and perspectives of the participants. Four investigators (YC, BS, TG, and AT) will be involved in the data analysis and review preliminary findings to ensure that the coding framework captures the full range and depth of the data collected. The preliminary findings will also be sent to all participants (i.e. member checking) for feedback and comment, and to add relevant perspectives as needed. This can enhance the analytical framework and ensure that the results reflect the perspectives of the participants.

Phase 4: International online Delphi consensus survey

We will conduct an international online Delphi survey to gain consensus on the outcome domains that are critically important to all stakeholder groups for trials in ADPKD. The Delphi survey has been successfully used to reach consensus on core outcome sets for various health conditions or treatments [6164], including in haemodialysis and kidney transplantation [65, 66]. The Delphi technique usually involves three rounds of surveys completed sequentially and anonymously by a panel of experts with experience or expertise on the topic of interest [30, 51]. Respondents contribute their individual perspectives (for example, rating the importance of an outcome) and, in subsequent rounds, participants can view their previous scores and the results of the group, reflect on the group results, and have the option of revising their opinion. This process results in equitable contribution from all participants as it allows individual respondents to express independent thought and minimises direct confrontation. Previous SONG Delphi surveys have demonstrated that participants do not appear to be unduly influenced by the group results (for example, increases in scores occur in both the patient/caregiver and health professional groups), and the scores of both stakeholder groups converge over the three rounds.

Participants and recruitment

The majority of Delphi studies used to develop core outcomes across medical specialties have reported sample sizes ranging from 13 to 222 participants. Over 1000 participants from more than 70 countries have participated on the Delphi panel for the studies conducted as part of the SONG-HD and SONG-Tx project [65, 66]. For the SONG-PKD Delphi panel, we will aim to recruit a minimum sample size of 1000 respondents where patients/caregivers comprise at least 50% of the sample size. Specifically, we will aim to recruit patients/caregivers (n = 500), clinicians (nephrologists/surgeons (i.e. urologists, transplant surgeons)/geneticists (n = 400)), nurses and allied health professionals (psychologists, genetic counsellors, and dietitians (n = 60)), researchers (n = 20), industry (n = 20), and policy makers (n = 20).

We will use multiple sampling strategies to be as broadly inclusive as possible. Participants will be identified by using a purposive strategy similar to those detailed in Phase 3 to ensure broad representation of views and snowballing whereby participants can nominate other eligible individuals to participate. Participants will be recruited globally through participating university institutions/hospitals of the SONG-PKD Steering Group and investigators, the SONG Initiative database, and patient/consumer organisations. All participants will be required to register their name and email address via www.songinititative.org to receive a standard study information sheet. Informed consent will be obtained from all participants.

Data collection

The Delphi survey will include approximately 30 outcome domains identified from Phases 1–3 of SONG-PKD. Each outcome will include a definition written in plain language. The draft Delphi survey will be reviewed by the SONG-PKD Steering Group and pilot tested with at least 10 patients/caregivers. The survey will be custom programmed and administered online using Qualtrics. All participants will be assigned a unique identifier based on their name and email address to link their responses across all three rounds of the Delphi survey. At least two reminders will be sent to participants per round. We aim to retain at least a 70% response rate across all rounds.

Round 1

Participants will rate the importance of each outcome domain using the GRADE nine-point Likert scale [67]. Ratings of 1–3 indicate outcomes of limited importance, ratings of 4–6 indicate outcomes of importance but not critical, and ratings of 7–9 indicate outcomes of critical importance. An option “unable to score” will also be available. In order to minimise the risk of ordering bias, the outcomes domains will be randomised. For each outcome, a text box is provided for participants to provide comments about their choices. After the rating scales are completed, participants can suggest new outcomes. All outcome domains that are suggested by more than 10% of the participants, and do not duplicate outcomes in the original survey, will be included in Round 2.

We will review the distribution of scores across all outcomes for each stakeholder group, i.e. patients/caregivers and health professionals. Any outcomes with a median and mean of more than 7 will be retained for Round 2. Any new outcomes suggested by more than 10% of the participants will also be included in the next round. This is based on criteria established for previous SONG Delphi surveys [38, 66]. Any outcomes excluded in subsequent rounds will be listed as “outer tier” outcomes, i.e. important to some or all stakeholder groups to consider for trials; or “middle tier” outcomes, i.e. critically important to some stakeholder groups to report in some trials (Fig. 2).
Fig. 2
Fig. 2

SONG conceptual schema of core outcomes

Round 2

The participants will be shown their previous score for each outcome (highlighted in the rating scale), and will be presented with a column graph of the distribution of scores for all participants (equally weighted by stakeholder group), patients/caregivers, and health professionals. A plain language explanation and an example will be provided to help participants understand how to interpret the graph. Participants will also see comments from patients/caregivers and health professionals in separate scroll down boxes. Participants will be asked to re-rate the outcomes on the same GRADE 9-point Likert scale. A text box will be provided for optional comments such as reasons for their ratings.

An outcome with a median and mean of more than 7, and with 70% or more participants in both stakeholder groups (i.e. patient/family member and health professionals) rating the outcome to be of critical importance (7–9), will be included in Round 3.

Round 3

In this final round of survey, participants will see their previous score highlighted and again review the distribution of scores for all participants, patients/caregivers, and health professionals, and comments from Round 2. They will be asked to re-rate the importance of each outcome, and have the opportunity to provide additional comments in a free-text box. To assess the relative importance of the outcomes, we will include a Best–Worst Scale survey [68]. Participants will be presented with approximately six blocks of six outcome domains in which they select the most important and the least important outcome from the list.

Data analysis

For all three rounds, we will show the distribution of scores and calculate the mean, median, and proportion of participants rating the outcome of critical importance. For the Best–Worst Scale Survey, multinomial logistic regression models will be used to calculate the relative importance score for each outcome domain normalised to the range of 1 (least important) to 9 (most important). This will be calculated separately for patients/caregivers and health professionals.

For feasibility, three to five outcome domains are included in the core set and all other outcomes are classified as middle and outer tier outcomes (Fig. 2). It is possible that more than five outcomes may be identified based on the SONG pre-specified definition of consensus. For outcomes to be included in the core set, the outcome must have: a median score of greater than or equal to 8; a mean score greater than or equal to 7.5; the proportion of participants rating the outcome as ‘critically important’ being greater than or equal to 75%, and a median score of less than 10 in the forced ranking question [65, 66]. As we are unable to determine the distribution of scores, the thresholds for inclusion of an outcome domain as a core outcome may need to be determined post-hoc and discussed at the consensus workshop (Phase 5). Several subgroup analyses will be conducted according to country, gender, CKD stage, and primary role (e.g. patient/caregiver, health professionals).

Phase 5: Consensus workshop

A face-to-face stakeholder consensus workshop will be convened to obtain feedback and discuss the proposed core outcomes for trials in PKD as identified from Phases 1–4. The meeting will be chaired and facilitated by members of the SONG-PKD Steering Group, and held in conjunction with an international nephrology conference to maximise participation. Based on previous consensus workshops, and to ensure feasibility and group manageability [40, 69], we will invite approximately 60 participants, including at least 20 patients with ADPKD and their caregivers. Health professionals (nephrologists, surgeons, geneticists, nursing, allied health professionals, researchers, policy makers, and industry) with a range of clinical and research experience in ADPKD, including in clinical trials, or who have leadership or advisory roles in major research, funding, and regulatory agencies will be invited to attend. Patients and caregivers attending the workshop will be reimbursed for transportation and parking.

A copy of results from Phases 1–4 will be provided to participants approximately 2 weeks before the workshop so that participants can have the opportunity to ask questions or seek clarification by email, and understand the SONG-PKD process and results to inform the discussion. The workshop will comprise of three sessions.

Session 1: Introduction

We will present an introduction to the SONG-PKD initiative, including the process and results from the Phases 1–4, and the proposed set of core outcome domains with the threshold and rationale for inclusion.

Session 2: Breakout group discussion

Participants will be allocated to six breakout groups with up to 12 participants in each group (including a facilitator and co-facilitator). Each group will include at least three patients/family members/caregivers to encourage exchange of different perspectives and opinions, and breadth of discussion. All facilitators will attend a briefing session and be provided with a question guide prior to the session commencement. Participants will be asked to reflect on and discuss their opinions on the proposed core outcome domains.

Session 3: Plenary discussion

The Chair of the workshop will moderate a plenary discussion. A nominated spokesperson from each breakout group will present a summary of the discussion to the wider group who will have the opportunity to respond and discuss the issues raised by other groups. The Chair will summarise the key points raised across all breakout groups.

All breakout and plenary discussion will be audiotaped and transcribed verbatim. An investigator (YC) will summarise the discussion with input from other investigators (BS, AT, and TG) to ensure that the summary is comprehensive and captures all the points raised in the discussion. A draft plain language report will be circulated to the workshop attendees and collaborators for feedback within a 2-week timeframe. The readability of the plain language report will be at the sixth-grade level, which will be of lower literacy level than the eight-grade level as recommended by the Centers for Disease Control and Prevention (CDC) [70]. Additional comments will be integrated into the final report.

Establishing the SONG-PKD core outcome domains

The SONG-PKD proposed core outcomes will be uploaded on the website for 3 weeks for public comment. The link will also be sent to the SONG Initiative database and collaborating organisations. All feedback received will be reviewed by the SONG-PKD Steering Group to finalise the SONG-PKD set of core outcome domains.

Ethics

This study has been approved by the Human Research Ethics Committee of The University of Sydney (2015-228), Westmead Hospital (2009/6/4.14), Monash Medical Centre (2010.0.31), and Princess Alexandra Hospital (HREC/17/QPAH/112).

Discussion

The SONG-PKD project engages patients/caregivers and health professionals in a systematic, transparent, and equitable consensus process to establish a set of critically important core outcome domains to be reported in all trials involving patients with ADPKD. We will establish validated, feasible, and robust core outcome measures for each core outcome domain identified through SONG-PKD.

Reporting outcomes that are critically important to all stakeholder groups, particularly patients, will improve the quality and relevance of research evidence to inform treatment decisions. We will actively disseminate the core outcome domains through publications, online media, and collaborating patient, research, professional, and policy organisations, and develop targeted strategies to ensure implementation of the core outcome domains in trials. Standardised reporting of patient-important outcomes in clinical trials is expected to translate to improved patient care and outcomes in ADPKD.

Study status

Recruitment and data collection have commenced.

Abbreviations

ADPKD: 

Autosomal dominant polycystic kidney disease

ANZCTR: 

Australian and New Zealand Clinical Trials Registry

CKD: 

Chronic kidney disease

COMET: 

Core Outcome Measures in Effectiveness Trials

COREQ: 

Consolidated Criteria for Reporting Qualitative Health Research

eGFR: 

Estimated glomerular filtration rate

ESKD: 

End-stage kidney disease

OMERACT: 

Outcome Measures in Rheumatology

PKD: 

Polycystic kidney disease

RCT: 

Randomised controlled trial

RRT: 

Renal replacement therapy

SONG: 

Standardised Outcomes in Nephrology

SONG-PKD: 

Standardised Outcomes in Nephrology—Polycystic Kidney Disease

Declarations

Acknowledgements

Not applicable.

Funding

YC is supported by the National Health and Medical Research Council (NHMRC) Early Career Fellowship (1126256). AT is supported by the NHMRC Career Development Fellowship (1106716). ATP is partially supported by the NHMRC Program Grant BEAT CKD (APP1092957). This project is supported by the NHMRC Program Grant 1092597 and the Polycystic Kidney Disease (PKD) Foundation of Australia. The funding bodies do not have a role in the design, collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.

Availability of data and materials

Not applicable.

Authors’ contributions

YC, BS, GR, JCC, and AT conceived the study, contributed to the design of the study, collected and analysed the data, and drafted the manuscript. TH conceived the study, contributed to the design of the study, and participated in data collection and analysis. AO, AC, CA, DC, HC, JTK, RG, RP, VT, YP, PGK, JR, TG, MH, AJ, KEM, ATP, and LAH contributed to the design of the study, and participated in data collection and analysis. GR, AO, AC, CA, DC, HC, JTK, RG, RP, TH, and VT are members of the SONG-PKD Steering Group. All authors provided important intellectual input, and revised and approved the final manuscript.

Authors’ information

Not applicable.

Ethics approval and consent to participate

The University of Sydney (2015/228) provided ethical review and approval. Ethics have also been approved by Westmead Hospital (2009/6/4.14), Monash Medical Centre (2010.0.31), and Princess Alexandra Hospital (HREC/17/QPAH/112). We will obtain informed consent from all participants in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Nephrology, Princess Alexandra Hospital, 199 Ipswich Road, Woolloongabba, Brisbane, QLD, 4102, Australia
(2)
Australasian Kidney Trials Network, University of Queensland, Brisbane, Australia
(3)
Translational Research Institute, Brisbane, Australia
(4)
Department of Nephrology and Clinical Immunology, Tours Hospital, University Francois Rabelais, INSERMU1246, Tours, France
(5)
Centre for Transplant and Renal Research, Westmead Institute for Medical Research, The University of Sydney, Sydney, Australia
(6)
Sydney School of Public Health, The University of Sydney, Sydney, Australia
(7)
Centre for Kidney Research, The Children’s Hospital at Westmead, Sydney, Australia
(8)
Academic Nephrology Unit, Department of Infection Immunity & Cardiovascular Disease, University of Sheffield, Sheffield, UK
(9)
Department of Medicine, The University of Chicago, Chicago, USA
(10)
Division of Nephrology, Seoul National University Hospital, Seoul, South Korea
(11)
Kidney Institute, Department of Nephrology, Shanghai Changzheng Hospital, Second Military Medical University, Shanghai, China
(12)
Polycystic Kidney Disease Foundation of Australia, Sydney, Australia
(13)
School of Medicine, Fu Jen Catholic University, Taipei, Taiwan
(14)
Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
(15)
Faculty of Medical Sciences, University Medical Center Gronigen, Gronigen, The Netherlands
(16)
Division of Nephrology, Tufts University School of Medicine, Boston, USA
(17)
Polycystic Kidney Disease International, London, UK
(18)
Department of Nephrology and Hypertension, Mayo Clinic, Rochester, USA
(19)
Division of Nephrology and Division of Genomic Medicine, University of Toronto, Toronto, Canada
(20)
Department of Nephrology, Monash Medical Centre and Monash University, Melbourne, Australia
(21)
Division of Nephrology, Albert Children’s Hospital, University of Calgary, Calgary, Canada

References

  1. Levy M, Feingold J. Estimating prevalence in single-gene kidney diseases progressing to renal failure. Kidney Int. 2000;58(3):925–43.View ArticlePubMedGoogle Scholar
  2. US Renal Data System. USRDS annual data report: atlas of chronic kidney disease and end-stage renal diseases in the United States. Bethesda: National Institute of Diabetes and Digestive and Kidney Diseases; 2011.Google Scholar
  3. Australia and New Zealand Dialysis and Transplant Registry. 39th annual report: Chapter 1. Incidence of end stage kidney disease. Adelaide, Australia: ANZDATA; 2016.Google Scholar
  4. Iglesias CG, Torres VE, Offord KP, Holley KE, Beard CM, Kurland LT. Epidemiology of adult polycystic kidney disease, Olmsted County, Minnesota: 1935–1980. Am J Kidney Dis. 1983;2(6):630–9.View ArticlePubMedGoogle Scholar
  5. What is ADPKD? www.pkdinternational.org/what-is-pkd/adpkd.
  6. Torres VE, Harris PC. Autosomal dominant polycystic kidney disease: the last 3 years. Kidney Int. 2009;76(2):149–68.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Belibi FA, Reif G, Wallace DP, Yamaguchi T, Olsen L, Li H, Helmkamp Jr GM, Grantham JJ. Cyclic AMP promotes growth and secretion in human polycystic kidney epithelial cells. Kidney Int. 2004;66(3):964–73.View ArticlePubMedGoogle Scholar
  8. Torres VE, Wang X, Qian Q, Somlo S, Harris PC. Gattone 2nd VH. Effective treatment of an orthologous model of autosomal dominant polycystic kidney disease. Nat Med. 2004;10(4):363–4.View ArticlePubMedGoogle Scholar
  9. Borreli L. Indian man with cysts has world’s largest kidneys removed, each the size of a newborn baby. http://www.medicaldaily.com/indian-man-cysts-has-worlds-largest-kidneys-removed-each-size-newborn-baby-330448. Accessed 23 Apr 2015.
  10. Fick-Brosnahan GM, Belz MM, McFann KK, Johnson AM, Schrier RW. Relationship between renal volume growth and renal function in autosomal dominant polycystic kidney disease: a longitudinal study. Am J Kidney Dis. 2002;39(6):1127–34.View ArticlePubMedGoogle Scholar
  11. Thong KM, Ong AC. The natural history of autosomal dominant polycystic kidney disease: 30-year experience from a single centre. QJM. 2013;106(7):639–46.View ArticlePubMedGoogle Scholar
  12. Lentine KL, Xiao H, Machnicki G, Gheorghian A, Schnitzler MA. Renal function and healthcare costs in patients with polycystic kidney disease. Clin J Am Soc Nephrol. 2010;5(8):1471–9.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Spithoven EM, Kramer A, Meijer E, Orskov B, Wanner C, Abad JM, Areste N, de la Torre RA, Caskey F, Couchoud C, et al. Renal replacement therapy for autosomal dominant polycystic kidney disease (ADPKD) in Europe: prevalence and survival—an analysis of data from the ERA-EDTA Registry. Nephrol Dial Transplant. 2014;29 Suppl 4:iv15–25.View ArticlePubMedGoogle Scholar
  14. Serra AL, Poster D, Kistler AD, Krauer F, Raina S, Young J, Rentsch KM, Spanaus KS, Senn O, Kristanto P, et al. Sirolimus and kidney growth in autosomal dominant polycystic kidney disease. N Engl J Med. 2010;363(9):820–9.View ArticlePubMedGoogle Scholar
  15. Torres VE, Chapman AB, Devuyst O, Gansevoort RT, Grantham JJ, Higashihara E, Perrone RD, Krasa HB, Ouyang J, Czerwiec FS, et al. Tolvaptan in patients with autosomal dominant polycystic kidney disease. N Engl J Med. 2012;367(25):2407–18.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Bolignano D, Palmer SC, Ruospo M, Zoccali C, Craig JC, Strippoli GF. Interventions for preventing the progression of autosomal dominant polycystic kidney disease. Cochrane Database Syst Rev. 2015;7:CD010294.Google Scholar
  17. Tong A, Tunnicliffe DJ, Lopez-Vargas P, Mallett A, Patel C, Savige J, Campbell K, Patel M, Tchan MC, Alexander SI, et al. Identifying and integrating consumer perspectives in clinical practice guidelines on autosomal-dominant polycystic kidney disease. Nephrology (Carlton). 2016;21(2):122–32.View ArticleGoogle Scholar
  18. Miskulin DC, Abebe KZ, Chapman AB, Perrone RD, Steinman TI, Torres VE, Bae KT, Braun W, Winklhofer FT, Hogan MC, et al. Health-related quality of life in patients with autosomal dominant polycystic kidney disease and CKD stages 1–4: a cross-sectional study. Am J Kidney Dis. 2014;63(2):214–26.View ArticlePubMedGoogle Scholar
  19. Caroli A, Perico N, Perna A, Antiga L, Brambilla P, Pisani A, Visciano B, Imbriaco M, Messa P, Cerutti R, et al. Effect of long-acting somatostatin analogue on kidney and cyst growth in autosomal dominant polycystic kidney disease (ALADIN): a randomised, placebo-controlled, multicentre trial. Lancet. 2013;382(9903):1485–95.View ArticlePubMedGoogle Scholar
  20. Fassett RG, Coombes JS, Packham D, Fairley KF, Kincaid-Smith P. Effect of pravastatin on kidney function and urinary protein excretion in autosomal dominant polycystic kidney disease. Scand J Urol Nephrol. 2010;44(1):56–61.View ArticlePubMedGoogle Scholar
  21. Higashihara E, Nutahara K, Horie S, Muto S, Hosoya T, Hanaoka K, Tuchiya K, Kamura K, Takaichi K, Ubara Y, et al. The effect of eicosapentaenoic acid on renal function and volume in patients with ADPKD. Nephrol Dial Transplant. 2008;23(9):2847–52.View ArticlePubMedGoogle Scholar
  22. Ruggenenti P, Remuzzi A, Ondei P, Fasolini G, Antiga L, Ene-Iordache B, Remuzzi G, Epstein FH. Safety and efficacy of long-acting somatostatin treatment in autosomal-dominant polycystic kidney disease. Kidney Int. 2005;68(1):206–16.View ArticlePubMedGoogle Scholar
  23. Autosomal Dominant Polycystic Kidney Disease Impact Scale (ADPKD-IS). https://eprovide.mapi-trust.org/instruments/autosomal-dominant-polycystic-kidney-disease-impact-scale.
  24. Williamson PR, Altman D, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting BG. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS One. 2013;8(7):e66844.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202.View ArticlePubMedGoogle Scholar
  27. Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.View ArticlePubMedGoogle Scholar
  29. Kirkham JJ, Gorst S, Altman DG, Blazeby J, Clarke M, Devane D, Gargon E, Williamson PR, Group C-S. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials. 2015;16:373.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Boers M, Kirwan JR, Tugwell P, Beaton D, Bingham CO, Conaghan PG. The OMERACT handbook. Ottawa: OMERACT; 2014. https://www.omeract.org/pdf/OMERACT_Handbook.pdf.
  32. Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d'Agostino MA, Conaghan PG. Bingham 3rd CO, Brooks P, Landewe R, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67(7):745–53.View ArticlePubMedGoogle Scholar
  33. Kirkham JJ, Boers M, Tugwell P, Clarke M, Williamson PR. Outcome measures in rheumatoid arthritis randomised trials over the last 50 years. Trials. 2013;14:324.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Kirkham JJ, Gargon E, Clarke M, Williamson PR. Can a core outcome set improve the quality of systematic reviews? A survey of the Co-ordinating Editors of Cochrane Review Groups. Trials. 2013;14:21.View ArticlePubMedPubMed CentralGoogle Scholar
  35. Kirkham JJ, Clarke M, Williamson PR. A methodological approach for assessing the uptake of core outcome sets using ClinicalTrials.gov: findings from a review of randomised controlled trials of rheumatoid arthritis. BMJ. 2017;357:j2262.View ArticlePubMedPubMed CentralGoogle Scholar
  36. Prinsen CA, Vohra S, Rose MR, King-Jones S, Ishaque S, Bhaloo Z, Adams D, Terwee CB. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials. 2014;15:247.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39.View ArticlePubMedPubMed CentralGoogle Scholar
  38. Evangelidis N, Craig JC, Tong A, Committee SE. Investigators. Standardised outcomes in nephrology—haemodialysis (SONG-HD): using the Delphi method to gain consensus on core outcomes for haemodialysis trials. J Ren Care. 2015;41(4):211–2.View ArticlePubMedGoogle Scholar
  39. Tong A, Manns B, Hemmelgarn B, Wheeler DC, Tugwell P, Winkelmayer WC, van Biesen W, Crowe S, Kerr PG, Polkinghorne KR, et al. Standardised outcomes in nephrology—haemodialysis (SONG-HD): study protocol for establishing a core outcome set in haemodialysis. Trials. 2015;16:364.View ArticlePubMedPubMed CentralGoogle Scholar
  40. Tong A, Manns B, Hemmelgarn B, Wheeler DC, Evangelidis N, Tugwell P, Crowe S, Van Biesen W, Winkelmayer WC, O'Donoghue D, et al. Establishing core outcome domains in hemodialysis: report of the standardized outcomes in nephrology—hemodialysis (SONG-HD) consensus workshop. Am J Kidney Dis. 2017;69(1):97–107.View ArticlePubMedGoogle Scholar
  41. Tong A, Budde K, Gill J, Josephson MA, Marson L, Pruett TL, Reese PP, Rosenbloom D, Rostaing L, Warrens AN, et al. Standardized outcomes in nephrology—transplantation: a global initiative to develop a core outcome set for trials in kidney transplantation. Transplant Direct. 2016;2(6):e79.View ArticlePubMedPubMed CentralGoogle Scholar
  42. Tong A, Samuel S, Zappitelli M, Dart A, Furth S, Eddy A, Groothoff J, Webb NJ, Yap HK, Bockenhauer D, et al. Standardised Outcomes in Nephrology—Children and Adolescents (SONG-Kids): a protocol for establishing a core outcome set for children with chronic kidney disease. Trials. 2016;17:401.View ArticlePubMedPubMed CentralGoogle Scholar
  43. Initiative. S. The SONG handbook, vol. 1.0. Sydney: SONG Initiative; 2017.Google Scholar
  44. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Eng J Med. 2011;364(9):852–60.View ArticleGoogle Scholar
  45. Samuels J. Use of surrogate outcomes in nephrology research. Adv Chronic Kidney Dis. 2016;23(6):363–6.View ArticlePubMedGoogle Scholar
  46. Jun M, Turin TC, Woodward M, Perkovic V, Lambers Heerspink HJ, Manns BJ, Tonelli M, Hemmelgarn BR. Assessing the validity of surrogate outcomes for ESRD: a meta-analysis. J Am Soc Nephrol. 2015;26(9):2289–302.View ArticlePubMedPubMed CentralGoogle Scholar
  47. Aronson JK. Biomarkers and surrogate endpoints. Br J Clin Pharmacol. 2005;59(5):491–4.View ArticlePubMedPubMed CentralGoogle Scholar
  48. Velentgas P, Dreyer NA, Wu AW. Chapter 6: Outcome definition and measurements. In: Velentgas P, Dreyer NA, Nourjah P, editors. Developing a protocol for observational comparative effectiveness research: a user’s guide. Rockville: Agency for Healthcare Research and Quality (US); 2013.Google Scholar
  49. NIH Definitions Working Group. Biomarkers and surrogate endpoints in clinical research: definitions and conceptual model. In: Downing GJ, editor. Biomarkers and surrogate endpoints. Amsterdam: Elsevier; 2000. p. 1–9.Google Scholar
  50. Corner J, Wright D, Hopkinson J, Gunaratnam Y, McDonald JW, Foster C. The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer. 2007;96(6):875–81.View ArticlePubMedPubMed CentralGoogle Scholar
  51. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–80.View ArticlePubMedPubMed CentralGoogle Scholar
  52. Black N. Consensus development methods. Qualitative research in health care. Malden: Blackwell; 2006.Google Scholar
  53. Howell M, Tong A, Wong G, Craig JC, Howard K. Important outcomes for kidney transplant recipients: a nominal group and qualitative study. Am J Kidney Dis. 2012;60(2):186–96.View ArticlePubMedGoogle Scholar
  54. Urquhart-Secord R, Craig JC, Hemmelgarn B, Tam-Tham H, Manns B, Howell M, Polkinghorne KR, Kerr PG, Harris DC, Thompson S, et al. Patient and caregiver priorities for outcomes in hemodialysis: an international nominal group technique study. Am J Kidney Dis. 2016;68(3):444–54.View ArticlePubMedGoogle Scholar
  55. Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ. 2008;337:a1035.View ArticlePubMedGoogle Scholar
  56. Tong A, Winkelmayer WC, Craig JC. Qualitative research in CKD: an overview of methods and applications. Am J Kidney Dis. 2014;64(3):338–46.View ArticlePubMedGoogle Scholar
  57. Chapelle O, Metlzer D, Zhang Y, Grinspan P. Expected reciprocal rank for graded relevance. In: 18th ACM conference on information and knowledge management 2009. New York: ACM; 2009:621–630.Google Scholar
  58. Kuper A, Reeves S, Levinson W. An introduction to reading and appraising qualitative research. BMJ. 2008;337:a288.View ArticlePubMedGoogle Scholar
  59. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.View ArticlePubMedGoogle Scholar
  60. Tong A, Winkelmayer WC, Wheeler DC, van Biesen W, Tugwell P, Manns B, Hemmelgarn B, Harris T, Crowe S, Ju A, et al. Nephrologists’ perspectives on defining and applying patient-centered outcomes in hemodialysis. Clin J Am Soc Nephrol. 2017;12(3):454–66.View ArticlePubMedGoogle Scholar
  61. Blackwood B, Ringrow S, Clarke M, Marshall J, Rose L, Williamson P, McAuley D. Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study. Trials. 2015;16:368.View ArticlePubMedPubMed CentralGoogle Scholar
  62. MacLennan S, Bekema HJ, Williamson PR, Campbell MK, Stewart F, MacLennan SJ, N'Dow JM, Lam TB. A core outcome set for localised prostate cancer effectiveness trials: protocol for a systematic review of the literature and stakeholder involvement through interviews and a Delphi survey. Trials. 2015;16:76.View ArticlePubMedPubMed CentralGoogle Scholar
  63. Chiarotto A, Deyo RA, Terwee CB, Boers M, Buchbinder R, Corbin TP, Costa LO, Foster NE, Grotle M, Koes BW, et al. Erratum to: Core outcome domains for clinical trials in non-specific low back pain. Eur Spine J. 2015;24(9):2097.View ArticlePubMedGoogle Scholar
  64. Chiarotto A, Deyo RA, Terwee CB, Boers M, Buchbinder R, Corbin TP, Costa LO, Foster NE, Grotle M, Koes BW, et al. Core outcome domains for clinical trials in non-specific low back pain. Eur Spine J. 2015;24(6):1127–42.View ArticlePubMedGoogle Scholar
  65. Evangelidis N, Tong A, Manns B, Hemmelgarn B, Wheeler DC, Tugwell P, Crowe S, Harris T, Van Biesen W, Winkelmayer WC, et al. Developing a set of core outcomes for trials in hemodialysis: an international Delphi survey. Am J Kidney Dis. 2017;70(4):464–75.View ArticlePubMedGoogle Scholar
  66. Sautenet B, Tong A, Manera KE, Chapman JR, Warrens AN, Rosenbloom D, Wong G, Gill J, Budde K, Rostaing L, et al. Developing consensus-based priority outcome domains for trials in kidney transplantation: a multinational Delphi survey with patients, caregivers and health professionals. Transplantation. 2017;101:0875–1886.View ArticleGoogle Scholar
  67. Schunemann H, Brozek J, Oxman A, editors. GRADE handbook for grading quality of evidence and strength of recommendations. Version 3.2 [updated March 2009]. The GRADE Working Group, 2009. Available from http://www.cc-ims-net/gradepro.
  68. Cheung KL, Wijnen BFM, Hollin IL, Janssen EM, Bridges JF, Evers SMAA, Hiligsmann M. Using best–worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34(12):1195–209.Google Scholar
  69. Tong A, Gill J, Budde K, Marson L, Reese PP, Rosenbloom D, Rostaing L, Wong G, Josephson MA, Pruett TL, et al. Toward establishing core outcome domains for trials in kidney transplantation: report of the standardized outcomes in nephrology-kidney transplantation consensus workshops. Transplantation. 2017;101(8):1887–96.View ArticlePubMedGoogle Scholar
  70. Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010;468(10):2572–80.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2017

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement