Skip to main content

Table 3 Summary of item scores by survey round and outcome of consensus meeting

From: Development of a standardised set of metrics for monitoring site performance in multicentre randomised trials: a Delphi study

Metric and domain

Round 1 scores

Round 2 scores

Round 3 scores

% of consensus meeting participants voting to retain item

1–3

4–6

7–9

10

1–3

4–6

7–9

10

1–3

4–6

7–9

10

n (%) respondents per rating category (1-3 ‘not important’, 4-6 ‘important but not critical’, 7-9 ‘critical’, 10 unable to score)

Recruitment and retention

1. Total actual recruitment versus total target recruitment a,c

1 (0.5)

36 (17.1)

172 (81.5)

2 (0.9)

0

16 (7.6)

194 (91.9)

1 (0.5)

0

14 (6.6)

197 (93.4)

0

100

2. Time from the site opening to first participant randomised

8 (3.8)

108 (51.2)

94 (44.5)

1 (0.5)

3 (1.4)

118 (55.9)

90 (42.7)

0

5 (2.4)

126 (59.7)

80 (37.9)

0

 

3. Number of days/weeks since the most recent participant was randomised

21 (10.0)

112 (53.1)

77 (36.5)

1 (0.5)

12 (5.7)

142 (67.3)

57 (27.0)

0

11 (5.2)

155 (73.5)

45 (21.3)

0

 

4. Percentage of potential participants screened who have been randomised

4 (1.9)

88 (41.7)

117 (55.5)

2 (0.9)

2 (0.9)

83 (39.3)

125 (59.2)

1 (0.5)

0

76 (36)

134 (63.5)

1 (0.5)

 

5. Percentage of potential participants who could have been screened, who were screened

13 (6.2)

97 (46.0)

92 (43.6)

9 (4.3)

10 (4.7)

90 (42.7)

103 (48.8)

8 (3.8)

 

6. Percentage of potential participants screened who were eligible

9 (4.3)

106 (50.2)

93 (44.1)

3 (1.4)

6 (2.8)

110 (52.1)

92 (43.6)

3 (1.4)

 

7. Percentage of potential participants eligible who have consented b,c

8 (3.8)

81 (38.4)

119 (56.4)

3 (1.4)

3 (1.4)

77 (36.5)

128 (60.7)

3 (1.4)

95

8. Percentage of potential participants who have consented and have been randomiseda

5 (2.4)

71 (33.6)

131 (62.1)

4 (1.9)

2 (0.9)

57 (27)

150 (71.1)

2 (0.9)

35

9. Percentage of randomised participants who have withdrawn consent to continue in the studya,c

8 (3.8)

76 (36.0)

125 (59.2)

2 (0.9)

4 (1.9)

60 (28.4)

147 (69.7)

0

4 (1.9)

46 (21.8)

161 (76.3)

0

83

10. Percentage of randomised participants lost to follow-upa

10 (4.7)

59 (28.0)

140 (66.4)

2 (0.9)

3 (1.4)

38 (18)

169 (80.1)

1 (0.5)

3 (1.4)

24 (11.4)

183 (86.7)

1 (0.5)

22

11. Percentage of screening logs returned on time out of all those that should have been returned

40 (19.0)

135 (64.0)

33 (15.6)

3 (1.4)

29 (13.7)

159 (75.4)

22 (10.4)

1 (0.5)

23 (10.9)

167 (79.1)

20 (9.5)

1 (0.5)

 

12. Percentage of screening items completed of those required

32 (15.2)

105 (49.8)

67 (31.8)

7 (3.3)

20 (9.5)

114 (54)

72 (34.1)

5 (2.4)

18 (8.5)

117 (55.5)

72 (34.1)

4 (1.9)

 

13. Percentage of randomised participants with a consent form that is incomplete or inaccuratea

11 (5.2)

51 (24.2)

148 (70.1)

1 (0.5)

8 (3.8)

31 (14.7)

172 (81.5)

0

9 (4.3)

14 (6.6)

187 (88.6)

1 (0.5)

13

14. Percentage of all expected forms that have been receiveda

    

8 (3.8)

69 (32.7)

128 (60.7)

6 (2.8)

4 (1.9)

50 (23.7)

154 (73)

3 (1.4)

39

15. Percentage of randomised participants with any issues or problems with consenta

10 (4.7)

68 (32.2)

129 (61.1)

4 (1.9)

6 (2.8)

53 (25.1)

150 (71.1)

2 (0.9)

4 (1.9)

34 (16.1)

169 (80.1)

4 (1.9)

26

16. Percentage of randomised participants for whom documentation of consent is missing from their medical recordsa

15 (7.1)

69 (32.7)

123 (58.3)

4 (1.9)

9 (4.3)

47 (22.3)

154 (73)

1 (0.5)

7 (3.3)

31 (14.7)

172 (81.5)

1 (0.5)

0

Data quality

17. Percentage of randomised participants with the time between data collection and either data entry (electronic case report form) or central receipt of paper case report form within the target timeframe

12 (5.7)

129 (61.1)

66 (31.3)

4 (1.9)

8 (3.8)

156 (73.9)

45 (21.3)

2 (0.9)

7 (3.3)

170 (80.6)

32 (15.2)

2 (0.9)

 

18. Percentage of randomised participants with a query/queries for primary outcome data a,c

4 (1.9)

59 (28.0)

145 (68.7)

3 (1.4)

3 (1.4)

36 (17.1)

170 (80.6)

2 (0.9)

4 (1.9)

23 (10.9)

182 (86.3)

2 (0.9)

65

19. Percentage of randomised participants with query/queries for secondary outcome data

16 (7.6)

128 (60.7)

65 (30.8)

2 (0.9)

8 (3.8)

156 (73.9)

46 (21.8)

1 (0.5)

8 (3.8)

162 (76.8)

40 (19)

1 (0.5)

 

20. Time taken between sending a data query and resolution of the query

17 (8.1)

140 (66.4)

52 (24.6)

2 (0.9)

10 (4.7)

164 (77.7)

36 (17.1)

1 (0.5)

9 (4.3)

167 (79.1)

34 (16.1)

1 (0.5)

 

21. Percentage of randomised participants with complete data for primary and important secondary outcomesa,c

2(0.9)

44 (20.9)

163 (77.3)

2 (0.9)

1 (0.5)

20 (9.5%)

189 (89.6)

1 (0.5)

1 (0.5)

11 (5.2)

198 (93.8)

1 (0.5)

96

22. Percentage of randomised participants with complete data

3 (1.4)

85 (40.3)

120 (56.9)

3 (1.4)

0

88 (41.7)

122 (57.8)

1 (0.5)

0

91 (43.1)

119 (56.4)

1 (0.5)

 

23. Percentage of unresolved serious adverse event queries > 30 calendar days from the date the query was generated a

3 (1.4)

44 (20.9)

163 (77.3)

1 (0.5)

1 (0.5)

24 (11.4)

186 (88.2)

0

1 (0.5)

12 (5.7)

198 (93.8)

0

9

24. Total number of adverse events and serious adverse events reported per number of randomised participants

32 (15.2)

101 (47.9)

74 (35.1)

4 (1.9)

19 (9)

130 (61.6)

59 (28.0)

3 (1.4)

17 (8.1)

138 (65.4)

53 (25.1)

3 (1.4)

 

25. Number of serious adverse events reported per number of randomised participants

25 (11.8)

84 (39.8)

98 (46.4)

4 (1.9)

16 (7.6)

90 (42.7)

102 (48.3)

3 (1.4)

15 (7.1)

102 (48.3)

91 (43.1)

3 (1.4)

 

26. Number of adverse events reported per number of randomised participants b,c

40 (19)

106 (50.2)

60 (28.4)

5 (2.4)

27 (12.8)

136 (64.5)

45 (21.3)

3 (1.4)

24 (11.4)

148 (70.1)

36 (17.1)

3 (1.4)

81

Protocol compliance

27. Percentage of randomised participants with at least one protocol violation a,c

6 (2.8)

78 (37)

124 (58.8)

3 (1.4)

1 (0.5)

64 (30.3)

145 (68.7)

1 (0.5)

0

47 (22.3)

163 (77.3)

1 (0.5)

76

28. Percentage of randomised participants receiving allocated intervention as intended per protocol a,c

2 (0.9)

48 (22.7)

158 (74.9)

3 (1.4)

0

19 (9.0)

191 (90.5)

1 (0.5)

0

11 (5.2)

199 (94.3)

1 (0.5)

100

29. Number of missed visits per number of randomised participants a

7 (3.3)

93 (44.1)

107 (50.7)

4 (1.9)

5 (2.4)

75 (35.5)

128 (60.7)

3 (1.4)

4 (1.9)

52 (24.6)

152 (72)

3 (1.4)

10

30. Number of late visits per number of randomised participants

18 (8.5)

128 (60.7)

61 (28.9)

4 (1.9)

10 (4.7)

157 (74.4)

41 (19.4)

3 (1.4)

9 (4.3)

162 (76.8)

37 (17.5)

3 (1.4)

 

31. Number of critical or major audit findings per number of randomised participants a

6 (2.8)

43 (20.4)

152 (72)

10 (4.7)

4 (1.9)

23 (10.9)

179 (84.8)

5 (2.4)

3 (1.4)

14 (6.6)

190 (90)

4 (1.9)

0

Staff

32. Number of contacts from site staff to the central trial team within a given time period

79 (37.4)

112 (53.1)

14 (6.6)

6 (2.8)

76 (36)

124 (58.8)

6 (2.8)

5 (2.4%)

81 (38.4)

120 (56.9)

6 (2.8)

4 (1.9)

 

33. Time between protocol amendment being sent and principal investigator sign-off

22 (10.4)

110 (52.1)

73 (34.6)

6 (2.8)

16 (7.6)

127 (60.2)

65 (30.8)

3 (1.4)

15 (7.1)

140 (66.4)

53 (25.1)

3 (1.4)

 

34. Cumulative number of staff included on the delegation of duties log

105 (49.8)

87 (41.2)

12 (5.7)

7 (3.3)

116 (55.0)

83 (39.3)

7 (3.3)

5 (2.4)

 
  1. 1–3 not important, 4–6 important but not critical, 7–9 critical and 10 unable to score
  2. Scores shown for the 211 participants who completed all three rounds of the Delphi survey. Cells containing a dash indicate metrics that were added at round 2 and therefore not scored in round 1
  3. aMetrics (n = 15) reaching consensus in status after survey round 3 and taken forward to the consensus meeting
  4. b Metrics 7 and 26 were also discussed and voted on at the meeting
  5. cMetrics receiving > 50% of the vote at the meeting and retained in the final set