- Study protocol
- Open Access
Does rating the operation videos with a checklist score improve the effect of E-learning for bariatric surgical training? Study protocol for a randomized controlled trial
Trials volume 18, Article number: 134 (2017)
Laparoscopic training has become an important part of surgical education. Laparoscopic Roux-en-Y gastric bypass (RYGB) is the most common bariatric procedure performed. Surgeons must be well trained prior to operating on a patient. Multimodality training is vital for bariatric surgery. E-learning with videos is a standard approach for training. The present study investigates whether scoring the operation videos with performance checklists improves learning effects and transfer to a simulated operation.
This is a monocentric, two-arm, randomized controlled trial. The trainees are medical students from the University of Heidelberg in their clinical years with no prior laparoscopic experience. After a laparoscopic basic virtual reality (VR) training, 80 students are randomized into one of two arms in a 1:1 ratio to the checklist group (group A) and control group without a checklist (group B). After all students are given an introduction of the training center, VR trainer and laparoscopic instruments, they start with E-learning while watching explanations and videos of RYGB. Only group A will perform ratings with a modified Bariatric Objective Structured Assessment of Technical Skill (BOSATS) scale checklist for all videos watched. Group B watches the same videos without rating. Both groups will then perform an RYGB in the VR trainer as a primary endpoint and small bowel suturing as an additional test in the box trainer for evaluation.
This study aims to assess if E-learning and rating bariatric surgical videos with a modified BOSATS checklist will improve the learning curve for medical students in an RYGB VR performance. This study may help in future laparoscopic and bariatric training courses.
German Clinical Trials Register, DRKS00010493. Registered on 20 May 2016.
Minimally invasive surgery (MIS) plays an important role in a number of surgical disciplines i.e., bariatric surgery. Surgeons require different skills and abilities for MIS compared to open surgery . Over the past two decades, there has been a great patient demand for MIS, requiring laparoscopic training for surgeons [2, 3]. Laparoscopic techniques have created a new paradigm in surgical training. Traditionally, residents and surgeons learned skills hands-on in the operation room (OR), but that approach delays their training in MIS since they are only able to perform few maneuvers [3, 4]. Learning technical and non-technical skills outside the OR is vital for MIS due to additional difficulties that prolong the learning curve. These include pivot and fulcrum effects, lack of haptic feedback, and lack of a three-dimensional view . Currently, there are several laparoscopic training modalities: box trainers, organ models, cadavers, cadaveric organs, live animals, and virtual reality (VR) . With the use of real laparoscopic instruments, box trainers provide a realistic platform for learning . VR has proven to be a safe and effective training modality for MIS, creating a virtual environment for laparoscopic basic skills and operations [7, 8].
The laparoscopic approach to bariatric surgery is considered the “gold standard” for the surgical management of obesity . Laparoscopic Roux-en-Y gastric bypass (RYGB) is the most common bariatric procedure performed [10, 11]. RYGB can be a technically challenging operation for surgeons and trainees. In order to perform the surgery, trainees should first master the basic MIS technique to perform a safe surgery . RYGB has a complication rate that is almost three times higher than suspected during the learning curve . E-learning websites provide videos of surgeries with explanations of the techniques, the relevant anatomy, and perioperative management [14, 15]. The efficacy of E-learning modalities has been studied with positive results for E-learning both alone and in combination with other training modalities . Bariatric Objective Structured Assessment of Technical Skill (BOSATS) is currently the only procedure-specific rating scale specifically developed and validated for use in RYGB. BOSATS was intentionally designed to address multiple approaches to RYGB, increasing its transferability between surgeons and institutions . Checklists, such as BOSATS, have been shown to provide trainees with structured formative feedback and to improve learning curves . Implementation of the BOSATS scale has the potential to provide trainees with objective structured feedback, facilitate deliberate practice, and shorten learning curves in the operating room .
We hypothesize that using the BOSATS checklist during E-learning will improve the learning curve and facilitate transfer to practice. The present study aims to explore whether trainees will have an improved learning curve for RYGB on the VR trainer by E-learning and rating videos with a modified BOSATS checklist than just by E-learning without the use of a checklist.
The primary objective of this study is to identify if students in group A, who undergo E-learning and rate surgical videos with a modified BOSATS checklist, will have a better learning curve while performing an RYGB with the VR trainer than students in the control group, who use E-learning without rating the videos. Secondary goals include the transfer of skills to laparoscopic small bowel suturing using an Objective Structured Assessment of Technical Skill (OSATS) scale [19, 20] (Fig. 1). The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) schedule is given in Fig. 2.
This is a prospective, single-center, two-arm, parallel-group randomized controlled trial.
Settings and trainees
This study is carried out in the MIS training center of the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital. This study offers voluntary laparoscopic training courses to medical students at Heidelberg University during their clinical years of study (3rd to 6th year).
Inclusion and exclusion criteria
Inclusion criteria for the study are students enrolled at Heidelberg University Medical School during their clinical years. Exclusion criteria are students who are not in their clinical years or who have already participated in basic laparoscopy training courses for more than 2 hours, who have experience in laparoscopic suturing and knot tying, or who have experience assisting in laparoscopic surgeries for more than 2 hours.
This curriculum uses multiple modalities of training to verify and ascertain any advantage in each one. The training groups will participate in a standardized and structured multimodality training curriculum involving E-learning, VR trainer and laparoscopic box trainers. Basic skills are trained with the VR trainer and box trainer in a standardized and structured curriculum (Table 1). For E-learning two different websites are used, www.webop.de and www.websurg.com, and three RYGB videos. During E-learning, group A will rate all three videos with a modified BOSATS checklist (Table 2); the control group (group B) will not be using the checklist. After E-learning, both groups will perform a 4-step RYGB in the VR trainer and will be evaluated with a modified BOSATS scale by an experienced member of the staff (Table 2). As an additional test, using a laparoscopic box trainer, trainees will suture a small bowel incision and will be rated with an OSATS scale (Table 3) to evaluate their performance. Additional to this step, trainees will also be evaluated using a knot quality checklist (Table 4) with a maximum of 5 points. As a last step, all trainees will take a technical knowledge test to evaluate their RYGB post-test knowledge (Table 5, Fig. 1).
Introduction to the training modalities in the training center
The trainees receive a standardized introduction and instructions to use the VR trainer, box trainer, and instruments by trained staff. All students can familiarize themselves with the training center and training devices before starting the tests and exercises.
Basic skills training
All trainees will attend the MIS training center of the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital and perform 10 hours of standardized basic skills training. This includes instrument coordination tasks as well as laparoscopic suturing and knot tying exercises with box trainers. At the end the trainees will perform basic skills tasks with the VR trainer for one hour as a pre-test (Table 1).
The pre-test for both groups includes the laparoscopic basic skills training tasks in the VR trainer. Groups A and B will perform eight basic skills tasks before starting with E-learning. The objective for these exercises is to learn about the VR trainer management and functions to train for their RYGB performances (Table 1).
Trainees are randomly allocated to either the checklist group (group A) or control group (group B) with the sealed envelopes technique. The randomization of subjects is performed in a 1:1 ratio by block randomization with a variable block length using a computer-generated randomization list. Trainees are allocated to groups without stratification by gender or previous operative experience. The employee responsible for the randomization and group assignment is otherwise not involved with the training, tests, and data from the present study. As student recruitment to the study will be completed before randomization, any influence of randomization results or subsequent task assignments is considered minimal. We aim to compare both groups following data acquisition.
Introduction to laparoscopic Roux-en-Y gastric bypass by E-learning
All trainees work with E-learning modalities for three hours as an introduction to RYGB after randomization. This is done in a standardized fashion by using the same room at the Department of Surgery at Heidelberg University Hospital with identical surrounding conditions in order to rule out any difference between trainees. The trainees are given an explanatory introduction by trained staff in a standardized way to begin the RYGB modalities on www.webop.de and www.websurg.com. During this introduction, trainees are asked to study and understand the anatomy, illustrations, and videos of the procedural techniques. Following this general overview, the trainees will watch three anonymized RYGB videos to get a clearer view of the surgical techniques. Group A will rate the correct performance of the operative technique with the BOSATS checklist, while group B will not use a checklist (Table 2).
The post-test includes the RYGB on the VR trainer and a modified BOSATS evaluation. Groups A and B will perform the VR trainer post-test at the end of the training curriculum. Both groups will perform RYGB on the VR trainer three times and will be evaluated with the modified BOSATS by an experienced staff member who is blinded to the training status of trainees (Table 2).
Transfer of training test
The additional test includes suturing a small bowel incision with the laparoscopic technique. After the post-test, groups A and B will suture a 3-cm incision on cadaveric porcine small bowel in a laparoscopic box and will be evaluated by the blinded staff with an OSATS score for suturing and knot tying (Table 3) and a knot quality checklist (Table 4).
As a last step, all trainees will take a multiple choice (MC) technical knowledge test to evaluate their knowledge on the RYGB technique after the training curriculum (Table 5).
The primary endpoint is the performance of a 4-step RYGB on the VR trainer based on the modified BOSATS score evaluated by a blinded expert rater . RYGB steps on the VR trainer include (1) dissection of the gastro-phrenic ligament and creation of the gastric pouch, (2) location of Treitz ligament and measurement, (3) creation of gastro-jejunal anastomosis, and (4) creation of the jejuno-jejunal anastomosis.
The secondary endpoints include the time spent on the VR trainer to perform RYGB; time will be taken at all three times the students perform the procedure. VR trainer subscores and single parameters for each trainee will be evaluated. Also, trainees’ laparoscopic small bowel suturing performance will be included and evaluated with an OSATS scale. Additional endpoints include subgroup analyses of gender differences, gaming experience, and questionnaire evaluations of training after the course. Previous studies state that since surgery has been traditionally a male field, male students acquire surgical skills faster and have superior visuospatial skills than female students [21–24].
For both groups, the distribution of continuous data will be presented using mean, standard deviation (SD), minimum, maximum, and median, and for categorical variables, absolute and relative frequencies will be used. The primary endpoint, which is the modified BOSATS score, will be compared between both groups using a t test with a significance level of 0.05. Comparisons regarding secondary endpoints will be performed by the chi-square test for categorical data and the t test for continuous variables. Resulting p values from secondary analyses will be interpreted descriptively.
Sample size determination
Sample size determination was calculated for the BOSATS score. Previous published data from a study by Zevin et al. was used. The data was modified according to the BOSATS with a maximum score of 115 points. Group 1 had a mean score of 95.8 points with an SD of 9.9, while group 2 had a mean of 82.9 points with an SD of 15.0. Calculation was done for a significance level of α = 0.05 and a power of 1 − β = 0.8. An additional 10% was added to each group to compensate for the adjustment of the data. With these data differences can be detected with a minimum of 24 trainees in each group. To account for possible drop-outs the planned group size is 40 trainees per group.
This study evaluates if students who rate videos with a checklist during E-learning will have a better learning curve while performing an RYGB in the VR trainer than those who do E-learning without the ratings and checklist. Rating videos seems like an extra training for students; therefore, expectations are that trainees who perform the video ratings will have a better performance than those who just use E-learning and no rating. The continuous data recording of the VR trainer and the tests will help us understand if there is a difference in learning curves between both training groups . The assessments of the study trainees will help us to understand the possible factors of influence for successful surgical education. It is important to ascertain which module will have a better outcome to be implemented into further laparoscopic and bariatric surgery training.
Limitations of the study
There are some limitations to the study; subjects are limited to be medical students in their clinical years. Participants’ lack of surgical knowledge and bariatric surgery experience may influence their performance during the study. On the other hand, the inclusion of laparoscopy-naïve medical students allows for better differentiation of intervention effects, as the study group is very homogenous concerning surgical experience. In addition, the students have a total of 11 hours of laparoscopy training using the box trainer and the VR trainer before performing the virtual RYGB after extensive E-Learning for this procedure. Due to the fact that the trainees are laparoscopic novice medical students, the results cannot be transferred directly to more experienced surgeons. However, the results will provide a better perspective for designing optimal bariatric surgery training.
Recruitment started in April 2016 and the collection of data was finished in August 2016. Data analyses are currently running.
Bariatric Objective Structured Assessment of Technical Skill
Multiple choice test
Minimally invasive surgery
Objective Structured Assessment of Technical Skill
Laparoscopic Roux-en-Y gastric bypass
Hendrie JD, Nickel F, Bruckner T, Kowalewski K-F, Garrow CR, Mantel M, et al. Sequential learning of psychomotor and visuospatial skills for laparoscopic suturing and knot tying — study protocol for a randomized controlled trial “The shoebox study”. Trials. 2016;17:14.
Hamad GG, Curet M. Minimally invasive surgery. Am J Surg. 2010;199:263–5.
Yiannakopoulou E, Nikiteas N, Perrea D, Tsigris C. Virtual reality simulators and training in laparoscopic surgery. Int J Surg Lond Engl. 2015;13:60–4.
Friedman RL, Pace BW. Resident education in laparoscopic cholecystectomy. Surg Endosc. 1996;10:26–8.
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458–64.
Nickel F, Bintintan VV, Gehrig T, Kenngott HG, Fischer L, Gutt CN, et al. Virtual reality does not meet expectations in a pilot study on multimodal laparoscopic surgery training. World J Surg. 2013;37:965–73.
Nickel F, Jede F, Minassian A, Gondan M, Hendrie JD, Gehrig T, et al. One or two trainees per workplace in a structured multimodality training curriculum for laparoscopic surgery? Study protocol for a randomized controlled trial — DRKS00004675. Trials. 2014;15:137.
Stefanidis D, Heniford BT. The formula for a successful laparoscopic skills curriculum. Arch Surg Chic Ill 1960. 2009;144:77–82. discussion 82.
Nguyen NT, Goldman C, Rosenquist CJ, Arango A, Cole CJ, Lee SJ, et al. Laparoscopic versus open gastric bypass: a randomized study of outcomes, quality of life, and costs. Ann Surg. 2001;234:279–89. discussion 289–91.
Livingston EH. The incidence of bariatric surgery has plateaued in the U.S. Am J Surg. 2010;200:378–85.
Seki Y, Kasama K. Current status of laparoscopic bariatric surgery. Surg Technol Int. 2010;20:139–44.
Sánchez-Santos R, Estévez S, Tomé C, González S, Brox A, Nicolás R, et al. Training programs influence in the learning curve of laparoscopic gastric bypass for morbid obesity: a systematic review. Obes Surg. 2012;22:34–41.
Schauer P, Ikramuddin S, Hamad G, Gourash W. The learning curve for laparoscopic Roux-en-Y gastric bypass is 100 cases. Surg Endosc. 2003;17:212–5.
Pape-Koehler C, Chmelik C, Aslund AM, Heiss MM. An interactive and multimedia-based manual of surgical procedures: Webop—an approach to improve surgical education. Zentralblatt Für Chir. 2010;135:467–71.
Mutter D, Vix M, Dallemagne B, Perretta S, Leroy J, Marescaux J. WeBSurg: an innovative educational Web site in minimally invasive surgery--principles and results. Surg Innov. 2011;18:8–14.
Pape-Koehler C, Immenroth M, Sauerland S, Lefering R, Lindlohr C, Toaspern J, et al. Multimedia-based training on Internet platforms improves surgical performance: a randomized controlled trial. Surg Endosc. 2013;27:1737–47.
Zevin B, Bonrath EM, Aggarwal R, Dedy NJ, Ahmed N, Grantcharov TP, et al. Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg. 2013;216:955–65. 1031–33.
Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med J Assoc Am Med Coll. 1998;73:993–7.
Nickel F, Hendrie JD, Stock C, Salama M, Preukschas AA, Senft JD, et al. Direct observation versus endoscopic video recording-based rating with the objective structured assessment of technical skills for training of laparoscopic cholecystectomy. Eur Surg Res Eur Chir Forsch Rech Chir Eur. 2016;57:1–9.
Nickel F, Brzoska JA, Gondan M, Rangnick HM, Chu J, Kenngott HG, et al. Virtual reality training versus blended learning of laparoscopic cholecystectomy: a randomized controlled trial with laparoscopic novices. Medicine (Baltimore). 2015;94:e764.
Donnon T, DesCôteaux J-G, Violato C. Impact of cognitive imaging and sex differences on the development of laparoscopic suturing skills. Can J Surg J Can Chir. 2005;48:387–93.
Thorson CM, Kelly JP, Forse RA, Turaga KK. Can we continue to ignore gender differences in performance on simulation trainers? J Laparoendosc Adv Surg Tech A. 2011;21:329–33.
Ali A, Subhi Y, Ringsted C, Konge L. Gender differences in the acquisition of surgical skills: a systematic review. Surg Endosc. 2015;29:3065–73.
Nickel F, Hendrie JD, Kowalewski K-F, Bruckner T, Garrow CR, Mantel M, et al. Sequential learning of psychomotor and visuospatial skills for laparoscopic suturing and knot tying—a randomized controlled trial “The Shoebox Study” DRKS00008668. Langenbecks Arch Surg Dtsch Ges Chir. 2016;401(6):893–901.
Nickel F, Hendrie JD, Bruckner T, Kowalewski KF, Kenngott HG, Müller-Stich BP, et al. Successful learning of surgical liver anatomy in a computer-based teaching module. Int J Comput Assist Radiol Surg. 2016;11(12):2295–301.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8:18.
The present research was conducted within the setting of the SFB/Transregio 125 “Cognition-Guided Surgery” funded by the German Research Foundation.
This study is partly sponsored by the European Social Fund of the State Baden Wuerttemberg.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
FN, RDLG, BPMS, MWS, HGK, LF and KFK conceived and designed the study. MWS, MF, KFK, and RDLG acquired the data. TB, KFK, and FN performed the statistical analysis. RDLG, FN, MWS, KFK, and TB analyzed and interpreted the data. RDLG, MF, FN, KFK, and MWS drafted the manuscript; BPMS, HGK, LF and FN critically revised it. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Written informed consent for participation in the study and publication of the acquired anonymous data is obtained from each participant.
Ethics approval and consent to participate
All data for the study are recorded anonymously, treated confidentially, and evaluated by authorized staff for scientific purposes only. Trainees’ names are kept separate from all study data and are not used for the study. Each participant is assigned a designated code that is used for the entire study documentation and data collection. The study courses are offered in addition to compulsory university courses. Participation in the study is voluntary and may be ended at any time. There are no foreseeable negative consequences for trainees related to participation. Participants are not allowed to be part of any laparoscopic training throughout the duration of the study. The participating staff of the Heidelberg MIS center is experienced in the handling of training devices and in tutoring MIS. The benefits of training for students are numerous: stamina, concentration, and manual adroitness are enhanced and practiced, surgical interest may be stimulated or invigorated, and students are able to begin their first practical laparoscopic experience, which may be used during later work. In the event that a participant’s physical or mental health becomes jeopardized due to participation in the present study, the participant will be dismissed immediately and excluded from the study. Ethical approval was obtained from the Ethics Committee of the Medical Faculty at Heidelberg University prior to the beginning of the study (Code S-334/2011, Amendment 07/05/2012). The Consolidated Standards of Reporting Trials (CONSORT) guidelines for randomized controlled trials and SPIRIT guidelines (Fig. 2, Additional file 1) for implementation of study protocols were followed [26, 27]. This trial was registered with the German Clinical Trials Register (DRKS) in Freiburg, Germany on 20 May 2016 under trial registration number DRKS00010493.
Nickel F reports receiving travel support for conference participation as well as equipment provided for laparoscopic surgery courses by KARL STORZ, Johnson & Johnson, and Medtronic. Friedrich M, Kowalewski KF, Schmidt M, De La Garza JR, Kenngott HG, Bruckner T, Fischer L, and Müller-Stich BP have no conflicts of interest or financial ties to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
De La Garza, J.R., Kowalewski, K., Friedrich, M. et al. Does rating the operation videos with a checklist score improve the effect of E-learning for bariatric surgical training? Study protocol for a randomized controlled trial. Trials 18, 134 (2017). https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-017-1886-7
- Minimally invasive surgery
- Human mirror system
- Serious gaming
- First-person view