Published online: 2 January, TAPS 2018, 3(1), 42-49
DOI: https://doi.org/10.29060/TAPS.2018-3-1/SC1059

Bikramjit Pal, M. V. Kumar, Htoo Htoo Kyaw Soe, Sudipta Pal

Melaka Manipal Medical College, Manipal University, Malaysia

Abstract

Introduction: Simulation is the imitation of the operation of a real-world process or system over time. Innovative simulation training solutions are now being used to train medical professionals in an attempt to reduce the number of safety concerns that have adverse effects on the patients.

Objectives: (a) To determine its usefulness as a teaching or learning tool for management of surgical emergencies, both in the short term and medium term by students’ perception. (b) To plan future teaching methodology regarding hi-fidelity simulation based on the study outcomes and re-assessment of the current training modules.

Methods: Quasi-experimental time series design with pretest-posttest interventional study. Quantitative data was analysed in terms of Mean, Standard Deviation and standard error of Mean. Statistical tests of significance like Repeated Measure of Analysis of Variance (ANOVA) were used for comparisons. P value < 0.001 was considered to be statistically significant.

Results: The students opined that the simulated sessions on high fidelity simulators had encouraged their active participation which was appropriate to their current level of learning. It helped them to think fast and the training sessions resembled a real life situation. The study showed that learning had progressively improved with each session of simulation with corresponding decrease in stress.

Conclusion: Implementation of high fidelity simulation based learning in our Institute had been perceived favourably by a large number of students in enhancing their knowledge over time in management of trauma and surgical emergencies.

Keywords:      High fidelity simulation, Simulation in medical education, Stress in simulation

I. INTRODUCTION

High fidelity simulation is an innovative and effective strategy to address increasing student enrolment, faculty shortages, and limited clinical sites (Schoenig, Sittner, & Todd, 2006). The value of simulation in undergraduate medical education is now well established; it basically animates the curriculum. Medical training in the current era is multi-modular and simulation based learning may play a pivotal role in improving training standards in medical schools (Joseph et al., 2015). High Fidelity Patient Simulators replicate patient care scenarios in a realistic environment and have advantage of repetition of the same scenario in a controlled environment which allows practice without risk to patient thereby minimizing chances of medical error and thus, make them a useful tool for student assessment. It is also recognized that putting the learners into a simulated critical care environment subjects them to stresses which have not been well studied. There were many studies on the use of simulation (mainly low and medium fidelity) in medical education but few studies were done on the effectiveness of high fidelity simulation based teaching in under-graduate medical students. The importance of simulation in training medical students are being recognized by academic institutions around the world. In spite of proven benefits, it has so far not been formally introduced as a part of curriculum in medical colleges in our settings. With this background, this study was conducted to explore the perception of medical students on the usefulness of high fidelity patient simulation.

II. METHODS

This was an ongoing research study about the impact of high fidelity patient simulation in undergraduate medical education. METIman Pre-Hospital High Fidelity Patient Simulator (Serial number: MMP-0418/2013; CAE Healthcare, USA) was used in this study. The final year MBBS students of MMMC were the subjects of this study. The students of these batches who volunteered were recruited during their surgical posting after obtaining their informed consent. The proportion of students who volunteered was 92.73% (204 students out of total number of 220 students).

The simulation sessions were conducted with one sub-group of 12 to 15 students which were further divided into 3 teams of 4 to 5 students. The participants were briefed about the simulation sessions and expected learning outcomes. The duration of each simulation session was 50 minutes: Briefing (10 minutes), Simulation (25 minutes) and Debriefing (15 minutes).  A theoretical briefing was given by the investigator on ATLS protocol for trauma management and management of surgical emergencies like hypovolemic shock, tension pneumothorax and head injury. This briefing was done as an interactive lecture to the whole subgroup. Each team then participated in a trauma simulation session and the scenario was chosen randomly from among the conditions mentioned above. The three teams in a subgroup were assigned three different scenarios. The teams were then debriefed in order to achieve the learning outcomes. The same team had participated second time in the simulation of the same scenario after 1 week and third time after 3 – 4 weeks to test their short to medium term retention of knowledge and practical skills, followed by final debriefing. Thus, at the end of the course, each student was expected to perform satisfactorily any of the roles during the management of the standard scenarios.

Each student was assessed individually in terms of their progress in knowledge, confidence and stress reduction. We developed a standardized five point (very poor to excellent) Likert scale questionnaire (Appendix I), to collect initial background knowledge of the students on the first day of the training course in terms of ATLS protocol for management of acute trauma, management of hypovolemic shock, management of tension pneumothorax and management of head injury. The same questionnaire was repeated after every session for post-session knowledge assessment.

Another set of five point Likert scale questionnaire was designed to obtain participant feedback after each session on the relevance and usefulness of the simulation experience, effectiveness of briefing and debriefing and stressor assessment. It was an ordinal scale used by respondents to rate the degree to which they agree or disagree to a statement (Appendix II). The stressor questionnaire contained thirteen items (Appendix III). They resembled pre and post tests for comparison to note progress in confidence and stress reduction.  Finally, a third set of questionnaire was administered to the students at the completion of training for their feedback assessment of training course (Appendix IV).

We used Microsoft Excel for data entry and SPSS software (SPSS Inc. Released 2009. PASW Statistics for Windows, Version 18.0. Chicago: SPSS Inc.) for data analysis. We calculated descriptive statistics such as frequency and percentage for categorical data; mean and standard deviation for total score of knowledge, simulation assessment and stressor assessment. We used one-way repeated measure ANOVA to determine the statistically significant difference in simulation assessment (total score) and stressor assessment (total score). We also used Friedman test to determine the statistically significant difference in individual item of knowledge assessment, simulation assessment and stressor assessment. P value <0.001 was taken to be statistically significant in our study.

III. RESULTS

The cohort of 204 participants in this study were selected from four batches of final year MBBS students (October 2015 to April 2016).

Friedman test of simulation assessment for individual items showed significant difference of simulation assessment over time. One-way repeated measure ANOVA of stressor assessment (total score) revealed statistically significant difference (p < 0.001) of total score of stressor assessment over time. Total score of stressor assessment was decreased from 27.09 (Mean) / 7.41(SD) at pre-simulation to 25.63 (Mean) / 8.06(SD) at post-simulation I, to 23.92 (Mean) / 8.92(SD) at post-simulation II and to 23.75 (Mean) / 9.77(SD) at post-simulation III.

For assessment of stress during simulation sessions, we used Likert scale of 1 to 5 (low stress to maximum stress). There was significant difference (p < 0.001) of stressor assessment during simulation over time (Friedman test) for following individual items where majority of the students had the opinion of “moderate stress” regarding “Competition with team members”, “Limited time during simulation sessions”, “Participation in debriefing” and “high stress” regarding “Death of simulated patient”.

Table 1. Simulation assessment at pre-simulation, post-simulation I, post-simulation II and post-simulation III (individual item)

IV. DISCUSSION

High fidelity simulators have revolutionised training as almost any emergency situation can be replicated. Simulation sessions have provided opportunity for clinical students to collaborate and apply both cognitive and psychomotor skills. Our main objective was to determine usefulness of high fidelity simulation as a teaching or learning tool for management of surgical emergencies. The study showed that high fidelity simulators had made a difference in enhancing the knowledge over time in management of trauma and surgical emergencies as perceived by our students. It also showed that learning had significantly improved with each session of simulation and learners’ attitudes were supportive of simulation. Wayne, Barsuk, O’Leary, Fudala, & McGaghie (2008) showed that internal medicine residents had increased knowledge and skills using simulation technology and deliberate practice. Participants in one study (Okuda et al., 2009) felt simulation based teaching was a reliable tool for assessing learners by providing good feedback on performance which was similar to our observation. In a study by Founds, Zewe, & Scheuer (2011), participants felt that high fidelity simulators can present simulations that were closer to real life situations which was similar to opinion of most of our students. The study revealed that simulation sessions with high fidelity simulators encouraged active participation of students who need further sessions on simulation for better understanding of clinical problems and knowledge acquisition. The finding in one of the main area of study: “Clinical management more easily learned” was not satisfactorily documented (p < 0.996). It showed that simulation did not always help in better understanding of management of clinical problems. The drop in stress was significant at week II and III but flattened out in week IV which might be due to participants’ increasing adaptability to simulated atmosphere. The time for briefing was adequate but participants felt that time for debriefing was inadequate and debriefing did not help them to learn better. This was an area of utmost concern to us as we concluded that there was a definite lacuna in our debriefing process. We planned to rectify our shortcomings and deficiencies in this matter. For long term assessment we planned to get these students back during their internship period and do a re-test to validate the improved outcome. Most of the students had a favourable perception about high fidelity patient simulation indicating that it has bright prospect for its inclusion in under-graduate curriculum in near future.

V. CONCLUSION

Few studies have been done with regards to students’ perception on the effectiveness of high fidelity simulators in training under-graduate medical students. This innovative training method may help to improve the quality of medical care and safety of the patient. The limitation of this study: This was a single centre study and participants who volunteered were only recruited. Hence findings may not be applicable to other settings. Even though training with high fidelity simulators was perceived positively by the students, it remained unclear whether the learning skills acquired with this teaching methodology would translate into improved patient care.

Notes on Contributors

Bikramjit Pal MBBS, DA (Anesthesiology), MS (Surgery); Associate Professor, Department of Surgery and Chairman of Clinical Skills Learning Committee, Member of the International Forum of Teachers of the UNESCO Chair in Bioethics, Melaka-Manipal Medical College, Manipal University, Malaysia

MV Kumar BSc, MBBS, FRCS Ed; Professor and Ex-HOD, Department of Surgery and member of Clinical Skills Learning Committee, Melaka-Manipal Medical College, Manipal University, Malaysia

Htoo Htoo Kyaw Soe MBBS, MPH, PhD; Associate Professor, Department of Community Medicine and member of Statistics Committee, Melaka-Manipal Medical College, Manipal University, Malaysia

Sudipta Pal MBBS, PGDMCH; Lecturer, Department of Community Medicine, Melaka-Manipal Medical College, Manipal University, Malaysia

Ethical Approval

Duly taken from the IRB & IEC, Melaka-Manipal Medical College.

Acknowledgements

The final year MBBS students of Melaka-Manipal Medical College who had participated in this research project, the faculty of the Department of Surgery, the lab assistants and technicians of Clinical Skills Lab, the Management of Melaka-Manipal Medical College and last but not the least, Prof. Dinker Pai (Ex-Head of Department of Surgery, MMMC), for his inspiration, initiation and conceptualization of this research project.

Declaration of Interest

The authors have not received any funding or benefits from industry or elsewhere to conduct this study and have no conflicts of interest.

References

Founds, S. A., Zewe, G., & Scheuer, L. A. (2011). Development of high fidelity simulated clinical experiences for baccalaureate nursing students. Journal of Professional Nursing, 27(1), 5–9.

Joseph, N., Nelliyanil, M., Jindal, S., Utkarsha., Abraham, A.E., Alok, Y., … Lankeshwar, S. (2015). Perception of Simulation based Learning among Medical Students in South India. Annals of Medical and Health Sciences Research, 5(4), 247–252. doi: 10.4103/21419248.

Okuda, Y., Bryson, E. O., De Maria, S. Jr., Jacobson, L., Quinones, J., Shen, B., & Levine, A. l. (2009). The utility of simulation in medical education: What is the evidence? Mount Sinai Journal of Medicine, 76(4), 330–43. doi: 10.1002/msj.20127.

Schoenig, A. M., Sittner, B. J., & Todd, M. J. (2006). Simulated clinical experience: nursing students’ perceptions and the educators’ role. Nurse Education, 31 (6), 253-8.

Wayne, D. B., Barsuk, J. H., O’Leary, K. J., Fudala, M. J., & McGaghie, W. C. (2008). Mastery Learning of Thoracentesis Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice. Society of Hospital Medicine, 3(1), 48-54. doi: 10.1002/jhm.268.

*Bikramjit Pal
Associate Professor (Surgery),
Melaka-Manipal Medical College
Jalan Batu Hampar, Bukit Baru,
75150, Melaka, Malaysia.
Mobile: +60-1118728085.
Landline (Office): +60-6-2896662-1116; +60-6-2925849/50/51.
FAX: +60-6-2817977.
E-mail: bikramjit.pal@manipal.edu.my
Alternative email: drbikramjitpal@gmail.com

Published online: 2 January, TAPS 2018, 3(1), 38-41
DOI: https://doi.org/10.29060/TAPS.2018-3-1/SC1040

Sean Wu¹, Julia Farquhar^{1, 2}, Scott Compton¹

¹Duke-NUS Medical School, Singapore; ²School of Medicine, Duke University, USA

Abstract

Aim: Evidence suggests that Team Based Learning (TBL) is an effective teaching method for promoting student learning. Many people have also suggested that TBL supports other complex curriculum objectives, such as teamwork and communication skills.  However, there is limited rigorous, substantive data to support these claims. Therefore, the purpose of this study was to assess medical educators’ perceptions of the outcomes affected by TBL, thereby highlighting the specific areas of TBL in need of research.

Methods: We reviewed the published research on TBL in medical education, and identified 21 unique claims from authors regarding the outcomes of TBL.  The claims centred on 4 domains: learning, behaviours, skills, and wellbeing. We created a questionnaire that asked medical educators to rate their support for each claim.  The survey was distributed to the medical educators with experience teaching via TBL and who were active users of the Team Based Learning Collaborative listserv.

Results: Fifty responses were received.  Respondents strongly supported claims that TBL positively impacts behaviours and skills over traditional, lecture based teaching methods, including the promotion of self-directed learning, active learning, peer-to-peer learning, and teaching. In addition, respondents strongly supported claims that TBL promotes teamwork, collaboration, communication and problem solving.  Most participants reported that TBL is more effective in promoting interpersonal, accountability, leadership and teaching skills.

Conclusion: Medical educators that use TBL have favourable perceptions of the practice across a variety of domains.  Future research should examine the actual effects of TBL on these domains.

Keywords:      Team Based Learning; Medical Education; Teaching

I. INTRODUCTION

Team-based learning (TBL), originally developed for use in business schools, is now a growing teaching modality in medical education. The process of TBL is comprised of three components: advance preparation, individual and team readiness assurance tests (RATs), and in-class application of content through team exercises (Parmelee, Michaelsen, Cook, & Hudes, 2012). In a TBL course, students are typically assigned into teams of 5 to 7, and receive preparatory materials before class. At the beginning of the TBL process, each individual completes a readiness assurance test (I-RAT). Students next re-take the same readiness assurance test during class as a team (T-RAT), reaching consensus to select a single best answer, and receive immediate feedback. Students typically then have the option to write evidence-based appeals for their wrong answers, and finally an instructor clarifies lingering questions. Following the readiness assurance phase, the teams then work on solving applied problems, followed by whole class discussions.  Evidence suggests that TBL is a popular teaching modality, with approximately one-third of US schools and colleges of pharmacy implementing it into the curriculum (Allen et al., 2013).

A number of studies have shown TBL to be as academically effective or more effective than traditional lecture-based courses by promoting mastery over several core medical subjects (Fatmi, Hartling, Hiller, Campbell, & Oswald, 2013; Koles, Stolfi, Borges, Nelson, & Parmelee, 2010; Nieder, Parmelee, Stolfie, & Hudes, 2005). These studies have focused specifically on the academic outcomes of TBL, yet TBL as a teaching modality is often assumed to support complex curriculum objectives such as teamwork and communication skills. Despite these assumptions, little substantive data exists to support these claims.  For the current body of research to progress and to improve our understanding of the true effects of TBL, these assumed outcomes need to be explicitly described and articulated. Following which, individual outcomes can be studied and interventions can be proposed to further develop TBL as a medical teaching modality. Therefore, the purpose of this study was to assess TBL medical educators’ perceptions of the outcomes affected by TBL, thereby providing a framework for future research.

II. METHODOLOGY

A. Development of Questionnaire

We identified 21 claims of the positive effects of TBL in various published papers and textbooks.  We categorized these claims into 4 domains: learning, behaviours, skills, and well-being. Examples of references to each claim are available from the corresponding offer. We subsequently compiled those claims and used them as the basis for a questionnaire in order to assess the support for each claim among TBL users.

The final questionnaire consisted of 6 demographic questions, followed by the claims within each domain: learning (4 questions), behaviours (4 questions), skills (8 questions) and wellbeing (5 questions).  In addition, we asked 3 questions related to the extent that TBL skills learned in classroom settings translates to clinical settings.

Cognitive interviews were conducted with ten TBL educators to determine if the questions were clear and interpreted as intended. Relevant edits were made and the questionnaire was revised. We then assessed reliability using the test-retest method. Eight TBL educators responded to the questionnaire twice within two weeks. The test-retest reliability of the instrument was 0.927 (df = 8, p<0.001) based on Spearman’s correlation coefficient, thus demonstrating its suitability for research use.

B. Dissemination

The final survey was uploaded onto surveymonkey.com and submitted to the Team-Based Learning Collaborative (TBLC) listserv with a request for medical educators to respond. The TBLC listserve is comprised of people interested in TBL, and is open to all educators. The introductory email made explicit that medical educators were the target of the survey.

III. RESULTS

A total of 50 medical educators replied. The participants were a seasoned group of medical educators, having an average of 14 years’ experience in the field. Most respondents were senior faculty members (25% professor, 36% associate professors), while 19% were assistant professors and 20% had other academic appointments. Overall the respondents had substantial experience in using TBL: 96% had created TBL materials, 98% had facilitated TBL discussions, and 78% had designed course curriculum with TBL as the primary educational modality.

As shown in Figure 1, respondents tended to strongly support the various claims that suggest TBL positively impacts learning, behaviour, and skills.  In fact, most participants reported that TBL promotes “Deep Learning” and “Application of knowledge” much more, 83% and 98% respectively, than conventional lecture based modalities.  Nearly all (96%) reported that TBL is more effective in promoting retention of knowledge than lecture and most (64.5%) replied that TBL is more effective in promoting breadth of knowledge. A minority (25%) of respondents indicated that there is no difference between the two teaching modalities in student learning.

In terms of student behaviour, respondents reported that TBL promotes self-directed learning, active learning, peer-to-peer learning and teaching much more than lecture based teaching modalities. Most (98%) answered that TBL is more effective in teaching self-directed learning, peer-to-peer learning, and peer-to-peer teaching than lecture. All answered that TBL promotes active learning more than traditional lecture based teaching. Only one replied that TBL and lecture-based teaching are equally effective in promoting the aforementioned student behaviours.

Most respondents replied that TBL is more effective than lecture-based teaching in promoting a multitude of student skills. All answered that TBL is better at promoting teamwork, collaboration, communication and problem solving. Most of participants reported that TBL is more effective in promoting interpersonal, accountability, leadership and teaching skills.

Figure 1 also demonstrates that there is less support for the claims surrounding TBL promotion of wellbeing.  Nearly half (44%) of the respondents answered that TBL resulted in more competition than lecture-based teaching whereas a quarter replied the opposite. Half reported that TBL leads to more anxiety whereas a quarter replied otherwise. More than half reported that TBL promoted empathy more so than lecture, whereas 35% of respondents were unsure or thought the two teaching modalities equal. Most respondents answered that lecture-based teaching was easier than TBL, yet also answered that TBL was more enjoyable. Interestingly, respondents also reported that TBL-learned skills translated well to the clinical setting. Most, 89.5%, 83.3%, and 66.7% respectively, answered that teamwork, communication, and leadership skills learned in a TBL classroom were applicable to the clinical rotations.

Figure 1. Medical educators’ ratings: lecture-based teaching versus team based learning in promoting different skills

IV. DISCUSSION

The results of the study indicate that TBL medical educators believe TBL to be better than traditional lecture-based teaching in achieving a vast variety of outcomes. According to the survey results, educators believe that TBL promotes students’ mastery of subject content by encouraging deep and active learning as well as the retention and application of knowledge. They also believe that TBL develops behaviours such as peer-to-peer learning and teaching. An important point is that most TBL educators believe TBL fosters development of key skillsets such as accountability, teamwork, collaboration, and communication that play vital roles in the successful transition of students from classroom learning to clinical clerkships. Yet respondents to this survey may be wary that TBL may negatively affect student wellbeing by increasing competition and difficulty. Interestingly, many of the TBL educators appeared confident that TBL aids in the development of medical students that are independent learners, teamwork-oriented and effective leaders.  However, to date, there is little substantive data to suggest that any of these claims are true.  Likewise, there was strong support for the notion that skills learnt in classroom TBL sessions translate to the clinical setting. Again, little evidence currently exists to support this claim.

This study is limited by the small convenience sample drawn from the TBL collaborative listserv. The participants may not be representative of TBL educators as a whole. However, the participant profiles reveal that these respondents have extensive experience in medical education and using TBL in the classroom.

V. CONCLUSION

This study should serve as a call to action to educational researchers to investigate the claims of the effects of TBL, particularly those associated with non-academic factors, such as teamwork, leadership, and communication. In fact, the results of this study could serve as the foundation for future research. Ensuing studies can provide evidence towards determining the validity of these assumptions, thereby providing direction for the development of TBL as a teaching modality in medical education.

Notes on Contributors

Sean Wu, MD, is a graduate of the Duke-NUS Medical School and currently an Internal Medicine resident at NUHS.  Julia Farquhar is a medical student at the Duke School of Medicine, Durham, NC, USA.

Scott Compton, PhD, is the Associate Dean and Associate Professor of Medical Education, Research, and Evaluation at Duke-NUS Medical School.

Ethical Approval

The approval for this study was obtained from the Institutional Review Board, National University of Singapore.

Declaration of Interest

The authors declare that they have no competing interests.

References

Allen, R. E., Copeland. J., Franks, A. S., Karimi, R., McCollum, M., & Riese, D. J. (2013). Team-based learning in US colleges and schools of pharmacy. The American Journal of Pharmaceutical Education, 77(6).  DOI: 10.5688/ajpe776115

Fatmi, M., Hartling, L., Hillier, T., Campbell, S., & Oswald, A.E. (2013). The effectiveness of team-based learning on learning outcomes in health professions education: BEME Guide No. 30.  Medical Teacher, 35(12), 1608-24. DOI: 10.31.09/0142159X.2013.849802

Koles, P. G., Stolfi, A., Borges, N. J., Nelson, S., & Parmelee, D.X. (2010). The impact of team-based learning on medical students’ academic performance. Academic Medicine, 85(11), 1739-1745.  DOI: 10.1097/ACM.0b013e3181f52bed

Nieder, G. L., Parmelee, D. X., Stolfi, A., & Hudes, P. D. (2005). Team‐based learning in a medical gross anatomy and embryology course. Clinical Anatomy, 18(1), 56-63. DOI: 10.1002/ca.20040

Parmelee, D., Michaelsen, L.K., Cook, S., & Hudes, P.D. (2012). Team-based learning: A practical guide: AMEE Guide No. 65. Medical Teacher, 34(5), e275-e287. DOI: 10.3109/0142159X.2012.651179

* Scott Compton, PhD
Medical Education, Research and Evaluation Department
Duke-NUS Medical School
8 College Road, Singapore 169857.
Tel: +65 6601 1565.
Email: scott.compton@duke-nus.edu.sg

Published online: 4 September, TAPS 2018, 3(3), 39-42
DOI: https://doi.org/10.29060/TAPS.2018-3-3/SC1057

Lim Mien Choo Ruth, Keith Tsou Yu Kei, Chooi Peng Ong, Sabrina Wong Kay Wye, Gilbert Tan Choon Seng, Winnie Soon Shok Wen, Joanne Quah Hui Min & Marie Stella P. Cruz

Division of Graduate Medical Studies, Yong Loo Lin School of Medicine, National University of Singapore

Abstract

This paper describes the revision of a national post-graduate medical examination to incorporate formal quality assurance and psychometrics. We discuss the considerations and rationale leading to the new format, challenges faced and lessons learned in making the change. The processes described were successfully implemented in the 2015 examination administration. We continue to reflect on and analyse these processes to improve the examination.

Keywords:           Post-graduate, Examination Reform, Quality Assurance, Psychometrics, Family Medicine, Standardised Patient

I. CONTEXT

In Singapore, population growth and a swiftly inverting population pyramid have diametrically changed the job scope of the family physician, impacting training and certification.

Postgraduate Family Medicine certification in Singapore formally began with the Membership of the College of General Practitioners (MCGP) in 1972, and was replaced in 1993 by the Master of Medicine in Family Medicine (MMed FM) examination. This examination comprised a theory portion, a viva voce and a clinical examination (Table 1).

In 2011, Singapore adopted the American Accreditation Council for Graduate Medical Education-International (ACGME-I) Residency system. Family Medicine Residency culminates in a summative multiple-choice question (MCQ) examination administered by the American Board of Medical Specialties (ABMS) with no clinical component.

With these training and contextual changes, the time was appropriate for a review and revision of the MMed FM examination. We describe the considerations and processes leading to the new MMed FM examination in 2015.

II. FROM THE OLD TO THE NEW

A. Family Medicine Examination Committee

In 2012, the Family Medicine Examination Committee (FMEC) was set up to administer and review the examination, reporting to the Division of Graduate Medical Studies (DGMS), with education consultative support from the Centre for Medical Education, National University of Singapore.  Senior family physicians from different training institutions form the FMEC.

B. Reviewing the Old Examination

Table 1. Summary of main components of the old and 2015 examinations

The previous long case involved assessing the patient and discussing management with the examiner. The short case was a physical examination station. Two long cases provided insufficient insight into the candidate’s mastery of the breadth of the Family Medicine curriculum (Swanson, Norman, & Linn, 1995). With real patients, standardisation of testing points across a cohort was not possible. Management was discussed with the examiner only, so communication with the patient about management was not assessed.

The old pass mark was pre-defined at 50%, regardless of difficulty of cases.

C. Developing the New Examination

We developed the new examination following review of literature (Swanson, Norman, & Linn, 1995; Van der Vleuten & Schuwirth, 2005; Mookherjee, Chang, Boscardin, & Hauer, 2013) and discussion to determine best practice. Our priorities were to improve the representativeness and reliability of testing within a practicable framework.

The new examination includes theory and clinical components, and psychometrics to analyse candidate performance. The theory component is now the ABMS MCQ examination. Candidates who are successful may sit the clinical examination.

The re-designed clinical examination comprises thirteen stations, with the slides and viva voce stations being holdovers from the old format. The remaining eleven comprise five short consultations, three long consultations, and three physical examination stations, an increase over the previous six stations (Swanson et al, 1995). This increases the number of sampling points and testing time and hence reliability (Van der Vleuten & Schuwirth, 2005).

For the consultations, the candidate evaluates, and then discusses management with the patient. He is judged on his handling of the consultation, not on his interaction with examiners. A long consultation tests multiple and more complex issues than a short consultation. Consultations are set in a primary care context, representing Family Medicine practice in Singapore.

The standardised patient (SP) was introduced, and consultations involve real and standardised patients. The three physical examination stations continue to use real patients.

The viva voce was retained to allow testing of professional communication and discussion of clinical reasoning.

Instead of a pre-defined standard, we introduced the use of psychometrics to determine the pass marks, and collaborated with a medical educationist who has been trained in assessment psychometrics. At the same time, we implemented examiner training (Van der Vleuten, Luyk, Ballegooijen, & Swanson, 1989), calibration, and standard setting. The passing standard is defined as that of a proficient family physician on the Dreyfus model of skill acquisition (Goh & Ong, 2014).

A new mark sheet, assessing clinical skills and global assessment of the candidate’s performance, was introduced.

D. Evaluating the New Examination

The FMEC committed to incorporating quality assurance measures into the design and implementation of the new clinical examination.

For the first time, an examination blueprint was formally developed. Testing points were chosen across the curriculum to reflect competencies required in Family Medicine practice (Mookherjee et al, 2013).

The blueprint guided recruitment of real patients and SP case development. FMEC members tested the SP cases for authenticity and reasonableness. Before each clinical examination bloc, all examiners participated in calibration sessions during which clinical history and signs were confirmed and salient points agreed upon.

Borderline regression and modified Angoff methods were used to set pass marks. Performance and reliability of individual stations and the entire examination were studied. Examiner performance was analysed and feedback given.

The examination, conducted under the auspices of the DGMS, conformed to standard operating procedures, access and security measures used in other DGMS examinations.

An external examiner (EE) observed the examination (Associate Professor Neil Spike). This EE had observed previous MMed FM examinations.

III. CHALLENGES AND LESSONS

The old examination format had been in place for forty-three years (1972 – 2014). The new format introduced standard setting, examination validity as an evidence-based concept, the use of SPs, and the use of statistics to analyse results.

Stakeholder engagement was, and remains, paramount. There was a need for consistent, sustained communication to trainers, residents and examiners.

With the consultation as test encounter, we are assessing for relevant history and hypothesis-driven physical examination. This differs from traditional assessment that rewards comprehensiveness. Examiners debated the boundaries of relevance and hypothesis-driven practice, to help form their expert judgement of what is required of proficiency.

With the many changes introduced, we set a requirement for examiners to have attended examiner training sessions before appointment. This is common in many examinations but new to us.

The format, as presented here, will remain for 2016 and 2017, after which time we will review the programme of assessment. At this time, we are in the process of gathering feedback and reviewing the outcomes for the examination administrations of 2015 and 2016.

Notes on Contributors

Dr Ruth Lim is the Chairperson of the Family Medicine Examination committee, Chief Examiner for the Master of Medicine (Family Medicine) Examination and Director of Education for SingHealth Polyclinics.

Dr Keith Tsou is the Deputy Chairperson of the Family Medicine Examination committee, Deputy Chief Examiner for the Master of Medicine (Family Medicine) Examination and Director of Clinical Services for National University Polyclinics.

Dr Ong Chooi Peng is member of the Family Medicine Examination committee, Senior Consultant of the Department of Family Medicine, National University Health System.

Dr Sabrina Wong is member of the Family Medicine Examination committee, Assistant Director of Clinical Services for National Healthcare Group Polyclinics.

Dr Gilbert Tan is member of the Family Medicine Examination committee, Assistant Director of Clinical Services for SingHealth Polyclinics.

Dr Winnie Soon is member of the Family Medicine Examination committee, Consultant Family Physician for National Healthcare Group Polyclinics.

Dr Joanne Quah is member of the Family Medicine Examination committee, Director for Outram Polyclinics under SingHealth Polyclinics.

Dr Marie Stella P. Cruz is member of the Family Medicine Examination committee, Family Physician and Adjunct Lecturer for Department of Medicine and Division of Graduate Medical Studies, National University of Singapore.

Ethical Approval

No IRB approval is involved.

Declaration of Interest

Authors declare no conflict of interest.

References

Swanson, D. B., Norman, G. R., & Linn, R. L. (1995). Performance-based assessment: Lessons from the health professions. Educational Research, 24(5), 5-11. https://doi.org/10.3102/0013189X024005005.

Van der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39, 309-17. Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-2929.2005.02094.x.

Vleuten, C. V., Luyk, S. V., Ballegooijen, A. V. & Swanson, D. B. (1989). Training and experience of examiners. Medical Education, 23(3), 290-6.

Goh, L. G. & Ong, C. P. (2014). Education and training in family medicine: progress and a proposed national vision for 2030. Singapore Medical Journal, 55(3), 117-23. http://dx.doi.org/10.11622/smedj.2014031.

Mookherjee, S., Chang, A., Boscardin, C. K. & Hauer, K. E. (2013). How to develop a competency-based examination blueprint for longitudinal standardized patient clinical skills assessments. Medical Teacher, 35(11), 883-90. https://doi.org/10.3109/0142159X.2013.809408.

*Ruth Lim
Division of Graduate Medical Studies,
Yong Loo Lin School of Medicine,
National University of Singapore
Blk MD3, Level 2, 16 Medical Drive,
Singapore 117600
Email: ruth.lim@singhealth.com.sg
Contact: +65-63777018

Published online: 4 September, TAPS 2018, 3(3), 35-38
DOI: https://doi.org/10.29060/TAPS.2018-3-3/SC1062

Li-Phing Clarice Wee

Ng Teng Fong General Hospital, Singapore

Abstract

Objectives: Do Not Attempt Resuscitation (DNAR) orders have been used in hospitals worldwide for the past 30 years, but are still considered to be a challenging and difficult area of practice. Nurses being the frontline healthcare professionals should be involved during the decision-making process and are required to have good understanding of the DNAR order, in order to provide effective and efficient care. Our aim was to investigate: nurses’ involvement during decision-making process, level of understanding of issues surrounding DNAR orders; and how they perceive care for patients with DNAR orders.

Methods: A descriptive crossed sectional study design using electronic questionnaires was adopted for the study.  The study was conducted among 400 nurses at a tertiary hospital in Singapore.

Results: This study showed that 44.5% of nurses reported physicians did not involve them in decisions for DNAR orders; only 8% felt that they should be involved in the decision-making process. Even if they did not agree with the order, 63.2% would still comply whilst 21% of them were willing to discuss this further with the treatment teams. Most agreed that antibiotics, intravenous fluids, oxygen therapy and artificial feeding were appropriate for patients with DNAR orders. Majority (57.1%) expressed uneasiness in discussing end of life issues with patients even in specialty areas.

Conclusion: Nurses should be encouraged to advocate for their patient and take part in the decision-making process. Communication between the medical team and nurses can be improved and there is an obvious need for further improvement in education and collaboration in this area.

Keywords:            Do Not Attempt resuscitation, End-of-Life, Withdrawal, Palliative Care

I. INTRODUCTION

Do-Not-Attempt Resuscitation (DNAR) orders have been used in hospitals worldwide for over the past 30 years with the intention of preventing non-beneficial treatments for hospitalized patients. These orders provide direction for the medical team and often take place after discussion with the patient, their family members and the multidisciplinary team as this decision is often complex with possible legal, ethical and moral implications. Although these decisions are typically initiated by physicians, nurses play a key role in DNAR discussions as well (Gendt et al., 2006).

Nurses are often the first point of contact for a patient and have the responsibility to act as patient advocates (Fitz, Fuld, Haydock & Palmer, 2010). They are in the best position to fulfil a liaison role between physicians and patients; conveying patient preferences to the physicians and influencing acceptance by the patient or surrogate decision-maker of the medical recommendations of DNAR (Gendt et. al, 2006). Many studies have shown that healthcare workers tend to provide less medical care to DNAR patients and inconsistencies around what continuing care should be given to patients exist. Similarly, DNAR orders can also influence the delivery of nursing care. Fitz et. al (2010)’s study showed that a significant number of nurses believed a DNAR order altered nursing observation frequency, pain relief and fluids administered.

Though end-of-life care and patients with DNAR orders can co-exist, there might be a fine line in terms of treatment goals and outcomes. A study done by Stewart and Baldry (2010) found that a DNAR decision is perceived by some as equivalent to withdrawal of active treatment. The aim of the study was to identify nurses’ involvement during the decision-making process, level of understanding of issues surrounding DNAR orders; and how they perceive care for patients with DNAR orders.

II. METHODS

A. Design and Setting

A descriptive crossed sectional study design using electronic questionnaires was adopted for the study. It was conducted at 700-bedded hospital in Singapore where approximately 2500 nurses work full-time.

 B. Sample

Convenience sampling method was adopted for the recruitment of nurses in the study and 400 nurses participated in the study.

C. Instruments Used

The questionnaire was adapted from a previous study with the author’s permission (O’Hanlon, O’Connor, S., Peters & O’Connor, M., 2013) and piloted on 50 nurses working in the ICU and High Dependency Unit (HDU). Thereafter, the questionnaire was further modified and consisted of 24 questions to improve the clarity of some questions and to better address the aims of the study.

 D. Data Collection and Analysis

Ethical approval was obtained from the Institutional Review Board prior to data collection. Data was collected over a period of 5 months (June – October 2016). Consent was assumed if the survey was completed. Responses were collected anonymously through an electronic platform with a secured login. Responses were confidential and descriptive statistics analysis was used to analyse the data.

III. RESULTS

Majority of respondents were 21-30 years old females, with an average work experience of less than 5 years. Most of the participants were Chinese and an equal number of participants had a degree or Masters as their highest qualification. Majority of the participants were staff nurses who work in a medical ward, whilst other responses were collected from nurses working in the ICU, emergency department (ED) and surgical wards.

 A. Involvement in the Decision-Making Process

Nurses thought that the patient (75.5%), family members (83.5%) and physicians (79%) should be involved in DNAR decision-making process as shown in Figure 1. However, only 8% thought that nurses should be involved in this process. A proportion of nurses (13.8%) reported that they had previously disagreed with a DNAR decision, but 63.2% said that they would still comply with the orders even if they did not agree. Majority (57.1%) of the nurses reported that they were not comfortable discussing end-of-life issues with patients and their family members and a sub-group analysis revealed that this was also true for nurses working in specialty areas like the Intensive Care Unit, High Dependency Unit and Emergency Department.

Figure 1. Involvement in the decision-making process (N=400)

B. Perceptions and Understanding

The results in Figure 1 showed that majority (63%) felt that withholding and withdrawing life-sustaining therapy were equivalent. There was no consensus with regards to the survival rate for out-of-hospital cardiac arrest and in-hospital cardiac arrest rates and these were generally overestimated. If a patient had a DNAR order in place, therapies like antibiotics, intravenous fluids, oxygen therapy and nasogastric feeds were deemed to be appropriate. Other therapies like intubation, cardiac compression, defibrillation, surgery, transfer to HD or ICU, dialysis and antiarrhythmic for life threatening cardiac rhythm were thought to be less appropriate. There were no differences observed between junior and senior nurses.

IV. DISCUSSION

To our knowledge, this study is the first to investigate nurses’ involvement during the decision-making process; and how they perceive care for patients with DNAR orders in a Singapore.

The study provided insight to gaps in communication that exist amongst the medical and nursing team; and the central importance of communication between these two teams. From the study, an alarming number of nurses revealed that they were not comfortable discussing end-of-life issues with patients or family members, even if they worked in critical areas.

Unlike Western countries, Singapore does not have a national policy on DNAR or an established framework to guide end-of-life care. Various studies have reported that the distinction between withholding and withdrawing treatment is not always recognised, and an assumption is often made that DNAR equates to withdrawal of active treatment (Stewart & Baldry, 2010). This is also observed in this study as majority of the respondents felt that withholding and withdrawing life-sustaining measures were equivalent. There is often an overlap between the two areas but more importantly, these decisions should be individualised to patients.

Results of the study showed that respondents of this study generally overestimated the survival rate of out-of-hospital cardiac arrest rates, in-hospital survival rates, recovery with good neurological function and survival to discharge. This study also revealed that nurses regarded therapies like IV fluids, antibiotics, oxygen therapy and nasogastric tube feeding as appropriate for patients with a DNAR order. The findings differed from O’Hanlon et. al (2013)’s study that surfaced some confusion over palliation and DNAR orders and only 36% of nurses felt that nasogastric tube feeding was appropriate for patients with DNAR orders. A recent study exploring physician attitudes towards withholding and withdrawal of life-sustaining treatments in end-of-life care saw that most were more ready to withhold and withdraw haemodialysis than enteral feeds (Phua et. al, 2015). This difference could perhaps reflect cultural differences as the Asian culture consider food and nutrition as basic rather than medical care, representing love and hope for health and an expression of filial piety (Phua et. al, 2015).

Being a single-centred questionnaire-based study, there are limitations to be considered. Half of the respondents only had 5 years or less of working experience. Religious beliefs of the participants were also not included in this study and this could potentially contribute to the nurses’ views on end-of-life care. As such, findings should not be generalised to the nursing population in Singapore or Asia and a larger study is recommended to replicate the study findings.

A. Implications for Clinical Practice and Further Research

This study has multiple important implications. Firstly, there is clearly a need for more training and increased confidence to improve nurses’ involvement in the DNAR process. Nursing education has traditionally been focused on nursing diagnoses and the practical aspects of care. However, good communication skills surrounding end-of-life conversations are also required in order to advocate and better care for their patients. Perhaps nursing schools and hospitals should consider developing a structured programme to help improve communication skills surrounding difficult topics like end-of-life care and DNAR orders. Nurses should also be encouraged to be more involved in the DNAR process, encouraging open communication especially when expectations are not aligned with patients or family members and the treatment team.

Further research is recommended with a bigger population of nurses from various hospitals throughout Singapore and even Asia to provide more insight into this topic. Future research should also focus on testing the validity of the tool used in this study since it has not been tested after modification of the questionnaire.

As DNAR policies and issues surrounding withdrawal and withholding of care for these patients can vary from hospital to hospital, a national policy guiding physicians and nurses the matter could be developed to better equip healthcare workers with the necessary information needed to care for patients.

V. CONCLUSION

This study gives first insight into the perceptions and understanding of nurses surrounding DNAR orders and the care rendered for these patients. Being at the forefront, nurses should be encouraged to communicate openly with the medical team and advocate for their patients. In order to do so, structured programs and national policies should be considered so as to guide and support nurses through this process.

Notes on Contributors

Wee Li-Phing Clarice is an Advanced Practice Nurse at Ng Teng Fong General Hospital, Singapore.

Institutional Review Board (IRB)

Standard institutional review board (IRB) procedures were followed and approval was obtained by the author. IRB number 2016/00413.

Acknowledgements

The author would like to thank Dr. Ross Freebarin and Dr. Faheem Khan for their valuable suggestions towards the study design and questions; and Dr. Yanika Kowitlawakul for her help in reviewing the article. In addition, the author would also like to thank Ms Syairah Binte Mohd Reduan and Mr. Johnny Wong for their assistance during the pilot study that contributed towards the final questionnaire used in this study.

Declaration of Interest

The author does not have any declaration of conflict of interest, including financial, consultant, institutional and other relationships that might lead to bias.

References

Fitz, Z., Fuld, J., Haydock, S. & Palmer, C. (2010). Interpretation and intent: A study of the (mis)understanding of DNAR orders in a teaching hospital. Resuscitation, 81(9), 1138-1141. https://doi.org/10.1016/j.resuscitation.2010.05.014.

Gendt, C. D., Bilsen, J., Stichele, R. V., Noortgate, N. V. D., Lambert, M. & Deliens, L. (2007). Nurses’ involvement in ‘do not resuscitate’ decisions on acute elder care wards. Journal of Advanced Nursing, 57(4), 404-409.

O’Hanlon, S., O’Connor, S., Peters, C. & O’Connor, M. (2013). Nurses’ attitudes towards Do Not Attempt Resuscitation orders. Clinical Nursing Studies, 1(1), 43-50.

Phua, J., Joynt, G. M., Nishimura, M., Deng, Y., Myatra, S. N., Chan, Y. H. … Koh, Y. (2015). Withholding and withdrawal of life-sustaining treatments in intensive care units in Asia. JAMA Internal Medicine, 175(3), 363-371. https://doi.org/10.1001/jamainternmed.2014.7386.

Stewart, M. & Baldry, C. (2010). The over-interpretation of DNAR. Clinical Governance: An International Journal, 16(2), 119-128. https://doi.org/10.1108/14777271111124473.

*Wee Li-Phing Clarice
Email: clarice_wee@nuhs.edu.sg

Published online: 4 September, TAPS 2018, 3(3), 31-34
DOI: https://doi.org/10.29060/TAPS.2018-3-3/SC1056

Christie Anna¹ & Lian Dee Ler²

¹National Healthcare Group Polyclinics (NHGP), Singapore; ²Health Outcomes and Medical Education Research (HOMER), National Healthcare Group, Singapore

Abstract

Aims: The evidence on how reflection associates with clinical teaching is lacking. This study explored the reflection pattern of nursing clinical instructor trainees on their clinical teaching and its association with their teaching performance.

Methods: Reflection entries on two teaching sessions and respective teaching assessment data of a cohort of Registered Nurses participating in the National Healthcare Group College Clinical Instructor program (n=28) were retrieved for this study. Reflection entries were subjected to thematic analysis. Each reflection statement was coded and scored according to topics in relevance to three clinical teaching phases – preparation, performance and evaluation. Teaching assessment scores were then used to group the participants into different performance group. Reflection patterns derived from the coding scores were compared across these groups.

Results: Participants’ reflections focused on the performance phase (57% of reflected items), followed by preparation (30%) and evaluation (13%) phases. To assess the reflection pattern of trainees with differing teaching performance, participants whose teaching assessment scores were already high from first teaching session were classified into Consistently High group (score>22). Remaining participants were further categorized based on their improvement in teaching assessment scores into Higher Change (score difference>1) and Lower Change (score difference≤1). Compared to Lower Change group, participants in the Consistently High and Higher Change groups had higher trend of reflection focus on performance (57% and 59% vs 48%) and evaluation phases (14% and 14% vs 8%), but lower on preparation phase (29% and 27% vs 44%).

Conclusions: The finding suggests a possible role of reflection in teaching performance of nurse clinical instructors, warranting further investigation.

Keywords:            Registered Nurses, Clinical Instructor, Reflective Thinking, Clinical Teaching, Reflective Journal

I. INTRODUCTION

The concept of reflection was first articulated by Dewey (Dewey, 1933) and it can be defined as ‘careful consideration and examination of issues of concern related to an experience’ (Kuiper & Pesut, 2004). Since its introduction, reflection has gained a lot of attraction in multiple disciplines and professions, including medicine and nursing. David Kolb, who developed the idea of ‘reflective learning’, is one of the most influential authors who proposed the incorporation of reflection as an educational approach  (Kolb, 2015).  Kolb suggested that learning should be a dialectic and cyclical process, and in his experiential learning model, suggested that experience is the basis of learning, which cannot take place without reflection. Meanwhile, reflection must be connected to action. In line to this conception, Donald Schön emphasized the importance of the ‘reflective practitioner’, describing reflective practice as the practice by which professionals can become aware of their implicit knowledge base and learn from their action or experience (Schön, 1983).

In the clinical setting, the focus of clinical teaching and learning has shifted from doing, to knowing and understanding (Won & Wong, 1987). In order to understand, one must process information or knowledge in a structured manner and reflect or link new experiences and knowledge with past experiences. This allows the learner to be a ‘reflective practitioner’ and learning to be meaningful. In addition, developing learners’ reflective skills enhances coherence between theoretical and practical components of education programmes (Hatlevik, 2012). Using reflection as a means to bridge the gap between theory and practice, and develop and articulate tacit knowledge is a widely used method in nursing literature (Johns & Joiner, 2002). As reflection is increasingly recognized as a crucial component in nursing education and practice (Nguyen, Fernandez, Karsenti, & Charlin, 2014), it was a timely initiative of the National Healthcare Group College (NHGC) to conduct a Clinical Instructor (CI) programme which factored reflection as an important aspect of the curriculum while assessing the association of reflection to nurses’ teaching performance.

The NHGC conducted a CI programme for the training and development of nursing clinical instructors. The development of reflective skills was incorporated as a methodology in the curriculum. The CI programme consisted of a total of sixteen days, of which six days were spent in the classroom, followed by ten days in the clinical setting as clinical practicum. In addition to the ten days clinical practicum period, participants were granted a one-month period, beginning from the last day of the programme to complete all assessment requirements of the programme. Participants of the programme were Registered Nurses (RNs) with a minimum of three years working experience in the clinical setting and a keen interest in the clinical teaching role. The aim of this study was to explore the reflective thinking of these nurse clinical instructors and its association with their clinical teaching performance during placement of the clinical practicum.

II. METHODOLOGY

Twenty-eight RNs were enrolled in the CI program and participated in this study which used quantitative thematic analysis to establish the relationship of reflective thinking and clinical teaching. During the one-month clinical practicum period of the programme, the CI trainees were required to carry out two clinical teaching sessions and log these cases into their Clinical Practicum Logbooks (CPL).

Clinical practicums took place in the same clinical setting where the RNs worked in. Following each clinical teaching session, they were to reflect upon their teaching and document their reflections in the Reflective Journal (RJ). The RJ contained a set of prompt questions to facilitate reflections: 1) what they had learnt from their clinical teaching; 2) what new learnings from classroom sessions they applied to the clinical setting; and 3) any concerns or doubts arose from their clinical teaching experiences. The CPL and RJ were in partial assessment requirements to successfully complete the CI programme.

At the end of the one-month clinical attachment period, both these documents were retrieved from the database for this study. The RJ was first subjected to thematic coding and analysis. The themes were coded deductively according to three clinical teaching phases – preparation, performance and evaluation. Two researchers who were initially blinded to participants’ teaching performances analysed the reflective journal. Each one independently read and coded each statement of the reflection from the first teaching log according to topics in relevance to the three clinical teaching phases. The two researchers’ respective coding results were then compared to ensure inter-coder reliability. Differences in interpretation were resolved through discussion until consensus was reached. To quantify the reflection themes, each reflective statement was coded as one reflected item. The frequency of reflected item for each clinical teaching phase was thus derived. For example, if a participant reflected on the preparation teaching phase, this reflected item was counted towards the reflection frequency for the preparation phase theme.

We also extracted the teaching assessment scores for both teaching sessions from participants’ CPL. Performance scores were then used to group participants according to their teaching performances; how groups were categorised is explained in detail in the results section.  In order to analyse patterns of reflective thinking across these groups, the frequency of reflected item for each clinical teaching phase was converted to percentage over the total reflected items.

III. RESULTS

As suggested by Kolb, reflection is critical to connect experience to learning (Kolb, 2015). We thus analysed participants’ RJ on their first clinical teaching experience and explored their learning for clinical teaching through the change in their clinical teaching assessment scores between the first and second teaching sessions. Generally, participants’ reflections focused more on the performance (mean no. of reflected items = 3.8, 57%) as compared to preparation and evaluation (mean no. of reflected items are 2.0 and 0.9, or 30% and 13%, respectively) phases of clinical teaching (Figure 1). The teaching assessment scores of all participants either remained the same or improved across the two clinical teaching. The participants were classified into two groups according to their teaching assessment score in the first teaching. Fourteen participants whose teaching assessment scores were already high from the first teaching log were in the Consistently High group (score > 22). The remaining participants (score ≤ 22) were further classified based on the difference of teaching assessment scores between first and second teaching into Higher Change (difference > 1) and Lower Change (difference ≤ 1) for those with higher or lower improvements in clinical teaching, respectively. Participants in the Consistently High and Higher Change groups had higher trend of reflection focus on performance (57% and 59% vs 48%) and evaluation phases (14% and 14% vs 8%), but lower trend of reflection focus on preparation phase (29% and 27% vs 44%) when comparing to the Lower Change group.

Figure 1. Percentage breakdown of participants’ reflections focus according to their group category

IV. DISCUSSION

The use of clinical logs and reflective journals in the programme was to allow learners to define their experiences in their own words through reflective writing following a practical experience, and thus to validate their own unique experiences. Kobert (1995) illustrated that clinical logs and reflection allowed the learner to see his/her world within a larger context. In addition, learning shifts from a passive to an active process (Callister, 1993). Reflecting on their practice, the clinical logs and reflective journals provided the learners with tools for developing and improving their metacognitive awareness. The ability to develop and subsequently improve reflective thinking skills begins with the awareness of one’s own thinking patterns (i.e. their metacognition) (Fonteyn & Cahill, 1998).

This pilot study set out to describe patterns of reflective thinking among RNs following experiencing clinical teaching sessions. Results from this study provide preliminary support of the relationship between the reflection on different clinical teaching phases and clinical teaching performance of CI trainees. While all participants’ reflection focused mostly on performance phase, those who attained consistent and improved teaching performance showed a higher trend of reflection focus in the performance and evaluation phases but a lower trend of reflection focus in the preparation phase when compared with other participants. In line with this research effort devoted to identify effective reflective practice education (Asselin & Fain, 2013), the understanding of how reflection associates with clinical teaching can inform curriculum design on how nurse educators can teach or scaffold reflection to enhance clinical instructors’ clinical teaching skills.

The assessment structure of the CPL could be a contributing factor in participants’ reflection focus of the teaching phases. The overall weightage of the CPL teaching assessment was 30% of the CI programme. Other assessment requirements for CI programme included developing a lesson plan based on a microteaching conducting session, class participation and presentations during classroom sessions, and doing an online quiz before commencement of the programme. The RJ entries carried a total weightage of 20%, which meant each reflective entry carried a hefty 10%. The CPL teaching performance of the trainees was assessed by their total performance scores in each of the three teaching phases, with the performance phase carrying the highest scores. The preparation and evaluation phases each carried 5 marks, while the performance phase carried 15 marks. The weightage distribution given to each clinical teaching phase could also have been a factor which contributed to participants’ reflection focus – where they tended to focus on the performance phase as it carried the highest weightage of scores among the three teaching phases.

It should be noted, that study limitations included the use of a small sample, all recruited from a single cohort of the CI programme.  All participants of this cohort of the CI program were included in this study, with no exclusion criteria or eliminating factors. The RNs were of varying nationalities, with differing years of working experience, holding various educational qualifications, and even differing in age and gender. These factors were not considered in this study on their impact or influence towards their clinical teaching ability, depth of reflection potential or the capacity to translate reflective thoughts into writing. Future research, taking these factors into account, will be critical to further establish the relationship of reflective thinking and clinical teaching.

V. CONCLUSION

Different patterns of reflection were associated with clinical teaching in nurse clinical instructors. Those with consistently good teaching performance, when comparing to those with low or no improvement in teaching, have higher trend of reflection focus in the performance and evaluation phases but lower in preparation phase of their teaching. Similar reflection pattern was also observed in those showing high improvement in teaching. The finding thus suggested a possible role of reflection in clinical teaching performance of nurse clinical instructors which warrants further investigation.

Notes on Contributors

Christie Anna is a nurse educator at the National Healthcare Group Polyclinics, Singapore. She is involved in the development of nursing education.

Lian Dee Ler is a research analyst at NHG-HOMER, the research unit within Education Office of National Healthcare Group, Singapore. She is involved in health profession education research.

Ethical Approval

This study does not require ethics approval as determined by the NHG Domain Specific Review Board (DSRB), Reference 2017/00156.

Declaration of Interest

C. A. is an employee of National Healthcare Group Polyclinics and L.D.L. an employee of National Healthcare Group Singapore. The authors have no financial, consultant and other relationships that might lead to competing interests.

Acknowledgements

This pilot research project was supported by the following colleagues who provided their expertise that greatly assisted the research:

Lim Yong Hao, Research Analyst, NHG-HOMER, Group Education & Research;

Eileen So, Executive, NHG College;

Dong Lijuan, Assistant Director of Nursing, Nursing Services, NHG Polyclinics; and

Poh Chee Lien, Assistant Director of Nursing, NHG HQ, Group Education & Research.

Funding

The authors have no financial relationships.

References

Asselin, M. E., & Fain, J. A. (2013). Effect of reflective practice education on self-reflection, insight, and reflective thinking among experienced nurses: a pilot study. Journal for Nurses in Professional Development, 29(3), 111–119.
https://doi.org/10.1097/NND.0b013e318291c0cc.

Callister, L. C. (1993). The use of student journals in nursing education: Making meaning out of clinical experience. Journal of Nursing Education, 32(4), 185–186.

Dewey, J. (1933). How we think: a restatement of the relation of reflective thinking to the educative process. Lexington (Massachusetts): D. C. Heath.

Fonteyn, M. E., & Cahill, M. (1998). The use of clinical logs to improve nursing students’ metacognition: A pilot study. Journal of Advanced Nursing, 28(1), 149–154.

Hatlevik, I. K. R. (2012). The theory‐practice relationship: reflective skills and theoretical knowledge as key factors in bridging the gap between theory and practice in initial nursing education. Journal of Advanced Nursing, 68(4), 868–877.

Johns, C., & Joiner, A. (2002). Guided reflection: Advancing practice. Osney Mead, Oxford: Blackwell Science.

Kobert, L. J. (1995). In our own voice: Journaling as a teaching/learning technique for nurses. Journal of Nursing Education, 34(3), 140–142.

Kolb, D. A. (2015). Experiential Learning: Experience as the Source of Learning and Development. Upper Saddle River, New Jersey: Pearson Education.

Kuiper, R. A., & Pesut, D. J. (2004). Promoting cognitive and metacognitive reflective reasoning skills in nursing practice: Self-regulated learning theory. Journal of Advanced Nursing, 45(4), 381–391.

Nguyen, Q. D., Fernandez, N., Karsenti, T., & Charlin, B. (2014). What is reflection? A conceptual analysis of major definitions and a proposal of a five-component model. Medical Education, 48(12), 1176–1189.
https://doi.org/10.1111/medu.12583.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic books.

Won, J., & Wong, S. (1987). Towards effective clinical teaching in nursing. Journal of Advanced Nursing, 12(4), 505–513. https://doi.org/10.1111/j.1365-2648.1987.tb01360.x.

*Christie Ann
Email: christie_anna@nhgp.com.sg
*Lian Dee Ler
Email: lian_dee_ler@nhg.com.sg

Published online: 2 January, TAPS 2019, 4(1), 55-58
DOI: https://doi.org/10.29060/TAPS.2019-4-1/SC2000

Ikuo Shimizu, Junichiro Mori & Tsuyoshi Tada

Centre for Medical Education and Clinical Training, Shinshu University, Japan

Abstract

Consensus formation among faculties is essential to curriculum reform, especially for the clinical curriculum. However, there is limited evidence on the model of workshop that is necessary for curriculum reform. In the current project, we aimed at developing a beneficial workshop method for building consensus and reaching educational goals in curriculum reform. We compared the two types of workshop models. First, we conducted workshops using standard group work model with fixed group members. Then we used a revised workshop model. In the revised model, all but one group member moved their seats after the first round of discussion. In addition, we reserved time for plenary presentations and discussions after each round. We called the model “Modified World Café” workshop named after World Café, a collaborative dialogue method. With this design, not only we were able to achieve significant improvement of appropriate products and better consensus, but also attained several educative goals. Since the model combines characteristics of the standard group work and World Café concept, it might be useful in facilitating the sharing of new knowledge and creating consensus.

Keywords:            Curriculum Reform, Workshop, Consensus Building, World Café

I. INTRODUCTION

When medical education managers conduct the curriculum reform in medical schools, they must plan for an effective implementation of the new outcomes of the curriculum, especially in the curriculum reform of clinical years when more diverse community clinician-educators at various educational environments are involved (Takamura, Misaki, & Takemura, 2017). Therefore, it is desirable that the curriculum is reformed by taking into accounts of the opinions from stakeholders at multiple specialties and reaching a consensus among them. In medical education, workshops have been performed for such a reform (Steinert, 2014). The standard workshop consists of small group discussions with fixed members throughout the schedule (Figure 1A), however, the effectiveness of this workshop design has not been well explored.

The World Café (Brown & Isaacs, 2005), a series of short group dialogues with table rotations (Figure 1B), is useful for building consensus since it relies on a participatory and collaborative network of conversations (Health Profession Opportunity Grants, 2015). However, one concern to apply World Café for establishing outcomes is that it may not be suitable to establish a new curriculum since it is just a dialogue process rather than a workshop to construct valid products and short discussion time may make discussions superficial and unfruitful.

Therefore, we designed a hybrid program of the standard workshop described above and World Café: the “Modified World Café” (Figure 1C). In this design, participants started the first round by providing a goal within a group to work on. After the first round, they share the discussion in the interim plenary presentation, then all members except one in each group (a “host”) move to a new table and start a new round of discussion. Finally, they return to their original table to summarise their discussions and finalise the product with a final presentation. While “Modified World Café” is similar to World Café in terms of process whereby all members except for one in each group move seats, there are two major differences: (1) each round is longer to discuss more deeply, and (2) participants have to collect opinions twice for interim and final presentations. These revisions are intended to finalise the products. We aimed at (1) evaluating the effectiveness of the “Modified World Café” to mix the various opinions and reach better consensus on the expected outcomes among participants, and (2) exploring the participants’ perceptions.

Note: 1A. The standard workshop (i.e., the model without moving tables),
1B. World Café dialogues (Brown, 2005), and
1C. “Modified World Café” workshop.
Figure 1. Three types of workshop models discussed in this article

II. METHODS

A. Setting

Shinshu University School of Medicine in Japan extended its clinical clerkship program from 15 to 21 months in 2015, by adding a 6-month elective clerkship program. In the new program, students are expected to rotate among the community teaching hospitals to acquire clinical competencies. Because of this reform, we conducted workshops to establish the learning outcomes of the new clinical clerkship program and share the outcomes from the 2014 to 2015 academic year. The workshop participants are clinician-educators of community teaching hospitals who teach medical students in the clinical clerkship program. Every teaching hospital was asked that at least one clinician-educator per hospital participated in one of the series of workshops.

B. Design of workshop

In the workshops, we used the following two workshop models for comparison. Firstly, we conducted two workshops with the standard group work model (Figure 1A), in which the participants engaged in two rounds of small group discussions. The groups consisted of five to seven participants each and were formed according to their clinical specialty, and members were fixed. Each group was asked to establish outcomes of its specialty for the reformed curriculum during the couple of rounds. Faculties of Shinshu University Hospital participated and facilitated the discussions. There were two presentation opportunities: interim and final sharing on what they have discussed.

We, then, conducted three workshops using the “Modified World Café”. The expected product, number of participants, the number and length of rounds, and the presence of facilitators were the same as the standard group work model. As described in the Introduction part (Figure 1C), all members except one in each group (a “host”) move to a new table in the second round. Two presentations (interim and final) were performed. The participants were briefed on process at the beginning of each workshop.

C. Evaluation

To evaluate these workshop models, the participants were asked to complete two anonymous questionnaires. The first one was an open-response evaluation sheet designed for participants to write their own perceptions (e.g. benefits, demerits, suggestions) on the group work in the workshop. The comments were analysed using thematic analysis, which involved iteratively exploring the data and comparing and refining the identified categories. Two team members read the data iteratively and analysed them thematically using the affinity diagram method (KJ method) (Scupin, 1997), identifying, analysing, and reporting themes within the data and developing an interpretive synthesis of the topic.

The second survey used a questionnaire comprising the following items: (1) the degree of perceived usefulness of the products, and (2) the degree of perceived consent of the products, and (3) the degree of active participation in the workshop. The responses were collected with a 5-point Likert scale ranged from 1 for “strongly disagree” to 5 for “strongly agree,” which we analysed statistically with the t-test.

III. RESULTS

A total of 120 educators from 37 hospitals participated in the five workshops (71 at the initial model and 49 at the revised model). Among them, 77.5% had been involved in postgraduate (residency) clinical teaching. The response rate of the questionnaire was 89.4%.

Analysing the open-response sheet in the initial model enabled the identification of four themes: preference for outcomes, usefulness of collaboration, attitude for participation, and need of inter-specialty discussions. From the comments after the Modified World Café, two more themes were emerged in addition to the four themes in the initial model: (1) suggestive opinions about educational views (“How should we address the diverse characteristics of students?” “Near-peer learning with residents can be useful”) and (2) an educational impact (“teaching strategies at the workplace,” “the importance of outcome-based education,” “an overview of legitimate peripheral participation”), all of which are theoretical bases of clinical clerkships.

The rating of the items in the questionnaire on the 5-point scale revealed that the mean degree of the usefulness of the products significantly improved from 3.77 to 4.00 (p = 0.017). The mean degree of perceived consent of the products also significantly improved from 3.61 to 4.05 (p = 0.001). The degree of active participation in the workshop did not change (4.05 and 3.95; p = 0.256).

IV. CONCLUSION

Our “Modified World Café” workshop shows not only improved the perceived usefulness and built better consensus but also yielded educational achievement from the analysis of open-response sheet. It is probably because of a benefit of World Café, which can visualize and share collective intelligence (Brown & Isaacs, 2005). By hybridizing the characteristics of the standard group work and World Café, our “Modified World Café” workshop can be a more suitable design. The degree of active participation did not change probably because the participants did not join in the workshop voluntarily. Better achievements with the same degree of active participation will still indicate a merit of “Modified World Café”.

On the other hand, there are some limitations. Firstly, since the participants attended only one workshop, our results might not be from any experimental design. Also, only three items were used in the questionnaire, although we observed significant improvement in some perceptions.

Since workshop tends to be dependent on its context (Steinert, 2014), the educational innovation in its design has been hindered. With this regard, our “Modified World Café” workshop may be applicable in other contexts which require highly collective processes for consensus creation, such as patient safety and interprofessional management.

Notes on Contributors

Ikuo Shimizu MD, MHPE, is an assistant professor at the Centre for Medical Education and Clinical Training in Shinshu University. He is also involved in the safety management office at Shinshu University Hospital.

Junichiro Mori, MD, PhD, is a lecturer at the Centre for Medical Education and Clinical Training in Shinshu University.

Tsuyoshi Tada, MD, PhD, is a professor at the Centre for Medical Education and Clinical Training in Shinshu University.

Ethical Approval

This work was approved by the Institutional Research Board at Shinshu University and carried out in accordance with the Declaration of Helsinki; in particular, anonymity was guaranteed and consented for analysis was obtained.

Acknowledgements

We thank all the other faculty members at the Centre for Medical Education and Clinical Training and the Shinshu University Hospital who contributed to the series of this workshop.

Funding

No funding is involved for this paper.

Declaration of Interest

The authors report no declarations of interest.

References

Brown, J., & Isaacs, D. (2005). The World Café: Shaping our futures through conversations that matter. San Francisco, CA: Berrett-Koehler Publishers.

Health Profession Opportunity Grants (2015). Using the World Café to Improve Instructor Engagement: A Guide for Health Profession Opportunity Grants Programs. Retrieved 1 March 2018, from https://www.acf.hhs.gov/ofa/resource/using-the-world-cafe-to-improve-instructor-engagement-a-guide-for-health-profession-opportunity-grants-programs.

Scupin, R. (1997). The KJ method: A technique for analysing data derived from Japanese ethnology. Human organisation, 56(2), 233-237.

Steinert, Y. (2014). Faculty development: Future directions. In Faculty Development in the Health Professions. Springer Netherlands.

Takamura, A., Misaki, H., & Takemura, Y. (2017). Community and interns’ perspectives on community – Participatory medical education: From passive to active participation. Family Medicine, 49(7), 507–513.

*Ikuo Shimizu, MD, MHPE
Safety Management Office
Shinshu University Hospital
3-1-1 Asahi, Matsumoto 3908621, Japan
Tel: +81 263 37 3359
Email: ishimizu@shinshu-u.ac.jp

Published online: 2 January, TAPS 2019, 4(1), 51-54
DOI: https://doi.org/10.29060/TAPS.2019-4-1/SC1052

Jill Cheng Sim Lee¹, Muhammad Fairuz Abdul Rahman¹, Weng Yan Ho¹, Mor Jack Ng², Kok Hian Tan^1,2 & Bernard Su Min Chern^1,2

¹Division of Obstetrics and Gynaecology, KK Women’s and Children’s Hospital, Singapore; ²SingHealth Duke-NUS Obstetrics and Gynaecology (OBGYN) Academic Clinical Program, SingHealth Duke-NUS Academic Medical Centre, Singapore

Abstract

Background: Residency selection panels commonly use time consuming manual voting processes which are easily subjected to bias and influence of others to select successful candidates. We explored the use of an electronic audience response system (ARS) or ‘clickers’ in obstetrics and gynaecology resident selection; studying the voting process and examiner feedback on confidentiality and efficiency.

Methods: All 10 interviewers were provided with clickers to vote for each of the 25 candidates at the end of the residency selection interview. Votes were cast using a 5-point Likert scale. The number of clickers provided to each interviewer was weighted according to the rank of the interviewer. Voting scores and time for each candidate was recorded by the ARS and interviewers completed a questionnaire evaluating their experience of using clickers for resident selection.

Results: The 10 successful candidates scored a mean of 4.28 (SD 0.27, range 3.86-4.73), compared to 2.99 (SD 0.71, 1.50–3.79) for the 15 unsuccessful candidates (p<0.001). Average voting time was 26 seconds per candidate. Total voting time for all candidates was 650 seconds. All interviewers favoured the use of clickers, for its confidentiality, instantaneous results, and more discerning graduated response.

Conclusion: Clickers provide a rapid and anonymous method of collating interviewer decisions following a rigorous selection process. It was well-received by interviewers and highly recommended for use by other residencies in their selection process.

Keywords:            Resident Selection, Clickers, Electronic Audience Response System

PRACTICE HIGHLIGHTS

• Clickers provide a time-efficient way of collating resident selection interview outcomes following a rigorous structured selection process.
• It is important that individual interviewers are able to select successful candidates anonymously to reduce risk of bias from external influences as use of multiple observers rather than single interviewers improves reliability of the resident selection interview.
• Numerical ratings using clickers provide objective and transparent data easily available should an inquiry arise about the resident selection process.
• Clickers are increasingly becoming standard educational tools within the medical classroom. Faculty and resident familiarity with audience response systems allows development of creative extensions of clickers beyond the classroom context.

I. INTRODUCTION

There is increasing evidence that radiofrequency electronic Audience Response System (ARS) or “clickers” are useful in undergraduate and postgraduate medical education (Caldwell, 2007). ARS instantly collects real-time data through hand-held keypads and graphs participant responses. Each clicker unit has a unique signal allowing answers from each assessor to be identified and recorded.

Clickers bridge the communication gap between speaker and audience, and is used to assess understanding, engage attention, enhance learner enjoyment and interaction, improve knowledge retention and encourage clinical reasoning and problem solving. The positive uptake, feedback and experience of clickers by the medical education community has led to its introduction in innovative new areas, providing solutions to many problems within medical education. Outside medicine, clickers have been used for research data collection, enhanced social norms marketing campaigns and surveying vulnerable, low literacy groups.

The SingHealth Obstetrics and Gynaecology (OBGYN) Residency Program recently explored clickers as a way to improve efficiency and efficacy in the residency selection process. Traditional selection interviews typically concluded with a voting process where members of the selection panel raised hands to decide on the best candidates. This was time-consuming and individuals’ voting decisions could be swayed by openly visible votes of other panel members. Furthermore, hand-raising only allowed binary responses. This paper describes the OBGYN resident selection process using clickers and studies interviewer feedback on its confidentiality and efficiency. To our knowledge, no literature exists on the use of ARS during recruitment interviews.

II. METHODS

The SingHealth OBGYN residency selection process was conducted over two interview sessions assessing 25 candidates for selection for the academic year of 2015. Prior to this, candidates were shortlisted from an annual national specialty training application process open to final year medical students, house officers and medical officers. Following review of applications, portfolios, medical school grades and letters of support, candidates participated in a national level multiple mini interview prior to undergoing selection at individual sponsoring institutions.

The SingHealth selection panel comprised ten interviewers; the Program Director (PD), two Associate Program Directors (APDs), Academic Chair, four core faculty and two chief residents. The selection format comprised a round robin three-station interview; a large panel interview with the PD, Academic Chair, an APD, a core faculty member and chief resident; a small panel interview with an APD and one core faculty; and a less formal ‘bull pen’ interview with two core faculty and a chief resident. Candidates were interviewed alone in the large panel, where they were asked about self-appraisal and reactions to residency and healthcare industry. In the small panel, they were interviewed about goals, ambitions and work experience. Candidates awaiting panel interviews were interviewed about personal information and life questions in a group in the ‘bull pen’. Interviewers convened at the end to discuss candidate performances and review the multisource feedback obtained from within SingHealth OBGYN. Interviewers then scored each candidate using clickers.

All interviewers were given clickers to vote for each candidate. The ARS in this study was the Classroom Performance Systems Pulse, utilising the INTERWRITERESPONSE® 6.0 software. The number of clickers given to each interviewer was weighted, with the PD receiving three and APDs and Academic Chair receiving two each. Other interviewers each received one. This weightage policy was decided by the program in recognition of leadership and experience. Program administrators allocated numbered clickers to each interviewer. Interviewers were blinded regarding which clickers were allocated to other interviewers.

Candidates’ names and photographs would be shown on screen (Figure 1) and interviewers were asked to vote using a 5-point Likert scale, with 1 being “Highly Not Recommended”, 2, “Not Recommended”, 3, “Neutral”, 4, “Recommended” and 5, “Highly Recommended”. The ARS instantly processed and displayed results to interviewers, enabling immediate visualisation of scores and ranking of candidates. Time taken for interviewers to vote for each candidate was recorded as part of the ARS. No limit was imposed to the voting time taken by interviewers to decide on each candidate. The total time taken for voting each candidate was measured as the time from which ARS voting was activated for the candidate until all clicker responses were recorded. Interviewers were not allowed to abstain from voting. The 10 candidates with the best mean score were admitted into the SingHealth OBGYN Residency Program.

All interviewers were asked the following questions via email a week later:

1. Did you like the anonymity of the response system?
2. Did you like the graduated response allowed by the clickers instead of binary “Yes” or “No” responses?
3. Did you like the instantaneous results generated by the clicker software?

Microsoft Excel 2010 was used to analyse differences in scores between successful and unsuccessful candidates using t-test. Data on time and questionnaire responses were analysed using descriptive statistics.

This study was undertaken as part of a larger study to understand residency selection interviews in OBGYN. SingHealth Centralised Institutional Review Board exempted this study from further review.

Figure 1: Example of voting display screen

III. RESULTS

Ten interviewers participated in the selection process of which a total of 15 clickers were utilised in accordance with the weightage described above.

The mean score for all 25 candidates was 3.51 (SD 0.86, 1.50-4.73), with a combined score of 87.75. The mean score for the 10 successful candidates was 4.28 (SD 0.27, range 3.86-4.73), compared to 2.99 (SD 0.71, 1.50–3.79) for the 15 unsuccessful candidates (p<0.001). The total time taken to vote for 25 candidates was 650 seconds with a mean of 26 seconds per candidate. All interviewers recorded decisions within two minutes.

All (100%) ten interviewers returned the questionnaire and answered “Yes” to the three questions, favouring the confidentiality, instantaneous results, and graduated response provided by ARS.

IV. DISCUSSION

Clickers are increasingly used in medical education and familiar to most college faculty members (Lewin, Vinson, Stetzer, & Smith, 2016). The ARS in this study is a shared system used by all SingHealth OBGYN faculty and residents. The familiarity of the interviewers with clickers made it a rapid and accurate way for administrators to process votes following the interview. The total time taken for all interviewers to record votes for all 25 candidates was under 11 minutes. Average cost for the recruitment of one postgraduate year 1 position was US$9899, of which 96% were attributed to efforts, and therefore time, contributed by the faculty, chief residents, and administrative staff (Brummond et al., 2013). Time-saving strategies are crucial to reduce the growing costs of resident recruitment. No studies were available studying time efficiency following implementation of clickers apart from one that reported no difference in time spent lecturing between clicker and non-clicker classes in the University-level science, technology, engineering and mathematics setting (Lewin et al., 2016). However, our use of clickers did not involve lecturing and further studies are needed to compare the time difference between traditional and clicker voting methods in resident selection.

The interviewers liked the anonymity provided by clickers. Anonymity to vulnerable groups through clickers has been echoed in non-medical literature (Keifer, Reyes, Liebman, & Juarez-Carrillo, 2014). In the selection interview context, it reduces uncontrolled swaying of votes by dominant individuals causing severe bias to voting results. We believe clickers allow investigators to control the influence of each assessor with the use of predetermined and prior agreed weighted votes through the allocation of greater numbers of clickers to residency leaders in recognition of their greater experience. In other situations, equal weightage for each voter may be more appropriate and can be manipulated to suit needs. Immediate tabulation of results further adds transparency. Should an inquiry arise about the resident selection process, data can easily be reviewed through the program administrator as to decisions recorded by each interviewer.

Interviewers liked the instant response provided by clickers. This mirrors feedback of other ARS users such as a graduate student population study which reported the primary benefit of clickers related to providing immediate feedback (Benson, Szucs, & Taylor, 2016).

This study has limitations. This single cohort study utilised a single ARS which may not reflect practices of other programs or ARS platforms. However, many currently available ARS platforms share common functions. We did not compare the time taken to record the decisions of interviewers during interview sessions which did not utilise clickers nor did we evaluate reasons for delay in decision-making time beyond the mean. Further study of these factors may identify individual and system-based problems such as interviewers’ variation in familiarity with clickers and coping with multiple keypads.

V. CONCLUSION

In summary, study of interviewer feedback suggests that clickers enhance residency selection by providing a rapid and anonymous method of collating interviewer decisions following a rigorous selection process. The introduction of clickers to the selection process was well-received by all interviewers in this study and highly recommended for similar use by other residencies.

This simple, novel extension to the use of clickers beyond the classroom illustrates how we can extend the use of facilities already available within our medical institutions to improve existing systems. Such attitudes need to continue to be encouraged within healthcare services.

Research is currently in progress to study the multi-station interview process and its correlation with success during residency.

Notes on Contributors

Dr. Jill Cheng Sim Lee is a senior resident in OBGYN at SingHealth and former Chief Resident for Education within her department. She has a Master of Science in Clinical Education and is involved in undergraduate medical education at Lee Kong Chian School of Medicine, Nanyang Technological University.

Dr. Muhammad Fairuz Abdul Rahman is a 4^th year resident in OBGYN at SingHealth.

Dr. Weng Yan Ho is a senior resident in OBGYN at SingHealth and former Chief Resident for Administration within her department.

Mor Jack Ng is the Manager of the SingHealth-Duke-National University of Singapore (NUS) OBGYN Academic Clinical Program (ACP).

Prof. Kok Hian Tan is Senior Associate Dean of Academic Medicine at Duke-NUS, Group Director of Academic Medicine at SingHealth and Head of Perinatal Audit and Epidemiology at KK Women’s and Children’s Hospital (KKH). He is also editor-in-chief of the Singapore Journal of Obstetrics and Gynaecology.

A/Prof. Bernard Su Min Chern is Chairman of Division of OBGYN and Head and Senior Consultant of both the Department of OBGYN and Minimally Invasive Surgery Unit at KKH. He is also Chairman of the SingHealth-Duke-NUS OBGYN ACP and was formerly Program Director of the SingHealth OBGYN Residency Program.

Ethical Approval

This study was undertaken as part of a larger study to understand residency selection interviews in OBGYN. SingHealth Centralised Institutional Review Board approved this study with an exempt status.

Acknowledgements

The authors would like to acknowledge the support and cooperation provided by the faculty and staff at the SingHealth OBGYN Residency Program during the period of this study.

Funding

Funding for this study was borne internally by SingHealth OBGYN Residency Program. No external funding sources were required.

Declaration of Interest

All authors have no potential conflicts of interest.

References

Benson, J. D., Szucs, K. A., & Taylor, M. (2016). Student Response Systems and Learning: Perceptions of the Student. Occupational Therapy In Health Care, 30(4), 406–414. https://doi.org/10.1080/07380577.2016.1222644.

Brummond, A., Sefcik, S., Halvorsen, A. J., Chaudhry, S., Arora, V., Adams, M., … Reed, D. A. (2013). Resident Recruitment Costs: A National Survey of Internal Medicine Program Directors. The American Journal of Medicine, 126(7), 646–653. https://doi.org/10.1016/j.amjmed.2013.03.018.

Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE-Life Sciences Education, 6(1), 9–20.

Keifer, M. C., Reyes, I., Liebman, A. K., & Juarez-Carrillo, P. (2014). The use of audience response system technology with limited-english-proficiency, low-literacy, and vulnerable populations. Journal of Agromedicine, 19(1), 44–52.

Lewin, J. D., Vinson, E. L., Stetzer, M. R., & Smith, M. K. (2016). A Campus-Wide Investigation of Clicker Implementation: The Status of Peer Discussion in STEM Classes. Cell Biology Education, 15(1), ar6,1-ar6,12. https://doi.org/10.1187/cbe.15-10-0224.

*Dr Jill C. S. Lee
Email: jill.lee.c.s.@singhealth.com.sg
Division of Obstetrics and Gynaecology,
KK Women’s and Children’s Hospital,
100 Bukit Timah Road, Singapore 229899
Tel: +65 6225 5554

Published online: 2 January, TAPS 2019, 4(1), 48-50
DOI: https://doi.org/10.29060/TAPS.2019-4-1/SC1067

Nattabborn Bunplook¹, Mechita Kongphiromchuen¹, Priyapat Phatinawin¹ & Panadda Rojpibulstit²

¹Medical Education Centre Buddhasothorn Hospital, Faculty of Medicine, Thammasat University, Thailand; ²Department of Biochemistry, Preclinical Science Institute, Faculty of Medicine, Thammasat University, Thailand

Abstract

Aim: To encourage first year medical students to have better health by drinking more water and being role models in health promoting lifestyle.

Methods: The campaign was launched in February 2016 with seven main activities, convinced via social networking apps (Line and Facebook). The pre and post questionnaires were launched via Google Forms regarding the effective activities and participants’ drinking behaviour. Data were analysed by descriptive statistics and dependent t-test.

Results: Response rate for the questionnaire was 74.71% (127/170). After 4 weeks of the campaign, the average water intake was significantly increased (t = 6.359, p < 0.05). The average consumption of sugar-sweetened beverages was significantly decreased (t = -8.256, p < 0.05). Additionally, when comparing from the seven activities, the most top three preference activities are ‘Info & suggestions’, ‘Sugar Hunt’ and ‘Sugar Free Be Rich’.

Conclusions: Nowadays, people are preferring to consume sugary drinks instead of regular water and many may assume that computer games and social media are prevalent issues. But in reality if we use technology wisely, it would even provide us with a much more efficient method to reach the target group. Our study, thereby, then brings upon the appropriate aspect of games and social media in order to resolve the current issues. However, some activities have to be improve to attract more of the target group and the duration of the campaign should be increased for better long-term behavioural change.

Keywords:            Health Promotion, Medical Students, Obesity, Lifestyle

I. INTRODUCTION

Global prevalence of non-communicable diseases (NCDs) is prominently increasing worldwide and kills more than 36 million people each year (World Health Organization, 2016). The 4 main types of non-communicable diseases are cardiovascular diseases, cancers, chronic pulmonary diseases and diabetes. These diseases are caused as a direct result of life style and environmental factors that include genetics, ageing, poverty and globalisation of unhealthy lifestyle. For example, poverty is a barrier to accessing health service and globalisation of unhealthy life style like physical inactivity or expose to harmful products.

Up to this present, changing of the world leads to unhealthy lifestyles like physical inactivity, smoking, alcohol abuse and eating unhealthy diets. Eating and drinking behaviour have been changed. Changing of eating behaviour are overeating, eating high calories or fast food. In addition, when considering on drinking behaviour, a variety of beverages which are easily accessible, delicious and inexpensive causes the preference of sugar-sweetened beverages (SSB) over pure water. Drinking a much more level of these SSB increases the risk of overweight, obesity, and diabetes (Al-Qahtani, 2016; Shah et al., 2014).

Despite the fact that the medical students are the upcoming doctors who should be the role models of health promotion, many studies have found that medical students have unhealthy dietary habits especially drinking SSB (Frank, Carrera, Elon, & Hertzberg, 2006).

Therefore, the campaign “You are what you drink” was launched in order to encourage the medical students to consume water instead of sweetened beverages. The promising outcome from these promoted activities is to drive them to have better health and additionally help them to be role models in health promoting lifestyle.

II. METHODS

A purposive sampling method was used to recruit 170 of first year medical students, Thammasat University, to participate the interventional study which designed by 7 main activities as shown in the details below. In brief, there were (1) Plant Nanny, (2) Info & suggestions, (3) Infused water, (4) Drunk in love, (5) Sugar Free be rich, (6) Drink for… and (7) Sugar hunt. All of these activities were launched via social networking apps (Line and Facebook) so these activities are directly broadcast to the target group.

Additionally, before starting and after finishing the interventional period, the participants were assigned to complete pre and post self-administered questionnaires were launched via Google Forms. The online questionnaires consist of 5 items regarding the SSB consumption and the effective activities that could improve the beverage consumption. These questionnaires were used to collect the participants’ drinking behaviour data which were analysed by descriptive statistics and dependent t-test.

Importantly, to attract the participants’ attention, prizes was offered in each activity. The students who mark their water intake every day will have a chance to win the special prizes at the end of the project.

III. RESULTS

The number of participated first year medical students was 127 from 170 (response rate of 74.71%). 36 of the participants are male and 91 are female. They have mean age of 17.62+0.64 year old. After 4 weeks of the campaign, the average water intake was significantly higher than the beginning (t = 6.359, p < 0.05). Additionally, the average consumption of SSB was statistically significant decreased (t = -8.256, p < 0.05). Moreover, the most preference activities based on number of votes were also verified, there were ‘Info & suggestions’, ‘Sugar Hunt’ and ‘Sugar Free Be Rich’.

Figure 1. The 7 main activities to encourage medical student (left) and
the percentage of the preference activities based on number of votes (right)

IV. DISCUSSION

From the results of the campaign, we found that the participants’ average water intake was increased and the average consumption of SSB was decreased. Importantly, the most preference activities were ‘Info & suggestions’, ‘Sugar Hunt’ and ‘Sugar Free Be Rich’.

First of all, ‘Info & suggestions’, it might be because the participants received the information and suggested tips for drinking more water and reducing SSB consumption, so this activity is one of the most popular activities in our campaign.

Second, ‘Sugar Hunt’, this activity is one of the most preference activities due to its incentive of winning the competition.

Third, ‘Sugar Free Be Rich’, a reason of the activity’s popularity might be its ability to raise participants’ awareness of amount of money wasted on SSB.

However, some activities were not suitable for them such as ‘Infused water’. It might be because finding fruits is hard work for them. In addition, ‘Drunk in Love’ during the Valentine’s week and ‘Drink for…’ did not improve the drinking behaviour because its influence was in the short duration; therefore, they could not create enough impact.

V. CONCLUSION

In the current era of the fast changing world, people are preferring to consume SSB instead of regular water and many may assume that computer games and social media are prevalent issue. But in reality if we use technology wisely, it would even provide us with a much more efficient method to reach the target group. Our study, thereby, then brings upon the appropriate aspect of games and social media in order to resolve the current issues and improve the quality of life. The issues are not only being the high rate of SSB consumption, but the collective social matter as a whole.

Notes on Contributors

Nattabhorn Bunplook, Mechita Kongphiromcheu, Priyapat Phatinawin are medical students from Collaborative Project to Increase Production of Rural Doctor (CPIRD) at Medical Education Centre Buddhasothorn Hospital. All of them contributed equally in the conception, experimental design and implementation of the study.

Panadda Rojpibulstit is an Associate Professor in Biochemistry and a former Vice Dean of Student Affairs and Health Promotion of the Faculty of Medicine, Thammasat University. She contributed in drafting and revision the manuscript.

All authors read and approved the final manuscript.

Ethical Approval

This study was approved by the Human Ethics committee of Thammasat University No. I (Faculty of Medicine) (IRB No. 097/2560).

Funding

There was no funding support.

Acknowledgements

We would like to appreciate all of the participants; 1st year medical students in Thammasat University (MEDTU26). We would like to acknowledge the Faculty of Medicine, Thammasat University and the Foundation for Medicine and Public Health, CPIRD Bhuddhasothorn Thailand for the traveling grants to join with 14th APMEC conference, 2017.

Declaration of Interest

The authors declare that they have no competing interests.

References

Al-Qahtani, M. H. (2016). Dietary Habits of Saudi Medical Students at University of Dammam. International Journal of Health Sciences, 10(3), 353-362.

Frank, E., Carrera, J. S., Elon, L., & Hertzberg, V. S. (2006). Basic demographics, health practices, and health status of U.S. medical students. American Journal of Preventive Medicine, 31(6), 499–505. https://doi.org/10.1016/j.amepre.2006.08.009.

Shah, T., Purohit, G., Nair, S. P., Patel, B., Rawal, Y. & Shah, R. M. (2014). Assessment of Obesity, Overweight and Its Association with the Fast Food Consumption in Medical Students. Journal of Clinical and Diagnostic Research, 8(5), CC05-CC07. https://doi.org/10.7860/JCDR/2014/7908.4351.

World Health Organization. (2016). Global Report on Diabetes. Retrieved from http://apps.who.int/iris/bitstream/10665/204871/1/9789241565257_eng.pdf.

*Panadda Rojpibulstit
Biochemistry Department, Preclinical Sciences Institutes,
Faculty of Medicine, Thammasat University,
Pathumthani, Thailand, 12121
Tel: +66 9269710
Email: panadda@tu.ac.th

Published online: 7 May, TAPS 2019, 4(2), 52-57
DOI: https://doi.org/10.29060/TAPS.2019-4-2/SC2035

Marianne Meng Ann Ong¹ & Sandy Cook²

¹Department of Restorative Dentistry, National Dental Centre, Singapore; ²Academic Medicine Education Institute, Duke-NUS Medical School, Singapore

Abstract

Aim: To describe residents’ expectations of faculty using the One-Minute Preceptor (OMP) in microskills and their ratings of faculty performing them during clinical sessions.

Methods: Prior to the start of residency, residents were invited to participate in a survey on residents’ expectations of faculty performing the OMP microskills in clinical teaching activities using a 4-point Likert scale. At the end of Year 1, they rated faculty on their use of the OMP microskills using a 4-point Likert scale using a second survey.

Results: Sixteen Year 1 residents completed the first survey and 15 residents completed the second survey. Prior to residency, correcting mistakes with feedback was the highest rated microskill (3.78) and obtaining a commitment was the lowest rated (3.15). At the end of Year 1, residents rated faculty performing getting a commitment as the highest (3.63) and giving feedback the lowest (3.17).

Conclusion: In this small cohort of residents, expectations were high around the OMP microskills. Residents felt faculty performed them well in their first year of residency. However, residents view of the importance of the five OMP microskills by faculty differed from their perception of how well the faculty demonstrated them. Future studies could explore if residents’ perceptions of importance changed over time or were related to their view on the quality of performance by faculty. Faculty will be further encouraged to employ the five OMP microskills to maximise their teaching moments with residents managing patients in busy outpatient clinics in National Dental Centre Singapore.

Keywords:            Learner Perception, Expectation, Evaluation, Clinical Teaching

I. INTRODUCTION

Some challenges faced by learners and faculty in clinical teaching include work demands, time constraints, multiple levels of learners and lack of active participation of learners (Spencer, 2003). The use of the One Minute Preceptor (OMP) in microskills in clinical teaching has been shown to be easy to learn and effective in helping faculty improve their teaching (Furney et al., 2001) in addition to maximising their teaching moments with learners in busy outpatient clinics (Neher, Gordon, Meyer, & Stevens, 1992). The OMP framework consists of five microskills: obtaining a commitment (OAC); probing for supporting evidence (PSE); teaching general rules (TGR); providing positive feedback (PPF); and correcting mistakes with feedback (CMF). Depending on the situation, each microskill can be used either on its own or in any sequence to cater to different learning contexts. Earlier studies done in National Dental Centre Singapore (NDCS) assessed whether OMP in microskills faculty development workshops had an impact on in-flight residents’ perceptions of clinical teaching (Ong, Yow, Tan, & Compton, 2017) and obtained information on past residents’ perceptions on the importance and frequency of the OMP microskills in residency clinics (Ong, Woo, & Cook, 2016). However, we found no studies in the literature that explored incoming dental residents’ expectations of faculty using the OMP microskills during clinical sessions in their residency programmes, nor their perceptions of faculty performing them in their first year of the programme. This descriptive study thus describes Year 1 residents’ expectations and perceptions of clinical teaching activities performed by faculty.

II. METHODS

A. Study Design

This is a descriptive study on residents’ expectations and perceptions of clinical teaching activities performed by faculty. The protocol was sent to SingHealth CIRB (Ref: 2018/2137) and it was deemed exempt from review.

B. Subjects

During the Academic Year (AY) 2015 orientation session in June 2015, prior to the start of residency programmes, all sixteen residents in five dental specialities (Endodontics, Oral and Maxillofacial Surgery, Orthodontics, Periodontics and Prosthodontics) were invited to participate in this survey.

C. Surveys

Two surveys were developed to explore Year 1 residents’ expectations and perceptions of clinical teaching activities in relation to the OMP microskills. The first survey (PRE) (refer Appendix A) had 14-items grouped by the microskills: OAC (3 items), PSE (3 items), TGR (3 items), PPF (3 items) and CMF (2 items). Residents rated the importance they placed on faculty engaging in teaching activities related to them on a 4-point Likert scale (1= not important to 4= very important).

The second survey (POST) (refer Appendix B), administered at the end of their first year (June 2016), had 13-items that were also grouped by the microskills: OAC (4 items), PSE (3 items), TGR (2 items), PPF (2 items) and CMF (2 items). While similar in the microskills classification as the PRE survey, these items were focused on how well faculty performed them on a 4-point Likert scale (1= inadequate to 4= excellent) and included slightly different phrasing. Both survey items were reviewed by 2 residents in Year 2 of their residency programmes for relevance, ease of use, language and clarity of items.

D. Administration of Surveys

At the AY 2015 orientation session, paper copies of the PRE survey were handed out to residents after the overview briefing session of NDCS residency training programmes by the Director of Education. At the end of Year 1, during the annual programme evaluation session, paper copies of the POST survey were handed out to residents to fill in by an Academic Clinical Programme (ACP) office executive.

E. Data Analysis

The PRE and POST survey data were collated by an executive in the ACP office. De-identified data were sent to the staff of Duke-NUS Office of Education to provide basic descriptive statistics. Individual items in each microskill category for both surveys were averaged to get a general sense of the residents’ views by category. No further statistical analysis was done as different rating scales were used and there was slight variation in the phrasing of items in both surveys.

III. RESULTS

Sixteen Year 1 residents (100% response rate) completed the PRE with overall mean ratings of importance ranging from 3.15 (OAC) to 3.78 (CMF) for the five microskills (Figure 1). Fifteen residents (93.75% response rate) completed the POST (1 resident dropped out of a residency programme before the end of Year 1) with overall mean ratings of quality ranging from 3.17 (PPF) to 3.63 (OAC) (Figure 1).

 Note: While the broad categories are similar, rating scales and individual items that were averaged by microskills category were different on each survey. The PRE scale was 1=not important to 4=very important. The POST scale was 1=inadequate to 4=excellent).
Figure 1. Overall mean ratings on the importance of OMP activities (PRE) and mean ratings on quality of faculty performing OMP activities (POST)

IV. DISCUSSION

In this study, clinical teaching was defined as performance of teaching activities related to OMP microskills. Residents in this cohort rated faculty CMF (PRE: 3.78) as the element of most importance of the five OMP microskills prior to the start of residency. Yet, it was not one of the OMP microskills rated as done well (POST: 3.23) as the rest. This is an area NDCS faculty can improve upon. The OMP microskill rated as best performed by faculty was OAC (POST: 3.63), was actually the least important to residents (PRE: 3.15). But overall residents perceived faculty performing the five OMP microskills well (>3.0) in clinical teaching.

Earlier data obtained from past learners in the residency programmes from a 2015 general survey done in NDCS had revealed they valued the use of the OMP microskills in clinical teaching. The two microskills with the highest mean ratings on importance were OAC and CMF (Ong et al., 2016). The frequently demonstrated microskills in that survey were OAC and TGR. In contrast, a survey done with in-flight residents in 2014 exploring the short-term follow up after an OMP in microskills workshop for faculty, had revealed TGR as the most adequately performed microskill by faculty (Ong et al., 2017). The differences in perceptions in OMP microskills in clinical teaching by faculty can be attributed to the different cohorts of learners surveyed at different periods of time. It is not unexpected that learners at various stages of their residency would have different expectations and perceptions of the five microskills in clinical sessions.

Limitations of this study include its small sample size, residents responded to survey items based on their own subjective expectation of faculty demonstrating the OMP microskills in clinical teaching and their own subjective assessment of how well faculty performed them during their first year of residency in the clinics. The nature of the study, different rating scales and variation in the phrasing of the survey items limited our ability to determine any change in perceptions on importance and quality in the OMP microskills.

V. CONCLUSION

In this small cohort of residents, expectations were high around certain of the OMP microskills. Residents felt the faculty performed them well. However, residents view of the importance of the five OMP microskills by faculty differed from their perception of how well the faculty demonstrated them. Future studies could explore if residents’ perceptions of importance changed over time or were related to their view on the quality of performance by faculty. Faculty will be further encouraged to employ the five OMP microskills in clinical teaching to maximise their teaching moments with residents managing patients in busy outpatient clinics in NDCS.

Notes on Contributors

Dr Marianne M. A. Ong, Cert Perio (Michigan), MS (Michigan); Adjunct Associate Professor, Duke-NUS Medical School, Singapore; Senior Consultant & Director, Education National Dental Centre Singapore.

Dr Sandy Cook, PhD; Senior Associate Dean, Deputy Head of Office of Education; Deputy Director, Academic Medicine Education Institute; Professor, Duke-NUS Medical School.

Ethical Approval

The protocol was sent to SingHealth CIRB (Ref: 2018/2137) and was deemed not requiring a formal review as it reports on residents’ expectations and perceptions of clinical teaching activities performed by faculty and satisfaction with their first-year experience.

Acknowledgements

The authors would like to give special thanks to Ms Kia-Mun Woo from the MD Programme Department for her help in the data analysis.

Funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

Declaration of Interest

The authors have no conflict of interest.

References

Furney, S. L., Orsini, A. N., Orsetti, K. E., Stern, D. T., Gruppen, L. D., & Irby, D. M. (2001). Teaching the one-minute preceptor: A randomized controlled trial. Journal of General Internal Medicine, 16(9), 620-624.
https://doi.org/10.1046/j.1525-1497.2001.016009620.x

Neher, J. O., Gordon, K. C., Meyer, B., & Stevens, N. (1992). A five-step “microskills” model of clinical teaching. Journal of the American Board of Family Practice, 5(4), 419-424. Retrieved from https://www.jabfm.org/content/5/4/419

Ong, M. M., Woo, K. M., & Cook, S. (2016). A general survey on learners and faculty perspectives on educational activities held in the National Dental Centre, Singapore. Proceedings of Singapore Healthcare. 25(3), 158-168.
https://doi.org/10.1177/2010105816642246

Ong, M. M., Yow, M., Tan, J., & Compton, S. (2017). Perceived effectiveness of one-minute preceptor in microskills by residents in dental residency training at National Dental Centre Singapore. Proceedings of Singapore Healthcare, 26(1), 35-41. https://doi.org/10.1177/2010105816666294

Spencer, J. (2003). Learning and teaching in the clinical environment. British Medical Journal, 326(7389), 591-594. https://doi.org/10.1136/bmj.326.7389.591

*Marianne Ong
Department of Restorative Dentistry
5 Second Hospital Avenue, Singapore 168938
National Dental Centre Singapore
Tel: +65 63248925
E-mail: marianne.ong.m.a@singhealth.com.sg

Published online: 7 May, TAPS 2019, 4(2), 48-51
DOI: https://doi.org/10.29060/TAPS.2019-4-2/SC2070

Zhun Wei Mok, Jill Cheng Sim Lee & Manisha Mathur

Division of Obstetrics and Gynaecology, KK Women’s and Children’s Hospital, Singapore

Abstract

Introduction: At KK Women’s and Children’s Hospital’s (KKWCH) Department of Obstetrics and Gynaecology (O&G), a junior doctor’s handbook exists to guide safe practice. A challenge remains in ensuring relevant, current, and readily accessible content. The onus of re-editing is left to senior clinicians with heavy clinical and supervisory roles, leading to a lack of sustainability. Mobile applications (apps) can be a sustainable ‘just-in-time’ learning resource for junior doctors as they balance new responsibilities with relative inexperience.

Methods: The app was developed in-house with the Residency’s EduTech Office. A focus group comprising junior doctors identified content deemed useful. The alpha version was launched in August 2017 and trialled amongst the wider junior doctor population. Data on usefulness were collected through serial focus groups and analysed using grounded theory.

Results: An online survey disseminated to all 100 junior doctors showed that 100% owned a smartphone. 97.1% supported this new resource. Consultative discussions recommended inclusion of (i) Procedural and consent information; (ii) Risk calculators; and (iii) Clinical pathways and management algorithms. Mobile learning apps entreat the user to immediately reflect and conceptualise their concrete experiences, and actively experiment with the content to build on his/her current knowledge. Learners become stakeholders in creating their own learning material. Qualitative feedback indicated a continued interest to contribute, underscoring the app’s sustainability potential.

Conclusions: Apps can be a sustainable on-the-go resource developed by junior doctors, for junior doctors. Learners become stakeholders in creating their own learning material through continued reflection, conceptualisation and active experimentation. This can be scaled for wider clinical use.

Keywords:            Sustainable Mobile Learning, Mobile Applications, On-the-Go Resource, Junior Doctors, Obstetrics and Gynaecology

I. INTRODUCTION

Mobile technology is an integral part of an ongoing technological revolution within medical practice. Junior doctors are often the first-line medical staff in contact with patients, tasked to provide adequate patient counselling and uphold standards of care in the ambulatory and inpatient settings while negotiating a fresh learning curve. Harnessing this technology in the form of mobile apps serves as an on-the-go resource when needed, and supports the new doctors’ preparation for patient encounters as they balance newly increased responsibilities with relative inexperience (Bullock et al., 2015; Guze, 2015).

At the KK Women’s and Children’s Hospital (KKWCH) Department of Obstetrics and Gynaecology (O&G), whilst a physical handbook of departmental guidelines currently exists to guide safe practice, there remains a challenge in ensuring sustainability on three levels: relevance to existing knowledge gaps, currency with new evidence, and accessibility on demand. In addition, the onus of re-editing was often left to senior clinicians, who had to undertake this portfolio in addition to heavy clinical and supervisory roles, leading to a challenge in sustainability. Given the widespread use of smartphones and apps in the workplace, harnessing this technology may change the learning ecology and transform this into a sustainable on-the-go resource to access much-needed knowledge amongst junior doctors.

Thus, a pilot project aimed at developing a sustainable mobile app for junior doctors for easy consultation of key knowledge to support their clinical practice was mooted. Central to this was the basis that the understanding of common O&G procedures, their risks and complications, access to department-specific guidelines, algorithms and risk calculators will ideally translate into better counselling of patients and ensure standards of care. This is especially so now that the newly updated Singapore Medical Council (SMC) Ethical Code and Ethical Guidelines 2016 has identified the provision of informed consent to be a cornerstone of every new doctor’s practice.

Following the proof-of-concept of this tool, this will be the first step towards making learning more accessible outside the conventional pedagogical setting, and be responsive to the juniors’ developing knowledge. Learners can be engaged to take a stake in creating and sustaining their own learning material.

II. METHODS

An online survey on the use of an app to replace the existing handbook was first disseminated electronically to all junior O&G doctors (House Officer, Medical Officers, Residents and Registrars) in KKWCH, which aimed to assess the feasibility of this new tool, and the receptivity towards it. An invitation was simultaneously extended to interested junior doctors to form a focus group to identify initial content deemed best able to fill existing knowledge gaps.

Preliminary discussions were then undertaken with the Programme Director, Academic Chair, SingHealth Designated Institutional Officer and Faculty. The app was developed in-house with the SingHealth Residency’s EduTech Office with funding from the Academic Clinical Program (ACP). The focus group were asked to identify content deemed most useful for themselves and their peers. The alpha version was launched in August 2017 and was trialled amongst the wider junior doctor audience. Data on usefulness and weaknesses were collected through serial focus groups and analysed using grounded theory. Pre- and post-implementation surveys will be conducted to determine the usefulness of the app in daily clinical practice, with a view for broadening the scope of content and scalability to other departments.

III. RESULTS

In September 2016, 100 KKWCH O&G junior doctors were invited to participate in an online survey, with a response rate of 70%. It showed that 100% of respondents currently own a smartphone, had experience of using apps and 98.6% use their smartphones at work. Of these, 77.1% use an iOS platform and the remaining 22.9% use an Android platform. An overwhelming majority (97.1%) supported the use of a mobile app for teaching of common O&G topics/procedures. 13 respondents agreed to participate in the working focus group to develop this resource.

Results of the pilot survey lent credence to the conceptualisation of a mobile learning app to meet the needs of the user population.

Following the launch of the alpha version in August 2017, serial 3-monthly focus groups were recruited to trial the alpha version for iterative improvements. This was not restricted to the original focus group members, and any junior doctor with suggestions could attend. Between 6 and 14 participants were present at each meeting, with a mix of recurrent and new focus group attendees. This represented an ongoing interest in the continued development of the app, and in future, may reflect the self-sustaining capability of the app with a constant inflow and outflow of junior personnel who will be able to access this knowledge base and contribute towards its expansion.

Topics of commonest reference amongst O&G junior doctors were identified and these key areas were recommended for inclusion:

i. Procedural and consent information
ii. Risk calculators for ambulatory counselling
iii. Clinical pathways and algorithms for inpatient and ambulatory management

Practice guidelines unique to the department were deemed most pertinent, as this obviated the need for internet referencing, especially in filtering content that was not specific to their needs. These recommendations were incorporated into the beta version. Clinical content for the app was prepared by junior doctors from the focus groups. Technical changes were made to improve the ease and intuitiveness of use. Beta-testing is currently underway with ongoing qualitative data collection as part of post-implementation tracking.

IV. DISCUSSION

The development of this mobile app ties in with the theory of a learning ecology, where there is a diverse variety of learning options, allowing each clinician to access and learn according to their own immediate needs (Rashid-Doubell, Mohamed, Elmusharaf, & O’Neill, 2016). The use of technology has been noted as a key element of learning by the US-based Accreditation Council for Continuing Medical Education (ACCME). In their 2015 report, it has been estimated that physicians spend an estimated 993 hours of instruction on internet searching and learning, which may be better served by the use of a dedicated mobile app serving their area of practice. The provision of evidence-based information in a mobile app is in alignment with the Best Evidence Medical Education (BEME) Collaboration, in the development of evidence-based education in the medical and health professions. To date, leading medical institutions such as Johns Hopkins Medicine have put in place mobile apps such as the ‘Hopkins Guides’ – a medical mobile app with monthly updates, covering clinically relevant topics such as antibiotic stewardship, HIV, Diabetes and Psychiatry. The development of this mobile app is based upon previous successful use of technology in these hospitals.

Based on preliminary focus group findings, harnessing this form of mobile technology can be well utilised as a ‘just-in-time’ information resource in daily clinical practice, particularly when other sources are not available. This provides timely access to key facts and encourages learning in context.

While this should not be seen as a replacement for conventional pedagogical learning, mobile learning will complement the existing learning platforms through textbooks and e-learning portals in existence which may not be easily accessible on the ward or at the bedside.

This use of mobile technology is affiliated with Kolb’s learning cycle (Bullock & Webb, 2015; Kolb, 1984) where the app is an adjunct to experiential and active learning. Mobile learning apps provide on-the go knowledge which allows the user to immediately reflect and conceptualise their concrete clinical experiences and then through active experimentation, using the content of the mobile app, come up with a decision and solution that he/she uses to build on his/her current knowledge.

In addition to its benefits on learning, this project allows learners to become stakeholders in creating their own learning material that is responsive to their knowledge gaps. Common themes extracted from qualitative feedback at the focus groups included the app being ‘current and readily available’, and the idea of taking ownership of their learning. The app can also overcome the issue of sustainability. Many have indicated interest in continuing to contribute and encouraging their peers to contribute as it is seen to be ‘for the greater good especially for incoming juniors’. The consistent mix of new and experienced attendees at the 3-monthly focus groups underscores the potential of the app’s sustainability, which is driven by the very users of the app.

Upon completion of Beta testing, the uptake of this app is expected to be seamless in view of the almost ubiquitous use of mobile and smartphones in the clinician population, with easy access to the internet and medical ‘apps’. Its user rate and utility can be further assessed, and thus made scalable for use in other departments within our sponsoring institution. Continued funding for this project has been promised by the ACP.

V. CONCLUSION

App development can provide a sustainable on-the-go knowledge resource developed by junior doctors, for junior doctors. Following the proof-of-concept through the alpha version and focus group data, its use as an up-to-date knowledge resource tailored to the individual needs of junior clinicians is justified. Learners can be engaged to take a stake in creation of their own learning material through reflection, conceptualisation and active experimentation, and ensure its sustainability. This resource can be easily replicated and provide benefit to wider areas of clinical care.

Notes on Contributors

Z. W. Mok reviewed the literature, and co-wrote the article with J. C. S. Lee; M. Mathur reviewed the article. All authors approved the final version.

Acknowledgements

The authors acknowledge the funding support, inputs from Faculty and support from the junior clinicians in their valuable feedback towards app improvement.

The authors would also like to acknowledge the technical assistance and expertise provided by A. D. H. Lu, I. C. Y. Goh, C. H. C. Tan, W. S. Tey, and M. J. Ng from the SingHealth Residency, Education Technology (EduTech), Centre for Resident and Faculty Development (CRAFD) in developing and improving the app.

Ethical Approval

A waiver of ethical approval was obtained from the Centralised Institutional Review Board at SingHealth Research.

Funding

Funding support was obtained from the Academic Clinical Program (ACP) to engage web development personnel.

Declaration of Interest

There was no conflict of interest in this paper.

References

Bullock, A., Dimond, R., Webb, K., Lovatt, J., Hardyman, W., & Stacey, M. (2015). How a mobile app supports the learning and practice of newly qualified doctors in the UK: An intervention study. BioMed Central Medical Education, 15, 71. https://doi.org/10.1186/s12909-015-0356-8

Bullock, A., & Webb, K. (2015). Technology in postgraduate medical education: A dynamic influence on learning? Postgraduate Medical Journal, 91(1081), 646-650. https://doi.org/10.1136/postgradmedj-2014-132809

Guze, P. A. (2015). Using technology to meet the challenges of medical education. Transactions of the American Clinical and Climatological Association, 126, 260-270.

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice‐Hall.

Rashid-Doubell, F., Mohamed, S., Elmusharaf, K., & O’Neill, C. S. (2016). A balancing act: A phenomenological exploration of medical students’ experiences of using mobile devices in the clinical setting. British Medical Journal Open, 6(5), e011896. https://doi.org/10.1136/bmjopen-2016-011896

*Zhun Wei Mok
100 Bukit Timah Road,
Singapore 229899
KK Women’s and Children’s Hospital
Tel: 6225 5554