Submitted: 16 December 2022
Accepted: 25 June 2024
Published online: 1 October, TAPS 2024, 9(4), 6-13
https://doi.org/10.29060/TAPS.2024-9-4/OA2928

Pooja Sachdeva & Derrick Chen-Wee Aw

Department of General Medicine, Sengkang General Hospital, Singapore

Abstract

Introduction: Case write-ups and reflective journals have been used as assessment tools of learning in clinical rotations in Yong Loo Lin school of medicine. It is timely to review the current process of conducting these assessments and effectiveness as an assessment tool. This study aims to understand faculty outlook towards these assessments.

Methods: This is a study that involves a survey-based questionnaire with both closed and open-ended questions, sent out to faculty marking the students’ assignments. This survey was anonymous & voluntary and was disseminated by administrative assistants. The purpose of this survey was to collect the feedback from faculty about current process with intentions of improving the effectiveness of these assessments. The suggestions for improvement were incorporated in the survey and faculty was invited to comment over these suggestions and provide further suggestions if any.

Results: Fifty-two responses from faculty were collected and analysed. Ninety percent of respondents thinks that this is an effective tool to assess and promote self-directed learning. Qualitative feedback was received about need of improvement in a) alignment of the submissions timings with rotation postings b) marking rubric to incorporate factors such as case complexity and weightage to different components of case write-ups, c) timely feedback to students, and d) follow up on action plans.

Conclusion: Case write-ups and reflective journals are still effective learning and assessment tools. They promote self-directed learning and clinical analysis in students. Feedback and action plans are the backbone of these assessments and optimal utilisation of these is recommended.

Keywords:           Undergraduate Medical Education, Case Write-ups, Medical Assessments, Reflective Journals

Practice Highlights

• Case write-ups promote critical analysis & clinical judgement and reflection develops metacognition.
• Students should be guided and encouraged to choose cases to promote self-directed learning.
• Marking rubrics need revising and faculty development on how to utilise them.
• Timing of submission needs to be improved to facilitate feedback and follow-up.
• Direct and timely feedback to students and follow up on actions plans improve utility.

Table 1. Comparison of Mini CEX and case write-ups as assessment tools

*For direct feedback, assessor needs to have follow up communication with the student.

**it can be made objective with descriptors provided in the, marking template for each domain that is being assessed.

Evidence has shown that case write-ups do provide assessors the ability to understand students’ learning and analytical skill (McLeod, 1989) and unlike Mini CEX, it involves reflections by students that deepen learning and memory (Fortson & Sisk, 2007). Similarly, Bjerkvik & Hilli (2019) emphasised that reflective journals promote deeper understanding, critical analysis, metacognition and promotes self-development. Onishi (2008) noted that case presentations during clinical rotations promote the assessor’s understanding of student learning which is consistent with our faculty response as most of our faculty (n=47, 90%) agree that case write-ups are important tools to assess learning. However, since these are not observed assessments at the bedside and require submission, there is no face-to-face contact with the student to give instant feedback and discuss action plans, if any. The discussion of feedback and action plan requires separate communication such as a meeting or phone calls or emails between the assessor and student. If appropriate feedback is not provided or communicated, it is a lost opportunity for improvement in students’ learning.

Results from our survey have highlighted a few areas that needed the attention in optimal utilisation of these assessments. First is, the choice of cases, either by faculty or by students is not uniform. The case selection by students promotes self-directed learning. Presently, students are given a list of cases that may help them select one, but the enlisted cases may not be encountered during their rotation. In such situations, faculty suggesting the choice of cases can provide directions to students. Lee et al. (2010) demonstrated that students who were encouraged to choose their cases based on their knowledge gaps, learning strategies, and study time, were more inclined towards self-directed learning. Self-directed learning has been a cornerstone of adult learning, and it provides learners autonomy and control over their learning and prepares them for lifelong self-management outside educational institutions (Goldman, 2009; Lee et al., 2010). Understandably, students’ choice of cases is influenced by the curriculum, tutorials, and objectives of a particular rotation. Case write-ups and Reflective Journals in our context included both inpatient and outpatient encounters hence providing the choice for both acutely sick vs stable chronic patients. Since the students were encouraged to choose their cases for these assessments on their own, it provided them the opportunity to meet their personal goals and learning deficits. However, if asked for guidance, slightly more than half the faculty (56%, n=29) would assign the cases themselves, and of note, such selection of cases, in various forms of frequency and complexity, did affect the marking by faculty by a considerable amount (about 30 to 45%). Nonetheless, the reflective journals involve writing about one’s learning evolution about a case from preset knowledge to acquired knowledge after encounter. Hence, both these written assignments, case write-ups and reflective journals on any encounter tend to improve learning by making students do literature search and individualise this current evidence-based management search in context of the chosen patient. It still serves the purpose of learning, although personal selection of patients encourages students to work on their own interest, at their own pace and promotes deeper understanding tailored to one’s own pre-existing gaps or deficiencies in knowledge. The authors think that faculty assigning cases will inadvertently compromise the extent of self-directed learning to a significant degree and adjusting the marking based on the frequency and complexity is a qualitative component that warrants further investigation. We recommend the school generate a simple set of guidelines to help students to make an informed and wise selection of suitable cases for their written assignments. Focus group discussions with tutors who regularly mark students on such assignments may provide useful directions in the guidelines’ construction.

Results of this survey also raised the inconsistency in faculty use of marking rubrics. McLeod (1989) raised the major concerns about the variability of criteria and standards of evaluation of these written assessments.

Kogan & Shea (2003) addressed these concerns and assessed the evaluation of write-ups against a pre-set evaluation form that increased the validity and reliability of scoring these write ups. Peggy (2014) also highlighted the use of standardised scoring rubric for case write-ups to reduce the interrater variability and improve the reliability of these assessments. Hence, the scoring rubrics must have descriptors for faculty to mark the students against their performances and it not only makes the assessment fair, but also contributes to individualised and appropriate feedback for students for further improvement in respective domains (Cyr et al., 2014; Kogan & Shea, 2005). Thus, the author recommends that universities or schools must emphasise on faculty training and thereby its use in marking these assignments.

This also brought about the feedback, provided by faculty in this survey about components of this rubric. Reflective journals and case reports assess similar yet different components of learning. While case reports accounts more for critical analysis, clinical judgement, evidence-based management for a particular patient, the reflective journals assess the student’s ability to assimilate an encounter with new acquired knowledge and reflect on their individual learning and growth (Sandars, 2009). Hence, rubric should be tailored to these assessments’ subcomponents accordingly. A small fraction of faculty (12%, n=6) in our survey responded that the discussion and analysis of information by students influenced their marking of the write-up as it provided them invaluable insight into students’ clinical reasoning. In line with this, half of the faculty (53%, n=27) recommended that analytic skills be ranked higher in weightage as compared to data collection on patient history and examination. Hence, marking rubric should emphasise more on clinical judgement and critical analysis in case write-ups than components such as history taking and examination as latter can be assessed in detail with other assessments such as Mini CEX or OSCE examinations. At the same time, complexity of the case and atypical presentation must also account for separate marks to encourage students for choosing challenging and difficult cases. Similarly, for reflective journals rubrics must have weightage on self-reflection cycle, changes in attitudes and perception and how the encounters have changed one’s learning and future practice. The role of rubric in standardisation of these written assessments is paramount as former provides a structure of written submissions for students and reliable and valid scoring tool for faculty.

Results of our survey also highlighted pertinent inadequacy in these assessments that is inability to provide the face-to-face feedback to the students in timely fashion, contributed by timing of submissions of these assessments towards the end of rotation. There is ample literature to support that feedback is a backbone of any formative assessment (Clynes & Raftery, 2008; Nicol & Macfarlane‐Dick, 2006) as it promotes self-regulation of training and highlights the discrepancies in the trainee’s current vs expected learning outcome. Hence like every other assessment, the templates of case write-ups are imbued with spaces for feedback and action plans which the majority of faculty (60%, n=31) have personally communicated to the students. Face-to-face feedback has a higher impact on performance improvement than written indirectly communicated or no feedback as the former provides two-way engagement, however, this impact depends upon the supervisor’s training and content & organisation of feedback (Johnson et al., 2020; Pelgrim et al., 2012). The final stage of providing any feedback as per Pendleton’s rule (Pendleton, 1984). Pendleton’s rule is an agreement between the learner and assessor for a joint action plan for improvement. Presently, there is no timeline for students to submit these written assignments to their core supervisor, hence if these are submitted towards the end of the rotation, the opportunity for face-to-face feedback and discussion on action plans is underutilised. Hence, it would be worthwhile to align the submission with weeks of rotation so that timely and personal feedback can be provided and agreed action plans can be followed within the rotation.

This study highlighted that though an action plan was agreed upon, it was not actively followed up with students 90% (n=47) of the time – as such, the accountability of this assessment is reduced. The most common reason for the inability to follow up on action plans was coordination (having to schedule a meeting between the assessor and student when the student may have moved on to the next rotation). This can be modified if students are instructed to submit their assignments at least one or two weeks before the end of posting to allow ample time for both parties to schedule a meet-up. Alternatively, there could be an end-of-posting mandatory meet-up with the clinical supervisor to provide overall feedback for rotation and to discuss action plans. Additionally, the school could also mandate a follow-up meeting, over an interactive online platform if a face-to-face meeting is not feasible, for a supervisor to review the outcomes of the actions undertaken by the student. The school may even consider recruiting student mentors to follow up instead.

Overall, this survey has provided useful insight into these assessments’ conduct and has highlighted the factors that limit the utility of these written assessments. With the faculty agreeing that these assessments are still worthy educational and learning assessment tools, there is a need to improve marking standardisation of these assessments and like other assessments, formative feedback to students on gaps in knowledge must be provided. These assessments have been part of curriculum for decades and their role in students learning must be utilised to its full potential.

There are limitations in our study such as lower number of respondents and qualitative feedback. As survey is voluntary and anonymous, it is limited in its research capability for recommendations and qualitative feedback as latter is respondent dependent. Hence, further qualitative research such as focussed group discussions is required to understand the ways, these assessments can be utilised to their full potential as learning and assessment tools.

VI. CONCLUSION

In conclusion, written assignments are still worthy and useful tools to assess the learning of students during clinical rotations. It promotes self-directed learning by allowing students to select their case and provides the opportunity for the assessor to assess the knowledge gaps of students about case management. Since the case choice affects the marking by the assessor, authors recommend that students are given free reign, within a set of recommended guidelines.

Marking and scoring should be adjusted to include variables such as case selection and complexity in the marking rubric provided to the assessors, hence authors also recommend updating marking rubrics in consultation with faculty, with subsequent faculty development for compulsory use of this rubric.

For these assessments to be more effective, structured, timely and direct feedback should be given to students with action plans that must be followed. The hurdles in following up on action plans such as change of rotations can be dealt with by adjusting the timing of submission of these assessments during a posting and creating opportunities for follow-up. Hence, authors also recommend face-to-face feedback by ensuring adequate timing of assessments and appropriate follow up for action plans to maximise educational improvement opportunities.

Notes on Contributors

Dr. Pooja Sachdeva has contributed to the conceptual development of this study, survey questionnaire development, dissemination of the survey to faculty, data collection, and analysis. This manuscript has been written, read, and finally approved by her.

Dr. Derrick Aw has contributed to the conceptual development of this study, survey questionnaire development, and student and faculty engagement. This manuscript was read, edited, and finally approved by him.

Ethical Approval

The study was approved by the Singhealth Institutional Review Board (IRB) with reference no 2020/2688.

Data Availability

The data that supports the findings of this study are openly available in Figshare repository at https://doi.org/10.6084/m9.figshare.24471661.v1  (Sachdeva & Aw, 2023).

Acknowledgement

We acknowledge the contributions of Dr Shweta Rajkumar Singh for editing the final manuscript.

Funding

There is no funding received for the study.

Declaration of Interest

There are no conflicts of interest.

References

Bjerkvik, L. K., & Hilli, Y. (2019). Reflective writing in undergraduate clinical nursing education: A literature review. Nurse Education in Practice, 35, 32–41. https://doi.org/10.1016/j.nepr.2018.11.013

Boyd, E. M., & Fales, A. W. (1983). Reflective learning: Key to learning from experience. Journal of Humanistic Psychology, 23(2), 99–117. https://doi.org/10.1177/0022167883232011

Bussard, M. E. (2015). Clinical judgment in reflective journals of prelicensure nursing students. The Journal of Nursing Education, 54(1), 36–40. https://doi.org/10.3928/01484834-20141224-05 

Clynes, M., & Raftery, S. (2008). Feedback: An essential element of student learning in clinical practice. Nurse Education in Practice, 8(6), 405–411. https://doi.org/10.1016/j.nepr.2008.02.003

Cyr, P. R., Smith, K. A., Broyles, I. L., & Holt, C. T. (2014). Developing, evaluating and validating a scoring rubric for written case reports. International Journal of Medical Education, 5, 18–23. https://doi.org/10.5116/ijme.52c6.d7ef

Fortson, A. (n.d.). Reflective journaling as assessment and teaching. http://www.reap.ac.uk

Gibbs, G. (1998). Learning by Doing: A guide to teaching and learning methods. Oxford Brookes University.

Goldman, S. (2009). The educational kanban: Promoting effective self-directed adult learning in medical education. Academic Medicine, 84(7), 927–934. https://doi.org/10.1097/ACM.0b013e3181a8177b

Johnson, C. E., Weerasuria, M. P., & Keating, J. L. (2020). Effect of face-to-face verbal feedback compared with no or alternative feedback on the objective workplace task performance of health professionals: A systematic review and meta-analysis. BMJ Open, 10(3), e030672. https://doi.org/10.1136/bmjopen-2019-030672

Joshi, M., Singh, T., & Badyal, D. (2017). Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. Journal of Postgraduate Medicine, 63(2), 100–105. https://doi.org/10.4103/0022-3859.201411

Kogan, J. R., Bellini, L. M., & Shea, J. A. (2002). Implementation of the mini-CEX to evaluate medical students’ clinical skills. Academic Medicine, 77(11), 1156–1157. https://doi.org/10.1097/00001888-200211000-00021

Kogan, J. R., & Shea, J. A. (2003). An assessment measure to evaluate case write-ups in a medicine core clerkship. Medical Education, 37(11), 1035–1036. https://doi.org/10.1046/j.1365-2923.2003.01660.x

Kogan, J. R., & Shea, J. A. (2005). Psychometric characteristics of a write-up assessment form in a medicine core clerkship. Teaching and Learning in Medicine, 17(2), 101–106. https://doi.org/10.1207/s15328015tlm1702_2

Kunin, M., Julliard, K. N., & Rodriguez, T. E. (2014). Comparing face-to-face, synchronous, and asynchronous learning: Postgraduate dental resident preferences. Journal of Dental Education, 78(6), 856–866.

Larsen, D. P., London, D. A., & Emke, A. R. (2016). Using reflection to influence practice: Student perceptions of daily reflection in clinical education. Perspectives on Medical Education, 5(5), 285–291. https://doi.org/10.1007/s40037-016-0293-1

Lee, Y.-M., Mann, K. V., & Frank, B. W. (2010). What drives students’ self-directed learning in a hybrid PBL curriculum. Advances in Health Sciences Education: Theory and Practice, 15(3), 425–437. https://doi.org/10.1007/s10459-009-9210-2

McGlade, K., Cargo, C., Fogarty, D., Boohan, M., & McMullin, M. (2012). Handwritten undergraduate case reports. The Clinical Teacher, 9(2), 112–118. https://doi.org/10.1111/j.1743-498X.2011.00494.x

McLeod, P. J. (1987). Faculty assessments of case reports of medical students. Journal of Medical Education, 62(8), 673–677. https://doi.org/10.1097/00001888-198708000-00008

McLeod, P. J. (1989). Assessing the value of student case write-ups and write-up evaluations. Academic Medicine, 64(5), 273–274. https://doi.org/10.1097/00001888-198905000-00016

Mello, L. V., & Wattret, G. (2021). Developing transferable skills through embedding reflection in the science curriculum. Biophysical Reviews, 13(6), 897–903. https://doi.org/10.1007/s12551-021-00852-3

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63.

Mortaz Hejri, S., Jalili, M., Masoomi, R., Shirazi, M., Nedjat, S., & Norcini, J. (2020). The utility of mini-clinical evaluation exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59. Medical Teacher, 42(2), 125–142. https://doi.org/10.1080/0142159X.2019.1652732

Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090

Onishi, H. (2008). The role of case presentation for teaching and learning activities. The Kaohsiung Journal of Medical Sciences, 24(7), 356–360. https://doi.org/10.1016/s1607-551x(08)70132-3

Patrício, M. F., Julião, M., Fareleira, F., & Carneiro, A. V. (2013). Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Medical Teacher, 35(6), 503–514. https://doi.org/10.3109/0142159X.2013.774330

Pelgrim, E. A. M., Kramer, A. W. M., Mokkink, H. G. A., & van der Vleuten, C. P. M. (2012). The process of feedback in workplace-based assessment: Organisation, delivery, continuity. Medical Education, 46(6), 604–612. https://doi.org/10.1111/j.1365-2923.2012.04266.x

Pendleton, D. (1984). The consultation: An approach to learning and teaching. Oxford University Press.

Sachdeva, P. & Aw D. C. W. (2023). Case write-ups survey responses [Dataset]. Figshare. https://doi.org/10.6084/m9.figshare.24471661.v1  

Sandars, J. (2009). The use of reflection in medical education: AMEE Guide No. 44. Medical Teacher, 31(8), 685–695. https://doi.org/10.1080/01421590903050374

Schon, D. A. (1984). The reflective practitioner: How professionals think in action. Basic Books.

Stufflebeam, D. (2002). The CIPP model for evaluation. In The International Handbook of Educational Evaluation, 49, 279–317. https://doi.org/10.1007/0-306-47559-6_16

*Pooja Sachdeva
110 Sengkang East Way,
Singapore 544886
96170342
Email address: pooja.sachdeva@singhealth.com.sg

Submitted: 3 July 2023
Accepted: 18 June 2024
Published online: 1 October, TAPS 2024, 9(4), 61-64
https://doi.org/10.29060/TAPS.2024-9-4/SC3071

Victoria Scudamore, Sze Yi Beh, Adam Foster & Michaela Goodson

School of Medicine, Newcastle University Medicine Malaysia, Malaysia

Abstract

Introduction: This study compares online and in-person delivery of a weekly clinical reasoning seminar for fourth-year medical students at a Malaysian medical school. During the easing of COVID-19 pandemic restrictions, the initial eight seminars took place online, followed by eleven in-person seminars. This study looks at student preference for online or in-person delivery and how these reasons differ due to gender.

Methods: An online questionnaire was sent to fourth-year medical students after returning to in-person seminars. The response rate was 60/128 (46.88%) and the data was analysed using SPSS software.

Results: 65% of students preferred in-person seminars and a larger proportion of female students (71.43%) preferred in-person sessions compared to male students (50.00%), although this was not statistically significant (p=0.11). A significantly larger proportion of female students preferred in-person seminars for the following reasons compared to male students: enjoyment (p=0.041), developing history-taking skills (55.56%) and for formulating differential diagnoses (p=0.046). Students were asked whether online or in-person seminars were most appropriate for eighteen reasons, they felt in-person seminars were most appropriate for 16/18 of these reasons.

Conclusions: More students preferred in-person clinical reasoning seminars and a higher proportion of these students were female. A significantly larger proportion of female students felt in-person seminars were better for; enjoyment and for developing history-taking skills and formulating differential diagnoses, compared to male students. The students preferred online seminars for home comforts and ease of travel, but they preferred in-person seminars for the other 16/18 reasons listed including all reasons linked to learning skills and interreacting with others.

Keywords:            Online Teaching, In-person Teaching, Gender, Clinical Reasoning, Medical Students

I. INTRODUCTION

Fourth-year medical students at Newcastle Medical School Malaysia attend weekly clinical reasoning seminars as part of their ‘Clinical Decision Making’ (CDM) module. Each week of CDM covers a different medical speciality and students attend a seminar where the group works through a presentation with patient cases and they discuss how to diagnose, investigate and manage the patient. The sessions are attended by eleven students and the groups remain the same throughout the year. In 2021-22 the initial eight seminars took place online using Zoom video conferencing software and as COVID-19 restrictions eased in Malaysia the final eleven sessions took place in-person.

The academic performance of students undertaking online and in-person clinical reasoning seminars has been researched and third-year medical student academic performance was comparable in both settings (Babenko et al., 2022). However, there is currently no research regarding medical student preference for online or in-person delivery of clinical reasoning seminars. Medical student preference for online or in-person delivery of all parts of the curriculum has been analysed and second-year medical students at a US medical school had a preference for online lectures and there was a correlation between these students and those who felt online lectures reduced stress (Altaf et al., 2022).

A cohort study analysed participation in a teaching programme for US graduate physicians and this showed female students asked and answered less questions during in-person sessions compared to online sessions (Cromer et al., 2022). The results of this study are contrasting with my observations whereby female students participated less in online seminars and their participation increased when seminars returned to an in-person setting. This could be due to differences in the research environments or due to the group of observed students being small with less statistical significance.

My first research question was to understand medical student preference and reason for preference of online or in-person delivery of clinical reasoning seminars. My second research question was to establish if student preference differed due to gender and why.

II. METHODS

The data was collected using survey methodology with a self-developed questionnaire made using Microsoft forms. The questionnaire was emailed to all fourth-year medical students after they had experienced both session deliveries. Students were provided with a consent form and informed the research project was optional and were asked to provide voluntary consent before participating. Participants were informed they could withdraw from the project at any time up until the data was anonymised during data collection.

The survey response rate was 60/128 (46.88%), the low participation numbers are likely due to the data collection being optional and undertaken in the students own time. This could lead to a nonresponse bias, as it is likely the more engaged students participated and students with less motivation who did not participate may have responded differently. The data was analysed using SPSS software. Chi-squared tests were used to cross-tabulate the results and to calculate p-values to indicate data with statistical significance.

III. RESULTS

Overall 65% of students preferred in-person seminars and 71.43% (30/42) of female students preferred in-person sessions compared to 50.00% (9/18) of male students (p=0.11). The students were asked if they felt online or in-person seminars were best for eighteen different reasons (see table 1). There were three statistically significant reasons female students preferred in-person sessions more than male students (p<0.05). These were Enjoyment (p=0.041), developing history-taking skills (p=0.011) and formulating differential diagnoses (p=0.046).

The students felt in-person sessions were most appropriate for 16/18 of the reasons listed in (table 1). The reasons with the highest proportion of students feeling in-person were the most appropriate were; interaction with friends (95.00%), interaction with the facilitator (91.67%), and developing clinical reasoning skills (91.67%). There were only two reasons students felt online sessions were most appropriate, these were home comforts (98.33%) and ease of travel (91.67%).

Original data can be accessed in Figshare at https://doi.org/10.6084/m9.figshare.23616627.v1 and https://doi.org/10.6084/m9.figshare.23616630.v1

Results are ranked from reasons with the highest proportion of students thinking in-person was most appropriate for that reason. P-values have been calculated to establish if there is statistical significance between the results for male and female students, significant results are highlighted in bold.

	All students	Female students	Male students	P-value
Interaction with friends	57/60 (95.00%)	39/42 (92.86%)	18/18 (100.00%)	0.245 (>0.05)
Interaction with the facilitator	55/60 (91.67%)	39/42 (92.86%)	16/18 (88.89%)	0.610 (>0.05)
Developing clinical reasoning skills	55/60 (91.67%)	39/42 (92.86%)	16/18 (88.89%)	0.610 (>0.05)
Learning from the facilitator	51/60 (85.00%)	35/42 (83.33%)	16/18 (88.89%)	0.581 (>0.05)
Flow	47/60 (78.33%)	33/42 (78.57%)	14/18 (77.78%)	0.945 (>0.05)
Developing history-taking skills	46/60 (76.67%)	36/42 (85.71%)	10/18 (55.56%)	0.011 (<0.05)
Developing knowledge	45/60 (75.00%)	33/42 (78.57%)	12/18 (66.67%)	0.329 (>0.05)
Ease of sharing opinions	45/60 (75.00%)	33/42 (78.57%)	12/18 (66.67%)	0.329 (>0.05)
Enjoyment	44/60 (73.33%)	34/42 (80.95%)	10/18 (55.56%)	0.041 (<0.05)
Learning from peers	41/60 (68.33%)	28/42 (66.67%)	13/18 (72.22%)	0.672 (>0.05)
Formulating differential diagnoses	41/60 (68.33%)	32/42 (76.19%)	9/18 (50.00%)	0.046 (<0.05)
Interpreting clinical data	40/60 (66.67%)	29/42 (69.05%)	11/18 (61.11%)	0.550 (>0.05)
Formulating management plans	39/60 (65.00%)	29/42 (69.05%)	10/18 (55.56%)	0.315 (>0.05)
Better for mental health	36/60 (60.00%)	26/42 (61.90%)	10/18 (55.56%)	0.645 (>0.05)
Developing communication skills	35/60 (58.33%)	22/42 (52.38%)	13/18 (72.22%)	0.153 (>0.05)
Interpreting investigations	35/60 (58.33%)	27/42 (64.29%)	8/18 (44.44%)	0.153 (>0.05)
Ease of travel	5/60 (8.33%)	5/42 (11.90%)	0/18 (0.00%)	0.126 (>0.05)
Home comforts	1/60 (1.67%)	1/42 (2.38%)	0/18 (0.00%)	0.509 (>0.05)
Table 1. The proportion of medical students who felt in-person sessions were the most appropriate for the listed reasons.

IV. DISCUSSION

Overall, more students in our cohort preferred in-person clinical reasoning seminars and a larger proportion of these students were female than male, however, the difference in preference based on gender did not show statistical significance. This could be due to a smaller cohort of male respondents (18/60) compared to female respondents (42/60). However, even in a study of 488 medical and dental students there was no significant difference in preference for online or in-person delivery when asked about all sessions in the curriculum (Al-Azzam et al., 2020). A larger sample size of medical students will need to be analysed to establish if gender significantly influences student preference for online or in-person delivery of clinical reasoning seminars.

Students felt in-person seminars were better for 16/18 of the listed reasons. This included all reasons pertaining to interaction with other students and staff and all reasons regarding learning a variety of skills. The only two reasons students felt online sessions were better were ease of travel and home comforts. Therefore, this data suggests the only reasons the medical students preferred online seminars were due to the convenience of the setting, and they felt the learning and interaction were superior in in-person seminars.

Of these eighteen reasons, three reasons showed a significant difference in response based on gender, with more female students preferring in-person seminars for the following reasons: enjoyment, development of history-taking skills and formulating differential diagnoses. History-taking and formulating differential diagnoses are more commonly undertaken by doctors within in-person environments. Therefore, female medical students may have a stronger preference for learning skills in the same setting they will be undertaken in when they are doctors.

This study helps to identify the components of clinical reasoning seminars male or female students prefer to undertake online or in-person. Future research could try to identify the reasons for these preferences and to establish if female students have a stronger preference for learning a skill in the same environment it would be undertaken in when they become a doctor.

This research will have most transferability to educators designing clinical reasoning modules to undergraduate students. It may also have some transferability to any undergraduate seminars and to postgraduate medical education. Also, understanding the environment each gender prefers to learn in and why, could help to designing future educational programmes. Especially if these programmes have previously shown differing participation or attainment based on gender.

V. CONCLUSION

In conclusion, students preferred in-person clinical reasoning seminars compared to online seminars and a higher percentage of female students preferred in-person compared to male students, although this was not statistically significant. Students had the strongest preference for in-person sessions due to interaction with friends and the facilitator and for developing clinical skills. Students had the strongest preference for online sessions due to home comforts and ease of travel. Female students preferred in-person seminars compared to male students for the following statistically significant reasons: enjoyment, developing history-taking skills and formulating differential diagnoses.

Notes on Contributors

Dr Victoria Scudamore was involved in data collection, data analysis and wrote this manuscript in full.

Dr Sze Yi Beh was involved in data collection and data analaysis.

Dr Adam Foster was involved in data collection and data analysis.

Dr Michaela Goodson supervised and advised on data collection and analysis.

Ethical Approval

Research and ethics approval was granted by the research committee at Newcastle University Medicine Malaysia on 08/02/2022 (Approval number: 18547/2022).

Data Availability

The data used in this paper is available in the Figshare repository through the following links with associated DOI’s https://doi.org/10.6084/m9.figshare.23616627.v1 and https://doi.org/10.6084/m9.figshare.23616630.v1. The data is shared on the Figshare repository under the CC0 licence (no rights reserved) as there is no copywritten information included.

Funding

No additional funding was used to undertake this project.

Declaration of Interest

There are no potential conflicts of interest to declare.

References

Al-Azzam, N., Elsalem, L., & Gombedza, F. (2020). A cross-sectional study to determine factors affecting dental and medical students’ preference for virtual learning during the COVID-19 outbreak. Heliyon, 6(12), 4. https://doi.org/10.1016/j.heliyon.2020.e05704

Altaf, R., Kling, M., Hough, A., Baig, J., Ball, A., Goldstein, J., Brunworth, J., Chau, C., Dybas, M., & Jacobs, R. J. (2022). The association between distance learning, stress level, and perceived quality of education in medical students after transitioning to a fully online platform. Cureus, 14(4), 3. https://doi.org/10.7759/cureus.24071

Babenko, O., Ding, M., & Lee, A. S. (2022). In-person or online? The effect of delivery mode on team-based learning of clinical reasoning in a family medicine clerkship. Medical Sciences, 10(3), 41. https://doi.org/10.3390/medsci10030041

Cromer, S. J., D’Silva, K. M., Phadke, N. A., Lord, E., Rigotti, N. A., & Baer, H. J. (2022). Gender differences in the amount and type of student participation during in-person and virtual classes in academic medicine learning environments. JAMA Network Open, 5(1), 1. https://doi.org/10.1001/jamanetworkopen.2021.43139

*Dr Victoria Scudamore
Newcastle University Medicine Malaysia
No. 1 Jalan Sarjana 1,
Kota Limu, EduCity@Iskandar,
Iskandar Puteri, Johor, Malaysia, 79200
Email: victoria.scudamore@nhs.net

Submitted: 5 July 2023
Accepted: 12 December 2023
Published online: 2 July, TAPS 2024, 9(3), 1-14
https://doi.org/10.29060/TAPS.2024-9-3/OA3045

Dujeepa D Samarasekera¹, Shuh Shing Lee¹, Su Ping Yeo¹, Julie Chen², Ardi Findyartini^3,4, Nadia Greviana^3,4, Budi Wiweko^3,5, Vishna Devi Nadarajah⁶, Chandramani Thuraisingham⁷, Jen-Hung Yang^8,9, Lawrence Sherman¹⁰

¹Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Family Medicine and Primary Care/ Bau Institute of Medical and Health Sciences Education, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong; ³Medical Education Center, Indonesia Medical Education & Research Institute (IMERI), Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia; ⁴Department of Medical Education, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia; ⁵Department of Obstetrics and Gynaecology, Faculty of Medicine Universitas Indonesia/Cipto Mangunkusumo Hospital, Jakarta, Indonesia;  ⁶IMU Centre of Education and School of Medicine, International Medical University, Kuala Lumpur, Malaysia; ⁷Department of Family Medicine, School of Medicine, International Medical University, Kuala Lumpur, Malaysia; ⁸Medical Education and Humanities Research Center and Institute of Medicine, College of Medicine, Chung Shan Medical University, Taichung, Taiwan; ⁹Department of Dermatology, Chung Shan Medical University Hospital, Taichung, Taiwan; ¹⁰Meducate Global, LLC, Florida, USA

Abstract

Introduction: Continuing medical education and continuing professional development activities (CME/CPD) improve the practice of medical practitioners and allowing them to deliver quality clinical care. However, the systems that oversee CME/CPD as well as the processes around design, delivery, and accreditation vary widely across countries. This study explores the state of CME/CPD in the East and South East Asian region from the perspective of medical practitioners, and makes recommendations for improvement.

Methods: A multi-centre study was conducted across five institutions in Hong Kong, Indonesia, Malaysia, Singapore and Taiwan. The study instrument was a 28-item (27 five-point Likert scale and 1 open-ended items) validated questionnaire that focused on perceptions of the current content, processes and gaps in CME/CPD and further contextualised by educational experts from each participating site. Descriptive analysis was undertaken for quantitative data while the data from open-ended item was categorised into similar categories.

Results: A total of 867 medical practitioners participated in the study. For perceptions on current CME/CPD programme, 75.34% to 88.00% of respondents agreed that CME/CPD increased their skills and competence in providing quality clinical care. For the domain on pharmaceutical industry-supported CME/CPD, the issue of commercial influence was apparent with only 30.24%-56.92% of respondents believing that the CME/CPD in their institution was free from commercial bias. Key areas for improvement for future CME/CPD included 1) content and mode of delivery, 2) independence and funding, 3) administration, 4) location and accessibility and 5) policy and collaboration.

Conclusion: Accessible, practice-relevant content using diverse learning modalities offered by unbiased content providers and subject to transparent and rigorous accreditation processes with minimal administrative hassle are the main considerations for CME/CPD participants.

Keywords:        Medical Education, Health Profession Education, Continuing Professional Development, Continuing Medical Education, Accreditation

Practice Highlights

• Identifying professional practice gaps of clinicians should be the first step.
• The state of CME/CPD varies among countries and addressing relevant needs is crucial.
• Clinicians agreed that CME/CPD improves their skills and knowledge but lacked time to participate.
• Potential improvements include relevant content free from commercial bias and delivery mode.
• Systematic governance and aligned regulations by physician credentialing agencies is recommended.

I. INTRODUCTION

Lifelong learning is an essential skill for all healthcare professionals. This is particularly true when new models of healthcare delivery are being implemented and there is increased focus on outcomes and values such as shorter hospital stay, greater accountability and transparency and emphasis on patient engagement (Sachdeva, 2016; Vinas et al., 2020). Recent literature highlights that continuing medical education and continuing professional development programs (CME/CPD) are crucial in providing current contextually relevant educational and developmental activities in maintaining knowledge, skills, and performance for clinicians and have proven to be effective (Cervero & Gaines, 2015; Drude et al., 2019; Forsetlund et al., 2009). CME is defined as “educational activities which serve to maintain, develop, or increase the knowledge, skills, and professional performance and relationships that a physician uses to provide services for patients, the public, or the profession” (Accreditation Council for Continuing Medical Education, n.d.), while CPD is usually a broader and more inclusive term referring to the combination of formal CME and other activities type that are designed to assist healthcare professionals to acquire skills and knowledge essential for their professional growth (Sherman & Chappell, 2018). Critical systematic reviews of the literature have shown that CME/CPD improves practice and support professional activities of medical practitioners to deliver best patient care (Cervero & Gaines, 2015; Sachdeva, 2016).

Although CME/CPD has undergone enormous changes and growth over the past 25 years, the advancement in CPD still considerably lag behind as compared to undergraduate and graduate medical education (Sachdeva et al., 2016). Goals and objectives in CME/CPD are often poorly defined and there is a paucity of the curricular structure for medical practitioners (Sachdeva, 2016). Despite consistent evidence sharing that formal CME/CPD activities, such as conferences and workshops, have little or no long-lasting effect on medical practitioners, many CME/CPD providers continue to include these approaches as their major educational offerings while clinicians continue to attend to improve their practice (Mann, 2002). Additionally, there are environments where CME/CPD is not mandatory, and in some instances, non-existent (Sherman & Nishigori, 2020).

Despite CME/CPD’s importance, the state of CME/CPD varies widely across regions and countries. Unlike Europe and the United States, there is no parallel accreditation system for CME/CPD in Asia. CME/CPD does not follow a standard process in all countries and the requirements are also different. A short summary of the CME/CPD system in the countries which are studied in this article is provided in Appendix 1. However, there is still a lack of empirical data in understanding the CME/CPD in Asia. Only one study was conducted in Japan to assess the state of CPD in the country and to identify the gaps in the understanding of the medical practitioners’ needs (Sherman & Nishigori, 2020). Hence, this study aims to explore the state of CME/CPD in the East and South East Asian region from the perspective of medical practitioners, and make recommendations for improvement.

A. Theoretical Framework

Researchers have been proposing few theoretical frameworks which are related to CME/CPD. For this study, we will be using the Process of Change and Learning framework by Fox et al. (1989) to provide an overarching view on the process of change and learning among medical practitioners. This will be further enhanced by using adult learning theory (Knowles, 1989).

No discussion of practice informing theory in CME could exclude the work of Fox et al. (1989), who studied the process of change and learning in the lives of medical practitioners. They interviewed more than 350 medical practitioners to find out the types of learning activities that clinicians undertake and the important factors in the process of learning and change. The framework is illustrated in Figure 1.

Figure 1. Theoretical Framework related to CME/CPD using the process of change and learning (Fox et al., 1989) and adult learning theory (Knowles, 1989)

This framework clearly illustrated how change and learning occurs through several processes and how these changes were influenced by three forces. The actual process of change involves three iterative steps – preparing for the change, making the change, and sustaining or implementing the change in practice.

Through validated studies, we understand that there are three forces to prepare for the change, mainly personal, professional and social forces. Professional forces were found to be the most frequently motivated change. Personal forces, such as the desire for personal well-being, were infrequent and usually not the sole force for change. More often they were combined with professional forces, e.g. the desire to further one’s career. Social forces were also cited, usually combined with professional forces, e.g. relationships with colleagues.

Once the image of change has been developed, medical practitioners will evaluate the discrepancy between what new knowledge and skills are needed to achieve the change and estimate their current capacities. As shown in Figure 1, the perceived discrepancy is positively correlated with the effort that a medical practitioner will put in in learning. Therefore, the next step may involve attending a formal CME event if the discrepancy is high – to understand what is required and to assess or verify one’s own capabilities.

Although the Process of Change and Learning Framework provides us a big picture on how medical practitioners engaged in change and learning, it is insufficient to understand the humanist approach in understanding learning for human growth. It is widely recognised that autonomy and self-directed learning are the developmental nature for human desire to learn (Personal Forces). This behaviour is usually motivated by a mixture of external and internal motivation. This is important for the development of individuals toward autonomy, the self-directed learning, reflective practice and critical reflection, experiential learning, and transformative learning.

II. METHODS

This is a multi-centred study which employed a survey using a validated questionnaire and the section below will describe the data collection process, sampling of participants and data analysis coupled with a qualitative data gathering focus group with educational experts from each place participating. Five sites were involved in this study: Hong Kong, Indonesia, Malaysia, Singapore and Taiwan. Ethical approval was obtained from the respective Institutional Review Board [Reference Number: DSRB-2019-0449 (Singapore), UW 19-840 (HKU/HA HKW IRB) (Hong Kong), KET-1035/UN2.F1/ETIK/PPMetc.00.02/2019 (Indonesia), (CCH-IRB-200425) (Taiwan), IMU 467/2019 (Malaysia)].

The same questionnaire that was used and validated previously in Japan was modified for use in this region (Nishigori and Sherman, 2018). The questionnaire is a self-administered, 28-item test comprising 27 single or multiple-choice questions and an open-ended question for comments. Respondents were asked to rate on a 5-point Likert scale (Strongly Disagree to Strongly Agree) for some of the questions. Demographic questions were included at the start of the survey (e.g. specialty, years of practice, prior participation in CME/CPD activities) followed by the following domains:

• Perceptions and satisfaction of clinicians with regard to current CME/CPD available for them
• Adequacy of the current CME/CPD available
• Impacts of CME/CPD in content coverage, evaluation, and development of learning
• Gaps in CME/CPD
• Future areas to focus on

The items were finalised following a group of experts’ meeting held in Singapore (March 2019) whereby the representatives (medical educationalists and medical practitioners) from participating sites discussed and went through the questions thoroughly. The meeting was moderated by an expert with over 28 years of experience in CME/CPD, and who designed the original study questionnaire. To add more local context and ensure that respondents were able to answer accurately, the questionnaire was translated into the native language and terms by the representatives in some locations.

Medical practitioners were invited to participate in the study.  The study was conducted from July 2019 until May 2020. Voluntary, convenience, and snowball sampling was used and the representatives either disseminate the questionnaire link to their mailing list or through the various national organisation/institutions (Table 1) who then informed their members/faculty, in accordance with the ethics protocol guidelines. Reminders were sent until the response rate no longer increased. Implied consent was obtained from the participants when they proceeded to complete the survey after reading the information about the study on the first page.

Submitted: 2 September 2023
Accepted: 9 January 2024
Published online: 2 July, TAPS 2024, 9(3), 15-21
https://doi.org/10.29060/TAPS.2024-9-3/OA3129

Uzma Khan

Department of Clinical Sciences, College of Medicine, Al Rayan Colleges, Saudi Arabia

Abstract

Objectives: To compare the cut scores and pass/fail rates achieved by arbitrary 60% method and Hofstee method in an undergraduate year 4 end semester objective structured clinical examination (OSCE) and check the possibility of using Hofstee method of standard setting in future exams.

Method: 102 medical students of year 4 underwent a 10 station OSCE exam conducted in a state of art simulation lab in 3 cycles. The cut scores were calculated using arbitrary method aiming at 60% of task achievement and by Hofstee method. The student’s obtained scores were compared for cut scores and pass rates for individual stations and the entire exam.

Results: The arbitrary and Hofstee methods of standard setting leads to different cut scores. For the individual stations it was 60% vs 65-70% and for the overall score it was 60% vs 70%. The percentage of students failing the exam is 13.7% based on arbitrary scores and is 29.4% when Hofstee cut score is applied.

Conclusions: The two methods lead to different cut scores and students’ failure rates. Overall, Hofstee method is more appropriate for assessing competencies in an OSCE exam in medical schools as it leads to calculation of cut scores based on the difficulty level of the station/exam and the examiners expected level of performance by the students.

Keywords:          Objective Structured Clinical Examination, Standard Setting, Hofstee Method, Arbitrary Method

Practice Highlights

• Standard settings of OSCE identifies objective, reliable and valid cutoff scores.
• Arbitrary method scrutinises the test content and nominates the percentage of items to be answered correctly.
• Hofstee method is calculative and avoids illogical very high and low scores.
• A retrospective descriptive study design assessing applicability of Hofstee method in low stake exam in a private medical school in Saudi Arabia.
• Students’ failure rate increased with applying Hofstee standard settings in OSCE.

Figure 1. Calculation of final cut score based on examiners provided four Hofstee values

Arbitrary cut score of 60% is applied to students achieved scores and the pass/fail percentage is calculated and recorded in table 2.

III. RESULTS

The OSCE went on without any significant issues. The data provided by the Exam and evaluation Unit (EEU) according to their software showed that the mean score was 75% with a standard deviation of 3.99% and an alpha coefficient of 1.03. Station wise descriptive results show a range in mean scores from 61 to 84%, illustrating a range in station difficulty. The detailed analysis of each station is shown in table 1.

Table 2. Station wise descriptive statistics, the two cut scores and students pass rates according to cut scores

The mean score for the station reflects its level of difficulty ranging from 61.17% to 83.3%. The cut score of the individual stations for the Hofstee method was higher than the cut score for the arbitrary method. So is the difference in pass rates, pass rates achieved with arbitrary cut scores are higher than with Hofstee method, as shown in Table 3.

Table 3. Comparison of overall cut scores and pass rates

Using Hofstee method and cut of score of 14 out of 20 passing rates achieved is 72 out of 102 which in percentage makes 70.5%. When compared with arbitrary method and cut score of 12 out of 20, students pass rate increased to 88 out of 102 leading to 86% overall. This study points out a higher pass rate for the students by arbitrary method, which creates a doubt on the competency of passing students.

IV. DISCUSSION

In this research, the results of end semester OSCE exam are compared by two methods, arbitrary fixed 60% standard setting method used at our college for the last 4 years and a compromise Hofstee method, which is applied for the very first time.

According to our study, the failure rate has increased from 13.7% to 29.4%, and has almost doubled. In fact, this increase is higher than what had been usually observed previously. This gives the impression that the students who have not yet achieved the required competency would have been allowed to pass. The same observation was made by Dudas et all who did apply Hofstee standard setting to a historic cohort of 116 Johns Hopkins University School of Medicine students from the academic year 2012–2013 to assess the potential impact on grade distributions (Dudas & Barone 2014).

According to the results of a study conducted by Doaa Kamal in Suez Canal University, Egypt in 2020 where four standard methods, the modified Cohen’s, borderline regression, Hofstee methods, and the fixed 60% arbitrary method were compared in determining the passing score in ophthalmology OSCE exam, it was concluded that 60% fixed arbitrary method resulted in a marked difference in failure and pass rates among students and Hofstee method yielded low pass rates which is consistent with my research (Kamal et al., 2020).

Since our exam was dealing with the assessment of multiple competencies, so Hofstee method is more likely to produce a standard appropriate with the purpose of assessment. Secondly the cut scores were calculated by the academic staff who were very much familiar with the OSCE as an assessment tool, the curriculum and the students as well. They were all content experts, fair and open-minded. Some of them were teaching in Taibah university the same content so they were well aware of the acceptable students’ performance. The examiners were meeting the criteria set by Downing et al., so their decision regarding the cut scores and estimation of number of failing students was accepted (Downing et al., 2006).   

Schindler et al in his research paper has applied Hofstee cut off scores and found that it can even be used for a multi assessment surgical clerkship and for assigning grades as well and concluded that this method has all the characteristics of an appropriate standard setting method (Schindler et al., 2007).

V. CONCLUSION

Since different competencies reflect the different level of difficulties, the cut scores need to be set for each station dealing with that competency. The arbitrary 60 % method is not appropriate to the purpose of an OSCE exam, but a cut off score calculated by using data from experts’ judgments provides a reasonable result with acceptable failing rates. Thorough and thoughtful preparation on the judges’ part is deemed important. The data gathered from this exam can be reviewed and acted in accordance with to create a standard each academic year.

Notes on Contributors

The author herself contributed to the design of the research, carried out the data acquisition and analysis, interpreted the data and prepared the manuscript.

Ethical Approval

Approval was obtained from the Institutional Research Ethics Committee (IREC) for the collection and publication of student data with approval No. HA-03-M-122-046. Informed consent is taken from the students and special permission from the dean of the college is obtained for the use of students result for this research purpose.

Data Availability

The data generated and analysed in this study are included in this published article. The raw data set analysed can be provided by the author if special interest in it is shown.

Acknowledgement

I would like to thank Dr. Yasir Naseem Khan from the department of basic medical sciences, Al-Rayan college of medicine for useful suggestions and guidance at the final stage of the project.

I would also like to thank the Dean and the Exam and evaluation Unit (EEU) of the Al-Rayan college of medicine for their support.

Funding

The study does not require any funding.

Declaration of Interest

The author declares that she has no competing interests.

References

Boulet, J. R., De Champlain, A. F., & McKinley, D. (2003). Setting defensible performance standards on OSCEs and standardised patient examinations. Medical Teacher, 25(3), 245-249. https://doi.org/10.1080/0142159031000100274

Cusimano, M. D. (1996). Standard setting in medical education. Academic Medicine, 71(10), S112-20. https://doi.org/10.1097/00001888-199610000-00062

Dent, J., Harden, R. M., & Hunt, D. (2021). A practical guide for Medical Teachers (6th ed.). Elsevier.

Downing, S. M., Tekian, A., & Yudkowsky, R. (2006). Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teaching and Learning in Medicine, 18(1), 50-57. https://doi.org/10.1207/s15328015tlm1801_11

Dudas, R. A., & Barone, M. (2014). Setting standards to determine core clerkship grades in pediatrics. Academic Pediatrics, 14(3), 294-300. https://doi.org/10.1016/j.acap.2014.01.008

Dwivedi, N., Vijayashankar, N. P., Hansda, M., Ak, D., Nwachukwu, F., Curran, V., & Jillwin, J. (2020). Comparing standard setting methods for objective structured clinical examinations in a Caribbean medical school. Journal of Medical Education and Curricular Development, 7. https://doi.org/10.1177/2382120520981992

Gormley, G. (2011). Summative OSCEs in undergraduate medical education. The Ulster Medical Journal, 80(3), 127. 

Harden, R. M., & Gleeson, F. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13(1), 39-54. https://doi.org/10.1111/j.1365-2923.1979.tb00918.x

Harden, R. M., Stevenson, M., Downie, W., & Wilson, G. M. (1975). Assessment of clinical competence using objective structured examination. The British Medical Journal, 1(5955), 447–451. https://doi.org/10.1136/bmj.1.5955.447

Hodges, B. (2003). OSCE! Variations on a theme by Harden. Medical Education, 37(12), 1134-1140. https://doi.org/10.1111/j.1365-2923.2003.01717.x

Hofstee, W. K. (1983). The case for compromise in educational selection and grading. On Educational Testing, 109-127. 

Humphrey‐Murto, S., & MacFadyen, J. C. (2002). Standard setting. Academic Medicine, 77(7), 729-732. https://doi.org/10.1097/00001888-200207000-00019

Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34(4), 353-366. https://doi.org/10.1111/j.1745-3984.1997.tb00523.x

Kamal, D., Sallam, M. A., Gouda, E., & Fouad, S. (2020). Is there a “best” method for standard setting in OSCE exams? Comparison between four methods (a cross-sectional descriptive study). Journal of Medical Education, 19(1), Article e106600. https://doi.org/10.5812/jme.106600

Kaufman, D., Mann, K., Muijtjens, A., & Van Der Vleuten, C. P. (2000). A comparison of standard-setting procedures for an OSCE in undergraduate medical education. Academic Medicine, 75(3), 267-271. https://doi.org/10.1097/00001888-200003000-00018

Khan, K., Ramachandran, S., Gaunt, K., & Pushkar, P. (2013). The objective structured clinical examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Medical Teacher, 35(9), e1437-e1446. https://doi.org/10.3109/0142159x.2 013.818634

Khan, K., Gaunt, K., Ramachandran, S., & Pushkar, P. (2013). The objective structured clinical examination (OSCE): AMEE Guide No. 81. Part II: Organisation & administration. Medical Teacher, 35(9), e1447-e1463. https://doi.org/10.3109/0142159x. 2013.818635

McKinley, D., & Norcini, J. J. (2014). How to set standards on performance-based examinations: AMEE Guide No. 85. Medical Teacher, 36(2), 97-110. https://doi.org/10.3109/0142159x.2013.8 53119

Norcini, J. J., Anderson, M. B., Bollela, V. R., Burch, V., Costa, M. J., Duvivier, R., Galbraith, R. M., Hays, R., Kent, A., Perrott, V., & Roberts, T. (2011). Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference. Medical Teacher, 33(3), 206-214. https://doi.org/10.3109/0142159x.2011.551559

Pell, G., Fuller, R., Homer, M., & Roberts, T. (2010). How to measure the quality of the OSCE: A review of metrics – AMEE Guide No. 49. Medical Teacher, 32(10), 802-811. https://doi.org/10.3109/0142159x.2010.507716

Pugh, D., & Smee, S. (2013). Guidelines for the development of objective structured clinical examination (OSCE) cases. Ottawa: Medical Council of Canada. https://doi.org/10.13140/RG.2.1.462 2.0003

Schindler, N., Corcoran, J., & DaRosa, D. A. (2007). Description and impact of using a standard-setting method for determining pass/fail scores in a surgery clerkship. The American Journal of Surgery, 193(2), 252-257. https://doi.org/10.1016/j.amjsurg. 2006. 07.017

Smee, S., Coetzee, K., Bartman, I., Roy, M., & Monteiro, S. (2022). OSCE standard setting: Three borderline group methods. Medical Science Educator, 32(6), 1439-1445. https://doi.org/10.1007/s40670-022-01667-x

Zayyan, M. (2011). Objective structured clinical examination: The assessment of choice. Oman Medical Journal, 219-222. https://doi.org/10.5001/omj.2011.55

*Dr. Uzma Khan
Department of Clinical Sciences,
Al Rayan College of Medicine,
Al Rayan National Colleges
Madina Munawara, Saudi Arabia
Contact: +966542754680
Email: uziik2019@gmail.com, uk.yasser@amc.edu.sa

Submitted: 4 July 2023
Accepted: 9 January 2024
Published online: 2 July, TAPS 2024, 9(3), 22-31
https://doi.org/10.29060/TAPS.2024-9-3/OA3081

Jia Le Ivan Tan¹ & Sashikumar Ganapathy²

¹School of Biological Sciences, Nanyang Technological University, Singapore; ²Department of Emergency Medicine, KK Hospital, Singapore

Abstract

Introduction: Maintenance of procedural skills is crucial for paediatric emergency medicine (PEM) physicians to provide high-quality care. A study by Lin-Martore et al. (2021) in the US identified key themes in conceptualising procedural skill maintenance (PSM), its motivations, and barriers to maintenance. However, the difference in culture brings into question the validity of their findings in other contexts. To determine its applicability specifically in an Asian context, this study aims to replicate the study at KK’s Women and Children Hospital (KKH), focusing on PEM physicians. While the findings are limited to a single hospital, they provide valuable insights into challenges encountered by PEM physicians.

Methods: A general qualitative approach was used through semi-structured interviews. Participants were recruited through email. Interviews were conducted via Zoom and subsequently de-identified and transcribed. The data was coded manually through thematic analysis, identifying key themes.

Results: Fifteen PEM physicians were interviewed. Participants conceptualised PSM through technical aspects and measured competence through objective and subjective measures. General motivation themes found the (1) desire to provide optimal patient care, (2) procedural competence as part of the identity of a PEM physician who teaches and performs procedures, and (3) desire for choice when alternatives are present. Barriers included limited time, support, and opportunities.

Conclusion: The study found that the themes from the original study are applicable in KKH, featuring SDT concepts prominently. Practical recommendations for KKH were proposed. Future research can focus on the challenges and gaps in maintaining procedural skills and develop strategies to improve PSM in PEM physicians.

Keywords:          Procedural Skill Maintenance, Singapore, Emergency Medicine, Qualitative, Paediatric Medicine

Practice Highlights

• Comparisons with the original study in the US were made.
• Understanding of procedural skill maintenance (PSM) was assessed.
• Motivators of PSM are identified.
• Barriers to skill maintenance are constant across studies.
• Current healthcare standards are maintained but medical advancements are limited.

I. INTRODUCTION

Procedural skills are critical in the training and competency of healthcare professionals. Paediatric Emergency Medicine (PEM) physicians may need to perform a wide range of procedures, including intubation, cannulation, and other life-saving interventions and are often the primary care physician first contacting the patient. As such, maintaining the procedural skills of PEM physicians is critical to ensure patient safety and high-quality care. However, many critical procedures are not common and the wide variety of procedures that fall under their purview makes maintenance of such skills challenging (Cabalatungan et al., 2020). Hence, there is a need to investigate the current state of procedural skill maintenance (PSM) in physicians to evaluate current systems and implement new policies if necessary.

According to a study by Lin-Martore et al. (2021), three themes revolving around motivation for PSM have been identified based on self-determination theory components of autonomy, competence, and relatedness (Ryan & Deci, 2000): (1) desire for optimal patient care and fear of suboptimal performance, (2) procedural competence is part of a PEM physician’s identity who teach and perform procedures and (3) desire for accessibility and choice in PSM.

As the research was conducted in the US, it is important to determine if such findings are generalisable to and replicable within an Asian society that may have different values than Western, Educated, Industrialised, Rich, and Democratic (WEIRD) societies (Henrich et al., 2010; Jones, 2010; National Academies of Sciences et al., 2019).

This paper aims to replicate the research design by Lin-Martore et al. (2021). It examines the unique challenges of maintaining proficiency in a high-stress environment within the paediatric emergency department (PED) of KK’s Women and Children’s Hospital (KKH) in Singapore.

This study examines the following research questions: (1) To what extent do the three themes characterising motivation for PSM identified from the Lin-Martore study in the US apply to Singapore? (2) To what extent do the barriers to motivation and skill maintenance identified from the Lin-Martore study in the US apply to Singapore?

Additionally, this study aims to identify any gaps or challenges in the current skill maintenance process and make recommendations for improving the maintenance of procedural skills in PED physicians.

II. METHODS

To tackle this hypothesis, the study design involved conducting qualitative interviews with participants to elicit key themes through thematic analysis. The themes would then be compared to the original study to determine its validity and subsequently formulate applications for use in Singapore.

Participants were recruited using the Purposeful Sampling Method through an email invitation. The selection criteria required participants to be faculty currently working in the Emergency Department of KKH with varying years of experience. They must specialise in PEM and have been working in the Emergency Department of KKH for at least 2 years. Participants consented to participation and recordings. The research team consisted of an undergraduate student (IT) and a PEM senior consultant (SG) with 12 years of experience in PEM.

A. Interview Questions

Based on the original study, most of the interview questions were kept the same and revolved around the conceptualisation of PSM using SDT concepts of autonomy, relatedness, and competence together with its motivations and barriers (Bradbury-Jones et al., 2017; Kahlke, 2014). However, the original question regarding autonomy was replaced to determine the receptiveness of participants toward novel techniques when existing protocols are already enacted by one investigator (SG).

B. Interviews

One investigator (IT) conducted interviews and was involved in the development and refinement of the interview guide and review of the transcribed interviews. This addressed the potential for reduced participant candour due to established relationships and mitigated the potential for less explicit responses due to familiarity with certain jargon. Similar to the original study, 30 – 45-minute semi-structured interviews were conducted and recorded via video-conferencing software (Zoom Cloud Meetings) using a general qualitative approach from July 2022 to February 2023. Following this, they were de-identified and transcribed. No participant was interviewed more than once. Interviews were conducted until data saturation was achieved.

C. Thematic Analysis

The data underwent structural coding for categorisation based on research questions and topics by one investigator (IT). In line with thematic analysis, ideas from the initial study were used as a framework for identifying key ideas relating to concepts from SDT by both investigators (IT and SG). Additional ideas elicited from interviews were then integrated into the codebook. Key concepts and patterns were then identified by both investigators.

The data was manually coded using Microsoft Office. Having perspectives from a PEM physician and nonphysician in the research team provided greater theoretical analysis and practical applications of SDT in the context of KKH.

III. RESULTS

A. Participant Demographics

Fifteen PEM physicians with 6-32 years of experience were interviewed (Table 1). Participants worked shifts with and without learners, including medical students.

Table 1. Participant demographics (n=15)

Note: Shifts are 8 hours long.

B. Thematic Analysis

The themes were organised based on the research results and findings of the original study (Table 2).

Table 2. Major themes in procedural skill maintenance

C. Conceptualisation of Procedural Skill Maintenance (PSM)

1) PSM is understood through technical aspects: Participants intuitively understood PSM and operationalised it through technical aspects such as practicing the procedure, attending courses and refreshers, keeping up to date with literature, revisiting the content, and teaching others.

“There should be constant upgrading of yourself either by doing the procedure itself or also upgrading by constantly reading literature and updated books and materials to check new things.”

(interview 3)

“For my TNS, although I try to teach some of the new MOs that come, I have to read back or watch from previous videos to maintain my skills.”

(interview 5)

The ability to teach is based on the protégé effect where through teaching others, knowledge retention increases (Baugh & Scandura, 1999; Chase et al., 2009; Gates, 2019, p. 20). It also allows senior physicians to gauge their competence relative to others. This self-evaluation means that PSM is an abstract concept understood by physicians through a form of social comparison with others.

2) Measuring competence through objective and subjective measures: Physicians listed objective measures implemented for junior doctors such as having third-party physicians evaluate and provide immediate feedback on skills.

“All of us must go through a skill test in the sense that you have to do supervised procedures – X number in the first six months of joining. So, there is a senior who makes sure that you know what you’re doing and [according to] the correct method, process including you know consent taking and post-procedure care, etc. And once we are supervised, we are ticked off as competent by the senior staff.”

(interview 9)

Confidence and the ability to foresee problems and deal with them are subjective measures that highlight the difficulties in measuring competencies. Many physicians feel confident in their ability to perform a procedural skill if they can do it without hesitation and without committing mistakes.

“If you are quite confident, it is just as easy as using a spoon and fork. You don’t have to think much about it, and you do it at ease with all confidence. It’s like muscle memory plus decision-making.”

(interview 6)

“I think we always know what our backup plan is. So, if it’s incubation then I know if I need to, I can code, call for airway code. There’s another senior on shift who might be able to help. I can call the anaesthetist without even coding if I need to … I anticipate what’s the problem that’s going to happen.”

(interview 12)

D. General Motivation

On motivation, three themes were identified with the SDT components of autonomy, competence, and relatedness serving as tenets.

1) Desire to provide optimal patient care (competence):

“The satisfaction that you have done a good job and that has resulted in a good outcome, gives you the motivation, satisfaction to update, maintain, upskill your skills.”

(interview 1)

Providing optimal patient care is a key motivation. The idea of competence is highlighted through the need to feel capable and effective in providing patient care.

“As a doctor, you are responsible and accountable for your patients, right? You must treat your patient at your best, right? If you’re not maintaining your procedural skills, you cannot serve your patient to the best of your knowledge and skill.”

(interview 7)

The concept of duty was highlighted. Physicians felt responsible for providing optimal patient care as part of their values and the right thing to do.

2) Procedural competence as part of the identity of a PEM physician who teaches and performs procedures (competence and relatedness): Participants listed several key procedures as part of their repertoire. Essentially, these skills were part of their identity as a PEM physician, making the skills important to maintain.

“The fact that you are the consultant and you need to know everything …  you might be the only one who is the most senior to be able to do it. If nobody can get it done, then it comes down to you.”

(interview 11)

Being the sole physician available to perform a procedure confers responsibility. Hence, skill maintenance is paramount to clinical practice with responsibility acting as a motivator.

“Although we have a lot of experienced doctors who can do it, but for me as a senior, I think I have to learn so that I can supervise in the future.”

(interview 13)

Senior physicians often teach and supervise junior doctors, driving senior physicians to maintain their skills. As senior physicians, individuals may feel a sense of connection and responsibility to other physicians who look up to them as role models.

3) Desire for choice when alternatives are present (autonomy): Physicians highlighted a need to ascertain the suitability of certain procedures to the patient. This indicates a desire for choice and autonomy, for the physician to make their judgement on the situation.

“If there is a good alternative that’s already in place, unless there’s a departmental guideline to force us to change, then I don’t see the need to change that. It’s sort of stipulated by what they [the department] wants.”

(interview 11)

Regarding novel techniques, physicians tend to fall back on existing, proven techniques. Without adequate reason, physicians are less likely to experiment with new skills unless there is a need to such as inadequacy of existing skills or department policy.

E. Barriers to Maintenance of Procedural Skills

On barriers, the lack of opportunities, time, and support served as the greatest obstacles.

1) Lack of opportunities: This encompasses external constraints such as patient flow, allowing junior doctors to perform procedures, and limited course slots

“We need to prioritise who to see for us because if, for example, I just keep doing trauma, it takes me 30 minutes to one hour. Rather than seeing medical cases in which in one hour I can see three to four patients. So, I think it’s more of how busy the children’s emergency is.”

(interview 10)

Patient flow and load are factors physicians consider as constantly prioritising learning creates delays in patient care, especially seen in high patient volumes. Prioritising efficiency, physicians fall back on well-versed competent skills that are already well-maintained. 

“At this stage, we are not doing the procedure ourselves yeah so it’s mainly the junior so that would be the barrier for the senior doctors.”

(interview 8)

After senior physicians reach a certain level, there is a transition in roles from a participatory to a supervisory one. With less direct participation in procedures, there are reduced opportunities to practice.

“There are limited slots every 4 years for APLS so the interval of the course is every four years to maintain their accreditation… there is a wait time for it.”

(interview 2)

Accreditation is an objective measure to determine competency in a particular skill. However, limited slots and infrequent sessions create long wait times, leading to some disparity in skill across the faculty.

2) Lack of time:

“Time is the biggest factor for anybody. If things like practicing or attending courses are outside of the shift, it would be difficult because you have other things to do and the shifts usually are quite busy.”

(interview 1)

Time limits a physician’s ability to practice skills. Within typical working hours, they need to balance clinical hours with practicing skills not commonly used. Higher clinical hours would mean less time available for workshops or courses for skill maintenance.

3) Lack of support:

“I think our limitation is more because we are unable to spare the manpower for dedicated time for updates or refresher courses for the seniors or the permanent staff…

Ideally, we would do it more frequently but we have to balance it with the clinical needs.”

(interview 4)

Reduced manpower equates to reduced capacity to send physicians for workshops and courses at regular intervals without compromising on clinical care. As the number of patients remains high, the reduced manpower means that the department cannot afford to send them for courses as there would be insufficient physicians available, constituting a lack of support for PSM.

IV. DISCUSSION

A. Comparison to the Original Study

The purpose of this study is to replicate the findings of Lin-Martore (2021) to determine applicability in an Asian context. A summary comparing the themes in the current study and the original study has been created (Table 3).

Topic	Current Study Themes	Original Study Themes
Conceptualisation of PSM	PSM is understood through technical aspects	PSM lacks a clear definition
	Measuring competence through objective and subjective measures	Ambivalence regarding requirements and the importance of adapting any requirements to practise needs
Motivation	Desire to provide optimal patient care (competence)	Desire to provide optimal patient care and fear of unsuccessful performance (competence)
	Procedural competence as part of the identity of a PEM physician who teaches and performs procedures (competence and relatedness)	Procedural competence as part of the identity of a PEM physician who teaches and performs procedures (competence and relatedness)
	Desire for choice when alternatives are present (autonomy)	Desire for accessibility and choice in maintaining procedural skills (autonomy)
Barriers to PSM	Lack of opportunities, time, and support	Lack of opportunities, time, and support

Table 3. Comparison of major themes

Singapore and the US have different social settings, values, and tenets which may help explain some of the results presented in this study. Singaporean physicians use social comparison within the department for performance evaluation and decision-making. US physicians take a more individualistic approach to self-evaluation with physicians sourcing for courses and defining their own standards (Lin‐Martore et al., 2021).

Regarding the conceptualisation of PSM, the current study did not find that participants had difficulty defining PSM, contrasting with the original study where conceptualisations were intuitive but difficult to verbalise and define. The same tautology exists where being competent was defined as completing the skill, however, the need to know if one is competent before performing the skill is not as prominent in the current study. This is due to the availability of senior staff and specialties available to assist, creating a space that allows independent learning with minimal patient risk. Additionally, the ambivalence regarding requirements is not as present in the current study. Although junior doctors have requirements, autonomy among senior physicians is important and highly respected (Chen & Chung, 2002; Fischer, 2008).

Interestingly, many mentioned confidence as a primary factor determining competence. This contrasts with the original study where participants noted that confidence alone failed to demonstrate competency adequately. In an Asian society where collectivistic efforts are highly valued, being sufficiently confident in performance could be a more accurate representation of competence compared to Western societies.

Regarding motivation, the desire to provide optimal patient care is consistent between studies (Deber, 1994, p. 19; Eisenberg, 1985; Nease & Brooks, 1995). The current study did not find the fear of unsuccessful performance as prominent as that in the original study. A possible reason could lie in the culture of reliance on other specialties. Within KKH, there is a culture of learning, and rather than a complete handover of the patient to other specialties, PEM physicians tend to assist with complex procedures, promoting interdisciplinary learning across faculties. As such, rather than a fear of unsuccessful performance, this is perceived as an opportunity to learn new skills.

Procedural competence is a vital part of a PEM physician’s identity across studies. The additional responsibility of being a senior is a key theme in this study. Should seniors fail to upkeep their skills, there is a sense of shame especially when juniors seek aid. Additionally, the failure to upkeep skills also impedes patient care, violating their sense of competence. This holds the greater the seniority.

The desire for autonomy differs in scope across studies. In the original study, skill prioritisation depends on the availability of individuals who possess that skill. Should there be someone with greater experience in that one skill, participants would choose to practice other skills. In the current study, skill prioritisation is dependent on the reliability of the skill and the presence of alternative skills. Should there be an existing skill that is highly reliable, physicians are likely to prioritise it.

Regarding barriers to PSM, a lack of time, opportunities and support are consistent throughout the studies.

B. Challenges of Maintenance

In this paper, three challenges are identified from the barriers to PSM: practice constraint, advancement constraint, and knowledge constraint.

Practice constraint comes from the barriers of time and opportunity. PEM physicians often have busy work schedules, making it difficult to practice less common but critical procedures during shifts. Additionally, the rarity of critical procedures limits opportunities to use them. With non-use, skills and procedural knowledge decay at a faster rate than knowledge by a year after training (Perez et al., 2013, p. 20; Yang et al., 2012).

Advancement constraint arises from the barriers of support and time. Rapid advancements can make it difficult for physicians to keep up with the latest techniques and procedures. The tyranny of choice heightens the barrier to entry to these new techniques, especially when tried-and-tested techniques exist and have proven to be reliable. Without departmental support pushing for more experimentation, there is little incentive to develop novel techniques, impeding the advancement of medical knowledge.

Knowledge constraint arises from barriers of support and opportunities. Different healthcare institutions have different critical skills important to the faculty, causing a knowledge disparity between physicians across facilities (Mittiga et al., 2019). With new physicians, proper onboarding is critical in ensuring the same standard of care and level of specialised knowledge. As a result, teaching and maintaining the procedural skills of physicians is an ongoing challenge.

C. Applications in KKH

Firstly, the workflow in KKH targets practice constraints and knowledge constraints. The use of senior physicians as a safety net allows junior doctors to learn and practice procedures on the job (practice constraint) while resolving knowledge gaps as senior physicians provide timely, informative feedback for optimal learning (knowledge constraint) (K. Ericsson et al., 1993, p. 199). Providing the space for junior doctors to learn by trial leads to an increase in performance accuracy while still allowing a safe environment for senior physicians to intervene (Crowe et al., 2018; Docherty-Skippen et al., 2020; K. Ericsson et al., 1993; Firdaus, 2018; Trowbridge & Cason, 1932).

This is seen in how participants understand PSM through technical aspects such as the ability to teach and how competence is operationalised objectively (third-party rating) and subjectively (confidence). Teaching others requires identifying the learner’s weaknesses and assigning tasks that build on their pre-existing knowledge for optimal learning. Based on the Levels of Processing Framework, deeper engagement with the material also allows deeper retention of the material for the teacher (Craik & Lockhart, 1972).

Next, the nature of a physician’s career relies heavily on competence. Motivating factors such as the desire to provide the best patient care demonstrate advancement constraints as physicians desire to improve their own skill set. However, as a physician’s priority is patient care, there is little incentive to explore new treatment options with unknown reliability especially with established treatment options (K. Ericsson et al., 1993, p. 199). Participants are less motivated to explore regional blocks unless it is a department regulation. With more options, physicians tend to fall back on what has been tried and tested, enforcing a mental set as exploring new options that may not succeed could lead to damage to their reputation or hinder patient care.

Finally, the lack of time, manpower, and opportunities is a consistent problem. Most PEM physicians spend their hours tending to patients, leaving little time for courses within stipulated working hours. Attending courses creates a greater workload for other physicians due to a decreased doctor-patient ratio. Rosters to send physicians to courses in a staggered formation have been a common suggestion however, many recognise the difficulties of doing so with limited manpower. As external parties conduct some courses, there is a big pause between their conduct. In-house trainers allow for more frequent conduct due to accessibility. However, the lack of manpower reduces the number of accredited trainers that can facilitate such sessions.

In the short term, current attitudes towards PSM appear to be geared towards crisis management, dealing with the large influx of patients with low manpower. PEM physicians revise only the most essential skills, whittling away those that are less necessary and can be executed by specialists. In the long term, these attitudes could impede innovation as deliberate practice is inherently unenjoyable and requires effort over a long period (K. A. Ericsson, 2004). PEM physicians could display expert mastery over their skills however, they would face difficulties exhibiting eminent performance to make a unique contribution to the field.

D. Practical Recommendations

Hospital administrators can consider extending the practice of enforced protected learning time to all PED physicians, rather than just junior doctors. This protected time could drive autonomous learning through workshops or practice resources. It also enhances proficiency and confidence in real-life procedures while providing opportunities to try novel treatment options and determine their reliability in various contexts. Complementary to this would be promoting self-directed learning. Making available various educational resources such as online modules provide opportunities to enhance and maintain skills outside of formal training, fostering a culture of life-long learning.

Finally, current workflows should be evaluated with a shuffle in resource allocation to minimise unnecessary administrative work while optimising availability for procedural skill practice. This includes streamlining documentation workflows, re-distribution of non-clinical work, or promoting work-life integration. Support services such as wellness programs and counselling can be extended and promoted. Through this, focus and concentration can be redirected toward performance.

E. Limitations

This study was conducted at a single academic centre with policies, workflows, and culture that may not be generalisable to other academic healthcare institutions. The voluntary nature of participants means that the findings presented might not include all faculty. Future studies can consider a larger and more diverse sample from several hospitals to take this into account. Having a comparison group of Adult Emergency Medicine (AEM) physicians or non-emergency medicine departments could be included in future studies to elicit unique challenges faced by PEM physicians.

V. CONCLUSION

Overall, the themes from the original study largely translate to and are applicable in KKH. SDT concepts are highly applicable in characterising motivation for PSM. Although the scope of autonomy differs, the resultant finding from the current study still utilises its principles. Additionally, the barriers to motivation and skill maintenance hold, allowing for the formulation of practical recommendations to complement the existing framework in KKH.

While findings may not be fully generalisable, they highlight important areas for improvement. Subjective and objective measures of competence highlight the need for evaluating and maintaining competence between various healthcare institutions. Understanding these could allow for policy development to ensure high fidelity of procedural skills. Future research should explore strategies to overcome these barriers and promote effective PSM among PEM physicians.

Notes on Contributors

IT reviewed the literature, conducted, and transcribed interviews, and wrote the manuscript. SG advised the design of the study and interview methods, gave critical feedback on the writing of the manuscript, and sent out invitations to the study. Both authors designed the study, developed the methodological framework, analysed transcripts, and read and approved the final manuscript.

Ethical Approval

This study was approved by the Ethics Committee of SingHealth IRB (Ref: 2022/2228). All participants have given written consent for their data to be used in the research and for publication.

Data Availability

It is not possible to share data since we did not obtain consent from participants to share all data collected. 

Acknowledgments

We thank the Emergency Department of KK Hospital for their participation.

Funding

This study is not funded by any organisations.

Declaration of Interest

The authors have no potential conflicts to disclose.

References

Baugh, S. G., & Scandura, T. A. (1999). The effect of multiple mentors on protege attitudes toward the work setting. Journal of Social Behavior and Personality, 14(4), 503. 

Bradbury-Jones, C., Breckenridge, J., Clark, M. T., Herber, O. R., Wagstaff, C., & Taylor, J. (2017). The state of qualitative research in health and social science literature: a focused mapping review and synthesis. International Journal of Social Research Methodology, 20(6), 627–645. https://doi.org/10.1080/13645579. 2016.1270583

Cabalatungan, S. N., Thode, H. C., & Singer, A. J. (2020). Emergency medicine physicians infrequently perform pediatric critical procedures: a national perspective. Clinical and Experimental Emergency Medicine, 7(1), 52–60. https://doi.org/ 10.15441/ceem.19.004

Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334–352. https://doi.org/10.1007/s10956-009-9180-4

Chen, G.-M., & Chung, J. (2002). Seniority and superiority: a case analysis of decision making in a Taiwanese religious group. Communication Studies Faculty Publications, 11(1), 41-56. https://web.uri.edu/iaics/files/03-Guo-Ming-Chen-Jensen-Chung.pdf

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11(6), 671–684. https://doi.org/10.1016/S0022-5371(72)80001-X

Crowe, S., Ewart, L., & Derman, S. (2018). The impact of simulation based education on nursing confidence, knowledge and patient outcomes on general medicine units. Nurse Education in Practice, 29, 70–75. https://doi.org/10.1016/j.nepr.2017.11.017

Deber, R. B. (1994). Physicians in health care management: 7. The patient-physician partnership: changing roles and the desire for information. CMAJ: Canadian Medical Association Journal, 151(2), 171–176.

Docherty-Skippen, S. M., Karrow, D., & Ahmed, G. (2020). Doing science: pre-service teachers’ attitudes and confidence teaching elementary science and technology. Brock Education: A Journal of Educational Research and Practice, 29(1), 24–34.

Eisenberg, J. M. (1985). Physician utilisation: the state of research about physicians’ practice patterns. Medical Care, 40(11), 1016-1035.

Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 79(10 Suppl), S70-S81. https://doi.org/10.1097/00001888-200410001-00022

Ericsson, K., Krampe, R., & Tesch-Roemer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406. https://doi.org/10.1037/ 0033-295X.100.3.363

Firdaus, A. R. (2018). Application of everyone is a teacher here learning model to improve self-confidence students. PrimaryEdu: Journal of Primary Education, 2(2), Article 2. https://doi.org/10.22 460/pej.v2i2.1007

Fischer, R. (2008). Rewarding seniority: Exploring cultural and organisational predictors of seniority allocations. The Journal of Social Psychology, 148(2), 167–186. https://doi.org/10.3200/SO CP.148.2.167-186

Gates, A. (2019). The protégé effect in the retention of underrepresented minority undergraduate teaching assistants in geoscience: preliminary indications from Newark, New Jersey. Journal of Geoscience Education, 67(4), 417–426. https://doi.org/ 10.1080/10899995.2019.1661760

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Beyond WEIRD: towards a broad-based behavioral science. Behavioral and Brain Sciences, 33(2–3), 111–135.  https://doi.org/10.1017/S0 140525X10000725

Jones, D. (2010). Psychology. A WEIRD view of human nature skews psychologists’ studies. Science, 328(5986), 1627. https://doi.org/10.1126/science.328.5986.1627

Kahlke, R. M. (2014). Generic qualitative approaches: pitfalls and benefits of methodological mixology. International Journal of Qualitative Methods, 13(1), 37–52. https://doi.org/10.1177/1609 40691401300119

Lin‐Martore, M., Kant, S., & O’Brien, B. C. (2021). Procedural skill maintenance: perspectives and motivations of pediatric emergency medicine faculty. AEM Education and Training, 5(4), Article e10696. https://doi.org/10.1002/aet2.10696

Mittiga, M. R., FitzGerald, M. R., & Kerrey, B. T. (2019). A survey assessment of perceived importance and methods of maintenance of critical procedural skills in pediatric emergency medicine. Pediatric Emergency Care, 35(8), 552–557. https://doi.org/10.1097/PEC.0000000000000991

National Academies of Sciences, E., Affairs, P., and G., Committee on Science, E., Information, B. on R. D. and, Sciences, D. on E. and P., Statistics, C. on A. and T., Analytics, B. on M. S. and, Studies, D. on E. and L., Board, N. and R. S., Education, D. of B. and S. S. and, Statistics, C. on N., Board on Behavioral, C., & Science, C. on R. and R. in. (2019). Replicability. In Reproducibility and Replicability in Science. National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK547524/

Nease, R. F., & Brooks, W. B. (1995). Patient desire for information and decision making in health care decisions: the autonomy preference index and the health opinion survey. Journal of General Internal Medicine, 10(11), 593–600. https://doi.org/10.1007/BF02602742

Perez, R. S., Skinner, A., Weyhrauch, P., Niehaus, J., Lathan, C., Schwaitzberg, S. D., & Cao, C. G. L. (2013). Prevention of surgical skill decay. Military Medicine, 178(10 Suppl), 76–86. https://doi.org/10.7205/MILMED-D-13-00216

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55(1), 68-78. https://doi,org/10.1037//0003-066x.55.1.68

Trowbridge, M. H., & Cason, H. (1932). An experimental study of Thorndike’s theory of learning. Journal of General Psychology, 7(2), 245–260. https://doi.org/10.1080/00221309.1932.9918465

Yang, C.-W., Yen, Z.-S., McGowan, J. E., Chen, H. C., Chiang, W.-C., Mancini, M. E., Soar, J., Lai, M.-S., & Ma, M. H.-M. (2012). A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation, 83(9), 1055–1060. https://doi.org/10.1016/j.resusci tation.2012.02.027

*Tan Jia Le Ivan
School of Biological Sciences
Nanyang Technological University
50 Nanyang Ave
Singapore 639798
Email address: B200021@e.ntu.edu.sg

Submitted: 1 September 2023
Accepted: 29 January 2024
Published online: 2 July, TAPS 2024, 9(3), 32-40
https://doi.org/10.29060/TAPS.2024-9-3/OA3127

Michiko Goto¹, Ryota Sakamoto², Hideki Wakabayashi³ & Yousuke Takemura⁴

¹Department of General Medicine, Mie University School of Medicine, Japan; ²Department of Medical Informatics, Mie University Hospital, Japan; ³Department of Community Medicine, Mie University School of Medicine, Japan; ⁴Department of General Medicine, Tokyo Women’s Medical University, Japan

Abstract

Introduction: From the late 1960s to the present, physicians’ dress codes have been actively studied in Western countries. Until the early 21st century, patients tended to prefer a conservative dress style, such as “shirt and tie or skirt” with white coats for physicians. However, as attitudes toward dress codes have changed, knowledge regarding this issue needs to be updated. A variety of colours of scrubs are currently commonly used by medical professionals, but it is not known whether all colours are acceptable to patients. The current study sought to investigate the acceptability of various dress codes for physicians from the patients’ perspective, to inform medical education.

Methods: Outpatients and their family members at a university hospital and a small-to-medium-sized hospital were surveyed. We inquired about which of the different styles of white coats and different colours of scrubs were most desirable for male and female physicians. We used Scheffe’s paired comparison method to determine rankings.

Results: Patients and their family members expected their physicians to wear white coats rather than scrubs. Furthermore, a more traditional and formal dress code was preferred. The least preferred colour of scrubs was yellow.

Conclusion: The current results indicated that patients’ preference for a traditional, conservative appearance has not changed over time. This finding does not match current perspectives on infection prevention. Both patient preferences and infection prevention are important for informing education and gaining patient trust.

Keywords:           White Coat, Scrub Colour, Physicians’ Appearance, Medical Education, Doctor-Patient Communication

Practice Highlights

• The physician’s traditional white coat may be associated with a sense of trust, and is most preferred by patients and their family members. This trend has not changed over time.
• Among the scrub colours, bright colours are not preferred, and black and red may have a negative meaning for patients and their family members.
• Regarding dress code education, patient/family perspectives, infection prevention, and sociocultural background should all be considered.

Figure 1. Scrubs in 15 different colours

1: black, 2: grey, 3: light blue, 4: light purple, 5: dark blue, 6: blue-green, 7: dark blue-green, 8: dark green, 9:  yellow, 10: orange, 11: brown, 12: pink, 13: dark pink, 14: red, 15: red-purple

Figure 2. Nine types of attire for mature and young male physicians, each in order of patient preference with scale chart of average degree of preference

*The yard stick values were Y_0.05=0.052 for young men and 0.054 for mature men

A total of 264 individuals reported 354 images showing unacceptable appearances, of which 40% showed the physician wearing scrubs (Table 1).

3) Young women: The descending order of preference for images of young women was as follows: ① white coat, ③ sandals, ② open-front white coat, ④ mask, ⑤ rolled-up sleeves, ⑥ scrubs + white coat, and ⑦ scrubs (Figure 3). Moreover, ① white coat and ③ sandals formed the top group; ② open-front white coat and ④ mask formed the middle group; and ⑤ rolled-up sleeves, ⑥ scrubs + white coat and ⑦ scrubs formed the lower group. There were no significant differences between the groups (Figure 3).

A total of 403 individuals reported 535 images showing unacceptable appearances, of which 57% featured rolled-up sleeves and 33% featured scrubs (Table 1).

Table 1. The characteristics of images that were identified as showing an unacceptable appearance by patients and their family members, and the percentage of respondents that deemed the image unacceptable

4) Mature women: The descending order of preference for images of mature women was as follows: ① white coat, ③ sandals, ④ mask, ② open-front white coat, ⑦ scrubs, ⑤ rolled-up sleeves, and ⑥ scrubs + white coat (Figure 3). There was no significant difference between the components ① white coat and ③ sandals in the top group. Moreover, ① white coat and ③ sandals were components of the top group, similar to the case for images of young women, and there was no significant difference between them (Figure 3).

Figure 3. Seven types of attire for mature and young female physicians, each in order of patient preference with scale chart of average preference

*The yard stick values are Y_0.05=0.054 for young women and 0.061 for mature women

A total of 172 individuals reported 222 images showing unacceptable appearances, of which 48% featured scrubs (Table 1).

We examined the trends by participants’ gender and age. The results revealed no differences between men and women and in each age group. However, participants over 70 years old tended to prefer tie + closed- and open-front white coat compared with participants under 70, and tended not to favour female doctors with rolled-up sleeves.

B. Question 2

1) Scrub colour: The colours and percentages of scrubs that were identified as unacceptable were, in descending order, as follows: 52%, yellow; 46%, red; 42%, dark pink; 33%, pink; 27%, black; 23%, orange; 20%, blue-green; 16%, grey; 17%, red-purple; 14% brown; 11%, dark blue-green; 8%, light blue; 7%, dark green; 6%, light purple; and 4%, dark blue (Figure 4).

Figure 4. Percentage of scrub colours not preferred by patients and their family members.

IV. DISCUSSION

Many patients and their family members expected their physicians to wear white coats rather than scrubs. On average, a traditional and formal dress style was most preferred by patients and their family members (buttoned white coat and tie for men, leather shoes, and buttoned white coat for women). In a 2010 study conducted in Japan by Yamada et al., the most favoured formality attire by patients was white coats (but skirts for women). Pronchik, who investigated the benefits of neckties prior to the BBE policy (King & Infection Prevention and Control Nurse, 2022), concluded that in an emergency room in the United States (US), patients preferred their physicians to wear ties, and patient satisfaction was high (Pronchik et al., 1998). The survey found that people in older age groups in particular preferred doctors to wear ties.

The results suggest that patients’ preferences regarding physicians’ appearance have not changed substantially. One study found that patients in the United Kingdom (UK) who were briefed on the BBE policy felt that conservatively dressed doctors looked more professional (Toquero et al., 2011).

The second-most preferred attire for male physicians was the Casey, followed by the white coat with rolled-up sleeves for younger men. However, this attire was not ranked highly for women or mature men. This indicates that patients perceived the Casey as a traditional style of dress for men, and that the preference was not necessarily based on the prevalence of the BBE concept or concerns about cleanliness. The pros and cons of rolling up the sleeves of white coats are often discussed in medical student dress guidance, including in a study by Bond (Bond et al., 2010). In the current study, rolled-up sleeves were not preferred by patients. However, it is known that the sleeves of white coats can become contaminated (Banu et al., 2012), and Wong et al. pointed out that the risk of contamination may be increased by pathogens in ties, cuffs, and pockets (Wong et al., 1991). The current results indicated that patients in their 70s and older were not comfortable with female doctors rolling up the sleeves of their white coats. Although this finding may be related to cultural factors, to the best of our knowledge, this phenomenon has not been previously reported in the literature. The results suggest that patients’ concepts of professionalism and infection prevention are not directly connected. When instructing students about dress code, they should be told that patients may not approve of rolling up their sleeves.

Men wearing masks were rated less favourably, while women wearing masks were not rated less favourably. For women, there may be something to compensate for the facial expressions hidden by masks. Because wearing a mask and other prophylactic devices has been essential for physicians since the beginning of the COVID-19 pandemic, the impressions of patients and family members should be examined in future studies.

Although scrubs are often rated as hygienic (Bond et al., 2010; Lightbody & Wilkie, 2013), the current survey revealed that scrubs were not rated as a preferred attire for physicians by patients and their family members. In the survey by Aitkin et al. mentioned above, scrubs also received the lowest ratings. However, previous studies in the US reported no influence of scrubs and other types of attire on patient satisfaction and perceptions of professionalism (Fischer et al., 2007; Li & Haber, 2005). Additionally, a recent survey in the UK reported a clear downward trend in resistance to scrubs, with a survey by Palazzo and Hocken reporting that white coats and ties are no longer expected by patients (Palazzo & Hocken, 2010). In a study in the US, patients undergoing surgery reported that scrubs were most favoured in outpatient settings (Edwards et al., 2012). These findings suggest that the resistance to scrubs in Japan may disappear in the near future.

Brightly coloured (yellow, red, and pink) and black scrubs were considered to be less acceptable than others, and those in pale and cold colours, such as dark blue and light blue, were preferred. This may have occurred because red and black are associated with negative emotions such as anger, anxiety and fear, whereas cold colours are associated with calm and quiet (Oyama et al., 1963). Some patients commented that black reminded them of death and red reminded them of blood. Bright colours may be irritating to patients. To the best of our knowledge, no previous studies have examined colour preferences for scrubs among patients, suggesting that these findings are novel.

In the 20th century, physicians in many countries were required to wear white coats (Gooden et al., 2001; Harnette, 2001). It is not surprising that preferences regarding physicians’ appearance have changed in the 21st century, as many people have started to become more familiar with the threat of infection and changes in the global medical environment. However, the concept of BBE is still not pervasive, and the appearance of attire required to be worn by physicians has not changed significantly. The current results are consistent with the findings of the largest study of this issue conducted in the UK (Jacob, 2007), which reported the following: “if there is deemed to be no significant infection risk from any given variation of workwear, our patients would like us to wear a shirt and tie.” Patients’ awareness regarding infection needs to be investigated, but patients will accept physicians’ suggestions if they understand the need for them (Collins et al., 2013).

Shelton et al. conducted an experiment in the UK to reduce cross-infection between doctors and patients, and reported that there was no significant difference in patient preferences between different types of attire before explaining the importance of clothing to patients; however, after the explanation, scrubs and short-sleeved shirts were most preferred (Shelton et al., 2010). When considering physicians’ dress code, we need to understand both the patient’s preferences and infection control factors. We propose that a dress code should be developed that does not compromise patients’ trust and confidence, but also takes safety into consideration.

A. Limitations

Although the types of clothing shown in the photos in the current study were based on a prior survey, it may not have been comprehensive. Moreover, using different models in the photographs may have influenced the results, and the potential effects of measurement bias cannot be excluded. It is unclear from the current findings why certain appearances were preferred or deemed unacceptable. Furthermore, the current study did not examine doctors’ preferences. Medical practitioners’ preferences need to be taken into account when making workplace attire choices in hospitals. Further research will be needed to identify the preferred attire for both patients and doctors.

V. CONCLUSION

The current findings indicated that patients exhibited a preference for physicians dressed in traditional attire. Even though times have changed, people may still associate trust, credibility, and respect with the formal appearance of their physicians. The current findings also suggested that patients were not aware of the BBE policy. The results of this study may be helpful for informing teaching approaches regarding the appearance of medical students and residents.

Notes on Contributors

MG developed the research idea and design with YT. The data collection was performed by MG. The data were analysed by RS. HW performed the data interpretation with MG. MG wrote the article with revision by HW. All the authors read and agreed with the final manuscript.

Ethical Approval

Verbal informed consent was obtained from all participants for publication and this procedure was approved by the Mie University Ethics Committee. The Ethical Review Committee of Mie University committee approved this study (No. 1237). All methods were performed in accordance with the relevant guidelines and regulations.

Data Availability

The data that support the findings of this RCT study are openly available at a Figshare repository, https://doi.org/10.6084/m9.figshare.23936379.v1

Acknowledgements

We would like to express our heartfelt gratitude to the models for their cooperation in creating the photograph stimuli, and to Goto F, Makita H, Yin M, Kasyo K, Sakaguchi R, Masukawa E, Tsunoda K, Shimada K, and Tanaka K, for collecting the data. We thank Benjamin Knight, MSc., from Edanz (https://edanz.com) for editing a draft of this manuscript. This paper has been preprinted. M Goto et al. What dress code do we teach students and residents? A survey of patients’ and their families’ preferences regarding physicians’ appearance. 23 Mar, 2022（Version 1）available at Research Square (https://doi.org/10.21203/rs.3.rs-1430222/v1).

Funding

This study was supported by research grants from the Kameyama City Department of Community Medicine [No. J12061L005] and the Tsu City Department of Community Medicine [No. J12061L008].

Declaration of Interest

No conflict of interest, financial or otherwise, exists.

References

Aitken, S. A., Tinning, C. G., Gupta, S., Medlock, G., Wood, A. M., & Aitken, M. A. (2014). The importance of the orthopaedic doctors’ appearance: A cross-regional questionnaire-based study. The Surgeon, 12(1), 40-46. https://doi.org/10.1016/j.surge.2013. 07.002

Au-Yeung, P. K. K. (2005). SARS changed medical dress code. BMJ, 330(7505), 1450. https://doi.org/10.1136/bmj.330.7505.14 50-b

Banu, A., Anand, M., & Nagi, N. (2012). White coats as a vehicle for bacterial dissemination. Journal of Clinical and Diagnostic Research, 6(8), 1381-1384. https://doi.org/10.7860/JCDR/2012/42 86.2364

Baxter, J. A., Dale, O., Morritt, A., & Pollock, J. C. (2010). Bare below the elbows: Professionalism vs infection risk. The Royal College of Surgeons of England Bulletin, 92(7), 248-251. https://doi.org/10.1308/147363510X510581

Blumhagen, D. W. (1979). The doctor’s white coat. The image of the physician in modern America. Annals of Internal Medicine, 91(1), 111-116. https://doi.org/10.7326/0003-4819-91-1-111

Bond, L., Clamp, P. J., Gray, K., & Van Dam, V. (2010). Patients’ perception of doctors clothing: Should we really be ‘bare below the elbow’? The Journal of Laryngology and Otology, 124(9), 963-966. https://doi.org/10.1017/s0022215110001167

Brandt, L. J. (2003). On the value of old dress code in the new millennium. Archives of Internal Medicine, 163(11), 1277-1281. https://doi.org/10.1001/archinte.163.11.1277

Collins, A. M., Connaughton, J., & Ridgway, P. F. (2013). Bare below the elbows: A comparative study of a tertiary and district general hospital. Irish Medical Journal, 106(9), 272-275. 

Edwards, R. D., Saladyga, A. T., Schriver, J. P., & Davis, K. G. (2012). Patient attitudes to surgeons’ attire in outpatient clinical setting: Substance over style. American Journal of Surgery, 204(5), 663-665. https://doi.org/10.1016/j.amjsurg.2009.09.001

Fischer, R. L., Hansen, C. E., Hunter, R. L., & Veloski, M. S. (2007). Does physician attire influence patient satisfaction in an outpatient obstetrics and gynecology setting? American Journal of Obstetrics and Gynecology, 196(2), e1-e5. https://doi.org/10.1016/ j.ajog.2006.09.043

Gherardi, G., Cameron, J., West, A., & Crossley, M. (2009). Are we dressed to impress? A descriptive survey assessing patient’s preference of doctors’ attire in the hospital setting. Clinal Medicine, 9(6), 519-524. https://doi.org/10.7861/clinmedicine.9-6-519

Gjerdingen, D. K., Simpson, D. E., & Titus, S. L. (1987).  Patients’ and physicians’ attitudes regarding the physician’s professional appearance. Archives of Internal Medicine, 147(7), 1209-1212.

Gledhill, J. A., Warner, J. P., & King, M. (1997). Psychiatrists and their patients: Views on forms of dress and address. The British Journal of Psychiatry, 171, 228-232. https://doi.org/10.1192/bjp. 171.3.228

Gooden, B. R., Smith, M. J., Tattersall, S. J. N., & Stockler, M. R. (2001). Hospital patients’ views on doctors and white coat. The Medical Journal of Australia, 175(4), 219–222. https://doi.org/10.5694/j.1326-5377.2001.tb143103.x

Harnette, P. R. (2001). Should doctors wear white coat? The Medical Journal of Australia, 174(7), 343–344. https://doi.org/10.5694/j.1326-5377.2001.tb143310.x

Hennessy, N., Harrison, D. A., & Aitkenhead, A. R. (1993). The effect of the anaesthetist’s attire on patient attitudes. The influence of dress on patient perception of the anaesthetist’s prestige. Anesthesia, 48(3), 219-222. https://doi.org/10.1111/j.1365-2044 1 993.tb06905.x

Hippocrates. (1923). Hippocrates (W. H. S. Jones, Trans.) (Vol. 2). Harvard University Press.

Ikusaka, M., Kamegai, M., Sunaga, T., Narita, N., Kobayashi, H., Yonenami, K., & Watanabe, M. (1999). Patients’ attitude toward consultations by a physician without a white coat in Japan. Internal Medicine, 38(7), 533–536. https://doi.org/10.2169/internalmedici ne.38.533

Jacob, G. (2007). Uniform and workwear: An evidence base for developing local policy. National Health Service. https://data.parliament.uk/DepositedPapers/Files/DEP2009-0656/DEP2009-0656.pdf

Keenum, A. J., Wallace, L. S., & Stevens, A. R. (2003). Patients’ attitude regarding physical characteristics of family practice physicians. Southern Medical Journal, 96(12), 1190-1194. https://doi.org/10.1097/01.smj.0000077011.58103.c1

Keio University Hospital. (2014, May 20). Cool Biz no jisshi ni tsuite [About the implementation of Cool Biz]. Keio University Hospital. https://www.hosp.keio.ac.jp/oshirase/hosp/detail/38033/

King, C., & Infection Prevention and Control Nurse. (2022). Hand hygiene policy: Including bare below the elbows (Ver.9). Leicestershire Partnership National Health Service Trust. https://www.leicspart.nhs.uk/wp-content/uploads/2022/06/Hand-Hygiene-Policy-inc-bare-below-the-elbows.pdf

Kurihara, H., Maeno, T., & Maeno, T. (2014). Importance of physicians’ attire: factors influencing the impression it makes on patients, a cross-sectional study. Asia Pacific Family Medicine, 13(1), 2. https://doi.org/10.1186/1447-056x-13-2

Li, S. F., & Haber, M. (2005). Patient attitude toward emergency physician attire. The Journal of Emergency Medicine, 29(1), 1-3. https://doi.org/10.1016/j.jemermed.2004.12.014

Lightbody, K. A., & Wilkie, M. D. (2013). Perceptions of doctors’ dress code: ENT patients’ perspective. Clinical Otolaryngology, 38(2), 188-190. https://doi.org/10.1111/coa.12084

Menahem, S., & Shvaretzman, P. (1998). Is our appearance important to our patients? Family Practice, 15(5), 391-397. https://doi.org/10.1093/fampra/15.5.391

Ministry of the Environment. (2017, April 25). Heisei 29 nendo Cool Biz ni tsuite [About Cool Biz in 2017]. Ministry of the Environment, Japan. https://www.env.go.jp/press/103993.html

Nair, B. R., Mears, S. R., Hitchcock, K. I., & Attia, J. R. (2002). Evidence-based physicians’ dressing: A crossover trial. The medical Journal of Australia, 177(11-12), 681-682. https://doi.org/10.5694/j.1326-5377.2002.tb05017.x

Neinstein, L. S., Stewart, D., & Gordon, N. (1985).  Effect of physician dress style on patient-physician relationship. Journal of Adolescent Health Car, 6(6), 456-459.  https://doi.org/10.1016/S0 197-0070(85)80053-X

Nishihara, N., Haneda, M., & Tanabe, S. (2010). A study of office workers’ clothes and subjective evaluations in an office with a preset air-conditioning temperature of 28°C in summer. Journal of Home Economics of Japan, 61(3), 169-175. https://doi.org/10.114 28/jhej.61.169

Nishina, D., Murakawa, S., & Uemura, Y. (2007). A Study on the Effects of the Cool Biz introduced into the Regional Joint Government Buildings in Hiroshima. Technical papers of annual meeting, the Society of Heating, Air-Conditioning and Sanitary Engineers of Japan. 1463-1466. https://doi.org/10.18948/shasetai kai.2007.2.0_1463

Oyama, T., Tanaka, Y., & Haga, J. (1963). Color-affection and color-symbolism in Japanese and American students. The Japanese Journal of. Psychology, 34(3), 109-121. https://doi.org/ 10.4992/JJPSY.34.109

Palazzo, S., & Hocken, D. B. (2010). Patients’ perspectives on how doctors dress. The Journal of Hospital Infection, 74(1), 30-34. https://doi.org/10.1016/j.jhin.2009.08.021

Pronchik, D. J., Sexton, J. D., Melanson, S. W., Patterson, J. W., & Heller, M. B. (1998). Does wearing a necktie influence patient perceptions of emergency department care? The Journal of Emergency Medicine, 16(4), 541-543.  https://doi.org/10.1016/s07 36-4679(98)00036-5

Rehman, S. U., Nietert, P. J., Cope, D. W., & Kilpatrick, A. O. (2005). What wear today? Effect of doctor’s attire on the trust and confidence of patient. The American Journal of Medicine, 118(11), 1279-1286. https://doi.org/10.1016/j.amjmed.2005.04.026

Shelton, C. L., Raistrick, C., Warburton, K., & Siddiqui, K. H. (2010). Can changes in clinical attire reduce likelihood of cross-infection without jeopardizing the doctor-patient relationship? The Journal of Hospital Infection, 74(1), 22-29. https://doi.org/10.10 16/j.jhin.2009.07.031

Takemura, Y., Atsumi, R., & Tsuda, T. (2008). Which medical interview behaviours are associated with patient satisfaction? Family Medicine, 40(4), 253-258. https://pubmed.ncbi.nlm.nih.gov/18382837/

Toquero, L., Aboumarzouk, O., Owers, C., Chiang, R., Thiagarajah, S., & Amin, S. (2011, April 02). Bare below the elbows-The patient’s perspective. Webmed Central. WebmedCentral. https://www.webmedcentral.com/article_view/1401

Wong, D., Nye, K., & Hollis, P. (1991). Microbial flora on doctors’ white coats. British Medical Journal, 303(6817), 1602-1604. https://doi.org/10.1136/bmj.303.6817.1602

Yamada, Y., Takahashi, O., Ohde, S., Deshpande, G. A., & Fukui, T. (2010). Patient’s preference for doctor’s attire in Japan. Internal Medicine, 49(15), 1521–1526. https://doi.org/10.2169/internalme dicine.49.3572

*Hideki Wakabayashi
Department of Community Medicine,
Mie University School of Medicine,
2-174, Edobashi,
Tsu, Mie,
514-8507, Japan
+81-59-231-5290
Email address: hidekiwaka@med.mie-u.ac.jp

Submitted: 8 August 2023
Accepted: 9 January 2024
Published online: 2 July, TAPS 2024, 9(3), 50-54
https://doi.org/10.29060/TAPS.2024-9-3/SC3107

Chuu Ling Chan¹, Russell Lee², Lih Ing Goh¹, Nathanael Hao Kai Chong¹, Li Neng Lee² & Jun-Hong Ch’ng¹

¹Department of Microbiology and Immunology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Psychology, Faculty of Arts and Social Sciences, National University of Singapore, Singapore

Abstract

Introduction: We frequently associate microbes with infection, rarely expounding on their usefulness and importance to healthy development. For humanity to leverage these microbial “super powers”, learners from all backgrounds need to appreciate their utility and consider how microbes could help solve some of the most critical problems we face. However, learners are frequently uninterested or intimidated by microbiology. The card game “No Guts No Glory” was created to engage students by piquing curiosity and encouraging informal learning to change perceptions and advocate the value of microbes to good health.

Methods: Undergraduates from various faculties in the National University of Singapore were invited to play and give feedback on accessibility, engagement and self-reported learning gains.

Results: The game was well-received across disciplinary backgrounds with positive feedback (5-point scale) on game mechanics being fun (4.17±0.63), attractive artwork (3.83±1.00) and scientific snippets (3.79±1.04), positive re-playability (3.46±0.84), player engagement for those without foundational knowledge in microbiology (3.63±1.04), and usefulness of knowledge taught (3.54±1.10). Areas for improvement evidenced from feedback included unclear instructions (2.74±0.98), limited content taught (2.76±0.93), not generating interest to attend formal microbiome classes (2.88±1.17) and not prompting lifestyle changes (1.98±1.11).

Conclusion: This pilot study provided valuable insights from the target demographic, with concrete ideas on how to improve the educational potential of “No Guts No Glory”. Findings further lay the groundwork for the design of future instruments to objectively quantify learning gains from gameplay.

Keywords:           Game-based Learning, Microbiome, Microbiology, Card Game, No Guts No Glory

I. INTRODUCTION

Though responsible for infection and disease, microbes are also necessary for promoting and maintaining good health and are remarkably useful in many industries. Microbes are crucial and versatile tools which can be used to tackle some of the world’s most complex problems ranging from pandemics and climate change to sustainable foods and environmental remediation (Akinsemolu, 2018). Recognising the true potential of microbes could encourage their use in impactful problem-solving. However, students may perceive microbiology as a difficult subject and not associate microbes with anything positive.  To address this, we developed the card game “No Guts No Glory” that focuses on the positive aspects of microbes, particularly in the context of the gut microbiome which is closely tied to many aspects of our health and development (Lynch & Pedersen, 2016). We sought to engage learners from diverse disciplines by reducing the psychological barrier to learning microbiology, sparking curiosity and encouraging self-directed exploration.

Previous studies on card games, including one involving immunology (a related and equally complicated subject), have demonstrated the benefits of game-based learning (Barnes, 2022; Su et al., 2014): (a) games provide an attractive, innocuous entry point for individuals to explore complex subjects in a fun and enjoyable manner; (b) they remove barriers associated with traditional learning approaches, making it easier for learners to get started and actively participate; (c) additionally, game-based learning promotes active engagement; (d) through interactive gameplay, players become immersed in theoretical concepts which fosters deeper understanding of the subject and enhances retention.

In this study, we examined the impact of playing “No Guts No Glory” on participants’ interest in microbiology and garnered feedback for optimising game mechanics, instructions and artwork in preparation for future studies on the game’s impact on learning gains.

II. METHODS

Undergraduate students from various academic backgrounds were invited to participate in this pilot study on version 1 of the microbiome card game “No Guts No Glory”. Students were briefed on the details of the study and implied consent was given with their participation. Documentation of informed consent was waived as the collection of personal, sensitive information was kept to a minimum. A total of 41 participants were recruited – 29% from Medicine, 46% from Life Sciences, 20% from Psychology and 5% from other majors. Although not enforced, most participants played the game with others from the same discipline due to the way participants were recruited and how gameplay sessions were organised.

Participants first read through the game instructions and provided feedback on the instructions before playing two rounds of the game in groups of three or four. During the first round of gameplay, participants discussed and negotiated the rules among themselves based on what they had read, without input from the session facilitator. There was a break between the two rounds when the session facilitator highlighted deviations from the intended gameplay and answered questions about the game, before proceeding to the second round of gameplay. Participants then gave feedback via Qualtrics on their experience with the game. Throughout the two rounds of gameplay, the session facilitator also recorded observations of questions about instructions, disagreements about rules, deviations in gameplay and comments about game mechanisms, artwork, experiences and learning gains.

For quantitative feedback, participants rated the clarity and presentation of the instructions, the accessibility of the game, player engagement level and perceived learning gains, based on 5-point rating scales. Open-ended qualitative questions included: 1) suggestions to improve the instructions or the game, 2) elaboration on likely lifestyle/behavioural changes after playing and 3) key ideas they had learnt about the gut microbiome.

III. RESULTS

Quantitative feedback from participants after playing “No Guts No Glory” is summarised in Table 1 and qualitative feedback (individual comments and suggestions) is accessible at https://doi.org/10.6084/m9.figshare.23735211

Table 1. Participants’ quantitative feedback (5-point rating scales) on the microbiome card game “No Guts No Glory”

A. Instructions

Participants rated the ease of understanding and the presentation of the instructions near average (2.74±0.98 and 3.11±1.09 respectively). Qualitative feedback on the instructions revealed that many participants felt uncertain of game components, rules and mechanics (19 participants), that phrasing and overall wordiness of the instructions could be improved (14 participants), and that the instructions lacked emphasis on the goals of the game (3 participants). They suggested the need for more examples, visuals or even a demonstration video (10 participants) and reformatting the instructions (3 participants).

B. Game Accessibility and Player Engagement

Participants agreed that the length of one game was just right (3.02±0.42) and felt that the game was fun (4.17±0.63). They also had a good impression of the artwork (3.83±1.00) and scientific snippets included on the game cards (3.79±1.04), although 7 participants did not manage to read these snippets. Most participants responded positively regarding the replayability of the game, with a likelihood of playing the game if they owned it scoring 3.46±0.84, at an average frequency of 2.80±0.8. They perceived that players without any prior microbiome knowledge would be able to play and find the game engaging (3.63±1.04). Suggestions were given to enhance the game by changing the game mechanics to enrich learning (17 participants), improving the quality of game components (7 participants), including visual aids in the instructions (6 participants), and refining the artwork aesthetics (4 participants).

C. Experiences and Learning Gains

Gameplay helped participants to recognise the usefulness of microbiome knowledge (3.54±1.10), and to a lesser degree, connect topics (3.05±0.92) and raise curiosity (3.00±1.02). Participants who indicated more prior knowledge reported that the game was moderately effective in reinforcing existing content (3.19±1.05) while those with less prior knowledge indicated that the game was less effective in teaching content (2.76±0.93). In the qualitative feedback, suggestions for improving learning gains involved linking the scientific snippets found on the cards to gameplay (4 participants), and including a quiz component in the game mechanics (3 participants). Participants showed little interest in taking formal microbiome classes after gameplay (2.88±1.17), and only half (51.2%) indicated potential behavioural or lifestyle changes: 14 mentioned changes in their diet, two mentioned an increased curiosity in microbiome-related topics while one reflected on blindly chasing lifestyle/dietary trends.

D. Key Lessons Learnt by Students from Playing the Game

Drawing on open-ended feedback from participants, the top three ideas drawn from gameplay were the importance of the gut microbiome in health (8 out of 36 responses, 22%), how our microbiome is affected by multiple factors (19.4%), and the importance and definitions of pre/pro/syn-biotics (16.6%).

IV. DISCUSSION

Based on the feedback received, the game was enjoyable, with the inclusion of both attractive artwork and intriguing scientific snippets being crucial in generating interest. Participants acknowledged the value of the information presented in the game, which could inspire them to further explore specific areas of interest on their own. Participants also agreed that the game’s entry barrier was low enough, such that even individuals without a microbiology background could comprehend the gameplay and find it engaging. Positive replayability of the game could aid learning through increased exposure to information on the cards and instructions. Notably, the game’s effectiveness in reinforcing existing knowledge received higher ratings compared to its ability to teach new content in this study, aligning with similar findings published by others (Spandler, 2016; Su et al., 2014). While positive, these outcomes need to be viewed with caution as some of the students were likely to have been from classes taught by the researchers. Although feedback was anonymous and collected in the absence of their teachers, we acknowledge that this student-teacher/researcher relationship may have inadvertently introduced bias in this study.

This study also revealed four shortcomings of the game: 1) unclear instructions, 2) ineffectiveness at teaching new content, 3) generating little interest to enrol in microbiology classes and 4) low possibility of changing lifestyles. The latter three points allude to current game mechanics being ineffective in highlighting the relevance of microbiomes to students’ lives. To assess the concordance between the game’s intended learning objectives and students’ actual learning outcomes, self-reported key takeaways were examined. Although many of the mentioned themes corresponded to the learning objectives that were established during the game development phase, it was evident that certain learning objectives were insufficiently emphasised.

With valuable suggestions provided by participants to enhance learning impact, we anticipate that the revised game (version 2), which further integrates learning outcomes with game mechanics, will better showcase the importance and relevance of microbiomes. Furthermore, student insights from the current study have facilitated the development of assessment tools for quantifying learning gains in future studies through pre- and post-play testing methodologies. Also, since “No Guts No Glory” emphasises the beneficial impact of microbes on our health, future studies could compare the impact of this game to others which emphasise disease-causing pathogens and antimicrobial resistance, especially in how they shape perceptions about microbes.

V. CONCLUSION

In conclusion, our pilot study of “No Guts No Glory” highlighted its strengths in terms of accessibility and player engagement. However, it also brought to attention areas requiring improvement. These include simplifying the instructions to enhance clarity, ensuring that learning is effectively integrated with gameplay and better aligning game mechanics with the science.

Furthermore, we have identified key learning outcomes from unsupervised gameplay which will guide the development of assessment instruments in future studies, via pre- and post-play testing. Such testing will allow us to evaluate learning gains more effectively in subsequent studies involving both microbiology and non-microbiology undergraduates.

Notes on Contributors

Chuu Ling Chan was involved in conceptualisation, methodology, project administration and supervision, data analysis and writing (original draft and editing).

Russell Lee was involved in conceptualisation, methodology, project administration and supervision, data analysis and writing (editing).

Lih Ing Goh was involved in methodology, project administration and supervision, data analysis and writing (editing).

Nathanael Hao Kai Chong was involved in project administration and supervision.

Li Neng Lee was involved in conceptualisation, methodology and writing (editing).

Jun-Hong Ch’ng was involved in conceptualisation, methodology, data analysis and writing (editing).

Ethical Approval

This study was reviewed by the Learning and Analytics Committee on Ethics- Departmental Ethics Review Committee (LACE-DERC) from the National University of Singapore (NUS) Institute for Applied Learning Sciences and Educational Technology (ALSET) and Centre for Development of Teaching & Learning (CDTL), with an exemption from IRB review and the approval to conduct research at NUS (LACE Reference Code: L2021-12-01).

Data Availability

Qualitative study data can be accessed at https://doi.org/10.6084/m9.figshare.23735211.

Acknowledgement

We would like to acknowledge all the students who participated in this study.

Funding

This study is funded by the Teaching Enhancement Grant (TEG FY2023) from the Centre for Development of Teaching and Learning (CDTL), National University of Singapore (E-571-00-0001-01).

Declaration of Interest

The authors declare no conflicts of interest.

References

Akinsemolu, A. A. (2018). The role of microorganisms in achieving the sustainable development goals. Journal of Cleaner Production, 182, 139–155. https://doi.org/10.1016/j.jclepro.2018. 02.081 

Barnes, R. L. (2022). A protein purification card game develops subject knowledge and transferable skills. Journal of Biological Education, 56(4), 365–375. https://doi.org/10.1080/00219266.20 20.1799844

Lynch, S. V., & Pedersen, O. (2016). The human intestinal microbiome in health and disease. New England Journal of Medicine, 375(24), 2369–2379. https://doi.org/10.1056/NEJMra16 00266

Spandler, C. (2016). Mineral supertrumps: A new card game to assist learning of mineralogy. Journal of Geoscience Education, 64(2), 108–114. https://doi.org/10.5408/15-095.1

Su, T., Cheng, M.-T., & Lin, S.-H. (2014). Investigating the effectiveness of an educational card game for learning how human immunology is regulated. CBE Life Sciences Education, 13(3), 504–515. https://doi.org/10.1187/cbe.13-10-0197

*Jun-Hong Ch’ng
MD4, 5 Science Drive 2,
Singapore 117545
Email: micchn@nus.edu.sg

Submitted: 9 September 2023
Accepted: 29 January 2024
Published online: 2 July, TAPS 2024, 9(3), 55-57
https://doi.org/10.29060/TAPS.2024-9-3/PV3134

Sean B Maurice

Northern Medical Program, Division of Medical Sciences, University of Northern British Columbia, Prince George, Canada; Cellular and Physiological Sciences, Faculty of Medicine, University of British Columbia, Vancouver, Canada

I. INTRODUCTION

Risk management is a skillset that is embedded within clinical practice. Clinicians use protective equipment to safeguard themselves from pathogens carried by patients, learn de-escalation techniques to manage violent patients, and learn to ask for help. Patients of course are also at risk, because they present with illness or injury that may get worse despite our best efforts, and because there’s always a chance of iatrogenic injury or disease. Healthcare providers dedicate themselves to healing injury and illness, and to not causing further harm. In contrast, risk management is rarely considered with regards to teaching and learning, as they are not commonly understood to involve risk. When a teaching or learning experience feels risky, but we don’t think it should, then we don’t talk about it, and this can create cognitive dissonance and discourage us from engaging in teaching and learning.

Health systems around the world need more healthcare providers, at a time when they are dealing with very significant burnout (Office of the Surgeon General, 2022). In the face of this burnout, we may not be inclined to innovate, but Eva (2022) encourages us to step up and embrace the opportunity to make change, as during challenging times, “… it is critical that every stakeholder in the medical profession strive for excellence by adapting to modern realities rather than clinging to the seemingly safe status quo.” If we need to train more healthcare providers at a challenging time, when not making change is potentially more dangerous than innovating, then we should consider the metaphor of adventure to inspire our innovations and help us think about risk management in teaching and learning.

II. RISKS IN OUTDOOR ADVENTURE

In self-propelled outdoor adventure (hiking, mountaineering, kayaking), risks are largely obvious, and consequences can be very serious, so talking about risk management is both natural and normalised. Participants engage in ‘calculated risk-taking,’ which involves identifying risks and managing them as well as possible to keep the risk at a level considered reasonable by all who partake. Prior to the adventure, risks are managed by ensuring that the team have the right skills, training, and equipment; and judgement is used in choosing the objective and monitoring conditions like the weather, or snow conditions. During the adventure, participants need to make decisions in response to changes that occur over time, including changes in the weather, or the group condition (fatigue, minor ailments like blisters); along with the unpredictable (weather that washes out a key bridge, a bear stole all your food). Additionally, in self-propelled outdoor adventures, the team are limited by the resources that they brought with them, so working as a team to overcome challenges with limited resources is inherent.

For those who choose to embark on self-propelled outdoor adventures as a leisure activity, the idea that fun can involve hard/tedious work and a degree of suffering, is not foreign. Some mountaineers proudly talk about “Type II fun,” with phrases like “It doesn’t have to be fun, to be fun” ­– an acknowledgement that a challenging adventure can be worthwhile even though it involves hard/tedious work, risk, and some discomfort. In fact, the reward is often greater because of the effort required (within reason). The motivation for these adventures is intrinsic, and the journey is as important as the destination.

Adventure leaders must ensure all members of the group are appropriately prepared for the challenge ahead and stay safe and engaged. This involves having the ability to teach the physical skills required, ensuring the group has appropriate equipment for the conditions, and making judgement calls as conditions change. In addition, adventure leaders need to care deeply for the wellness of participants, watching for non-verbal cues that a participant might be suffering physically or mentally, and use wisdom to decide when and how to intervene, to improve participant satisfaction, and reduce the chances of a problem escalating.

III. RISKS IN HEALTH PROFESSIONS EDUCATION

Risks in teaching and learning include the fear of public speaking (common, though rarely acknowledged), the risk of embarrassment (from getting something wrong in front of others), and the risk of losing control (if you hand over too much control to the learners and can’t reign them back in). In the clinical environment, there’s a risk of learner harm and loss of empathy if we don’t prepare and support them adequately during their training, there’s a risk of losing clinical faculty if we make unreasonable requests of them, and there’s a risk of compromised patient outcomes if we don’t consider impact on patients due to our innovations, or lack thereof.

IV. IMPLICATIONS OF AN ADVENTURE METAPHOR

A. Academic Teaching

For academic faculty, an adventure mindset might encourage learning about teaching practices and experimenting with new techniques with some risk that they might not be successful. It also encourages innovations in teaching and scholarship to meet important social needs, even if these don’t seem like the most fruitful or safe endeavours from the perspective of traditional tenure and promotion metrics. If the academy would acknowledge the importance of unconventional approaches to scholarship and teaching to meet social needs, then it would need to reconsider how it evaluates performance.

B. Clinical Teaching

There’s a growing recognition that demonstrating vulnerability and empathy, can lead to more effective patient care and more effective clinical education, while also being more rewarding for preceptors. Many physicians have long since given up wearing a lab coat, and some are comfortable being on a first-name basis with medical trainees, as part of “an ethic of caring” which ensures learners feel safe and are better able to learn (Balmer et al., 2016). When teaching while providing clinical care, the clinician needs to think about how they are perceived by the patient, as well as by the trainee. It may be possible to balance the need to maintain credibility, while being vulnerable and modelling the clinical reasoning process for both student and patient, by exhibiting “Intellectual Candour” (Molloy & Bearman, 2019). Showing vulnerability and empathy might seem like a loss of formality, and this might feel risky, yet if this is a calculated risk, with opportunities for gain in the form of a more meaningful teaching experience and more impactful learning for students, then this might be a worthwhile adventure to embark on.

C. Underserved Populations

Some of the most medically underserved people in Canada (and globally), are rural and Indigenous peoples. If we are trying to train more healthcare providers to meet the needs of equity deserving groups, we need to consider how we are currently discouraging this. If cultural safety is not always experienced by Indigenous peoples (especially on their own, colonized lands), we need to keep cultural safety at the top of our list of priorities and we need to invest in it. If we currently provide a largely specialist curriculum which discourages rural generalist practice, we need to look at how we can make the curriculum more of a generalist curriculum. If we acknowledge that family practice has become less appealing as many family physicians now work in clinics with more limited scope, and less longitudinal relationships with patients (albeit more reasonable hours), then we should consider how to better support learners to consider full scope family practice.

In discussions about the characteristics of rural family physicians who provide full scope care, people often talk about ‘rational risk takers,’ as physicians who are more willing to accept risk, because they work in locations where timely access to specialist and subspecialist care is often not available, and working near or beyond the limit of their training is the alternate to seeing some patients not receive care at all. This is a form of calculated risk-taking and has recently been described as “Clinical Courage” (Konkin et al., 2020). If clinical courage is necessary for care providers serving our most underserved populations, then we need to encourage it, to reduce the healthcare provider maldistribution. This means ensuring that characteristics of clinical courage are embedded: when admitting students to our programs, in both our pre-clinical and clinical curricula, and in our assessments, for all learners.

V. CONCLUSION

Teaching and learning in the health professions should be fun, though a serious sort of fun. Our learners are now much more diverse than in the past, and they are advocating for needed changes in the healthcare system, while our clinicians are struggling. If we must innovate to sustain and improve what we do, then an adventure metaphor will encourage and inform how we approach this.

Health professions programs should ensure that working as a team, managing risks, and overcoming challenges with limited resources, are embedded within our curricula. We should also focus on intrinsic motivations of learners and faculty, and emphasise the importance of the journey, alongside the destination. Our systems need to ensure that clinical faculty have the capacity to care about the wellbeing of learners, alongside providing patient care. Embracing the metaphor of adventure should help invigorate our teaching and learning, and counteract burnout, while we work towards needed change in our health systems.

Notes on Contributors

The author conceived and wrote this manuscript.

Acknowledgement

The idea for this manuscript came from the author’s teaching philosophy and teaching dossier prepared for the Society for Teaching and Learning in Higher Education, 3M National Teaching Fellowship (2022 recipient). The idea has been shared at the Centre for Health Education Scholarship (CHES) Day of Scholarship (October 2022), the International Congress on Academic Medicine (ICAM, April 2023), and the Asia Pacific Medical Education Conference (APMEC, May 2023) and the idea has been improved and clarified through the critical feedback of peers at these meetings.

Funding

No funding was required for this study.

Declaration of Interest

The author has no conflict of interest, including financial, institutional, or other relationship that might lead to bias.

References

Balmer, D. F., Hirsh, D. A., Monie, D., Weil, H., & Richards, B. F. (2016). Caring to care: Applying Noddings’ philosophy to medical education. Academic Medicine, 91(12), 1618-1621. https://doi.org/10.1097/ACM.0000000000001207

Eva, K. W. (2022). An open letter to all stakeholders involved in medicine and medical education in Canada. Canadian Medical Education Journal, 13(4), 1-2. https://doi.org/10.36834/cmej.75549

Konkin, J., Grave, L., Cockburn, E., Couper, I., Stewart, R. A., Campbell, D., & Walters, L. (2020). Exploration of rural physicians’ lived experience of practising outside their usual scope of practice to provide access to essential medical care (clinical courage): An international phenomenological study. BMJ Open, 10, Article e037705. http://dx.doi.org/10.1136/bmjopen-2020-037 705

Molloy, E., & Bearman, M. (2019). Embracing the tension between vulnerability and credibility: ‘Intellectual candour’ in health professions education. Medical Education, 53(1), 32-41. https://doi.org/10.1111/medu.13649

Office of the Surgeon General (OSG). (2022). Addressing health worker burnout: The U.S. Surgeon General’s advisory on building a thriving health workforce. US Department of Health and Human Services. https://www.hhs.gov/sites/default/files/health-worker-wellbeing-advisory.pdf

*Sean B Maurice
3333 University Way,
Prince George, BC,
Canada, V2N 4Z9
1-250-960-5443
Email: sean.maurice@unbc.ca

Submitted: 19 October 2023
Accepted: 25 March 2024
Published online: 2 July, TAPS 2024, 9(3), 58-60
https://doi.org/10.29060/TAPS.2024-9-3/CS3159

Wing Yee Tong¹, Bin Huey Quek¹, Arif Tyebally² & Cristelle Chow³

¹Department of Neonatology, KK Women and Children’s Hospital, Singapore; ²Emergency Medicine, KK Women and Children’s Hospital, Singapore; ³Department of Paediatrics, KK Women and Children’s Hospital, Singapore

I. INTRODUCTION

Neonatology is considered a ‘niche’ paediatric subspecialty. Most junior doctors posted to the department have limited prior exposure to the neonatal population, and require quick and effective training to help them function safely on the clinical floor. In recent years, postgraduate medical teaching has found the use of blended learning to be effective (Liu et al., 2016). Blended learning is defined as a combination of classroom face-time with online teaching approaches, and there is currently paucity of literature on its efficacy in ‘up-skilling’ relatively inexperienced healthcare professionals in a subspecialty setting. Hence, the aim of this study was to design and evaluate the efficacy of a blended-learning orientation programme in improving neonatal clinical knowledge and procedural skills amongst junior doctors.

II. METHODS

A. Study Setting and Participants

This study was set in the largest academic tertiary paediatric hospital in Singapore.

B. Curriculum Development

We adopted the Kern’s six-step approach for curriculum development (Thomas et al., 2022), as it systematically identifies and addresses learner needs, and its cyclical nature also allows for constant modifications and improvements.

1) Step 1: Problem identification and general needs assessment

We conducted a quantitative survey to identify the general issues with our current programme, which consisted of daily face-to-face, largely didactic lectures over the first month of the posting. We noticed that many junior doctors missed teaching sessions due to work obligations, resulting in ‘piecemeal’ and ineffective learning. The one-month programme was also considered excessively lengthy.

2) Step 2: Targeted needs assessment

Most junior doctors considered themselves to be ‘novice’ learners in neonatology. This emphasised the importance of starting with foundational teaching concepts to avoid overwhelming them. Junior doctors also preferred interactive learning methods.  

3) Step 3: Goals and objectives

Our main objective was for the junior doctors to be competent and safe members of the clinical team, with basic neonatal clinical knowledge and the ability to perform and assist in neonatal procedures.

4) Step 4: Educational strategies: Course content development

We identified a list of core topics and procedural skills which formed the programme curriculum (Figure 1).

The teaching format was changed from mainly didactic lectures to case-based scenarios in both online and face-to-face sessions, as this has been shown to better motivate students towards self-directed learning and develop problem-solving skills. Case-based scenarios would also facilitate greater peer discussion and interactivity amongst learners in the face-to-face sessions.

We worked with IT specialists to convert specific topics to six online learning modules, and included interactive components such as clickable elements and narration to better engage learners (Choules, 2007). Each module was designed to be completed within 30 minutes.

For neonatal procedural skills, learners were expected to watch online demonstration videos created by the department prior to attending hands-on practical sessions.  

Figure 1. Outline of blended-learning orientation programme

5) Step 5: Implementation

The blended learning programme was implemented with junior doctors across two batches from July 2022 to January 2023. Majority were from post-graduate year three to five, with approximately half having no prior working experience in neonatology. All participated in the face-to-face sessions and completed the online modules.

We used our institution’s online learning management system to deliver the e-learning modules, and department faculty members conducted the face-to-face sessions. Designated ‘protected teaching time’ was implemented to facilitate attendance during office hours.

6) Step 6: Evaluation and feedback

We designed a pre-and-post-programme assessment consisting of 24 multiple-choice questions covering the following aspects – (1) clinical scenarios with interpretation of laboratory and radiological results, (2) factual knowledge and (3) questions on procedural skills.

The junior doctors also completed an online survey which assessed the learners’ perceptions on blended learning. Consent for the survey data to be used for research was implied in their participation.

III. RESULTS

The junior doctors had a positive experience with blended learning. All participants agreed that the learning content was relevant and appropriate for their level of experience. Almost all participants felt that there was ease of access to the online learning modules, with minimal technical issues. Learners also found specific online modules such as respiratory conditions ‘useful’, but enjoyed the face-to-face nature of sessions such as ventilatory strategies, as it gave them the opportunity to clarify doubts with their facilitator. Overall, the duration of the face-to-face orientation sessions was halved, and there was a significant improvement in the mean MCQ score.

IV. DISCUSSION

A blended learning programme designed for novice learners in Neonatology is effective in preparing junior doctors for clinical work.

Learning theories suggest that adult learners are motivated to invest time in learning if they understand its relevance (Taylor & Hamdy, 2013). The shift towards case-based learning bridges theory and practice, and motivates participation in clinical decision-making. This is an effective form of learning as demonstrated by an improvement in the mean post-test MCQ score of the participants. The experience was also deemed a positive one in qualitative feedback. In addition, the accessibility of online modules provided learners with autonomy to control their pace of learning. However, it is important to strike the right balance between online and classroom teaching, as learners still value the interactivity offered by face-to-face teaching.

We should work to create a supportive infrastructure to support blended learning methods by training more clinician-educators in online learning approaches and designing ‘reusable’ learning resources, which can be modified and integrated into other medical courses in future (Singh et al., 2021).

The limitations of our study include reliance on multiple choice tests to assess knowledge, and a lack of formal evaluation of procedural skills. Competency-based evaluations, as well as practical skills evaluations can be implemented in future runs to evaluate the efficacy of the courses.

V. CONCLUSION

Technology enhanced learning is fast becoming an integral part of medical education. Through this study, we demonstrate that blended learning programmes can be successfully integrated into the training of junior doctors in a subspecialty setting.

Notes on Contributors

WT led the design and conceptualisation of this work, implemented the education programme, and drafted the manuscript. BQ provided feedback and guidance on creating the content of the education programme. CC provided guidance on the evaluation of teaching programme. CC, AT and BQ provided feedback on the manuscript. All authors approve the publishing of this manuscript.

Funding

The authors received a Singhealth Duke-NUS Academic Medicine Education Institute Education Grant 2021 (funding number EING2205) to support the development of curriculum content for our programme.

Declaration of Interest

All authors declare that there are no conflicts of interest.

References

Choules, A. P. (2007). The use of elearning in medical education: A review of the current situation. Postgraduate Medical Journal, 83(978), 212-216. https://doi.org/10.1136/pgmj.2006.05 4189

Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., & Yan, W. (2016). The effectiveness of blended learning in health professions: Systematic review and meta-analysis. Journal of Medical Internet Research, 18(1), e2.

Singh, J., Steele, K., & Singh, L. (2021). Combining the best of online and face-to-face learning: Hybrid and blended learning approach for COVID-19, post vaccine, & post-pandemic world. Journal of Educational Technology Systems, 50(2), 140-171. https://doi.org/10.1177/00472395211047865

Taylor, D. C., & Hamdy, H. (2013). Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. Medical Teacher, 35(11), e1561-e1572. https://doi.org/10.3109/0142159x.2013.828153

Thomas, P. A., Kern, D. E., Hughes, M. T., Tackett, S. A., & Chen, B. Y. (Eds.). (2022). Curriculum development for medical education: A six-step approach. Johns Hopkins University Press.

*Tong Wing Yee
100 Bukit Timah Road
Singapore 229899
Email: tong.wing.yee@singhealth.com.sg

Submitted: 19 September 2023
Accepted: 9 January 2024
Published online: 2 July, TAPS 2024, 9(3), 61-63
https://doi.org/10.29060/TAPS.2024-9-3/CS3137

Yoshikazu Asada¹, Chikusa Muraoka², Katsuhisa Waseda³ & Chikako Kawahara⁴

¹Medical Education Center, Jichi Medical University, Japan; ²School of Health Sciences, Fujita Health University, Japan; ³Medical Education Center, Aichi Medical University, Japan; ⁴Department of Medical Education, Showa University, Japan

I. INTRODUCTION

The COVID-19 epidemic has prompted the spread of ICT-based education, with many university classes being conducted remotely. Some education systems use asynchronous tools such as learning management systems (LMSs); others use synchronous tools such as web conference systems. This trend has affected not only lectures but also exercises among students and clinical practice. Game-based education is no exception, and classes that require direct face-to-face interaction have become difficult to implement. Escape rooms (ERs) are one example of game-based education.

ERs are defined as “live-action team-based games where players discover clues, solve puzzles, and accomplish tasks in one or more rooms in order to accomplish a specific goal (usually escaping from the room) in a limited amount of time” (Nicholson, 2015). Originally intended for entertainment purposes, ERs now also serve educational purposes (Davis et al., 2022). As an educational tool, ERs are mainly used for teaching specific content knowledge and content-related skills, general skills, and affective goals (Veldkamp et al., 2020). In addition, since ERs are categorised as game-based education, they are also useful for motivating students.

ERs may be conducted either face-to-face or online. Online-based ERs, known as “Digital Educational Escape Rooms” (DEERs), have become common since the COVID-19 pandemic (Makri et al., 2021). DEERs combine the (1) possibility of digital and analog hybrid style, (2) the potential to provide immediate feedback, and (3) the suitability for some learning objectives such as social skills.

This study is intended to design and develop DEERs based on Moodle and Zoom for teaching basic professionalism, with a focus on peer collaboration for medical students.

II. METHODS

The authors made an online-based DEER with Moodle LMS and used it for teaching team communication and reviewing basic CPR knowledge for second-year undergraduate medical students. In this case, students solve asynchronous DEER challenges in Moodle through synchronous discussion in Zoom breakout rooms.

The learning objectives were to “learn collaboratively with peers” and to “understand concepts related to interpersonal relationships and interpersonal behaviour.” Before the class, students submitted a short report on the important elements that are required for team medicine, which they had learned as first-year students. The class was 100 minutes in length. The first 10 minutes were used for orientation. The next 60 minutes were used for DEERs. Within 60 minutes, a hint for solving DEERs was provided via Google Documents; authors added the hint as time went on. After the game, 30 minutes were used for reflection, including the explanation of the DEER answers and the basic lectures. Despite the existence of two aforementioned learning objectives, the time limits made it particularly hard to assess students’ achievement. Therefore, after the class, a report was assigned on the topic “points to keep in mind when sharing information and communicating with your team online through the game experience.”

There were three stages to the DEER. A total game design is shown in Figure 1. The first stage consisted of a 360^o virtual room. Students had to explore the virtual room and solve two riddles. In this stage, some hints were hidden on the ceiling or the floor. Students had to find them by looking around the room. After solving the riddles, students inputted the answer to Moodle. If the answer was wrong, they had to wait one minute before inputting another answer. The second stage began after the two riddles. This stage had four puzzles related to CPR, such as concerning the placement of AED or metronome tempo of chest compression. Since the students learned about CPR when they were first-year students, these four puzzles were reviewed their understanding. The third stage was after the four CPR puzzles. In the third stage, students had to gather all the clues to clear the game.

Figure 1. A total game design

Program evaluation was based on students’ achievement results from the Moodle log and their comments from the questionnaires.

III. RESULTS

There were 29 groups, and each one had three to four students. While five groups were able to solve the riddle completely, one group could not even reach stage two. Moodle log data and questionnaires suggested that the difficulty of the riddles was appropriate, since only 8% of participants answered that the first stage was difficult, and other groups used about 15 minutes for the first stage from the logs.

In some groups, students turned off their cameras and solved the riddles individually. In this case, they shared almost nothing but the answers, and very little about the process for solving puzzles and riddles. In other groups, students turned on their cameras and shared the screen. In contrast to the previous groups, they solved the puzzles and riddles through live discussions.

IV. DISCUSSION

Some groups could not complete the DEERs, and one group could not reach stage two. In the group that could not finish stage one, students did not share the process of solving riddles. Moreover, they turned off their cameras, which made it difficult to define how they were approaching the tasks. The communication style of students potentially affects their achievement level. It is also connected to their learning objectives.

Despite the difficulty of teaching skills and attitude only with asynchronous distance learning, some scope exists for interactive content, for example, by having the students choose the correct tempo for chest compressions by sound with live discussion and feedback from others. Of course, it will be more effective to use face-to-face simulations to check psychomotor skills.

In this case, gathering students’ learning logs was easy since DEER was provided with Moodle. In addition, observing how students discuss in online was possible, since Zoom can track the activity status in breakout rooms. Although the design, development, and implementation of the DEERs will be complicated, the hybrid-style DEER, such as using LMS with synchronous classes, might make DEERs more attractive. Furthermore, it makes collection of a variety of data, such as the timestamp of answer, pattern of the failure, and manner of online communication, possible. These data would be useful to assess students and provide feedback to them.

V. CONCLUSION

DEERs are potentially useful for engaging student communication and discussion even in the online synchronous class. In the future, it will be possible to provide an integrated learning experience with a more appropriate difficulty level by accumulating Moodle log data and student recognition data.

Notes on Contributors

YA and CM designed and developed DEER. They also analysed the results.

KW and CK managed the class and facilitated the breakout room.

All authors have read and approved the final manuscript.

Acknowledgement

The authors would like to acknowledge the contributions of all participants.

Funding

This work was supported by FOST (Foundation for the Fusion of Science and Technology) 2019 and 2022 Research Grants.

Declaration of Interest

All authors declare no conflicts of interest.

References

Davis, K., Lo, H. Y., Lichliter, R., Wallin, K., Elegores, G., Jacobson, S., & Doughty, C. (2022). Twelve tips for creating an escape room activity for medical education. Medical Teacher, 44(4), 366–371. https://doi.org/10.1080/0142159X.2021.1909715

Makri, A., Vlachopoulos, D., & Martina, R. A. (2021). Digital escape rooms as innovative pedagogical tools in education: A systematic literature review. Sustainability, 13(8), Article 4587. https://doi.org/10.3390/su13084587

Nicholson, S. (2015, May 24). Peeking behind the locked door: A survey of escape room facilities. http://scottnicholson.com/pubs/erfacwhite.pdf

Veldkamp, A., van de Grint, L., Knippels, M. P. J., & van Joolingen, W. R. (2020). Escape education: A systematic review on escape rooms in education. Educational Research Review, 31, Article 100364. https://doi.org/10.1016/j.edurev.2020.100364

*Yoshikazu Asada
Medical Education Center
Jichi Medical University
3311-1, Yakushiji, Shimotsuke,
Tochigi, Japan
+81-285-58-7067
Email: yasada@jichi.ac.jp