Submitted: 19 June 2020
Accepted: 21 October 2020
Published online: 4 May, TAPS 2021, 6(2), 25-30
https://doi.org/10.29060/TAPS.2021-6-2/OA2327

Nicola Ngiam^1,2 & Chuen-Yee Hor¹

¹Centre for Healthcare Simulation, National University of Singapore, Singapore; ²Khoo Teck Puat-National University Children’s Medical Institute, National University Hospital, Singapore

Abstract

Introduction: Standardised patients (SPs) have been involved in medical education for the past 50 years. Their role has evolved from assisting in history-taking and communication skills to portraying abnormal physical signs and hybrid simulations. This increases exposure of their physical and psychological domains to the learner. Asian SPs who come from more conservative cultures may be inhibited in some respect. This study aims to explore the attitudes and perspectives of Asian SPs with respect to their role and case portrayal.

Methods: This was a cohort questionnaire study of SPs involved in a high-stakes assessment activity at a university medical school in Singapore.

Results: 66 out of 71 SPs responded. Racial distribution was similar to population norms in Singapore (67% Chinese, 21% Malay, 8% Indian). SPs were very keen to provide feedback to students. A significant number were uncomfortable with portraying mental disorders (26%) or terminal illness (16%) and discussing Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS, 14%) or Sexually Transmitted Diseases (STDs, 14%). SPs were uncomfortable with intimate examinations involving the front of the chest (46%, excluding breast), and even abdominal examination (35%). SPs perceive that they improve quality of teaching and are cost effective.

Conclusion: The Asian SPs in our institution see themselves as a valuable tool in medical education. Sensitivity to the cultural background of SPs in case writing and the training process is necessary to ensure that SPs are comfortable with their role. Additional training and graded exposure may be necessary for challenging scenarios and physical examination.

Keywords:            Standardised Patients, Perspective, Asian, Medical Education, Survey

Practice Highlights

• The Asian SPs in our institution see themselves as a valuable tool in medical education.
• Sensitivity to the cultural background of SPs in case writing and the training process is necessary to ensure that SPs are comfortable with the roles that they portray.
• Additional training and graded exposure for SPs may be necessary for challenging scenarios and physical examination in the Asian context.

Standardised patients (SPs) have been involved in medical education since the 1960s (Barrows & Abrahamson, 1964). SP methodology has been widely used in North America and Europe. By the 1990s, majority of American medical schools were using the SP methodology in teaching clinical skills, assessments and for providing feedback to learners (Anderson et al., 1994). The prevalence of employing SP methodology in medical education in Asia is presumed to be less ubiquitous. It is therefore imperative to understand the views of Asian SPs so that the SP methodology can be fostered.

SPs started out simulating medical symptoms and patient concerns as well as evaluating medical interviewing skills in 1976 (Barrows & Abrahamson, 1964; Stillman et al., 1976). Their role has evolved to demonstrating abnormal physical signs, providing feedback on medical interviewing skills and being involved in hybrid simulations. This increases their exposure to the medical environment and to different medical experiences that they may not have experienced before. Certain experiences may potentially cause psychological distress. The SP could be in a vulnerable position and personal attitudes and beliefs towards illness should be taken into consideration when engaging SPs to portray these roles.

This is particularly true in Asian SPs. For example, patients may not be willing to discuss mental health issues for fear of social stigma and shame (Kramer et al., 2002). Asian SPs are also likely to be more conservative and modest with regards to physical examination. This can be extrapolated from findings that cultural attitudes toward breast cancer screening tests and modesty are some reasons why Asian women are reluctant to seek out breast cancer screening (Parsa et al., 2006).

In the past, SPs were routinely employed in objective structured clinical examinations at our medical school. They were not required to provide any form of feedback to the learners. We endeavoured to develop a more structured SP training program at our institution. In the initial phase, this study was conducted to survey the attitudes of the SPs who work at our institution towards case portrayal and the value of SP methodology.

II. METHODS

This was an anonymous cohort questionnaire study. An online questionnaire was administered to standardized patients who were recruited to work at a high-stakes objective structured clinical examination at a university medical school. Participants were sent a link to an electronic survey by email after the event. Participation was voluntary. Questions about race, age, gender and years of experience as an SP were asked. The importance of the contribution of an SP to medical education and their comfort with discussing medical conditions, portraying abnormal signs and undergoing different physical examinations were evaluated. A Likert scale of 1-5 was used where appropriate. This questionnaire study is covered by the institutional review board approval (Study Reference Number: 09-288) of the standardised patient program in our institution. Being an anonymous, voluntary survey, the consent was implied when the participants filled and returned the completed survey.

Descriptive statistics and the electronic survey were generated using Vovici software version 6 (Vovici Corp, Dulles, Virginia, United States).

III. RESULTS

66 out of 71 SPs (93%) responded. 40% of the SPs were aged 31-40 years (Figure 1) and 72% were female. Racial distribution was similar to population norms in Singapore (67% Chinese, 21% Malay, 8% Indian, 4% others).

Figure 1: SP comfort with portrayal

With regards to their role, 95% of SPs felt it was important for them to be involved in teaching students and providing feedback. A significant number were uncomfortable with portraying mental disorders (26%) or terminal illness (16%) (Figure 1) and discussing HIV/AIDS (14%) or sexually transmitted diseases (14%) (Figure 2). With regards to death and dying, 6% of SPs were uncomfortable discussing this while another 6% were unsure about it. As expected, SPs were uncomfortable with examinations involving the front of the chest (46%, excluding breast examination) and even abdominal examination (35%). The 60% of the female SPs surveyed were uncomfortable with breast examination (Figure 3). SPs perceive themselves to improve the quality of teaching (98%) and to be cost effective (98%). The majority of this group of SPs (83%) felt that this was a viable option for sustainable employment.

Figure 2: SP comfort with discussing topic

Figure 3: SP comfort with physical examination

IV. DISCUSSION

The benefits of SP methodology in providing a safe environment for practice and experiential learning are well established. In an effort to expand the use of SP methodology at our institution, information regarding the acceptability and feasibility were required. In the past, SPs were mainly employed in summative assessment activities and did not provide learners with feedback. Before pushing the boundaries of the SP job description, it was important to understand the perspectives of our SPs and which areas of SP work they would feel comfortable or uncomfortable with.

The areas of interest were comfort with portraying roles that involved taboo topics such as mental health issues, sexually transmitted disease, death and dying. In many Asian cultures, mental illness is stigmatizing; it reflects poorly on family lineage and can influence others’ beliefs about the suitability of an individual for marriage. (Kramer et al., 2002). Many people of Asian descent view people with mental illnesses as dangerous and aggressive (Lauber & Rössler, 2007) and believe that mental illness is a punishment from God (Fogel & Ford, 2005). In China, mental health problems are believed to be a result of weak character, having evil spirits, or punishment for not respecting ancestors (Lam et al., 2006). Asian American women avoid seeking treatment for depression and suicide ideation because of Asian family and community stigma associated with mental health issues (Augsberger et al., 2015). With regards to sexual practices and sexually transmitted disease, literature shows that Chinese men regard homosexual-related stigma and discrimination as major barriers to HIV testing. Most men were reluctant to obtain an HIV test in fear that their homosexual identity would be exposed, and they sometimes encountered discrimination even from medical personnel (Wei et al., 2014). Living with HIV in an Asian society is fraught with difficulty in the context of fear and disapproval (Ho & Goh, 2017). Death and dying are generally considered taboo in Asian cultures. Open discussions about death are regarded as a bad omen (Hall & Hall, 1976). Even for those who are dying, discussion about death is avoided because it is believed that such talk may hasten the dying process or even cause death prematurely (Xu, 2007). The avoidance if discussion about death and dying in traditional Chinese culture has been found to impede the ability to discuss advanced care planning (Cheng, 2018). These cultural beliefs were reflected in the discomfort expressed by some study participants with portrayal of roles involving mental health issues, HIV or sexually transmitted diseases, terminal illness and death and dying. This is evidence that some of our SPs do have traditional Asian perspectives regarding these sensitive issues but it is encouraging that a larger proportion are comfortable with these issues. This informs us that SPs should be given advanced notice regarding the content of the case that they are expected to portray so that they can make an informed choice when accepting roles. This is especially important when taboo or sensitive content is involved. SPs should also be given an option to withdraw from the assignment if they feel uncomfortable with the content of the case after they have been trained for the case.

In view of the more conservative nature of Asians, the hypothesis was that there could be areas of the body that SPs would not be willing to have examined by students. Asian women seem to be more conservative as only 53% of respondents in a study did breast self-examinations (Sim et al., 2009; Tan et al., 2005) reported that, between 2000 and 2003, 21.5% of women in Singapore presented with stage III or IV breast cancer which may potentially be due to cultural attitudes toward breast cancer screening tests and modesty, which inhibit Asian women from participating in breast cancer screening (Parsa et al., 2006). Spiritual and religious beliefs were found to act as a barrier to breast cancer screening in Singaporean Malay women (Shaw et al., 2018). As expected, more than half of our female SPs were uncomfortable with breast examination. When both genders were considered, examination of the front of chest (excluding breast) and abdominal examination were also flagged as concerns. This made us aware of the hesitance of some SPs in this area and the need to explore this further while trying to expand the role of the SP. In developing our SP program, consent for physical examination needs to be explained in detail and comfort of the individual SP with any physical examination must be taken into consideration.

The SPs in our study perceived themselves to be of value in medical education. Standardized patients in a study in Switzerland felt motivated, engaged, and willing to invest effort in their task and did not mind the increasing demands of their work as long as the social environment in SP programs was supportive (Schlegel et al., 2016). This is encouraging for a developing SP program to know as we feel confident to expand the job scope of our SPs as long as adequate explanation and training is provided to support the SPs. With more structured coaching and exposure, we expect that SPs will become more comfortable with more challenging roles and would be willing to push the boundaries of their comfort zone.

One limitation of this study is the large majority of female participants. This was a convenience sample to optimize response rate. Further studies should aim to include a more balanced gender representation. Another limitation would be that only quantitative data was collected. In exploring perspectives, focused interviews with qualitative analysis would have provided a more in-depth understanding of the beliefs and values of the SPs.

V. CONCLUSION

This study provides initial insights into the perspectives of Asian SPs at a university medical school in an Asian country. They see themselves as a valuable tool in medical education and are willing to expand their role in the curriculum. Faculty and trainers need to be sensitive to the cultural background of our SPs in case writing and the training process to ensure that SPs are comfortable with the roles that they portray. This is of particular relevance to SP programs that employ predominantly Asian SPs. There is evidence from this study of discomfort with portraying patients with mental health issues, terminal illness and sexually transmitted diseases. The areas of exposure required in physical examination also need to be carefully considered. Additional training and graded exposure may be necessary for SPs willing to be involved in these scenarios and certain types of physical examination. Concerns about the scenario from the SPs may not be immediately apparent. The results presented here will make SP trainers more aware of the possibility of SP discomfort. Future research will be required on what type of training and what other factors will promote comfort with these scenarios as well as the impact of taking on such roles on the SPs.

Notes on Contributors

Nicola Ngiam conceptualized and designed the study, analyzed the data and interpreted the results, wrote the manuscript draft, revised it, read it and gave final approval of the manuscript.

Hor Chuen-Yee developed the methodological framework for the study, performed data collection and data analysis, revised the manuscript, read it and gave final approval of the manuscript.

Ethical Approval

This study is covered by the institutional review board approval (Study Reference Number: 09-288).

Acknowledgement

We thank Dr Dimple Rajgor for her assistance in editing, formatting, reviewing, and in submitting the manuscript for publication.

Funding

No funding source required.

Declaration of Interest

The authors have no conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Anderson, M. B., Stillman, P. L., & Wang, Y. (1994). Growing use of standardized patients in teaching and evaluation in medical education. Teaching and Learning in Medicine: An International Journal, 6(1), 15-22.

Augsberger, A., Yeung, A., Dougher, M., & Hahm, H. C. (2015). Factors influencing the underutilization of mental health services among Asian American women with a history of depression and suicide. BMC Health Services Research, 15, 542-542. https://doi.org/10.1186/s12913-015-1191-7.

Barrows, H. S., & Abrahamson, S. (1964). The programmed patient: A technique for appraising student performance in clinical neurology. Academic Medicine, 39(8), 802-805.

Cheng, H. W. B. (2018). Advance care planning in Chinese seniors: Cultural perspectives. Journal of Palliative Care, 33(4), 242-246.

Fogel, J., & Ford, D. (2005). Stigma beliefs of Asian Americans with depression in an internet sample. Canadian Journal of Psychiatry, 50(8), 470-478.

Hall, E. T., & Hall, E. (1976). How cultures collide. Psychology Today, 10(2), 66-74.

Ho, L. P., & Goh, E. C. L. (2017). How HIV patients construct liveable identities in a shame based culture: The case of Singapore. International Journal of Qualitative Studies on Health and Well-Being, 12(1), 1333899. https://doi.org/10.1080/17482631.2017.1333899

Kramer, E., Kwong, K., Lee, E., & Chung, H. (2002). Cultural factors influencing the mental health of Asian Americans. The Western Journal of Medicine, 176(4), 227-231.

Lam, C. S., Tsang, H., Chan, F., & Corrigan, P. W. (2006). Chinese and American perspectives on stigma. Rehabilitation Education, 20(4), 269-279. https://doi.org/10.1891/088970106805065368

Lauber, C., & Rössler, W. (2007). Stigma towards people with mental illness in developing countries in Asia. International Review of Psychiatry, 19(2), 157-178.

Parsa, P., Kandiah, M., Abdul, H. R., & Zulkefli, N. (2006). Barriers for breast cancer screening among Asian women: A mini literature review. Asian Pacific Journal of Cancer Prevention, 7(4), 509-514.

Schlegel, C., Bonvin, R., Rethans, J., & der Vleuten Van, C. (2016). Standardized patients’ perspectives on workplace satisfaction and work-related relationships: A multicenter study. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 11(4), 278-285.

Shaw, T., Ishak, D., Lie, D., Menon, S., Courtney, E., Li, S. T., & Ngeow, J. (2018). The influence of Malay cultural beliefs on breast cancer screening and genetic testing: A focus group study. Psycho‐Oncology, 27(12), 2855-2861.

Sim, H., Seah, M., & Tan, S. (2009). Breast cancer knowledge and screening practices: A survey of 1,000 Asian women. Singapore Medical Journal, 50(2), 132-138.

Stillman, P. L., Sabers, D. L., & Redfield, D. L. (1976). The use of paraprofessionals to teach interviewing skills. Pediatrics, 57(5), 769-774.

Tan, E., Wong, H., Ang, B., & Chan, M. (2005). Locally advanced and metastatic breast cancer in a tertiary hospital. Annals of the Academy of Medicine, Singapore, 34(10), 595-601.

Wei, C., Yan, H., Yang, C., Raymond, H., Li, J., Yang, H., Zhao, J., Huan, X., & Stall, R. (2014). Accessing HIV testing and treatment among men who have sex with men in China: A qualitative study. AIDS Care, 26(3), 372-378.

Xu, Y. (2007). Death and dying in the Chinese culture: Implications for health care practice. Home Health Care Management & Practice, 19(5), 412-414.

*Nicola Ngiam
Department of Medicine 
National University Health System
1E Kent Ridge Rd,
Singapore 119228
Email address: nicola_ngiam@nuhs.edu.sg

Submitted: 28 March 2020
Accepted: 23 September 2020
Published online: 4 May, TAPS 2021, 6(2), 9-24
https://doi.org/10.29060/TAPS.2021-6-2/OA2242

De Zhang Lee¹, Jia Yi Choo¹, Li Shia Ng², Chandrika Muthukrishnan¹ & Eng Tat Ang¹

¹Department of Anatomy, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Otolaryngology, National University Hospital, Singapore

Abstract

Introduction: Gamification has been shown to improve academic gains, but the mechanism remains elusive. We aim to understand how psychological constructs interact, and influence medical education using mathematical modelling.

Methods: Studying a group of medical students (n=100; average age: 20) over a period of 4 years with the Personal Responsibility Orientation to Self-Direction in Learning Scale (PRO-SDLS) survey. Statistical tests (Paired t-test) and models (logistic regression) were used to decipher the changes within these psychometric constructs (Motivation, Control, Self-efficacy & Initiative), with gamification as a tool. Students were encouraged to partake in a maze (10 stations) that challenged them to answer anatomical questions using potted human specimens.

Results: We found that the combinatorial effects of the maze and Script Concordance Test (SCT) resulted in a significant improvement for “Self-Efficacy” and “Initiative” (p<0.05). However, the “Motivation” construct was not improved significantly with the maze alone (p<0.05). Interestingly, the “Control” construct was eroded in students not exposed to gamification (p<0.05). All these findings were supported by key qualitative comments such as “helpful”, “fun” and “knowledge gap” by the participants (self-awareness of their thought processes). Students found gamification reinvigorating and useful in their learning of clinical anatomy.

Conclusion: Gamification could influence some psychometric constructs for medical education, and by extension, the metacognition of the students. This was supported by the improvements shown in the SCT results. It is therefore proposed that gamification be further promoted in medical education. In fact, its usage should be more universal in education.

Keywords:            Psychometric Constructs, Medical Education, Motivation, Initiative, Self-efficacy

Practice Highlights

• Student’s enjoyment (interest) of the curriculum will determine the eventual academic outcome.
• Metacognition (defined as the “learning of learning”, “knowing of knowing” and/ or the awareness of one’s thought processes) was improved with SCT and gamification.
• Gamification is useful as a form of augmentation for didactic teaching but should never replace it.
• Different type of psychometric scale (e.g. LASSI versus PRO-SDLS) used in your research will produce varying results.
• Gamification is resource intensive and needs extra time to prepare compared to didactic approaches.

I. INTRODUCTION

Psychology is integral to healthcare and education but has often been overshadowed, compared to the other basic disciplines (Choudhry et al., 2019; Pickren, 2007). This is ironical because human psyche needs to be properly understood in order to manage them effectively (Wisniewski & Tishelman, 2019). Presently, the study of psychology does not feature prominently in the medical curriculum (Gallagher et al., 2015) with the exception of psychiatry (Douw et al., 2019). This gap needs to be addressed (Paros & Tilburt, 2018). In this research, we seek to understand the constructs for good medical learning via gamification which has wide ranging effects (Mullikin et al., 2019). The psychometric constructs to be analysed were as follows: 1) “Motivation”; define as the desire to learn out of interest or enjoyment (Yue et al., 2019). 2) “Initiative”; refers to how proactive a student is to learning (Boyatzis et al., 2000). 3) “Control”; is how much influence one has over the circumstances (Sheikhnezhad Fard & Trappenberg, 2019). 4) “Self-Efficacy”; relates to how confident one is, to do what needs to be done (Michael et al., 2019). We believe that these constructs contribute to the student’s awareness of their own thought processes (metacognition) towards their medical education.

Gamification” is defined as a process of adding game-like elements to something so as to encourage more participation (Rutledge et al., 2018; Van Nuland et al., 2015). The idea of using games to “lighten up” medical education in the clinical setting was first proposed in 2002 (Howarth-Hockey & Stride, 2002). The authors observed increased engagement and participation during lunchtime medical quizzes in the hospital. They therefore concluded that medical education could be fun, and since then, gamification has been taken seriously by the community (Evans et al., 2015; Nevin et al., 2014). In essence, gamification could be something as simple as having board games (Ang et al., 2018) but importantly, its impact on students’ learning must be evaluated and validated. Most studies in the literature did not fulfil this requirement (Graafland et al., 2012). The impact of games on the behavioral and/or psychological outcomes should be studied (Graafland et al., 2017; Graafland et al., 2014).

A PubMed search would reveal that there are numerous self-reporting tools such as LASSI (Learning and Strategies Study Inventory (Muis et al., 2007), MSLQ (Motivated Strategies for Learning Questionnaire (Villavicencio & Bernardo, 2013), and the SRLPS (Self-regulated Learning Perception Scale) (Turan et al., 2009) etc. Given the choices, how does one decide which one to adopt for their studies? In our research, we chose to use the PRO-SDLS survey questions with some modifications. The choice was both serendipitous and practical, as we have previously validated it via the Cronbach alpha (>0.7). In our earlier work, feedback scores and results yielded inconclusive evidence to support enhanced motivation among our students. Furthermore, was this due to gamification? With the current endeavour, we aim to prove via mathematical modelling that there are indeed alterations to the psychometric constructs. Hence, we re-analyse the old data set together with additional new information, using statistical analysis tools such as the logistic regression model, Wilcoxon tests, and the Paired t-test.

Medical teaching and learning is a complex endeavour based on an apprenticeship model (Cortez et al., 2019), which may or may not be an ideal arrangement (Sheehan et al., 2010). Furthermore, the decision making is often delegated to the seniors (Chessare, 1998). Conversely, gamification could empower the students to take charge of one’s learning, including decision making (Shah et al., 2013). Furthermore, one needs to understand what works from what is empirical (Cote et al., 2017). While our initial research addressed the impact of the games on academic performance, we now sought to further understand its effects on the psychometric dimensions. This will help to understand the psychology of self-directed (or regulated) learning. We hypothesize that the amount of gamification will impact these constructs. In summary, we hope to achieve the following:

Aims:

• Understanding the role of psychometric constructs and gamification in medical education via suitable mathematical modelling.
• To decipher the interaction of different psychometric constructs (Motivation, Self-efficacy, Control and Initiatives) in producing desired learners’ behaviours (metacognition) via the anatomy maze.

II. METHODS

First-year medical students (M1) took part in this retrospective analytical research. Two randomised groups of medical students (n=75, median age: 20 years) consented to the study (Group 1 & 2). A randomised group of students (n=25) exposed to no gamifications (Group 0) served as the control. Every student was required to complete a pre- and post- PRO-SDLS for the research. There were no penalties for withdrawing from the IRB-approved project (See IRB: B-16-205).

Gamification was carried out according to the scheme in Figure 1. Each group was divided into 3 to 4 subgroups that would enter the maze with a clue card (see example in Figure 1) linked to a specific pot specimen. They were required to explore the museum for the next clue and had to answer the hidden questions (see examples in Appendix) which would provide further directions. At the conclusion, students were given a competitive pop quiz that had no impact on their summative academic grades.

The main purpose was to assess formative knowledge acquisition. The validated PRO-SDLS include the following psychometric constructs: “Motivation” (7 questions), “Initiative” (6 questions), “Control” (6 questions) and “Self-Efficacy” (6 questions). (See Sup. Materials). The responses are then collapsed into an average accordingly. A higher score indicated more agreeability towards that construct for self-directed learning (Ang et al., 2017). The survey was designed with backward scoring to ensure accuracy. For quantification purposes, we subtracted the pre-feedback from the post-feedback scores for each question. An increased score for a particular construct suggests improvement (Cazan & Schiopca, 2014). Furthermore, students in Group 2 were given Script Concordance Test (SCT) quizzes (See Sup. Materials) as part of gamification (Lubarsky et al., 2013; Lubarsky et al., 2018; Wan et al., 2018). SCT were meant to enhance clinical reasoning. All data were analysed from two perspectives:

• The magnitude of score increase (or decrease) of the post- PRO-SDLS survey responses, with respect to the pre- responses.
• The odds of a student reporting an increased score in the post- PRO-SDLS survey responses.

In (a), the paired differences for each student’s response were studied using a parametric approach (paired t-test). In (b), we studied the odds of increased score for each construct, and investigate if grouping affected these odds. More formally, for each construct k (where k is one of the four constructs), we define variable  as the probability of a student from group  showing an increase in score for construct  (and hence,  as the probability that the student’s score decreased or remained unchanged). The value of   can be estimated by dividing the number of students from group  with an increased score for construct  by the total number of students from group . If the interventions are unsuccessful, we would expect  to be around  since a student’s score would likely either increase or decrease at random, with an equal probability. This can be tested using the t-test.

An alternative approach would be to study the odds of success, which can be written as  . A common mathematical model used to study these odds is the logistic regression model. For each construct, the logistic regression model studies the odds of a student from a given group showing an increased or decreased score. The overall significance of the model can be tested using the p-value obtained from the likelihood ratio test, while the significance of the individual odds can be tested using the t-test. For more details on the logistic regression model, we refer the reader to (Agresti, 2003).

We utilised the open source software R  (Team, 2019) to perform our statistical analysis.

III. RESULTS

Participation rate in the gamification endeavour was consistently 90±5%, and there was zero withdrawal from it, accompanied by reported favourable qualitative comments.

A. Studying the Absolute Scores

The average change in scores across all the groups for each construct is given in Table 1. From these scores, we believe that our gamification exercises may have had a positive impact on “Self-Efficacy” and “Initiative”. To visualize the spread of responses, we have prepared box plots of the post – pre scores (available in Supplementary Materials).

We observe that the null hypothesis of no difference between pre and post intervention levels for all constructs are not rejected (under p=0.05) for the control group. Both tests also failed to show any significant change for the “Control” construct.

There is strong evidence that the classroom interventions employed by Groups 1 and 2 had an impact on “Initiative” levels of students, reflected by the small p-values obtained using both tests. The average increase in “Initiative” scores for students in Groups 1 and 2 are 0.71 and 0.67, respectively, which are similar. Recall that the students in Group 2 participated in the SCT, in addition to the maze which is common across both groups. This suggests that the SCT has a negligible impact on “Initiative”.

There is also strong evidence (p=0.05) to show that the games enhanced the “Self-Efficacy” levels among the students. The t-test also gives strong evidence (p=0.05) that there is a significant change in Group 2, and milder evidence for Group 1 (p=0.10). The average increase in “Self-Efficacy” levels for Groups 1 and 2 are 0.63 and 0.56, respectively. Again, the differences are negligible, and this suggests that the SCT has a negligible impact. Finally, there is mild evidence (p=0.10) of a significant change in “Motivation” for Group 2, but no such evidence for Group 1. The average increase in “Motivation” score for Group 2 is 0.55. This time, the SCT might have helped to improve students’ motivation.

B. Studying the Odds of Score Improvement

We will now turn our attention to modelling the odds of a student reporting an increase in construct score. Earlier, we defined  as the probability of a student from group  showing an increase in score for construct , and explained why we would expect  to be around  if the games have no impact on the odds of “success”. The t-test was used to test this, under the null hypothesis that  for all groups and constructs. The p-values obtained are summarised in Table 3.

Under the logistic regression model, not rejecting the null hypothesis for a given odds means that we assume it takes on the value 1. It should be noted that the individual coefficients should be examined when the model is determined to be significant under the likelihood ratio test, as the coefficients obtained under a poor model fit may not be meaningful.

We notice that the significant terms flagged out by the t-test (Table 3) largely agree with the significant terms of the logistic regression model, except for the “Self-Efficacy” odds for Group 1. However, the “Self-Efficacy” model was not determined to be a good fit using the likelihood ratio test.

The only model which was deemed to be a good fit was the one for the “Initiative” construct. The odds for Group 0 is deemed to be insignificant (and hence assumed to be 1), while the odds for Groups 1 and 2 are statistically significant. We can interpret this model as follows,

1. Since the odds for Group 0 is statistically insignificant under the t-test, we assume the odds to be 1. In other words, it is equally likely for a student from the control group to show an increase or decrease in score.
2. The odds for both Group 1 and 2 are statistically significant. The odds of success of Group 1 is 2.41, which can be translated to a roughly 7 in 10 chance (probability of 0.71) for a student in this group showing an improved score. A similar interpretation can be made for Group 2, which showed an odds of 1.99. This translates to a slightly lower probability of 0.67 for a student from Group 2 displaying an improved score.

 

With this, we have presented a logistic regression approach of mathematically modelling these odds. A search on Google Scholar and PubMed yielded no previous work which made us of this mathematical modelling approach on the PRO-SDLS survey data. With the derived odds, we can compare the degree of success of the various classroom interventions. The logistic regression modelling approach is therefore, proposed as a complement of the t-test approach, which is restricted to detecting the presence of statistically significant differences.

C. Qualitative Comments (underlined words underpinning for metacognition)

1) Positive feedbacks:

• “The maze games were the most helpful as they helped me to consolidate my learning, and also enables me to ask the tutor any questions that I have from class. They allowed me to learn anatomy in a fun, enjoyable and memorable way”
• “It allowed me to visualize the things that I was learning and helped with clarifying doubts”
• “The extra question posted at each station was helpful”
• “Wanting to be able to identify things in the museum makes me more motivated to prepare beforehand”
• “Allows me to identify the knowledge gap so that I can work on it”
• “I like the quiz as it motivates me to study beforehand and shows me the gaps in my knowledge”
• “The clinically relevant questions made me think a lot”

2) Negative feedbacks:

• “I prefer didactic teaching”
• “We did not interact much with the exhibits”
• “The maze was more of a mini quiz or test to check if we remember anything”
• “Perhaps we could go into more complex concepts”
• “More challenging questions”
• “Students just follow each other around the anatomy museum and it defeats the purpose of the maze”
• “The maze could have a competitive element to make it more exciting. Maybe more MCQ questions per model so we can make use of it more”

IV. DISCUSSION

We undertook this research to decipher how gamification as a concept helps medical students learn a basic subject like human anatomy. We also want to understand how psychometric constructs interact to produce behavioural changes towards self-directed learning. This was done by analysing the data from the PRO-SDLS via statistical tests. Put simply, one needs to understand that medical education is a very complex process that demands balance between apprenticeship (fellowship) (Sheehan et al., 2010), and a dose of self-directed learning (van Houten-Schat et al., 2018). With our initial research into gamification of anatomy education (Ang et al., 2018), there were other studies suggesting similar benefits (Felszeghy et al., 2019; Nicola et al., 2017; Van Nuland et al., 2015). We are therefore convinced that gamification could help to engage students and improve academic gains. However, the notion of gaming can be very broad (Virtual Reality, board games, digital apps etc.), so there is a need to understand the underlying psychology. With that in mind, we re-analyse our previous data with the existing, using proven statistical tools to decipher the learning psychology of these medical students, and their awareness of their own thought processes (metacognition).

We earlier hypothesised that gamification would influence these dependent constructs differently and indeed this was the outcome. In our analysis, we found that the combinatorial effects of the maze and SCT resulted in a significant improvement for “Self-Efficacy” and “Initiative”. While the maze alone did not significantly improve “Motivation”, we saw mild evidence of an improvement in terms of psychometric scores, when the SCT and maze were used in combination. In lay terms, the maze encouraged these students to learn on their own. By extension, one could also argue that gamification will help the students in making decisions since “Motivation” and “Initiatives” are key attributes (Vohs et al., 2008). The ability to make a simple clinical judgement, and the courage to act on them, are the virtues that we should be imbuing in the medical students, and some junior doctors. Interestingly, there is mild evidence that the “Control” construct was undergoing erosion in the students not exposed to gamification, as the course progresses. This adverse result is not seen in both groups exposed to the games. Perhaps the more relaxed classroom setting with gamification helped students to feel more in control of their learning process. Logically, this made a lot of sense across the education landscape.

A follow up question would be, does the feedback confirm the results given in our qualitative analysis? Recall that in our logistic regression model, students from both non-control groups displayed a statistically significant improvement in “Initiative” levels. This is supported by some of the positive feedback received for our endeavours, such as being “motivated to prepare beforehand”, “identify the knowledge gap” and work on them, as well as helping them to “think a lot” about the course content. Furthermore, some of the negative feedback, such as requests for more challenging questions, or more questions in general, suggests that the students are taking the initiative to learn more. This certainly adds credence to the findings of our proposed logistic regression model, as well as highlighting the importance of studying both qualitative and quantitative feedback.

There are caveats that one should be aware when implementing gamification. The formative part of the endeavour could be variable, and dependent on numerous factors such as the tutor involved, and the type of games, interventions, and reporting scales used. In the feedback, 76% of the participants felt that the maze should continue as an adjunct but not to totally replace didactic tutorials. In other words, introducing gaming elements into the curriculum should be done judiciously. With reverse scoring, it was shown that “Self-Efficacy” fell as the level of gamification is increased. In lay terms, students might be feeling that the maze trivialize the learning of the subject. As a counter measure, and to maintain quality assurance, we could introduce video lectures from previous years to allay these fears. In summary, we now confirmed that gamification works, and it influences learning outcomes as demonstrated by others (Burgess et al., 2018; Goyal et al., 2017; Kollei et al., 2017; Kouwenhoven-Pasmooij et al., 2017; Kurtzman et al., 2018; O’Connor et al., 2018; Patel et al., 2017; Savulich et al., 2017). Separately, there were criticisms as to why SCT was introduced into the research. We believed that such augmentation will add “fun” for the pre-clinical students to tackle the various clinical scenarios and clinical anatomy.

V. LIMITATIONS OF THE STUDY

Our research necessitated that the students take part in the maze and the SCT. Although it was not compulsory, no students opted out of it. Some critics would misconstrue this to be a form of forced play. According to Jane McGonigal, gamification should ideally not be mandated (Roepke et al., 2015).

VI. CONCLUSION

Through statistical modelling, we have shown how the “Initiative”, “Motivation”, and “Self-Efficacy” constructs could potentially benefit from gamification. The before-after experimental set up allowed for powerful comparisons to be made. Studying the odds of construct score improvement, alongside the raw scores, allowed us to study the data from different perspectives. Though this approach, we discovered how the potential benefits of our gamification exercises outweigh the potential adverse effects. Gamification had resulted in improved “Initiative” in these medical students. We believe that their decision-making skills will also be boosted if existing culture allows for more self-discovery (to improve “Initiative”, “Control” and “Self-efficacy”) and autonomy. If these recommendations are duly considered and implemented thoughtfully, there is little doubt that our future doctors will be better equipped to serve humanity. This may also help to avoid possible burnout in residents (Hale et al., 2019).

Stronger conclusion and potential for applications are as follows: In a continuum, we started gamifying anatomy education and proven that academic grades could be improved by the process (Ang et al., 2018). We then asked a fundamental question in how exactly it happened. This was done by carrying out a psychometric analysis on the participants. We discovered that psychometric constructs were important, and this was proven in this manuscript. The impact of gamification is now elevated given the COVID-19 pandemic that necessitated more online teaching. Moving forward, we believe that gamification should move towards creating an electronic application that the students may access 24/7. This will ensure that medical teaching will be fortified and be somewhat protected from further disruptions.

Notes on Contributors

Lee De Zhang graduated with a degree in Statistics and Computer Science. He reviewed the literature, analysed the data and wrote part of the manuscript.

Eng Tat Ang, Ph.D., is a senior lecturer in anatomy at the Department of Anatomy at the YLLSoM, NUS. He reviewed the literature, designed the research, collected and analysed the data. He developed the manuscript.

Choo Jiayi, BSc (Hons) graduated with a degree in life sciences. She executed the research, and help collected the data. She contributed to the development of the manuscript.

M Chandrika, MBBS, DO, MSc is an instructor at the Department of Anatomy at the YLLSoM, NUS. She helped to execute the research and collected the data.

Ng Li Shia, MBBS, Master of Medicine (Otorhinolaryngology), MRCS(Glasg) is a consultant at the Department of Otolaryngology, Head & Neck Surgery (ENT), National University Hospital. She developed the SCT questions.

Ethical Approval

This project has received full IRB and Ethical clearance (NUS IRB: B-16-205).

Acknowledgements

A big thank you to all students who took part in the research, and to the CDTL, NUS, for providing a teaching enhancement funds to support this research. Appreciation also due to Dr Patricia Chen (Dept. of Psychology, NUS) for her helpful advice.

Funding

NUS TEG AY2017/2018 was awarded to help the investigators pay Mr De Zhang Lee for the statistical modelling that gamification drove medical education via a MAZE.

Declaration of Interest

All authors have no conflict of interest to declare.

References

Agresti, A. (2003). Categorical data analysis. John Wiley & Sons.

Ang, E. T., Abu Talib, S. N., Samarasekera, D., Thong, M., & Charn, T. C. (2017). Using video in medical education: What it takes to succeed. The Asia Pacific Scholar. 2(3), 15-21.

Ang, E. T., Chan, J. M., Gopal, V., & Li Shia, N. (2018). Gamifying anatomy education. Clinical Anatomy, 31(7), 997-1005. https://doi.org/10.1002/ca.23249

Boyatzis, R. E., Murphy, A. J., & Wheeler, J. V. (2000). Philosophy as a missing link between values and behaviour. Psychological Reports, 86(1), 47-64. https://doi.org/10.2466/pr0.2000.86.1.47

Burgess, J., Watt, K., Kimble, R. M., & Cameron, C. M. (2018). Combining Technology and Research to Prevent Scald Injuries (the Cool Runnings Intervention): Randomized Controlled Trial. Journal of Medical Internet Research, 20(10), e10361. http://doi.org/10.2196/10361

Cazan, A. M., & Schiopca, B. A. (2014). Self-directed learning, personality traits and academic achievement. Procedia-Social and Behavioral Sciences (127), 640-644.

Chessare, J. B. (1998). Teaching clinical decision-making to pediatric residents in an era of managed care. Pediatrics, 101(4 Pt 2), 762-766; discussion 766-767. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/9544180

Choudhry, F. R., Ming, L. C., Munawar, K., Zaidi, S. T. R., Patel, R. P., Khan, T. M., & Elmer, S. (2019). Health literacy studies conducted in australia: A scoping review. International Journal of Environmental Research and Public Health, 16(7). https://doi.org/10.3390/ijerph16071112

Cortez, A. R., Winer, L. K., Kassam, A. F., Hanseman, D. J., Kuethe, J. W., Quillin, R. C., 3rd, & Potts, J. R., 3rd. (2019). See none, do some, teach none: An analysis of the contemporary operative experience as nonprimary surgeon. Journal of Surgical Education, 76(6), e92-e101. https://doi.org/10.1016/j.jsurg.2019.05.007

Cote, L., Rocque, R., & Audetat, M. C. (2017). Content and conceptual frameworks of psychology and social work preceptor feedback related to the educational requests of family medicine residents. Patient Education and Counseling, 100(6), 1194-1202. https://doi.org/10.1016/j.pec.2017.01.012

Douw, L., van Dellen, E., Gouw, A. A., Griffa, A., de Haan, W., van den Heuvel, M., Hillebrand, A., Van Mieghem, P., Nissen, I. A., Otte, W. M., & Reijmer, Y. D. (2019). The road ahead in clinical network neuroscience. Network Neuroscience, 3(4), 969-993. https://doi.org/10.1162/netn_a_00103 

Evans, K. H., Daines, W., Tsui, J., Strehlow, M., Maggio, P., & Shieh, L. (2015). Septris: a novel, mobile, online, simulation game that improves sepsis recognition and management. Academic Medicine, 90(2), 180-184. https://doi.org/10.1097/ACM.0000000000000611

Felszeghy, S., Pasonen-Seppänen, S., Koskela, A., Nieminen, P., Härkönen, K., Paldanius, K. M., Gabbouj, S., Ketola, K., Hiltunen, M., Lundin, M., & Haapaniemi, T.  (2019). Using online game-based platforms to improve student performance and engagement in histology teaching. BMC Medical Education, 19(1), 273. https://doi.org/10.1186/s12909-019-1701-0

Gallagher, S., Wallace, S., Nathan, Y., & McGrath, D. (2015). ‘Soft and fluffy’: medical students’ attitudes towards psychology in medical education. Journal of Health Psychology, 20(1), 91-101. https://doi.org/10.1177/1359105313499780

Goyal, S., Nunn, C. A., Rotondi, M., Couperthwaite, A. B., Reiser, S., Simone, A., Katzman, D. K., Cafazzo, J. A., & Palmert, M. R. (2017). A mobile app for the self-management of Type 1 Diabetes among adolescents: A randomized controlled trial. Journal of Medical Internet Research  mHealth and uHealth, 5(6), e82. https://doi.org/10.2196/mhealth.7336

Graafland, M., Bemelman, W. A., & Schijven, M. P. (2017). Game-based training improves the surgeon’s situational awareness in the operation room: a randomized controlled trial. Surgical Endoscopy, 31(10), 4093-4101. https://doi.org/10.1007/s00464-017-5456-6

Graafland, M., Schraagen, J. M., & Schijven, M. P. (2012). Systematic review of serious games for medical education and surgical skills training. British Journal of Surgery, 99(10), 1322-1330. https://doi.org/10.1002/bjs.8819

Graafland, M., Vollebergh, M. F., Lagarde, S. M., van Haperen, M., Bemelman, W. A., & Schijven, M. P. (2014). A serious game can be a valid method to train clinical decision-making in surgery. World Journal of Surgery, 38(12), 3056-3062. https://doi.org/10.1007/s00268-014-2743-4

Hale, A. J., Ricotta, D. N., Freed, J., Smith, C. C., & Huang, G. C. (2019). Adapting Maslow’s Hierarchy of Needs as a Framework for Resident Wellness. Teaching and Learning in Medicine, 31(1), 109-118. https://doi.org/10.1080/10401334.2018.1456928

Howarth-Hockey, G., & Stride, P. (2002). Can medical education be fun as well as educational? British Medical Journal, 325(7378), 1453-1454.  https://doi.org/10.1136/bmj.325.7378.1453 

Kollei, I., Lukas, C. A., Loeber, S., & Berking, M. (2017). An app-based blended intervention to reduce body dissatisfaction: A randomized controlled pilot study. Journal of Consulting and Clinical Psychology, 85(11), 1104-1108. https://doi.org/10.1037/ccp0000246  

Kouwenhoven-Pasmooij, T. A., Robroek, S. J., Ling, S. W., van Rosmalen, J., van Rossum, E. F., Burdorf, A., & Hunink, M. G. (2017). A blended web-based gaming intervention on changes in physical activity for overweight and obese employees: Influence and usage in an experimental pilot study. Journal of Medical Internet Research   Serious Games, 5(2), e6. https://doi.org/10.2196/games.6421

Kurtzman, G. W., Day, S. C., Small, D. S., Lynch, M., Zhu, J., Wang, W., Rareshide, C. A., & Patel, M. S. (2018). Social incentives and gamification to promote weight loss: The lose it randomized, controlled trial. Journal of General Internal Medicine, 33(10), 1669-1675. https://doi.org/10.1007/s11606-018-4552-1  

Lubarsky, S., Dory, V., Duggan, P., Gagnon, R., & Charlin, B. (2013). Script concordance testing: from theory to practice: AMEE guide no. 75. Medical Teacher, 35(3), 184-193. https://doi.org/10.3109/0142159X.2013.760036

Lubarsky, S., Dory, V., Meterissian, S., Lambert, C., & Gagnon, R. (2018). Examining the effects of gaming and guessing on script concordance test scores. Perspectives on Medical Education, 7(3), 174-181. https://doi.org/10.1007/s40037-018-0435-8  

Michael, K., Dror, M. G., & Karnieli-Miller, O. (2019). Students’ patient-centered-care attitudes: The contribution of self-efficacy, communication, and empathy. Patient Education and Counseling. https://doi.org/10.1016/j.pec.2019.06.004

Muis, K. R., Winne, P. H., & Jamieson-Noel, D. (2007). Using a multitrait-multimethod analysis to examine conceptual similarities of three self-regulated learning inventories. British Journal of Educational Psychology, 77(Pt 1), 177-195. https://doi.org/10.1348/000709905X90876

Mullikin, T. C., Shahi, V., Grbic, D., Pawlina, W., & Hafferty, F. W. (2019). First year medical student peer nominations of professionalism: A methodological detective story about making sense of non-sense. Anatomical Sciences Education, 12(1), 20-31. https://doi.org/10.1002/ase.1782 

Nevin, C. R., Westfall, A. O., Rodriguez, J. M., Dempsey, D. M., Cherrington, A., Roy, B., Patel, M., & Willig, J. H. (2014). Gamification as a tool for enhancing graduate medical education. Postgraduate Medical Journal, 90(1070), 685-693. https://doi.org/10.1136/postgradmedj-2013-132486

Nicola, S., Virag, I., & Stoicu-Tivadar, L. (2017). vr medical gamification for training and education. Studies in Health Technology and Informatics, 236, 97-103. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/28508784 

O’Connor, D., Brennan, L., & Caulfield, B. (2018). The use of neuromuscular electrical stimulation (NMES) for managing the complications of ageing related to reduced exercise participation. Maturitas, 113, 13-20. https://doi.org/10.1016/j.maturitas.2018.04.009

Paros, S., & Tilburt, J. (2018). Navigating conflict and difference in medical education: insights from moral psychology. BMC Medical Education, 18(1), 273. https://doi.org/10.1186/s12909-018-1383-z

Patel, M. S., Benjamin, E. J., Volpp, K. G., Fox, C. S., Small, D. S., Massaro, J. M., Lee, J. J., Hilbert, V., Valentino, M., Taylor, D. H., & Manders, E. S.  (2017). effect of a game-based intervention designed to enhance social incentives to increase physical activity among families: The BE FIT randomized clinical trial. Journal of the American Medical Association Internal Medicine, 177(11), 1586-1593. https://doi.org/10.1001/jamainternmed.2017.3458  

Pickren, W. (2007). Psychology and medical education: A historical perspective from the United States. Indian Journal of Psychiatry, 49(3), 179-181. https://doi.org/10.4103/0019-5545.37318  

Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R., & Maxwell, B. (2015). Randomized controlled trial of superbetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games for Health Journal, 4(3), 235-246. https://doi.org/10.1089/g4h.2014.0046

Rutledge, C., Walsh, C. M., Swinger, N., Auerbach, M., Castro, D., Dewan, M., Khattab, M., Rake, A., Harwayne-Gidansky, I., Raymond, T. T., & Maa, T. (2018). Gamification in action: Theoretical and practical considerations for medical educators. Academic Medicine, 93(7), 1014-1020. https://doi.org/10.1097/ACM.0000000000002183

Savulich, G., Piercy, T., Fox, C., Suckling, J., Rowe, J. B., O’Brien, J. T., & Sahakian, B. J. (2017). Cognitive training using a novel memory game on an ipad in patients with amnestic mild cognitive impairment (aMCI). International Journal of Neuropsychopharmacology, 20(8), 624-633. https://doi.org/10.1093/ijnp/pyx040

Shah, A., Carter, T., Kuwani, T., & Sharpe, R. (2013). Simulation to develop tomorrow’s medical registrar. The Clinical Teacher, 10(1), 42-46. https://doi.org/10.1111/j.1743-498X.2012.00598.x  

Sheehan, D., Bagg, W., de Beer, W., Child, S., Hazell, W., Rudland, J., & Wilkinson, T. J. (2010). The good apprentice in medical education. New Zealand Medical Journal, 123(1308), 89-96. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/20201158

Sheikhnezhad Fard, F., & Trappenberg, T. P. (2019). A novel model for arbitration between planning and habitual control systems. Frontiers in Neurorobotics, 13, 52. https://doi.org/10.3389/fnbot.2019.00052

Team, R. C. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

Turan, S., Demirel, O., & Sayek, I. (2009). Metacognitive awareness and self-regulated learning skills of medical students in different medical curricula. Medical Teacher, 31(10), e477-483. https://doi.org/10.3109/01421590903193521

van Houten-Schat, M. A., Berkhout, J. J., van Dijk, N., Endedijk, M. D., Jaarsma, A. D. C., & Diemers, A. D. (2018). Self-regulated learning in the clinical context: A systematic review. Medical Education, 52(10), 1008-1015. https://doi.org/10.1111/medu.13615

Van Nuland, S. E., Roach, V. A., Wilson, T. D., & Belliveau, D. J. (2015). Head to head: The role of academic competition in undergraduate anatomical education. Anatomical Sciences Education, 8(5), 404-412. https://doi.org/10.1002/ase.1498

Villavicencio, F. T., & Bernardo, A. B. (2013). Positive academic emotions moderate the relationship between self-regulation and academic achievement. British Journal of Educational Psychology, 83(Pt 2), 329-340. https://doi.org/10.1111/j.2044-8279.2012.02064.x

Vohs, K. D., Baumeister, R. F., Schmeichel, B. J., Twenge, J. M., Nelson, N. M., & Tice, D. M. (2008). Making choices impairs subsequent self-control: A limited-resource account of decision making, self-regulation, and active initiative. Journal of Personality and Social Psychology, 94(5), 883-898. https://doi.org/10.1037/0022-3514.94.5.883

Wan, M. S., Tor, E., & Hudson, J. N. (2018). Improving the validity of script concordance testing by optimising and balancing items. Medical Education, 52(3), 336-346. https://doi.org/10.1111/medu.13495  

Wisniewski, A. B., & Tishelman, A. C. (2019). Psychological perspectives to early surgery in the management of disorders/differences of sex development. Current Opinion in Pediatrics, 31(4), 570-574. https://doi.org/10.1097/MOP.0000000000000784

Yue, P., Zhu, Z., Wang, Y., Xu, Y., Li, J., Lamb, K. V., Xu, Y., & Wu, Y. (2019). Determining the motivations of family members to undertake cardiopulmonary resuscitation training through grounded theory. Journal of Advanced Nursing, 75(4), 834-849. https://doi.org/10.1111/jan.13923

*Ang Eng Tat
Department of Anatomy
Yong Loo Lin School of Medicine
MD10, National University of Singapore
Singapore 117599
Email address: antaet@nus.edu.sg

Submitted: 14 March 2020
Accepted: 20 July 2020
Published online: 5 January, TAPS 2021, 6(1), 30-39
https://doi.org/10.29060/TAPS.2021-6-1/OA2235

Yit Shiang Lui, Abigail HY Loh, Tji Tjian Chee, Jia Ying Teng, John Chee Meng Wong & Celine Hsia Jia Wong

Department of Psychological Medicine, National University Health System, Singapore

Abstract

Introduction: A good understanding of basic child-and-adolescent psychiatry (CAP) is important for general medical practice. The undergraduate psychiatry teaching programme included various adult and CAP topics within a six-week time frame. A team of psychiatry tutors developed two new teaching formats for CAP and obtained feedback from the students about these teaching activities.

Methods: Medical students were introduced to CAP via small group teaching in two different modes. One mode was the “Clinical Vignettes Tutorial” (CVT) and the other mode “Observed Clinical Interview Tutorial” (OCIT). In CVT, tutors would discuss clinical vignettes of real patients with the students, followed by explanations about theoretical concepts and management strategies. OCIT involved simulated-patients (SPs) who assisted by acting as patients presenting with problems related to CAP, or as parents for such patients. At each session, students were given the opportunity to interview “patients” and “parents”. Feedback was given following these interviews. The students then completed surveys about the teaching methods.

Results: Students rated very-positive feedback for the teaching of CAP in small groups. Almost all found these small groups enjoyable and that it helped them apply what they had learnt. Majority agreed that the OCIT sessions increased their level of confidence in speaking with adolescents and parents. Some students agreed that these sessions had stimulated their interest to know more about CAP.

Conclusion: Small group teaching in an interactive manner enhanced teaching effectiveness. Participants reported a greater degree of interest towards CAP, and enhanced confidence in treating youths with mental health issues as well as engaging their parents.

Keywords:           Child Adolescent Psychiatry, Medical Education, Small Group, Teaching

Practice Highlights

• Psychiatric disorders are among the most common medical conditions experienced by children and adolescents, and data from the Singapore Mental Health survey conducted in 2010 had shown the prevalence rates of emotional and behavioural problems among Singaporean youth to be at 12.5%.
• Most medical students had limited exposure to Child & Adolescent Psychiatry (CAP) in their medical curriculum due to reduced proportions of teaching time and opportune clinical exposures allocated to CAP programmes.
• This would be further compounded by the limited number of child and adolescent psychiatrists involved in teaching at medical schools and supervising clinical postings.
• This manuscript described synergistic teaching methods employed in educating medical students within the field of Child & Adolescent Psychiatry and examined the effectiveness and acceptability of CAP teaching using small-group teaching classes.
• The CAP small group interactive teaching sessions for medical students received good feedback from majority of the participants and translated to applicability and skillsets transferability.

Figure. 1. Open comment feedback to the survey question “The best things about the sessions were…”

In areas that the students indicated for further improvement, they had cited for a “shorter” duration in each teaching session (Figure 2). This was likely due to the nature of a full day programme of CAP teaching which could last eight hours in a day with a one-hour lunch break. Others had shared that they preferred “smaller” groups so students could get more chances to practice interviewing the SPs and also be provided “more time for discussion” to allow more in-depth feedback as well as discussion of each clinical condition. Some students remarked that Objective Structured Clinical Examination (OSCE) styled marking schemes could help enhance their learning experiences as this method might be more structured, compared to an open discussion.

Figure 2. Open comment feedback to the survey question “Some ways which I think can make the sessions better are…”

IV. DISCUSSION

This study evaluated the effectiveness and acceptability of small group tutorials for CAP conditions, which are packaged inseparably as part of a medical undergraduate psychiatry teaching programme. CVT and OCIT are synergistically designed to complement each other in the curriculum. The surveys used to compile the medical undergraduates’ responses had focused on their learning experience with the CAP curriculum. The effectiveness of the teaching methods namely CVT and OCIT would be determined from transferability of the requisite knowledge base and the clinical skills, as well as availability of opportunities to experience interviewing for the participants. The survey responses were also used to gauge the performance of the SPs and the clinical tutors’ usefulness. In addition, the degree of how impactful the teaching sessions had in generating interest towards CAP was also evaluated.

The fourth-year medical students gave good feedback for the small group teaching sessions. They reported that the CVT were enjoyable, beneficial and had allowed them to apply what they had learnt. For the OCIT, most of the respondents indicated that the session had helped them to learn psychiatric interviewing skills, increased their level of confidence in speaking with adolescents and parents, and had helped them to apply in clinical scenarios what they had learnt. There is discernible difference between the feedback for CVT and OCIT. The students’ feedback for CVT affirmed applicability of the knowledge content of CAP whereas those for OCIT concurred with transferability of interviewing skills in terms of confidence level.  

In the open feedback segment of the survey, respondents reported that they had particularly liked the interactive and hands-on aspect of the session, the frequent opportunities for evaluation and feedback, as well as for practice. However, they highlighted that certain factors such as the size of grouping, the length of the sessions and random allocation of conditions could be improved further to enhance their learning experience. Overall, their feedback still indicated positive experiences in these small group sessions, and this translated to an increased knowledge base, a heightened level of confidence, and burgeoned interest in CAP among the student participants.

This study’s limitations included the challenges inherent with attempting to accurately assess the students’ genuine experiences and feelings towards the sessions; with possible biases (recall and Hawthorne effect) in responding to questionnaires; and the lack of correlation to actual performances in real-world settings. Furthermore, what remained unanswered was how such sessions might truly generate interest leading to possibly pursuit of a career in CAP. In addition, it is uncertain whether changing the teaching methods with the curriculum could inspire more medical students and young doctors to consider specialising in this field and raise the number of residency applications. The data from our study did appear to be consistent with findings from other CAP clinical teaching programmes. In these programmes, more exposure to CAP and increased clinical opportunities did correlate with changes in impressions towards and appreciation of clinical interactions with children, increased positive views of CAP as part of medical practice, and heightened interest in CAP as a field of medical specialty (Dingle, 2010; Kaplan & Lake, 2008; Malloy et al., 2008; Martin, Bennett, & Pitale, 2005).

In the current undergraduate medical curriculum, the amount of time allocated to teaching CAP is relatively small compared to other topics. Child and adolescent psychiatric cases can be particularly complex and their management demand sensitive handling, which may pose challenges to real world practice. Youth patients and their parents may value privacy and sometimes do not allow medical students to be involved in initial assessments and subsequent follow-up consultations. These factors collectively pose unique challenges to teaching and equipping medical students with the skills and knowledge to address child and adolescent mental health disorders. While clinical contact and patient experience would be preferred and desirable for training, it may be impractical given the various constraints mentioned above (Kaplan & Lake, 2008). Hence, other creative methods of “exposure” to CAP patients should be incorporated into teaching rotations to offer medical students the opportunities to expand this knowledge base, apply the knowledge to practice scenarios, and further their clinical and communication skills. Small group sessions such as the CVTs and OCITs are teaching activities that can be used to overcome some of these challenges.

Our study showed that small group interactive teaching is effective in helping medical students to apply what they have learnt about CAP, increase their confidence in speaking to adolescents as patients and learn psychiatric interviewing skills. It also exposes them to a wide range of relevant CAP cases to which they can apply their theoretical knowledge and practice interview and management techniques. Furthermore, we have found that all this can be adequately achieved in a tailored environment that is conducive for learning. The collective constructive feedback had been used to further improve the content and deliverability style so as to enhance implementation in future batches. It has also been conceptualised to compare CVT and OCIT as individual teaching methods for future scholarly research.

V. CONCLUSION

The CAP small group interactive teaching sessions for medical students received good feedback from majority of the participants. This positive validation would spur the authors on to explore further how this pedagogy could help spark interests in Child and Adolescent Psychiatry among medical students given the shortfall of child and adolescent psychiatrists worldwide.

Notes on Contributors

AHYL analysed and interpreted data. CHJW, together with TJY and JCMW planned and conducted the child psychiatry small group teaching and collected feedback data from the medical students. TJY developed the feedback questionnaire. YSL, together with AHYL, CHJW and TTC planned and wrote the manuscript. All authors read and approved the final manuscript.

Ethical Approval

NHG DSRB reference number 2019/00431 for exemption.

Data Availability

Datasets generated and/or analysed during the current study are available from corresponding author on reasonable request.

Acknowledgements

The authors wish to thank the team from Centre for Healthcare Simulation, Yong Loo Lin School of Medicine, National University of Singapore for the invaluable support in recruiting and training the simulated patients for the CAP teaching program. We appreciate the participation of the simulated patients and medical students in the teaching programme.

Funding

There is no funding for this paper.

Declaration of Interest

As far as all the authors are concerned, we do not know of, or foresee any future competing interests. We are not aware of any issues relating to journal policies in submitting this manuscript. All the authors have approved of the manuscript for submission. The authors declare that they have no competing interests.

References

Baranne, M. L., & Falissard, B. (2018). Global burden of mental disorders among children aged 5–14 years. Child and Adolescent Psychiatry and Mental Health, 12(1), 19.

Breton, J. J., Plante, M. A., & St-Georges, M. (2005). Challenges facing child psychiatry in Quebec at the dawn of the 21st Century. The Canadian Journal of Psychiatry, 50(4), 203-212.

Costello, E. J., Egger, H., & Angold, A. (2005). 10-year research update review: The epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. Journal of the American Academy of Child & Adolescent Psychiatry, 44(10), 972-986.

Dingle, A. D. (2010). Child psychiatry: What are we teaching medical students? Academic Psychiatry, 34(3), 175-182.

Erskine, H. E., Moffitt, T. E., Copeland, W. E., Costello, E. J., Ferrari, A. J., Patton, G., … & Scott, J. G. (2015). A heavy burden on young minds: The global burden of mental and substance use disorders in children and youth. Psychological Medicine, 45(7), 1551-1563.

Hunt, J., Barrett, R., Grapentine, W. L., Liguori, G., & Trivedi, H. K. (2008). Exposure to child and adolescent psychiatry for medical students: Are there optimal “teaching perspectives”?. Academic Psychiatry, 32(5), 357-361.

Kaplan, J. S., & Lake, M. (2008). Exposing medical students to child and adolescent psychiatry: A case-based seminar. Academic Psychiatry, 32(5), 362-365.

Kyu, H. H., Pinho, C., Wagner, J. A., Brown, J. C., Bertozzi-Villa, A., Charlson, F. J., … & Fitzmaurice, C. (2016). Global and national burden of diseases and injuries among children and adolescents between 1990 and 2013: Findings from the global burden of disease 2013 study. JAMA Pediatrics, 170(3), 267-287.

Lim, C. G., Ong, S. H., Chin, C. H., & Fung, D. S. S. (2015). Child and adolescent psychiatry services in Singapore. Child and Adolescent Psychiatry and Mental Health, 9(1), 7.

Malloy, E., Hollar, D., & Lindsey, B. A. (2008). Increasing interest in child and adolescent psychiatry in the third-year clerkship: Results from a post-clerkship survey. Academic Psychiatry, 32(5), 350-356.

Martin, V. L., Bennett, D. S., & Pitale, M. (2005). Medical students’ perceptions of child psychiatry: Pre-and post-psychiatry clerkship. Academic Psychiatry, 29(4), 362-367.

Plan, S. (2002). A Call to Action: Children Need Our Help! American Academy of Child & Adolescent Psychiatry. Retrieved from https://www.aacap.org/app_themes/aacap/docs/resources_for_primary_care/workforce_issues/AACAP_Call_to_Action.pdf

Sawyer, M., & Giesen, F. (2007). Undergraduate teaching of child and adolescent psychiatry in Australia: Survey of current practice. Australian & New Zealand Journal of Psychiatry, 41(8), 675-681.

Thomas, C. R., & Holzer, C. E., 3rd (2006). The continuing shortage of child and adolescent psychiatrists. Journal of the American Academy of Child & Adolescent Psychiatry, 45(9), 1023-1031.

World Health Organization. (2014). Adolescent health epidemiology. Retrieved from http://www.who.int/maternal_child_adolescent/epidemiology/adolescence/en/

*Yit Shiang Lui
1E Kent Ridge Road
Tower Block, Level 9,
Singapore 119228
Tel: 6772 6331
Email address: yit_shiang_lui@nuhs.edu.sg

Submitted: 14 February 2020
Accepted: 1 July 2020
Published online: 5 January, TAPS 2021, 6(1), 40-48
https://doi.org/10.29060/TAPS.2021-6-1/OA2227

Shirley Beng Suat Ooi^1,2, Clement Woon Teck Tan^3,4 & Janneke M. Frambach⁵

¹Emergency Medicine Department, National University Hospital, National University Health System, Singapore; ²Department of Surgery, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Department of Ophthalmology, National University Hospital, National University Health System, Singapore; ⁴Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ⁵School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands

Abstract

Introduction: Almost all published literature on effective clinical teachers were from western countries and only two compared medical students with residents. Hence, this study aims to explore the perceived characteristics of effective clinical teachers among medical students compared to residents graduating from an Asian medical school, and specifically whether there are differences between cognitive and non-cognitive domain skills, to inform faculty development.

Methods: This qualitative study was conducted at the National University Health System (NUHS), Singapore involving six final year medical students at the National University of Singapore, and six residents from the NUHS Residency programme. Analysis of the semi-structured one-on-one interviews was done using a 3-step approach based on principles of Grounded Theory.

Results: There are differences in the perceptions of effective clinical teachers between medical students and residents. Medical students valued a more didactic spoon-feeding type of teacher in their earlier clinical years. However final year medical students and residents valued feedback and role-modelling at clinical practice. The top two characteristics of approachability and passion for teaching are in the non-cognitive domains. These seem foundational and lead to the acquisition of effective teaching skills such as the ability to simplify complex concepts and creating a conducive learning environment. Being exam-oriented is a new characteristic not identified before in “Western-dominated” publications.

Conclusion: The results of this study will help to inform educators of the differences in a learner’s needs at different stages of their clinical development and to potentially adapt their teaching styles.

Keywords:           Clinical Teachers, Medical Students, Residents, Cognitive/Non-Cognitive, Asian Healthcare, Faculty Development

Practice Highlights

• Approachability and teaching passion are foundational non-cognitive skills in effective clinical teachers.
• These foundational skills are more important for undergraduate than postgraduate teaching.
• Procedural residents can accept less ‘warm’ teachers if they can learn advanced clinical skills.
• Medical students value didactic ‘spoon-feeding’ type of teachers in their earlier clinical years.
• Final year medical students and residents value feedback and role-modelling at clinical practice.

I. INTRODUCTION

“The transformation of our students requires the engagement of innovative and outstanding clinician-teachers who not only supervise students in their development of technical skills and applied knowledge but also serve as role models of the values and attributes of the profession and of the life of a professional” (Sutkin, Wagner, Harris, & Schiffer, 2008). This statement nicely encapsulates the very important role played by outstanding clinical teachers in helping students to ultimately become professionals with the attributes our healthcare system desires. Previous research has extensively investigated characteristics of effective clinical teachers to inform faculty development (e.g. Branch, Osterberg, & Weil, 2015; Hatem et al., 2011; Hillard, 1990; Kernan, Lee, Stone, Freudigman, & O’Connor, 2000; Paukert & Richards, 2000; Singh et al., 2013; Sutkin et al., 2008; White & Anderson, 1995). However, despite the large body of existing research on effective clinical teaching, two issues related to the needs of different groups of learners need further investigation to enable more tailored faculty development.

First, effective clinical teaching may look different in undergraduate as compared with postgraduate education. In many healthcare institutions, clinical teachers are expected to teach across the medical education continuum, i.e., undergraduate medical students, graduate doctors in training, as well as part of continuing medical education, and teaching abilities are a necessary prerequisite in an academic environment (Hatem et al., 2011). Based on the conceptual framework of constructivism (Bednar, Cunningham, Duffy, & Perry, 1991)—­a theory which equates learning with creating meaning from experience or contextual learning­—Jonassen (1991) argues that constructive learning environments are most effective for acquiring knowledge in the advanced stage of knowledge, the stage between introductory and expert. According to Jonassen (1991), the initial or introductory stage of knowledge acquisition occurs when learners have very little directly transferable prior knowledge about a skill or content area. In this stage, knowledge is best acquired through more objectivistic approaches which can be described as ‘spoon-feeding’. Medical students in general would fit into this introductory stage, in varying degrees depending on their seniority and individual progress in learning. Jonassen’s (1991) second stage is advanced knowledge acquisition where the domains are ill-structured and more knowledge-based. This is in contrast to his third or final stage of knowledge acquisition of experts that require very little instructional support but are able to deal with elaborate structures, schematic patterns and seeing the interconnectedness in knowledge through experience. The stage of junior doctors in training would be fit into the second or advanced stage of learning. Constructivist teachers help students construct knowledge to become active learners rather than passive recipients of knowledge from the teachers or textbooks. In view of this constructivist framework, it appears logical to postulate that as medical students mature to become practicing doctors, their perceptions of effective clinical teachers may change from one who ‘spoon-feeds’ them with medical knowledge to one who encourages them to actively construct new meaning as they become clinically more experienced and have to deal with complex and ill-defined problems. Low, Khoo, Kuan, and Ooi (2020) showed that although the top four characteristics of effective medical teachers are consistent across all 5 years of medical school, characteristics that facilitate active learner participation are emphasised in the clinical years consistent with constructivist learning theory. However, as there is a paucity of comparative research on perceptions of effective clinical teachers among undergraduates as compared to postgraduates to plan more focused faculty development to address the attributes the learners look for in their clinical teachers, this warrants further research.

The second issue relates to potential differences in the clinical teaching role between Asian and Western settings. In Western studies, as noted above, effective clinical teachers are encouraged to stimulate students’ intellectual curiosity leading to more self-directed learning (Hillard, 1990; Kernan et al., 2000; White & Anderson, 1995). In contrast, feelings of uncertainty about the independence required in self-directed learning, a focus on tradition that respects ‘old ways’, hierarchy expecting ‘truths’ to come from persons of higher status, and an achievement orientation to pass and excel in examinations have been identified as more prominent in non-Western than in Western cultures (Frambach, Driessen, Chan, & van der Vleuten, 2012). This is despite the recent introductions of more student-centred education methods. In Singapore for example, there is a move in the Yong Loo Lin School of Medicine (YLLSoM) to try to embed students into healthcare teams (Jacobs & Samarasekera, 2012) and implement newer methods of learning such as flipped classroom. However, many teachers still employ traditional methods of lectures and small group tutorials focused on exam preparation. A comprehensive review study of 68 articles on effective clinical teaching (Sutkin et al., 2008), comprised only one article that reported research from a non-Western setting (Elzubeir & Rizk, 2001). In this article, originating from the United Arab Emirates, there is no discussion on whether there is a difference in the perception of a role model between medical students in Asian countries compared to the West (Elzubeir & Rizk, 2001). Another study conducted in Asia showed differences in the perceptions of first-year and fifth-year medical students in Singapore on what makes an effective medical teacher (Kua, Voon, Tan, & Goh, 2006). More first-year students preferred handouts in contrast to fifth-year students who were less reliant on ‘spoon-feeding’. Research on effective clinical teaching is growing in the Asian setting (Ciraj et al., 2013;  Haider, Snead, & Bari, 2016; Kikukawa et al., 2013; Mohan & Chia, 2017; Nishiya et al., 2019; Venkataramani et al., 2016) though there is still a paucity of literature in the Asian setting compared with studies conducted in the West and there are none that directly compared medical students with residents.

Another issue that deserves further attention is the role of non-cognitive domain skills in clinical teaching. Sutkin et al.’s (2008) review study described three main categories of characteristics of good clinical teachers: 1) physician characteristics, 2) teacher characteristics, and 3) human characteristics (Table 1). Approximately two-thirds of the characteristics were in non-cognitive domains (such as those involving relationship skills, emotional states, and personality types), and one-third in cognitive domains (such as those involving reasoning, memory, judgment, perception, and procedural skills). The article noted that cognitive abilities can be taught and learned, in contrast to non-cognitive attributes which are more difficult to develop and teach. Faculty development programmes currently often focus on traditional cognitive skills, such as curriculum design, large-group teaching, and assessment of learners (Searle, Hatem, Perkowski, & Wilkerson, 2006). In contrast, if non-cognitive domains are more important in contributing to outstanding teaching, they might need greater emphasis in the curricula of these workshops. The good news is that according to Schiffer, Rao, and Fogel (2003), non-cognitive behaviours are both measurable and alterable. Most of them have underlying neural networks which are entering our sphere of understanding. Hence non-cognitive skills, although much more challenging to develop than cognitive skills, have a potential to be developed. It is not clear whether there are differences in the distribution between cognitive and non-cognitive domains skills between the perceptions of medical students compared to residents of an effective clinical teacher.

The aim of this qualitative study is to explore the perceived characteristics of an effective clinical teacher among medical students compared to residents graduating from an Asian medical school and whether there is a difference regarding cognitive and non-cognitive domain skills.

II. METHODS

A. Participants

The participants consisted of final/fifth year medical students (M5s) from the Yong Loo Lin School of Medicine (YLLSoM), National University of Singapore (NUS) who were posted to the National University Hospital (NUH) to do their student internship posting in 2016. To ensure sufficient working experience, the National University Health System (NUHS) residents who had graduated from the YLLSoM and who had recently completed their intermediate specialty examinations were recruited. These were third to fifth year residents in different programmes. Maximal variation sampling of the M5s and the residents of both gender, different ethnic groups and from different specialties (for residents only) was done.

B. Design

A pragmatic qualitative research design (Savin-Baden & Howell Major, 2013) was used to get the participants to reflect on their own learning journey affecting their perceptions of the qualities that make an effective clinical teacher from the time they were first exposed to clinical medicine in year 3 (M3) of medical school to final year (M5) for the students, and to residency for the residents.

 C. Data Collection

Semi-structured one-on-one interviews using open-ended questions were conducted. A list of M5s doing their student internship programme in the various departments in NUH was invited via an e-mail invitation to participate in this study. To ensure maximal variation sampling, M5s of both gender and as far as possible different ethnic groups were recruited. As for the residents, through the Graduate Medicine Education Office in NUH, residents of both gender, from different ethnic groups and different specialties (both procedural and non-procedural) were selected from those who responded voluntarily to the invitation to participate in this study to ensure maximal variation sampling as residents from procedural specialties may have different perceptions of effective clinical teachers from non-procedural specialties.

Written consent after reading the Participant Information Sheet was taken from the interviewees before the interview was conducted in a quiet room. The interview was audiotaped and lasted between 30 and 45 minutes.

D. Data Analysis

The audiotaped interviews were transcribed. As all the 12 interviews were conducted by the principal investigator (PI) (SO) and although the coding and official analysis of the interviews were done after all the 12 interviews were transcribed, the PI had taken note of themes emerging and decided on ending the interviews after no substantial new themes had emerged.

In the first phase, open coding, initial categories of the information on characteristics of effective clinical teachers by segmenting information and assigning open codes were formed. In the second coding phase, broader categories were developed through conceptually related ideas. The third phase involved selective coding where the individual categories were counterchecked with Sutkin et al.’s (2008) categories of teacher, physician and human characteristics and whether they were in the cognitive or non-cognitive domains (Table 1). Further related categories according to Sutkin et al.’s (2008) classification were brought together.

Physician Characteristics

Submitted: 15 April 2020
Accepted: 5 June 2020
Published online: 5 January, TAPS 2021, 6(1), 49-59
https://doi.org/10.29060/TAPS.2021-6-1/OA2248

Amaya Tharindi Ellawala¹, Madawa Chandratilake² & Nilanthi de Silva²

¹Department of Medical Education, Faculty of Medical Sciences, University of Sri Jayewardenepura, Sri Lanka; ²Faculty of Medicine, University of Kelaniya, Sri Lanka

Abstract

Introduction: Professionalism is a context-specific entity, and should be defined in relation to a country’s socio-cultural backdrop. This study aimed to develop a framework of medical professionalism relevant to the Sri Lankan context.

Methods: An online Delphi study was conducted with local stakeholders of healthcare, to achieve consensus on the essential attributes of professionalism for a doctor in Sri Lanka. These were built into a framework of professionalism using qualitative and quantitative methods.

Results: Forty-six attributes of professionalism were identified as essential, based on Content Validity Index supplemented by Kappa ratings. ‘Possessing adequate knowledge and skills’, ‘displaying a sense of responsibility’ and ‘being compassionate and caring’ emerged as the highest rated items. The proposed framework has three domains: professionalism as an individual, professionalism in interactions with patients and co-workers and professionalism in fulfilling expectations of the profession and society, and displays certain characteristics unique to the local context.

Conclusion: This study enabled the development of a culturally relevant, conceptual framework of professionalism as grounded in the views of multiple stakeholders of healthcare in Sri Lanka, and prioritisation of the most essential attributes.

Keywords:           Professionalism, Culture, Consensus

Practice Highlights

• Medical professionalism is recognised as a culturally dependent entity.
• This has led to the emergence of definitions unique to socio-cultural settings.
• List-based definitions provide operationalisable means of portraying its meaning.
• A Delphi study was conducted to achieve consensus on locally relevant professionalism attributes.
• Using quantitative and qualitative methods, a conceptual framework of professionalism was developed.

I. INTRODUCTION

There is no single definition of medical professionalism that encompasses its many subtle nuances (Birden et al., 2014).  The realisation that professionalism is a dynamic, multi-dimensional entity (Van de Camp, Vernooij-Dassen, Grol, & Bottema, 2004), significantly dependent on context (Van Mook et al., 2009), and cultural backdrop (Chandratilake, Mcaleer, & Gibson, 2012), has led to the emergence of definitions specific to cultures and socio-economic backgrounds.

Many of the current definitions originate from Western societies. Certain Eastern cultures have embraced such definitions, though they are undeniably in conflict with local traditional views (Pan, Norris, Liang, Li, & Ho, 2013). In parallel however, countries such as Egypt, Saudi Arabia, Japan, China and Taiwan have explored how professionalism is conceptualised within their contexts (Al-Eraky, Chandratilake, Wajid, Donkers, & Van Merrienboer, 2014; Leung, Hsu, & Hui, 2012; Pan et al., 2013). Such studies have portrayed the interplay between cultural, socio-economic and religious factors in shaping perceptions on professionalism, further fuelling the notion that professionalism must be “interpreted in view of local traditions and ethos” (Al-Eraky et al., 2014, p. 14).

Culture is the embodiment of elements such as attitudes, beliefs and values that are shared among individuals of a community and is therefore, an entity that distinguishes one group of people from another (Hofstede, 2011). Various cultural theories provide insight into inter-cultural differences across the globe (Hofstede, n.d.; Schwartz, 1999). The Sri Lankan cultural context, while aligned with those of its closest geographical neighbours in South Asia in some ways, differs from them in other important aspects. 

Certain attempts have been made to explore the meaning of professionalism in Sri Lanka. Chandratilake et al. (2012) provided a degree of insight while comparing cultural similarities and dissonances in conceptualising professionalism among doctors of several nations. Monrouxe, Chandratilake, Gosselin, Rees, and Ho (2017) built on this work with their analysis of professionalism as viewed by local medical students. The sole regulatory authority of the medical profession in the country, the Sri Lanka Medical Council (SLMC, 2009) has delineated what it expects in terms of professionalism, by outlining the constituents of ‘good medical practice’, many of which converge with elements of professionalism described in the literature.

While the work mentioned here has shed some light on the topic, to our knowledge, there were no studies that focused solely on the local conceptualisation of professionalism, drawing on the views of diverse stakeholders of healthcare.

There exist two schools of thought on how professionalism can be defined: as a list of desirable attributes (Lesser et al., 2010), or as an over-arching, value-laden entity that transcends such lists (Irby & Hamstra, 2016; Wynia, Papadakis, Sullivan, & Hafferty, 2014). Unlike the latter, a list may not address the “foundational purpose of professionalism” (Wynia et al., 2014, p. 712); however, it will provide a tangible, operationalisable portrayal (Lesser et al., 2010). It is possibly for this reason that many studies have opted for list-based definitions, an approach that is supported in the East (Al-Eraky & Chandratilake, 2012; Al-Eraky et al., 2014; Pan et al., 2013).

The aim of this study was to develop a culturally appropriate conceptual framework of medical professionalism in Sri Lanka using a combination of qualitative and quantitative methods. We envisioned that identifying a list of desirable attributes would be appropriate, providing a definition that could readily be operationalised for teaching/learning, assessment and research purposes (Wilkinson, Wade, & Knock, 2009).

II. METHODS

A. The Approach

We followed a consensus approach, and opted for the Delphi technique as it was imperative to involve a large number of participants not limited by geographical location (Humphrey-Murto et al., 2017). The method offered the further advantage of providing participants with equal opportunity to express their opinions (De Villiers, De Villiers, & Kent, 2005), thereby negating the possible drawbacks of face-to-face interactions and resulting in a ‘process gain’ (Powell, 2003).

B. Participant Panel

The panel comprised nation-wide stakeholders of healthcare (Table 1), from both rural and urban regions who were presumably exposed to diverse forms of medical services and geographical variations in their distribution.

Submitted: 17 March 2020
Accepted: 3 June 2020
Published online: 5 January, TAPS 2021, 6(1), 60-69
https://doi.org/10.29060/TAPS.2021-6-1/OA2239

Frank Bate¹, Sue Fyfe², Dylan Griffiths¹, Kylie Russell¹, Chris Skinner¹, Elina Tor¹

¹University of Notre Dame Australia, Australia; ²Curtin University, Australia

Abstract

Introduction: In 2017, the School of Medicine of the University of Notre Dame Australia implemented a data-informed mentoring program as part of a more substantial shift towards programmatic assessment. Data-informed mentoring, in an educational context, can be challenging with boundaries between mentor, coach and assessor roles sometimes blurred. Mentors may be required to concurrently develop trust relationships, guide learning and development, and assess student performance. The place of data-informed mentoring within an overall assessment design can also be ambiguous. This paper is a preliminary evaluation study of the implementation of data informed mentoring at a medical school, focusing specifically on how students and staff reacted and responded to the initiative.

Methods: Action research framed and guided the conduct of the research. Mixed methods, involving qualitative and quantitative tools, were used with data collected from students through questionnaires and mentors through focus groups.

Results: Both students and mentors appreciated data-informed mentoring and indications are that it is an effective augmentation to the School’s educational program, serving as a useful step towards the implementation of programmatic assessment.

Conclusion: Although data-informed mentoring is valued by students and mentors, more work is required to: better integrate it with assessment policies and practices; stimulate students’ intrinsic motivation; improve task design and feedback processes; develop consistent learner-centred approaches to mentoring; and support data-informed mentoring with appropriate information and communications technologies. The initiative is described using an ecological model that may be useful to organisations considering data-informed mentoring.

Keywords:            Data-Informed Mentoring, Mentoring, Programmatic Assessment, E-Portfolio

Practice Highlights

• Students and mentors appreciated the introduction of data-informed mentoring.
• Assessment policies and practices should be integrated with data-informed mentoring.
• Data-informed mentoring presents curriculum challenges in task design and framing feedback.
• The student context informs the data-informed mentoring approach (learner-centred to mentor-directed).
• Data-informed mentoring requires supportive information and communications technologies.

I. INTRODUCTION

An often-cited definition of mentoring, highlights the role of experienced and empathetic others guiding students to re-examine their ideas, learning and personal and professional development (Standing Committee on Postgraduate Medical and Dental Education, 1998).

Heeneman and de Grave (2017) identify some subtle differences between traditional conceptions of mentoring and the type of mentoring that is required under programmatic assessment, which in this paper we refer to as Data-Informed Mentoring (D-IM). For example, D-IM is embedded in a curriculum in which rich data on student progress arises from student interaction with assessment tasks, informing and enhancing their progress (see Appendix). Further, in programmatic assessment, the learning portfolio is typically the setting in which the mentor-mentee relationship develops. This setting brings together institutional imperatives (e.g. assessable tasks), and personal imperatives such as evidence of competence and personal reflection. Situating mentoring in a curriculum and assessment framework impacts upon the mentoring relationship.

Meeuwissen, Stalmeijer, and Govaerts (2019) propose that a different type of mentoring is required under programmatic assessment. Mentors interpret data and feedback provided by content experts across domains of learning thus providing an evidence-base to facilitate student reflection. They might also take on a variety of roles (e.g. critical friend, coach, assessor) that could influence the mentoring relationship including the level of trust that is established with the student. These challenges suggest that conventional definitions of mentoring might not capture the essence of D-IM. Whilst the availability of rich information potentially enhances the mentoring experience and personalises learning, mentors and students are challenged to make sense and act upon this information; students might focus on issues that fall outside of the scope of the data provided (e.g. their wellbeing); mentors may also struggle to delineate boundaries between multiple roles or draw a line on where their scope of practice, as a mentor, begins and ends.

Mentoring is a social construct and as such is best considered through a holistic lens taking account of societal, institutional and personal factors (Sambunjak, 2015). The current study adopted Sambunjak’s “ecological model” (2015, p. 48) as a framework to help understand the impact of D-IM (Figure 1). Societal, institutional and personal forces are inter-related. For example, a student’s approach to D-IM might be influenced by financial circumstances resulting in the need to work part-time (societal); a medical school’s assessment policy (institutional); or a student’s learning style (personal). The model is presented as a set of cogs where the optimal educational experience is achieved if all elements work in harmony. The study uses the ecological model to help answer the central research question that guided the study: how did students and staff react and respond to D-IM?

Figure 1. An ecological framework for conceptualizing D-IM (modified from Sambunjak, 2015)

Note. *In the median test, a comparison is made between the median rating in each group to the ‘overall pooled median’ from both groups. **Values are less than or equal to the overall pooled median therefore Median Test could not be performed. ***Significant at p < 0.05 level.

Table 1. Student Perceptions of D-IM within Cohort 1 in 2017 and 2018–Median Tests for Individual Items

The only statistically significant difference noted for Cohort 1 was for the summaries of performance provided in Blackboard that were designed to underpin D-IM. The data provided in these summaries was less valued by students who engaged with D-IM in their second year.

The aggregate mean score in response to the statements on D-IM in the survey was positive in 2017 (M=4.02; SD=0.62; n=32). Mentoring continued to be well perceived by Cohort 1 as they progressed to second year in 2018 (M=3.80; SD=0.67; n=51). The slight difference in aggregate mean scores between 2017 and 2018 is not statistically significant (t=1.571; df=82, p=0.120). Table 2 compares first year students’ perceptions of D-IM.

Item	Overall Pooled Median*	Cohort 1 n=32		Cohort 2  n=47
		n> pooled median	<= pooled median	>  pooled median	<= pooled median	Median Test (chi square (χ2); df; p value)
The mentoring process was well organised	4	6	26	9	37	χ2 =0.008; df=1; p=0.928
My mentor was personally very well organised	5	0	32	0	47	n/a**
There were an appropriate number of mentoring meetings throughout the year	4	2	30	8	39	χ2 =0.998; df=1; p=0.158
My mentor was respectful	5	0	32	0	47	n/a**
My mentor listened to me	5	0	32	0	47	n/a**
My mentor asked thought-provoking questions which helped me to reflect	4	10	22	18	29	χ2 =0.413; df=1; p=0.520
My mentor added value to my learning	4	10	22	17	30	χ2 =0.205; df=1; p=0.651
My mentor helped me to set future goals that were achievable	4	9	23	17	30	χ2 =0.558; df=1; p=0.455
The summaries provided of my performance in the Blackboard Community Site were useful in helping me to reflect on my progress	3	17	16	17	30	χ2 =1.868; df=1; p=0.172

Note. *In the median test, a comparison is made between the median rating in each group to the ‘overall pooled median’ from both groups. **Values less than or equal to the overall pooled median therefore Median Test could not be performed.

Table 2. First Year Student Perceptions of D-IM –Median Tests between Cohort 1 and Cohort 2 for Individual Items

No statistically significant differences were noted between cohorts 1 and 2.

The aggregate mean score in response to the statements on D-IM in the survey was positive for Cohort 1 in 2017 (M=4.02; SD=0.62; n=32). Equally positive responses were noted in Cohort 2 in 2018 (M=3.91; SD=0.79; n=47). The difference between aggregate mean scores for first year students’ perceptions is not statistically significant (t=0.686; df=78, p=0.495).

Data from tables 1 and 2 reveals that students are highly satisfied with three aspects of mentoring: the personal organisation of the mentor along with their respectful and listening attributes. Students were also satisfied with the mentoring process, the number of mentoring meetings, the ability of the mentoring to assist in reflection and to add value to their learning, and also the propensity of the mentor to assist in action-planning. However, the summaries provided in the Blackboard environment were a source of dissatisfaction for students.

As discussed, qualitative data were collected from students through the questionnaire and staff through focus groups. The research team collated the qualitative data and confirmed that the qualitative data corroborated quantitative results with students and mentors appreciating the introduction of D-IM. For example, “Mentor sessions are important in providing support to students and…are a welcome introduction” (Yr1 Student, 2017); “Mentoring was useful to develop self-directed learning and to check where you were” (Yr2 Student, 2018); “You get to know the students, things were revealed which would not have been otherwise” (Mentor, 2017); and “Mentoring enabled me to facilitate more, listen more. Definitely a difference when you’re one-on-one with somebody” (Mentor, 2018).

In tune with the action research method adopted by the study which seeks to identify and respond to opportunities for improvement, the Research Team identified three concerns from the qualitative data: differing views of the purpose of D-IM and the role of the mentor; the provision of student feedback and information and communications technologies (ICT); and workload.

A. Differing Views of the Purpose of D-IM and the Role of Mentor

Mentors had differing conceptions of the purpose of D-IM and the role of a mentor. Some mentors perceived their primary function to be one of facilitating reflection and being encouraging whilst others were more directive, providing advice or sharing their own experiences. “I was…a sounding board to prompt their thoughts about how their progress was going. Rather than offering ways of solving problems it was more pointing where problems might lie and encouraging them to think of solutions” (Mentor, 2017); “The basic rule is to guide them… guide them properly, maybe to get them to change their study strategies and other things” (Mentor, 2017).

 B. Provision of Student Feedback and ICT

Students reported that feedback was inconsistent in timeliness and quality. Often feedback lacked guidance for improvement or was too late for it to help the student improve their learning: “More in-depth feedback on work, and returned in a timeframe that allows it to be relevant to our learning” (Yr1 Student, 2018); “Marking seemed thoughtless and halfhearted” (Yr2 Student, 2018).The use of a Blackboard Wiki to collate and present data points was also less than ideal with students finding the site difficult to navigate and use although they generally reported that it was safe and secure.

C. Workload

Students understood the role of reflection and appreciated having a mentor although there was some misunderstanding of the role of the portfolio with some students seeing it as extra work: “The amount of work required…was disproportionate” (Yr2 Student, 2018). Some students felt that the added stress and anxiety detracted from their study of medicine: “The portfolio actually detracts from spending time learning content that is essential to clinical years” (Yr2 Student, 2018). These concerns needed to be addressed by the School and are discussed in the context of changes that have and will be made to D-IM for preclinical students in the School.

IV. DISCUSSION

On the whole there was a positive response to D-IM implementation by students and staff. This response is consistent with Frei, Stamm, and Buddeberg-Fischer (2010, p. 1) who found that the “personal student-faculty relationship is important in that it helps students to feel that they are benefiting from individual advice.”

The findings of the research, however, reveal some tensions between the various elements of Sambunjak’s (2015) ecological model that link to the three areas of concern identified in the research. These tensions are shown in Figure 2.

Figure 2. The ecological framework to explore tensions in D-IM

A. Purpose and Role of Mentors and D-IM

The role of the mentor at the School is to support and guide students, and this role was not confused with other functions such as content expert or assessor. In this respect, the role conflict described by Meeuwissen et al. (2019) and Heeneman and deGrave (2017) was not evident at the School. However, mentoring approaches were situated on a continuum between learner-centred and mentor-directed. It is probable that the mentor’s style–empowering, checking or directing (Meeuwissen et al., 2019, p.605)–and their potentially different view of their role impacted on how D-IM sessions played out. Three ways of understanding the role of mentor in medical education have been identified: someone who can answer questions and give advice, someone who shares what it means to be a doctor and someone who listens and stimulates reflection (Stenfors-Hayes, Hult, & Owe Dahlgren, 2011). In a study of mentoring styles of beginning teachers, Richter et al. (2013) found that the mentor’s beliefs about learning have the greatest impact on the quality of the mentoring experience. Although professional development was provided to mentors on their role as facilitators of reflection and these issues were outlined and discussed, there were differences in interpretation of the role in the D-IM context.

Heeneman and de Grave (2017) argue that students need to be self-directed in order to be effective medical professionals. It is posited that a number of factors can influence the extent to which the mentor directs proceedings including the mentor’s experience, role clarity, rate of student progress, depth of student reflections and the perceived importance of the data required for assessment purposes.

In this study most students engaged positively with D-IM, though, albeit with variations in the extent and quality of reflection and action planning. A slight decrease in students’ enthusiasm towards D-IM was noted as they progressed from first to second year. This could be related to the novelty of D-IM diminishing over time that has been evident in other educational technology innovations (Kuykendall, Janvier, Kempton, & Brown, 2012). However, students also have a different mentor in each year. According to Sambunjak (2015), mentoring requires commitment sustained over a long period of time. At Maastricht University, for example, Heeneman and de Grave (2017) report that students are allocated the same mentor for a four-year medical course. It is, therefore, likely that in the current study the short timeframe for mentors to establish student relationships, and the introduction of a different mentor each year contributed to a reduction in student satisfaction.

B. Feedback and ICT Support

D-IM is dependent on quality data. That is, the perceived value of tasks that students engage with, and the feedback that they receive on, these tasks. Findings suggest that students found some tasks repetitive and feedback belated and superficial. Better task design and feedback practices are required. This finding is consistent with those of Bate, Macnish and Skinner (2016) in a study of task design within a learning portfolio. Findings also indicated dissatisfaction with Blackboard ICT environment. The portal was not intuitive and the structure and requirements for use of the template did not stimulate the desired level of reflection.

 C. Workload

Students at the School are “time poor” and many work part-time whilst studying. They are graduate entrants used to achieving academic success. Most are millennials comfortable with distilling and manipulating data and using online technologies. These characteristics are consistent with what Waljee, Copra, and Saint (2018) see as the new breed of medical students, being accustomed to distilling information and desirous of rapid career advancement. In these circumstances, it is unsurprising that students valued D-IM as it promoted focused data-driven discussion on their progress. However, it is also unsurprising that students were critical of anything that, in their opinion, did not support the “study of medicine”. Although students were sometimes critical of tasks that fed into D-IM (Bate et al., 2020), the reflective and action planning components of DI-M were not onerous and were at any rate optional.

For most students, grades rather than learning were paramount and this created a competitive environment which fuelled strategic learning in engaging with tasks underpinning D-IM. The School’s Assessment Policy has implications here. Progression is determined by passing discrete assessments and causes students to focus on grades rather than learning. These dispositions play out in D-IM sessions where, for example, goals are sometimes framed around passing examinations rather than addressing deficits in understanding. The School also distinguishes between formative and summative assessment with the result being that formative assessments are less valued by students. Opportunities to test understanding through formative testing are sometimes not taken up and result in less information for students and their mentor to gauge learning progress.

Bhat, Burm, Mohan, Chahine, and Goldszmidt (2018, p. 620) identified a set of “threshold concepts” in medicine that are crucial for students transitioning into clinical practice. Among these are self-directed, metacognitive and collaborative dispositions to learning. However, for a student in the preclinical years, these threshold concepts are not perceived to be the important factors that determine their progress through the course and their aim to become a doctor. Thus the tensions between students valuing mentoring but feeling that reflecting on their performance through D-IM is time-consuming and unrelated to their course progression is a source of tension within the model.

D. Actions as a Result of the Study

The action research approach of this study meant that in all results the Research Team was looking for ways to improve the system. Some issues could be improved quickly. A refinement of the Blackboard environment and a change to a software solution called SONIA was implemented in 2019 to improve the ICT interface and reduce workload. Continuing professional development (PD) for staff is undertaken and takes the research results into account. Within the mentor PD program, the Research Team saw that mentoring requires mentors to be able to diagnose the readiness and willingness of students to consider their learning educational journey. This means that, whilst the D-IM program needs a consistent view of D-IM where mentors see their role as facilitating reflection, different mentoring skills and behaviours are needed by mentors for different students. PD is also needed for students so that they understand the relationship between their achievement of learning and the role of D-IM in their journey.

Some issues are longer-term or resource dependent. A focus on the role of feedback in the system, especially for student reflection and its timeliness for mentoring sessions and action planning is critical to making D-IM valued by students. However, it is not always possible for staff to provide feedback in an optimum timeframe although the quality of the feedback can be improved by clear guidelines, expectations and an intuitive online interface.

Of great complexity and more difficult to resolve is the tension between developing the “threshold concepts” (Bhat et al., 2018); the generic skills which are built on self-reflection and are supported by D-IM and the ways in which a student progresses through the course. These are School-based rules of progression and produce a framework within which D-IM needs to operate.

V. LIMITATIONS OF THE STUDY

The study was conducted at one University and although it will ultimately cover a six-year timeframe, findings should be gauged within the context of this setting. Relatively low response rates were noted, and selection bias is a possibility with students most engaged with D-IM completing the questionnaire. Although professional development was provided to underpin the mentoring role, there was variation in the way tutors interpreted this role. The study was conducted at a time where other changes were occurring at the School (e.g. development of more continuous forms of assessment) and these changes might have impacted on D-IM. The questionnaire used in the study contained nine questions on mentoring. To gain a more nuanced understanding of D-IM at the School, it may be useful to use a comprehensive and validated questionnaire (e.g. Heeneman & de Grave, 2019) capturing the perceptions of mentees and mentors.

VI. CONCLUSION

The School aims to create quality, patient-centred and compassionate doctors who are lifelong learners (Candy, 2006). D-IM is an effective augmentation to the School’s educational program and the paper has demonstrated that it was well received by students and staff. Future directions include consideration of D-IM in clinical mentoring, development of more consistent learner-centred approaches to mentoring; improved task design and feedback; support for D-IM with appropriate ICT; and better integration of D-IM with assessment policies and practices.

Notes on Contributors

Associate Professor Frank Bate completed his PhD at Murdoch University. He is the Director of Medical and Health Professional Education at the School of Medicine Fremantle, University of Notre Dame Australia. He conceptualised and led the research, and was the primary author responsible for developing, reviewing and improving the manuscript.

Professor Sue Fyfe attained her PhD at the University of Western Australia and is an adjunct professor at Curtin University. She assisted in conceptualising the research design, conducted the qualitative data analysis and made a significant contribution to reviewing and improving the manuscript.

Dr Dylan Griffiths has a PhD from the University of Essex and is the Quality Assurance Manager at the School of Medicine Fremantle, University of Notre Dame Australia. He conducted data collection and assisted with preliminary analysis.

Associate Professor Kylie Russell obtained her PhD from the University of Notre Dame Australia. She is currently a Project Officer at the School of Medicine Fremantle, University of Notre Dame Australia. She assisted in the development of the research methodology and made a contribution to reviewing and improving the manuscript.

Associate Professor Chris Skinner completed his PhD at the University of Western Australia. He is Domain Chair of Personal and Professional development at the School of Medicine Fremantle, University of Notre Dame Australia. He assisted in conceptualising the research design and made a contribution to reviewing and improving the manuscript.

Associate Professor Elina Tor completed her PhD at Murdoch University and is the Associate Professor of Psychometrics at the School of Medicine Fremantle, University of Notre Dame Australia. She helped conceptualise the research design, led the quantitative data analysis, and made a significant contribution to reviewing and improving the manuscript.

Ethical Approval

The University of Notre Dame Australia Human Research Ethics Committee (HREC) has provided ethical approval for the research (Approval Number 017066F).

Acknowledgement

The authors acknowledge the thoughtful and insightful feedback provided by staff and students.

Funding

No internal or external funding was sought to conduct this research.

Declaration of Interest

There is no conflict of interest to declare.

References

Bate, F., Fyfe, S., Griffiths, D., Russell, K., Skinner, C., & Tor, E. (2020). Does an incremental approach to implementing programmatic assessment work: Reflections on the change process. MedEdPublish, 9(1), 55. https://doi.org/10.15694/mep.2020.000055.1

Bate, F., Macnish, J., & Skinner, C. (2016). The cart before the horse? Exploring the potential of ePortfolios in a Western Australian medical school. International Journal of ePortfolio, 6 (2), 85-94.

Bhat, C., Burm, S., Mohan, T., Chahine, S., & Goldszmidt, M. (2018). What trainees grapple with: A study of threshold concepts on the medicine ward. Medical Education, 52(6), 620–631. https://doi.org/10.1111/medu.13526

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101. https://doi.org/10.1191/1478088706qp063oa

Candy, P. (2006). Promoting lifelong learning: Academic developers and the university as a learning organization. International Journal for Academic Development, 1(1), 7-18. https://doi.org/10.1080/1360144960010102

Frei, E., Stamm, M., & Buddeberg-Fischer, B. (2010). Mentoring programs for medical students: A review of PubMed literature 2000-2008, BMC Medical Education, 10(32), 1-14. https://doi.org/10.1186/1472-6920-10-32

Gillis, A., & Jackson, W. (2002). Research for Nurses: Methods and Interpretation. Philadelphia, PA: F.A. Davis Co.

Heeneman, S., & de Grave, W. (2017). Tensions in mentoring medical students toward self-directed and reflective learning in a longitudinal portfolio-based mentoring system – An activity theory analysis. Medical Teacher, 39(4), 368-376. https://doi.org/10.1080/0142159X.2017.1286308

Heeneman, S., & de Grave, W. (2019). Development and initial validation of a dual-purpose questionnaire capturing mentors’ and mentees’ perceptions and expectations of the mentoring process. BMC Medical Education, 19(133), 1-13. https://doi.org/10.1186/s12909-019-1574-2

Kemmis, S., & McTaggart, R. (2000). Participatory action research. In N. K. Denzin & Y. S. Lincoln (Eds.) Handbook of Qualitative Research (2^nd Ed.; pp 567-606). New York: Sage Publications.

Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating Training Programs: The Four Levels (3rd ed.). Oakland, CA: Berrett-Koehler Publishers, Inc.

Kuykendall, B., Janvier, M., Kempton, I., & Brown, D. (2012). Interactive whiteboard technology: Promise and reality. In T. Bastiaens & G. Marks (Eds.), Proceedings of E-Learn 2012 – World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 1 (pp. 685-690). Retrieved from Association for the Advancement of Computing in Education (AACE), https://www.learntechlib.org/p/41669

Meeuwissen, N., Stalmeijer, R., & Govaerts, M. (2019). Multiple-role mentoring: Mentors conceptualisations, enactments and role conflicts. Medical Education, 53, 605-615. https://doi.org/10.1111/medu.13811

Richter, D., Kunter, M., Lüdtke, O., Klusmann, U., Anders, Y., & Baumert, J. (2013). How different mentoring approaches affect beginning teachers’ development in the first years of practice. Teaching and Teacher Education, 36, 166-177. https://doi.org/10.1016/j.tate.2013.07.012

Sambunjak, D. (2015). Understanding the wider environmental influences on mentoring: Towards an ecological model of mentoring in academic medicine. Acta Medica Academia, 44(1), 47-57. https://doi.org/10.5644/ama2006-124.126

Standing Committee on Postgraduate Medical and Dental Education. (1998). Supporting Doctors and Dentists at Work: An Enquiry into Mentoring. London: SCOPME.

Stenfors-Hayes, T., Hult, H., & Owe Dahlgren, L. (2011). What does it mean to be a mentor in medical education? Medical Teacher, 33(8), 423-428. https://doi.org/10.3109/0142159X.2011.586746 

Waljee, J. F., Copra, V., & Saint, S. (2018). Mentoring Millennials. JAMA, 319(15), 1547-1548. https://doi.org/10.1001/jama.2018.3804

*Frank Bate
Medical and Health Professional Education,
School of Medicine Fremantle,
University of Notre Dame Australia,
PO Box 1225, Fremantle,
Western Australia 6959
Telephone: +66 9433 0944
Email address: frank.bate@nd.edu.au

Submitted: 20 March 2020
Accepted: 28 July 2020
Published online: 5 January, TAPS 2021, 6(1), 70-82
https://doi.org/10.29060/TAPS.2021-6-1/OA2241

Sandra Widaty¹, Hardyanto Soebono², Sunarto³ & Ova Emilia⁴

¹Department of Dermatology and Venereology, Faculty of Medicine, Universitas Indonesia – Dr. Cipto Mangunkusumo Hospital, Indonesia; ²Department of Dermatology and Venereology Faculty of Medicine, Universitas Gadjah Mada, Indonesia; ³Pediatric Department, Faculty of Medicine, Universitas Gadjah Mada Indonesia, Indonesia; ⁴Medical Education Department, Faculty of Medicine, Universitas Gadjah Mada, Indonesia

Abstract

Introduction: Performance assessment of residents should be achieved with evaluation procedures, informed by measured and current educational standards. The present study aimed to develop, test, and evaluate a psychometric instrument for evaluating clinical practice performance among Dermatology and Venereology (DV) residents.

Methods: This is a qualitative and quantitative study conducted from 2014 to 2016. A pilot instrument was developed by 10 expert examiners from five universities to rate four video-recorded clinical performance, previously evaluated as good and bad performance. The next step was the application of the instrument to evaluate the residents which was carried out by the faculty of DV at two Universities.

Results: The instrument comprised 11 components. There was a statistically significant difference (p < 0.001) between good and bad performance. Cronbach’s alpha documented high overall reliability (a = 0.96) and good internal consistency (a = 0.90) for each component. The new instrument correctly evaluated 95.0% of poor performance. The implementation study showed that inter-rater reliability between evaluators range from low to high (correlation coefficient r =0.79, p < 0.001).

Conclusion: The instrument is a reliable and valid instrument for assessing clinical practice performance of DV residents. More studies are required to evaluate the instrument in different situation.

Keywords:            Instrument, Clinical Assessment, Performance, Resident, Dermatology-Venereology, Workplace-Based Assessment

Practice Highlights

• The residents’ performance will reflect on their professionalism and competencies. Furthermore, clinical care provided by Dermatology and Venereology field is unique, therefore a standard instrument is needed to assess their performance.
• Dermatology – Venereology Clinical Practice Performance Examination instrument is proven to be reliable and valid in assessing residents’ clinical performance

I. INTRODUCTION

Performance assessment in medical clinical practice has been a great concern for medical education programmes worldwide. (Holmboe, 2014; Khan & Ramachandran, 2012; Naidoo, Lopes, Patterson, Mead, & MacLeod, 2017). It is an accepted premise that performance may differ according to competency (Cate, 2014; Khan & Ramachandran, 2012). Performance also occurs within a domain; therefore, the assessment of performance should be separated from that of competency. Performance assessment of medical residents should also be informed by existing medical standards and performance criteria (Li, Ding, Zhang, Liu, & Wen, 2017; Naidoo et al., 2017).

Assessment of residents during their training programme is an important issue in postgraduate medical education, which has declared formative evaluation and constructive feedback as priorities (World Federation for Medical Education, 2015). An earmark of postgraduate medical specialist training is that it occurs in the workplace; therefore, the most appropriate measurement tools are Workplace-Based Assessments (WPBA). In medical education, these assessments emphasise on result and professionalism (Boursicot et al., 2011; Joshi, Singh, & Badyal, 2017).

In response to a standardisation programme for postgraduate medical specialist training (PMST), the World Federation for Medical Education (WFME) had published guidelines which adopted by several countries including Indonesia (Indonesian College of Dermatology and Venereology, 2008; World Federation for Medical Education, 2015). Clinical care provided in the Dermatology and Venereology (DV) field is unique; a brief examination of the patient is often useful before taking a lengthy history (Garg, Levin, & Bernhard, 2012). Privacy is a top priority, especially for venereology patients, patients with communicable diseases, cosmetic dermatology and skin surgery care.

Until now, no standard instrument has been available for performance assessment of PMST in DV; therefore, a variety of assessments are in use which may cause discrepancies (Jhorar, Waldman, Bordelon, & Whitaker-Worth, 2017). A valid and reliable method of assessment is required that can be used in various facilities and related to proficiency in both content and process (Kurtz, Silverman, Benson, & Drapper, 2003). Therefore, a study was conducted to focus on the development of a residents’ clinical performance assessment based on certain standards and principles such as the WPBA and WFME standards.

II. METHODS

A. Instrument Development

The instrument was developed and tested using qualitative and quantitative study designs. It started with a solicitation of inputs regarding expected performance from a variety of stakeholders in DV: patients, nurses, laboratory staff, newly graduated DV specialists, DV practitioners, and faculty. A literature review was performed, which included various documents such as the educational programme standards for DV residents, and documentation on available assessment tools (Cate, 2014; Hejri et al., 2017; Norcini, 2010). The instrument was developed according to the current standards (Campbell, Lockyer, Laidlaw, & MacLeod, 2007; McKinley, Fraser, van der Vleuten, & Hastings, 2000).

The resulted 11-items instrument was subsequently evaluated by faculty groups from various universities in Indonesia. Repeated revisions were carried out. Psychometric data for the instrument were provided through independent evaluations of performance videos of the residents and also through comparison of the results of the new instrument (Dermatology -Venereology Clinical Practice Performance Examination/DVP-Ex) and the compared instrument. The design was a validation study in which psychometric data for the instrument were provided. Further step was the assessment of residents’ performances when performing their clinical practice using the instrument to evaluate instrument reliability and feedback. Flowchart of the study process is shown in Appendix A.

B. Setting

The present study was conducted at the Department of Dermatology and Venereology, Dr. Cipto Mangunkusumo Hospital, a teaching hospital for Faculty of Medicine Universitas Indonesia, from 2014 to 2016. The study was conducted in four steps. When developing the instrument (Step 1), we included faculty members from five medical faculties in Indonesia that have DV Residency Programme (Universitas Indonesia, Universitas Sriwijaya (UNSRI), Universitas Padjajaran (UNPAD), Universitas Gadjah Mada, and Universitas Sam Ratulangi) through in depth interview and expert panel. The study received ethical approval from the Research Ethics Committee of the Faculty of Medicine Universitas Gadjah Mada Number KE/FK/238/EC.

The instrument that had been developed was sent to five senior faculty members from three universities (Department of Dermatology and Venereology, Universitas Indonesia, UNPAD and UNSRI) (Step 2). They were asked to give their assessment in order to have face and content validity. As a test of criterion validity, we recruited 10 faculty members of Faculty of Medicine, Universitas Indonesia, randomised them into two groups. Randomisation was performed to prevent bias against the instruments being tested. One group used DVP-Ex and the other used the current instrument. The single inclusion criterion was more than three years of teaching experience. After receiving some inputs, final correction was done and training was provided for faculty member who would use the instrument.

C. Performance Video

To obtain standardised performance of the residents, video recordings of the resident’s clinical practice were made. Two residents were voluntarily recruited and a special team recorded their clinical practice performance using scenarios created by the first author. (Campbell et al., 2007; McKinley et al., 2000)

There were four videos, each of which showed the clinical practice performance of the residents when they were presented with a difficult case (dermatomyositis) and a common case (leprosy tuberculoid borderline type). Patients had to sign informed consent before included in this study. A good (first and fourth video clips) and poor (second and third clips) standard of performance were demonstrated. Activities presented in the scenarios were those associated with patient care (Campbell et al., 2007; Iobst et al., 2010). After the recording session was finished, patients were managed accordingly and provided with rewards.

D. Training on the Performance Instrument

An hour-long training was provided for the 10 faculty members (the examiners). The faculty then practiced scoring, using the recorded video clips. During the training, we received some input and made necessary corrections to the rubrics. There was no training given for the comparison instrument because the entire faculty was already accustomed to this instrument. Step 3 is the step to produce validity, reliability and accuracy of performance instrument, which was conducted through a comparative study between two instruments of assessment; i.e. performance and control instruments. It evaluated the clinical practice performance of residents in the form of video film recording their performance

 E. Implementation of Resident Performance Assessment with Performance Instrument

This step was aimed to evaluate the reliability of the instrument and results of instrument implementation when it was used to assess the residents (Step 4). The sample included residents of Postgraduate Medical Specialist Training Programme in Dermatology and Venereology, Faculty of Medicine, University of Indonesia and UGM, who were at their basic level (residents who were on their 1st semester in clinical setting), intern level (semester II-V) and independent level (semester VI or higher). 

Sample size:  n = 3 – 4/level/Faculty of Medicine = 20. The evaluators were five lecturers/ Faculty of Medicine = 10, and each lecturer evaluated six residents.

F. Data Collection

One week after the training, the instrument was evaluated. Faculty members assessed the performance of the residents in the four video recordings at the same time. Three days later, the groups underwent a rotation to reassess the video with whichever of the two instruments they had not already used. The examiners were asked to provide feedback and information on the ease of completing the instrument and the clarity of its instructions.  For the implementation of resident assessment performance with the instrument, one resident was being evaluated by three lecturers simultaneously. The lecturers were grouped randomly; therefore, every lecturer could evaluate six residents out of ten residents from each group that would be assessed.

G. Data Analysis

The analyses aimed to evaluate validity, reliability, and precision of the instrument for discriminating the performance of the residents as poor, good, or excellent.

H. Validity and Reliability

A reliability test was performed, i.e. internal consistency in the form of responses against items in each field (Cronbach alpha coefficient). Face and content validity were assessed by addressing the relevant performance standards and criteria, and by optimizing clarity of instruction, specific criteria, acceptable format, gradation of responses, correct and comprehensive answers (including all assessed variables). The cut-off score of the instrument was determined using ROC (receiver operating curve) principles, which was then used to evaluate sensitivity, specificity, positive and negative predictive value. The accuracy of the instrument was determined to evaluate the precision of the instrument in distinguishing between good and poor performance.

I. Statistical Analysis

The statistical analysis was performed using SPSS 11.5 software. Total assessment scores of each examiner were analysed using analysis of variance (ANOVA). Internal consistency was determined using Cronbach’s α and Spearman analysis was performed to acquire p value for the validity. The accuracy was determined by comparing failed or passed score results and comparing it with the video. To obtain the intergroup difference, McNemar’s test and Kappa analysis were carried out. Qualitative analysis was also performed, especially to evaluate feedbacks by performing several analytical steps.

III. RESULTS

A performance instrument was developed with 11 competency components, for which evaluation responses were given in the form of rubric scale (Appendix B). All 10 faculty members completed an assessment of each of the four videos. Eight examiners had more than 3 years of teaching experiences, and five examiners were DV consultants.

A. Validity

For validity, face, content, and construct validity remain solid points of reference for validity evaluation (Colliver, Conlee, & Verhulst, 2012; Johnson & Christensen, 2008). Face content was evaluated by five experts from three universities. The evaluation was implemented to improve the instrument. The scale of the rubrics described the capacity of residents to perform activities according to the Standard Competency of DV specialist and the domain of performance for physicians has made the instrument evaluated as the instrument with good validity on its face, content and construction.

The results of the assessments made on performance videos with the DVP-Ex showed that examiners agreed that the performances of the first and fourth videos (the “good” videos) were good performance (>60); conversely, the second and third videos (the “bad” ones) were evaluated as poor performance by 10 and 9 out of 10 faculty members, respectively (Table 1).

Note: Chi Square, Kruskal–Wallis p < 0.001

Table 1. Assessment scores for each of the four videos (n = 10)

Faculty members also gave feedback suggesting that the instrument would be useful for assessing residents’ performance. They also commented that the instrument was more objective than the one currently in use, that it was challenging in that they had to read the instrument carefully in order to use it properly, and that the response options allowed several aspects of the residents’ performance to be assessed.

B. Validity and Reliability

Validity of the instrument was measure using Spearman analyses showed significant result for all of the competency component (p > 0.001). Reliability measure of the correlation between each item score and the total score on all relevant items (Cohen, Manion, & Morrison, 2008). Our analysis revealed good overall reliability, with Cronbach α = 0.96. All components of competency achieved internal reliability scores >0.95. The correlation between each item score on the competency components and the overall score was excellent (range: 0.64–0.99).

Submitted: 16 April 2020
Accepted: 24 June 2020
Published online: 5 January, TAPS 2021, 6(1), 83-92
https://doi.org/10.29060/TAPS.2021-6-1/OA2251

Eng Koon Ong 

Division of Supportive and Palliative Care, National Cancer Centre Singapore, Singapore; Assisi Hospice, Singapore

Abstract

Introduction: Physician empathy is declining due to an unproportionate focus on technical knowledge and skills. The medical humanities can counter this by allowing connection with our patients. This is a pilot study that aims to investigate the acceptability, efficacy, and feasibility of a humanities educational intervention to develop physician empathy.

Methods: Junior doctors at the Division of Supportive and Palliative Care at the National Cancer Centre Singapore between July 2018 and June 2019 attended two small-group sessions facilitated by psychologists to learn about empathy using literature and other arts-based materials. Feasibility was defined as a completion rate of at least 80% while acceptability was assessed by a 5-question Likert-scale questionnaire. Empathy was measured pre- and post-intervention using Jefferson’s Scale of Physician Empathy (JSPE) and the modified-CARE (Consultation and Relational Empathy) measure.

Results: Seventeen participants consented, and all completed the programme. Acceptability scores ranged from 18 to 50 out of 50 (mean 38, median 38). There was an increase in JSPE scores (pre-test mean 103.6, SD=11.0 and post-test mean 108.9, SD=9.9; t (17) =2.49, P=.02). The modified-CARE score increased between pre-test mean of 22.9(SD=5.8) and a post-test mean of 28.5(SD=5.9); t (17) = 5.22, P<0.001.

Conclusion: Results indicate that the programme was acceptable, effective, and feasible. The results are limited by the lack of longitudinal follow-up. Future studies that investigate the programme’s effect over time and qualitative analysis can better assess its efficacy and elicit the participants’ experiences for future implementation and refinement.

Keywords:            Empathy, Humanities, Literature, Palliative Medicine

Practice Highlights

• The medical humanities can be used to teach empathy by facilitating reflective practice.
• This novel educational programme was acceptable, effective, and feasible.
• Limitations include the lack of longitudinal follow-up and the quantitative nature of assessment.
• Future studies should investigate the programme’s effect over time and include qualitative analysis.

II. METHODS

A. Intervention Design

The Humanistic Aspirations as a Propeller of Palliative medicine Education (HAPPE) programme was conceived to introduce and develop a novel curriculum in empathy for junior doctors undergoing a palliative medicine rotation.  The overall goal of the study was to design an effective education programme based on the humanities to teach doctors empathy. Our study draws upon Schon’s work of Reflective Practice (see Table 2; Schön, 1987).

Submitted: 21 February 2020
Accepted: 13 July 2020
Published online: 5 January, TAPS 2021, 6(1), 93-108
https://doi.org/10.29060/TAPS.2021-6-1/OA2229

Kah Wei Tan¹, Hwee Kuan Ong² & Un Sam Mok³

¹Ministry of Health Holdings, Singapore; ²Department of Physiotherapy, Singapore General Hospital, Singapore; ³Division of Anaesthesiology and Peri-operative Medicine, Singapore General Hospital, Singapore

Abstract

Introduction: During resuscitations, healthcare professionals (HCPs) find balancing the need for timely resuscitation and adherence to infection prevention (IP) measures difficult. This study explored the effects of an innovative teaching method, using in-situ simulation and inter-professional education to enhance compliance to IP through better inter-professional collaboration. 

Methods: The study was conducted in the Surgical Intensive Care Unit (SICU) in a 1200-beds teaching hospital. HCPs working in the SICU were conveniently allocated to the intervention or control group based on their work roster. The intervention group attended an in-situ simulated scenario on managing cardiac arrest in an infectious patient. The control group completed the standard institution-wide infection control eLearning module. Outcomes measured were: (a) attitudes towards inter-professional teamwork [TeamSTEPPS Teamwork Attitudes Questionnaire (TAQ)], (b) infection prevention knowledge test, (c) self-evaluated confidence in dealing with infectious patients and (d) intensive care unit (ICU) audits on infection prevention compliance during actual resuscitations.

Results: 40 HCPs were recruited. 29 responded (71%) to the pre- and post-workshop questionnaires. There were no significant differences in the TeamSTEPPS TAQ and infection prevention knowledge score between the groups. However, ICU audits demonstrated a 60% improvement in IP compliance for endotracheal tube insertion and 50% improvement in parenteral medication administration. This may be attributed to the debriefing session where IP staff shared useful tips on compliance to IP measures during resuscitation and identified threats that could deter IP compliance in SICU.

Conclusion: Learning infection prevention through simulated inter-professional education (IPE) workshops may lead to increased IP compliance in clinical settings.

Keywords:            Inter-Professional Education, Simulation Infection Control, Resuscitation, Inter-Professional Teamwork

Practice Highlights

• Use of a simulated scenario to improve infection prevention during resuscitation.
• Improving attitudes towards inter-professional collaboration amongst healthcare professionals.
• Evaluating the efficacy of a simulated scenario through clinical audit.

I. INTRODUCTION

Adherence to infection prevention is paramount in the intensive care unit (ICU) as hospital acquired infections in the critically ill are associated with increased morbidity, mortality, length of stay and healthcare cost (Gandra & Ellison, 2014). However, during resuscitations, healthcare professionals (HCPs) may experience difficulty in balancing the need for resuscitation and adherence to infection prevention guidelines, resulting in suboptimal compliance to basic infection prevention measures (Steinemann et al., 2016). Moreover, resuscitation is a time-critical endeavour that requires good collaboration in a team comprising of different HCPs fulfilling different roles with different priorities, and lapses in teamwork may arise in a team comprising of HCPs with different roles and priorities (Barr, Koppel, Reeves, Hammick, & Freeth, 2009).

Inter-professional education (IPE) is defined as “occasions when two or more professions learn with, from and about each other to improve collaboration and quality of care” by the Centre for Advancement of Interprofessional Education (CAIPE) (Steinert, 2005). It is known to improve patient safety through improving communication, understanding and knowledge to encourage active participation from different HCPs (Oandasan, 2007; Wong, Lee, Allen, & Foong, 2020). Research has shown that active collaboration amongst HCPs in the workplace resulted in improved patient outcomes and provider satisfaction (Wagner, Parker, Mavis, & Smith, 2011). Previous studies had investigated the outcomes of using IPE workshops to teach infection prevention in non-emergency clinical settings or using standardised patients, and concluded that knowledge and confidence in infection prevention and inter-professional teamwork had improved (Mundell, Kennedy, Szostek, & Cook, 2013).

Currently, infection prevention education in our institute is didactic and web-based. Although this method is effective in disseminating information, there are no opportunities to learn with different HCPs or apply knowledge to real-life scenarios. On the basis that resuscitation is traditionally taught using simulation and has been proven to be highly effective (Perkins, 2007), we developed an IPE simulation workshop on infection prevention during resuscitation.

We hypothesised that the in-situ simulation workshop involving different HCPs will result in improved attitudes towards inter-professional teamwork and improved compliance to infection prevention guidelines, compared to our standard institutional infection control (IC) education.

 II. METHODS

A. Study Population

We conducted a non-randomised experimental study amongst HCPs working in the Surgical Intensive Care Unit (SICU) of the Singapore General Hospital (SGH). All HCPs working in the SICU were eligible to participate in the study. There were no exclusion criteria. Informed consent was obtained from all participating HCPs. Information on participants’ profession, the year they obtained their professional qualification, the number of years they had worked in critical care and prior experience in simulation training were collected. HCPs from Anaesthesiology, Nursing, Physiotherapy, Pharmacy, Speech and Language Therapy, Dietetics and Infection Prevention were involved.  A working day was picked to run the workshop and HCPs on duty that day were allocated to the intervention group, while those there were not on duty were assigned to the control group.

B. In-Situ Simulation Workshop

Participants in the intervention group (n=25) underwent a two-hour in-situ simulation workshop in the SICU on the scenario of a cardiac arrest in an infectious patient (Annex A). The training faculty comprised of HCPs from various professions such as Anaesthesiology, Nursing, Physiotherapy, Pharmacy, Speech and Language Therapy, Dietetics and Infection Prevention. Each workshop consisted of HCPs from five to seven different professions. The learning outcomes of the workshop were (i) to practice infection prevention precautions for transmission-based infections during resuscitations and (ii) to improve attitudes towards inter-professional teamwork during a crisis situation.

The workshop was designed based on Kolb’s experiential learning theory (Kolb & Fry, 1975), which is a four-stage learning cycle consisting of (i) concrete experience, (ii) reflective observation, (iii) abstract conceptualisation and (iv) active experimentation. Concrete experience was facilitated through in-situ simulation where participants experienced real-life constraints of resuscitating an infectious patient. Following the simulation, a debrief was held by the faculty that facilitated reflective observation and abstract conceptualisation. Learning points discussed include (i) issues faced in adhering to infection prevention guidelines in a resuscitation setting, (ii) inter-professional teamwork and (iii) threats that could deter IC compliance in SICU. The final stage, active experimentation, was facilitated through actual application of learning points surmised from the workshop, and evaluated through real-time ICU audits.

C. Evaluation of Outcomes

Evaluation of the effectiveness and impact of the workshop was done in accordance with Kirkpatrick’s evaluation framework (Kirkpatrick, 1994), which emphasised the need to go beyond the immediate reactions of participants by assessing them on four different levels, which are (i) Reaction, (ii) Learning, (iii) Behaviour and (iv) Results.  “Reaction” was evaluated through the post-workshop questionnaire and participants’ responses in the post-workshop debrief with regards to the effectiveness of simulated workshops in improving inter-professional teamwork and encouraging compliance to infection prevention guidelines. The second level, “Learning”, was evaluated by discussing learning points of the workshop in the debrief, and getting participants to note down their most important takeaways regarding infection prevention and IPE in the post-workshop questionnaire. “Behavioural changes” were assessed through a real-time observational study that assessed participants’ ability to observe proper infection prevention practices during actual resuscitations in the SICU. Results, the fourth level, were difficult to evaluate due to the small sample size that we recruited.

D. Outcomes of Study

There are two primary outcomes of this study: (i) attitudes towards inter-professional teamwork and (ii) infection prevention knowledge and practices.

Changes in attitudes towards inter-professional teamwork were assessed using the TeamSTEPPS Teamwork Attitudes Questionnaire (TAQ 1.0), based on scores in the following subcategories; Team Structure, Leadership, Situation Monitoring, Mutual Support and Communication (Appendix 1). Qualitative comments on key learning points with regards to teamwork were also collected during the debriefing sessions.

Changes in infection prevention knowledge and practice were assessed in five ways:

1. An infection prevention self-evaluation questionnaire on a 5-point Likert Scale (Appendix 2).

2. A multiple-choice quiz developed by the Institutional Infection Prevention Nurse Educators (Appendix 3A and 3B).

3. Questionnaire on effectiveness of the simulation (for participants in the intervention group only) (Appendix 4).

4. Qualitative feedback on the learning points concerning infection prevention. (Appendix 5).

5. Clinical audit data that evaluated compliance to infection prevention  guidelines during resuscitations in the SICU. Two months after the simulation workshop, the data was collected by a trained hospital-based infection prevention team based on real-time observations. The audit checklist assessed proper use of personal protective equipment (PPE), hand hygiene, administration of parenteral medications and insertion of endotracheal tube (ETT).

Statistical analysis was performed using SPSS for Mac, Version 20.0 (SPSS Inc., Chicago, IL, USA). Continuous variables were analysed using the t-test, and categorical variables were analysed using the Fisher and Chi square test. A p-value of less than 0.05 was taken to be statistically significant.

III. RESULTS

The study recruited a total of 40 HCPs. Of the 40 clinical staff working in the SICU asked to rank their responses, 29 (71%) responded to the pre- and post-workshop TAQ, self-evaluation of infection prevention knowledge and infection prevention knowledge quiz. Healthcare professions represented included doctors (31%), nurses (34%), pharmacists (10%), physiotherapists (10%), speech and language therapists (10%) and dieticians (3%).  

The intervention and control groups were not significantly different in terms of the number of years post-graduation, years of working experience in critical care and the number of simulated training sessions they have attended, excluding basic cardiac life support (BCLS) and advanced cardiac life support (ACLS). Similarly, there were no significant differences in self-evaluation of infection prevention knowledge and infection prevention scores at baseline. However, we noted significantly higher scores in the control group in Team Structure (mean difference=2.76), Leadership (mean difference =3.8), and Communication (mean difference=2.54; Table 1).

Submitted: 8 February 2020
Accepted: 9 April 2020
Published online: 1 September, TAPS 2020, 5(3), 71-82
https://doi.org/10.29060/TAPS.2020-5-3/OA2226

May Eng Loo¹, Brenda Wong², Yee Mun Lee³

¹Health Outcomes and Medical Education Research (HOMER), National Healthcare Group, Singapore; ²Group Education, National Healthcare Group, Singapore; ³Department of Urology, Tan Tock Seng Hospital, Singapore

Abstract

In order to understand what constitutes unprofessional online behaviour from the perspectives of stakeholders, there have been studies that examine the perceptions of doctors and the public on hypothetical online postings. However, the considerations and reasoning of the participants when they evaluate online posts have not been explored in-depth. This project aimed to examine the main considerations and possible conflicting considerations of faculty and residents when they evaluate the appropriateness of Facebook posts, and how they might negotiate any conflicts. Faculty and residents from the National Healthcare Group – Alexandra Health Pte Ltd Residency in Singapore were randomly presented Facebooks posts as part of an online questionnaire, rated their appropriateness, and provided explanations for their ratings. Responses were coded for main considerations and responses with 2 or more conflicting considerations were further analysed to describe the conflict and the way they were negotiated. 182 faculty and residents rated the appropriateness of three out of six Facebook posts and explained their evaluation. Except for one post which was evaluated as ‘Neither appropriate nor inappropriate’ by the majority (37%) of the respondents, all other posts were rated as either ‘Very inappropriate’ or ‘Inappropriate’ by the majority of respondents (34%-69%). Despite similar evaluation of inappropriateness, faculty and residents take into account a wide range of considerations. These considerations tend to conflict with one another when the respondents considered freedom of expression of the poster and the educational purpose of a post. Understanding physicians’ negotiation of conflicting considerations provides insight into their outworking of professionalism in social media context.

Keywords:           Professionalism, Social Media, Postgraduate Education

Practice Highlights

• Doctors within the same Residency do not necessarily have a uniform set of professional priorities regarding social media.
• They may also have to manage conflicting professional and personal values in different contexts.
• Educators of professional values could recognise the complexity of such conflicts and be sensitive to this in their teaching.
• We recommend creating platforms for doctors to have conversations on social media use.

I. INTRODUCTION

In June 2017, Mark Zuckerberg announced that the Facebook community was officially two billion people. While seeking to understand how social media can be maximised for good, Chief Product Officer Chris Cox was conscious of Facebook’s responsibility to ‘curtail any way that it can be misused or turned into something sad’ (Constine, 2017). In healthcare education and practice, social media has both the potential to be maximised for good (e.g. facilitating communication and improving knowledge) as well as the potential to be misused, compromising patient confidentiality and eroding public confidence in the medical profession (Greysen, Kind, & Chretien, 2010; Hamm et al., 2013). In their commentary, Chretien and Kind (2014) propose that in order to reap the benefits of social media use for healthcare, physicians need to be first aware of the risks of using social media and address concerns of unprofessional online behaviour.

In order to understand what constitutes unprofessional online behaviour from the perspectives of stakeholders, there have been studies that examine the perceptions of doctors and the public on hypothetical online postings (Chretien, Farnan, Greysen, & Kind, 2011; Dawkins, King, Boateng, Nichols, & Desselle, 2017; Greysen et al., 2013; Jain et al., 2014; Kesselheim, Batra, Belmonte, Boland, & McGregor, 2014; Kind, Greysen, & Chretien, 2012; Rocha & de Castro, 2014). These studies found that online behaviour that participants judged as unprofessional include patient privacy violations, patient content in general and negative comments about faculty and staff.  However, the considerations and reasoning of the participants when they evaluate online posts have not been explored in-depth, and especially for posts that may be inappropriate but may not appear obviously unprofessional.  Such ‘grey posts’ may include patient storytelling posts where, for example, a patient’s identity is not revealed but details are shared to encourage reflection or request support through social media (Wells, Lehavot, & Isaac, 2015).  Professional or personal values may sometimes come into conflict when they seem equally important in such situations. However, not much is known about what these conflicting considerations are when doctors evaluate such online posts and how these conflicting considerations are resolved or negotiated.

In response to concerns about unprofessional behaviour on social media, guidelines have been published by medical associations in various countries like the USA, UK, Canada, Australia, New Zealand as well as Singapore (American College of Physicians Ethics, Professionalism, and Human Rights Committee, 2012; Australian Medical Association, 2010; Canadian Medical Association, 2011; General Medical Council, 2013; Singapore Medical Council, 2016). These existing guidelines are generally comprehensive, providing principles for social media use and examples of unprofessional behaviour to avoid. Regarding posting online, concerns about patient confidentiality, professionalism, collegiality and preserving patients’ and the public trust in the medical profession are key issues raised by the guidelines. In particular, the Singapore Medical Council gives a list of inappropriate posts, examples include ‘speaking and writing in an indiscreet, bigoted, rude, and obscene or profane manner’ and ‘posting personal or derogatory comments about patients or colleagues’ (Singapore Medical Council, 2016).

Our study seeks to explore the considerations and possible competing or conflicting considerations of faculty and residents in the National Healthcare Group – Alexandra Health Pte Ltd (NHG-AHPL) Residency program when they evaluate online postings, especially grey posts. This study also aims to examine how the participants resolve or negotiate these conflicting considerations. Identifying the considerations of faculty and residents when they evaluate the posts sheds light on what constitutes unprofessional online posts and provides a platform for further discussion on existing social media use guidelines. Furthermore, understanding how doctors negotiate at times conflicting norms and obligations also provides insight into how they perceive the outworking of professionalism in the social media context.

II. METHODS

Six Facebook posts deemed inappropriate were identified from an online search. One author (L.M.E.) searched Google News on 30th June 2016 using a Boolean search: (social media OR Facebook OR twitter OR blog OR online OR internet OR web) AND (doctor OR physician OR medical student OR hospital OR clinic OR ward) AND (inappropriate OR unprofessional OR wrong OR unethical). The search was restricted to the first 20 pages of results (10 hits/page) and four posts (Posts 1, 2, 4 and 5) that appeared inappropriate but were not clearly unprofessional were selected. The remaining two posts (Post 3 and 6) were re-used from a pilot phase of the study. The characteristics of the posts are described in Table 1 and the screenshots of the six posts can be found in Appendix A.

Table 1. Characteristics of posts

As part of an online questionnaire administered between 8^th September and 30^th October 2016 on social media use, faculty and residents from the NHG-AHPL Residency program in Singapore were asked to rate the appropriateness of three of these posts and to explain their evaluation. All respondents provided informed consent at the start of the questionnaire. Considerations were identified using structural coding which is a question-based code that acts as a labelling and indexing device (Saldana, 2012). For example, the following response was coded ‘Purpose’, ‘Language’ and ‘Respect’ based on the question “What do respondents consider when they evaluate posts?”

Someone has just died. Instead of showing empathy to the family of the deceased who had just lost a loved one, she uses swear words in a callous, insensitive way. Even though her intentions might have been good to ask people to wear a crash helmet, the way she said it was offensive and highly inappropriate given the circumstances. (F5.3)

In the first round of coding, two authors (L.M.E, B.W.) coded each response independently and resolved coding discrepancies through discussions. Using a codebook developed from the first round of coding as a guide, each response was revisited and coded. Coding discrepancies were resolved through more extensive discussions. The second round of coding added new codes to the codebook. Responses with two or more conflicting considerations were further analysed to describe the conflict and the way they were negotiated.  Conflicting considerations were identified based on the plain reading of the response, prompted by the use of conjunctions such as ‘but’, ‘even though’ and ‘while’. Appendix B gives a full description of the codes used. Ethics approval was obtained from the institution’s Domain Specific Review Board. 

III. RESULTS

A total of 182 respondents (36.8%)–64 faculty and 118 residents–rated the appropriateness of three out of six Facebook posts. 463 responses–169 responses from faculty and 294 responses from residents–were analysed. Table 2 provides a summary of the results for each post.

Table 2. Summary of results for Posts 1-6

A. Evaluation and Considerations

Except for Post 4 which was evaluated as ‘Neither appropriate nor inappropriate’ by the majority (37%) of the respondents, all other posts were rated as either ‘Very inappropriate’ or ‘Inappropriate’ by the majority of respondents (34%-69%). In particular, Post 5 has a relatively high proportion of respondents (10%) rating it as either ‘Appropriate’ or ‘Very appropriate’. Respondents have a broad range of considerations with ‘Professionalism’, ‘Language’, ‘Behaviour’ and ‘Platform’ repeated across all six posts.

‘Platform’ was the main consideration for Post 2 and Post 6. Respondents considered the inappropriateness of posting on a public platform what should only be said or done in private, “FB is a public forum. People should settle personal grievances NOT online but privately” (R2.57; Post 2); “Publicly criticizing a colleague is inappropriate. Feedback should always be given in a private setting” (R6.33; Post 6).

For the other posts, the main considerations were ‘Consent’ (Post 1), ‘Professionalism’ (Post 3), ‘Freedom’ (Post 4) and ‘Confidentiality’ (Post 5). More than half of the respondents (55%) for Post 1 considered whether consent or permission was obtained before posting. For Post 3, more than half of the respondents (59%) considered professionalism generally or more specifically in terms of attitude. For Post 4, 42% of the respondents considered the freedom of speech of the poster including the freedom of the poster to post personal matters in a personal capacity, with personal accountability. Confidentiality and privacy of patients was considered by 24% of respondents on Post 5.

B. Negotiation of Conflicting Considerations

Among the six posts, Post 5 had the most number of respondents with conflicting considerations (n=15), followed by Post 4 (n=13).

For Post 5, fourteen respondents considered that the poster has an educational purpose versus other considerations and one respondent considered freedom of expression versus language.  To negotiate the conflicting considerations for Post 5, six respondents reasoned that the educational message of the post and the poster’s good intentions need to be subjected to (or at least seen in the light of) considerations of confidentiality, language, professionalism and respect. For example, two faculty members, F5.3 and F5.5, thought that although the poster had good intentions and the message was educational, her language and tone were inappropriate.

Someone has just died. Instead of showing empathy to the family of the deceased who had just lost a loved one, she uses swear words in a callous, insensitive way. Even though her intentions might have been good to ask people to wear a crash helmet, the way she said it was offensive and highly inappropriate given the circumstances (F5.3).

F5.5 also considered the possible identification of the victim, “although message was ‘correct’ strong language including expletives used which is inappropriate timing and location of post may allow identification of victim” (F5.5).

However, there were nine respondents for Post 5 who considered educational purpose and freedom of expression at the same level of importance as other considerations. Rating Post 5 as ‘Neither inappropriate nor appropriate’, F5.6 reasoned that the possible breach in Professionalism/Confidentiality may be justified by educational purpose, “I don’t think it’s appropriate to talk about patient experiences on Facebook but I believe she has done so with the intention of encouraging others to wear helmets” while R5.39 reasoned that the inappropriate language use may also be justified by the freedom of the poster to say what she wants as long as there were no patient identifiers, “While her choice of words may not be the best, there were no patient identifiers. Again, she can say what she wants”.

For Post 4 which had 13 respondents with conflicting considerations, 12 respondents considered that the poster should have the freedom to express his personal opinion versus a variety of other considerations, including that of ‘Professionalism’,’ Language’, ‘Audience’,’ Behaviour’, ‘Context’, ‘Platform’ and ‘Consequences’. One respondent considered norms on social media versus the harsh language used.

To negotiate the conflicting considerations for Post 4, six respondents prioritised professionalism and other considerations like audience and consequences over freedom of expression.  For example, a resident reasoned that while medical professionals can have their own political views, the poster should consider the possible consequences of such a post being seen by a wider audience and being used against him, “Everyone is entitled to his own view, medical professionals can have their own political views, but caution needs to be exercised if this information is used against him. might need to restrict to close friends only” (R4.43).

Seven respondents, however, considered freedom of expression and norms on social media at the same level of importance as other considerations. For example, respondents R4.51 and R4.60 rated Post 4 as ‘Neither inappropriate nor appropriate’. They reasoned that the harsh language used or the negative consequences of the post may be justified by the poster’s entitlement to freedom of expression, “He is entitled to his beliefs and freedom of speech about political and religious issues. Even though these comments are harsh and mean, we can’t possibly control how everyone speaks” (R 4.51); “Everyone is entitled to free speech and he has every right to say what he wants. But again this does not look good on his reputation” (R 4.60).

The number of respondents with conflicting considerations for the other posts ranged from 1 to 6. In general, respondents negotiated the conflicting considerations mainly by prioritising professionalism over and above the other considerations.

IV. DISCUSSION

A total of 182 faculty and residents (36.8%) from the NHG-AHPL Residency program in Singapore rated the appropriateness of Facebook posts and 463 evaluation responses were analysed. One of three postgraduate medical education programs in Singapore, the NHG-AHPL Residency comprises 27 residency programs within a few institutions including hospitals and nine polyclinics. Faculty and residents who explained their evaluation took into account a wide range of considerations and these considerations tended to conflict with one another when the respondents considered freedom of expression of the poster and the educational purpose of a post. These findings suggest that doctors within the same Residency do not necessarily have a uniform set of professional priorities and may have to manage conflicting professional and personal values in different contexts.  The teaching and evaluation of professionalism should thus increasingly take conflict and context into consideration (Ginsburg et al., 2000). Educators of professional values could recognise the complexity of such conflicts and be sensitive to this in their teaching, whether formally or informally.

Freedom of speech or expression appears to be a recurrent consideration for posts not just in the personal domain (Posts 2 and 4) but also those in the work-related domain as well (Posts 3 and 5). Given that freedom of expression is not commonly encouraged or emphasised in Asian cultures (Kim & Sherman, 2007), it may be somewhat surprising that this was a recurrent consideration for doctors in Singapore. This consideration thus deserves more attention from developers of social media use guidelines and online professionalism course facilitators who can seek to start addressing these considerations by first recognising and acknowledging them. While existing social media guidelines could and should be tempered by recognising doctors’ right to free speech, doctors and medical students would also need to exercise critical judgment to consider whether their freedom of expression is appropriate in view of the guidelines (Farnan et al., 2009).

This study found that the post that elicited the most number of conflicting considerations was Post 5. This post by a junior doctor contained insensitive language and patient-related content but had an educational purpose. It is unclear whether if the post had been written more sensitively, it would elicit even more conflicting considerations. This may be a valid cause for concern given the potential negative consequences of such posts (Wells et al., 2015). Although the victim was not named, it is still possible that enough details are given for him to be identified. Such a post may also violate family members’ expectations of privacy. Public trust could also possibly be undermined because of the insensitivity and lack of empathy displayed by a doctor. It may be helpful for faculty to engage residents or students with such issues informally or formally during relevant courses and to explore alternative avenues for patient storytelling.

Based on the findings of our study, we recommend creating platforms for conversations on social media use to take place among faculty and residents. Using existing social media use guidelines and relevant findings from studies as material for discussion, faculty and residents could consider various perspectives, discuss how guidelines may moderate some considerations and explore further considerations arising from the conversation. Discussions like these require doctors to exercise critical judgments on ethical dilemmas and arrive at possible ways to negotiate conflicting considerations in various circumstances within the social media context. In her commentary on social media and medical professionalism, Fenwick (2014) discusses how doctors in contemporary practice must deal with conflicting priorities and urges a more pluralistic approach to understanding the notion of professionalism while thinking critically about social media’s current and future implications for practice.  Such conversations can provide opportunities for the doctors to do so and perhaps more crucially, to consider how social media can be used creatively for better patient and health outcomes.

In a recent systematic review on the use of social media in graduate medical education, the authors found ten studies pertaining to resident professionalism (Sterling, Leung, Wright, & Bishop, 2017). However, most were exploratory, surveying residents about their social network behaviour and exploring how program directors use social media to monitor unprofessional behaviour of residents. Although our study was limited to the faculty and residents in one Residency in Singapore, to our knowledge, this is the first study to shed light on how residents as well as faculty negotiate conflicting considerations when evaluating online posts. Instead of hypothetical Facebook posts (whereby it would be possible to address potential biases by varying factors like the age or gender of the poster), we used authentic posts (except Post 6) which respondents might find more relevant and might be more motivated to evaluate and discuss them. Another limitation of the study was that we did not consider how factors like the respondents’ age, type of residency program or their actual usage of social media may influence their evaluation of the posts. However, we followed a systematic, transparent process for coding the written responses and we achieved a high interrater reliability through extensive discussions.

Future work should focus on understanding the complexities of how doctors negotiate conflicting professional values. In this study, respondents were asked to explain their evaluation after rating the appropriateness of the posts and the length of responses varied from one word to several sentences. Due to the limitation of such a study design, the reasoning process of some respondents could not be examined. Conducting interviews or focus groups could enable more in-depth analysis of how participants negotiate conflicting values. Research in moral psychology has shed light on how a person makes moral judgments.  According to cognitive developmentalists a person may spontaneously have a new intuition that contradicts the initial intuitive judgment during the course of thinking about a situation (Kohlberg, 1969; Piaget, 1965). He/she comes to see a dilemma from more than one perspective and experiences competing intuitions. In particular, focus-group discussions could shed light on how the moral judgments of participants in the group might influence one another (Haidt, 2001). Studies designed to focus on examining how doctors resolve conflicting professional values can deepen our understanding of medical professionalism and what it constitutes, within the social media context and beyond.

V. CONCLUSION

This study aimed to explore the considerations and possible competing or conflicting considerations of faculty and residents in the NHG-AHPL Residency program when they evaluate online postings, especially grey posts. Faculty and residents who explained their evaluation took into account a wide range of considerations and these considerations tended to conflict with one another when the respondents considered freedom of expression of the poster and the educational purpose of a post. These findings suggest that doctors within the same Residency do not necessarily have a uniform set of professional priorities and may have to manage conflicting professional and personal values in different contexts. Educators of professional values could recognise the complexity of such conflicts and be sensitive to this in their teaching, whether formally or informally. We recommend creating platforms for conversations on social media use to take place among faculty and residents.

Notes on Contributors

Ms Loo May Eng is Research Analyst at Health Outcomes and Medical Education Research (HOMER), National Healthcare Group. She is the primary author of this manuscript and was involved in the study design, data collection and analysis of this project.

Dr Lee Yee Mun is Senior Consultant at the Department of Urology, Tan Tock Seng Hospital. He was involved in the conceptualisation of this project.                                

Ms Brenda Wong is Program Coordinator at Group Education, National Healthcare Group. She was involved in data collection and analysis.

Ethical Approval

Ethics approval was obtained from the institution’s Domain Specific Review Board (NHG DSRB Ref: 2015/00584).

Acknowledgements

The authors would like to thank all residents and faculty who participated in the questionnaire.

Funding

The research did not receive any specific grant from funding agencies.

Declaration of Interest

There is no conflict of interest to declare.

References

American College of Physicians Ethics, Professionalism, and Human Rights Committee. (2012). American College of Physicians Ethics Manual: Sixth edition. Annals of Internal Medicine, 156(1 Pt 2), 73–104. https://doi.org/10.7326/0003-4819-156-1-201201031-00001

Australian Medical Association. (2010). Social media and the medical profession. Retrieved from https://ama.com.au/article/social-media-and-medical-profession

Canadian Medical Association. (2011). Social media and Canadian physicians: Issues and rules of engagement. Retrieved from https://www.cma.ca/En/Pages/social-media-use.aspx

Chretien, K. C., Farnan, J. M., Greysen, S. R., & Kind, T. (2011). To friend or not to friend? Social networking and faculty perceptions of online professionalism. Academic Medicine, 86(12), 1545-1550.  https://doi.org/10.1097/ACM.0b013e3182356128

Chretien, K. C., & Kind, T. (2014). Climbing social media in medicine’s hierarchy of needs. Academic Medicine, 89(10), 1318–1320. https://doi.org/10.1097/ACM.0000000000000430

Constine, J. (2017, June 27). Facebook now has 2 billion monthly users… and responsibility. Retrieved October 20, 2017, from TechCrunch website http://social.techcrunch.com/2017/06/27/facebook-2-billion-users/

Dawkins, R., King, W. D., Boateng, B., Nichols, M., & Desselle, B. C. (2017). Pediatric residents’ perceptions of potential professionalism violations on social media: A US national survey. JMIR Medical Education, 3(1). https://doi.org/10.2196/mededu.5993

Farnan, J. M., Paro, J. A. M., Higa, J. T., Reddy, S. T., Humphrey, H. J., & Arora, V. M. (2009). Commentary: The relationship status of digital media and professionalism: it’s complicated. Academic Medicine, 84(11), 1479–1481. https://doi.org/10.1097/ACM.0b013e3181bb17af

Fenwick, T. (2014). Social media and medical professionalism: Rethinking the debate and the way forward. Academic Medicine, 89 (10), 1331–1334. https://doi.org/10.1097/ACM.0000000000000436

General Medical Council. (2013). Doctors’ use of social media. Retrieved from http://www.gmc-uk.org/guidance/ethical_guidance/21186.asp

Ginsburg, S., Regehr, G., Hatala, R., McNaughton, N., Frohna, A., Hodges, B., … Stern, D. (2000). Context, conflict, and resolution: A new conceptual framework for evaluating professionalism. Academic Medicine, 75(Supplement), S6–S11.

Greysen, S. R., Johnson, D., Kind, T., Chretien, K. C., Gross, C. P., Young, A., & Chaudhry, H. J. (2013). Online professionalism investigations by state medical boards: First, do no harm. Annals of Internal Medicine, 158(2), 124–130. https://doi.org/10.7326/0003-4819-158-2-201301150-00008

Greysen, S. R., Kind, T., & Chretien, K. C. (2010). Online Professionalism and the Mirror of Social Media. Journal of General Internal Medicine, 25(11), 1227–1229. https://doi.org/10.1007/s11606-010-1447-1

Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108 (4), 814–834.

Hamm, M. P., Chisholm, A., Shulhan, J., Milne, A., Scott, S. D., Klassen, T. P., & Hartling, L. (2013). Social media use by health care professionals and trainees: A scoping review. Academic Medicine, 88(9), 1376–1383. https://doi.org/10.1097/ACM.0b013e31829eb91c

Jain, A., Petty, E. M., Jaber, R. M., Tackett, S., Purkiss, J., Fitzgerald, J., & White, C. (2014). What is appropriate to post on social media? Ratings from students, faculty members and the public. Medical Education, 48(2), 157–169. https://doi.org/10.1111/medu.12282

Kesselheim, J. C., Batra, M., Belmonte, F., Boland, K. A., & McGregor, R. S. (2014). New professionalism challenges in medical training: An exploration of social networking. Journal of Graduate Medical Education, 6(1), 100–105. https://doi.org/10.4300/JGME-D-13-00132.1

Kim, H. S., & Sherman, D. K. (2007). “Express yourself”: Culture and the effect of self-expression on choice. Journal of Personality and Social Psychology, 92(1), 1-11. https://doi.org/10.1037/0022-3514.92.1.1

Kind, T., Greysen, S. R., & Chretien, K. C. (2012). Pediatric clerkship directors’ social networking use and perceptions of online professionalism. Academic Pediatrics, 12(2), 142–148. https://doi.org/10.1016/j.acap.2011.12.003

Kohlberg, L. (1969). Stage and sequence: The cognitive development approach to socialization. In D. Goslin (Ed.), Handbook of socialization theory (pp. 347–480). Chicago: Rand McNally.

Piaget, J. (1965). The moral judgment of the child. Retrieved from http://archive.org/details/moraljudgmentoft005613mbp

Rocha, P. N., & de Castro, N. A. A. (2014). Opinions of students from a Brazilian medical school regarding online professionalism. Journal of General Internal Medicine, 29(5), 758–764. https://doi.org/10.1007/s11606-013-2748-y

Saldana, J. (2012). The Coding Manual for Qualitative Researchers. London: SAGE.

Singapore Medical Council. (2016). smc handbook on medical ethics. Retrieved from http://www.healthprofessionals.gov.sg/content/hprof/smc/en/topnav/guidelines/ethical_code_and_ethical_guidelines.html

Sterling, M., Leung, P., Wright, D., & Bishop, T. F. (2017). The Use of Social Media in Graduate Medical Education: A Systematic Review. Academic Medicine, 92(7), 1043–1056. https://doi.org/10.1097/ACM.0000000000001617

Wells, D. M., Lehavot, K., & Isaac, M. L. (2015). Sounding Off on Social Media: The Ethics of Patient Storytelling in the Modern Era. Academic Medicine, 90(8), 1015–1019. https://doi.org/10.1097/ACM.0000000000000668

*Loo May Eng
Health Outcomes and Medical Education Research,
National Healthcare Group
Tel: (65) 6359 6362
Email: may_eng_loo@nhg.com.sg