Submitted: 18 January 2021
Accepted: 19 May 2021
Published online: 5 October, TAPS 2021, 6(4), 107-117
https://doi.org/10.29060/TAPS.2021-6-4/OA2470

Sarah Brown & Justin L C Bilszta

Department of Medical Education, Melbourne Medical School, University of Melbourne, Australia

Abstract

Introduction: Use of complementary and alternative medicine (CAM) is popular in the general population and medical practitioners may not be fully equipped in their knowledge of CAM to advise patients appropriately. The aim of this paper was to perform a scoping review of current literature describing undergraduate medical student use, attitudes, and knowledge of CAM as a means of better understanding the educational needs of these students.

Methods: A systematic search of Medline, PubMed and the Education Resources Information Center (ERIC) databases with keywords related to “complementary and alternative medicine” and “undergraduate medical students” for relevant articles published until August 2020.  

Results: Of 131 papers identified, 38 underwent full review. It was found 13-80% of medical students use CAM, and overall have a positive attitude towards CAM therapies.  Female medical students and those with religiosity had more positive attitudes towards CAM than their male colleagues and those without a religion. Knowledge of CAM is lacking with approximately only half of students feeling they were knowledgeable about CAM therapies. Popular information resources are the Internet and social media, but students expressed they want more teaching of CAM in the undergraduate medical curriculum.

Conclusion: Evidence suggests high usage of CAM amongst undergraduate medical students, and positive attitudes towards CAM therapies; however, knowledge of CAM is poor, and students want more CAM teaching to upskill them in counselling patients interested in CAM therapies. Further areas for research include a better understanding of resources medical students use for their knowledge and how gender and religiosity influence attitudes towards CAM.  

Keywords:           Undergraduate Medical Student, Complementary and Alternative Medicine, CAM, Attitude, Knowledge, Use

Practice Highlights

• Medical student’s personal use of CAM is significant, with the most popular therapies being massage, meditation and herbal medicine.
• Medical students have a positive attitude towards CAM but potential differences between attitudes of preclinical and clinical student populations exist.
• Medical student’s knowledge of CAM is lacking, and this impacts their ability to advise patients appropriately.
• Medical students want more CAM teaching integrated into their UGME curriculum, and believe conventional western medicine could benefit from CAM methods and ideas.
• Better integration of the principles of EBM rather than teaching related to specific CAM therapies can provide medical students with the skills to critique claims of CAM efficacy.

Complementary and alternative medicine (CAM) encompasses a range of health practices including, but not limited to, acupuncture, naturopathy, chiropractic traditional Chinese medicine (TCM), herbal, vitamin & homeopathic therapies (Zollman & Vickers, 1999). Complementary refers to the practice of something alongside conventional Western medicine, whilst alternative refers to the practice of something instead of conventional Western medicine (Zollman & Vickers, 1999).

In comparison to conventional Western medicine, the eclectic range of CAM is often cited as having a poor-quality evidence-base regarding its efficacy (Australian Medical Association, 2018), yet it remains popular with the general public (Frass et al., 2012). With surveys suggesting 10–76% of the public has used CAM (Ernst, 2000; Harris et al., 2012) the demand for CAM is evident. How knowledgeable medical practitioners are, and their attitudes towards CAM, can influence the advice they might provide to patients who seek information about CAM therapies or want to use CAM in lieu of conventional Western medicine. Janamian et al. (2011) have reported general practitioners feel they do not have the education to appropriately advise patients on CAM (Janamian et al., 2011). This may deter patients from seeking guidance from their healthcare provider and result in a breakdown of communication and lack of congruence over health goals (Xue et al., 2007).

Medical students’ insights into their prevailing attitudes and knowledge of CAM can help determine if CAM teaching within undergraduate medical education (UGME) should be expanded to better meet the needs of growing public demand. The last significant review of medical students’ attitudes and knowledge of CAM was published in 2016 (Joyce et al., 2016), and reported overall medical students believed they lacked knowledge of CAM but were generally positive towards CAM education and thought CAM teaching should be incorporated into UGME. What this review did not explore was the rates of CAM use by medical students, and whether this influenced attitudes towards CAM. Importantly, since the publication of Joyce et al’s review, not only have there been additional studies exploring medical students’ use, attitudes and knowledge of CAM, but the increasing use of the Internet and social media as a source of information means it is timely to re-evaluate the findings of this earlier study and determine whether the results presented remain relevant.

We conducted this scoping review of primary studies to evaluate undergraduate medical student use, attitudes, and knowledge of CAM as a means of better understanding the educational needs of these students. The specific research questions were: 1) what is the usage and knowledge of CAM among medical students?; 2) what are medical students’ attitudes towards CAM?; 3) what factors influence medical students’ attitudes towards CAM, and where do they seek information about CAM? and; 4) what are medical students’ views on the current teaching they receive in UGME about CAM?

II. METHODS

This study adopted the “Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews” (PRISMA-ScR) protocol (Tricco et al., 2018).

A. Search Strategy

Electronic databases Medline (Ovid), PubMed and ERIC were searched for full text articles describing undergraduate medical students use, attitudes, and knowledge of CAM (see Appendix 1). Additional papers were found through a hand search of reference lists of articles identified through the database search. There was no limit on publication date.

B. Selection of Sources

Papers were included if published in English and reported on undergraduate medical students. Papers were excluded if they were not published in English; opinion pieces or review articles; reported on non-undergraduate medical students or; reported on osteopathic medical students. This group was excluded due to the potential bias these students may have due to their medical training including specific CAM teaching.

C. Key Words and Boolean Operators

The search strategy (see Appendix 1) included Boolean operators (AND, NOT, OR), Medical Subject Headings (MeSH) and specific key words.

D. Data Extraction and Charting

Data extraction was performed using a predetermined checklist, and included

• Article details: first author and publication year
• Research aim: purpose of the study
• Method: methods of data collection
• Sample size: number of students invited to participate and the number who responded
• Student CAM use: number of students using CAM, including use of specific therapies and timeframe of use
• Student attitudes/perceptions to CAM: student attitudes to CAM including desire to undertake further education about CAM, beliefs about CAM efficacy, role of CAM in conventional Western medicine and, sources of CAM information
• Influencing factors: factors reported to influence student’s attitudes/perceptions to CAM
• Limitations: identified study limitations

E. Synthesis of Results

Included studies were described by author, publication year, and characteristics described above. Thematic analysis was conducted to identify commonality between included studies. No inferences were made about students’ use, attitudes, or knowledge of CAM if this was not explicitly stated.

Literature searching, title and abstract screening, full text review & data extraction, and charting were undertaken by the 1st author (SB). Where there was uncertainty regarding the aforementioned, these articles were reviewed independently by the 2nd author (JB) and discussed until consensus was reached. The 2nd author also independently reviewed the data extraction and charting results once this process was completed by the 1st author.

III. RESULTS

From the initial search strategy, 131 articles were obtained for screening based on title and abstract. Following screening, 17 duplicate citations were excluded, leaving 114 papers. Abstract and title screening excluded papers focused on pharmacy or nursing students, and those that did not centre on the research questions. Following this, 58 articles underwent full review. Of these, 21 were excluded as they were not in English, did not provide enough detail regarding medical students specifically or focused heavily on medical curricula. The total number of articles included is 38 (see Figure 1); a full version of the outcome harvesting data can be found at https://doi.org/10.6084/m9.figshare.14471250 (Brown & Bilszta, 2021).

Figure 1: PRISMA Diagram

A. CAM usage

Twenty-two studies assessed CAM usage in medical students (Baugniet et al., 2000; Chaterji et al., 2007; Chez et al., 2001; Ditte et al., 2011; Donald et al., 2010; Flaherty et al., 2015; Frye et al., 2006; Greenfield et al., 2000, 2002; Hegde et al., 2018; Hopper & Cohen, 1998; James et al., 2016; Lie & Boker, 2004, 2006; Nicolao et al., 2010; Oberbaum et al., 2003; Rees et al., 2009; Sadeghi et al., 2016; Samara et al., 2019; Shani-Gershoni et al., 2008; Wong et al., 2006; Yildirim et al., 2010). Rates of usage ranged from 13% to 80% (Chez et al., 2001; Wong et al., 2006). The most common CAM therapy used by medical students was difficult to evaluate due to inconsistent methods of collecting this information with studies either asking students to nominate usage from a predefined list of therapies (Baugniet et al., 2000; Chaterji et al., 2007; Frye et al., 2006; Hegde et al., 2018; James et al., 2016; Lie & Boker, 2004, 2006; Nicolao et al., 2010; Sadeghi et al., 2016; Samara et al., 2019; Shani-Gershoni et al., 2008; Wong et al., 2006; Yildirim et al., 2010); self-report the therapies they had engaged with (Donald et al., 2010; Greenfield et al., 2000, 2002) or; indicating CAM usage without nominating a specific therapy (Chez et al., 2001; Ditte et al., 2011; Flaherty et al., 2015; Hopper & Cohen, 1998; Oberbaum et al., 2003; Rees et al., 2009). The challenge in evaluated CAM usage is highlighted by two separate studies (Lie & Boker, 2004, 2006) which reported a range of practices to be popular including massage, meditation and yoga compared to vitamins, meditation and spirituality, respectively (Lie & Boker, 2004, 2006).

B. CAM perception/attitudes

Overall, the general attitude of medical students towards CAM is positive (Ahmed et al., 2017; Akan et al., 2012; Chaterji et al., 2007; Chez et al., 2001; Ditte et al., 2011; Flaherty et al., 2015; Frye et al., 2006; Furnham & McGill, 2003; Godin et al., 2007; Greenfield et al., 2002, 2006; Hegde et al., 2018; Hopper & Cohen, 1998; James et al., 2016; Lie & Boker, 2004, 2006; Loh et al., 2013; Nicolao et al., 2010; Oberbaum et al., 2003; Perkin et al., 1994; Rees et al., 2009; Sadeghi et al., 2016; Samara et al., 2019; Templeman et al., 2015; Torkelson et al., 2006; Wong et al., 2006; Xie et al., 2020; Yeo et al., 2005) with positive attitudes ranging from 49% to 60% (Ditte et al., 2011; Perkin et al., 1994; Sadeghi et al., 2016). Different methods were used to collect this data including validated measures (for example: (Flaherty et al., 2015; Frye et al., 2006; Rees et al., 2009)), rating scales with various stages of validation (for example: (Ahmed et al., 2017; Oberbaum et al., 2003; Sadeghi et al., 2016; Samara et al., 2019)) or qualitative interviews (for example: (Templeman et al., 2015)).

Differences in the attitude of preclinical and clinical cohorts varied between studies. Akan et al., Furnham & McGill and Syverstad et al. reported preclinical students had a more positive attitude compared to clinical students (Akan et al., 2012; Furnham & McGill, 2003; Syverstad et al., 1999), however others found clinical students were more positive, perhaps due to them receiving more CAM education (Chaterji et al., 2007; Xie et al., 2020). However, multiple studies found no difference between preclinical and clinical cohorts (Ditte et al., 2011; Flaherty et al., 2015; Hopper & Cohen, 1998; Rees et al., 2009). Additionally, all studies bar one (Hübner et al., 2012) asked students to consider attitudes and perceptions in relation to ‘…their clinical practice…’ rather than associated with a specific clinical context or medical condition.

Attitudes towards specific CAM therapies demonstrated acupuncture received consistent positive perceptions ranging from 77% to above 90% (Loh et al., 2013; Torkelson et al., 2006; Yeo et al., 2005). Other therapies with consistent positive perceptions include massage and meditation, with more than 70% of medical students feeling positively towards these practices (Furnham & McGill, 2003; Loh et al., 2013; Torkelson et al., 2006). Conversely, Greenfield et al. and Loh et al. both found homeopathy had the most negative perception from students (Greenfield et al., 2006; Loh et al., 2013). Interestingly, studies indicate ambivalence towards chiropractic practice, with some reporting positive attitudes whilst others reported negative attitudes (Greenfield et al., 2006; Loh et al., 2013). When asked about the integration of CAM with conventional Western medicine, there was overwhelming belief from medical students that benefits may come from applying these practices together. Ahmed et al. and Chez et al. reported a large percentage of their cohorts believed conventional Western medicine could benefit from CAM methods and ideas (67% and 89%, respectively) (Ahmed et al., 2017; Chez et al., 2001). Similar findings have been reported by others, with the percentage of students wanting the integration of CAM and conventional Western medicine ranging from 71% to 92% (Chez et al., 2001; Hopper & Cohen, 1998; Lie & Boker, 2004; Loh et al., 2013; Nicolao et al., 2010; Torkelson et al., 2006; Xie et al., 2020; Yeo et al., 2005).

An important question to consider when evaluating medical student knowledge and attitudes towards CAM is whether they believe CAM works. This question arose frequently in the reviewed literature, with investigators trying to determine if there was congruence between having a positive perception of CAM and belief in its efficacy. When asked if participants found the results of CAM were mainly due to the placebo effect, there were mixed findings with 39-59% of medical students agreeing the CAM efficacy was mainly due to the placebo effect (Baugniet et al., 2000; Nicolao et al., 2010; Yeo et al., 2005; Yildirim et al., 2010). Interestingly, recent studies by Xie et al. and Samara et al. reported only 10% and 35%, respectively, believed CAM efficacy was due to the placebo effect (Samara et al., 2019; Xie et al., 2020). Whether this is simply a cohort effect or a reflection that over time students may be becoming less sceptical of CAM, is unable to be confirmed.

Factors that influence attitudes towards CAM should be considered as they may alter how medical students perceive CAM therapies. The most frequent influence reported was gender, with female medical students overall tending to have a more positive attitude than male colleagues (Akan et al., 2012; Chaterji et al., 2007; Ditte et al., 2011; Donald et al., 2010; Flaherty et al., 2015; Frye et al., 2006; Furnham & McGill, 2003; Godin et al., 2007; Greenfield et al., 2002, 2006; Hübner et al., 2012; Oberbaum et al., 2003; Rees et al., 2009; Schmidt et al., 2005; Xie et al., 2020). Interestingly, Ditte et al. found male medical students feared the social stigma of using CAM more than female students (45% vs 36%) and Donald et al. reported male students were more likely to be more sceptical of CAM than female students (50% vs 44%) (Ditte et al., 2011; Donald et al., 2010).

Religion may also influence attitudes to CAM. Yeo et al. reported nearly two thirds of their cohort believed spiritual/religious beliefs influenced attitudes towards CAM (Yeo et al., 2005). Furnham & McGill reported similar findings as female students and students that rated themselves higher religiously, were more eager for CAM to be on the curriculum compared to males and those who were less religious (Furnham & McGill, 2003). In contrast, Loh et al. reported students with no religion tended to be less interested in CAM (Loh et al., 2013).

C. CAM knowledge

The included studies suggest approximately only half of medical students felt knowledgeable about CAM (Hopper & Cohen, 1998; James et al., 2016; Sadeghi et al., 2016; Shani-Gershoni et al., 2008) and the most common individual therapies students were subjectively knowledgeable in were massage (Baugniet et al., 2000; Chez et al., 2001; Frye et al., 2006; Furnham & McGill, 2003), acupuncture (Akan et al., 2012; Nicolao et al., 2010; Sadeghi et al., 2016; Yeo et al., 2005), chiropractic (Baugniet et al., 2000; Chez et al., 2001; Frye et al., 2006) and herbal medicine (Akan et al., 2012; Baugniet et al., 2000; Chez et al., 2001). It is clear, however, medical students still perceive significant gaps in their knowledge of CAM. Nicolao et al. reported although students indicated acupuncture and homeopathy as their most knowledgeable areas, this was only for a minority of the cohort (34% and 43%, respectively) and the majority of students felt their level of knowledge, and therefore ability to appropriately advise patients, was poor (Nicolao et al., 2010). In one Australian study, investigators found out of ten common CAM therapies (acupuncture, chiropractic, herbal medicine, homeopathy, hypnosis, massage, meditation, naturopathy, reflexology and spiritual healing) 56% of their cohort had no knowledge of any of these therapies (Hopper & Cohen, 1998).

The Internet appears to be a popular avenue for finding CAM information with Lie and Boker reporting 81% of their cohort used online resources as their main information source (Lie & Boker, 2004). Although the specific online resources used were not described, journals, books and health databases were used less frequently in comparison (41%, 38% and 28%, respectively) (Lie & Boker, 2004). A study of Chinese medical students with CAM teaching integrated into their UGME, found their cohort’s main source of information was from teachers, making up 91% with only 2% using the Internet (Xie et al., 2020). Samara et al. reported 73% of their cohort used social media as their main source of information (Samara et al., 2019).

It is clear medical students want more teaching about CAM in their UGME. Flaherty et al. reported just over two thirds of students across all year levels had a desire to study CAM, however, there was a positive correlation between those students who wanted more teaching and a positive attitude towards CAM (Flaherty et al., 2015). Similar findings were reported by Perkins et al. and Frye et al. with approximately 85% of clinical and preclinical students believing they should learn about CAM in UGME (Frye et al., 2006; Perkin et al., 1994). In terms of this would look like in the curriculum, Greenfield et al. reported 67% wanted to study the theory, with Greiner et al. and Yeo et al. finding 72% and 86%, respectively, wanted clinical exposure (Greenfield et al., 2006; Greiner et al., 2000; Yeo et al., 2005). In semi-structured qualitative interviews of Australian medical students, CAM education was considered important regardless of specialisation and CAM literacy was an ethical responsibility (Templeman et al., 2015).  

IV. DISCUSSION

In this Discussion, we provide commentary on our findings, consider how these might be applied to the design of UGME curriculum, and suggest areas of further research.

In the current study, we found a significant number of medical students use CAM, however reported rates varied considerably, and our analysis suggests inconsistency in approaches used to assess CAM use may explain the observed differences. As highlighted, data collection was split between those studies that provided a predefined list of specific CAM therapies, with a limited number of response options, and the exact therapies between papers incongruent; those that allowed students to self-report usage and; those indicating CAM usage without nominating a specific therapy (see Table 1). Cultural differences may also explain usage variation; for example, the high use of yoga in Indian medical students (Hegde et al., 2018), TCM in Chinese medical students (Wong et al., 2006) and, traditional Iranian medicine in Iranian medical students (Sadeghi et al., 2016).

Table 1. Example of the differing methods used to collect student CAM usage data. Data collection of medical student CAM usage was split between those studies that provided a predefined list of specific CAM therapies; those that allowed students to self-report usage and; those indicating CAM usage without nominating a specific therapy. The above examples highlight the inconsistent approaches to collecting this information.

Although medical students believe conventional Western medicine could benefit from CAM methods and ideas, an important caveat to this finding is it does not consider how they feel about specific CAM therapies. For example, would medical students think conventional Western medicine could benefit from integrating herbal medicine or homeopathy approaches, compared to osteopathy or acupuncture? Given the diversity of CAM, it is arguably unfeasible (Wetzel et al., 1998) to teach students about each individual therapy, so an EBM teaching model equipping students with skills needed to critically appraise evidence presents a potential solution, and several medical schools have designed curricular in this way (Bailey et al., 2015; Forjuoh et al., 2003; Hassed, 2004; Jeffries, 2001; Mahapatra et al., 2017; Owen & Lewith, 2001; Perlman & Stagnaro-Green, 2010). This addresses a thematically overwhelming and resource intensive area of medical education where learned knowledge and skills can be adapted for differing CAM therapies.

Only three studies (Lie & Boker, 2004; Samara et al., 2019; Xie et al., 2020) specifically explored how medical students use digital resources to supplement formal CAM teaching, so this is a clear direction for further research. It is important medical students have access to high quality evidence-based information and the skills to determine the validity of evidence presented on digital platforms. Whilst there is evidence to demonstrate resources such as blogs, social networking sites and online support groups are useful platforms for health education, social connection and experience sharing, they are at significant risk of misinformation and mischaracterisation of CAM (Delgado-López & Corrales-García, 2018; Molassiotis & Xu, 2004). Sharma et al. (2016) have underscored the challenges in developing approaches for identifying the reliability of CAM-related information on the Internet, which may not be supported by a reliable evidence-base (Sharma et al., 2016). This means medical students require skills to critique claims related to CAM efficacy for their own education needs, but also so they can appropriately counsel patients. As advocated, an EBM teaching model would provide students with the skills to critique claims of CAM efficacy as well as empower patients to make informed decisions relevant to their health needs.

V. LIMITATIONS OF THE STUDY

As we did not conduct a systematic review of the literature, we are unable to draw any conclusions about the quality of the included studies. We excluded non-English studies, and this potentially creates a gap in the literature and may have altered the findings as CAM use, attitudes and knowledge may be different in non-English speaking countries (for example, the high use of acupuncture or Traditional Chinese medicine in Asian countries). The lack of consistency in data related to medical student use, attitudes and knowledge of CAM therapies is reflected in its heterogeneity. This makes generalisations related to ‘medical students’, even within the same medical school or UGME curriculum, difficult. Despite this significant limitation, the diversity of reported outcomes reflects important contextual differences in medical programs, and medical student cohorts, across the globe.

VI. CONCLUSION

Overall, medical students have a positive attitude towards CAM, with significant influencing factors being gender and religion, and potential differences between preclinical and clinical cohorts. The included literature indicates medical students’ personal use of CAM is significant, with therapies such as massage, meditation and herbal medicine being more popular than others. It is clear medical students’ CAM knowledge is lacking and integration of the principles of EBM may be beneficial in addressing this, ultimately leading to better educated doctors who have better relationships with their patients.

Notes on Contributors

JB: conceptualised the review; JB, SB: designed the search strategy; SB: conducted the search and screened the literature; JB, SB: reviewed the included studies; SB: analysed the data; JB: drafted the manuscript; JB, SB: critically revised the manuscript. All authors had full access to all study data, read and approved the final version of the manuscript.

Ethical Approval

As all the data were retrieved from public databases, this study did not require institutional review board approval.

Data Availability

The data that support the findings of this study are openly available in Figshare repository, https://doi.org/10.6084/m9.figshare.14471250. The data extraction sheet used during the current study is available from the corresponding author on request. All data is based on published studies.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Declaration of Interest

None of the listed authors have financial and personal relationships with organisations or people that could inappropriately influence their work.

References

Ahmed, S. M., Al-Mansour, M. A., Mohamed, E. Y., Medani, K. A., Abdalla, S. M., & Mahmoud, W. S. (2017). Medical Students’ Opinion Toward the Application of Complementary and Alternative Medicine in Healthcare. Saudi Journal of Medicine & Medical Sciences, 5(1), 20-25. https://doi.org/10.4103/1658-631X.194255

Akan, H., Izbirak, G., Kaspar, E. C., Kaya, C. A., Aydin, S., Demircan, N., Bucaktepe, P. G., Ozer, C., Sahin, H. A., & Hayran, O. (2012). Knowledge and attitudes towards complementary and alternative medicine among medical students in Turkey. BMC Complementary & Alternative Medicine, 12, 115. https://doi.org/10.1186/1472-6882-12-115

Alwhaibi, M., AlRuthia, Y., & Meraya, A. (2019). Gender differences in the prevalence of complementary and alternative medicine utilization among adults with arthritis in the United States. Evidence-Based Complementary & Alternative Medicine: eCAM. https://doi.org/10.1155/2019/8739170

Alwhaibi, M., & Sambamoorthi, U. (2016). Sex differences in the use of complementary and alternative medicine among adults with multiple chronic conditions. Evidence-Based Complementary & Alternative Medicine: eCAM. https://doi.org/10.1155/2016/2067095

Australian Medical Association. (2018). AMA Position Statement, Complementary Medicine. https://ama.com.au/position-statement/ama-position-statement-complementary-medicine-2018

Bailey, M. L., Chudgar, S. M., Engle, D. L., Moon, S. D., Grochowski, C. O., & Clay, A. S. (2015). The impact of a mandatory immersion curriculum in integrative medicine for graduating medical students. Explore (NY), 11(5), 394-400. https://doi.org/10.1016/j.explore.2015.07.003

Barraco, D., Valencia, G., Riba, A., Nareddy, S., Draus, C., & Schwartz, S. (2005). Complementary and alternative medicine (CAM) use patterns and disclosure to physicians in acute coronary syndromes patients. Complementary Therapies in Medicine, 13(1), 34-40. https://doi.org/ 10.1016/j.ctim.2005.02.003

Baugniet, J., Boon, H., & Ostbye, T. (2000). Complementary/alternative medicine: comparing the view of medical students with students in other health care professions. Family Medicine, 32(3), 178-184.

Bell, R., Suerken, C., Grzywacz, J., Lang, W., Quandt, S., & Arcury, T. (2006). Complementary and alternative medicine use among adults with diabetes in the United States. Alternative Therapies in Health and Medicine, 12, 16-22.

Ben-Arye, E., Schiff, E., Vintal, H., Agour, O., Preis, L., & Steiner, M. (2012). Integrating complementary medicine and supportive care: Patients’ perspectives toward complementary medicine and spirituality. Journal of Alternative and Complementary Medicine, 18(9), 824-831. https://doi.org/10.1089/acm.2011.0327

Bishop, F., & Lewith, G. (2010). Who uses CAM? A narrative review of demographic characteristics and health factors associated with CAM use. Evidence-Based Complementary & Alternative Medicine: eCAM, 7(1), 11-28. https://doi.org/10.1093/ecam/nen023

Brown, S., & Bilszta, J. L. C. (2021). Medical students use, attitudes towards, and knowledge of complementary and alternative medicine: A scoping review [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.14471250

Chao, M., Wade, C., & Kronenberg, F. (2008). Disclosure of complementary and alternative medicine to conventional medical providers: Variation by race/ethnicity and type of CAM. Journal of the National Medical Association, 100(11), 1341-1349. https://doi.org/10.1016/s0027-9684(15)31514-5

Chaterji, R., Tractenberg, R. E., Amri, H., Lumpkin, M., Amorosi, S. B., & Haramati, A. (2007). A large-sample survey of first- and second-year medical student attitudes toward complementary and alternative medicine in the curriculum and in practice. Alternative Therapies in Health & Medicine, 13(1), 30-35.

Chez, R. A., Jonas, W. B., & Crawford, C. (2001). A survey of medical students’ opinions about complementary and alternative medicine. American Journal of Obstetrics and Gynecology, 185(3), 754-757. https://doi.org/10.1067/mob.2001.117484

Davis, E. L., Oh, B., Butow, P. N., Mullan, B. A., & Clarke, S. (2012). Cancer patient disclosure and patient-doctor communication of complementary and alternative medicine use: A systematic review. The Oncologist, 17(11), 1475-1481. https://doi.org/10.1634/theoncologist.2012-0223

Delgado-López, P., & Corrales-García, E. (2018). Influence of Internet and social media in the promotion of alternative oncology, cancer quackery, and the predatory publishing phenomenon. Cureus, 10(5), e2617. https://doi.org/10.7759/cureus.2617

Ditte, D., Schulz, W., Ernst, G., & Schmid-Ott, G. (2011). Attitudes towards complementary and alternative medicine among medical and psychology students. Psychology, Health & Medicine, 16(2), 225-237. https://doi.org/10.1080/13548506.2010.532559

Donald, G. K., Mackereth, P., & Tobin, I. (2010). Medical students and acupuncture: A short sharp placement experience! Acupuncture Medicine, 28(1), 12-15. https://doi.org/10.1136/aim.2009.001602

Ellison, C. G., Bradshaw, M., & Roberts, C. A. (2012). Spiritual and religious identities predict the use of complementary and alternative medicine among US adults. Preventive Medicine, 54(1), 9-12. https://doi.org/10.1016/j.ypmed.2011.08.029

Ernst, E. (2000). Prevalence of use of complementary/alternative medicine: A systematic review. Bulletin of the World Health Organization, 78(2), 252-257.

Flaherty, G., Fitzgibbon, J., & Cantillon, P. (2015). Attitudes of medical students toward the practice and teaching of integrative medicine. Journal of Integrative Medicine, 13(6), 412-415. https://doi.org/10.1016/s2095-4964(15)60206-0

Foley, H., Steel, A., Cramer, H., Wardle, J., & Adams, J. (2019). Disclosure of complementary medicine use to medical providers: A systematic review and meta-analysis. Scientific Reports, 9(1573). https://doi.org/10.1038/s41598-018-38279-8

Forjuoh, S. N., Rascoe, T. G., Symm, B., & Edwards, J. C. (2003). Teaching medical students complementary and alternative medicine using evidence-based principles. The Journal of Alternative & Complementary Medicine, 9(3), 429-439.

Frass, M., Strassl, R. P., Friehs, H., Müllner, M., Kundi, M., & Kaye, A. D. (2012). Use and acceptance of complementary and alternative medicine among the general population and medical personnel: A systematic review. Ochsner Journal, 12(1), 45-56.

Frye, A. W., Sierpina, V. S., Boisaubin, E. V., & Bulik, R. J. (2006). Measuring what medical students think about complementary and alternative medicine (CAM): A pilot study of the complementary and alternative medicine survey. Advances in Health Sciences Education, 11(1), 19-32. https://doi.org/10.1007/s10459-005-1587-y

Furnham, A., & McGill, C. (2003). Medical students’ attitudes about complementary and alternative medicine. The Journal of Alternative & Complementary Medicine, 9(2), 275-284. https://doi.org/10.1089/10755530360623392

Godin, G., Beaulieu, D., Touchette, J. S., Lambert, L. D., & Dodin, S. (2007). Intention to encourage complementary and alternative medicine among general practitioners and medical students. Behavioral Medicine, 33(2), 67-77.

Greenfield, S. M., Brown, R., Dawlatly, S. L., Reynolds, J. A., Roberts, S., & Dawlatly, R. J. (2006). Gender differences among medical students in attitudes to learning about complementary and alternative medicine. Complementary Therapies in Medicine, 14(3), 207-212. https://doi.org/10.1016/j.ctim.2005.12.001

Greenfield, S. M., Innes, M. A., Allan, T. F., & Wearn, A. M. (2002). First year medical students’ perceptions and use of complementary and alternative medicine. Complementary Therapies in Medicine, 10(1), 27-32. https://doi.org/10.1054/ctim.2002.0501

Greenfield, S. M., Wearn, A. M., Hunton, M., & Innes, M. A. (2000). Considering the alternatives: A special study module in complementary therapy. Complementary Therapies in Medicine, 8(1), 15-20.

Greiner, K. A., Murray, J. L., & Kallail, K. J. (2000). Medical student interest in alternative medicine. The Journal of Alternative & Complementary Medicine, 6(3), 231-234. https://doi.org/10.1089/acm.2000.6.231

Harris, P. E., Cooper, K. L., Relton, C., & Thomas, K. J. (2012). Prevalence of complementary and alternative medicine (CAM) use by the general population: A systematic review and update. International Journal of Clinical Practice, 66(10), 924–939. https://doi.org/10.1111/j.1742-1241.2012.02945.x

Hassed, C. S. (2004). Bringing holism into mainstream biomedical education. The Journal of Alternative and Complementary Medicine, 10(2), 405-407. https://doi.org/10.1089/107555304323062428

Hegde, S. V., Rao, S. K., Menezes, R. G., Kotian, S. M., & Shetty, S. (2018). Knowledge, attitude, and practice of yoga in medical students: Assessment of anthropometry and lifestyle factors. International Journal of Yoga Therapy, 28(1), 9-14. https://doi.org/10.17761/2018-00005r1

Heller, T., Kloos, C., Mueller, N., Roemelt, J., Keinki, C., Wolf, G., Mueller, U. A., & Huebner, J. (2020). Complementary and alternative medicine is positively associated with religiousness/ spirituality. Journal of Complementary & Integrative Medicine, Advance online publication. https://doi.org/10.1515/jcim-2018-0023

Hopper, I., & Cohen, M. (1998). Complementary therapies and the medical profession: A study of medical students’ attitudes. Alternative Therapies in Health and Medicine, 4(3), 68-73.

Hsiao, A.-F., Wong, M. D., Miller, M. F., Ambs, A. H., Goldstein, M. S., Smith, A., Ballard-Barbash, R., Becerra, L. S., Cheng, E. M., & Wenger, N. S. (2008). Role of religiosity and spirituality in complementary and alternative medicine use among cancer survivors in California. Integrative Cancer Therapies, 7(3), 139-146. https://doi.org/10.1177/1534735408322847

Hübner, J., Wicker, S., & Münstedt, K. (2012). Survey of students regarding their attitudes toward complementary and alternative medicine in oncology. Forschende Komplementarmedizin, 19(6), 319-321. https://doi.org/10.1159/000345973

James, P. B., Bah, A. J., & Kondorvoh, I. M. (2016). Exploring self-use, attitude and interest to study complementary and alternative medicine (CAM) among final year undergraduate medical, pharmacy and nursing students in Sierra Leone: A comparative study. BMC Complementary & Alternative Medicine, 16, 121. https://doi.org/10.1186/s12906-016-1102-4

Janamian, T., O’Rourke, P., Myers, S. P., & Eastwood, H. (2011). Information resource needs and preference of Queensland general practitioners on complementary medicines: Result of a needs assessment. Evidence-Based Complementary & Alternative Medicine: eCAM, https://doi.org/10.1155/2011/810908

Jawahar, R., Yang, S., Eaton, C. B., McAlindon, T., & Lapane, K. L. (2012). Gender-specific correlates of complementary and alternative medicine use for knee osteoarthritis. Journal of Women’s Health, 21(10), 1091-1099. https://doi.org/10.1089/jwh.2011.3434

Jeffries, W. B. (2001). A fourth-year elective course in alternative medicine. Academic Medicine, 76(5), 525-526. https://doi.org/10.1097/00001888-200105000-00060

Joyce, P., Wardle, J., & Zaslawski, C. (2016). Medical student attitudes towards complementary and alternative medicine (CAM) in medical education: A critical review. Journal of Complementary & Integrative Medicine, 13(4), 333-345. https://doi.org/10.1515/jcim-2014-0053

Kristoffersen, A. E., Stub, T., Salamonsen, A., Musial, F., & Hamberg, K. (2014). Gender differences in prevalence and associations for use of CAM in a large population study. BMC Complementary & Alternative Medicine, 14, 463. https://doi.org/10.1186/1472-6882-14-463

Lie, D., & Boker, J. (2004). Development and validation of the CAM Health Belief Questionnaire (CHBQ) and CAM use and attitudes amongst medical students. BMC Medical Education, 4, 2. https://doi.org/10.1186/1472-6920-4-2

Lie, D. A., & Boker, J. (2006). Comparative survey of Complementary and Alternative Medicine (CAM) attitudes, use, and information-seeking behaviour among medical students, residents & faculty. BMC Medical Education, 6, 58. https://doi.org/10.1186/1472-6920-6-58

Loh, K. P., Ghorab, H., Clarke, E., Conroy, R., & Barlow, J. (2013). Medical students’ knowledge, perceptions, and interest in complementary and alternative medicine. The Journal of Alternative & Complementary Medicine, 19(4), 360-366. https://doi.org/10.1089/acm.2012.0014

Mahapatra, S., Bhagra, A., Fekadu, B., Li, Z., Bauer, B. A., & Wahner-Roedler, D. L. (2017). Incorporation of integrative medicine education into undergraduate medical education: A longitudinal study. Journal of Integrative Medicine, 15(6), 442-449. https://doi.org/10.1016/s2095-4964(17)60367-4

McCurdy, E. A., Spangler, J. G., Wofford, M. M., Chauvenet, A. R., & McLean, T. W. (2003). Religiosity is associated with the use of complementary medical therapies by pediatric oncology patients. Journal of Pediatric Hematology Oncology, 25(2), 125-129.

Molassiotis, A., & Xu, M. (2004). Quality and safety issues of web-based information about herbal medicines in the treatment of cancer. Complementary Therapies in Medicine, 12(4), 217-227. https://doi.org/10.1016/j.ctim.2004.09.005

Nicolao, M., Täuber, M. G., Marian, F., & Heusser, P. (2010). Complementary medicine courses in Swiss medical schools: Actual status and students’ experience. Swiss Medical Weekly, 140(3-4), 44-51.

Oberbaum, M., Notzer, N., Abramowitz, R., & Branski, D. (2003). Attitude of medical students to the introduction of complementary medicine into the medical curriculum in Israel. Israel Medical Association Journal, 5(2), 139-142.

Owen, D., & Lewith, G. T. (2001). Complementary and alternative medicine (CAM) in the undergraduate medical curriculum: the Southampton experience. Medical Education, 35(1), 73-77. https://doi.org/10.1046/j.1365-2923.2001.00762.x

Pedersen, C. G., Christensen, S., Jensen, A. B., & Zachariae, R. (2013). In God and CAM we trust. Religious faith and use of complementary and alternative medicine (CAM) in a nationwide cohort of women treated for early breast cancer. Journal of Religion and Health, 52(3), 991-1013. https://doi.org/10.1007/s10943-012-9569-x

Perkin, M. R., Pearcy, R. M., & Fraser, J. S. (1994). A comparison of the attitudes shown by general practitioners, hospital doctors and medical students towards alternative medicine. Journal of the Royal Society of Medicine, 87(9), 523-525.

Perlman, A., & Stagnaro-Green, A. (2010). Developing a complementary, alternative, and integrative medicine course: One medical school’s experience. Journal of Alternative & Complementary Medicine, 16(5), 601-605. https://doi.org/10.1089/acm.2009.0276

Rees, C. E., Wearn, A. M., Dennis, I., Amri, H., & Greenfield, S. M. (2009). Medical students’ attitudes to complementary and alternative medicine: Further validation of the IMAQ and findings from an international longitudinal study. Medical Teacher, 31(2), 125-132. https://doi.org/10.1080/01421590802139724

Sadeghi, M., Rabiepoor, S., Forough, A. S., Jabbari, S., & Shahabi, S. (2016). A survey of medical students’ knowledge and attitudes toward complementary and alternative medicine in Urmia, Iran. Journal of Evidence-Based Integrative Medicine, 21(4), 306-310. https://doi.org/10.1177/2156587215605751

Samara, A. M., Barabra, E. R., Quzaih, H. N., & Zyoud, S. H. (2019). Use and acceptance of complementary and alternative medicine among medical students: A cross sectional study from Palestine. BMC Complementary & Alternative Medicine, 19(1), 78. https://doi.org/10.1186/s12906-019-2492-x

Schmidt, K., Rees, C., Greenfield, S., Wearn, A. M., Dennis, I., Patil, N. G., Amri, H., & Boon, H. (2005). Multischool, international survey of medical students’ attitudes toward “holism”. Academic Medicine, 80(10), 955-963. https://doi.org/10.1097/00001888-200510000-00017

Shani-Gershoni, Z., Freud, T., Press, Y., & Peleg, R. (2008). Knowledge and attitudes of internists compared to medical students regarding acupuncture. Israel Medical Association Journal, 10(3), 219-223.

Sharma, V., Holmes, J., & Sarkar, I. (2016). Identifying complementary and alternative medicine usage information from internet resources. A systematic review. Methods of Information in Medicine, 55(4), 322-332. https://doi.org/10.3414/ME15-01-0154

Syverstad, D., Norheim, A., & Fonnebo, V. (1999). Changes in medical students’ attitudes toward acupuncture: The possible impact of role models. Education for Health: Change in Learning & Practice, 12(2), 195-210.

Templeman, K., Robinson, A., & McKenna, L. (2015). Student identification of the need for complementary medicine education in Australian medical curricula: A constructivist grounded theory approach. Complementary Therapies in Medicine, 23(2), 257-264. https://doi.org/10.1016/j.ctim.2015.02.002

Thomson, P., Jones, J., Evans, J. M., & Leslie, S. L. (2012). Factors influencing the use of complementary and alternative medicine and whether patients inform their primary care physician. Complementary Therapies in Medicine, 20(1-2), 45-53. https://doi.org/10.1016/j.ctim.2011.10.001

Torkelson, C., Harris, I., & Kreitzer, M. J. (2006). Evaluation of a complementary and alternative medicine rotation in medical school. Alternative Therapies in Health and Medicine, 12(4), 30-34.

Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMAScR): Checklist and explanation. Annals of Internal Medicine, 169, 467-473. https://doi.org/10.7326/M18-0850

Wetzel, M. S., Eisenberg, D. M., & Kaptchuk, T. J. (1998). Courses involving complementary and alternative medicine at US medical schools. Journal of the American Medical Association, 280(9). https://doi.org/10.1001/jama.280.9.784 

Wong, W. C., Lee, A., Wong, S. Y., Wu, S. C., & Robinson, N. (2006). Strengths, weaknesses, and development of Traditional Chinese Medicine in the health system of Hong Kong: Through the eyes of future Western doctors. Journal of Alternative & Complementary Medicine, 12(2), 185-189; 191-182. https://doi.org/10.1089/acm.2006.12.185

Xie, H., Sang, T., Li, W., Li, L., Gao, Y., Qiu, W., & Zhou, H. (2020). A survey on perceptions of complementary and alternative medicine among undergraduates in China. Evidence-Based Complementary & Alternative Medicine: eCAM, 2020, 9091051. https://doi.org/10.1155/2020/9091051

Xue, C. C., Zhang, A. L., Lin, V., Da Costa, C., & Story, D. F. (2007). Complementary and alternative medicine use in Australia: A national population-based survey. Journal of Alternative & Complementary Medicine, 13(6), 643-650. https://doi.org/10.1089/acm.2006.6355

Yeo, A. S., Yeo, J. C., Yeo, C., Lee, C. H., Lim, L. F., & Lee, T. L. (2005). Perceptions of complementary and alternative medicine amongst medical students in Singapore – A survey. Acupuncture in Medicine, 23(1), 19-26. https://doi.org/10.1136/aim.23.1.19

Yildirim, Y., Parlar, S., Eyigor, S., Sertoz, O. O., Eyigor, C., Fadiloglu, C., & Uyar, M. (2010). An analysis of nursing and medical students’ attitudes towards and knowledge of complementary and alternative medicine (CAM). Journal of Clinical Nursing, 19(7-8), 1157-1166. https://doi.org/10.1111/j.1365-2702.2009.03188.x

Zollman, C., & Vickers, A. (1999). What is complementary medicine? British Journal of Medicine, 319(7211), 693-696. https://doi.org/10.1136/bmj.319.7211.693

*Justin Bilszta
Department of Medical Education,
Melbourne Medical School,
University of Melbourne, Australia
Parkville, Australia 3052
Tel: +61 3 8344 7826
Fax: +6567775702
Email: jbilszta@unimelb.edu.au

Submitted: 6 November 2020
Accepted: 27 January 2021
Published online: 5 October, TAPS 2021, 6(4), 92-106
https://doi.org/10.29060/TAPS.2021-6-4/OA2459

Ardi Findyartini^1,2, Natalia Widiasih Raharjanti³, Nadia Greviana^1,2, Gregorius Ben Prajogi⁴ & Daniar Setyorini²

¹Department of Medical Education, Faculty of Medicine Universitas Indonesia, Indonesia; ²Medical Education Center, Indonesian Medical Education and Research Institute (IMERI), Faculty of Medicine, Universitas Indonesia, Indonesia; ³Department of Psychiatry, Faculty of Medicine, Universitas Indonesia, Indonesia; ⁴Department of Oncology Radiation, Faculty of Medicine, Universitas Indonesia, Indonesia

Abstract

Introduction: E-portfolio may facilitate the entrustment process from supervisors to residents in postgraduate medical education. The present study was aimed at identifying necessary features of an e-portfolio application and to conduct pilot study in a teaching hospital.

Methods: Six programs participated. Eight interviews with education directors and six focus group discussions with residents and supervisors were completed for the needs analysis stage. The application was developed based on the thematic analysis of the needs analysis stage. The mobile-app e-portfolio pilot was conducted for four weeks and a modified version of the System Usability Scale (SUS) was distributed to participants following the pilot program.

Results: Key features of the e-portfolio were identified. A total of 45 supervisors and 66 residents participated in the pilot study. The residents utilised the application according to the clinical activities and supervision level, and the information was fed to the supervisors as per the application design. Challenges during the pilot study in terms of feedback provision and residents’ workload which influence the e-portfolio use for entrustment decisions are discussed.

Conclusion: Current e-portfolio features were created for supervision and are potential to facilitate the entrustment process in Entrustable Professional Activities (EPA) implementation. The pilot study highlighted challenges of the implementation which should be considered for future improvement.

Keywords:           Postgraduate Medical Education, E-Portfolio, Entrustable Professional Activities, Supervision, Needs Analysis, Pilot Study

Practice Highlights

• Development of assessment application, including e-portfolio, should consider input from the stakeholders.
• The e-portfolio should further be defined to emphasise the documentation, reflection, and feedback processes.
• Consideration of adequate evidence for EPA should be determined based on the aim of the e-portfolio.
• A robust e-portfolio implementation will potentially support the implementation of EPA.
• Challenges in e-portfolio implementation should be aware of and tackled for future improvement.

I. INTRODUCTION

Supervision is a critical component of postgraduate medical education as it allows residents to have a gradual achievement of competencies while still fulfilling patient safety standards. Patient care conducted by residents under adequate supervision can have results comparable to the care provided by more experienced medical doctors (Farnan et al., 2012). To entrust the resident, a supervisor considers several factors, including incidents experienced during supervision, the resident’s characteristics, the results of assessments, and encountered situations (Cianciolo & Kegg, 2013).

Ten Cate et al. (2015) introduced the Entrustable Professional Activities (EPA) concept to facilitate the implementation of a competency-based curriculum in the workplace (Ten Cate et al., 2015). EPAs are observable and measurable units of work that can correspond to competency milestones, allowing for safe and effective performance (Carraccio et al., 2017).  Easily accessible and interpreted information about a resident’s past performance using the EPA concept is, therefore, suggested to facilitate the entrustment process, both for ad-hoc and summative purposes (Hauer et al., 2013; Ten Cate et al., 2015).

Entrustment decision-making refers to a supervisor’s decision to trust a resident to carry outpatient care ‘without supervision’ (Crossley et al., 2011; Ten Cate, 2006; Weller et al., 2014). Given the need for assured patient safety, ‘without supervision’ refers to the reduction of educational supervision and the provision of ‘relational autonomy’, whereby interdependence between the resident, the supervisor, the healthcare team, and the healthcare system becomes critical (Holmboe et al., 2011). However, the amount of data accessible about a resident’s performance should be analysed comprehensively to enable ad-hoc entrustment (Sandhu, 2018).

Logbooks and portfolios have been considered as workplace-based assessment methods that would enable summarising a resident’s performance as well as maintaining assessment results during training (van Tartwijk & Driessen, 2009). Electronic portfolios, or e-portfolios, are believed to increase the accessibility of a portfolio in medical training provided that the purpose of the portfolio development is well defined, such as for learning, assessment, or continuing professional development (Deketelaere et al., 2009, Tepper et al, 2020; van Tartwijk & Driessen, 2009). Generally, an e-portfolio aims to monitor a resident’s competency development and to stimulate the capacity for self-reflection (Meeus et al., 2006).

An e-portfolio may consist of a list of a resident’s performance, the supporting evidence, and the resident’s self-reflection (van der Schaaf, et al., 2017). The content is specific according to the e-portfolio’s purpose and the required activities/competencies for the resident at each level (Mulder et al., 2010). The use of an e-portfolio with a mobile application and updated evidence of residents’ performance potentially enhance informed decision-making in the entrustment process, and hence, it can be embedded in the resident supervision system (Ten Cate et al., 2016). An exploratory study in Australia showed that an EPA-driven e-portfolio model assisted trainees and supervisors in agreeing upon expected trainees’ performance in order to obtain competence level (Bramley et al., 2020).

Implementation of e-portfolio as a supporting system for EPA should consider supporting and hindering factors specific to an institution. Implementation of best practices in this area can be accompanied by an exploration of the most suitable system according to the stakeholders’ needs. Therefore, the current study aimed to explore stakeholders’ needs and develop a mobile-app e-portfolio model in a teaching hospital with various postgraduate medical education programmes located in a resource-limited setting. The study also aims to identify challenges of e-portfolio implementation. The research questions of this study were: (1) What are mobile-app e-portfolio features needed to support EPA implementation in the current teaching hospital? (2) How do stakeholders perceive the use of the developed mobile-app e-portfolio during the pilot study? 

II. METHODS

A. Context

Cipto Mangunkusumo General Hospital (CMGH) is the main teaching hospital for the Faculty of Medicine Universitas Indonesia (FMUI) and is located in Jakarta, Indonesia. All 31 study programmes implement a competency-based medical curriculum and EPA approach. CMGH is obliged to have all study programmes participate in clinical activities, which are gradually entrusted to residents in accordance with the resident’s level. The EPA document in form of a matrix is then used to develop supervision system. Supervisors are trained to provide feedback through compulsory clinical teacher training conducted by the faculty. Assessment documentation has been conducted mostly manually in hardcopies for all study programs; consequently, tracking the residents’ performance to provide formative or summative EPA decisions has been challenging in the current setting.

B. Design

This study is participatory action research with a mixed -method approach. The exploration stage was aimed at exploring the needs and features expected from the mobile application. Following the exploration stage, the research team in collaboration with a professional vendor developed the mobile-app e-portfolio according to the results of the exploration stage. The pilot implementation phase aimed to implement and train residents and supervisors in the use of the application. A follow-up evaluation was conducted to assess users’ perception about the use of the e-portfolio application.

C. Respondents

Respondents of this study were education leaders from the medical school and the teaching hospital. Residents, clinical teachers, and administrators from six study programs representing medicine, surgery, and diagnostic: anaesthesiology and intensive care, obstetrics and gynaecology, psychiatry, oncology radiation, radiology, and internal medicine at CMGH/FMUI were recruited with maximum variety sampling method taking into account gender, age, and length of study (residents) or work (clinical teachers). Detailed information of the study was given to all participants who then provided written consents prior to the interviews and completion of the questionnaire.

D. Data Collection

The exploration stage involved interviews with: (1) residency programme directors; (2) education leaders from the medical school and the teaching hospital. Focus group discussions (FGDs) were also conducted with clinical teachers and residents from the six study programmes. The guiding questions of the interviews and FGDs were developed according to e-portfolio and EPA concepts used in medical education, and are shown in Table 1. The data obtained were used as a reference for further development of the application’s content and features. In addition, the EPA document from each study program was also analysed as baseline data for developing the platform.

Following the development of the application, a four-week-long pilot implementation was conducted in November 2018. After week four, the back-end system data were analysed. Supervisors’ and residents’ perceptions of the use of mobile-app e-portfolio for supporting EPA were collected using a modified questionnaire of System Usability Scale (SUS) (Brooke, 1986). Active commentaries were also added in the questionnaire to obtain users’ suggestions:

Table 1. Guiding questions of interviews and focus groups

E. Data Analysis

The transcripts of interviews and FGDs of the need analysis stage were analysed using thematic analysis. The agreed subthemes and themes and the results of the analysis of EPA documents from each study program were then translated into a computational framework to be developed as mobile application e-portfolio’s features, also considering the best practice and theoretical framework of e-portfolio development and EPA practice.

Back-end data of the activities, case variations, and supervisors’ feedback were descriptively analysed using SPSS IBM 22.0. The words submitted on the reflection-on-action section of each activity were counted along with the feedback recorded by the supervisors. Residents’ reflections were categorised into those with descriptions of difficulties (DD), lessons learned (LL), and action plans (AP). The feedback recorded was classified into descriptive (D), constructive (C), and neither descriptive nor constructive (N) categories.

The modified SUS questionnaires were descriptively analysed using chi-square/fisher exact analysis (SPSS 22.0) to compare residents’ and supervisors’ perceptions. The active commentaries in the questionnaire were also thematically analysed to identify areas of improvement in the e-portfolio development.

III. RESULTS

A. Exploration Stage

Eight in-depth interviews were conducted with each residency program director as well as with the leader of FMUI and with teaching hospital director representatives. Two FGDs were conducted, each with three study programmes faculty members. In addition, three FGDs were conducted, each with two study programmes resident representatives. The number and characteristics of the participants are described in Table 2.

Table 2. Characteristics of Focus Group Discussion (FGD) Participants in Exploration Stage

Four main themes and 22 subthemes were identified during the exploration stage. The main themes identified were (1) resident supervision; (2) current implementation of workplace-based assessments; (3) current implementation of portfolios and logbooks; and (4) development of the mobile application e-portfolio.

	Theme	Subtheme	# of Quotes	Quotations
1	Resident supervision	Constraints in supervision (faculty members, patient/clinical cases)	59	“For example, during the mid-level programme, residents were required to be fully supervised, but when they should be fully supervised was not specifically stated.” (W1, p. 1)
		Form of supervision	46	“…we should report to the supervisor when anything goes wrong to get feedback; otherwise, it will become our fault.” (FGD 2, [R5], p. 2)
		Determination of entrustment (written curriculum, residents and patients factor, workplace, agreement among faculty members)	42	“At some point, we might feel that the resident is eligible to perform the procedure after he/she has observed several times, and then we try to let him/her give it a try, but we are actually risking our trust…” (W7, p. 5)
		Background and determination of supervision (workplace, resident’s educational stage)	26	“Supervision was given hierarchically; the third-year residents attain duty as the chief, supervisor, and the highest form was division of weekly rounds.” (FGD 4, [R1], p. 2)
		Documentation of supervision level	10	“Our logbook contains a very clear explanation about supervision level, such as what cases should be achieved so that we can evaluate and reflect on our progress in one semester.” (FGD 3, [R6], p. 9)
2	The current implementation of workplace-based assessment	Scheduled WBA	17	“During each rotation, residents and consultants were scheduled with case-based discussions or miniCEX; while for surgical DOPS, [it] is not specifically scheduled because it depends on the availability of the case.” (W3, p. 1)
		WBA as stage step-up procedure	6	“In our programme, there are several cases we need to get done at each level, usually cases written in the logbook.” (FGD 2, [R7], p. 3)
		Based on the written curriculum	4	“In our programme, we arrange evaluation at every level. We evaluate every intern according to the required competency they should achieve.” (FGD 4, [R3], p. 7)
3	The current implementation of portfolio and logbook	Technical portfolio and logbook implementation (as monitoring, communication, and assessment instrument)	26	“On a routine basis, the form was filled in each afternoon, when the patient’s information, such as name, weight, and height, are recorded.” (FGD 2, [R7], p. 4)
		Constraints in portfolio and logbook implementation (lack of ownership, faculty member factor)	38	“Due to the enormous workload every day, such as examining patients, attending division rounds, filling out electronic health records, and receiving so many instructions, they couldn’t complete the logbook routinely. Besides, the supervisor might find it unnecessary to write down their supervision in the log book after discussions, rounds, or treatments.” (W2, p. 10)
		Understanding of portfolio and logbook for residents’ development (clinical, academic, non-academic)	24	“Portfolio consists of one’s creativity, innovation, and organisation experiences.” (W8, p. 1)
4	Development of the mobile application-based e-portfolio	Residents’ performance and experience (clinical and academic) tracking	20	“We evaluated one intern’s competency in ultrasound, and the result wasn’t quite satisfying. It turned out that this intern lacked of practice that could’ve been done every day since the cases were quite abundant.” (W6, p. 10)
		User-friendliness and real-time access	18	“The application may have to be ‘consultant-friendly,’ so they can finish it in a click.” (FGD 2, [R3], p. 7)
		Feedback documentation	13	“In my opinion, there has to be some immediate feedback.” (FGD 2, [R5], p. 7)
		Accessibility and confidentiality	11	“As they save the record of the treatment that has been checked by the doctor in charge, they can no longer change it because only the supervisor has the authority to change.”(FGD 4, [R4,] p. 19)
		Encompass achievement of competence and modules within the curriculum	7	“We want it to be comprehensive, so it can be evaluated. This is about clinical skills, but then the related academic skill is also important.” (W4, p. 9)
		Ensuring patients were treated by competent and authorised residents	4	“We need to ensure that the patient is treated by competent, authorised, and certified doctors.” (W7, p. 10)
		Reminder and consequences for undocumented activity	5	“There must be something to force the resident to write down his activity so the next day, he can keep up with the new tasks.” (W2, p. 10)
		Paperless attribute, but printable if needed	6	“The record and the format will be the students’ database as well as the supervisors’. Therefore, it needs to be printable for our benefit.” (FGD 2, [R3], p. 7)
		Integrated with service care system	3	“We have to write down the same thing repetitively in so many books and records.” (FGD 3, [R8], p. 12)
		Collect important evidence of accomplishments (cases and complications, image documentation)	4	“Compiling the number of cases is required, and reporting complications in cases is equally important. Let’s say they are in the third level, but complications occurred in most of the procedures they performed.” (W3, p. 7)

Table 3. Themes and Subthemes Identified in Exploration Stage

Submitted: 19 October 2020
Accepted: 12 April 2021
Published online: 5 October, TAPS 2021, 6(4), 80-91
https://doi.org/10.29060/TAPS.2021-6-4/OA2456

Balakrishnan Ashokka^1,2, Tat Leang Lee¹ & Daniëlle M.L. Verstegen³

¹Department of Anaesthesia, National University Health System, National University Hospital, Singapore; ²Centre for Medical Education (CenMED), Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Department of Educational Development and Research, Faculty of Health Medicine and Life Sciences, Maastricht University, Netherlands

Abstract

Introduction: There are certain factors in exam preparedness that are not well studied in the postgraduate medical education context. Non-academic predictors have been extensively researched but usually in isolation.

Methods: The study involved a sequential explanatory mixed methods research design. The study was conducted among anaesthesia postgraduates appearing for high-stake nation-wide primary examination. Data obtained by a questionnaire assessing pre-examination attributes were compared with the students’ reflections through focus group discussions (FGD) after the formal declaration of results. The examination had an overall pass rate of 42.9% (18 out of 42).

Results: The study showed that pre-examination questionnaire could identify attributes and study behaviours in the postgraduates who passed. Passers procrastinated three times lesser, pursuing a timetable-based study (conscientiousness); had higher metacognitive self-regulation (p value<0.05) applying concentrated self-directed learning & effective group study and higher self-efficacy compared to those who failed. The focus group discussions affirmed of these attributes in candidates who ‘breeze through exams’. Postgraduate success required better ‘work-study’ balance, self & cross regulation and peer and faculty support.        

Conclusion: Implementing a composite tool to assess ‘exam preparedness’, we propose, would help the learners and teachers to skim for non-academic factors (metacognitive self-regulation, self-efficacy, conscientiousness) that influence the chances of success. Understanding & predicting this would help educators to identify the ‘candidates with difficulty’ and delegate personalised faculty attention. This could guide the exam candidates to have a ‘reality check’ to plan and pace their effort with peer learning, consolidated study and goal orientation.

Keywords:           Postgraduate Exam Success, Non-Academic Predictors, Self-Regulation

Practice Highlights

• Non-academic attributes impact success in postgraduate examinations.
• Postgraduate exam success necessitates work-study & work-life balance.
• Time on task, self-regulation to task demands is needed when assessments are tougher and high stake.
• Exam preparedness: A collective attribute is proposed with a questionnaire to measure predictability of exam success.
• Shunning away from ‘shame of mock vivas’ spirals down to poor chance of passing.

Submitted: 16 January 2021
Accepted: 17 May 2021
Published online: 5 October, TAPS 2021, 6(4), 65-79
https://doi.org/10.29060/TAPS.2021-6-4/OA2447

Yee Cheun Chan¹, Chi Hsien Tan¹ & Jeroen Donkers²

¹Department of Medicine, National University Health System, Singapore; ²Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Netherlands

Abstract

Introduction: Reflection is a critical component of learning and improvement. It remains unclear as to how it can be effectively developed. We studied the impact of reflective writing in promoting deep reflection in the context of learning Accreditation Council for Graduate Medical Education (ACGME) competencies among residents in an Internal Medicine Residency programme.

Methods: We used a convergent parallel mixed-methods design for this study in 2018. We analysed reflective writings for categories and frequencies of ACGME competencies covered and graded them for levels of reflection. We collected recently graduated residents’ perceptions of the value of reflective writings via individual semi-structured interviews.

Results: We interviewed nine (out of 27) (33%) participants and analysed 35 reflective writings. 30 (86%) of the writings showed a deep level (grade A or B) of reflection. Participants reflected on all six ACGME competencies, especially ‘patient care’. Participants were reluctant to write but found benefits of increased understanding, self-awareness and ability to deal with similar future situations, facilitation of self-evaluation and emotional regulation. Supervisors’ guidance and feedback were lacking.

Conclusion: We found that a reflective writing programme within an Internal Medicine Residency programme promoted deep reflection. Participants especially used self-reflection to enhance their skills in patient care. We recognised the important role of mentor guidance and feedback in enhancing reflective learning.

Keywords:           Reflective Writing, ACGME Competencies, Internal Medicine, Residency

Practice Highlights

• Reflection is a critical component of learning and improvement.
• Written reflections offer theoretical advantages over other forms of reflections in requiring more commitment and ownership of experience, promoting critical thinking and offering more opportunities for feedback.
• Written reflections can be a record for mentored reflection, included in a portfolio, used in ongoing self-assessment and longitudinal integration of learning.
• Practitioners reported benefits of increased understanding, self-awareness and ability to deal with similar future situations, facilitation of self-evaluation and emotional regulation.
• Supervisors’ guidance and feedback are important for enhancing reflective learning.

I. INTRODUCTION

Medical competencies are developed through experience and application, not just knowledge acquisition (Frank et al., 2010). Kolb (1984) conceptualises experiential learning in a four-stage cyclical process.  An experience triggers a reflection on that experience that leads to the formation of abstract concepts and generalisations. These are then tested in future situations, resulting in new experiences.  Reflection is an essential aspect of the learning experience. It remains unclear how it can be developed most effectively.

Reflection is a complex concept that has been defined in several ways. One definition describes it as the process of engaging self in attentive, critical, exploratory, and iterative interactions with one’s thoughts and actions, and their underlying conceptual frame, with a view on the change itself (Nguyen et al., 2014). Thus, reflection has an iterative dimension which describes a cyclic process with phases triggered by experience, which produces new understanding, and then an intention to act differently in future encounters of similar experience (Mann et al., 2009). There is also a vertical dimension correlating to the depth of reflection. The surface levels are more descriptive and less analytical than the deeper levels. For example, Boud et al. (1985) described iterative phases of returning to experience, attending to feelings, re-evaluation of experience and outcome/resolution. Mezirow (1991) described increasing depth of reflection as habitual action, thoughtful action/understanding, reflection, critical reflection. Evidence suggests that deeper levels of reflections are associated with deep approaches to learning (Leung & Kember, 2003).

Reflective writing is a commonly utilised method in developing reflective learning but evidence for its value remains limited. Theoretically, written reflections offer advantages over other types of reflections e.g. verbal discussions. Creating an artefact by writing involves a commitment to learning, ownership of experience, promotes critical thinking and offers more opportunities for feedback (Aronson, 2011). The writings can be a record for mentored reflection, included in a portfolio, used in ongoing self-assessment and longitudinal integration of learning. A systematic review (Winkel et al., 2017) looking at the impact of reflection in graduate medical education found only three studies (Epner & Baile, 2014; Levine et al., 2008; Winkel et al., 2010) that involved reflective writings. Levine et al. (2008) found that the process of narrative writings encouraged deepening of reflection leading to reconsideration of core values and priorities, improved self-awareness, provided an emotional outlet and motivation to improve. However, the study did not formally gauge the depth of reflections in the writings.

We aimed to further study the impact of reflective writing in promoting reflection and the learning of medical competencies. Better understanding this will guide the development of reflective learning skills in training programmes for medical trainees.

A. Research Question

Does reflective writing promote deep reflection in the context of learning core competencies defined by the Accreditation Council for Graduate Medical Education (ACGME) (Accreditation Council for Graduate Medical Education, 2013)?

II. METHODS

A. Research Paradigm and Design

Our study adopted a phenomenological approach. We used a convergent parallel mixed-methods design (Figure 1). Quantitative data included the tabulation of the categories of ACGME competencies and the frequency they were covered in the reflective writings. Quantitative scoring of levels of reflections in the reflective writings was done using two grading scales. Qualitative data included graduates’ perceptions of the value and effects of reflective writings on learning ACGME competencies. The quantitative and qualitative data were analysed, compared and related together to answer the research question.

Figure 1. Convergent parallel mixed-methods design to study the role of reflective writing in promoting reflective learning of ACGME competencies

B. Study Setting and Subjects

The study setting was the Internal Medicine Residency of a single tertiary university hospital in 2018. We have used reflective writing as a tool for developing reflective learning and practice in our Internal Medicine Residency. Our programme has a competency-based curriculum using the ACGME framework. Residents are encouraged to write their reflections on how an encounter or situation helped them develop one or more of the competencies. They are required to include at least two such reflective writings in their portfolio each year. The reflective writings are not graded but are read by the residents’ supervisors as part of their portfolio’s content during regular reviews and by the competency review committee during 6-monthly meetings. They provide insight into the residents’ competencies development.

We invited all past residents (27) who graduated from the programme one year earlier to participate. We used a convenience sampling method. We determined the final number of participants after data saturation was reached in the analysis of the collected qualitative data.

The study was approved by the National Healthcare Group Domain Specific Review Board (NHG DSRB) (Reference number: 2017/01219). We obtained informed consent from each participant.

C. Data Collection

We collected and analysed reflective writings from the participants’ three years of residency. We used individual semi-structured interviews to gather participants’ perceptions to avoid bias from others’ opinions. One researcher (YCC) conducted, recorded and transcribed the interviews. Box 1 shows the main questions that were asked. An interactive approach was used, and interviews conducted till thematic saturation was reached.

Box 1. Main questions asked during interviews

D. Data Analysis

Reflective writings from participants were analysed for the categories as well as frequencies of ACGME competencies covered. They were graded for levels of reflection using grading rubrics. To reduce possible interpretation bias or conflicts related to confidentiality and power relationships, grading was done by an ‘external’ co-researcher (CHT) who was a faculty member of the Neurology residency programme. Two grading scales were used. The first (Box 2) had a simple grading scale from A to F (Moon, 2004). The other grading rubric provided more categorical details and was based on that used by Tsingos et al. (2015) (Supplementary Table 1). The rubric graded the reflective writings on seven stages of reflection based on the model by Boud et al. (1985) and categories of non-reflector, reflector or critical reflector according to Mezirow’s model (Mezirow, 1991). The co-researcher read through each reflective writing and first determined if stages of ‘returning to experience’, ‘attending to feelings’, ‘association’, ‘integration’, ‘validation’, ‘appropriation’ and ‘outcomes of reflection’ were present. He then assessed if the written content related to these stages fit the descriptors for non-reflector, reflector or critical reflector as given in the rubric. Finally, he graded the reflective writing on the simple grading scale of A to F according to the descriptors given (Box 2).

Submitted: 1 December 2020
Accepted: 5 April 2021
Published online: 5 October, TAPS 2021, 6(4), 49-64
https://doi.org/10.29060/TAPS.2021-6-4/OA2443

Yasushi Matsuyama¹, Hitoaki Okazaki¹, Kazuhiko Kotani², Yoshikazu Asada³, Shizukiyo Ishikawa¹, Adam Jon Lebowitz⁴, Jimmie Leppink⁵ & Cees van der Vleuten⁶

¹Medical Education Center, Jichi Medical University, Japan; ²Center for Community Medicine, Jichi Medical University, Japan; ³Center for Information, Jichi Medical University, Japan; ⁴Department of General Education, Jichi Medical University, Japan; ⁵Hull York Medical School, University of York, United Kingdom; ⁶School of Health Professions Education, Maastricht University, The Netherlands

Abstract

Introduction: Previous studies indicate that professional identity formation (PIF), the formation of a self-identity with the internalised values and norms of professionalism, may influence self-regulated learning (SRL). However, it remains unclear whether a PIF-oriented intervention can improve SRL in clinical education. The aim of this study was to explore whether a PIF-oriented mentoring platform improves SRL in a clinical clerkship.

Methods: A mixed-methods study was conducted. Forty-one students in a community-based clinical clerkship (CBCC) used a PIF-oriented mentoring platform. They articulated the values and norms of professionalism in a professional identity essay, elaborated on future professional self-image, and reflected on their current compared to future selves. They made a study plan while referring to PIF-based self-reflection and completed it. The control group of 41 students completed CBCC without the PIF-oriented mentoring platform. Changes in SRL between the two groups were quantitatively compared using the Motivated Strategies for Learning Questionnaire. We explore how PIF elements in the platform affected SRL by qualitative analysis of questionnaire and interview data.

Results: A moderate improvement in intrinsic goal orientation (p = 0.005, ε² = 0.096) and a mild improvement in critical thinking (p = 0.041, ε² = 0.051) were observed in the PIF-oriented platform group. Qualitative analysis revealed that the PIF-oriented platform fostered professional responsibility as a key to expanding learning goals. Gaining authentic knowledge professionally fostered critical thinking, and students began to elaborate knowledge in line with professional task processes.

Conclusion: A PIF-oriented mentoring platform helped students improve SRL during a clinical clerkship.

Keywords:           Self-Regulated Learning, Professional Identity Formation, Clinical Clerkship

Practice Highlights

• Encourage students to verbalise their future self-image as a medical professional.
• Encourage students to reflect on their current selves compared with their perceived future ones.
• Promote in-depth communication between students and role models to foster self-regulated learning.
• Train mentors to become professional role models as self-regulated learners.

I. INTRODUCTION

Rapid advances in clinical knowledge require medical professionals to update their knowledge autonomously throughout their practice. Self-regulation in life-long learning has therefore become an important competency, and competency-based undergraduate medical education has emphasised students’ self-regulated learning (SRL) (Berkhout et al., 2018; Brydges & Butler, 2012; Frank, 2005; Sandars & Cleary, 2011). SRL is defined as learners’ active participation in their own learning processes from metacognitive, motivational, and behavioural perspectives (Zimmerman, 1989). In undergraduate education, SRL has been related to academic achievements (Artino, Cleary et al., 2014; Artino, Dong et al., 2012; Song et al., 2011; Turan & Konan, 2012), clinical skills (Cleary & Sandars, 2011) and emotional management (van Nguyen et al., 2015).

Several reports have claimed that drastic changes in learning context, from structured learning in preclinical years to less-structured and complex learning in clinical clerkships, may be too challenging for students and lead to insufficient learning (Berkhout et al., 2015, 2018; Cho et al., 2017; van Houten-Schat et al., 2018). This question appears more serious in East Asian countries, including Japan, where strong teacher instruction in pre-university education and teacher-centred curricula are the norm (Iwata & Doi, 2017; Lam & Lam, 2009; Tagawa, 2008). In order to make the typically limited clinical clerkship period a fruitful learning opportunity, remediation for struggling students from the perspective of SRL might be important (Durning et al., 2011; van Houten-Schat et al., 2018).

Several reports have shown that individualised mentoring intervention is effective in fostering SRL in clinical practice. For example, Aho et al. (2015) found that mentor-assisted SRL for surgical habits of residents led to more frequent practice and improved skills compared to peers. In Stuart et al. (2005), individualised guidance on strategies and learning plans raised students’ awareness of the learning process. However, the educational interventions undertaken in this study will focus on another contextual characteristic that may facilitate SRL – Professional Identity Formation (PIF) – defined according to Cruess et al. (2014, p.1447) as “a representation of self, achieved in stages over time during which the characteristics, values, and norms of the medical profession are internalised”.

In response to advances in medical science and the increasingly diverse needs of society, “professional” attributes such as autonomy, self-regulation, and social responsibility have been emphasised, in addition to traditional moral and ethical education emphasising healer roles (Cruess & Cruess, 1997, 2019). Furthermore, formation of professional identity can result in individuals thinking, acting, and feeling like physicians (Cruess et al., 2014; Cruess & Cruess, 2019). During the formation of professional identity, medical students begin to perceive belonging to a professional community and increase attention to role models (Jarvis-Selinger et al., 2012; Kalet, Buckvar-Keltz, Harnik et al., 2017; Kalet, Buckvar-Keltz, Monson et al., 2018). Emulation of role models’ self-regulation in learning behaviour is also expected.

People are more likely to interpret difficult experiences as task important when an accessible identity feels congruent to the task (Oyserman et al., 2017). In the context of this study, growing professional identity as a ‘physician-to-be’ might strengthen the perceived importance of engaging in challenges during clinical clerkships, and in self-regulating learning behaviours. In addition, when physicians perceive their identity as professionals, they begin to view daily learning tasks as high-stakes, and to self-regulate learning behaviours as coping strategies (Matsuyama et al., 2018). Another study has suggested that an explicit future professional self-image in medical students leads to self-reflection, increased attention to learning strategies of professional role models, and diversification of learning strategies (Matsuyama et al., 2019). Given that PIF is associated with motivational states, self-reflection, and diversified learning strategies, SRL may be facilitated by introduction of a PIF-oriented intervention.

This study specifically focused on PIF as a facilitating factor for SRL, because previous studies  have suggested possible benefits of PIF-oriented education even for East Asian medical students, who are generally considered to have less SRL due to influence of pre-university education, with its strong faculty instruction and in-university teacher-centred curricula (Matsuyama et al., 2018, 2019).

The purpose of this study was to examine whether SRL during clinical training can be fostered using a mentorship tool emphasising PIF, in addition to conventional mentorship by faculty members. In the pre- and post-clinical clerkship mentorship, students were asked to elaborate on their future self-image as professionals and compare their current and future selves to strengthen self-reflection under mentor support. Learners were aided in articulating their values and norms of medical professionalism by using the Professional Identity Essay (PIE) (Kalet, Buckvar-Keltz, Harnik et al., 2017; Kalet, Buckvar-Keltz, Monson et al., 2018), a self-administered questionnaire with 9 questions relevant to PIF. This encouraged mentors to understand the developmental stage of each learner’s professionalism and to provide individualised feedback on PIE and their future self-image. The feedback was also aimed at remediation for those whose self-images showed underdeveloped professionalism (low developmental stages in PIE). Study plans in the clinical clerkship were developed with reference to PIE-based self-images. We have named this platform ‘PIF-oriented mentoring platform for SRL (PIF-SRL)’. An overview of PIF-SRL is provided in Figure 1.

Figure 1. Overview of the PIF-SRL and research data collection

This studied centred on two research questions:

1. Does PIF-SRL improve SRL during the period around the clinical clerkship?
2. If so, how does the PIF-oriented elements in PIF-SRL improve SRL?

This study was approved by the ethics committee of Jichi Medical University (reference number: 19-001).

II. METHODS

A. Settings

1) Community-based clinical clerkship in Jichi Medical University: The mission of Jichi Medical University (JMU) is to educate students to become general practitioners competent in rural settings. Students are accepted per a quota system from each of Japan’s 47 prefectures. In the current curriculum at JMU, students complete lectures on almost every basic and clinical medicine area before the end of Year 3. From Year 4 to Year 6, students are permitted to participate in a clinical clerkship during which they receive training centred on taking patient histories and providing physical examination. Previously, most ward placements took place at the University’s affiliated hospital providing little opportunity for in-depth communication with role models in rural settings.

The community-based clinical clerkship (CBCC) was introduced in 1998 (Okayama & Kajii, 2011). For 2 weeks from late August to early September in Year 5, CBCC students stay at a community hospital or clinic in their home prefecture where JMU graduates work. Every year, two to five JMU graduates per prefecture are appointed to be CBCC mentors based on their motivation to teach in their prefectural communities. JMU faculty ensures the instructional quality of mentors by conducting annual face-to-face faculty development sessions. In addition, standards for learning activities are proposed, including ambulatory care, home care, hospital care, placement in mobile clinics, on-call work, rehabilitation, health education, health check-ups, vaccination, day services, and placement in welfare facilities (welfare institutions or nursing homes for the aged) (Okayama & Kajii, 2011).

Prior to the regular CBCC without PIF-SRL, students had several opportunities to communicate with mentors by telephone or e-mail. However, these communications did not provide sufficient opportunity for students to develop an image of future professionalism. We therefore felt that the communication framework in the regular CBCC did not fulfil its potential to stimulate PIF.

2) PIF-SRL for the CBCC: The PIF-SRL platform (Matsuyama et al., 2021) used an online communication platform, Google Forms. Before the CBCC began, mentors were briefed multiple times in writing and verbally on the purpose of the PIF-SRL to ensure their mentorship was PIF-oriented. They were also asked to read a manual which provided specific step-by-step responses from their first interaction with medical students on Google Forms in mid-July to their post-practice reflection in early to mid-September (Figure 1).

In the pre-clerkship phase, participants were asked to write their reflexive PIE. We used PIE because it is useful for helping learners articulate their own values and norms of medical professionalism, and for teachers providing feedback with rubrics based on Kegan’s constructive developmental theory (Kalet, Buckvar-Keltz, Harnik et al., 2017; Kalet, Buckvar-Keltz, Monson et al., 2018; Kegan, 1994). The present study used a Japanese version of the original English-language PIE form. Translation to Japanese was conducted by the main author (YM). To validate the translation accuracy, back-translation to English was conducted by a co-author (AJL), an American professor living in Japan who is literate in both English and Japanese. In accordance with in-depth talks based on PIE contents, students were encouraged to verbalise their future self-image and reflect on their current selves compared with their perceived future ones. The Question 6 of PART 2 in the PIF-SRL asked alumni mentors to describe their present self-image (Matsuyama et al., 2021). Students in PIF-SRL can also refer to this information when verbalising their future self-image.

Additionally, students in PIF-SRL were asked to create study plans for CBCC based on gaps between their current and future selves, and identify one or more learning goals. Referring to these plans, alumni mentors observed students and gave just-in-time feedback. Also, mentors provided students with learning strategies to help them overcome potential future challenges. Apart from these instructions, mentors were essentially independent in their education of the medical students. After the two-week clerkship, students were asked to re-articulate their own future image and received feedback from their mentors by Google Forms (Figure 1).

B. Subjects

First, PIF-SRL mentors were selected. In 2018 and 2019, 94 JMU graduates registered as attending rural physicians for the CBCC. Among them, 20 candidates for PIF-SRL mentors in 2018 and 2019 were randomly selected and informed consent for their contribution to PIF-SRL in this study was requested. Eventually, 17 and 13 JMU alumni agreed to participate in 2018 and 2019, respectively, 8 of whom participated in both years.

Independent of this study, the JMU Center for Community Medicine matched one alumnus with one to three students for the CBCC. The 30 PIF-SRL alumni-mentors were paired with 1 to 3 students each before informed consent was obtained. In this study, students paired with the 30 PIF-SRL alumni were chosen as candidates for the PIF-SRL group subjects. There were 22 and 20 candidates in 2018 and 2019, respectively. One candidate in 2019 declined participation. Eventually, 41 students were registered as subjects in the PIF-SRL group. Simultaneously, 41 control subjects were chosen from the same school year cohort and informed consent to participate was obtained. Control subjects experienced the regular CBCC mentorship without PIF-SRL. Because previous studies have shown that gender (Ray et al., 2003) and academic performance (Lucieer et al., 2016) might independently influence SRL development, participants in both groups were paired by gender and academic ranking from the previous year (Year 4).

C. Procedures

A convergent mixed method was chosen for the first research question ‘Does PIF-SRL improve SRL during the period around the clinical clerkship?’ to identify common data between quantitative and qualitative results (Creswell & Clark, 2017). Next, an explanatory mixed method was used to address the second research question ‘How does the PIF-oriented elements in PIF-SRL improve SRL?’. A rationale for this method is that follow-up qualitative approaches can explain quantitative results (Creswell & Clark, 2017). We conducted this mixed method study in the paradigm of pragmatism, which emphasises solutions to research questions and integrates qualitative and quantitative research results to obtain general findings (Shannon-Baker, 2016).

1) Quantitative approach: Learner SRL levels were measured with a Japanese-language version of the Motivated Strategies for Learning Questionnaire (MSLQ-J) before (mid-July) and after (mid-September) subjects participated in CBCC with or without PIF-SRL. The MSLQ (Pintrich et al., 1991) is composed of 81 items with seven-point Likert scales quantifying levels of 9 types of SRL strategy (rehearsal, elaboration, organisation, critical thinking, metacognitive self-regulation, time and study environment, effort regulation, peer learning, and help seeking), and 6 variables of motivation states (intrinsic goal orientation, extrinsic goal orientation, task value, control of learning beliefs, self-efficacy for learning and performance, and test anxiety). All 81 items of the MSLQ were used as done previously in a medical school context (Cho et al., 2017) because it was believed the 15 SRL-related categories could multi-dimensionally identify differences between the PIF-SRL group and control group. Translation to Japanese was conducted by the main author (YM) and back-translation to English by a co-author (AJL). For the MSLQ validation, the Cronbach alpha and McDonald omega were measured for 15 categories (Matsuyama et al., 2021).

2) Qualitative approach: To explore changes in motivation, strategies and reflective behaviours from self-reflection and study during the clinical clerkship, we created a questionnaire composed of seven questions (Matsuyama et al., 2021). All participants answered the questionnaire within two weeks after post-CBCC PIF-SRL mentoring. In late September 2019, one-on-one interviews were also conducted after intensive qualitative data analysis of the questionnaire from 41 participants in 2018 and 2019. Three interviewers familiar with the CBCC but not engaged in the assessment of Year 5 students conducted interviews in order to encourage interviewees to openly articulate their own perceptions. Twelve students in the 2019 PIF-SRL group consented to participate in interviews conducted in a semi-structured manner using an interview form with similar questions to those in the questionnaire (Matsuyama et al., 2021). The interviewers were instructed beforehand by the main author (YM) to obtain data about changes in perception regarding motivation, strategies and reflective behaviours after experiencing PIF-SRL. After collecting interview data from 10 students, the two main authors (YM and HO) found no additional meaningful codes emerging and, concluding that data saturation had been reached (Hennink et al., 2017), and stopped further interview data collection.

D. Analysis

1) Quantitative approach: The 15 MSLQ-J pre-intervention subcategory scores of the PIF-SRL group and control group were compared using Kruskal-Wallis one-way analysis of variance (ANOVA). After confirming that there were no statistically significant differences between the two groups, subtracted (post-pre) scores in the 15 MSLQ-J subcategories were compared between the two groups using Kruskal-Wallis one-way ANOVA. This non-parametric method was used because of considerable skewness in distribution in the scales of several items and outliers in other scales in MSLQ-J. A p-value < 0.05 was considered statistically significant. The effect sizes for comparisons were also calculated using ε² values, wherein small effect sizes ranged from 0.01 to <0.08, medium effect sizes ranged from 0.08 to <0.26 and large effect sizes ranged from ≥0.26. We used JAMOVI version 1.0.7.0 for statistical analysis.

2) Qualitative approach: Qualitative data from PIE, the questionnaire and interviews were analysed using thematic analysis. Anonymised qualitative data were analysed in accordance with the six phases proposed by Braun and Clarke (2006). Initial coding was conducted by the two Japanese researchers (YM and HO). YM, the lead author, was involved in the development of PIF-SRL and has previous experience with qualitative studies relevant to SRL. HO was not directly engaged in PIF-SRL but has had experience in qualitative studies relevant to SRL. The transcripts were thoroughly analysed using an inductive coding approach until agreement on coding was achieved through repetitive face-to-face meetings between the pair.

The focus was on changes in SRL (motivation, learning strategies, and reflective behaviours), and student opinion of the effects of PIF-SRL on SRL. Representative codes and statements were translated into English by an American professor literate in both English and Japanese (AJL). In the final phase, two other authors (JL and CV; education psychologists familiar with SRL) joined the discussion, and a higher-level synthesis of the codes was developed.

III. RESULTS

A. Quantitative Data

Mean averages, standard deviations, and median averages for fifteen MSLQ-J categories plus gender and academic rank data at pre-intervention are shown in Table 1. No categories significantly differed between the PIF-SRL and control groups.

The subtracted (post- minus pre-intervention) between-group scores in the 15 MSLQ-J categories are shown in Table 2. Improvements in 1. Intrinsic goal orientation and 10. Critical thinking were significantly better in the PIF-SRL group than the control group with ε² values 0.096 (p = .005) and 0.051 (p = .041), respectively. The quantitative data used in this study are accessible (Matsuyama et al., 2021).

A higher-level synthesis of the codes eventually resulted in three major themes corresponding to the second research question, ‘How does the PIF-oriented elements in PIF-SRL improve SRL?’ 

1) Active expansion of learning goals based on professional responsibility: The first theme consisted of subthemes which included learning motivated from responsibility, clear learning goals based on explicit self-images, and wider learning goals based on wider perceptions of professional roles.

Students viewed in-depth communication with mentors through the PIF-SRL platform with PIE helpful for imagining their future professional responsibilities in a positive and objective manner.

‘A lot of opening questions were, for example, what do you expect from work, what is the worst that can happen if you failed to live up to the expectations you have set for yourself, that’s the situation you’re working under when you’re a doctor, and the first time I really felt this was the time I really should be aware of this, it was positive, and actually although I was still just a student, I could objectively assess what I was thinking…’

(Interview, 1D-10)

During PIF-SRL mentoring, they were able to realise that knowledge beyond what they were currently learning in the curriculum was required of them as professionals. They were actively trying to set learning goals that they could connect to what they would need to learn in the future.

 

‘Knowing what skills and knowledge the region expects of you, you can create a working image of your future situation, and this becomes motivation to learn about new areas you weren’t aware of before.’

 (Questionnaire, 2019-19)

Aside from the expansion of perceived learning goals, students also began to see that self-study was required to ensure the authenticity of medical knowledge applicable to their future professional work. This was linked to Theme 3.

‘What I got was that incomplete understanding or recall wasn’t going to cut it when actually applying knowledge in the clinic. I began to train with the awareness that I wouldn’t be useful there if I didn’t understand and remember all I learned about disease.’

(Questionnaire, 2018-21)

2) Elaboration by linking future professional task processes to daily self-study contents: The second theme was comprised of subthemes which included focus on the task processes of role models and relating daily self-study content to future roles.

After PIF-SRL, students began to imagine what they would do as professionals in future medical practice at a task process-based level. Because their vivid future professional image helped them identify deep responsibilities for their professional tasks, they began to carefully observe mentors’ complete professional tasks in the clinical clerkship and articulate task processes. This in turn encouraged students to select learning strategies which linked information in daily self-study materials (e.g. textbooks) to professional task processes, which is referred to as ‘elaboration’ in MSLQ.

‘I started to learn in terms of what I would do if it were me. I also started to think about the details and not just the general flow of things, and to apply them as much as possible to reality.’

(Questionnaire, 2018-8)

‘Since the clerkship, I’ve been able to relate and recall what I actually encountered in the clinical clerkship, and when I was actually reading textbooks in self-study, I was able to align it with my future work content, and if there were things that applied, I would emphasise them… The study method that I use to impress upon myself was strengthened in the mentoring and clerkship…’

(Interview, 1D-3)

3) Critical thinking based on the pursuit of authentic medical knowledge: The third theme was comprised of subthemes which included realisation of the significance of authenticity pursuit, access to a wide variety of educational materials, and critical reflection.

Because they began to perceive that what they were learning would affect the lives of individual patients, they recognised the significance of gaining authentic knowledge that could be applied to each patient, differentiated from textbook-based universal knowledge.

‘It’s really important to learn about disease by studying the texts and gaining comprehensive knowledge, but since that tertiary material is insufficient for responding to clinical questions and individual patient backgrounds, I’m not sure that knowledge is useful in clinical practice. For that, what’s most meaningful are secondary materials like UpToDate, or if you still have doubts then primary material research papers.

(Questionnaire, 2019-1)

After beginning to pursue authenticity of medical knowledge, students found diversity and inconsistency in information from learning materials. To deal with this, they began to formulate inquiries focusing on self-study, a variety of information resources, and different viewpoints. Through this strategic shift, critical thinking emerged in an intrinsic manner.

‘I’ve come to think opinions are going to diverge to some extent as you get down to actually asking opinions of several different doctors, and these are choices you have to make, the stages you go through when studying.’

(Interview, 1C-7)

‘Having the ability to doubt, and because it’s science not taking everything at face value, becoming sceptical, I think that’s necessary.’

(Interview, 3C-14)

IV. DISCUSSION

Regarding the first research question ‘Does PIF-SRL improve SRL during the period around the clinical clerkship?’, our findings of a moderate improvement in intrinsic goal orientation and a mild improvement in critical thinking in the PIF-SRL group compared with the control group suggested that PIF-SRL promotes SRL. The qualitative analysis supports the MSLQ-J results. With regard to intrinsic goal orientation, students’ recognition of their future ‘professional responsibility’ was a key to fostering this. Further, recognition of professional responsibility led to critical thinking — critical evaluation of learning materials or their interpretation — as a means of pursuing authenticity of information for professional task processes. The consistency between quantitative and qualitative data was notable in understanding the outcomes of the PIF-oriented mentoring on SRL.

There was no statistical significance in the difference between the PIF-SRL and the control groups regarding elaboration in MSLQ-J data. However, qualitative analysis illuminated that PIF-SRL students’ attention to professional task processes triggered a strategy toward elaboration of knowledge in accordance with their future professional task processes. Reasons for this discrepancy might include the number of participants and sensitivity of the outcome measurement (Tonkin-Crine et al., 2016). Further research is needed to address these issues. However, we believe that in order to remediate learning strategies necessary for professional tasks or professionalism, it is useful to reflect on daily self-learning in accordance with the process of professional work shown by role models.

Recent research pertaining to educational intervention for SRL emphasises analysing the learning process of a particular task in detail and remediating the individual process (Artino et al., 2014; Durning et al., 2011; Gandomkar et al., 2016). While we generally concur with this, learning tasks that take place in clinical practice are limitless, so it would be beneficial to explore a foundational intervention applicable to a variety of tasks in clinical settings. The PIF-oriented mentoring in this study is presented as a foundational SRL intervention for a variety of clinical settings wherein learners can recognise professional identities and role models can suggest learning strategies.

The major strength of this study is that the subjects were Japanese students, who are considered to engage less in self-regulation than their Western counterparts (Iwata & Doi, 2017; Matsuyama et al., 2018; Tagawa, 2008). We believe that our study can provide educators with evidence that PIF-oriented educational schemes promote better learning behaviours in institutions desiring to promote undergraduate SRL. Another strength is that there were few formal classes or training programmes – i.e., intervening confounders — except for PIF-SRL and CBCCs between pre- and post-data collection (Figure 1). We believe the study schedule, without other educational interventions, optimised learning comparison outcomes between the two groups. For instance, changes in accessing learning materials for critical thinking (e.g. UpToDate) can be attributed mostly to the experiences in PIF-oriented mentoring and the subsequent CBCCs.

This study also has some limitations. First, we did not collect one-on-one interview data from 2018 cohort. Second, we did not examine student SRL changes from the mentor’s perspective, despite the fact that mentors’ perceptions of changes in students’ SRL would be as useful as, or more useful than students’ self-administered data. Third, reference to mentors’ self-images in the PIF-SRL could help students in PIF-SRL construct their professional self-image more clearly; however, this may cause bias student statements in the PIF-SRL form regardless of their real professionalism. Future research needs to measure PIF and SRL in more multifaceted and objective manners. Lastly, we do not have long-term outcomes of PIF-SRL. According to previous studies (Cruess et al., 2014; Cruess & Cruess, 2019; Kalet et al., 2017; Oyserman et al., 2017), identity is flexibly attuned to immediate situations rather than fixed in memory. Plus, frequently and fluently cued identities form stable ones. Accordingly, we speculate that the repetitive use of PIF-SRL could strengthen learners’ SRL. Validation of this notion would require a longitudinal cohort study.

Evaluation of these results suggests that the ability of PIF-SRL to work effectively requires that the quality of mentors be guaranteed. One reason for the success of the relatively random combination of students and mentors in this PIF-SRL group is that all are future rural physicians, and their mentors are also alumni of JMU in rural practice. On the other hand, for medical students from other universities who can follow diverse specialties and career paths, use of the PIF-SRL will require pairing medical students with appropriate mentors who can respond to students’ identities or future images. Also, it is important to pair students not only by their interest in future expertise, but also by looking at the mental and physical traits of individual students as they relate to motivation and student career choices (Henning et al., 2017). Moreover, in-depth conversations that would foster professionalism might not be done only through text messages in the PIF-SRL platform, but also through video calls and in-person meetings that would convey the participants’ voices, facial expressions, and mood. We think it is important for mentors to actively provide opportunities for direct dialogue with students. In light of this challenge, the present study supports PIF-oriented intervention as a method for improvement in SRL.

V. CONCLUSION

Allowing for these limitations and the need for further research, this study indicated that PIF-oriented education in a clinical clerkship with alumni mentors increased immediate intrinsic goal orientation and promoted a shift to SRL. Their SRL was characterised as task process-based elaboration, with critical thinking emerging from the pursuit of authenticity in medical practice.

Notes on Contributors

Yasushi Matsuyama reviewed the literature, designed the study, conducted both quantitative and qualitative data analysis, and wrote the manuscript.

Hitoaki Okazaki conducted qualitative data analysis.

Kazuhiko Kotani designed the study, and collected both quantitative and qualitative data.

Yoshikazu Asada collected both quantitative and qualitative data.

Shizukiyo Ishikawa collected both quantitative and qualitative data.

Adam Jon Lebowitz contributed to Japanese-English translation of data collection tools and qualitative data from questionnaire and interviews.

Jimmie Leppink reviewed the literature, designed the study, conducted both quantitative and qualitative data analysis, and wrote the manuscript.

Cees van der Vleuten reviewed the literature, designed the study, conducted both quantitative and qualitative data analysis, and wrote the manuscript.

All the authors have read and approved the final manuscript.

Ethical Approval

This study was approved by the ethics committee of Jichi Medical University (Reference number: 19-001). Consent was obtained from all participants for the research study.

Acknowledgements

We would like to express our sincere gratitude to Dr. Adina Kalet and Dr. Verna Monson for their consultation and provision of supplementary materials. We would also like to thank Drs. Yasuko Aoyama, Yoshitaka Maeda, and Maiko Watanabe for their support in conducting one-on-one interviews. We would also like to thank Ms. Akemi Watanabe and Yasuko Koguchi for their helpful assistance. 

Funding

This work was supported by JSPS KAKENHI [Grant number JP17K08924]. 

Declaration of Interest

The authors report no conflicts of interest.

Data availability

The quantitative data used in this study, Supplemental files are available at https://doi.org/10.6084/m9.figshare.14312507

Representative qualitative data translated into English are shown in the Result section (Matsuyama et al., 2021). All qualitative data written in Japanese are available from the corresponding author on reasonable request.

A preprint of the previous version of our manuscript, which is not peer-reviewed, is available at https://www.researchsquare.com/article/rs-12667/v1

References

Aho, J. M., Ruparel, R. K., Graham, E., Zendejas-Mummert, B., Heller, S. F., Farley, D. R., & Bingener, J. (2015). Mentor-guided self-directed learning affects resident practice. Journal of Surgical Education, 72(4), 674–679. https://doi.org/10.1016/j.jsurg.2015.01.008o

Artino, A. R., Jr., Cleary, T. J., Dong, T., Hemmer, P. A., & Durning, S. J. (2014). Exploring clinical reasoning in novices: A self-regulated learning microanalytic assessment approach. Medical Education, 48(3), 280–291. https://doi.org/10.1111/medu.12303

Artino, A. R., Jr., Dong, T., DeZee, K. J., Gilliland, W. R., Waechter, D. M., Cruess, D., & Durning, S. J. (2012). Achievement goal structures and self-regulated learning: Relationships and changes in medical school. Academic Medicine, 87(10), 1375–1381. https://doi.org/10.1097/ACM.0b013e3182676b55

Berkhout, J. J., Helmich, E., Teunissen, P. W., van den Berg, J. W., van der Vleuten, C. P., & Jaarsma, A. D. (2015). Exploring the factors influencing clinical students’ self-regulated learning. Medical Education, 49(6), 589–600. https://doi.org/10.1111/medu.12671

Berkhout, J. J., Helmich, E., Teunissen, P. W., van der Vleuten, C., & Jaarsma, A. (2018). Context matters when striving to promote active and lifelong learning in medical education. Medical Education, 52(1), 34–44. https://doi.org/10.1111/medu.13463

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brydges, R., & Butler, D. (2012). A reflective analysis of medical education research on self-regulation in learning and practice. Medical Education, 46(1), 71–79. https://doi.org/10.1111/j.1365-2923.2011.04100.x

Cho, K. K., Marjadi, B., Langendyk, V., & Hu, W. (2017). Medical student changes in self-regulated learning during the transition to the clinical environment. BMC Medical Education, 17(1), 59. https://doi.org/10.1186/s12909-017-0902-7

Cleary, T. J., & Sandars, J. (2011). Assessing self-regulatory processes during clinical skill performance: A pilot study. Medical Teacher, 33(7), e368–e374. https://doi.org/10.3109/0142159X.2011.577464

Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. SAGE Publications.

Cruess, R. L., & Cruess, S. R. (1997). Teaching medicine as a profession in the service of healing. Academic Medicine, 72(11), 941–952. https://doi.org/10.1097/00001888-199711000-00009

Cruess, R. L., Cruess, S. R., Boudreau, J. D., Snell, L., & Steinert, Y. (2014). Reframing medical education to support professional identity formation. Academic Medicine, 89(11), 1446–1451. https://doi.org/10.1097/ACM.0000000000000427

Cruess, S. R., & Cruess, R. L. (2019). The development of professional identity. In T. Swanwick., K. Forrest., B. C. O’Brein (Eds.), Understanding medical education; Evidence, theory, and practice (3^rd ed., pp. 239–254). https://doi.org/10.1002/9781119373780.ch17

Durning, S. J., Cleary, T. J., Sandars, J., Hemmer, P., Kokotailo, P., & Artino, A. R. (2011). Perspective: Viewing “strugglers” through a different lens: How a self-regulated learning perspective can help medical educators with assessment and remediation. Academic Medicine, 86(4), 488–495. https://doi.org/10.1097/ACM.0b013e31820dc384

Frank, J. R. (2005). The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care. The Royal College of Physicians and Surgeons of Canada.

Gandomkar, R., Mirzazadeh, A., Jalili, M., Yazdani, K., Fata, L., & Sandars, J. (2016). Self-regulated learning processes of medical students during an academic learning task. Medical Education, 50(10), 1065–1074. https://doi.org/10.1111/medu.12975

Henning, M. A., Krägeloh, C. U., Booth, R., Hill, E. M., Chen, J., & Webster, C. S. (2017). Profiling potential medical students and exploring determinants of career choice. The Asia Pacific Scholar, 2(1), 7-15. https://doi.org/10.29060/TAPS.2017-2-1/OA1019

Hennink, M. M., Kaiser, B. N., & Marconi, V. C. (2017). Code saturation versus meaning saturation: How many interviews are enough? Qualitative Health Research, 27(4), 591–608. https://doi.org/10.1177/1049732316665344

Iwata, K., & Doi, A. (2017). Can hybrid educational activities of team and problem based learning program be effective for Japanese medical students? International Journal of Medical Education, 8, 176–178. https://doi.org/10.5116/ijme.591b.2bc8

Jarvis-Selinger, S., Pratt, D. D., & Regehr, G. (2012). Competency is not enough: Integrating identity formation into the medical education discourse. Academic Medicine, 87(9), 1185–1190. https://doi.org/10.1097/ACM.0b013e3182604968

Kalet, A., Buckvar-Keltz, L., Harnik, V., Monson, V., Hubbard, S., Crowe, R., Song, H. S., & Yingling, S. (2017). Measuring professional identity formation early in medical school. Medical Teacher, 39(3), 255–261. https://doi.org/10.1080/0142159X.2017.1270437

Kalet, A., Buckvar-Keltz, L., Monson, V., Harnik, V., Hubbard, S., Crowe, R., Ark, T. K., Hyuksoon, Song, H. S., & Yingling, S. (2018). Professional Identity Formation in medical school: One measure reflects changes during pre-clerkship training. MedEdPublish, 7(1), 41. https://doi.org/10.15694/mep.2018.0000041.1

Kegan, R. (1994). In over our heads: The mental demands of modern life. Harvard University Press.

Lam, T. P., & Lam, Y. Y. (2009). Medical education reform: The Asian experience. Academic Medicine, 84(9), 1313–1317. https://doi.org/10.1097/ACM.0b013e3181b18189

Lucieer, S. M., Jonker, L., Visscher, C., Rikers, R. M. J. P., & Themmen, A. P. N. (2016). Self-regulated learning and academic performance in medical education. Medical Teacher, 38(6), 585–593. https://doi.org/10.3109/0142159X.2015.1073240

Matsuyama, Y., Nakaya, M., Okazaki, H., Leppink, J., & van der Vleuten, C. (2018). Contextual attributes promote or hinder self-regulated learning: A qualitative study contrasting rural physicians with undergraduate learners in Japan. Medical Teacher, 40(3), 285–295. https://doi.org/10.1080/0142159X.2017.1406074

Matsuyama, Y., Nakaya, M., Okazaki, H., Lebowitz, A. J., Leppink, J., & van der Vleuten, C. (2019). Does changing from a teacher-centered to a learner-centered context promote self-regulated learning: A qualitative study in a Japanese undergraduate setting. BMC Medical Education, 19(1), 152. https://doi.org/10.1186/s12909-019-1550-x

Matsuyama, Y., Okazaki, H., Kotani, K., Asada, Y., Ishikawa, S., Lebowitz, A. J., Leppink, J., & van der Vleuten, C. (2021). Professional identity formation-oriented mentoring technique as a method to improve self-regulated learning: A mixed-method study. [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.14312507

Matsuyama, Y., Okazaki, H., Kotani, K., Asada, Y., Ishikawa, S., Lebowitz, A. J., Leppink, J., & van der Vleuten, C. (2020). Education in professional identity formation enhances self-regulated learning: A mixed-method exploratory study from a community-based clinical clerkship in Japan. [Preprint]. Research Square. January 28, 2020. [cited June 10, 2020] Available from: https://doi.org/10.21203/rs.2.22088/v1

Okayama, M., & Kajii, E. (2011). Does community-based education increase students’ motivation to practice community health care? A cross sectional study. BMC Medical Education, 11(1), 19. https://doi.org/10.1186/1472-6920-11-19

Oyserman, D., Lewis, Jr., N. A., Yan, V. X., Fisher, O., O’Donnell, S. C., & Horowitz, E. (2017). An identity-based motivation framework for self-regulation. Psychological Inquiry, 28(2-3), 139-147. http://doi.org/10.1080/1047840X.2017.1337406

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (Technical Report 91-B-004). The Regents of the University of Michigan. https://files.eric.ed.gov/fulltext/ED338122.pdf

Ray, M. W., Garavalia, L. S., & Gredler, M. E. (2003, April 21-25). Gender differences in self-regulated learning, task value, and achievement in developmental college students. Annual Meeting of the American Educational Research Association. https://files.eric.ed.gov/fulltext/ED476176.pdf

Sandars, J., & Cleary, T. J. (2011). Self-regulation theory: Applications to medical education: AMEE Guide No. 58. Medical Teacher, 33(11), 875–886. https://doi.org/10.3109/0142159X.2011.595434

Shannon-Baker, P. (2016). Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research, 10(4), 319-334. https://doi.org/10.1177/1558689815575861

Song, H. S., Kalet, A. L., & Plass, J. L. (2011). Assessing medical students’ self-regulation as aptitude in computer-based learning. Advances in Health Sciences Education: Theory and Practice, 16(1), 97–107. https://doi.org/10.1007/s10459-010-9248-1

Stuart, E., Sectish, T. C., & Huffman, L. C. (2005). Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambulatory Pediatrics, 5(5), 298–301. https://doi.org/10.1367/A04-091R.1

Tagawa, M. (2008). Physician self-directed learning and education. The Kaohsiung Journal of Medical Sciences, 24(7), 380–385. https://doi.org/10.1016/S1607-551X(08)70136-0

Tonkin-Crine, S., Anthierens, S., Hood, K., Yardley, L., Cals, J. W., Francis, N. A., Coenen, S., van der Velden, A. W., Godycki-Cwirko, M., Llor, C., Butler, C. C., Verheij, T. J., Goossens, H., Little, P., & GRACE INTRO/CHAMP consortium (2016). Discrepancies between qualitative and quantitative evaluation of randomised controlled trial results: Achieving clarity through mixed methods triangulation. Implementation Science, 11, 66. https://doi.org/10.1186/s13012-016-0436-0

Turan, S., & Konan, A. (2012). Self-regulated learning strategies used in surgical clerkship and the relationship with clinical achievement. Journal of Surgical Education, 69(2), 218–225. https://doi.org/10.1016/j.jsurg.2011.09.003

van Houten-Schat, M. A., Berkhout, J. J., van Dijk, N., Endedijk, M. D., Jaarsma, A. D. C., & Diemers, A. D. (2018). Self-regulated learning in the clinical context: A systematic review. Medical Education, 52(10), 1008–1015. https://doi.org/10.1111/medu.13615

Van Nguyen, H., Laohasiriwong, W., Saengsuwan, J., Thinkhamrop, B., & Wright, P. (2015). The relationships between the use of self-regulated learning strategies and depression among medical students: An accelerated prospective cohort study. Psychology, Health & Medicine, 20(1), 59–70. https://doi.org/10.1080/13548506.2014.894640

Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329.

*Yasushi Matsuyama
3311-1
Yakushiji, Shimotsuke,
Tochigi, 329-0498
Email: yasushim@jichi.ac.jp

Submitted: 2 November 2020
Accepted: 8 February 2021
Published online: 5 October, TAPS 2021, 6(4), 37-48
https://doi.org/10.29060/TAPS.2021-6-4/OA2425

Stephen Bradley¹, Aaron Ooi², Kerry Stafford³, Shuvayon Mukherjee¹ & Marcus A. Henning⁴

¹Department of Paediatrics, Lakes District Health Board, New Zealand; ²Department of Paediatrics, Waikato District Health Board, New Zealand; ³Department of Paediatrics, Christchurch Hospital, New Zealand; ⁴Centre for Medical and Health Sciences Education, University of Auckland, New Zealand

Abstract

Introduction: The paediatric team handover process is a crucial workplace practice and comprises the transfer of patient information from one shift to another involving medical professionals and students.  A qualitative study was performed to analyse the feasibility, functionality, benefits and limitations of the dramaturgical approach when applied to examining a handover session. 

Methods: Data relating to one handover were collected and analysed from video and audio recordings, notes created by two independent observers and a de-identified copy of the handover sheet. 

Results: The dramaturgical constructs and subsequent findings allowed us to make informed inferences about the dynamics of the handover procedure. The directors/lead actors consisted of a consultant and a registrar.  One consultant was transitory and the remaining 12 attendees were either major support, support or bit actors.  The students (bit actors/audience) were included when a learning point was emphasised.  The script was informal and improvised as the discussion emphasised certain facets of patient care or accentuated learning points.  The staging involved the seating arrangement, a whiteboard, computer screen and ongoing data presentation.  The performance suggested a handover of two halves: one emphasising learning and the other allocation of patient care responsibility. 

Conclusion: We concluded that the real-life drama occurring within a handover was feasibly analysed, with its functionality demonstrated, using the dramaturgical investigative system.  The multifaceted recordings enabled researchers to review the ‘authentic’ handover system without censorship. These findings have implications for educational and organisational research.

Keywords:           Dramaturgical, Handover, Paediatric, Methodology

Practice Highlights

• Dramaturgical methodology provided a unique, authentic and detailed analysis of the handover.
• The dramaturgical research methodology used to evaluate the handover was feasible and functional.
• This research methodology can be used to analyse education within similar team based settings.
• This research methodology can be applied to the team handovers and other complex health meetings.
• This research methodology identifies important clinical/educational roles and dynamics within teams.

I. INTRODUCTION

Hospital team handovers involve effective transfer of information and responsibility from one health professional to another, ensuring continuity of patient care (Australian Medical Association Limited, 2006; Hilligoss & Cohen, 2011). The level of communication needs to be comprehensive, unambiguous and coherent so that patient information is easily understood, thus optimising patient care through the meaningful and efficient transfer of patient information (Fujikawa et al., 2021). This is crucial given the ramifications for optimising patient care and minimising potential treatment error, including miscued transfer of knowledge, insertion of faulty or misleading information, treatment delay, and poor patient outcomes (Arora et al., 2005; Bomba & Prakash, 2005). To explore the nuances occurring in handover practice from organisational behaviour and educational perspectives, different methodological approaches need to be developed.

In this paper, we propose that the dramaturgical approach can optimally analyse handover dynamics, as it is an integrated, pragmatic and multidimensional approach. This approach uses multi-source feedback from video and audio recordings, observer records, and transcripts of analysis. The dramaturgical approach argues that the individuals present in the activity assume different roles that influence the way they communicate and behave (Canary et al., 2008; Goffman, 1959; Henderson, 2005). Using this approach, the handover activity can be analysed much like a drama or element of theatre. The dramaturgical approach has the potential to offer insights into the clinical and educational handover components, much like the insights drawn when applying this approach to analysing healthcare simulations (Crea, 2017), decision-making aspects of an emergency department triage (Forde, 2014), and behaviour of radiographers and their patients (Murphy, 2009). This analytical approach enables the researcher to be present at the moment of the occurrence, rather than relying on retrospective data obtained when subsequently interviewing participants. Interviews can be a powerful means of obtaining information, but require participants to accurately retell their experiences of the activity (DiCicco‐Bloom & Crabtree, 2006).  The dramaturgical approach tells it for what it is, and allows researchers the ability to see and hear the authentic process of communication (Goffman, 1959; Murphy, 2009). We surmised that the dramaturgical approach would be a more comprehensive evaluative system and thus well suited for collecting observational data that could inform training and development initiatives within hospital systems.

The research aim of this study was to explore the feasibility and functionality of the dramaturgical methodological system of analysis not yet applied to the handover procedure. 

The research questions driving this study include:

1. How can the dramaturgical approach feasibly be applied to the handover system?
2. How does the dramaturgical approach describe the functional dynamics of the handover procedure?
3. What are the benefits and limitations associated with applying this research methodology?

 

II. METHODS

A. Phenomenon of Interest

Feasibility, functionality and quality were informed by examples taken from one handover involving team discussion regarding patients admitted to a general paediatric ward (20 beds) and a Special Care Baby Unit (8 cots) in a New Zealand secondary-level hospital (Hensher et al., 2006).

B. Sample/Participants

All the health professionals and medical students involved in one handover were asked to volunteer for the study, with eligibility determined by consent and approval from hospital management. Informed consent was obtained after eligible participants read a detailed information sheet, provided by an administrator, followed by a consent form that they then signed. 

C. Data Collection

Data were obtained from several sources.

1. Five video cameras were situated in the handover room to obtain multiple angles of the handover. Two audio recorders were placed in the room and served as the primary sources of data for transcription.
2. The final transcription of events was checked by all authors using data from the cameras.
3. Notes on the salient aspects of handover interactions were made by two present ‘unknown’ observers (i.e., one medical student and one medical educationalist).

A diagram of the seating positions of each participant was constructed (see Figure 1).

Figure 1: Handover room layout depicting seating arrangements, participants (P1-P15, with original position participants sat in), 2 observers (Ob1 and Ob2) and equipment.

Submitted: 17 October 2020
Accepted: 12 April 2021
Published online: 5 October, TAPS 2021, 6(4), 26-36
https://doi.org/10.29060/TAPS.2021-6-4/OA2420

Chee Yang Chin¹, Si Qi Tan², Swee Leng Kui^1,2, Kurugulasigamoney Gunasegaran¹ & Jill Cheng Sim Lee³

¹Department of Cardiovascular Medicine, National Heart Centre Singapore, Singapore; ²Ministry of Health Holdings, Singapore; ³Department of Obstetrics and Gynaecology, KK Women’s and Children’s Hospital, Singapore

Abstract

Introduction: Sleep deprivation impacts clinical performance. However, literature is conflicting, with insufficient focus on patient outcomes. The aim of this study was to assess if patient satisfaction and prescription errors in outpatient clinics were adversely affected when consulting post-call versus non-post-call registrars.

Methods: This prospective, quantitative study was set in a large teaching hospital in Singapore. Between November 2015 and February 2016, patients from clinics run by a registrar after 24-hour shift were recruited to post-call group. Patients from non-post-call clinics run by the same registrar were controls. Outcome measures were patient satisfaction, using 5-item 4-point Likert scale questionnaire, and prescribing error rate, defined as number of errors over number of orders. Differences were analysed using chi-squared test.

Results: 103 of 106 (97%) patients in 9 post-call clinics and 93 of 105 (90%) patients in 9 non-post-call clinics were recruited. Questionnaire completion rate was 99%. 536 and 526 prescriptions were ordered in post-call and non-post-call groups, respectively. Percentage of top-box responses (greatest satisfaction) was higher in post-call group overall (79.3% versus 62.4%, p<0.001), and for each questionnaire item. There was no significant difference in prescribing errors (1.31% versus 2.28%, p=0.23).

Conclusion: Patient satisfaction and prescribing error rates in outpatient clinics were not detrimentally affected. This provides some objective evidence that patients may safely consult post-call registrars. True impacts of sleep deprivation remain poorly understood, and larger, longer term, multicentre studies would inform generalisability. Qualitative studies of fatigue may shed light on complex interactions of emotions that compensate for tiredness.

Practice Highlights

• Studies of sleep deprivation and work-hour restrictions are conflicting, with few on patient outcome.
• We compared patient satisfaction and prescribing errors of clinics post-call, versus non-post-call.
• We demonstrate quantitative evidence that patients may safely consult post-call registrars in clinic.
• Impairment may be mitigated by adaptive effort to be conscientious and post call euphoria.
• No recommendations were made to change the current practice of registrars running post-call clinics.

I. INTRODUCTION

Sleep deprivation and fatigue are thought to have detrimental effects on trainee doctors’ performance. It was previously shown that the Epworth Sleepiness Scale score in junior doctors was equivalent to that of patients with sleep apnoea and narcolepsy (Mustafa et al., 2005). Indeed, cognitive psychomotor performance after 24 hours of sustained wakefulness was found to be equivalent to a blood alcohol concentration twice the prohibited level of alcohol intoxication (Dawson & Reid, 1997). The performance and safety to practise of fatigued trainee doctors, especially at the end of a long shift, is thus of utmost concern.

Increasing awareness of fatigue-related medical errors has led to reforms in trainee doctor working hours. Work hours were limited by the Accreditation Council for Graduate Medical Education (ACGME) in the USA in 2003 to 80 hours per week, and by the European Working Time Directive (EWTD) in 2009 to 48 hours per week. A subsequent ACGME stipulation in 2017 further restricted work periods to 24 consecutive hours for first‐year residents (Rosenbaum & Lamas, 2012). Still, doubts remain as to the clinical impact of these restrictions (Baldwin et al., 2011; Fletcher et al., 2005). A systematic review of ACGME’s 2011 work hour restrictions found no improvements in patient care or resident well-being, and in fact revealed unintended negative impacts on resident education (Bolster & Rourke, 2015). Indeed, low job satisfaction was found to be associated with a higher rate of attrition from training (Lee et al., 2019). Thus, the optimal working hours and working patterns for trainee doctors remain ill-defined.

Studies on the impact of sleep deprivation and fatigue on trainee doctors’ performance and wellbeing have yielded mixed results. Chronic sleep deprivation was shown to be particularly detrimental to tasks requiring sustained concentration (Bertram, 1988). After an extended night shift, intensive care unit physicians made 36% more serious medical errors (Landrigan et al., 2004), were 61% more likely to suffer a percutaneous injury (Ayas et al., 2006) and 2.3 times more likely to have a motor vehicle collision (Barger et al., 2005). Hostility and anger in trainee doctors were also more prevalent after a night’s sleep loss (Saadat et al., 2016; Samkoff & Jacques, 1991). Cardiologists were particularly prone to sleep deprivation due to frequent call coverage responsibilities, and in a recent 2019 survey of 481 cardiologists, 46% felt work performance hindrance and 8.6% described sleep-deprivation related adverse events (Lobo et al., 2019).

Yet, other studies have not observed a similar impairment of ability amongst fatigued trainee doctors. There were fewer errors in cognitive tests and cardiopulmonary resuscitation (CPR) manoeuvres in residents after on-call duty (Domínguez et al., 2009; Hamui-Sutton et al., 2013), and psychomotor performance and acquisition of laparoscopic and robotic simulator skills in post-call trainee doctors were also found to be no different from non-post-call trainee doctors (Robison et al., 2018; Yi et al., 2013).

Notably, the outcome measures in the vast majority of these published studies were either subjective through interviews with the trainee doctors themselves, which introduces bias in results, or based on performance on robotic surgical simulators, which is not reflective of true patient contact. Few studies focused on actual patient outcome measures.

Trainee doctors (“registrars”) at our institution, a high volume tertiary referral centre for cardiovascular medicine in Singapore, may be expected to run busy outpatient clinics after a 24-hour continuous in-hospital shift. Typically, up to 20 patients are seen within 4 hours in these clinics. Concerns linger regarding the suitability of post-call registrars running these high-intensity clinics, both in terms of psychomotor ability as well as proneness to error.

With the lack of published literature on the impact of fatigue on trainee doctor performance in outpatient clinics, we aimed to study for ourselves whether patient outcomes were adversely affected if they consulted a post-call registrar, versus a non-post-call registrar. To achieve this, we decided on two practical and measurable outcomes: patient satisfaction and prescribing error rates. Quantifying patient satisfaction allowed an assessment of the registrars’ ability to communicate and show empathy despite their fatigued state. Prescribing error rates allowed an objective evaluation of patient safety. Using two very different outcome measures was a means to allow some triangulation of results and to reduce bias.

Our hypothesis was that both patient outcome measures – patient satisfaction and prescribing error rate – would be worse off in the clinics run by registrars when they were post-call versus when they were not post-call.

II. METHODS

A. Study Setting and Population Design

This study was set in a high volume, specialised Cardiology clinic in a large teaching hospital in Singapore. Registrars, who are trainee doctors in the final stages of training prior to specialist accreditation, work overnight in-hospital shifts as part of service and training commitments. “On-call” duties last from 7am to 1pm the following day (30 continuous hours), during which they are responsible for all urgent inpatient and Emergency department Cardiology referrals. “Post-call” refers to the last 6 hours of the continuous 30-hour shift, during which they may be posted to run busy outpatient clinics.

B. Patient Recruitment

Between November 2015 and February 2016, all patients who attended clinics run by a post-call registrar were recruited to the post-call group. Patients who attended clinics run by the same registrar on a non-post-call day were recruited as controls into the non-post-call group. This provided internal control by comparing performances by the same registrar post-call and non-post call. Patients were approached for participation after their clinic consultation so as not to bias the consultation. They were informed of the purpose of the study but not informed whether the registrar they had just consulted was post-call or not, so as not to bias their responses. All patients, whether new or established, were approached.

C. Patient Satisfaction Questionnaire

Patient satisfaction was evaluated using a 5 item questionnaire (Appendix A). These questions were adapted from the Hospital Consumer Assessment of Healthcare Providers And Systems (HCAHPS), a well-established standardised instrument for the measurement of patient perspectives on hospital care (Giordano et al., 2009; Goldstein et al., 2005).

1) During today’s consultation, did the doctor treat you with courtesy and respect?

2) During today’s consultation, did you feel the doctor listened carefully to you?

3) During today’s consultation, were you given the opportunity to ask your questions?

4) During today’s consultation, did the doctor explain things in a way you could understand?

5) How would you rate your overall satisfaction with today’s consultation?

For each item, as per the HCAHPS questionnaire format, patients were asked to rate responses on a 4-point Likert scale. Smiley faces were incorporated to provide visual representations. The questionnaire was in English and Chinese text, which would have been understood by the vast majority of the patient population. Relevant translations in Chinese, Malay or Tamil languages were provided by research assistants for illiterate patients or those who required further clarification, with care not to influence the patients’ responses. Only fully completed questionnaires were included in the data analysis.

D. Prescribing Error Rates

All prescription orders entered electronically were logged. As is standard procedure for our outpatient clinics, all prescriptions are reviewed by the Pharmacy Department prior to dispensing. All suspected prescription errors will be verified via a phone call to the prescribing doctor, who will confirm whether that prescription was intended or if it truly was an error. All verified prescription errors, including medication name, dose or frequency, or omission of a necessary drug, were recorded regardless of severity. The Pharmacy Department conducts regular internal audits to ensure that this process is accurate and consistent.

The total number of medications ordered and total number of errors were retrospectively summed. Prescribing error rate was calculated by dividing total number of prescribing errors by total number of medications ordered.

E. Consent and Ethics

In accordance with local Institutional Review Board (IRB) guidelines, the study protocol was exempted from full IRB review as it was non-interventional and intended primarily for medical education. All recruited patients provided informed consent to participate. All data were anonymised, with no personal identifiers recorded.

F. Data Analysis

For the patient satisfaction questionnaire, the “top-box” response was defined as the most positive response on the Likert scale (for example “all of the time” or “very satisfied”). The proportions of “top-box” responses were calculated and reported for each questionnaire item and overall items. Only “top-box” responses were sought, to target a higher quality of performance.

For both patient satisfaction and prescribing error outcomes, comparisons between post-call and non-post-call groups were tested for statistical significance with p<0.05 using the chi-squared test. All statistical analyses were reviewed by a biostatistician.

III. RESULTS

The recruitment period consisted of 9 outpatient clinics run by 8 post-call registrars. Two of these clinics were run by the same registrar who was post-call on separate days. We then identified a further 9 clinics run by the same registrars during days when they were not post-call. Thus, 7 registrars had one post-call clinic and one non-post-call clinic studied, while 1 registrar had two post-call clinics and two non-post-call clinics studied.

All patients seen were invited to participate. 103 of 106 (97%) patients in the post-call group and 93 of 105 (90%) in the non-post-call group consented. The number of patients seen per clinic ranged from 7 to 20 (median 11) in the post-call group, and 7 to 17 (median 10) in the non-post-call group.

Basic demographic data on the registrars and the distribution of patients in these clinics were summarised in Table 1.

Table 1: Basic demographic data of registrars and distribution of patients seen and number of medications ordered per clinic.

Note: ^{[1] †}Numbers in brackets indicate number of patients who declined participation. *C1 and C2 refer to the same registrar, who had 2 post-call and 2 non-post-call clinics from whom patients were recruited for the study.

Figure 1: Summary of patient questionnaire responses according to registrars’ post-call status. Questionnaire items were shown on the y-axis and percentage of top-box responses were shown on the x-axis.

B. Prescribing Error Rates

536 and 526 medications were ordered in the post-call and non-post-call group respectively. Seven prescribing errors were made in the post-call group and 12 in the non-post-call group (Table 2).

Overall, there was no significant difference in prescribing error rates (1.31% versus 2.28%, p=0.231).

Table 2: Distribution of medications ordered and prescribing errors made according to registrar.

IV. DISCUSSION

To our knowledge, this is one of very few studies on post-call trainee doctor performance in the context of a realistic outpatient setting, centred on patient outcomes. In our study, the results of both patient satisfaction and prescription errors were congruent in demonstrating that these measurable patient outcomes were not adversely affected when patients were seen by a post-call registrar in an outpatient clinic setting.

A. Patient Satisfaction

Patient satisfaction is an important measure as it demonstrates a doctor’s ability to communicate, which is a key element of any patient-doctor encounter. Fatigue leading to increased stress levels and hostility could have negatively influenced this.

In our study, rather than create a novel and unvalidated questionnaire, we adapted the HCAHPS, a well-established standardised instrument for the measurement of patient perspectives on hospital care (Giordano et al., 2009; Goldstein et al., 2005). As per HCAHPS, only the proportions of top-box responses (i.e., the percentage of patients who rated a particular questionnaire item the most positively) were taken into account in the analysis. This was a means of assessing and aiming for the highest quality of care possible.

The results from overall and individual item top-box responses in our questionnaire demonstrated that patient satisfaction was statistically significantly (p-value <0.001) higher in the post-call group. Indeed, 6 out of 8 registrars had higher percentages of overall top-box responses when post-call, adding consistency to this finding. Nonetheless, given the small cohort of registrars surveyed, there were still the likelihood that this was a chance finding, and previous studies in this area had shown only subtle differences in patient satisfaction (Hoellein et al., 2004; Liu & Wissow, 2011). The key take away point was that patient satisfaction in our study was not adversely affected when patients saw a post-call registrar.

There were two other studies examining post-call trainee doctors in outpatient clinics. In Liu and Wissow’s study (2011), 170 pre-recorded paediatric clinic consultations were analysed to reveal that parents were twice as likely to request a post-call doctor to repeat explanations, both post-call doctors and parents used paraphrasing more frequently to check that they understood each other, and trainee doctors were perceived as less willing to help the patient accomplish their goals during the visit. This suggested that communication was less effective when trainee doctors were post-call. The earlier study by Hoellein et al. (2004), using a 7-item 10-point Likert scale questionnaire, found that patients were significantly less satisfied (mean score 8.99) when seen by a post-call trainee doctor than when seen by a non-post-call trainee doctor (mean score 9.31). Differences in study design and clinic setup could explain the discrepancy between our findings and those of these 2 previous studies. In these 2 studies, patient volume was low at 3 to 7 per clinic, trainee doctors and patients were aware that they were being recorded, and some clinics surveyed were the trainee doctors’ “continuity clinics” where visits were more likely returning patients with already established rapport. In our clinic setting, patient volumes were higher, patients were only approached after the consultation, and none of the clinics were “continuity clinics”.

Another possible explanation for the lack of reduction in patient satisfaction in our study was the phenomenon of “post-call euphoria”, where trainee doctors experience an upbeat mood associated with being able to look forward to the end of a long 30-hour shift and finishing work earlier than usual, at 1 p.m. instead of at 6 p.m. They may be emotionally less stressed in clinic than if they had a long day of work ahead, particularly if they were just starting an on-call shift. In Liu and Wissow’s study (2011), trainee doctors reported more anxiety and frustration when running morning clinics when they still had the rest of the day’s work ahead of them. Additionally, they described themselves as tired, but not more unhappy, when they were post-call as opposed to when they had left on time the day before. Furthermore, post-call trainee doctors made attempts to cope with their fatigue, adaptively or maladaptively, and were seemingly able to compensate for compromised abilities. This counter-intuitive psychological effect warrants further study.

B. Prescribing Error Rates

Prescribing errors were evaluated as a marker of patient safety. Errors have been shown to be provoked by situations of high workload, stress and fatigue (Keers et al., 2013; Tully et al., 2009). Overall case volume in particular was associated with increased rate of major diagnostic discrepancies for junior residents (Hanna et al., 2016), which is relevant to our high-volume clinic setting. The questionnaire among anaesthesiologists from Santa Catarina by Erdmann et al. (2016) revealed that most respondents committed more than one error in drug administration, with distraction and fatigue being the highest contributing factor of 64.9%. Interns made substantially more serious medical errors in the intensive care unit when they worked frequent shifts of 24 hours or more, than when they worked shorter shifts (Landrigan et al., 2004). There was only one other study that dealt specifically with medication errors in the post-call setting. In a retrospective study of 8,195 inpatient prescriptions, trainee doctors were found to be significantly more likely to commit an error on-call and post-call, than when they were off-call, by 2.16%, with odds ratio 1.44 (Hendey et al., 2005).

Our study in the outpatient setting revealed no statistically significant difference in prescribing error rates between patients who had seen a post-call registrar (1.31%) and those who had seen a non-post-call registrar (2.28%) across 536 and 526 prescriptions respectively. The overall prescribing error rate seen in our study was comparable to that of Hendey et al. (2005) (1.79% versus 2.16%, respectively), supporting the validity of our results. Importantly, prescribing error rates were low in both groups, providing reassurance for patient safety.

Previous studies on the impact of fatigue on clinical performance show mixed results. Surgical trainees awake the previous night made 20% more errors in stimulators (Taffinder et al., 1998). Gastroenterologists who performed emergent procedures the previous night had a significant 24% decrease in adenoma detection rates on colonoscopies performed post-call (Benson et al., 2014). While long work hours may contribute to mistakes, the lack of supervision, faulty handovers and large patient caseloads were important factors that further compounded these errors.

Yet, other studies have observed that performance was no worse amongst fatigued trainee doctors. Yi et al. (2013) found no significant difference in psychomotor performance of general surgery residents performing laparoscopic simulator tasks after a 24-hour call work shift versus a 12-hour night-float shift. Robison et al. (2018) reported no significant difference in acquisition of robotic simulator skills between pre-call and post-call general surgery residents despite higher levels of fatigue, and Domínguez et al. (2009) and Hamui-Sutton et al. (2013) observed fewer errors in cognitive tests and CPR manoeuvres in residents after on-call duty. This paradox of improved execution might be explained by an increased motivation to execute tasks as efficiently as possible, even if cognitive processes were theoretically impaired by sleep deprivation (Ayas et al., 2006; Barger et al., 2005; Taffinder et al., 1998).

Additionally, it was postulated that alertness and task performance were impaired immediately in the first 2 hours of awakening due to sleep inertia, but these detrimental effects dissipate with time (Jewett et al., 1999). Post-call residents in our study might therefore have had a chance to recover by the time clinic started.

C. Strengths and Limitations

A particular strength of our study was that registrars who had post-call clinics surveyed also had a non-post-call clinic surveyed. This enabled them to act as controls for themselves and reduced potential bias that may be introduced as a result of differences in registrar demographics, such as age, gender and training experience. Second, there was a high patient participation rate in both the post-call group (97%) and non-post-call group (90%), with a high questionnaire return rate (99% in both groups). This increased the validity of our findings. Third, our study measured two different patient outcomes as a means of triangulation, and both were congruent in showing better outcomes in the post-call group. Fourth, we approached patients for their participation only after their clinic consultation, so as not to have influenced their interaction with the doctor. Fifth, patients were not informed if the registrar that they had just consulted was post-call or not, so as not to influence their questionnaire responses.

Nonetheless, being a single-centre study, caution should be exercised when attempting to generalise these findings to other healthcare settings. Second, only 8 of 18 eligible registrars in the department had clinics surveyed during the study period; thus the findings may not be reflective of the entire cohort. Third, there may have been factors outside the doctor-patient encounter that affected overall patient satisfaction, such as waiting times. Fourth, we did not quantify how busy each registrar’s on-call shift was and what amount of rest they obtained; the results might have been influenced by the amount of rest obtained. Fifth, we could not control for the complexity of the patients seen in the clinics; it is perceivable that a clinic with more straightforward patients would have been more easily managed by a fatigued registrar and vice versa.

For future studies, increasing cohort size of registrars over a longer period of time, and at more categories of call status (such as post-call, on-call and neither post- nor on-call) could improve the generalisability of the findings and reveal influences of call status on performance. In addition, incorporating qualitative data on registrars’ emotions and perceptions of their own performances in clinic would improve understanding of potential factors that may influence performance.

V. CONCLUSION

Our study showed that patient satisfaction and prescription error rates were not adversely affected when patients consulted a post-call registrar versus a non-post-call registrar in an outpatient clinic. This provides reassurance that the current practice of post-call trainee doctors running clinics is not harmful to patients. Complex interactions of associated positive and negative emotional factors could possibly compensate for any physical and mental tiredness. Impairment may be mitigated by adaptive effort to be conscientious. In our institution, no recommendations were made to change the current practice of registrars running post-call clinics. Ultimately, the relationship between being post-call with physician performance and patient outcomes remains poorly understood.

Notes on Contributors

Dr. Chin Chee Yang is a Consultant in Cardiology at the National Heart Centre Singapore and Clinical Assistant Professor at Duke-NUS Medical School. He has a Master of Science in Clinical Education. CCY designed and directed the study, implemented the research, data analysis and writing of the manuscript.

Dr. Tan Si Qi is a Medical Officer with the Ministry of Health Holdings, Singapore. TSQ contributed to the data analysis and writing of the manuscript.

Dr. Kui Swee Leng is a senior resident in Cardiology at Singhealth. She is an Adjunct Research Fellow with Singhealth Duke-NUS Cardiovascular Sciences Academic Clinical Programme (ACP) and Singhealth Associate in Education. KSL contributed to design and implementation of the research.

​Assistant Professor Kurugulasigamoney Gunasegaran is a Senior Consultant at the National Heart Centre Singapore. KG contributed to the design and overall supervision of the research.

Dr. Jill Cheng Sim Lee is a Consultant in Obstetrics and Gynaecology and Associate Programme Director to the SingHealth Obstetrics and Gynaecology Residency Programme. She has a Master of Science in Clinical Education. JCSL contributed to the design of the research and writing of the manuscript.

All authors have read and approved the final manuscript.

Ethical Approval

The study was categorised as an educational study hence formal Institutional Board Review (IRB) was exempted. We have attached the Local IRB Exception waiver document.

Funding

No funding source was utilised in this study. 

Declaration of Interest

Authors report no declaration of interest.

Data Availability

In our study, all raw data from questionnaires was collected on hard copies and not scanned into a repository. All other data are presented in the direct manuscript.

References

Ayas, N. T., Barger, L. K., Cade, B. E., Hashimoto, D. M., Rosner, B., Cronin, J. W., Speizer, F. E., & Czeisler, C. A. (2006). Extended work duration and the risk of self-reported percutaneous injuries in interns. Journal of the American Medical Association, 296(9), 1055–1062. https://doi.org/10.1001/jama.296.9.1055

Baldwin, K., Namdari, S., Donegan, D., Kamath, A. F., & Mehta, S. (2011). Early effects of resident work-hour restrictions on patient safety: A Systematic review and plea for improved studies. The Journal of Bone and Joint Surgery-American Volume, 93(2), e5(1)-e5(9). https://doi.org/10.2106/JBJS.J.00367

Barger, L. K., Cade, B. E., Ayas, N. T., Cronin, J. W., Rosner, B., Speizer, F. E., & Czeisler, C. A. (2005). Extended work shifts and the risk of motor vehicle crashes among interns. New England Journal of Medicine, 352(2), 125–134. https://doi.org/10.1056/NEJMoa041401

Benson, M., Grimes, I., Gopal, D., Reichelderfer, M., Soni, A., Benson, H., Austin, K., & Pfau, P. (2014). Influence of previous night call and sleep deprivation on screening colonoscopy quality. The American Journal of Gastroenterology, 109(8), 1133–1137. https://doi.org/10.1038/ajg.2014.28

Bertram, D. A. (1988). Characteristics of shifts and second-year resident performance in an emergency department. New York State Journal of Medicine, 88(1), 10–14.

Bolster, L., & Rourke, L. (2015). The Effect of Restricting Residents’ Duty hours on patient safety, resident well-being, and resident education: An updated systematic review. Journal of Graduate Medical Education, 7(3), 349–363. https://doi.org/10.4300/JGME-D-14-00612.1

Dawson, D., & Reid, K. (1997). Fatigue, alcohol and performance impairment. Nature, 388(6639), 235–235. https://doi.org/10.1038/40775

Domínguez, P., Grosso, M. L., Pagotto, B., Taliercio, V., & Allegri, R. (2009). Efectos de la privación de sueño en el desempeño de los médicos residentes de pediatría [Effects of sleep deprivation on medical performance of pediatric residents]. Archivos Argentinos De Pediatria, 107(3), 241–245.

Erdmann, T. R., Garcia, J. H. S., Loureiro, M. L., Monteiro, M. P., &amp; Brunharo, G. M. (2016). Profile of drug administration errors in anesthesia among anesthesiologists from Santa Catarina. Brazilian Journal of Anesthesiology (Elsevier), 66(1), 105–110. https://doi.org/10.1016/j.bjane.2014.06.011

Fletcher, K. E., Underwood, W., Davis, S. Q., Mangrulkar, R. S., McMahon, L. F., & Saint, S. (2005). Effects of work hour reduction on residents’ lives: A systematic review. Journal of the American Medical Association, 294(9), 1088–1100. https://doi.org/10.1001/jama.294.9.1088

Giordano, L. A., Elliott, M. N., Goldstein, E., Lehrman, W. G., & Spencer, P. A. (2009). Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review, 67(1), 27–37. https://doi.org/10.1177/1077558709341065

Goldstein, E., Farquhar, M., Crofton, C., Darby, C., & Garfinkel, S. (2005). Measuring hospital care from the patients’ perspective: An overview of the CAHPS® hospital survey development process: Measuring hospital care from the patients’ perspective. Health Services Research, 40(6p2), 1977–1995. https://doi.org/10.1111/j.1475-6773.2005.00477.x

Hamui-Sutton, L., Barragán-Pérez, V., Fuentes-García, R., Monsalvo-Obregón, E. C., & Fouilloux-Morales, C. (2013). Efectos de la privación de sueño en las habilidades cognitivas, psicomotoras y su relación con las características personales de los médicos residentes [Sleep deprivation effects on cognitive, psychomotor skills and its relationship with personal characteristics of resident doctors]. Cirugia Y Cirujanos, 81(4), 317–327.

Hanna, T. N., Loehfelm, T., Khosa, F., Rohatgi, S., & Johnson, J. (2016). Overnight shift work: factors contributing to diagnostic discrepancies. Emergency Radiology, 23(1), 41-47. https://doi.org/10.1007/s10140-015-1355-0

Hendey, G. W., Barth, B. E., & Soliz, T. (2005). Overnight and postcall errors in medication orders. Academic Emergency Medicine, 12(7), 629–634. https://doi.org/10.1197/j.aem.2005.02.009

Hoellein, A. R., Feddock, C. A., Griffith, C. H., Wilson, J. F., Barnett, D. R., Bass, P. F., & Caudill, T. S. (2004). Are continuity clinic patients less satisfied when the resident is postcall? Journal of General Internal Medicine, 19(5p2), 562–565. https://doi.org/10.1111/j.1525-1497.2004.30165.x

Jewett, M. E., Wyatt, J. K., Ritz-De Cecco, A., Khalsa, S. B., Dijk, D. J., & Czeisler, C. A. (1999). Time course of sleep inertia dissipation in human performance and alertness. Journal of Sleep Research, 8(1), 1–8. https://doi.org/10.1111/j.1365-2869.1999.00128.x

Keers, R. N., Williams, S. D., Cooke, J., & Ashcroft, D. M. (2013). Prevalence and nature of medication administration errors in health care settings: A Systematic review of direct observational evidence. Annals of Pharmacotherapy, 47(2), 237–256. https://doi.org/10.1345/aph.1R147

Landrigan, C. P., Rothschild, J. M., Cronin, J. W., Kaushal, R., Burdick, E., Katz, J. T., Lilly, C. M., Stone, P. H., Lockley, S. W., Bates, D. W., & Czeisler, C. A. (2004). Effect of reducing interns’ work hours on serious medical errors in intensive care units. New England Journal of Medicine, 351(18), 1838–1848. https://doi.org/10.1056/NEJMoa041406

Lee, J. C. S., Kee, X. L. J., Wiener-Ogilvie, S., Chern, B. S. M., & Chin, C. Y. (2019). What drives attrition amongst obstetrics and gynaecology residents in Singapore. The Asia Pacific Scholar, 4(1), 34–41. https://doi.org/10.29060/TAPS.2019-4-1/OA2005

Liu, C.-C., & Wissow, L. (2011). How post-call resident doctors perform, feel and are perceived in out-patient clinics: Post-call residents seeing outpatients. Medical Education, 45(7), 669–677. https://doi.org/10.1111/j.1365-2923.2010.03912.x

Lobo, A. S., Sandoval, Y., Burke, M. N., Sorajja, P., Mooney, M., Traverse, J., Henry, T. D., Chavez, I., Gössl, M., Lips, D. L., Bradley, S. M., Poulose, A., Wang, Y., & Brilakis, E. S. (2019). Sleep deprivation in cardiology: A multidisciplinary survey. The Journal of Invasive Cardiology, 31(6), 195–198.

Mustafa, M., Erokwu, N., Ebose, I., & Strohl, K. (2005). Sleep problems and the risk for sleep disorders in an outpatient veteran population. Sleep and Breathing, 9(2), 57–63. https://doi.org/10.1007/s11325-005-0016-z

Robison, W., Patel, S. K., Mehta, A., Senkowski, T., Allen, J., Shaw, E., & Senkowski, C. K. (2018). Can fatigue affect acquisition of new surgical skills? A prospective trial of pre- and post-call general surgery residents using the da Vinci surgical skills simulator. Surgical Endoscopy, 32(3), 1389–1396. https://doi.org/10.1007/s00464-017-5820-6

Rosenbaum, L., & Lamas, D. (2012). Residents’ Duty hours — Toward an empirical narrative. New England Journal of Medicine, 367(21), 2044–2049. https://doi.org/10.1056/NEJMsr1210160

Saadat, H., Bissonnette, B., Tumin, D., Thung, A., Rice, J., Barry, N., & Tobias, J. (2016). Time to talk about work-hour impact on anesthesiologists: The effects of sleep deprivation on Profile of Mood States and cognitive tasks. Pediatric Anesthesia, 26(1), 66–71. https://doi.org/10.1111/pan.12809

Samkoff, J. S., & Jacques, C. H. (1991). A review of studies concerning effects of sleep deprivation and fatigue on residents’ performance. Academic Medicine, 66(11), 687–693. https://doi.org/10.1097/00001888-199111000-00013

Taffinder, N., McManus, I., Gul, Y., Russell, R., & Darzi, A. (1998). Effect of sleep deprivation on surgeons’ dexterity on laparoscopy simulator. The Lancet, 352(9135), 1191. https://doi.org/10.1016/S0140-6736(98)00034-8

Tully, M. P., Ashcroft, D. M., Dornan, T., Lewis, P. J., Taylor, D., & Wass, V. (2009). The causes of and factors associated with prescribing errors in hospital inpatients: A Systematic Review. Drug Safety, 32(10), 819–836. https://doi.org/10.2165/11316560-000000000-00000

Yi, W. S., Hafiz, S., & Sava, J. A. (2013). Effects of night-float and 24-h call on resident psychomotor performance. Journal of Surgical Research, 184(1), 49–53. https://doi.org/10.1016/j.jss.2013.03.029

*Chin Chee Yang
National Heart Centre Singapore
5 Hospital Drive, Singapore 169609
Tel: +65 6704 8962
E-mail: chin.chee.yang@singhealth.com.sg

Submitted: 18 October 2020
Accepted: 1 March 2021
Published online: 5 October, TAPS 2021, 6(4), 17-25
https://doi.org/10.29060/TAPS.2021-6-4/OA2418

Xin Rong Goh¹, Chee Wai Ku^2,4, Rajeswari Kathirvel^1,2,4,5 & Kok Hian Tan^1,3,4

¹Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore; ²Department of Obstetrics and Gynaecology, KK Women’s & Children’s Hospital, Singapore; ³Department of Maternal Fetal Medicine, KK Women’s & Children’s Hospital, Singapore; ⁴Duke-NUS Medical School, Singapore; ⁵Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: Disease outbreaks (DO) result in unprecedented changes to the healthcare industry with far-reaching implications for medical education. The need to adapt to the fluidity during DO requires the delivery of the clinical medical curriculum to be flexible and effective. There is a lack of well-established guidelines on how medical education should be delivered during DO. This study aimed to explore the efficacy of teleconferencing-based platforms (TBP) as a teaching modality to overcome the challenges of clinical year medical education amidst a global pandemic and possibility of its use when there are no disease outbreaks (NDO).

Methods: A cross-sectional survey amongst 144 undergraduate clinical year students from a medical school in Singapore was conducted from May to June 2020, to explore their perspectives on TBP compared to physical venue-based platforms (PVBP). The survey consisted 5-point Likert scale and open-ended questions. Statistical and thematic analyses were performed.

Results: TBP provides greater convenience in travelling, note-taking and ability to overcome administrative challenges. Students strongly recommended its use in DO and NDO. However, students faced increased distractibility, decreased engagement and ease of raising questions, with a lower efficacy in content delivery. The above is dependent on the type of lessons delivered – clinical skills-based or didactic sessions.

Conclusion: TBP is a promising teaching modality for DO with promising possibility of extending its use to NDO. We propose a tri-faceted approach to target improvement in content delivery on TBP, mainly with measures to target propensity for decreased engagement and increased distractibility and to address the technology-related concerns.

Keywords:           Medical Education, Teleconferencing, Teaching Modalities, Medical Students, COVID-19, Disease Outbreaks, Pandemics

Practice Highlights

• TBP was strongly recommended as substitute teaching modality during disease outbreaks.
• Students recommended TBP as a supplement for teaching even when there are no disease outbreaks.
• TBP enabled more convenience in travelling, note-taking and overcoming administrative challenges.
• Decreased engagement and increased distractibility noted when sessions are conducted on TBP.
• TBP can be improved via use of teaching aids and contextualising it to the lesson type.

I. INTRODUCTION

Singapore has experienced the escalation of its Disease Outbreak Response System Condition (DORSCON) status to ‘Orange’ due to COVID-19 outbreak on 7th February 2020. This has had significant implications in multiple sectors of the healthcare industry, including that of medical education (Samarasekera, Goh & Lau, 2020). It called for the rapid remodelling of the delivery of the medical education curriculum during a time when social containment and avoidance of large groups gatherings were enforced (Samarasekera & Gwee, 2021). In particular, to contain and prevent the spread of COVID-19, clinical rotations in healthcare institutions were temporarily suspended during the DORSCON Orange period (Chandratre, 2020; Samarasekera, Goh, Yeo et al., 2020).

Many institutions had since employed teleconferencing-based platforms (TBP) to deliver curriculum (Kanneganti et al., 2020; Srinivasan, 2020). TBP is defined as an avenue of e-learning that is ‘internet and local-networking based’ (Al-Shorbaji et al., 2015), enabling the provision of ‘synchronous’ (Dhir et al., 2017) real time audio-visual online interactions across different locations (Lamba, 2011). The literature exploring the use of TBP during DO is limited. Most studies on TBP focused on its use when there are no disease outbreaks (NDO). They described its facilitation of direct teacher-student engagement with quieter students (Fox, 2004) and raised its comparable effectiveness in fulfilling learning outcomes to traditional face-to-face lectures (Bertsch et al., 2007). TBP nonetheless faces deterrents to its implementation, including those of financial implications (Lim et al., 2009) and technical difficulties (Boatin et al., 2015; Lamba, 2011). During DO, it had helped to overcome concerns of social distancing and allowed the continuation of medical training (Kanneganti et al., 2020; Lim et al., 2009).

In spite of the above, it should be recognised that the temporary suspension of clinical rotations represent the absence of a cornerstone in delivery of the medical curriculum (Govindarajan et al., 2018; Jacobs & Samarasekera, 2012; Lim et al., 2009; Rawekar et al., 2016). This study thus chose to specifically focus on the clinical year students owing to the potential application of these findings to post-graduate clinical trainings. Furthermore, the different content and focus during the pre-clinical years (e.g. lack of clinical rotations, presence of anatomy and science practical) would likely raise concerns that are unique and non-generalisable to students in the clinical years. Pre-clinical students were therefor excluded from the current study.

While the use of TBP is largely commended (Fatani, 2020), it is pertinent to compare its efficacy to traditional physical venues-based platforms (PVBP) as a potential replacement or supplement for the delivery of the clinical curriculum during DO. This study aims to understand the ground-up perspective of clinical year students on TBP as an education tool during DO, as compared to PVBP. It also explored TBP’s suitability and effectiveness for the delivery of the undergraduate medical education when there are no disease outbreaks (NDO).

II. METHODS

A. Ethics Review

The following study was approved by NTU Institutional Review Board (IRB Reference number: 2020-05-003) with an exempt status.

B. Survey Design and Study Population

All undergraduate clinical year medical students from the Lee Kong Chian School of Medicine who had used teleconferencing as a part of their curriculum were invited to participate in an online survey via email and social-messaging platform that included an explanation of the study’s details. Informed consent was implied when the participant accessed the online survey administered via SurveyMonkey^Ó platform. Personal identifiers were not collected.

The cross-sectional survey consisted of a mix of 5-point Likert scale questions and open-ended questions. It was anticipated to be completed in 10 minutes. The questions were designed to evaluate their familiarity with teleconferencing and their opinions on teleconferencing as an educational tool as compared to physical venue-based sessions, e.g. traditional lectures, bedside and small group tutorials, and team-based learning. Their willingness to extend teleconferencing as a supplementary delivery tool for medical education in both DO and NDO situations were also explored.

The survey questions were developed de-novo with reference to the existing literature. The themes of platform accessibility and ease of raising questions were adapted from Al-Neklawy (2017) while that of engagement and distractibility were adapted from the Danielson’s Framework for Teaching – The classroom environment (Alvarez & Anderson-Ketchmark, 2011). The survey was then piloted amongst a group of medical students and faculty to assess content and face validity.

C. Statistical Analysis

Self-reported estimates on the number of prior use of teleconferencing were divided by 52 to determine weekly usage. Numerical values were awarded for the 5-point Likert scale as follows: Strongly agree (5), agree (4), neutral (3), disagree (2) and strongly disagree (1).

Paired 2 tailed T test was performed to determine the significance of difference in subjective efficacy of PVBP and TBP. To compare effect of gender on subjective efficacy for the two platforms and recommendations for TBP, independent T test and Fisher’s exact test were performed respectively. Pearson Correlation testing was performed to look for correlations between the variables. Kruskal-Wallis H test was used to study the effect of year of study on the responses. A p value of < 0.05 indicated statistical significance. Data analysis was performed using SPSS^TM software (V.24.0).

D. Qualitative Analysis

Thematic analysis was performed for the elaborations and justifications provided by survey respondents. Concepts that were similar were used to guide the development of themes.

III. RESULTS

A. Sociodemographic Characteristics

A total of 144 out of 315 clinical year medical students completed the survey over a 3-week period from May to June 2020 (Year 3: n = 51; Year 4: n = 64; Year 5: n = 29).  The survey response rate was 45.7%. Majority of students who responded were female (58.3%) and Year 4 (44.4%).

B. Teleconferencing Usage Patterns

Majority (73.6%) of students had used a teleconferencing platform, for educational and non-education purposes, in the past year prior to declaration of DORSCON Orange. The top three teleconferencing platforms by corrected mean weekly usage prior to DORSCON orange were Skype^© (0.2), FaceTime^© (0.2) and ZOOM^© (0.1). During DOSRCON orange, the top three platforms for educational purposes by corrected mean weekly usage were ZOOM^© (4.0), Skype^© (0.3), Facetime^© (0.1). (Table 1)

Table 1. Use of teleconferencing-based platforms by medical students

Corrected weekly usage was calculated as follows: (A) Total usage divided by 52, (B) Total usage divided by number of weeks from declaration of DOSRSCON orange to point of survey (For example: 1 week + 1 day would be considered as 2 weeks)

Others included: Whatsapp©, Microsoft Teams©, Google Hangouts©, WebEx©, Discord©, Houseparty©

Table 2: Comparison of subjective efficacy of physical venue-based vs teleconferencing based platforms by clinical year medical students surveyed from May to June 2020 on a 5-point Likert Scale

Numerical values were assigned as follows: Strongly disagree (1), Disagree (2), Neutral (3), Agree (4), Strongly agree (5)

D. Correlations Between Responses

Based on Pearson’s correlation, students who were engaged on PVBP, tend to be more comfortable in raising questions (r = 0.301, p <0.001) and less distracted (r = -0.337, p <0.001) with PVBP. Similarly, students who felt engaged on TBP tend to feel comfortable in raising questions on TBP (r = 0.301, p <0.001), less distracted (r = – 0.353, p <0.001) and agree on its efficacy in content delivery (r = 0.570, p <0.001).  Students who felt more distracted on PVBP tend to also be more distracted on TBP (r = 0.176, p <0.05). No significant correlations were found between prior teleconferencing experience on self-reported rating of PVBP and TBP.

Kruskal-Wallis H test showed a statistically significant difference in rating of convenience for TBP between different years of study (χ2 (2) = 18.1, p <0.001, mean (Year 3) 87.1, (Year 4) 69.8, (Year 5) 53.0). There were no statistically significant effects of gender on the responses.

E. Recommendations of Teleconferencing Platform

An overwhelming majority of students recommended use of TBP in DO (n = 143, 99.3%), 54.9% (n = 79) recommended it as a substitute, and 44.4% (n = 64) as a supplement for learning, whereas one student did not recommend it at all. A significant proportion continued to favour its use even during NDO (n = 121, 84.0%), although there was a slight shift in preference for it to be used as a supplement (Supplement: n = 108, 75%; Substitute: n = 13, 9.0%). (Table 3A)

There were no statistically significant effects of gender or year of study on the responses.

Table 3. Recommendations and suggested improvements for use of teleconferencing platforms during disease and non-disease outbreak situations by clinical year medical students surveyed from May to June 2020

N refers to the total number of elaborations provided for each type of recommendations, of which percentage (%) calculated reflects the percentage of respondents who raised a particular theme in their elaborations. n refers to the number of times the theme appeared per respondent.

Representative elaborations were quoted and tagged by the respondent’s demographic (Recommendation, Year of Study, Gender)

F. Qualitative Analysis

Thematic analyses revealed 10 main areas of discussion. These were categorised into three main themes that were common for both DO and NDO: convenience, content delivery and context of lesson (Table 3B). For the analysis for DO recommendations, a new theme materialised: Pandemic-specific considerations. Suggestions for improvements of TBP mainly fell into 3 categories – enhanced use of TBP teaching aids, ZOOM^© fatigue and connectivity and technological concerns.

1) Convenience: Students indicated that TBP reduced their need to travel to teaching destinations, thereby saving on expenditure and time. This was especially useful when they were scheduled with a full day of lectures that did not require any clinical setting training, and during the exam period. For example, one student reported, “I value the time saved on travelling… more than the increased concentration and engagement of a face-to-face tutorial, given that exams are looming.” (Year 4, Female)

The ability to stream these sessions from home was advantageous as students could take additional measures to help sustain their attention, including the ease of taking refreshment breaks. Students valued the ease of taking notes as elaborated by one student as “easier to take screenshots of the slides so we don’t waste time copying the points.” (Year 5, Female)

Students noted TBP to be more convenient in arranging lectures across institutions. Specific to NDO, TBP was suggested as a potential mean to allow “teaching very large groups of people… combined stream classes” (Year 3, Male) and across medical schools. TBP also offered flexible scheduling arrangements. Examples included situations where participants are “feeling unwell but are worried about missing class” (Year 3, Female), “tutors with very busy or unpredictable schedules… which would require students to meet at odd or inconvenient timings” (Year 4, Male), hence minimising need for students to “stay back” (Year 5, Female) in the late evening.

2) Content delivery: Students discussed how triadic interactions between tutors and students, and amongst themselves were better afforded on PVBP. Nuances of social cues like facial expressions were cited as potentially beneficial for tutors to gauge students’ engagements. One student shared that “the tutor can gauge whether or not the content is appropriate based on the body language/facial expressions of the students which may be hard to do so on an online platform.” (Year 5, Female) Students also noted how TBP changed the nature of social interactions, making participants “less likely to speak up or participate… [with] the tutor [feeling] more distant.” (Year 4, Female)

The interactions between students as a reinforcement for learning surfaced as a crucial component. As one student shared, PVBP gave the “chance for the team to meet and learn from each other… quizzing each other, clarifying doubts, or sharing fun facts and important information [that]… cannot be done over teleconferencing.” (Year 3, Female)

Students reported that it was harder to stay focused during TBP sessions as one was in a “more comfortable location with no one monitoring your movements and actions” (Year 3, Male), unable to see the tutor in person and may have concomitant activities surrounding them. The implications of such distractibility were noted that TBP made it easier “to hide away and therefore not be present.” (Year 4, Female)

3) Context of lesson: Students felt their subjective assessment of TBP versus PVBP should factor in the type of lessons being delivered. It was unanimous that clinical-based teachings required use of PVBP to develop soft skills and appreciate clinical signs. To illustrate, one student shared, “medicine is an apprenticeship – there are many skills that cannot be learnt theoretically but must be observed in a clinical setting… Physical sessions are still very important to teach such skills, and should proceed with adequate safety measures in place. Otherwise, we will become doctors with good theoretical knowledge but poor practical/people skills” (Year 5, Female). However, many agreed that didactic lectures which involved pure unidirectional delivery of content could be suitably delivered over TBP as “lecturer… can still speak and present slides as usual.” (Year 3, Male)

4) Pandemic specific considerations: Students acknowledged the risk-benefit ratio of disease transmission and accommodation for the schedule of clinical tutors. One student summarised, “Additional benefits gained through physical venue-based outweighed by risks of disease transmission due to physical interaction.” (Year 3, Male) TBP was hence considered to be a safe and only suitable alternative to ensure continuity of curriculum delivery during DO.

5) Identified areas of improvement: Students raised suggestions to overcome the drawbacks of TBP in 3 main domains. Firstly, they recommended the enhanced used of teaching aids to increase engagement peer-peer interactions. Secondly, they raised the phenomenon of “ZOOM^© fatigue” and the need to manage scheduling of tutorials. For example, one student shared, “People seem to think that online lectures are less mentally taxing since you’re at home, and proceed to pack the entire day full of lectures for couple weeks on end… there is no time to consolidate knowledge especially in the absence of opportunity to apply in a clinical setting.” (Year 4, Male) Hence, students suggested tutors to avoid “bombarding students with back-to-back tutorials, leading to information overload” (Year 4, Female) and decrease the maximum time per day to less than 4-5 hours. The inclusion of breaks between sessions as a mental break since tutors who are “swapping in… would not know if students are still engaged or fatigued.” (Year 4, Male)

Lastly, presence of poor connectivity contributed to streaming lags, with negative implications on students “concentration and ability to follow the lecture.” (Year 4, Female) The possibility of increasing tutors’ familiarity with the platform was cited to have facilitated a smoother conduct of lessons.

IV. DISCUSSION

A. Summary of Results

An overwhelming majority of students recommended the continued use of TBP during both DO and NDO. This is most likely due to the increased convenience in travelling, taking down notes, overcoming administrative challenges, and its suitable replacement for didactic lectures. The student’s perception of convenience of TBP correlated to the year of study, with the earlier clinical year students finding it more convenient than the final year students. Nevertheless, this should be evaluated against the background that end users range from enthusiastic ‘technophiles’ to ‘technophobes’, which influences the receptiveness towards e-learning platforms (Bruce, 1997; Fox, 2004).

Specifically during DO, it was viewed to be the best available alternative when considering the disease transmission risk and the busy schedules of the clinical tutors as they may have new responsibilities as front-liners in pandemics (Branch et al., 1997; Cook, 2006; Goh, 2020; Harden & Crosby, 2000; Ramani & Leinster, 2008).

The shift in preference of use of TBP as a substitute to supplement during NDO was noticeable. This can be due to many factors. There is a decreased engagement due to the reduced ability of tutors to assess nuances of social cues (Fox, 2004),  greater distractibility and decreased ease of raising questions on TBP. Notably, peer-peer interaction is lost over TBP. Each student’s interaction is multidirectional: student with resource materials, with educator, and between students (Dhir et al., 2017). The value of such interactions and study groups in medical education should be duly taken into consideration (Burgess et al., 2014). Poorer engagement were further compounded by challenges with technology-related concerns and risks of “ZOOM^© fatigue”.

Distractibility on TBP could be due to learning in a non-classroom environment (Fox, 2004) with a device that could also be used for non-educational purposes (Walsh, 2015). “ZOOM^©  fatigue” was likely due to lack of pre-fixed maximal time duration and the tendency for tutors to over-deliver and pile extra study materials (Fox, 2004)  causing student’s to “feel tired and restless” (Srinivasan, 2020).

Interestingly, students’ innate personality and learning attitudes potentially influenced their perceptions of PVBP and TBP. Students who were more engaged on a platform, also felt more comfortable raising questions, were less distracted and believed that the platform was efficacious in delivering content. This demonstrates the difference between the “active and engaged learners” and the majority who are “silent lurkers (Fox, 2004).  Similarly, students who felt more distracted on PVBP appeared to also be more distracted on TBP.

B. Recommendations

TBP as a teaching modality has an indisputable role during both DO and NDO. It is well-established that medical students experience significant psychological impact during DO (AlAteeq et al., 2020; Ullah & Amin, 2020), possibly related concerns on the impact of their studies (Lyons et al., 2020). Hence with the increased use of TBP, it is imperative that medical educators develop a deeper understanding on the potential short-comings of the platform and how best to maximise its utility as a teaching modality.

Nonetheless, it is vital to recognise that the efficacy of any new education tool depends on both educators and students. It would be better facilitated if the end user is familiar with the platform. It should be considered in light of its application and configuration (Cook, 2006) and ideally be managed and monitored by trained staff (Dhir et al., 2017; Harden, 2018). Moving forward, we recommend a tri-faceted approach to improve the content delivery of TBP during both DO and NDO (Figure 1).

Figure 1. Correlation of domains assessed for efficacy of teleconferencing-based platforms (TBP) with themes identified and suggestions to improve content delivery on TBP

1) Domain 1: Contextual use of TBP: Disease outbreak situations:

• Where physical appreciation of clinical skills through bedside tutorial or real-life clinical interactions might not be feasible, consider use of media and standardised patients to simulate clinical exposure.
• The supplemental use of standardised patients (Peters & Thrien, 2020) hones student’s clinical competence while facilitating the training of relevant skills needed for future practice (Khoo et al., 2020), a notable example in these evolving recent times also includes that of telemedicine (Williams & Song, 2016).

No disease outbreak situations

• TBP should mainly be used for didactic or team-based learning sessions, large group teaching sessions and for students who are unable to physically attend the lecture.

2) Domain 2: Targeting decreased engagement and increased distractibility:

• Use of multi-modal teaching aids, e.g. virtual quizzes, breakout rooms for small group engagement, drawing functions to better visually illustrate explanations.
• Monitoring of students’ current fatigue level in session via live polls at regular hourly intervals.
• Engagement of a central coordinator to regulate the maximum hours of online sessions per day, with considerations of suitable breaks between sessions.
• Prior assessment of student’s learning attitudes and class dynamics to enable tailored use of teaching aids.

3) Domain 3: Smoothening transition to TBP:

• Engagement of central coordinator to brief tutors on the functionality of TBP of choice to minimise technology unfamiliarity and maximise its utility in conducting sessions.
• Educational institutions to consider providing infrastructural support to students and educators, including dedicated spaces, electronic devices with internet connections, and easy access to technical support. These would help minimise network connectivity challenges and providing a conducive study environment for those who might face difficulties accessing TBP off-campus.
• Selective use of TBP as a supplement during NDO to increase both students and educator’s familiarity with the platform and as a learning tool, thereby enabling a smoother transition during DO.

C. Limitations

Our study has few limitations. We surveyed clinical year students from a single medical school and therefore, our study may not reflect the views of students from other medical schools with different teaching pedagogies and those of pre-clinical year students. Future studies should consider exploring the opinions of clinical tutors on PVBP and TBP and their perspective as educators in delivering such curriculum. In addition, given our limited sample size, some statistically significant sub-group patterns may not have been evident.

V. CONCLUSION

TBP is an important teaching modality during DO and NDO, especially with its increased convenience. However, it has certain issues including context-specific use, decreased engagement, increased distractibility and technological challenges. Our proposed potential interventions may help to maximise its utility and facilitate transition of its use in subsequent DO.

Notes on Contributors

Xin Rong Goh designed the study, recruited the participants, administered the survey, analysed the data and wrote the first and subsequent drafts of the manuscript. Rajeswari Kathirvel and Chee Wai Ku contributed to analysis of the data and writing of the manuscript. Kok Hian Tan contributed to the study design and reviewed the manuscript. All authors have read and approved the final manuscript.

Ethical Approval

The following study was approved by NTU Institutional Review Board (IRB Reference number: 2020-05-003) with an exempt status. 

Data Availability

The ethical approval by NTU Institutional Review Board was based on the conditions that while the data is deidentified data, only study team members will have access to the raw data that will be stored on a password protected PC. The data could thus not be uploaded on a public data sharing platform.

Acknowledgement

The authors acknowledge Dr Joel Shi Quan Tan, Yong Loo Lin School of Medicine, National University of Singapore, for his contribution to the statistical analysis of the paper. 

Funding

No funding sources was used for this research study. 

Declaration of Interest

The authors have no conflicts of interest to declare.

References

AlAteeq, D. A., Aljhani, S., & AlEesa, D. (2020). Perceived stress among students in virtual classrooms during the COVID-19 outbreak in KSA. Journal of Taibah University Medical Sciences, 15(5), 398–403. https://doi.org/10.1016/j.jtumed.2020.07.004

Al-Neklawy, A. F. (2017). Online embryology teaching using learning management systems appears to be a successful additional learning tool among Egyptian medical students. Annals of Anatomy, 214, 9–14. https://doi.org/10.1016/j.aanat.2017.07.001

Al-Shorbaji, N., Atun, R., Car, J., Majeed, A., & Wheeler, E. (2015). eLearning health professional for undergraduate education. World Health Organistion. https://www.who.int/hrh/documents/14126-eLearningReport.pdf

Alvarez, M. E., & Anderson-Ketchmark, C. (2011). Danielson’s framework for teaching. Children and Schools, 33(1), 61–63. https://doi.org/10.1093/cs/33.1.61

Bertsch, T. F., Callas, P. W., Rubin, A., Caputo, M. P., & Ricci, M. A. (2007). Applied research: Effectiveness of lectures attended via interactive video conferencing versus in-person in preparing third-year internal medicine clerkship students for clinical practice examinations (CPX). Teaching and Learning in Medicine, 19(1), 4–8. https://doi.org/10.1080/10401330709336616

Boatin, A., Ngonzi, J., Bradford, L., Wylie, B., & Goodman, A. (2015). Teaching by teleconference: A model for distance medical education across two continents HHS public access. Open Journal of Obstetrics and Gynecology, 5(13), 754–761. https://doi.org/10.4236/ojog.2015.513106

Branch, W. T., Kroenke, K., & Levinson, W. (1997). The clinician-educator – Present and future roles. Journal of General Internal Medicine, 12(Suppl. 2), 1–4. https://doi.org/10.1046/j.1525-1497.12.s2.16.x

Bruce, B. C. (1997). Literacy technologies: What stance should we take? Journal of Literacy Research, 29(2), 289–309.

Burgess, A., McGregor, D., & Mellis, C. (2014). Medical students as peer tutors: A systematic review. BMC Medical Education, 14,115. https://doi.org/10.1186/1472-6920-14-115

Chandratre, S. (2020). Medical students and COVID-19: Challenges and supportive strategies. Journal of Medical Education and Curricular Development, 7. https://doi.org/10.1177/2382120520935059

Cook, D. A. (2006). Where are we with web-based learning in medical education? Medical Teacher, 28(7), 594–598. https://doi.org/10.1080/01421590601028854

Dhir, S. K., Verma, D., Batta, M., & Mishra, D. (2017). E-learning in medical education in India. Indian Pediatrics, 54(10), 871–877. https://doi.org/10.1007/s13312-017-1152-9

Fatani, T. H. (2020). Student satisfaction with videoconferencing teaching quality during the COVID-19 pandemic. BMC Medical Education, 20(1), 396. https://doi.org/10.1186/s12909-020-02310-2

Fox, R. (2004, December). SARS epidemic: Teachers’ experiences using ICTs. In R. Atkinson., C. McBeath., D. Jonas-Dwyer. & R. Phillips (Eds.), Beyond the comfort zone: Proceedings of the 21st Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) (pp. 319-327). Australasian Society for Computers in Learning in Tertiary Education. http://www.ascilite.org.au/conferences/perth04/procs/fox.html

Goh, C. T. (2020, May 3). COVID-19: Guarding against burnout, compassion fatigue and trauma in frontline healthcare workers. Channel News Asia. https://www.channelnewsasia.com/news/singapore/covid-19-guarding-against-burnout-trauma-ttsh-ncid-frontline-12669280

Govindarajan, S., Vasanth, G., Kumar, P. A., Priyadarshini, C., Radhakrishnan, S. S., Kanagaraj, V., Balasubramanian, N., Kumar, P. N., Divya, T. J., & Aishwarya, C. N. (2018). Impact of a comprehensive early clinical exposure program for preclinical year medical students. Health Professions Education, 4(2), 133–138. https://doi.org/10.1016/j.hpe.2017.06.002

Harden, R. M. (2018). Medical teacher E-learning-caged bird or soaring eagle? Medical Teacher, 30(1), 1–4. https://doi.org/10.1080/01421590801938878

Harden, R. M., & Crosby, J. (2000). AMEE guide no 20: The good teacher is more than a lecturer – The twelve roles of the teacher. Medical Teacher, 22(4), 334–347. https://doi.org/10.1080/014215900409429

Jacobs, J. L., & Samarasekera, D. D. (2012). How we put into practice the principles of embedding medical students into healthcare teams. Medical Teacher, 34(12), 1008–1011. https://doi.org/10.3109/0142159X.2012.731097

Kanneganti, A., Sia, C.-H., Ashokka, B., & Ooi, S. B. S. (2020). Continuing medical education during a pandemic: An academic institution’s experience. Postgraduate Medical Journal, 96, 384- 386. https://doi.org/10.1136/postgradmedj-2020-137840

Khoo, S. A., Aswin, W., Shen, G. Q. Y., Haq, H. M., Junaidah, B., Yiew, J. L., Abiramy, M., & Sashikumar, G. (2020). Improving provider-patient communication skills among doctors and nurses in the children’s emergency department. The Asia Pacific Scholar, 5(3), 28–41. https://doi.org/10.29060/TAPS.2020-5-3/OA2160

Lim, E. C. H., Oh, V. M. S., Koh, D. R., & Seet, R. C. S. (2009). The challenges of “continuing medical education” in a pandemic era. Annals of the Academy of Medicine Singapore, 38(8), 724–726.

Lyons, Z., Wilcox, H., Leung, L., & Dearsley, O. (2020). COVID-19 and the mental well-being of Australian medical students: Impact, concerns and coping strategies used. Australasian Psychiatry, 28(6), 649–652. https://doi.org/10.1177/1039856220947945

Lamba, P. (2011). Teleconferencing in medical education: A useful tool. Australasian Medical Journal, 4(8), 442–447. https://doi.org/10.4066/AMJ.2011.823

Peters, T., & Thrien, C. (2020). The digital use of simulated patients in times of the corona pandemic – Considerations and proposals. GMS Journal for Medical Education, 37(7), Doc 93. https://doi.org/10.3205/zma001386

Ramani, S., & Leinster, S. (2008). AMEE guide no. 34: Teaching in the clinical environment. Medical Teacher, 30(4), 347–364. https://doi.org/10.1080/01421590802061613

Rawekar, A., Jagzape, A., Srivastava, T., & Gotarkar, S. (2016). Skill learning through early clinical exposure: An experience of Indian medical school. Journal of Clinical and Diagnostic Research, 10(1), JC01–JC04. https://doi.org/10.7860/JCDR/2016/17101.7022

Samarasekera, D. D., Goh, D. L. M., & Lau, T. C. (2020). Medical school approach to manage the current COVID-19 crisis. Academic Medicine, 95(8), 1126–1127. https://doi.org/10.1097/ACM.0000000000003425

Samarasekera, D. D., Goh, D. L. M., Yeo, S. P., Ngiam, N. S. P., Aw, M. M., Lim, M. M., Pillai, S., Lee, S. S., Mahadevan, M., Kow, A., Chong, Y. S., & Lau, T. C. (2020). Response and lessons learnt managing the covid-19 crisis by school of medicine, National University of Singapore. MedEdPublish, 9(1), 92, https://doi.org/10.15694/mep.2020.000092.1

Samarasekera, D. D., & Gwee, M. C. E. (2021). Adapting to change during challenging times. The Asia Pacific Scholar, 6(1), 1-2. https://doi.org/10.29060/TAPS.2021-6-1/EV6N1

Srinivasan, D. K. (2020). Medical students’ perceptions and an anatomy teacher’s personal crisis experience using an e-learning platform for tutorials during the Covid-19 crisis. Anatomical Sciences Education, 13(3), 318-319. https://doi.org/10.1002/ase.1970

Ullah, R., & Amin, S. (2020). The psychological impact of COVID-19 on medical students [Letter]. Psychiatry Research, 288, 113020. https://doi.org/10.1016/j.psychres.2020.113020

Walsh, K. (2015). Mobile learning in medical education: Review. Ethiopian Journal of Health Sciences, 25(4), 363–366. https://doi.org/10.4314/ejhs.v25i4.10

Williams, B., & Song, J. J. Y. (2016). Are simulated patients effective in facilitating development of clinical competence for healthcare students? A scoping review. Advances in Simulation (London, England), 1, 6. https://doi.org/10.1186/s41077-016-0006-1

*Goh Xin Rong
11 Mandalay Road,
Singapore 308232
Email: xinronggoh@yahoo.com.sg

Submitted: 26 September 2020
Accepted: 1 March 2021
Published online: 5 October, TAPS 2021, 6(4), 7-16
https://doi.org/10.29060/TAPS.2021-6-4/OA2415

Rasika Manori Jayasinghe¹, Indika Priyanthi Thilakumara¹, Bandara Dhanushka Leuke², Gishan Edirisinghe³, Manil Christopher Nishan Fonseka³, Manjula Attygalla⁴ & Ruwan Duminda Jayasinghe²

¹Department of Prosthetic Dentistry, Faculty of Dental Sciences, University of Peradeniya, Sri Lanka; ²Department of Oral Medicine and Periodontology, Faculty of Dental Sciences, University of Peradeniya, Sri Lanka; ³Department of Restorative Dentistry, Faculty of Dental Sciences, University of Peradeniya, Sri Lanka; ⁴Department of Oral Surgery, Faculty of Dental Sciences, University of Peradeniya, Sri Lanka

Abstract

Introduction: E-learning resulted in a revolution in dental education with continuous educational experiences. General objective of this study was to assess the perspective of undergraduate dental students on effectiveness of e-learning in the Bachelor of Dental Surgery (BDS) programme. Specific objectives were to identify students’ opinion on the factors that promote e-learning, awareness on the effectiveness of using different platforms and social media, barriers imposed and suggestions for the improvement of e-learning in the BDS programme.

Methods: This was a cross sectional descriptive study among undergraduate dental students of the Faculty of Dental Sciences, University of Peradeniya, Sri Lanka using a self-administered pre-tested questionnaire administered via a web-based survey form. Frequencies and percentages were obtained for categorical data and Chi-square test was used to determine the association between variables.

Results: The majority received the e-learning experience well. Fifty-four percent of the participants felt it was better compared to traditional face-to-face learning and the difference between the semesters (p=0.000) and genders was statistically significant (p=0.000). Difference in overall satisfaction on e-learning material across the different semesters was statistically significant. Students felt that e-learning should be used as a supplementary tool mainly by means of procedural videos during delivery of the skill component. Fifty percent participants felt that conducting assessments online is fair for all the students.

Conclusion: A positive learning experience was achieved through the e-learning modalities in comparison to traditional face-to-face learning though in terms of skills training, e-learning modalities should only be considered as a supplementary tool.

Keywords:           E-Learning, Perspective, Dental Undergraduates, Dental Education

Practice Highlights

• Majority felt e-learning was better compared to traditional face-to-face learning.
• Majority were satisfied with interactions with teachers and teacher responses to clarifications.
• Students felt that e-learning used as a supplementary tool during delivery of the skill component.

I. INTRODUCTION

E-learning has resulted in a revolutionising dental education to an extent that more interactive and intuitive e-learning options have evolved which provide students with an enjoyable and meaningful continuum to their educational experience. During the current COVID-19 pandemic electronic learning tools have been invaluable in the delivery of knowledge in many higher education institutions (Li & Lalani, 2020). One of the great advantages of e-learning is the possibility of transferring knowledge and skills to a large number of recipients irrespective of the time of delivery of knowledge (Asiry, 2017). In addition, students can repeatedly peruse the material and follow it in their own pace at a relatively low cost. This is considered an excellent method of overcoming certain difficulties faced with the traditional system of teaching such as the lack of space especially within the lecture halls and a shortage of human resources (Asiry, 2017). However, socio-economic factors play an important role in deciding the type of teaching. Students in the developed countries prefer e-learning modalities while students in the developing countries prefer it as a supplementary method (Schlenz et al., 2020). A study by Asiry (2017) identified e-learning as a helpful supplementary learning method among Saudi Arabian dental students rather than a replacement for traditional teaching methods. According to authors’ experience, though there was much resistance in adopting to e-learning at its infancy, it has now been universally accepted as a learning modality not only by teachers but also by the students. Many are of the view that assimilation of knowledge could be best achieved purely through e-learning modalities where as it should be supplementary in nature in skills acquisition. The popularity of books and hard copies of journals have seen a steady decline due to the availability of e-learning resources and even publishers have realised this trend and supply both hard and electronic versions of their publications which in future may progressively be replaced by electronic educational materials.

The success of e-learning is dependent on a multitude of factors. Computer literacy, availability of appropriate technology, accessibility and having a good high bandwidth internet connection are some of the key factors (Asiry, 2017; Linjawi & Alfadda, 2018). According to a study by Gunawardane and Wijekoon (2017), it was revealed that dental students in the Faculty of Dental Sciences, University of Peradeniya, Sri Lanka had adequate computer literacy and facilities. Although Linjawi and Alfadda (2018) identified that the perceived impact of e-learning and readiness for e-learning though popular and satisfactory among Saudi Arabian dental students, declined as they proceeded to higher semesters. Imparting the skills component has been identified as a major challenge in using e-learning and students were in need of more support in the skills domain (Linjawi & Alfadda, 2018).  Students have been shown to have better skills and motivation for use of online tools for personal or non-educational purposes than for learning purposes. Thus, this should be carefully considered when developing a successful strategy to motivate them for e-learning (Linjawi & Alfadda, 2018). Apart from the above mentioned factors, student characteristics, cognitive factors (performance expectations) and the social environment (learning climate) are considered as some predictors of the perceived satisfaction among students (Venkatesh et al., 2019). Most of the students prefer a combined approach comprising traditional and e-learning methods than relying on e-learning methods alone. Some of the preferred methods of delivery of e-learning resources for pre-clinical dental undergraduates were online flash lectures and procedural videos (Asiry, 2017). Video demonstrations enabled students to reflect on their predominant learning approaches, which emphasise self-directed learning and avoid surface learning techniques (Chonkar et al., 2019). Some dental schools in the developed countries have introduced tools such as Technology Enhanced Learning (TEL) in both pre-clinical and clinical courses with the development of innovative learning platforms and they further assess and monitor student performance in relation to the new approach (Wong et al., 2020). Further, Faculty of Dentistry, National University of Singapore has implemented a multi-stage framework for the academic staff to grasp technology in various aspects such as reflecting teaching practice, designing new technology options and identifying learner impact with changes on teaching methods. 

However, not everyone is in agreement with the benefits of e-learning. University administrators and experts in information technology working in universities have varying opinion (ranging from majority of teaching on traditional classroom mode to predominant e-learning mode) regarding the need of administrational change created by e-learning, generation of resources, impact on enrolment, responsibility for course design and content and the impact on the mission of the university. However, there is a general agreement on challenges such as the requirement of resources and cost of maintenance, need for motivating the teachers to keep abreast with the latest technology and the necessity for cooperation to improve e-learning (Hillenburg et al., 2006). 

The COVID-19 pandemic has made it necessary to impart changers in the modalities of education all over the world. Since schools and universities have been closed to prevent the spread of infection, conducting e-learning sessions has become the prime mode of teaching. The situation in Sri Lanka was without exception. Most programmes including the Bachelor of Dental Surgery had to comply with this requirement in order to provide the students a continuous educational experience during the pandemic situation. E-learning has been used in a very primitive level in the Faculty of Dental Sciences, University of Peradeniya for a long time but had not been utilised to its full potential. As dentistry is mostly a skill based professional degree programme, delivering course content online was a challenge. In addition, resistance to change was evident among the staff due to the fact that they were comfortable with the traditional time tested methods. However, with the new norms, e-learning platforms such as Zoom, Google Meet and WhatsApp discussions were used at the Faculty of Dental Sciences for the learning along with narrated PowerPoint presentations, video demonstrations and online quizzes uploaded onto the existing e-learning platform Dent Moodle. Due to this novel experience, it was considered important to analyse the students’ opinion on e-learning, the difficulties faced by them and expectations to improve e-learning in the field of dentistry. Further, it would be beneficial to identify barriers of e-learning when designing and revising the dental undergraduate curriculum in the future.

II. OBJECTIVES

General objective of the study was to assess the undergraduate dental students’ perception on the effectiveness of e-learning utilised in the Bachelor of Dental Surgery (BDS) programme. The specific objectives were to identify students’ opinion on the factors that promote e-learning, awareness on the effectiveness of using different platforms and social media, barriers imposed and suggestions for the improvement of e-learning in the BDS study programme.

III. MATERIALS AND METHODS

A cross sectional descriptive study was carried out on a  sample comprising of undergraduate dental students of the Faculty of Dental Sciences, University of Peradeniya who had enrolled and followed all online lectures and assignments based on the content taught via e-learning in the first, third, fifth and seventh semesters. A self-administered and pre-tested (25 students representing all batches for the pilot and their responses were excluded from the study sample) questionnaire was administered via a web-based survey form. A covering letter, information sheet and consent form were also annexed to this form. Those respondents who gave the informed consent were able to fill the questionnaire. The survey link was disseminated among the dental students via the email by the faculty administration unit. The questionnaire used in this study comprised of 23 close-ended questions which consisted of demographic data and specific questions on awareness, attitude and practice of e-learning. Examples of e-learning platform stated in the questionnaire were live Zoom presentations, Google Meet, Power Point presentations with voice narrations and supplementary material such as web links, video demonstrations and YouTube and other video links.

Confidentiality and anonymity of the data provided were assured by keeping the questionnaire anonymous. No identification details were included in the questionnaire. Data management and statistical analysis was performed using the statistical software SPSS version 21.0. Frequencies and percentages were obtained for categorical data and Chi-square test was used to determine the association between variables. Ethical clearance was obtained from the Ethics Review Committee of the Faculty of Dental Sciences, University of Peradeniya (ERC/ FDS/UOP/I/2020/21).

IV. RESULTS

The response rate was 82% (250 participants out of 305 of all undergraduates). The sample comprised 19% from the first semester, nearly 25% from the third semester, 26% from the fifth semester and nearly 30% from the seventh semesters of the Bachelor of Dental Surgery (BDS) programme. Seventy-two percent (72%) were females. Approximately 41% claimed that they have either very good or excellent literacy on information technology (IT). Fifty-four percent of the respondents claimed that their English language literacy is very good or excellent. A majority of the population accessed e-learning content using their mobile devices. Half of the sample had experienced some kind of interruptions during e-learning.

A. Satisfaction on E-learning

Seventy-one percent (71%) responded that they were either satisfied or highly satisfied about e-learning materials they received. More than one third of the respondents (39%) felt encouraged to collaborate and interact with their teachers during e-learning time compared to the in-class teaching sessions. A similar percentage of respondents felt that they were more encouraged to work on course materials compared to the traditional learning sessions received at the university. Around 74% of the students felt that the teachers were friendly during e-learning sessions. Moreover, 62% of them were either satisfied or highly satisfied with the provision to ask questions during e-learning sessions. Sixty percent (60%) of respondents was satisfied or highly satisfied with the answers received for the clarifications they requested. Seventy percent (70%) of the students claimed that the interactions between the lecturers and students during question and answer sessions, case scenarios and case discussions were essential for the success of e-learning experience. Fifty-five percent (55%) of them were satisfied with the interactions they experienced during e-learning sessions in their study period. Interestingly, more than one third of the respondents were of the opinion that online lectures were more effective than traditional live lectures. Moreover, 54% of the sample felt that the overall e-learning experiences was better than the traditional learning opportunities they received in the university and the possibility of following the coursework at their own pace (14.8%) was chosen as the main reason for this response.

Females had better overall e-learning experiences than the learning opportunities they received at the university compared to male students and this difference was statistically significant (p=0.000) (Table 1).

Surprisingly, only 1% of the respondents had identified social media as a positive influence on e-learning.

Table 1 describes participants’ response to the questionnaire according to their gender.

Table 1. Comparison of participants’ response according to their gender (in percentages)

B. Preference on Methods of E-learning

Sixty-four percent (64%) of the participants were familiar with Zoom as an e-learning platform. Majority of the students preferred PowerPoint presentations with narrations (68%) to receive lectures. In addition, 69% felt that e-learning materials had helped them to improve their knowledge on the course content specified in the Intended Learning Outcomes (ILOs). Around 34% had identified video demonstrations as an important tool in imparting skills training to achieve the course ILOs. Over 70% rated it as highly useful when students’ opinion is taken in consideration in improving knowledge on the course content.

Sixty-four percent (64%) felt that that e-learning material should be used as supplementary materials for in-class learning within the skills domain. The e-learning materials considered appropriate were video demonstrations, procedural steps described in text documents, YouTube links and case discussions. Majority of the courses in the first semester (Anatomy, Dental Anatomy and Physiology related), third semester (Tooth morphology and occlusion, Human Diseases, Dental Biomaterials), fifth semester (Operative Dentistry, Population Oral Health, Clinical and Diagnostic Oral Sciences [CDOS1] and Child and Adolescent Oral Health [CAOH1]) and seventh semester (Adult Oral Health, CDOS2 and CAOH2) were rated as suitable for e-learning activities.

C. Opinion on Assessment via E-Learning Platforms

The Multiple Choice Questions (MCQ) and Short Answer Questions (SAQ) were the more common forms of assessments provided to the participants. However, some students (9%) had experience with essays too. These assessments were formative in nature whereby the questions were posted after each learning session in order to assess the effectiveness of the learning process. These questions were marked by the teachers or self-marked by the students. Sixty-nine percent (69%) felt that conducting assessments online is fair.

D. Opinion on Barriers Identified in E-Learning

Though the majority of first semester students had identified inadequate opportunities to work with other batch mates as a stumbling block, the majority of the seventh semester had identified technical difficulties in accessing the e-learning resources as the major barrier in e-learning. However, there was no significant difference in the responses between the genders and the semester of study of the student.

E. Suggestions to Improve E-Learning

When the students were requested to suggest ways to use the e-learning to improve the required skills, approximately half (52%) of the respondents preferred video demonstrations on clinical and lab procedures. Furthermore, multiple choice questions, use of charts to explain presentations, discussion forums, quizzes, assignments, web links and supplementary reading material were other ways of improving the learning experience as suggested by the participants. However, 3% of the students had clearly mentioned that e-learning could not be used to improve the required skills in dentistry.

There were multiple suggestions by the majority to improve the overall effectiveness of their e-learning experience. Some of the suggestions were uploading of lectures coupled with assignments, video demonstrations especially for practical scenarios, lectures followed by discussions and lectures on Zoom followed by case discussions. However, the statistical significance could not be assessed between genders or semesters as the number of responses for each option was small.

F. Analysis of Results According to the Respondents’ Current Semester

When the results were assessed according to the current semester, a majority of the first, third and seventh semesters students preferred video demonstrations for e-learning sessions. However, a majority of the fifth semester students preferred a combination of video demonstrations and a text document describing procedural steps or video demonstrations combined with YouTube links and discussion forums. Respondents from first semester had the highest overall satisfaction on e-learning materials they received and the difference between the semesters was statistically significant. (p=0.002). Over two-thirds of the seventh semester students (being the students who were exposed to conventional classroom learning experience for more than three years) confirmed that their overall e-learning experiences were better than the traditional classroom learning opportunities they received at the university and the difference was statistically significant (p=0.000) (Table 2). Further, students’ rating of e-learning materials received from teachers (p=0.002) (Figure 1) and encouragement to work on course material compared to the learning sessions received at the university (p=0.015) (Figure 2) were statistically significant when the semester of study were considered.

Table 2 describes participants’ response to the questionnaire according to their semester of study.

Table 2. Variations of participants’ response according to their semester of study (in percentages)

Figure 1 shows students’ rating of e-learning materials received from teachers according to their semester of study.

Figure 1. Students’ rating of e-learning materials received from teachers according to their semester of study

Figure 2 shows students’ feeling of encouragement to work on course material compared to conventional learning sessions received at the university.

Figure 2. Students’ feeling of encouragement to work on course material compared to conventional learning sessions received at the university according to their semester of study

When the results were analysed according to theory based courses (Anatomy, Oral Biology, Human Diseases, Tooth morphology and Occlusion, Dental Biomaterials) against practical, lab based and clinical training courses (Adult Oral Health, Population Oral Health, Clinical and Diagnostic Oral Sciences, Child and Adolescent Oral Health), the participants’ overall e-learning experience in theory based courses was significantly higher than practical and clinical courses (p= 0.03).

All data are available at https://figshare.com/s/c6c7b8f1d5cfd0076808.

V. DISCUSSION

E-learning is a broad term which encompasses the use of software applications to support learning, whether online or offline (Piccoli et al., 2001). The efficacy of an e-learning experience is dependent on many factors such as availability of devices, access to internet, knowledge on IT and uninterrupted internet connectivity (Asiry, 2017; Linjawi & Alfadda, 2018).

In the present study, half of the sample experienced some interruption during the e-learning sessions. It has been demonstrated in previous studies that students generally have a positive outlook to e-learning in spite of technical problems and related stresses (Brumini et al., 2014). Other studies have also highlighted positive experiences in isolated dental specialties (Bains et al., 2011; Gonzalez & Gadbury-Amyot, 2016; Qi et al., 2013; Reissmann et al., 2015). Our results revealed that students had a better learning experience with e-learning in comparison to traditional methods which was compatible with the findings of similar studies (Turkyilmaz et al., 2019). A study in Germany involving undergraduate dental students stated that participation in online learning was higher compared to “face-to-face” courses which was similar to the findings of the present study (Schlenz et al., 2020). Reasons for popularity of e-learning material over face-to-face learning are that it not only provides flexibility in the learning process (p=0.003) but also saves time (p = 0.012) (Anwar et al., 2021).

When the students’ perception on e-learning resources were analysed, males showed better IT literacy in comparison to female students. Even though the female students were not as confident as male students, a majority of them expressed that the overall e-learning experience was better than the conventional learning. These findings endorse the observations of Linjawi and Alfadda (2018) where the students claimed that they had better skills and motivation for e-learning than conventional learning activities. The identification of social media platforms for e-learning opportunities by a very few students in our study corroborate the fact that formal modalities of delivering e-learning resources should be implemented rather than relying on generic platforms.

Participants’ overall e-learning experience in theory based courses were significantly higher than practical and clinical courses due to the need of hands on experience, simulator training and working with patients for the practical and clinical courses than listening, reading and writing practice. This fact should be carefully considered when developing and planning the modes of content delivery as the knowledge and skills domains should be separately considered. More theory could be delivered through e-learning modalities and this time saved could be utilised for the face to face delivery of practical and clinical procedural teaching in predominantly skills based courses such as the Bachelor of Dental Surgery. However, the findings contradict the previous belief of the faculty that e-learning should only be used as a supplementary tool. There is a possibility to incorporate classroom and e-learning as a blended learning experience when imparting theory components in the future to save time, improve efficient use of limited infrastructure and to focus more on much needed skill development in dentistry. Incorporation of e-learning facilities to the maximum possible level should also be considered in future planning of the state universities in the delivery of undergraduate education in countries such as in Sri Lanka to enhance the quality of learning using the minimal physical and human resources available. Reports from experience in other countries highlight that e-learning processes are also cost effective, due to the availability of low cost or free online tools and platforms which can be used by teachers with a sound background of pedagogical knowledge (Goh, 2018).

First semester had the highest overall satisfaction on e-learning. This group consisted of students who hardly had any noteworthy face-to-face learning experience in the Faculty of Dental Sciences due to the abrupt closure during pandemic situation. Therefore, it seems that they could not effectively compare the two learning methods. Though less than first semester, more than 50% of the seventh semester students had a positive attitude towards e-learning experience. Our finding is in agreement with the findings of a previous study where it was shown that the perceived impact of e-learning and readiness for e-learning became less acceptable as students became mature (Linjawi & Alfadda, 2018).

Positive feedback regarding e-learning experience showed statistically significant differences between students of different semesters in relation to overall satisfaction on the e-learning materials received; encouragement to work on course materials; feeling that the teachers were friendly during the e-learning sessions; satisfaction on the student-teacher interaction during e-learning sessions. These findings should be considered when modes of learning for different components are planned by the Faculty of Dental Sciences during future revisions to the curriculum. Similarly, a recent report on undergraduate dental education during the COVID-19 pandemic has described that e-learning has been appreciated not only by students and teachers, but also in relation to the teacher-student interaction (Bennardo et al., 2020).

A majority of the students felt that the best way to receive the theory component was through PowerPoint presentation with voice narrations. This was statistically significant when compared among different semesters. This is in contrast with the preferred e-learning method among the Saudi Arabian dental students whereby online flash lectures were the most preferred method (Asiry, 2017).

Similar to our findings, a study on health care students belonging to 11 countries among the developed and developing nations also identified that 60% agreed that clinical and practical skills are best learned in clinics and laboratories (Abbasi et al., 2020). Another study involving over 3000 medical students of all years from over 13 schools in Libya has revealed only 21% agreed that e-learning could be used for clinical aspects, as compared with 55% who disagreed with this statement and 24% who were neutral (Alsoufi et al., 2020). The students ‘most preferred mode to learn skills as prescribed in the ILOs was through the use of procedural videos. A previous study carried out in the same institute to investigate the efficacy of procedural video versus live demonstrations revealed that that there was no appreciable difference in the development of skills in dental laboratory technology procedures (Thilakumara et al., 2018). This outcome could be applied to our findings that e-learning sessions, if followed by discussion forums with the teachers, as suggested by the participants would dramatically improve the e-learning experience.

E-learning with its virtual learning environment also requires different skills and a positive attitude from instructors. The positive attitude of the instructors towards technology, the interactive teaching style, and control over the technology were found to have an important influence on students’ reactions to the learning environment (Piccoli et al., 2001). This highlights the importance of professional development in two main aspects; technical and e-pedagogical areas (Conole & Oliver, 2002). The significance of institutional support should be highlighted. Therefore, the need of training for the teachers in the use of efficient methods for e-learning is emphasised. Furthermore, the faculty needs to have a long-term plan to encourage learning facilities through the use of different platforms, use of web links and production of supplementary video for demonstrations in order to encourage students towards a lifelong learning experience. Investment in more user-friendly and versatile software and resources to improve e-learning activities should be prioritised in order to make this endeavour more effective, interactive and receptive to the end users.

A. Limitations of the Study

Our study was performed in a single country and in a single institution as the country has only one dental school. Therefore, the results may not be generalised to other countries and to the region. The results must be validated through further multi-centre studies in order to obtain an overall opinion. Multi-national studies would provide a better understanding on how e-learning could replace the bulk of conventional dental undergraduate clinical learning and provide cost effective solutions for the developing countries with limited physical and human resources to avoid disruption of clinical training during difficult times such as the present COVID-19 pandemic. Another limitation is the cross-sectional nature of the study design, which limited our ability to identify causal associations.

VI. CONCLUSION

A positive learning experience was achieved through the e-learning modalities by the undergraduate dental students in comparison to traditional face-to-face learning. In terms of skills training, e-learning should only be considered as a supplementary tool. These findings will be useful when designing the study courses as a major part of theory component via e-learning mode and skill component with combined e-learning and conventional modes in the future.

We recommend adapting interactive e-learning platforms by using highly sophisticated technologies along with virtual clinical experience with combined clinical scenarios and assessments based on discussions of patient cases. Cost-effective solutions are needed to reduce disruption of undergraduate dental education especially in developing countries in the region like Sri Lanka and there should be new collective effort by the countries in the South Asian region in planning and sharing less expensive e-learning solutions in order to overcome financial issues. Such measures may vary from learning theory and virtual clinical and lab-based experience, followed by limited hands-on experience on live patients in a safe environment during difficult times of this nature.

Notes on Contributors

Principal and corresponding author (Rasika Jayasinghe) conceptualised the research and edited the initial draft and wrote the results after analysis. She also did the final approval of the version to be published and agreed to be accountable for all aspects of the work.

Manjula Attygalla edited the initial draft of the manuscript and supported in modification of the questionnaire. He also did the final approval of the version to be published and agreed to be accountable for all aspects of the work.

Manil Fonseka supported planning of the study, edited the research proposal and manuscript. He also did final editing and approval of the version to be published and agreed to be accountable for all aspects of the work.

Dhanushka Leuke Bandara supported planning of the study, designed the Google form, carried out the survey and entered the data in addition to editing of the manuscript. She also did final approval of the version to be published and agreed to be accountable for all aspects of the work.

Gishan Edirisinghe carried out the literature survey and did the final approval of the version to be published and agreed to be accountable for all aspects of the work.

Ruwan Jayasinghe supported planning of the study, prepared the initial draft of the research proposal and edited the final draft of the manuscript. He did the final approval of the version to be published and agreed to be accountable for all aspects of the work.

Indika Thilakumara supported planning of the study, edited the first draft and wrote the discussion. She did the final approval of the version to be published and agreed to be accountable for all aspects of the work.

Ethical Approval

Ethical clearance has been granted by Ethics Review Committee, Faculty of Dental Sciences, University of Peradeniya. (ERC/FDS/UOP/I/2020/21).

Data Availability

All data are available at https://figshare.com/s/c6c7b8f1d5cfd0076808 and can be accessed on request and approval from the corresponding author.

Funding

No funding sources are associated with this study.

Declaration of Interest

Authors declare that they do not have possible conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Abbasi, M. S., Ahmed, N., Sajjad, B., Alshahrani, A., Saeed, S., Sarfaraz, S.,   Alhamdan, R. S.,  Vohra, F., & Abduljabbar, T. E. (2020). E-Learning perception and satisfaction among health sciences students amid the COVID-19 pandemic. Work, 67(3), 549-556. https://doi.org/10.3233/WOR-203308

Alsoufi, A.,  Alsuyihili, A.,  Msherghi, A.,   Elhadi, A., Atiyah, H.,  Ashini, A.,  Ashwieb, A.,  Ghula, M.,   Hasan, H. B., Abudabuos, S.,  Alameen, H.,  Abokhdhir, T.,  Anaiba, M.,  Nagib, T.,   Shuwayyah, A.,  Benothman, R., Arrefae, G.,   Alkhwayildi, A.,  Alhadi, A., … Elhadi, M. (2020). Impact of the COVID-19 pandemic on medical education: Medical students’ knowledge, attitudes, and practices regarding electronic learning. PLOS ONE, 15(11), e0242905. https://doi.org/10.1371/journal.pone.0242905

Anwar, A.,   Mansoor, H., Faisal, D., & Khan, H. S. (2021). E-learning amid the COVID-19 lockdown: Standpoint of medical and dental undergraduates. Pakistani Journal of Medical Sciences, 37 (1), 217-222. https://doi.org/10.12669/pjms.37.1.3124

Asiry, M. A. (2017). Dental students’ perceptions of an online learning. The Saudi Dental Journal, 29(4), 167–170. https://doi.org/10.1016/j.sdentj.2017.03.005

Bains, M., Reynolds, P. A., McDonald, F., & Sherriff, M. (2011). Effectiveness and acceptability of face-to-face, blended and e-learning: A randomised trial of orthodontic undergraduates. European Journal of Dental Education, 15(2), 110–117. https://doi.org/10.1111/j.1600-0579.2010.00651.x

Bennardo, F., Buffone, C., Fortunato, L., & Giudice, A. (2020). COVID-19 is a challenge for dental education—A commentary. European Journal of Dental Education, 24(4), 822-824. https://doi.org/10.1111/eje.12555 

Brumini, G., Špalj, S., Mavrinac, M., Biočina-Lukenda, D., Strujić, M., & Brumini, M. (2014). Attitudes towards e-learning amongst dental students at the universities in Croatia. European Journal of Dental Education, 18(1), 15-23. https://doi.org/10.1111/eje.12068

Chonkar, S. P., Qi, H. L. C., Ha, T. C., Lim, M., Ng, M. J., & Tan, K. H. (2019). Can digital media affect the learning approach of medical students? The Asia Pacific Scholar, 4(1), 13- 23. https://doi.org/10.29060/TAPS.2019-4-1/OA1058

Conole, G. C., & Oliver, M. (2002). Embedding theory into learning technology practice with toolkits. Journal of Interactive Media in Education (8), 2- 3. https://doi.org/10.5334/2002-8

Goh, P. S. (2018). eLearning in medical education – Costs and value add. The Asia Pacific Scholar, 3(2), 58- 60. https://doi.org/10.29060/TAPS.2018-3-2/PV1073

Gonzalez, S. M., & Gadbury-Amyot, C. C. (2016). Using Twitter for teaching and learning in an oral and maxillofacial radiology course. Journal Dental Education, 80(2), 149–155. https://doi.org/10.1002/j.0022-0337.2016.80.2.tb06070.x

Gunawardane, S., & Wijekoon, P. (2017). Changes in perceived computer use and knowledge in Sri Lankan dental students in the last ten years. Scholars Journal of Dental Sciences, 4(5), 216-219.

Hillenburg, K. L., Cederberg, R. A., Gray, S. A., Hurst, C. L., Johnson, G. K., & Potter, B. J. (2006). E-learning and the future of dental education: Opinions of administrators and information technology specialists. European Journal of Dental Education, 10, 169–177. https://doi.org/10.1111/j.1600-0579.2006.00413.x

Li, C., & Lalani, F. (2020). The COVID-19 pandemic has changed education forever. This is how. World Economic Forum. https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning

Linjawi, A. I., & Alfadda, L. S. (2018). Students’ perception, attitudes, and readiness toward online learning in dental education in Saudi Arabia: A cohort study. Advances in Medical Education and Practice, 9, 855–863. https://doi.org/10.2147/AMEP.S175395

Piccoli, G., Ahmed, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25(4), 401-426. https://doi.org/10.2307/3250989

Qi, S., Yan, Y., Li, R., & Hu, J. (2013). The impact of active versus passive use of 3D technology: A study of dental students at Wuhan University, China. Journal of Dental Education, 77(11), 1536–1542.

Reissmann, D. R., Sierwald, I., Berger, F., & Heydecke, G. (2015). A model of blended learning in a preclinical course in prosthetic dentistry. Journal of Dental Education, 79(2), 157–165. https://doi.org/10.1002/j.0022-0337.2015.79.2.tb05870.x

Schlenz, M. A., Michel, K., Wegner, K., Schmidt, A., Rehmann, P., & Wöstmann, B. (2020). Undergraduate dental students’ perspective on the implementation of digital dentistry in the preclinical curriculum: A questionnaire survey. BMC Oral Health, 20, 78. https://doi.org/10.1186/s12903-020-01071-0

Thilakumara, I. P., Jayasinghe, R. M., Rasnayaka, S. K., Jayasinghe, V. P., & Abeysundara, S. (2018). Effectiveness of procedural video verses live demonstrations in teaching laboratory techniques to dental students. Journal of Dental Education; 82(8), 898-904. https://doi.org/ 10.21815/JDE.018.086

Turkyilmaz, I., Hariri, N. H., & Jahangiri, L. (2019). Student’s perception of the impact of e-learning on dental education. Journal of Contemporary Dental Practice, 20(5), 616-621.

Venkatesh, S., Rao, Y. K., Nagaraja, H., Woolley, T., Alele, F. O., Bunmi, S., & Malau-Aduli, B. S. (2019). Factors influencing medical students’ experiences and satisfaction with blended integrated e-learning. Medical Principles and Practice, 29, 296-402. https://doi.org/10.1159/000505210

Wong, M. L., Lee, T. W. O., Allen, P. F., & Foong, K. W. C. (2020). Dental education in Singapore: A journey of 90 years and beyond. The Asia Pacific Scholar, 5(1), 3- 7. https://doi.org/10.29060/TAPS.2020-5-1/GP1086

*RM Jayasinghe
Department of Prosthetic Dentistry,
Faculty of Dental Sciences,
University of Peradeniya
Tel: +94777806314
Email: manorija@pdn.ac.lk/ manoripathiraja@yahoo.com

Submitted: 19 July  2020
Accepted: 7 October 2020
Published online: 13 July, TAPS 2021, 6(3), 67-74
https://doi.org/10.29060/TAPS.2021-6-3/OA2347

Tess Lin Teo, Jia Hao Lim, Choon Peng Jeremy Wee & Evelyn Wong

Department of Emergency Medicine, Singapore General Hospital, Singapore

Abstract

Introduction: Singapore experienced the COVID-19 outbreak from January 2020 and Emergency Departments (ED) were at the forefront of healthcare activity during this time. Medical students who were attached to the EDs had their clinical training affected.

Methods: We surveyed teaching faculty in a tertiary teaching hospital in Singapore to assess if they would consider delivering clinical teaching to medical students during the outbreak and conducted a thematic analysis of their responses.

Results: 53.6% felt that medical students should not undergo clinical teaching in the ED and 60.7% did not wish to teach medical students during the outbreak. Three themes arose during the analysis of the data – Cognitive Overload of Clinical Teachers, Prioritisation of Clinical Staff Welfare versus Medical Students, and Risk of Viral Exposure versus Clinical Education.

Conclusion: During a pandemic, a balance needs to be sought between clinical service and education, and faculty attitudes towards teaching in high-risk environments can shift their priorities in favour of providing the former over the latter.

Keywords:           Disease Outbreak, Pandemic, Faculty, Medical Students, Attitudes, Clinical Teaching, Emergency Medicine

Practice Highlights

• In a pandemic, a balance needs to be sought between clinical education and risking learner exposure to the virus.
• A crisis situation can affect educators’ priorities and attitudes towards the provision of clinical education, in favour of providing crucial clinical services.

Since the first reported cases of COVID-19 infections in Wuhan, in December 2019, the month of January 2020 saw Singapore’s Ministry of Health (MOH) issue guidelines and implement a series of calibrated defensive measures to reduce the risk of imported cases and community transmission (Lin et al., 2020; WHO, 2020). Singapore has a Disease Outbreak Response System Condition (DORSCON) framework, which guides the nation’s response to various emerging infectious diseases outbreaks. The four-level colour-coded system of Green, Yellow, Orange and Red, describes the increasing severity of the outbreak in the community  (Quah et al., 2020).

The Department of Emergency Medicine (DEM) of Singapore General Hospital saw 130 000 visits in 2019 (SGH, 2019). It hosted 158 medical students (MS) through the year. Aside from some elective students, the majority were in their second year of clinical postings. Formal clerkships consisted of four weeks of clinical exposure in which they were expected to clerk and present cases to teaching faculty and perform minor procedures such as intravenous cannulation and insertion of bladder catheters etc., with about nine hours of classroom tutorials.

In early January 2020, DORSCON yellow was declared, indicating either a severe outbreak outside Singapore or that the disease was contained locally with no significant community spread (Quah et al., 2020). All DEM staff were required to wear personal protective equipment (PPE). Hospital elective surgeries were postponed. Other outbreak measures included setting up new isolation areas for patients. DEM staff had their leave embargoed to ensure that there was adequate manpower to staff these areas in anticipation of a gradually worsening outbreak (Chua et al., 2020).

On 7 February 2020, the outbreak alert rose to DORSCON Orange (DO) as there were cases of community transmissions (Quah et al., 2020). Based on previous experience managing the Severe Acute Respiratory Syndrome (SARS) outbreak 17 years prior, the DEM transitioned to an Outbreak Response Roster, where physicians and nurses of the DEM were split into teams that worked 12 hour shifts, with no overlapping shifts, hence limiting staff contact to only those within their teams (Chua et al., 2020). With DO in effect, the department needed to come to a rapid decision about whether or not to accept MS in the ED. A group of 12 MS that the DEM was supposed to host this April already had their clerkship cancelled due to concerns of breaching infection control and safe distancing measures. There have been no studies to date on faculty attitudes towards clinical teaching of MS during a pandemic, although papers have been published about students’ attitudes towards clinical training during disease outbreaks. The Clerkship Director conducted a short and focused survey amongst the faculty between the 27th-29th of March, amidst rising public concerns that the country might soon be locked down, to explore their attitudes on having MS clerkships during the COVID-19 pandemic. The results of this survey allowed the Director to quickly understand the sentiments of the faculty and thus decided that an entirely remote, online teaching program would be created instead. 9 days after the survey, on the 7th of April, the Singapore government officially announced the implementation of a lockdown, known locally as a ‘circuit breaker’ (Quah et al., 2020).

II. METHODS

Clinical teachers of the DEM were issued an anonymous survey over a period of three days via an online survey tool, SurveyMonkey (www.surveymonkey.com). Participants were informed prior to completing the survey that it was anonymous, and by proceeding with the survey they consented to the results being used for research purposes. The data collected included their professional appointments in the department and two yes/no questions: “Do you think medical students should be performing their EM clerkship during DO?” and “Are you keen on teaching MS clinically during DO?”. Participants answering “No” to the latter were asked to elaborate. All participants were asked to write about any concerns they had about having MS in the emergency department (ED) during DO. No other personal identifying information was sought. The survey was deliberately kept short and easy to answer to promote staff participation within the short timeframe the DEM had to make the decision about accepting students. Informed consent was waived as per the Institutional Review Board (IRB).

A simple descriptive quantitative analysis of responses to the 1^st two yes-no questions identified the overarching sentiment of the department towards hosting MS during DO and was followed by a thematic analysis of the free-text answers to the last two open-ended questions (Braun & Clarke, 2006).

As many participants used the last question (‘any other comments?’) to emphasise or elaborate on the preceding question (‘why aren’t you keen to teach?’), the majority of the qualitative data gathered pertains to the issues of having MS in the department during DO. There was a paucity of data detailing why participants were in favour of teaching MS, as the survey did not specifically ask this. Hence, the authors chose to focus on analysing the responses of participants who were not keen to teach during this time. This analysis yielded three different themes. However, a small number of respondents supportive of MS felt strongly about teaching and volunteered their reasons in response to the last question. While this data is insufficient to support a robust thematic analysis, a small section is included at the end in order to present as complete a discussion as possible.

III. RESULTS

A. Participant Background

Participants consisted of Emergency Medicine (EM) specialists, permanent registrars or middle grade staff and EM senior residents. These groups were chosen because they each hold significant roles, such as being named supervisors or clinical instructors of MS, and have considerably more contact time with MS in the DEM as opposed to nursing staff or junior doctors.

B. Quantitative Results

A total of 28 out of 45 (62.3%) responses were recorded. Except for two individuals, all other respondents in favour of hosting MS in the ED during DO (46.4%) were also keen to teach them. About two-thirds of the participants (60.7%) were not keen to teach MS during DO. However, of this latter group, 23.5% of respondents offered (without prompting) a compromise – where they proposed teaching only during the relatively less busy night shifts, in their response to ‘Any other comments?’  Table 1 shows the breakdown of responses.

C. Qualitative Results – Reasons Against

Each of the three themes presented here begins with a short paragraph that describes the situational context in which this survey took place, followed by a series of selected statements, and ends with a general summary and discussion of the responses within the respective theme. In order to maintain the authenticity of the data, each response is reproduced verbatim, sometimes in Singlish, the local colloquial variety of Standard Singaporean English (Bokhorst-Heng, 2005). Any edits to the text for clarification purposes have been clearly identified.

1) First theme: Cognitive overload of clinical teachers– There is only so much one can handle: Emergency physicians are no strangers to high stress environments, and are aware that as frontline workers they will be at the forefront in dealing with any emerging infectious disease. The move into DO represented the shifting of the local virus epidemiology from predominantly imported cases that could be easily identified and isolated, into the community-at-large. With this shift came changes to existing workflows and the re-arrangement of department space to form isolation areas for treating potential infectious cases. The implementation of a strict team-based roster described earlier meant that almost half the entire department would not physically meet the other half, and a surge in manpower requirements saw many junior doctors from other departments being rotated into the ED to help tackle the increased clinical load. Being new to the DEM, these new doctors required more supervision and assistance in adapting to the unfamiliar work environment. Responses that supported this theme include:

“High clinical load, long hours. Already cognitively overloaded. Not conducive for teaching. New [junior doctors] need to be taught also.”

Participant #6, Specialist

“Focus on daily evolving challenge first.” and in response to the last question “Please no.”

  Participant #2, Senior Resident 

“During DORSCON ORANGE we are in stress, if clinical teaching sessions start then other [doctors’] stress and workload level will increase.”

Participant #25, Staff registrar

“May be more a hassle if we have to look after the new [junior doctors] rotating and students [as well].”

Participant #4, Specialist

“We are also in a 12-hour outbreak roster which is physically, emotionally and mentally draining. Teaching students in this environment is far from ideal” and in response to the last question “Am fairly strongly against this idea”.

Participant #8, Specialist

“Day shifts no bandwidth to teach […] also can’t pay attention to [medical students] during day shifts, too tiring and too busy […] but I feel I can’t do [medical students] justice because I can’t debrief after a shift either, too tired.”

Participant #17, Specialist

Many of these responses conveyed a sense of exhaustion, reflecting the toll that constant workflow changes, longer work hours and relative social isolation was taking on the faculty. Teaching and supervising MS appeared to be viewed as a “hassle” or “extra work”, an additional drain to a clinical faculty’s energy during a busy and stressful shift.

This brought the department to a discussion on the provision of clinical services versus clinical education – whether teaching the next generation of future doctors was as important as treating the patient in front of us. One school of thought held that as clinician educators, physicians should – as the name implies – be clinicians first before educators. However, the interplay between these two roles is likely dependent on the faculty’s attitudes towards learners, as will be described later. Being cognitively overloaded naturally results in a shuffling of One’s priorities, which is seen next.

2) Second theme: Prioritisation of staff welfare – whose welfare is more important, staff or students? : It is well known that mental health can be adversely affected in crisis situations, and as the COVID-19 situation unfolded, boosting morale and maintaining the welfare of all staff became an important consideration (Matsuishi et al., 2012; McAlonan et al., 2007). At the forefront of this effort was the need to provide the staff with a supply of good quality personal protective equipment (PPE) so the staff would feel safe and confident in existing infection control measures. Although Singapore had yet to experience a shortage of PPE, there was still a concerted effort made by all hospitals to conserve these resources. Staff wellness was a theme seen in several responses:

“[I] can’t do the [junior doctors] justice because having a [medical student] attached to them is another stressor in an already stressful shift.”

Participant #17, Specialist

“Having to keep our doctors and nurses safe takes up a lot of energy. Students are young and naïve and will require even more time and resources to ensure they are safe.”

Participant #22, Specialist

“Furthermore, they will need to use PPE and again this should be conserved during the period of the outbreak.”

Participant #27, Specialist

“Medical students are important for future but I feel staff currently working in the department should be look after well.”

 Participant #25, Staff registrar

“Waste PPE.”

Participant #20, Specialist

The importance of conserving PPE during a pandemic is undisputed and the concern that MS would use them up is valid. It was interesting to note in these responses hints of an “us-versus-them” mentality, where MS were seen as competition for the limited resources of PPE, time, and energy. Students were not viewed as part of the DEM team and perceived more as “stressors”, who required attention because they were “young and naïve”, and their use of PPE was viewed as a “waste”. This identification of an “in-group” of staff and an “out-group” of students led to a prioritisation or favouring of the former over the latter. This behaviour can be explained by the Social Identity Theory (SIT), which states that part of an individual’s self-image or self-concept is derived from the social groups to which they perceive themselves to belong to (Hogg & Reid, 2006; Tajfel & Turner, 1979). Thus, in order to maintain a positive self-image, there is a tendency for people to favour the in-group and discriminate against the out-group. This phenomenon was famously demonstrated by Tajfel et al in their Minimal Group Paradigm studies, which essentially showed that the mere perception of belonging to one of two distinct groups was enough to trigger social discrimination between the groups (Tajfel et al., 1971). Behaviour like this is indicative that a significant number of the department hold the belief that there is a distinct divide between students and staff, rather than seeing MS as belonging to the wider group of the medical fraternity. Creating such a divide between staff and student is problematic because it hinders effective teaching, especially because MS will eventually transition from the “out-group” of students to the “in-group” of staff upon graduation, and clinician educators are responsible for providing a safe environment for them to learn in. However, beyond this discussion of intergroup competition, there were concerns amongst the faculty with regards to the appropriateness of siting clinical learning in the high-risk, front-line location of the ED in a pandemic, as discussed in the next theme.

3) Third Theme: Risk of viral exposure vs clinical education – What is the price to pay and who pays it?: During the initial period of DO, medical schools pulled MS out of the clinical environment and moved to online learning, with the aim of protecting them from unnecessary exposure to the virus and for safe distancing. However, when they proposed that students be allowed back into the hospitals after undergoing PPE training, this risk of exposure had not changed, as the number of positive cases was rising daily still. Responses that reflected this theme included:

“Don’t think it’s appropriate to have students around in a high-risk environment.”

Participant #4, Specialist

“Having medical students around not only will expose them to infection it will also compromise the rest of the staff in the event of a breach in infection [protocols]. Also, them just hanging around & not allowed to have hands-on [participation] in the procedures, clerking, [patient] contact etc will not be of any benefit [to them] at this time.”

Participant #7, Senior Resident

“Student safety issue. No minder to ensure students’ adherence to strict PPE as Doctors and Nurses will be busy with clinical service.”

Participant #11, Specialist

“I think medical students are not providing clinical care to patients and having them in the ED increases risk to patients (without the attendant benefits) and increases risk to themselves (without the moral obligation to do so as doctors) and their family.”

Participant #27, Specialist

“Can students be [held] responsible for their own health? Or the school or the department? As doctors, we know it as our duty and occupational hazard. But as students – their duty is to learn (best done in a safe environment), not put their health at risk.”

 Participant #6, Specialist

Responses that addressed the risk of virus exposure in the ED could be divided into two groups –those that were predominantly concerned about the students themselves catching the virus, and those that were more concerned about the consequences of such an event. The risk of catching the virus was seen as too high a cost – one that was borne not only by the individual student but by the patients and the staff as well. The benefits of clinical bedside instruction were called into question, as students’ movements would be restricted to low or medium-risk areas only. More than one participant raised the potential issue of students breaching infection control protocol or needing supervision in donning their PPE, despite reassurances given that schools would send MS for PPE training. This reflected a lack of trust in MS – themselves adult learners – who could be reasonably expected to understand the importance of infection control protocols. It begs the question of how big a role the educator plays in the personal safety of a MS and that of the patients and staff they interact with.

D. Qualitative Results – Reasons For

The survey design did not specifically ask responders about their reasons for supporting teaching MS during this pandemic. However, some participants felt strongly enough about this to advocate for clinical postings. Their reasons are shared below.

1) Theme: For the sake of tomorrow – In defence of teaching amidst a crisis:

“I feel we can still provide a meaningful learning experience for these students. We just need to lay out clear instructions and precautions for them to follow. It is a good opportunity to show to students how emergency medicine is adaptive, versatile, and for them to appreciate how quickly workflows can change, or how triage works in a disaster setting.”

Participant # 15, Specialist

“The way it is done has to be different […] the traditional method of teaching, where the students look to the seniors and may expect some form of spoon feeding […] Only when this mind-set is removed, will the tutors […] look at them as part of the team and incorporate them […], and will students see […] themselves as Drs to be [sic], practice safe habits from the very start and protect themselves as the patient’s doctor. This sense of ownership, accountability, professionalism can be started from that stage as a medical student. This is the perfect opportunity to state that this is what is expected and groom them likewise.”

Participant #19, Specialist 

“I feel that the teaching should as much as possible be a simulation of working life and that working in high-risk areas such as these gives a semblance of pressure which cultivates good habits such as mindfulness of hand hygiene, donning of PPE etc.”

Participant #26, Senior Resident

The responses share a commonality of seeing the pandemic as an opportunity for modelling positive attitudes that would benefit the student in the future. This point of view advocates for the acknowledgement of the realities of being a doctor and assumes that students are already part of the “in-group” of the medical team rather than the “out-group” as seen in the earlier discussion.

IV. DISCUSSION

A. Limitations

This study has its limitations, chiefly being the lack of qualitative data representing the opinions of those who were keen to teach MS as the initial survey was conducted with the purpose of gauging whether or not the department would be open to receiving MS during DO. This lack of data meant that this study is at best a one-sided representation of the department’s opinion.

Additionally, all four of the authors have a keen interest in the education of MS and two of the authors are actively involved in faculty development. They were all both participants in the study as well as its evaluators. Prior to evaluation of results, the authors themselves suspected that majority of the faculty would be too overwhelmed with the changes the pandemic wrought to want to teach students, which may have contributed to confirmation bias in the analysis of the data. However, throughout the analysis, every attempt was made to ensure that the themes uncovered remained true to the data, and much of the original data was reproduced here faithfully to maintain transparency, such that the reader may draw their own conclusions.  

Another limitation of the study was that the survey was unable to measure shifts in the attitudes of faculty as the pandemic evolved, which would have allowed us to understand the amplitude of the effect of the pandemic itself more clearly.

B. New Insights

It was worth noting that nearly two-thirds of the department did not want to teach MS during DO, despite each participant having taught MS routinely prior to this pandemic. Initial analysis of the reasons given for this refusal revealed three distinct themes of Cognitive Overload of Teachers, the need to Prioritise Staff Welfare and the Risk of Viral Exposure to Students – themes that are transferrable to many departments involved in pandemic response, regardless of locality.

Expounding further on this topic, it can be seen in some of the responses detailed under the themes of Cognitive Overload and Prioritising Staff Welfare, that there was a perceived increase in the need to supervise the new junior doctors rotating into the department on short notice (as opposed to the junior doctors who were already in the middle of their rotation and thus more familiar with the department’s protocols). This supervision is an important component of the continuing clinical education of junior doctors, which in itself is part of a larger debate surrounding the competing aims of clinical service versus clinical education that has been ongoing for many years (Woods, 1980). It is often the case in EM that when overwhelmed with patients, clinical education is sacrificed for clinical service without much short-term complications. Indeed, even amongst EM residents, more research is needed to define the optimal balance between service and education (Quinn & Brunett, 2009). However, a pandemic presents a rather unique situation in that most junior doctors will not have worked in a pandemic before. Thus, the need to educate junior doctors on both pandemic response and the importance of personal safety – with its direct impact on patient safety – now cannot be sacrificed without directly affecting the provision of clinical service.

It is beyond the scope of this paper to comment on whether educating MS on pandemics through clinical immersion programs during a pandemic better prepares them for future outbreaks, or in the broader sense, whether the clinical education of today’s MS by immersive learning can bolster the clinical service of tomorrow’s junior doctors. In fact, it seems almost premature to consider this question given the paternalistic attitude many of our faculty appeared to have towards students, perceiving them as learners to be looked after – to the extent that they could not even be trusted with their own safety and that of the patients and staff they interact with. Interestingly, this view seems to be shared by MS themselves – an electronic survey conducted at one of Singapore’s medical schools showed that a third of currently enrolled MS were concerned that they might introduce possible risks to the patient should they return to the clinical setting (Compton et al., 2020). These findings are indicative of a more deeply rooted mindset in which the social hierarchy draws a clear line between Teacher and Student. This becomes clearer when one considers that in Confucian Heritage Cultures such as Singapore (Biggs, 1998), the teacher holds great authority and students brought up in such cultures tend to defer to such authority as a matter of course (Ho, 2020). Given the multiple factors that contribute to this debate, it is unlikely that we will be able to arrive at a clear answer without further research, but what is certain is that medical students are not essential workers and, in a pandemic, medical schools need to balance their educational needs and ethical obligations to keep students safe (Menon et al., 2020).

Within our paper, it is heartening that many participants who were not keen to teach still tried to offer a compromise of teaching during the relatively less busy night shifts instead, and that 46% of our department were willing to accept MS during this period. COVID-19 allowed us to uncover some of the underlying attitudes towards MS and to consider them in the context of Singapore’s cultural heritage. These attitudes are important for us to address if we are to improve the delivery of medical education in the ED and we would like to invite the reader to consider whether the same uncovering has occurred in their respective departments.

V. CONCLUSION

The balance between clinical service and clinical education is a precarious one that appears to shift quickly in favour of the former in the high-risk environment of an evolving pandemic, which presents significant challenges even for experienced educators to overcome. As seen in our paper, cognitive overload of educators and the need to prioritise the welfare of junior staff inexperienced in pandemic response takes clear precedence over the education of MS. The paternalistic view that majority of our faculty hold leads to doubts about the ability of MS to keep themselves and their patients safe from virus exposure, doubts that are surprisingly shared by MS as well, and is indicative of the social hierarchy deeply ingrained in Confucian Heritage Cultures such as Singapore and surrounding countries in the region, where students tend to defer to authority as a matter of course. In order to improve as medical educators, we must place further effort into uncovering the underlying attitudes of both faculty and MS and address them in ways specific to our cultural heritage.

Notes on Contributors

Author Teo TL analysed the transcripts, conducted the primary thematic analysis and wrote the manuscript. Author Lim JH co-wrote the manuscript. Author Wee JCP conducted the literature review and developed the manuscript. Author Wong E designed and conducted the study, performed the data collection and developed the manuscript. All the authors have read and approved the final manuscript.

Ethical Approval

IRB approval for this study was obtained (SingHealth CRIB reference number 2020/2134).

Acknowledgement

The authors would like to acknowledge all participants of the survey.

Funding

No funding sources are associated with this study. 

Declaration of Interest

All authors work in SGH DEM and answered the survey as participants.

References

Biggs, J. (1998). Learning from the confucian heritage: So size doesn’t matter? International Journal of Educational Research, 29(8), 723–738. https://doi.org/10.1016/S0883-0355(98)00060-3

Bokhorst-Heng, W. D. (2005). Debating singlish. Multilingua, 24(3), 185–209. https://doi.org/10.1515/mult.2005.24.3.185

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa 

Chua, W. L. T., Quah, L. J. J., Shen, Y., Zakaria, D., Wan, P. W., Tan, K., & Wong, E. (2020). Emergency department ‘outbreak rostering’ to meet challenges of COVID-19. Emergency Medicine Journal, 37(7), 407–410. https://doi.org/10.1136/emermed-2020-209614

Compton, S., Sarraf-Yazdi, S., Rustandy, F., & Radha Krishna, L. K. (2020). Medical students’ preference for returning to the clinical setting during the COVID-19 pandemic. Medical Education, 54(10), 943–950. https://doi.org/10.1111/medu.14268 

Ho, S. (2020). Culture and learning: Confucian heritage learners, social-oriented achievement, and innovative pedagogies. In C. Sanger & N. Gleason (Eds.), Diversity and inclusion in global higher education (pp. 117–159). Palgrave Macmillan. https://doi.org/10.1007/978-981-15-1628-3

Hogg, M. A., & Reid, S. A. (2006). Social identity, self-categorization, and the communication of group norms. Communication Theory, 16(1), 7–30. https://doi.org/10.1111/j.1468-2885.2006.00003.x

Lin, R. J., Lee, T. H., & Lye, D. C. B. (2020). From SARS to COVID-19: The Singapore journey. The Medical Journal of Australia, 212(11), 497-502.e1. https://doi.org/10.5694/mja2.50623

Matsuishi, K., Kawazoe, A., Imai, H., Ito, A., Mouri, K., Kitamura, N., Miyake, K., Mino, K., Isobe, M., Takamiya, S., Hitokoto, H., & Mita, T. (2012). Psychological impact of the pandemic (H1N1) 2009 on general hospital workers in Kobe. Psychiatry and Clinical Neurosciences, 66(4), 353–360. https://doi.org/10.1111/j.1440-1819.2012.02336.x

McAlonan, G. M., Lee, A. M., Cheung, V., Cheung, C., Tsang, K. W. T., Sham, P. C., Chua, S. E., & Wong, J. G. W. S. (2007). Immediate and sustained psychological impact of an emerging infectious disease outbreak on health care workers. Canadian Journal of Psychiatry, 52(4), 241–247. https://doi.org/10.1177/070674370705200406

Menon, A., Klein, E. J., Kollars, K., & Kleinhenz, A. L. W. (2020). Medical students are not essential workers: Examining institutional responsibility during the COVID-19 pandemic. Academic Medicine, 95(8), 1149–1151. https://doi.org/10.1097/ACM.0000000000003478

Quah, L. J. J., Tan, B. K. K., Fua, T. P., Wee, C. P. J., Lim, C. S., Nadarajan, G., Zakaria, N. D., Chan, S. J., Wan, P. W., Teo, L. T., Chua, Y. Y., Wong, E., & Venkataraman, A. (2020). Reorganising the emergency department to manage the COVID-19 outbreak. International Journal of Emergency Medicine, 13(1), 32. https://doi.org/10.1186/s12245-020-00294-w

Quinn, A., & Brunett, P. (2009). Service versus education: Finding the right balance: A consensus statement from the council of emergency medicine residency directors 2009 academic assembly “Question 19” working group. Academic Emergency Medicine, 16(SUPPL. 2), 15–18. https://doi.org/10.1111/j.1553-2712.2009.00599.x

SGH. (2019). Hospital Overview – Singapore General Hospital.  Retrieved August 17, 2020, from https://www.sgh.com.sg/about-us/corporate-profile/Pages/hospital-overview.aspx

COVID-19 (Temporary Measures) Act 2020. (2020). https://sso.agc.gov.sg/Act/COVID19TMA2020

Tajfel, H., Billig, M. G., Bundy, R. P., & Flament, C. (1971). Social categorization and intergroup behaviour. European Journal of Social Psychology, 1(2), 149–178. https://doi.org/10.1002/ejsp.2420010202

Tajfel, H., & Turner, J. C. (1979). An integrative theory of intergroup conflict. In W. G. Austin & S. Worchel (Eds.), The social psychology of intergroup relations (pp. 33–47). Brooks/Cole.

WHO. (2020). IHR emergency committee for pneumonia due to the novel coronavirus 2019-nCoV transcript of a pressing briefing. Retrived January 30, 2020, from https://www.who.int/docs/default-source/coronaviruse/transcripts/ihr-emergency-committee-for-pneumonia-due-to-the-novel-coronavirus-2019-ncov-press-briefing-transcript-30012020.pdf?sfvrsn=c9463ac1_2

Woods, D. (1980). Service and education in residency programs. A question of balance. Canadian Medical Association Journal, 123(1), 44.

*Evelyn Wong
Department of Emergency Medicine,
Singapore General Hospital
Outram Road
Singapore 169608
Email: evelyn.wong@singhealth.com.sg