Submitted: 10 January 2022
Accepted: 22 April 2022
Published online: 5 July, TAPS 2022, 7(3), 23-32
https://doi.org/10.29060/TAPS.2022-7-3/OA2742

Tun Tun Naing¹, Yuriko Minamoto², Ye Phyo Aung¹ & Marlar Than¹

¹Department of Medical Education, Defence Services Medical Academy, Myanmar, ²Meiji University, Tokyo

Abstract

Introduction:  With the evolution of healthcare needs for the community and the changing trends in medical education in the 21^st century, medical educators need to be prepared for their tasks in the coming decades. Medical educator training is crucial but other factors can also affect the development of their competency. This study aims to measure the impact of the medical educators’ training course and find out the key challenges encountered by the medical educators in Myanmar.

Methods: A retrospective quantitative design was conducted on 45 respondents by four levels of Kirkpatrick’s model assessment consisting of 39 statements and 9 items of key challenges, using five-point Likert scale. The item scores were analysed as mean and standard deviation, ‘t’ test and ANOVA were used for relationship between impact of training and demographic background.

Results: There was significant association between the impact of training and the educational background (p=0.03), job position (p=0.02), and academic year attended (p=0.03). The respondents distinctly agreed that the training increased their knowledge and attitudes and that they could apply the learnt lessons practically in their workplace (minimum 3.75±0.60 and maximum 4.28±0.50). Regarding the key challenges, respondents viewed that their institution needed to support more scholarship opportunities and academic recognition; encourage networking and strengthen ICT-based medical education system (minimum 2.55±0.84 – maximum 4.17±0.71).

Conclusion: This study indicates that enhancing the competency of medical educators with medical educator training programs is effective and useful; but inadequacy of institutional support for faculty development and internet facilities posed challenges in the overall faculty development.

Keywords:           Medical Education, Faculty Development, Impact of Medical Educator’s Training, Kirkpatrick’s Model, Key Challenges

Practice Highlights

• Medical educator training program is crucial for enhancing competency in medical educators.
• Medical educator training program inspires medical educators to apply their knowledge and skills in their daily departmental activities.
• Beyond training, medical schools must address a balance of capacity for teaching, research and service functions for their faculty.
• Encouraging institutional support such as reward, recognition, and appreciation for their performance should be encouraged as an admirable institutional culture.
• Inadequacy of institutional support for faculty development and internet facilities posed challenges in the overall faculty development.

I. INTRODUCTION

Nowadays, with the evolution of healthcare needs for the community and changes in medical education trends, faculty development in medical education is challenging.  Sheets and Schwenk (1990) explained that any activity that enhances the knowledge and skills of individual faculty members are very fundamental to their performance in a department or residency program such as teaching skills, clinical skills, research and administrative skill (Sheets & Schwenk, 1990). Developing the skills of faculty members is not effortless. The ability to teach is not inborn, although the graduate from medical school is supposed to be capable of teaching (McLean et al., 2008). Besides content, teaching involves ‘process,’ and to develop the ‘art’ of teaching, academics required to support (Benor, 2000).

The present-day teacher must be aware of and become part of the far- reaching changes taking place in medical education. Accordingly, in the 21st century, many changes can be found, such as teachers’ conventional roles being shifted to different roles, such as facilitators, curriculum planners, resource developers, educational administrators, and researchers (Crosby, 2000). Significant trends in underpinned theories of medical education are now focusing on patient-centred and culturally competent medical doctors and the ultimate aim of medical education is to improve the patient and community outcomes by promoting competent and caring practitioners (competent medical doctors) (McLean et al., 2008).

According to Harden and Laidlaw (2017), the competencies and attributes expected of an effective teacher includes not only mastery of the content area but also the technical competencies necessary to serve as an information provider, a role model, a facilitator of learning, a curriculum planner, an assessor, a manager and a scholar” (p. 9). Most medical schools worldwide have been implementing specific training for medical educators to develop the necessary skills as medical educators. The study conducted by Steinert (2014), pointed out that nowadays, most medical schools and educational organisations offer various programs and activities in response to educational trends in teaching and assessment for helping faculty members improve their skills as teachers and educators. Additionally, the World Federation of Medical Education (WFME) entails that for a basic standard of staff development: “the medical school must have a staff policy that addresses a balance of capacity for teaching, research and service functions. It also encourages ensuring recognition of meritorious academic activities, with appropriate emphasis on both research attainment and teaching qualifications” (World Federation for Medical Education, 2015).

In Myanmar, there are five civilian medical schools and one military medical school (Defence Services Medical Academy); however, no private medical school exists yet. Defence Services Medical Academy (DSMA), was established on 19th November 1992 in Mingalodon, Yangon and is listed in the World Directory of Medical Schools. The Ministry of Health and Sports, Myanmar, established a medical educator’s training program in 2003 for the medical educators from the civilian medical schools; likewise, the Directorate of Medical Services under the Ministry of Defence also started the medical educator training program for the military medical school in 2011. Both these medical educators’ training programs emphasize on developing the competent skills of medical educators.

Although medical educator training is crucial to improve medical educators’ quality in medical school, other influencing factors can affect the competency of medical educators. The impact of training depends not only on the program design of the training program but also on other factors such as learner characteristics and educational environment (Iqbal & AlSheikh, 2018). In a study conducted by Peeraer and Van Petegem (2012), the faculty members were able to apply teaching strategies and methods in their teaching activities after the faculty development training. Nevertheless, they faced some challenges, such as time constraints and scarce resources that impede their behavioural changes from becoming sustainable.

The medical educator training program in DSMA is a distance learning, diploma course with four face-to-face hands-on workshops. Individual written assignments are given on 10 fundamental modules that provide medical educators with the knowledge and skills about educational psychology, curriculum design and planning, objectives and contents, teaching-learning strategy, teaching-learning media, assessment and evaluation, educational leadership, communication skills, as well as conducting an educational workshop and educational research: throughout the one-year course. Since 2011, nine successive medical educator training courses have been conducted, and approximately 200 medical teachers from DSMA have successfully completed these courses and graduated.

Currently, there is no published evidence-based research investigating the crucial issues to develop competent medical educators in Myanmar. This study intends to focus on the training perspectives and the commitment of institutional support for the development of the medical skills. Therefore, the main objective of this study is to find out the impact of the medical educator’s training course conducted in the military medical school and to explore the key challenges encountered by the trainees.

II. METHODS

A. Research Design

A retrospective design was used to investigate the impact of the medical educator training course and the medical educators’ perception regarding their key challenges. By applying the program theory, a logic model for training program was applied, illustrated in Figure 1.

Figure 1. Logic model for training evaluation Adapted from (Rossi, et al., 2004)

B. Kirkpatrick’s Model

To investigate the impact of the training, the Kirkpatrick evaluation measurement tool was utilized (Appendix 1). The four-levels in the Kirkpatrick’s model are: 1) Reaction evaluation, Learning evaluation Behaviour evaluation and Result evaluation (Kirkpatrick & Kirkpatrick, 2006).

C. Research Framework

Figure 2. Research framework

D. Data Collection

Survey questionnaires were formulated in three sections. The first session aimed to get demographic information, the second one intended for Kirkpatrick’s evaluation by 39 evaluation questions, and the last was constructed by nine items to explore the critical challenges for medical educators. Questionnaires were developed based on the contents and expected outcomes of the medical educator training course that has been to delivered throughout the courses. These are related to the knowledge, attitude and skills that gained after the course regarding educational psychology, curriculum design and planning, objectives and contents, teaching-learning strategy & media, assessment and evaluation, educational leadership, communication skills, conducting an educational workshop and educational research. Moreover, questionnaires attributed to possible challenges that have been encountered by medical educators were adapted from the research outcomes of Huwendiek et al. (2010). All evaluation questions were self-administered and had the five-point Likert scale items ranging from 5 (strongly agree) to 1 (strongly disagree). Informed verbal consent was obtained from the respondents in respective of respondents’ autonomy before data collection. Ethics approval was obtained from the Ethical Review Committee of the Defence Services Medical Academy, Yangon, Myanmar.

E. Sampling Procedure

Every medical educator is eligible if they had already completed the training and currently engaging in the undergraduate and postgraduate medical education programs at the military medical school, in Myanmar. Among the 120 persons who meet eligibility, approximately 30% of the population (45 participants) responded to the evaluation survey and convenience sampling was practiced.

F. Data Analysis

Data obtained from the survey was entered in Microsoft Excel files and analysed by SPSS software. The item scores of each evaluation were analysed as descriptive analysis such as mean and standard deviation to compare the response rate. The t-test and ANOVA analysis were used to determine the relationship between respondents’ demographic characteristics and Kirkpatrick’s four-level evaluation results. Descriptive analysis was used to explore the key challenges of medical educators.

III. RESULTS

Forty-five medical educator attendees who satisfactorily completed one of the medical education courses held at DSMA between 2011 and 2019, individually expressed their views on the impact of the medical educator training courses and disclosed the key challenges regarding faculty development in medical education. The data that support the findings of this study are openly available in Figshare at https://doi.org/10.6084/m9.figshare.17074637 (Naing et al., 2021).

There were 39 evaluation questions, and each evaluation level has specific items, respectively. The internal consistency of each of the scales was examined by using Cronbach’s alpha. The alpha score was satisfactory: 0.65 for reaction evaluation (11 items), 0.86 for learning evaluation (10 items), 0.81 for behavioral evaluation (11items), and 0.84 for result evaluation (7 items).

A. Demographic Characteristics

Of the demographic characteristics, three factors (age, gender, and expertise), had no significant association with the impact of the training, but the respondents’ educational background (p=0.03), job’s position (p=0.02), academic year they attended (p=0.03) showed statistically significant association with the impact of training (Table 1). However, the proportionally unequal number of respondents in each group may affect the results.

 

Demographic Characteristics	Reaction Evaluation	Learning Evaluation	Behavioral evaluation	Result Evaluation
	Mean ± SD	Mean ± SD	Mean ± SD	Mean ± SD
Age(N=45)
<=40 (N=26)	4.05±0.20	4.08±0.27	3.93±0.26	4.00±0.35
41-50 (N=11)	4.10±0.27	4.30±0.44	3.99±0.22	4.22±0.39
>= 50 (N=8)	4.06±0.28	4.12±0.38	4.01±0.44	4.10±0.40
‘F’ value	0.17	1.55	0.28	1.30
‘p’ value	0.83	0.22	0.75	0.28
Gender(N=45)
Male(N=33)	4.06±0.23	4.10±0.31	3.93±0.26	4.07±0.39
Female(N=12)	4.09±0.24	4.26±0.41	4.04±0.36	4.08±0.33
‘t’ value	-0.474	-1.44	-1.16	-0.07
‘p’ value	0.63	0.15	0.24	0.94
Education(N=45)
Master(N=31)	4.05±0.27	4.09±0.31	3.90±0.27	3.99±0.37
Doctoral(N=14)	4.09±0.20	4.25±0.39	4.09±0.28	4.25±0.32
‘t’ value	-0.50	-1.49	-2.18	-2-23
‘p’ value	0.61	0.14	0.03*	0.03*
Expertise in medical Sciences(N=45)
Basic Sciences(N=14)	4.04±0.26	4.10±0.33	3.86±0.30	4.06±0.33
Paraclinical(N=16)	4.09±0.19	4.25±0.33	4.09±0.24	4.20±0.36
Clinical(N=9)	4.02±0.28	4.03±0.33	3.90±0.29	3.96±0.40
Others(N=6)	4.15±0.21	4.10±0.41	3.90±0.32	3.92±0.45
‘F’ value	0.45	0.95	1.94	1.19
‘p’ value	0.71	0.42	0.13	0.32
Job Position (N=45)
Assistant lecturer(N=27)	4.04±0.21	4.07±0.27	3.93±0.26	3.99±0.35
Lecturer(N=11)	4.04±0.24	4.16±0.35	3.90±0.18	4.12±0.38
Associate Professor(N=3)	4.33±0.29	4.70±0.30	4.33±0.29	4.38±0.35
Professor(N=4)	4.11±0.30	4.15±0.52	4.02±0.56	4.25±0.45
‘F’ value	1.49	3.47	2.02	1.47
‘p’ value	0.23	0.02*	0.12	0.23
Teaching experience (N=45)
< = 5 Years(N=17)	4.02±0.24	4.11±0.37	3.88±0.27	4.13±0.45
(6-10) Years(N=15)	4.15±0.19	4.12±0.24	4.03±0.23	4.01±0.32
(11-15) Years(N=7)	4.00±0.23	4.11±0.28	3.90±0.44	4.08±0.29
> = 6 Years(N=6)	4.07±0.28	4.31±0.54	4.06±0.24	4.04±0.39
‘F’ value	1.00	0.55	1.05	0.24
‘p’ value	0.41	0.64	0.37	0.86
Year of Services (N=45)
< = 10 Years(N=5)	4.05±0.13	4.16±0.15	3.96±0.18	4.08±0.27
(11-15) Years(N=16)	4.08±0.23	4.04±0.29	3.90±0.29	3.96±0.37
(16-20) Years(N=12)	4.08±0.28	4.25±0.39	4.01±0.27	4.16±0.39
> = 21 Years(N=12)	4.04±0.23	4.16±0.40	3.98±0.35	4.13±0.40
‘F’ value	0.08	0.83	0.35	0.77
‘p’ value	0.97	0.48	0.78	0.51
Academic Year (N=45)
2011-2013(N=11)	4.03±0.25	4.25±0.35	4.00±0.31	4.10±0.32
2014-2016(N=9)	4.00±0.34	3.93±0.35	3.78±0.34	3.79±0.45
2017-2019(N=25)	4.11±0.17	4.17±0.32	4.00±0.24	4.16±0.33
‘F’ value	0.94	2.47	2.09	3.64
‘p’ value	0.40	0.10	0.13	0.03*

(*) means 0.05 level of significant

Table 1. Relationship between demographic characteristics and Kirkpatrick’s evaluation

B. Kirkpatrick’s Four Level of Evaluation

When looking at the reaction evaluation, most of the respondents gave favourable agreement on the management of training, teaching skills of the trainers, and training methods, however, there was relatively weakness in the proper preparation of training contents by facilitators (3.86±0.45), providing training materials (3.88±0.53). Looking upon the learning evaluation responses, all of the respondents expressed a high level of satisfaction on their understandings and awareness. Regarding learning behaviour, majority agreed that they can apply their learning to their workplace but comparatively lower response score was found in planning educational research (3.84±0.56), how to apply educational theories in medical education practices (3.84±0.52) and arrange an educational workshop programme (3.75±0.60). Although the medical educator training affects the respondents to get better improvement in most faculty development activities, there is still a need to develop the participation of respondents in research development activities with comparatively lowest evaluation scores of (3.80±0.66).

Table 2. Kirkpatrick’s Four Level of Evaluation of Medical Educator Training

C. Key Challenges in Faculty Development

It was found that majority of the evaluation scores for the statement items in the faculty development area were much lower than the scores for the items in the training’s impact. The most noticeable and lowest scores were found in the item of financial investment of medical educators (2.68±0.73), research collaboration in other universities (2.60±0.88) and networking with the international university for career development of medical educators (2.55±0.84). However, they agreed that the currently used undergraduate curriculum of the institution is appropriate for them to practically apply their pedagogic skills gained from the training course(4.17±0.71).

No.	Statement	Mean±SD
1.	Depending on the performance in faculty development activities, our institution appreciates rewards for medical educators as an institutional policy.	2.80±0.75
2.	Medical educators in our university have the opportunity to apply the Institutional scholarship program for research and carrier development.	2.84±0.79
3.	The financial investment for medical educators in our institution is acceptable in the current situation. (e.g. teaching-learning materials, research facilities, ICT based technology)	2.68±0.73
4.	There are adequate networks and collaboration with an international university to promote the carrier development of medical educators. (e.g. MOU)	2.60±0.88
5.	ICT based medical education system in our university is well functioned and applicable.	2.73±0.80
6.	The number of medical educators in our university is sufficient to achieve the mission of our university.	2.71±0.84
7.	The institutions’ current undergraduate curriculum is appropriate for us to practically apply the pedagogic skills gained from the training course.	4.17±0.71
8.	Research development activities of our university, including research funding, research skills of medical educators, and research facilities, are well functioning.	2.80±0.78
9.	The research collaboration with other universities or research centers to promote research innovation is acceptable.	2.55±0.84

Table 3. Key Challenges in faculty development in medical education by respondents

IV. DISCUSSION

This study investigates the medical educator training program, which aims to develop the competency skills of medical educators who are involved in the faculty development activities at DSMA, Myanmar. Regarding the overall impact of the training, the respondents expressed their positive perception on the organisation of the program; the training improved their knowledge and attitudes as a medical educator; and it was practically applicable in their daily work. The positive perception of the training was not surprising because most of the trainees voluntarily attended the course and were highly motivated to accomplish self- improvement after attending the course. The findings are consistent with another study where ‘overall satisfaction with faculty development programs was high, and they consistently found the program helpful, enjoyable and relevant to their objectives’ (Steinert et al., 2016, p.779). Similarly, another study done by Yolsal et al. (2003) showed that the participants who enrolled in the similar training of trainers (TOT) course explored positive perception, and they agreed it was required to be acquainted with those kinds of instructional practices (Yolsal et al., 2003). The possible reason for the respondents’ satisfaction with the impact of training in the current study, could also be due to the organized training preparation and training context and, the course itself used mainly a distance learning format with flexible training schedules for them when compared to other full-time courses in other specialties. It was also found that significant improvements in self-efficacy of medical educators in the domain of the teaching relevant subject contents and developing creative ways to cope with system constraints after experiencing 12 months faculty development program in Bhutan (Tenzin et al., 2019).

In this study, for learning evaluation, most respondents believed that they gained in terms of knowledge, attitude, and skills and that it had an impact on the teaching effectiveness. It revealed that most of the medical educators gained benefits after the training, and the increase in confidence facilitated them to become more involved in participating in faculty development activities and curriculum planning activities. Similarly, F.J. Ciller and N. Herman explained that as a goal of an educational development program, changes in attitudes and perception serves as a foundation for further changes in behaviour (Cilliers & Herman, 2010).Even more, the review articles of Steinert et al. (2016) highlighted most of the faculty development interventions focus on teaching effectiveness by improving their teaching skills, assessment skills, designing curriculum, and educational leadership skills (Steinert et al., 2016).

When analysing the relationship between the various demographic background and impact of the training, it was revealed that the job position, educational background, and difference in academic year among the respondents were significantly associated with the impact of training. There might be many possible reasons why this was significant. For example, the course preparation, the changes in placement of trainers in several years might also be a possible issue, and variation in individual performance also considered. However, in this survey, those factors were not explored. Fishbein et al. (2003) explained that interpersonal variation and the organization’s favorable situation could be impact factors for behavioural change at the organization level (Fishbein et al., 2003).

In this study, although the self-reported changes showed self-actualization in individual performance to some extent, respondents are not contented with their learning environment in terms of institutional support. Institutional support plays a vital role in faculty development in medical education and every institution can meet its institutional mission and goals by enabling its faculty member to fulfil their particular purposes as teachers, scholars, and leaders according to the research outcome (Boucher et al., 2006). Moreover, Steinert pointed out that many factors are impeding the faculty development, such as unsupportive leadership, resistance to change, lack of faculty motivation, and the unwillingness of faculty to acquire the teaching skills and knowledge (Steinert, 2000).

On exploring the critical challenges of the medical educators, all the respondents believed that their institution should support more scholarship opportunity, academic recognition, networking with other universities in terms of research and professional development, and ICT-based medical education system. According to the survey of (Huwendiek et al., 2010) 806 medical educators from Association for Medical Education Europe (AMEE) revealed that the critical challenges of medical education were lack of academic recognition (40%), funding (36%), faculty development (24%), time for medical education issues (22%), and institutional support (21%) (Huwendiek et al., 2010).

Therefore, to accelerate the faculty development as a holistic approach, it is needed to consider not only training for faculty members but also other factors reward and recognition, ICT system, and networking. This requisite is not only in a military medical school, but internationally most medical schools are trying to support their faculty in accordance with the staff policy requirements as stated in the WFME Global standards for Quality improvement. However, the current research could gather only some self-reported changes in behaviour and whether the changes actually occurred in the workplace has not been proven by observation. Nevertheless, as the opinions were obtained from closed ended questionnaire statements, a further exploratory qualitative study is needed to obtain accurate information on the magnitude of the problem and the specific areas that needed further support from the institution.

V. CONCLUSION

This study revealed the medical educator training could improve their required knowledge, attitude, and skills to practice in the teaching environment, the fundamental need for educational leadership, educational research, and communication skills in the health-care setting. The medical educators who need to be competent could not be motivated only from the training without institutional support. The respondents believed that their institution should encourage institutional support in terms of reward, recognition, scholar allowance, and collaboration with other academic institutions to promote research culture and professional development, ICT-based medical education.

To conclude, the findings of this study exclusively show that military medical schools in Myanmar still need to emphasize the professional identities of medical educators by encouraging institutional support, not just by only focusing on the faculty development training as a mandatory by institutional policy.

Notes on Contributors

Tun Tun Naing reviewed the literatures and developed the conceptual framework and conducted the data analysis and wrote the discussion and conclusion. Finally, he developed the manuscript to submit to TAPS.

Yuriko Minamoto was involved in the formulation of research question and research framework to conduct the research systematically, application of evaluation tools and technique and, proofreading of original thesis and the manuscript.

Ye Phyo Aung participated in the research in the writing of research methodology session, conducted the data collection, and supported choosing research design, proper sampling methods and data collection tools and technique and proofreading of manuscript.

Marlar Than supported the construction of survey questionnaires which is the back bone of the evaluation research and contribute to proofreading of current manuscript.

Ethical Approval

Ethics approval was granted by the Ethical Review Committee of the Defence Services Medical Academy, Yangon, Myanmar. (7 / Ethics 2019).

Data Availability

The data that support the findings of this study are openly available in Figshare repository,  http://doi.org/10.6084/m9.figshare.17074637

 Acknowledgement

I would also like to express my gratitude to Japanese Government through the Japan International Cooperation Center (JICE) for their support to conduct this research project for completion of master thesis program at Meiji University.

I would like to express my special thanks to my colleagues, and without their support, the survey of my research would not have been achieved. I am indebted to all medical educators from Military Medical Service, Myanmar, who help me respond to my survey sharing with their valuable opinions and experience.

Funding

Conducting the research was operated during the study period supported by JDS Program under the JICA. For this publication , it is conducted for personal and professional development and no funding is involved.

Declaration of Interest

There is no conflict of interest in the current research.

References

Benor, D. (2000). Faculty development, teacher training and teacher accreditation in medical education: Twenty years from now. Medical Teacher, 22(5), 503-512. https://doi.org/10.1080/01421590050110795

Boucher, B., Chyka, P., Fitzgerald, W., Hak, L., Miller, D., Parker, R., Phelps, S., Wood, G., & Gourley, D. (2006). A comprehensive approach to faculty development. American Journal of Pharma- ceutical Education, 70(2), 27. https://doi.org/10.5688/aj700227

Cilliers, F., & Herman, N. (2010). Impact of an educational development programme on teaching practice of academics at a research‐intensive university. International Journal for Academic Development, 15(3), 253-267. https://doi.org/10.1080/1360144x.2010.497698

Crosby, R. (2000). AMEE Guide No 20: The good teacher is more than a lecturer – the twelve roles of the teacher. Medical Teacher, 22(4), 334-347. https://doi.org/10.1080/014215900409429

Fishbein, M., Hennessy, M., Yzer, M., & Douglas, J. (2003). Can we explain why some people do and some people do not act on their intentions? Psychology, Health &Amp; Medicine, 8(1), 3-18. https://doi.org/10.1080/1354850021000059223

Harden, R. M., & Laidlaw, J. M. (2017). Essential Skills for a Medical Teacher (2nd ed., pp. 9). Elsevier.

Huwendiek, S., Mennin, S., Dern, P., Ben-David, M., Van Der Vleuten, C., Tönshoff, B., & Nikendei, C. (2010). Expertise, needs and challenges of medical educators: Results of an international web survey. Medical Teacher, 32(11), 912-918. https://doi.org/10.3109/0142159x.2010.497822

Iqbal, M., & AlSheikh, M. (2018). Factors affecting the transfer of training to the workplace after a faculty development programme: What do trainers think? Journal of Taibah University Medical Sciences, 13(6), 552-556. https://doi.org/10.1016/j.jtumed.2018.11.001

Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating Training Programs: The Four Levels (3rd ed.). Berrett-Koehler Publisher.

McLean, M., Cilliers, F., & Van Wyk, J. (2008). Faculty development: Yesterday, today and tomorrow. Medical Teacher, 30(6), 555-584. https://doi.org/10.1080/01421590802109834

Naing, T. T., Minamoto, Y., Aung, Y. P., & Than, M. (2021). Faculty development of medical educators: Training evaluation and Key Challenges. [Dataset].

Figshare. http://doi.org/10.6084/m9.figshare.17074637

Peeraer, J., & Van Petegem, P. (2012). The limits of programmed professional development on integration of information and communication technology in education. Australasian Journal of Educational Technology, 28(6), 1039-1056. https://doi.org/10.14742/ajet.809

Rossi, P., Lipsey, M., & Freeman, H. (2004). Evaluation: A Systematic Approach (7th ed.). Sage.

Sheets, K., & Schwenk, T. (1990). Faculty development for family medicine educators: An agenda for future activities. Teaching and Learning in Medicine, 2(3), 141-148. https://doi.org/10.1080/10401339009539447

Steinert, Y. (2000). Faculty development in the new millennium: key challenges and future directions. Medical Teacher, 22(1), 44-50. https://doi.org/10.1080/01421590078814

Steinert, Y. (2014). Faculty Development in Health Profession: A Focus on Research and Practice (1st ed.). Springer.

Steinert, Y., Mann, K., Anderson, B., Barnett, B., Centeno, A., Naismith, L., Prideaux, D., Spencer, J., Tullo, E., Viggiano, T., Ward, H., & Dolmans, D. (2016). A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Medical Teacher, 38(8), 769-786. https://doi.org/10.1080/0142159x.2016.1181851

Tenzin, K., Dorji, T., Choeda, T., & Pongpirul, K. (2019). Impact of faculty development programme on self-efficacy, competency and attitude towards medical education in Bhutan: A mixed-methods study. BMC Medical Education, 19(1), 468. https://doi.org/10.1186/s12909-019-1904-4

World Federation for Medical Education. (2015, January 1). Basic medical education WFME global standards for quality improvement. https://wfme.org/download/wfme-global-standards-for-quality-improvement-bme/.

Yolsal, N., Bulut, A., Karabey, S., Ortayli, N., Bahadir, G., & Aydin, Z. (2003). Development of training of trainer’s programmes and evaluation of their effectiveness in Istanbul, Turkey. Medical Teacher, 25(3), 319-324. https://doi.org/10.1080/0142159031000092779

*Tun Tun Naing
No. 94, D-1, Pyay Road,
Mingaladon Township
Yangon, Myanmar
Postal code – 11021
+95 95053402
Email: tuntunnaing@dsma.edu.mm, drhtun1984@gmail.com

Submitted: 30 May 2021
Accepted: 8 April 2022
Published online: 5 July, TAPS 2022, 7(3), 10-22
https://doi.org/10.29060/TAPS.2022-7-3/OA2539

Hanyi Li¹, Elaine Li Yen Tan^1,3, Mun Loke Wong² & Marianne Meng Ann Ong^1,3

¹National Dental Centre Singapore, Singapore; ²Faculty of Dentistry, National University of Singapore, Singapore; ³Oral Health Academic Clinical Programme, Duke-NUS Medical School, Singapore

Abstract

Introduction: As young healthcare professionals (HCPs) enter the workforce, they find challenges adapting as academic training and workplace settings often do not mirror each other. Mentorship is a possible solution to help bridge this transition. The aim of this study was to gather information from HCPs with regards to their views towards mentorship as a strategy to help in the transition of newly qualified HCPs from study to work.

Methods: Two 3-hour interactive workshops entitled “Bridging the Study-Work Chasm” were organised, and participants were invited to complete a survey voluntarily after the workshop. The survey comprised questions regarding the benefits of mentorship, qualification of mentors, time commitment for mentoring, elements of an effective mentorship programme, and barriers to its effectiveness. The anonymised responses were analysed descriptively.

Results: Fifty-two out of 62 participants from various healthcare backgrounds completed the survey. 96.2% of respondents felt a study-work chasm exists in the healthcare workplace with 90.4% indicating that a mentorship programme would help to bridge the chasm. More than 70% of participants agreed or strongly agreed that mentoring would boost confidence, reduce anxiety, and aid in study-work transition. It was identified that to produce a more effective mentorship programme, time commitment, training, and proper organisation of the programme would be necessary.

Conclusion: It was perceived that a mentorship programme can help to bridge the study-work chasm in the healthcare landscape in Singapore, and will best serve mentors and mentees by committing the proper time and training to ensure its effectiveness.

Keywords:           Training-Work Transition, Graduate, Healthcare Professionals, Mentorship

Practice Highlights

• Despite receiving extensive training during their education, healthcare professionals still experience many challenges as they enter the workforce.
• Globally it has been reported that differences in training and workplace demands, coupled with the need to take direct responsibility for patients, propagate anxiety and perceived incompetence in fresh graduates. This could result in compromised work performance and patient outcomes.
• A study-work chasm exists in the healthcare work space, and should be addressed.
• Mentorship was shown to be accepted as a popular solution amongst healthcare workers in Singapore, and the benefits were discussed.
• Elements of a good mentorship programme as well as challenges in setting one up were identified, laying the groundwork for future implementation of such programmes in local public healthcare institutions.

I. INTRODUCTION

Healthcare professionals (HCPs) are known to receive extensive training during their pre-qualification education. However, there still exists many challenges as they enter the workforce. These include increasing workload, encountering patients with more complex conditions, generational diversity in the workforce, performance anxiety, and bullying when transitioning to the workforce (Hofler & Thomas, 2016). While facing these challenges in a new working environment marks the beginning of a fresh process of learning, there are indications that this may be more than what newly qualified professionals can cope with (Teunissen & Westerman, 2011).

There are several contextual differences between pre-employment learning in the university and post-employment learning in the workplace. Fundamentally, the focus of education and real-world practice are different. The commonplace practice of test-taking in school focuses on knowledge retention, with minimal consideration for practical value in the workplace (Lave & Wenger, 1991). A good example of this is the learning of ethical guidelines, which can be easy to regurgitate in school assignments and tests, but are far more difficult to apply on the job (Le Maistre & Paré, 2004). In school, tasks and assignments follow a certain syllabus and scope, which are more structured and predictable than what is expected at the workplace. Similarly, these tasks and assignments in school are indicators of performance and avenues for feedback, while at the workplace such similar opportunities are limited (Wendlandt & Rochlen, 2008).

Healthcare training has had elements of practical application, but the adequacy of these exposures is questionable. Clinical shadowing and observation are known to be helpful, but cannot take the place of actual hands-on clinical experience (Brennan et al., 2010). With increased patient safety concerns, there has also been a shift towards simulation, which reduces meaningful contact with patients and poses challenges when students are expected to ultimately transfer their learning to real-life practice (Bleakley & Bligh, 2008).

The security of observing from a distance and the safe environment for experimentation and reflection that students experience in school have to be left behind as they enter the workplace, apply textbook knowledge in real-life situations, and deal with workplace systems and politics (Le Maistre & Paré, 2004). Efforts to prepare for this transition are misplaced (Kilminster et al., 2011), and the training and educational opportunities aimed to help with this transition, such as those in the transitional year, have been lacklustre and ineffective (Lambert et al., 2013). Therefore, a study-work chasm exists in many places, and is a pertinent and critical issue that requires addressing.

Among efforts to help in the transition of students to new HCPs at the workplace, mentorship has been seen as a possible solution (Andrews & Wallis, 1999; Dalgaty et al., 2017). Mentorship, as previously defined by The Standing Committee on Postgraduate Medical and Dental Education in the United Kingdom, is the guidance in “the development and re-examination of their own ideas, learning, and personal and professional development” by “listening and talking in confidence to the mentee” (Oxley & Standing Committee on Postgraduate Medical and Dental Education, 1998). It has also been built into medical practice guidelines, such as that in the UK (General Medical Council, 2012), as a key element in training and professional development.

Mentorship has been seen as a viable approach not only to address any gaps in professional skills, but also aid junior healthcare workers in personalised professional development in the workplace, to enhance job satisfaction, motivation, and self-esteem (Souza & Viney, 2014). The role of a mentor in medical education is to help mentees identify areas of strength and weaknesses in a comfortable and safe learning environment, offer guidance and advice, and motivate and support them to work towards their personal long-term goals (Burgess et al., 2018). Mentors have roles overlapping that of coaches and advisors in medical education. However, coaches tend to focus more on skills or knowledge-based content, and may have a relationship that is of shorter duration and of less depth than between mentors and mentees (Lin & Reddy, 2019). Advisors often oversee a group of individuals in an organisation. Therefore, they develop a more structured relationship compared to that between mentors and mentees, and tend to work towards an outcome based on the needs of the organisation (Hastings & Kane, 2018). Thus, mentoring has been a widely recognised method of helping young people learn, demonstrate their abilities and potential, as well as develop their identity (Fuller & Unwin, 1998). This is extensively explored and studied in many healthcare institutions and systems today.

The aim of this study was to gather information from HCPs about their views towards mentorship as a strategy to help in the transition of newly qualified HCPs from study to work.

II. METHODS

This was a descriptive study on the perceptions of the study-work chasm by HCPs. The protocols were sent to SingHealth Centralised Institutional Review Board (References: 2017/2687 and 2021/2044) and they were deemed exempt from review.

Two three-hour interactive workshops, entitled “Bridging the Study-Work Chasm”, were held in September 2017 and 2018. The workshops invited participation from HCPs in SingHealth, one of the three public healthcare clusters in Singapore.

Each 3-hour workshop comprised two short talks on ‘Is there a Chasm?’ and ‘Bridging the Chasm’, followed by small-group discussions, then sharing and discussions with the large workshop group. A round-up and summary was done by the respective facilitator after each large-group discussion.

The first talk on ‘Is there a Chasm?’ gave a definition of a study-work chasm and the small groups were then asked to describe how such a chasm manifested in the workplace with two questions: “How do we know if a study-work chasm exists?” and “What contributes to this study-work chasm?”  After the sharing by the small groups with the large group and some discussion, the facilitator then summarised differences between learning in an academic setting versus learning in the workplace that could contribute to the study-work chasm.

The second talk on ‘Bridging the Chasm’ invited participants to think about and discuss in their small groups how they could bridge the study-work chasm and what were some of the anticipated barriers in doing this. These were then shared with the large group followed by a summary by the facilitator on how coaching and mentoring could be used to bridge the chasm. Facilitators finally concluded the workshop session with a summary of points, addressing questions and answers, and getting participants to complete the survey and workshop evaluation forms.

The survey was carried out by having the participants fill up either an online survey form via scanning a QR code, or a hardcopy version. Implied consent was obtained with the submission of the survey. The survey consisted of 26 items and two open-ended questions (Appendix 1). The first part utilised dichotomous questions to survey participants about their past and current experiences with mentoring, and their views towards whether a study-work chasm existed and whether mentoring could be a solution to bridge this gap. The second part comprised two open-ended questions, asking participants about the elements of an effective mentorship programme, and barriers to having one. The third part allowed participants to rate their level of agreement with 15 statements, related to the benefits of mentoring, the qualification of mentors, and time commitment for mentoring, on a Likert-like scale of 1-5 (1: Strongly Disagree; 5: Strongly Agree). Questions were adapted from a questionnaire that was developed and pilot-tested in a previous study (Lopez et al., 2010).

The anonymised survey data was collated and analysed descriptively. Common repetitive words that appeared in the responses to the open-ended questions were noted and these were grouped into key themes. The percentage of responses representing each theme was tabulated.

III. RESULTS

A. Respondents

Sixty-two participants comprising HCPs from various backgrounds – doctors, dentists, nurses, ancillary workers, and allied health professionals – attended both workshops. There were no repeat participants who attended the two workshops. The data that support the findings of this study are openly available in Figshare at https://doi.org/10.6084/m9.figshare.14740332 (Li et al., 2021). Among the 62 participants, 52 completed the survey, achieving a response rate of 83.9%. A majority (78.8%) of the respondents were females. Dentists were the largest healthcare group represented, comprising 42.6% of respondents, followed by doctors, comprising 21.2%. The age group of 30 to 39 years old had the strongest representation of 44.2%. Most respondents had more than five years of working experience, with 44.2% of respondents having six to 10 years of experience, and 51.9% of respondents having more than 10 years of experience (Figure 1). Among the respondents, 96.2% (50 out of 52) felt that a study-work chasm existed in the healthcare workspace, with 90.4% (47 out of 52) indicating that a mentorship programme would help to bridge the chasm, and a majority (90.4%) of respondents agreed that mentoring would be a viable strategy (the remaining 5 respondents were neutral). A good majority of respondents (67.3%) noted they did not have a mentor when they first started work.

Figure 1. Demographics of survey respondents: gender, age, healthcare group, and number of years of working experience.

B. Key Features of An Effective Mentorship Programme

There was a total of 63 responses describing features of an effective mentorship programme, and the key themes were identified in Figure 2(a). Respondents raised attributes pertaining to the attitude and dynamics of the mentor and mentee, and the organisation of such a programme. Some respondents submitted multiple attributes.

Figure 2(a). Key themes describing key features of an effective mentorship programme

Pertaining to attitude and dynamics, both the mentor and mentee must firstly be engaged and willing to participate in the programme (11.1% of responses). The mentor and mentee should have attributes of open-mindedness and humility (19.0% of responses), and the mentor also ought to be patient, caring, and non-judgmental (17.5% of responses). Finally, the mentor-mentee pair must have good two-way communication, and some degree of interpersonal “chemistry” to bond (12.7% of responses).

Pertaining to the organisation of a mentor-mentee programme, many (12.7% of responses) raised that there should be planned regular meetings, and participants should set goals and expectations and discuss timely feedback (11.1% of responses). However, there was no indication of an ideal frequency of meeting. Some other organisational features were also mentioned. Regarding mentor-mentee pairing, it was noted that the mentor-mentee ratio was an important factor, though there was no indication of a preferred ratio in the responses; others raised that both mentor and mentee should be from relevant fields, for instance from the same department. Some also mentioned that the mentorship programme could be built into the career progression pathway of the mentee to ensure continuation, and even to build it into licensing requirements, such as in the housemanship year for junior doctors.

C. Barriers to Effective Mentorship Programme

There was a total of 53 responses describing barriers to an effective mentorship programme, and the key themes were identified in Figure 2(b). Respondents acknowledged that several aspects of effective mentorship previously mentioned were difficult to achieve, many of which revolved around commitment and mentor-mentee dynamics. A lack of time was identified as the top barrier to implementing a mentoring programme (49.1% of responses). As mentorship can be time consuming, there might be difficulty getting sufficient mentors to willingly commit the time; similarly, it might be challenging to match the availability of mentors and mentees around hectic work hours. Furthermore, there was significant mention (41.5% of responses) about whether the seniority of mentors may result in difficulty breaking the ice with the mentees, or if cultural differences may hinder the understanding of differing perspectives. Some respondents also raised that not all mentors may have the personality, skills or willingness to render effective mentorship, as evidenced in 17.0% of the responses. Finally, some respondents questioned if and how the role of mentors and work supervisors should be differentiated in the workplace.

Figure 2(b). Key themes describing barriers to an effective mentorship programme.

Percentages represent the proportion of responses mentioning words related to the theme.

D. Mentorship Benefits and Commitment

On mentorship benefits and commitment, there was strong agreement (more than 70% of participants agreed or strongly agreed) that having a mentor would boost confidence, reduce anxiety, and aid in the transition from study to work (Figure 3). It was also strongly expressed that the participants would have liked a mentor in their first year in the workplace (82.6% agreed), and that the mentorship should extend beyond their first year of work as well – more than half (55.8%) disagreed that the mentorship should only be in the first year, and 90.4% agreed that mentorship would help even in the later years of work. In spite of this, and while a strong majority of 71.2% expressed that they would have liked a long-term mentor-mentee relationship, a large proportion of respondents (44.2%) was neutral towards expecting regular mentor-mentee contact while 38.5% expected mentors to meet regularly with mentees (Figure 4).

Figure 3. Responses on benefits of mentorship. Numbers in the chart indicate the number of respondents.

Figure 4. Responses on time commitment of mentorship. Numbers in the chart indicate the number of respondents.

E. Mentor Qualifications and Background

On mentor qualifications, more than half of the respondents agreed that they had adequate knowledge (59.6%) and skills (53.8%) to function as effective mentors. Most of those who agreed (74.2% for knowledge, 67.9% for skills) were HCPs with more than 10 years of experience. A significant proportion responded neutrally as well, with 28.8% and 34.6% for knowledge and skills, respectively. Majority of them (71.4% for knowledge, 52.9% for skills) had 6 to 10 years of work experience in healthcare. Nevertheless, many more (78.8%) agreed that receiving prior training or orientation in mentoring could increase their effectiveness as mentors. When asked about whether young professionals two to three years into their professions make the best mentors, there was strong neutrality in the response (46.1%, the largest proportion). It was noted that all the neutral responses were from HCPs with more than 5 years of experience, with a significant proportion of them (52.2%) having 6 to 10 years of work experience. All except two of the respondents who gave neutral responses had experience as a mentor. Majority (59.6%) preferred mentors to be paired with mentees with similar backgrounds; similarly, there was significant uncertainty around whether mentors from dissimilar professions could be effective, with the same proportion (38.5%) of respondents agreeing and holding a neutral stance towards this view (Figure 5).

Figure 5. Responses on qualification of mentors. Numbers in the chart indicate the number of respondents.

IV. DISCUSSION

The results from this study showed that there was a perceived study-work chasm among this cohort of HCPs. Given the wide range of backgrounds and years of work experience of the respondents, it is also evident that the chasm was perceived across different HCPs and batches of new HCPs entering the workforce.

Indeed, it was reported in the literature that newly qualified HCPs face challenges in the new workplace, indicating a study-work chasm. One such challenge is the lack of hands-on clinical skills. Junior doctors beginning practice and taking direct responsibility for patients found themselves undertaking clinical tasks for which they felt inadequately trained for (Lambert et al., 2013). As such, they expressed anxiety over their competence, especially in areas like prescribing (Brennan et al., 2010; Matheson & Matheson, 2009). It was also reported that new doctors lacked knowledge in pathology and therapeutics (Lempp et al., 2004). These gaps can bring dire consequences to clinical care, influencing assessment, treatment, and outcome of patients (Smith & Poplett, 2002).

In addition, many professionals fresh from school experienced difficulties in communicating with patients. The immediate switch to a professional relationship with patients proved stressful for them, especially in the first few weeks of work (Lempp et al., 2004). The findings of the present study mirror this sentiment with a large proportion of the respondents citing their wish for a mentor in their first year or work.  The difficulty was especially evident in such school-work transitions revolved around breaking bad news, dealing with difficult and violent patients, and communicating with people with mental illnesses. Despite active practice and drilling with simulated scenarios in school, fresh graduates still expressed that they were inadequately prepared for these in real life (Matheson & Matheson, 2009).

Mentorship was seen very favourably as a possible solution to bridging the chasm. Among these, more than half of the respondents (67.3%) had not experienced mentoring during their initial years of work, indicating that mentorship had not been widely implemented, and can be a new strategy for exploration in the local healthcare system. It is worthy to note that many of the respondents who had experienced mentorship had also viewed mentorship positively. Similarly, residents of a medical residency programme in Singapore viewed mentorship as being crucial and beneficial to their training, as mentors were able to provide guidance and encouragement, and create a more well-rounded learning environment (Chua et al., 2020). This implies that mentorship could potentially exert a positive impact on HCPs.

The benefits of mentorship have been well-reported worldwide. In particular, there have been reports emphasising its benefits on the emotional and psychological health of healthcare workers. For instance, in the United Kingdom, senior doctors as mentors and teachers helped train fresh graduates on the job, and reduced their stress (Brennan et al., 2010), increased motivation, and boosted self-esteem (Souza & Viney, 2014). In Saudi Arabia, use of coaching appeared to reduce depression and anxiety among HCPs in the short-term (Aboalshamat et al., 2015). A study in Denmark reported improved professional identity among their fresh doctors who had undergone group mentorship sessions, and participants developed new ways to deal with their new professional roles with increased awareness of their thinking, feelings, and reactions, making the stressful transition into their workplace more bearable (de Lasson et al., 2016). All these resonate well with the responses from this study, that mentorship can reduce anxiety and boost confidence in new HCPs.

Studies have shown attributes of an effective mentorship programme. Many of these attributes relate to the attitudes of mentors and mentees – trust and respect, responsibility, and taking initiative. The importance of having good communication, connection, and common values between mentors and mentees were also emphasised. These were consistent across countries, as seen in studies in North America (Straus et al., 2013) and the United Kingdom (Ssemata et al., 2017). Notably, these attributes were mentioned significantly in the responses collected in this study as well, specifically those relating to mentor-mentee dynamics.

Barriers to effective mentorship have also been brought up in several studies, and draw similarities to the responses obtained from this study. It was summarised that challenges can arise when there is a mismatch of goals, expectations, and time commitment between mentors and mentees. There can also be hindrances to communication due to generational and personality differences (Zerzan et al., 2009). Qualitative studies targeting students, junior faculty mentees and senior faculty members at healthcare institutions uncovered similar issues (Ssemata et al., 2017; Straus et al., 2013).

In this study, questions and uncertainties were raised pertaining to whether mentors should be supervisors as well, and how regular meetings should be held. It is noteworthy that in the United Kingdom, Ssemata et al. (2017) reported findings that there presented conflicts of interest when mentors were also supervisors. Such an arrangement posed problems with performance judgement, competition, and the amount of honesty in the mentor-mentee relationship. In the same study, it was noted that the mentor-mentee pairing could likely be better maximised if the pairing was selected mutually by the mentor and mentee, instead of being pre-assigned, as this would ensure better chemistry and as a result better commitment and motivation towards the relationship. Regular time commitment was generally agreed upon in literature; however, discussions on frequency were inconclusive, similar to the responses of this present study. It was reported in North America that the preferred frequency of meet-ups according to participating mentors and mentees ranged widely from monthly to half-yearly (Straus et al., 2013).

The literature has identified some other barriers to setting up of an effective mentorship programme not surfaced in our study that are worthy of note. In particular, factors relating to the organisation or system play a big part in the effectiveness of the mentorship programme. Geographical location, for instance, is one barrier identified in systems where mentors and mentees may be situated at different sites, such as different hospital campuses, and have difficulty meeting (Soklaridis et al., 2015). This can be the case when HCPs are posted to different clinical sites, or when HCPs across sites are linked up to increase diversity in mentorship. The strong focus by organisations on delivering clinical service may also have spared little resources and manpower for mentorship programmes (Ploeg et al., 2008), rendering such programmes inadequate support. For instance, the use of manpower and facilities for mentorship activities in job-specific areas such as a clinical simulation can be limited when these are dedicated to patient care (Morgan et al., 2018). Finally, a strong culture of hierarchy seen in healthcare settings, such as among nurses, can be a hindrance to the open-mindedness and non-judgemental nature of mentoring that the programme aspires to achieve (Morgan et al., 2018).

Nevertheless, it is clear from existing literature that a formalised mentorship programme would ensure the best outcomes, as time and resources can be dedicated to the programme to ensure better recruitment, matching and co-ordination between mentors and mentees, proper structure and guidelines so as to ensure mentorship outcomes are reviewed at adequate intervals and achieved within a desired timeframe, and adequate training for mentors to render support and guidance to their mentees. A proper programme can likely reduce the challenges to effective mentorship as raised by the respondents of this study, as well as in other studies. Lack of experience of mentors, for instance, was a main failure in mentoring relationships (Straus et al., 2013). Similarly, in this study, a significant finding was that there was a larger proportion of respondents with relatively fewer years of work experience who gave neutral responses towards statements on whether they had sufficient knowledge and skills to function effectively as mentors, compared to statements on benefits of mentorship and their preferences towards having a mentorship programme. This indicated a lower perceived confidence of these respondents in their effectiveness as mentors. In line with this, there was strong agreement that receiving training and orientation in mentoring would have been preferred prior to them assuming such mentoring roles, presumably regardless of their prior work experiences. This reinforces the need for proper training and resources for mentors in order for the mentorship programme to see positive effects.

This study presents with some limitations. Firstly, while there was a diverse range of backgrounds among the workshop participants, the proportions may not be representative of the healthcare landscape in Singapore. As such, this may introduce a potential source of bias towards views of predominant groups of participants such as the dentists. Nevertheless, the findings provide a useful baseline understanding of how other healthcare professionals view mentoring. This could, in turn, set the stage for more extensive work in this area among other healthcare professionals. Secondly, the respondents participated in the workshop prior to responding to this survey. The workshop raised awareness towards a study-work chasm and presented mentorship as one of the possible solutions. Therefore, it might have resulted in a stronger perception of this chasm among respondents, and induced more favourable views towards mentorship. Nevertheless, the participants’ positive views towards mentorship could be viewed as indications of the strong agreement they had towards mentorship as a solution highlighted in the workshop. In future workshops, qualitative focused-group discussions can be carried out, and questionnaires can be administered before and after the workshop, so as to establish a more accurate perception of study-work chasm among participants.

V. CONCLUSION

This study has shown that with a perceived presence of a study-work chasm in the healthcare landscape in Singapore, there were strong preferences for a mentorship programme in bridging this chasm. The benefits of a mentorship programme, and the attributes and challenges of setting up such an effective programme were identified, and echo the findings from existing studies from other regions and healthcare systems in the world. In particular, there were emphases on positive attitudes of mentors and mentees, and alignment of expectations between them. There was also strong indication that mentors require proper training. A formalised mentorship programme will best serve mentors and mentees by committing the proper time and resources to ensure its effectiveness.

This study lays the groundwork for future efforts to help fresh graduates better transition into the healthcare workplace using mentorship programmes. In particular, the study targeted healthcare professionals working in the healthcare system in Singapore and revealed their sentiments and opinions towards mentorship, paving the way for local efforts to improve healthcare education and mentorship. Moving forward, further studies can be carried out in an in-depth manner to investigate specific areas of concern pertaining to setting up an effective mentorship programme, such as mentor training, time commitment, and mentor-mentee pairing. Studies can be expanded to larger pools of participants, and targeted at specific sectors and participant groups, for example medical doctors in their housemanship year, to better cater to specific professional needs. Focused-group discussions can be carried out to collect more detailed, qualitative responses and feedback from potential mentors and mentees. Furthermore, this information can be used to engage stakeholders in public healthcare institutions in Singapore and the region, and execute pilot tests of formalised mentorship programmes tailor-made to the institution’s structure and the newly qualified HCPs’ needs.

 Notes on Contributors

ELY Tan, ML Wong, and MMA Ong planned, developed, and conducted the workshops and data collection. H Li and ELY Tan, together with ML Wong and MMA Ong planned and wrote the manuscript. All authors read and approved the final manuscript.

Ethical Approval

The study protocols were sent to SingHealth Centralised Institutional Review Board (References: 2017/2687 and 2021/2044) and they were deemed exempt from review.

Data Availability

The data that support the findings of this study are openly available in Figshare with DOI: https://doi.org/10.6084/m9.figshare.14740332

Acknowledgement

The authors would like to thank the administrative team from SingHealth Duke-NUS Oral Health Academic Clinical Programme (Education) for their help and support in the smooth running of the workshops and collection of data.

Funding

No funding was sought for this study.

Declaration of Interest

The authors declare that they have no conflicts of interest, and do not foresee any future competing interests.

References

Aboalshamat, K., Hou, X.-Y., & Strodl, E. (2015). The impact of a self-development coaching programme on medical and dental students’ psychological health and academic performance: A randomised controlled trial. BMC Medical Education, 15(1), 134.  https://doi.org/10.1186/s12909-015-0412-4

Andrews, M., & Wallis, M. (1999). Mentorship in nursing: A literature review. Journal of Advanced Nursing, 29(1), 201–207. https://doi.org/10.1046/j.1365-2648.1999.00884.x

Bleakley, A., & Bligh, J. (2008). Students learning from patients: Let’s get real in medical education. Advances in Health Sciences Education, 13(1), 89–107. https://doi.org/10.1007/s10459-006-9028-0

Brennan, N., Corrigan, O., Allard, J., Archer, J., Barnes, R., Bleakley, A., Collett, T., & De Bere, S. R. (2010). The transition from medical student to junior doctor: Today’s experiences of tomorrow’s doctors. Medical Education, 44(5), 449–458. https://doi.org/10.1111/j.1365-2923.2009.03604.x

Burgess, A., van Diggele, C., & Mellis, C. (2018). Mentorship in the health professions: A review. The Clinical Teacher, 15(3), 197–202. https://doi.org/10.1111/tct.12756

Chua, M. J., Foo, G. L., & Kwek, E. B. K. (2020). A tiered mentorship framework in an Orthopaedic Surgery Residency Programme. The Asia Pacific Scholar, 5(1), 54–60. https://doi.org/10.29060/TAPS.2020-5-1/OA2095

Dalgaty, F., Guthrie, G., Walker, H., & Stirling, K. (2017). The value of mentorship in medical education. The Clinical Teacher, 14(2), 124–128. https://doi.org/10.1111/tct.12510

de Lasson, L., Just, E., Stegeager, N., & Malling, B. (2016). Professional identity formation in the transition from medical school to working life: A qualitative study of group-coaching courses for junior doctors. BMC Medical Education, 16(1), 165. https://doi.org/10.1186/s12909-016-0684-3

Fuller, A., & Unwin, L. (1998). Reconceptualising apprenticeship: Exploring the relationship between work and learning. Journal of Vocational Education & Training, 50(2), 153–173. https://doi.org/10.1080/13636829800200043

General Medical Council. (2012, March 12). Leadership and management for all doctors. https://www.gmc-uk.org/ethical-guidance/ethical-guidance-for-doctors/leadership-and-management-for-all-doctors

Hastings, L. J., & Kane, C. (2018). Distinguishing mentoring, coaching, and advising for leadership development. New Directions for Student Leadership, 2018(158), 9–22. https://doi.org/10.1002/yd.20284

Hofler, L., & Thomas, K. (2016). Transition of new graduate nurses to the workforce. North Carolina Medical Journal, 77(2), 133-136. https://doi.org/10.18043/ncm.77.2.133

Kilminster, S., Zukas, M., Quinton, N., & Roberts, T. (2011). Preparedness is not enough: Understanding transitions as critically intensive learning periods. Medical Education, 45(10), 1006–1015. https://doi.org/10.1111/j.1365-2923.2011.04048.x

Lambert, T. W., Surman, G., & Goldacre, M. J. (2013). Views of UK-trained medical graduates of 1999–2009 about their first postgraduate year of training: National surveys. BMJ Open, 3(4), e002723. https://doi.org/10.1136/bmjopen-2013-002723

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation (p. 138). Cambridge University Press. https://doi.org/10.1017/CBO9780511815355

Le Maistre, C., & Paré, A. (2004). Learning in two communities: The challenge for universities and workplaces. Journal of Workplace Learning, 16(1/2), 44–52. https://doi.org/10.1108/13665620410521503

Lempp, H., Cochrane, M., Seabrook, M., & Rees, J. (2004). Impact of educational preparation on medical students in transition from final year to PRHO year: A qualitative evaluation of final-year training following the introduction of a new Year 5 curriculum in a London medical school. Medical Teacher, 26(3), 276–278. https://doi.org/10.1080/248-0142159042000192046

Li, H., Tan, E. L. Y., Wong, M. L., & Ong, M. M. A. (2021). Tackling study-work chasm: Perceptions of the role of mentorship in the healthcare workplace [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.14740332

Lin, J., & Reddy, R. M. (2019). Teaching, mentorship, and coaching in surgical education. Thoracic Surgery Clinics, 29(3), 311–320. https://doi.org/10.1016/j.thorsurg.2019.03.008

Lopez, N., Johnson, S., & Black, N. (2010). Does peer mentoring work? Dental students assess its benefits as an adaptive coping strategy. Journal of Dental Education, 74(11), 1197–1205. https://doi.org/10.1002/j.0022-0337.2010.74.11.tb04993.x

Matheson, C., & Matheson, D. (2009). How well prepared are medical students for their first year as doctors? The views of consultants and specialist registrars in two teaching hospitals. Postgraduate Medical Journal, 85(1009), 582. https://doi.org/10.1136/pgmj.2008.071639

Morgan, M. C., Dyer, J., Abril, A., Christmas, A., Mahapatra, T., Das, A., & Walker, D. M. (2018). Barriers and facilitators to the provision of optimal obstetric and neonatal emergency care and to the implementation of simulation-enhanced mentorship in primary care facilities in Bihar, India: A qualitative study. BMC Pregnancy and Childbirth, 18(1), 420. https://doi.org/10.1186/s12884-018-2059-8

Oxley, J., & Standing Committee on Postgraduate Medical and Dental Education. (1998). Supporting doctors and dentists at work: An enquiry into mentoring. SCOPME.

Ploeg, J., de Witt, L., Hutchison, B., Hayward, L., & Grayson, K. (2008). Evaluation of a research mentorship program in community care. Evaluation and Program Planning, 31(1), 22–33. https://doi.org/10.1016/j.evalprogplan.2007.10.002

Smith, G. B., & Poplett, N. (2002). Knowledge of aspects of acute care in trainee doctors. Postgraduate Medical Journal, 78(920), 335. https://doi.org/10.1136/pmj.78.920.335

Soklaridis, S., López, J., Charach, N., Broad, K., Teshima, J., & Fefergrad, M. (2015). Developing a mentorship program for psychiatry residents. Academic Psychiatry, 39(1), 10–15. https://doi.org/10.1007/s40596-014-0163-2

Souza, B. D., & Viney, R. (2014). Coaching and mentoring skills: Necessities for today’s doctors. BMJ, 348, g4244. https://doi.org/10.1136/bmj.g4244

Ssemata, A. S., Gladding, S., John, C. C., & Kiguli, S. (2017). Developing mentorship in a resource-limited context: A qualitative research study of the experiences and perceptions of the makerere university student and faculty mentorship programme. BMC Medical Education, 17(1), 123. https://doi.org/10.1186/s12909-017-0962-8

Straus, S. E., Johnson, M. O., Marquez, C., & Feldman, M. D. (2013). Characteristics of successful and failed mentoring relationships: A qualitative study across two academic health centers. Academic Medicine, 88(1), 82–89. https://doi.org/10.1097/ACM.0b013e31827647a0

Teunissen, P. W., & Westerman, M. (2011). Opportunity or threat: The ambiguity of the consequences of transitions in medical education. Medical Education, 45(1), 51–59. https://doi.org/10.1111/j.1365-2923.2010.03755.x

Wendlandt, N. M., & Rochlen, A. B. (2008). Addressing the college-to-work transition: Implications for university career counselors. Journal of Career Development, 35(2), 151–165. https://doi.org/10.1177/0894845308325646

Zerzan, J. T., Hess, R., Schur, E., Phillips, R. S., & Rigotti, N. (2009). Making the most of mentors: A guide for mentees. Academic Medicine, 84(1), 140–144. https://doi.org/10.1097/ACM.0b013e3181906e8f

*Eileen Lew
National Dental Centre Singapore
5 Second Hospital Avenue,
Singapore 168938
Email: elaine.tan.l.y@singhealth.com.sg

Submitted: 13 December 2021
Accepted: 22 April 2022
Published online: 5 July, TAPS 2022, 7(3), 1-9
https://doi.org/10.29060/TAPS.2022-7-3/OA2722

Claire Jing-Wen Tan¹, Cathryn Cai², Farida Ithnin² & Eileen Lew²

¹Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Women’s Anaesthesia, KK Women’s and Children’s Hospital, Singapore

Abstract

Introduction: The COVID-19 pandemic has caused significant disruption to undergraduate medical education globally, with worldwide implementation of social distancing and lock-down measures. This systematic review aims to identify challenges in undergraduate medical education during the pandemic and report creative approaches that have been employed to ensure continuity of education.

Methods: A MEDLINE (PubMed) search was performed for articles published between 1 February 2020 and 1 September 2020, corresponding to the pandemic’s global upsurge. Studies, commentaries, perspectives, and correspondence reporting challenges, adaptations and innovations in undergraduate medical education were screened. Data was grouped in accordance with the research questions it sought to address and analysed qualitatively.

Results: 43 studies were included. The COVID-19 pandemic has resulted in lost opportunities for clinical exposure, reduced motivation, and heightened anxiety among medical students. Traditional teaching and assessment methods have transitioned to virtual platforms, harnessing video conferencing, social media, and virtual or augmented reality technologies. The long-term effectiveness of these solutions is uncertain, but immediate benefits conferred include increased access, improved time management, and cultivation of self-directed learning. The lack of authentic clinical experiences and patient interaction remains a major concern. Technical challenges and digital fatigue were also highlighted as pertinent challenges.

Conclusions: The COVID-19 pandemic has provided unexpected, yet invaluable opportunities to explore new pedagogies that may be instrumental in shaping medical education in the new norm. As future members of the healthcare workforce, medical students should be integrated into current healthcare systems to equip them with knowledge and skills to manage future pandemics.

Keywords:           COVID-19, Education, Medical, Undergraduate, Medical Student, Telemedicine

Practice Highlights

• The COVID-19 pandemic has caused significant disruption to undergraduate medical education globally.
• Undergraduate medical students faced diminished clinical exposure, poor motivation, and isolation.
• The pandemic has provided unexpected yet invaluable opportunities to explore new teaching pedagogies.
• Institutions should continually adapt these strategies as methods of teaching in the new norm.

I. INTRODUCTION

The appearance of the novel coronavirus (SARS-CoV-2) was first reported in December 2019 (Huang et al., 2020). Since then, the COVID-19 outbreak has become a full-blown global health emergency, with more than 5.3 million deaths reported to date, disruptions in economies and education systems, and increased mental health issues amidst a climate of uncertainty (Torales et al., 2020). 

Major disruptions to medical education and training have been experienced worldwide. At the height of the pandemic, universities and medical schools suspended in-campus learning and clinical training to curb the spread of infections (Murphy, 2020; Theoret & Ming, 2020). Non-essential health services were ceased to focus resources on managing the pandemic. With medical faculty deployed to the frontlines, little time and capacity was available for training and teaching. There were reservations deploying medical students in clinical areas, due to safety concerns and consumption of personal protective equipment (PPE) in the face of shortages (Hilburg et al., 2020).

Mitigating measures have challenged academic institutes to explore new pedagogies to sustain and bolster education efforts beyond the crisis. Creative approaches to ensure continuity of medical student education include the transition to e-learning and virtual clerkships, enabled through online video-conferencing platforms, social media, and virtual reality technology. Assessment formats have also been adapted to accommodate virtual evaluation of clinical competencies. While the effectiveness of these approaches remains to be proven, they are instrumental in allowing medical education to continue during the pandemic and into the new norm.

This qualitative systematic review aims to define the challenges associated with medical student education at the height of the COVID-19 pandemic, highlight creative approaches and innovations that were employed to sustain medical education in these unprecedented times, and review their limitations and effectiveness.

II. METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Page et al., 2021). Literature search was performed on MEDLINE (PubMed) using a combination of the following search terms: (COVID-19 OR pandemic) AND (undergraduate medical education OR undergraduate medical training). The search was tailored for the period of 1 February 2020 to 1 September 2020, to focus on the height of the pandemic.

Articles were included if they met the following criteria: published in English; reported challenges, strategies, and adaptations in conducting undergraduate medical education during the COVID-19 pandemic. To ensure comprehensive coverage, case reports, case and cohort studies, commentaries, perspectives, and correspondence were included. Articles pertaining to postgraduate medical education and non-medical education, non-English articles and systematic reviews were excluded.

Two reviewers (CJT and CC) independently screened the titles and/or abstracts of all articles and selected only articles that met inclusion criteria. Any disagreement was resolved by discussion with senior authors (FI and EL). Data extracted included: challenges encountered in the implementation of undergraduate teaching; innovations, strategies and adaptations employed to facilitate teaching and assessment; perceived limitations and effectiveness of described strategies; original authors’ conclusions. The data was collated on a spreadsheet, grouped by categories, and analysed qualitatively.

III. RESULTS

A total of 123 studies were identified using the search strategy, of which 43 studies eventually met inclusion criteria. The PRISMA study flow-diagram (Figure 1) demonstrates our research algorithm. The data that support the findings of this study are openly available in Figshare at http://doi.org/10.6084/m9.figshare.172968   56 (Tan et al., 2021).

Figure 1: PRISMA flow-diagram of study selection

Submitted: 7 December 2021
Accepted: 8 February 2022
Published online: 5 April, TAPS 2022, 7(2), 27-36
https://doi.org/10.29060/TAPS.2022-7-2/OA2718

Tan Charmaine¹ & Ivan Cherh Chiet Low^1,2

¹Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Human Potential Translational Research Programme, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: Practicals are core components of an undergraduate health sciences curriculum to promote experiential learning and motivation in students. With restrictions on traditional forms of face-to-face practicals during the COVID-19 pandemic, we designed and investigated the efficacy of remote practicals as a viable learning strategy in exercise physiology teaching.

Methods: Student volunteers were instructed to perform a graded exercise test in a remote setting and provide their collected data for subsequent discussion in an online lecture. The effectiveness of this remote practical in promoting students’ motivation and learning outcomes achievement was assessed via an anonymous questionnaire containing 29 closed-ended and 2 open-ended items. Continual Assessment (CA) results were also investigated. Unpaired student’s t-tests were performed for comparisons between interventions with significance level set at P<0.05.

Results: Sixty-one (out of 81; 75%) students responded to the questionnaires and 49 (60%) consented to the use of their CA results for this study. Results revealed that students were moderately motivated and attained strong achievement of learning outcomes. When compared to students who did not volunteer for the hands-on component of the remote practical, students who participated in the hands-on component reported significantly higher self-efficacy (P<0.05) in explaining the practical procedures to their peers. Qualitative analysis further revealed that experiential learning and real-life data analysis were the central reasons supporting the effectiveness of the remote practical. Students were generally satisfied and would recommend the remote practical to future students.

Conclusion: Our study highlights the potential of remote practicals as viable alternatives to traditional practicals.

Keywords:           Remote Practical, Experiential Learning, Student Motivation, Learning Outcomes

Practice Highlights

• Remote practical aids in promoting experiential learning in exercise physiology teaching.
• Remote practical can promote motivation by enabling students to see the relevance of their learning.
• Students perceived that they could achieve the necessary learning outcomes via remote practicals.

I. INTRODUCTION

Laboratory work or practical classes are considered as core components of health sciences curriculum in higher education (Colthorpe & Ainscough, 2021; Dohn et al., 2016; Hofstein & Lunetta, 2004). Past studies have revealed the strong educational value of practical classes in promoting student motivation (Bruce, 1988; Dohn et al., 2016), student learning outcomes achievement (Brinson, 2015) as well as the ability to draw theory to practical applications (Neves et al., 2017).

One of the key subjects in undergraduate health sciences education is human physiology, a discipline seeking to understand the underlying mechanisms and dynamics of the human body (Lellis-Santos & Abdulkader, 2020). The role of practical classes in the teaching and learning of physiology is highly valued by educators and students alike (Dohn et al., 2016; Neves et al., 2017). Experiential learning in physiology practicals commonly takes the form of interactive hands-on activities, real-time data collection and analysis of physiological responses. When such practicals are carried out in a traditional face-to-face manner, students are able to utilise laboratory equipment in an authentic experimental setting and generate real-time data from their peers and/or themselves (Colthorpe & Ainscough, 2021). Data analysis and discussion following the hands-on component of practicals can further promote contextualised learning and facilitate the understanding of the theoretical content (Lewis & Williams, 1994). It has been reported that such an interactive learning approach in physiology enhances the achievement of learning outcomes and increases the level of motivation for students (Dohn et al., 2016).

The emergence of the COVID-19 pandemic has led to increasing safety management restrictions being imposed on physical classes in higher education institutes around the world (Ali, 2020). As such, educators were faced with the sudden need to switch from face-to-face lessons to online and remote teaching (Ali, 2020; Lellis-Santos & Abdulkader, 2020). Amidst the uncertainty of this transition, traditional face-to-face practicals have seen a sharp decline (Ray & Srivastava, 2020; Vasiliadou, 2020). As we examine these educational trends during crises, it becomes apparent that harnessing creativity to adapt and invent novel solutions is vital to maintain and even advance current standards of teaching and learning. Lellis-Santos and Abdulkader (2020) rightfully exemplify this notion by proposing the use of smartphone applications as a creative teaching approach to enable scientific data collection and practical learning from home even during social isolation. Along similar lines, we have designed a remote practical for students to carry out hands-on experiments outside of a traditional laboratory environment as an innovative alternative to circumvent the restrictions on face-to-face practicals and to provide them with experiential learning opportunities on cardiovascular concepts in exercise physiology.

To the best of our knowledge, there are few studies conducted to date regarding the efficacy of such remote practicals, particularly in the field of life sciences and exercise physiology. Therefore, our study aims to address this research gap by evaluating the effectiveness of our remote exercise physiology practical on (1) student learning outcomes, (2) student motivation and (3) students’ perceptions on the effectiveness and relevance of the remote practical. In addition, we also compared the quantitative and qualitative responses between students who participated and did not participate in the hands-on component of the remote practical. The Continual Assessment (CA) results from these two groups of students were also compared to assess if differences in academic performance existed between the two groups.

II. METHODS

A. Description of the Module

LSM3212 Human Physiology: Cardiopulmonary System is a third-year module in Life Sciences conducted by the Department of Physiology, Yong Loo Lin School of Medicine at the National University of Singapore. A total of 81 undergraduate Life Sciences students were enrolled in the module in the Academic Year 2020/2021, Semester 2. Traditionally, both lectures and practicals are carried out in a face-to-face manner for this module. However, due to restrictions imposed by the COVID-19 pandemic, lessons were forced to go online. As a result, a remote practical was designed and conducted as an innovative alternative for this batch of students.

B. Description of the Remote Practical

The remote practical was designed to provide students with experiential learning on cardiovascular concepts in exercise physiology. Conventionally, the practical involved a treadmill-based graded exercise test performed by a student volunteer equipped with specialised electrocardiogram-based heart rate monitors to illustrate how the cardiovascular system changes with increasing exercise stress. For the remote practical, however, students were given a set of practical handouts comprising a novel graded exercise protocol developed by the authors and could choose to perform it in their own time asynchronously, or not carry it out at all. In the graded exercise protocol, students were instructed to carry out a series of graded knee raise exercises and record their heart rate measured via a smartphone application together with other subjective exercise prescription ratings (ratings of perceived exertion and talk test ratings) after each set of exercise. The graded exercise protocol was developed with the intent to encourage contextualised learning from the practical content to real-life exercise routines. The consolidated data was subsequently used for discussion in a virtual lecture to illustrate how heart rate responses and cardiovascular adaptations may differ across individuals, as well as how the consolidated data may serve to guide exercise prescription. Participation in the hands-on component (knee raise exercises) of the remote practical was optional. However, participation in the analysis and discussion of the collated data in the virtual lecture conducted after the graded exercise was made compulsory. Via this design, the remote practical (comprising both the graded exercise and post-exercise discussion) not only replicated the pedagogy of the original in-class practical, but also augmented the opportunity for students to volunteer and take part in the graded exercise component of the practical.

C. Instruments

After the virtual lecture, students completed an anonymous (no informed consent required) questionnaire containing 29 closed-ended and 2 open-ended items. The purpose of this self-report questionnaire was to evaluate students’ perceived effectiveness of the remote practical on their motivation and achievement of learning outcomes of the virtual lecture.

Student motivation was measured by the Lab Motivation Scale (Dohn et al., 2016) containing 21 closed-ended statements based on three aspects – student interest, effort and self-efficacy. Multiple instruments had previously been employed to assess dimensionality and reliability of the validated Lab Motivation Scale (Dohn et al., 2016). A set of six closed-ended items were employed to measure students’ perception on whether they had achieved the intended learning outcomes of the remote practical. Lastly, two closed-ended items were included to elicit a general satisfaction score from students regarding the remote practical and/or the virtual lecture. All the closed-ended statements in the questionnaire were scored on a 5-point Likert Scale, ranging from 5 (strongly agree) to 1 (strongly disagree).

Furthermore, there were two open-ended questions focusing on the effectiveness of the remote practical and the relevance of the remote practical to students’ daily lives. The first question was posed to uncover specific reasons supporting the analysis of the closed-ended items, while the second question aimed to encourage contextualisation of concepts learnt through the remote practical in students’ daily lives.

Finally, the CA results of students who participated in the hands-on component of the remote practical were compared with students who did not participate. To ensure a fair comparison, we took into consideration only the CA result from the questions corresponding to the content covered in the remote practical and virtual lecture. The questions taken into consideration made up 40% of the entire examination score.

D. Analysis

A mixed method approach was employed in analysing the questionnaire responses. An initial 66 questionnaire responses were collected but five incomplete responses were excluded, leaving a total of 61 responses that were included in the final analysis. Responses to the closed-ended items were coded accordingly to a 5-point Likert scale, ranging from strongly agree (5) to strongly disagree (1). Scores were reversed for statements phrased in a negative manner (items 9, 15 and 18). All closed-ended responses were presented in the form of mean ± standard deviation. As a measure of internal consistency, Cronbach’s α was used as an instrument and measured across all scales. Unpaired student’s t-tests were carried out to find out if differences between students who participated and did not participate in the hands-on exercise component were significant with significance level set at P < 0.05. All data analysis and statistical tests were performed using Microsoft Excel 2016.

Open-ended responses were analysed in a 3-part process: informal reviewing, open coding and thematic analysis. Firstly, all responses were informally reviewed to familiarise with general ideas and main themes were identified. Next, open coding was performed where each response was analysed in detail and coded to the most appropriate theme (Braun & Clarke, 2006). Finally, thematic analysis was carried out through ranking themes according to frequency and analysing the results (Braun & Clarke, 2006). The open-ended questions were made optional, and all responses collected were subsequently analysed.

III. RESULTS

Out of the 61 participants, 29 (48%) participated in the hands-on exercise component of the remote practical while 32 (52%) did not participate. Both groups attended the compulsory virtual lecture conducted after the remote practical, where the data collected from the remote practical was consolidated and discussed with the entire class.

Internal consistency was calculated using Cronbach’s α and the reliability coefficient was found to be 0.95 across all closed-ended items, indicating an excellent level of interrelatedness across the overall scale (Cronbach, 1951). Individual scales of learning outcomes and motivation were also subjected to the analyses of Cronbach’s α. The alpha coefficient value was calculated to be 0.86 for perceived achievement of learning outcomes and 0.94 for motivation (Tables 1 & 2). Motivation was further divided into three individual subscales assessing student interest, effort and self-efficacy, with the reliability coefficients returning 0.85, 0.88 and 0.88 respectively (Table 1). These reliability coefficients correlate strongly with those of Dohn et al. (2016), hence providing support for the internal consistency of the Lab Motivation Scale.  Data supporting these findings is openly available via Figshare at https://doi.org/10.6084/m9. figshare. 17170 964 (Low, 2021).

A. Remote Practical and Learning Outcomes

In general, students rated between “Agree” and “Strongly Agree” for perceived achievement of learning outcomes, with an overall mean score of 4.16 ± 0.68 on a 5-point Likert scale (Table 1). Students who participated in the hands-on component reported a mean score of 4.20 ± 0.54, which was similar to that rated by students who did not participate in the hands-on component (4.13 ± 0.68, P = 0.567; Table 1). For the first closed-ended statement: I have gained a stronger understanding of how heart rate responds to increasing exercise intensity, students who participated indicated a higher mean score of 4.52 ± 0.51 as compared to the lower mean score of 4.09 ± 0.59 (P = 0.004) for students who did not participate in the hands-on exercise component (Table 1).

Table 1. Students’ perceived achievement of learning outcomes in cardiovascular physiology

n = 61. Responses were coded from 1 (Strongly Disagree) to 5 (Strongly Agree). All means are shown with ±SD of the mean. **P < 0.01.

B. Remote Practical and Student Motivation

Students generally rated between “Agree” and “Somewhat Agree/Somewhat Disagree” for student motivation, with an overall mean score of 3.66 ± 0.71 (Table 2). Students who participated in the hands-on component reported a mean score of 3.70 ± 0.64, which was similar to that rated by students who did not participate in the hands-on component (3.62 ± 0.78, P = 0.187; Table 2). Students rated between “Agree” and “Somewhat Agree/Somewhat Disagree” regarding the three aspects of student motivation, with a score of 3.92 ± 0.69 for interest, 3.66 ± 0.75 for effort and 3.50 ± 0.68 for self-efficacy respectively (Table 2). For statement 23, students who participated indicated a higher mean score of 3.68 ± 0.61 as compared to the lower mean score of 3.29 ± 0.71 (P = 0.031) for students who did not participate in the hands-on component (Table 2).

Table 2. Students’ perceived motivation towards the remote practical

n = 61. Adapted from the Lab Motivation Scale (Dohn et al., 2016). Responses were coded from 1 (Strongly Disagree) to 5 (Strongly Agree). Scores were reversed for statements phrased in a negative manner. All means are shown with ± SD of the mean. **P < 0.05.

C. Qualitative Explanations on Perceived Effectiveness and Relevance of Remote Practical

The first open-ended item sought to investigate the reasons underlying the perceived effectiveness or ineffectiveness of the remote practical in enhancing students’ learning. Of the 37 responses, 34 (92%) felt that the remote practical was effective while 3 (8%) felt it was ineffective and of little to no added value to them (Table 3). Experiential learning and real-life data emerged as the most common themes cited across all responses (n = 12), followed by reinforc(ing) concepts taught in lecture (n = 10; Table 3). Experiential learning was reflected as the most common response among students who participated in the hands-on component (n = 10) in comparison to real-life data indicated by students who did not participate in the hands-on component (n = 8; Table 3).

The aim of the second open-ended item was to investigate the relevance and application of the remote practical to students’ daily lives. Of the 36 responses, 29 (81%) felt the remote practical was relevant while 7 (19%) felt that it was irrelevant to their daily lives (Table 3). Overall, the remote practical was found to be most relevant in improving current exercise routine (n = 12), followed by understanding one’s own fitness level (n = 10) and understanding the importance of exercise (n = 7; Table 3). This trend was similar for both students who participated and did not participate in the hands-on component of the remote practical (Table 3). 

Table 3. Themes identified from the open-ended responses, ranked by frequency

D. Remote Practical and Satisfaction Score

Students rated close to “Agree” for satisfaction, with an overall mean score of 3.95 ± 0.75 (Table 4). Those who participated in the hands-on component reported a mean score of 4.11 ± 0.70, which was similar to that rated by students who did not participate in the hands-on component (3.81 ± 0.77, P = 0.054; Table 4).

Table 4. Students’ satisfaction score

n = 61. Responses were coded from 1 (Strongly Disagree) to 5 (Strongly Agree). All means are shown with ± SD of the mean. **P < 0.05.

E. Remote Practical and Academic Performance

Out of the 49 students who consented to the use of their CA results for this research study, 30 (61%) participated in the hands-on component of the remote practical while 19 (39%) did not (Table 5). Only the questions corresponding to the content covered in the remote practical and relevant virtual lecture were taken into consideration for this study. The overall mean mark was 7.3 ± 1.64 out of 10 (Table 5). Students who participated in the hands-on component (7.3 ± 1.84) and did not participate in the hands-on component (7.3 ± 1.32) exhibited similar mean marks as well (P = 0.940; Table 5).

Table 5. Students’ CA results

n = 49. CA scores are shown as mean ± SD, with *P < 0.05 considered significant.

IV. DISCUSSION

This study sought to evaluate if remote physiology practicals could be viable alternatives to traditional face-to-face practicals, especially during emergency remote teaching in a pandemic. Our results demonstrated that the students who participated in the remote practical perceived that they could achieve the learning outcomes in cardiovascular and exercise physiology teaching with reasonable satisfaction, regardless of whether they participated in the hands-on component of the remote practical. However, students who had actively participated in the hands-on component (graded exercise) reported that the remote practical had particularly benefitted them in better achieving certain learning outcomes as compared to their classmates who did not participate in the graded exercise. Specifically, students indicated that their participation in the graded exercise allowed them to have a better grasp of the concepts concerning heart rate response to increasing exercise intensity. This finding was not unexpected as the remote graded exercise was specifically designed to provide experiential learning opportunities to better comprehend the concepts underlying this particular learning outcome.

Perceived scores for the achievement of learning outcomes were otherwise similar between the participated and non-participated group. This could be attributed to the fact that the remote practical was used as a complement to the virtual lecture, wherein the interpretation and analysis of data collected from the optional graded exercise was discussed with the whole class during the compulsory virtual lecture. The perception scores of learning outcome achievement were well supported by the students’ academic performance as all of them shared similar mean CA marks regardless of their participation in the remote practical. This similarity is especially prominent as the CA questions were set based on the principle of constructive alignment (Biggs, 1996; Bloom, 1956; Stamov Roßnagel et al., 2020).

Interestingly, open-ended responses revealed “experiential learning” as the key reason supporting the efficacy of the remote practical in students who participated in the graded exercise while “having real-life data which reinforces concepts taught in lecture” were key reasons indicated by students who did not participate in the hands-on component of the remote practical. These findings are in line with studies recommending experiential learning as one of the seven “principles of good practice” to achieve excellence in higher education (Chickering & Gamson, 2006). This is accomplished by generating real-life data to allow students to draw the link between theoretical content and practical applications, before applying it to analyse real-life situations in view of course material (Lewis & Williams, 1994). This suggests that the remote practical is able to foster environments which could encourage hands-on learning and real-time data generation to enhance student learning, even if not conducted in a traditional laboratory setting.

Overall, students were satisfied with the remote practical and/or virtual lecture, with those who participated in the hands-on component generally being more satisfied. Even though the remote practical was not compulsory, those who took part in the hands-on component generally did not view it as an additional academic burden and instead would recommend it to future batches of students. This reinforces the potential of such remote practicals in helping students to achieve learning outcomes without imposing unnecessary pressure on them. 

With motivation being a strong indicator of self-directed learning and academic achievement (Cortright et al., 2013), it is crucial for educators to assess and understand the importance of motivating students. In fact, educators play a critical role in determining the motivation levels of their students through the nature of their classes and assignments (Cortright et al., 2013). Specifically, Dohn et al. (2016) states that students’ motivation could be negatively impacted by limited equipment or restricted time for practicals. Majority of students face similar limitations for a graded exercise test carried out in a traditional face-to-face practical. Typically, only one volunteer would carry out the actual exercise experimental protocol due to equipment and time constraints, while other students would passively watch and learn from the data collected. The novel remote practical proposed in this study could potentially overcome these limitations as students are able to personally experience the hands-on exercise component within their own spaces at home and at their own convenience, thereby possibly enhancing their motivation levels.

Our results indicated that overall motivation towards the remote practical and/or the virtual lecture ranged from 3 to 5, corresponding to “somewhat agree” and “strongly agree”, with a mean value of 3.62 ± 0.78. This is comparable to the motivation scores previously reported by Dohn et al. (2016) for in-class biomedical laboratory classes. The positive motivation score could be explained by the fact that majority of students (81%) could see the relevance (Table 3) of the remote practical in their daily lives. Learning activities which guide students towards finding ‘personal meaning and value’ in the educational content is known to positively influence their motivation levels (Cortright et al., 2013). By providing opportunities for students to reflect on, find meaning and draw relevance to their personal lives, such remote practicals can potentially address common limitations of traditional practicals and boost student motivation and learning.

Delving further into the three aspects of student motivation – interest, effort and self-efficacy, students rated the highest scores for interest, followed by effort and lastly self-efficacy. The score for effort placed into the practical could have been understandably affected due to the non-compulsory nature of the graded exercise. The exercise component of the practical could not be made compulsory as not all students were medically/physically fit enough to undergo a graded exercise test. Nonetheless, the similarity in perceived learning outcomes and academic results between students who did and did not participate in the graded exercise suggests that the follow up analysis and peer-based discussion of the tabulated data involving the entire class was sufficient to bridge the learning gap between the two groups of students. Overall, the favourable perceived learning outcomes (ranging from “agree” to “strongly agree”) and academic scores (corresponding to a grade of “A-” to “A”) reinforces the value of the remote practical as a teaching strategy to promote learning in exercise physiology, regardless of the students’ ability or interest to participate in strenuous physical activity. However, whether the remote practical is more effective than a conventional face-to-face practical or no practical at all remains an interesting question which necessitates future research as this cannot be addressed given the limitations of our current study design.  

Notably, self-efficacy scores were rated the lowest amongst the three aspects of motivation. This could be due to the fact that students are not closely supervised during a remote practical, unlike face-to-face practicals. Without the physical presence and continuous guidance of an instructor, students could have faced uncertainty as to whether instructions were properly executed. Thus, strategies to enhance pre-practical instructions using asynchronous video instructions or the incorporation of remote supervision methods may aid to further enhance the effectiveness of the remote practical. Interestingly, participation in the hands-on component of the remote practical appeared to have nonetheless enhanced the confidence of students in explaining the procedures of the practical to their peers (Table 2). This finding is of particular importance, as the ability to teach and explain is an indication of higher order learning corresponding to the second and third levels of the Bloom’s taxonomy (Bloom, 1956).

A. Limitations

Our study sought to evaluate the effectiveness of a remote exercise physiology practical in promoting student motivation and learning in a cohort of Life Sciences undergraduates. However, the current study design does not permit immediate comparison with conventional face-to-face practicals as students could not be randomly allocated into different comparison groups (remote or face-to-face) owing to pandemic restrictions and ethical reasons. Also, we could only investigate the effects of practical participation on the effectiveness of the remote practical in enhancing student motivation and learning outcomes achievement using a quasi-experimental approach. This is so, for we were unable to randomly allocate students into two comparison groups given that not all students were medically/physically fit enough to undergo a graded exercise test for the hands-on component of the remote practical. Based on this study design, some degree of self-selection bias could have been present as physically active students who had volunteered to take part in the exercise component of the remote practical could have seen greater relevance to their daily lives and could have been more intrinsically motivated to partake and learn from the practical session. In spite of that, the overall student motivation score appeared comparable between the two groups of students who participated and did not participate in the exercise component of the practical, suggesting that the degree of self-selection bias may not be of significant concern in the present study.

V. CONCLUSION

Overall, students reported that experiential learning and real-life data were the main reasons supporting the effectiveness of the remote practical. With experiential learning and real-life data as key components of traditional practicals (Dohn et al., 2016; Randall & Burkholder, 1990), the present study demonstrates the potential of remote practicals as viable and innovative alternatives for face-to-face practicals in exercise physiology teaching. In cases of sudden shifts to emergency remote education, such alternatives offer the possibility of incorporating experiential learning even during social isolation. 

Notes on Contributors

Tan conducted the study, analysed and interpreted the data, and drafted the manuscript. I.C.C. Low was involved in experimental conception and design, as well as critically reviewed and revised the manuscript. All authors have read and approved the final version of the manuscript.

Ethical Approval

All students were provided with a participant information sheet containing the purpose and details of the research study. The questionnaire was made completely anonymous. Informed consent was obtained from students for use of their CA results only after the release and confirmation of their results. Circulation of research materials was done virtually and students were never approached directly for recruitment. The study was approved by the National University of Singapore – Institutional Review Board (NUS-IRB) with study code NUS-IRB-2020-631.

Data Availability

Data supporting these findings is openly available via Figshare at DOI: https://doi.org/10.6084/m9.figshare. 17170964.

Acknowledgements

We greatly appreciate the study participants who have spent their time and effort to complete the questionnaires and have provided consent in the use of their results for research purposes.

Funding

There was no funding support accorded for this study.

Declaration of Interest

No potential conflict of interest was reported by the authors.

References

Ali, W. (2020). Online and remote learning in higher education institutes: A necessity in light of COVID-19 pandemic. Higher Education Studies, 10(3), 16-25. https://doi.org/10.5539/hes.v10n3p16

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364. https://doi.org/10.1007/BF00138871

Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. David McKay.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brinson, J. R. (2015). Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: A review of the empirical research. Computers & Education, 87, 218-237. https://doi.org/10.1016/j.compedu.2015.07.003

Bruce, R. H. (1988). Enhancing student motivation and learning: Experiences with a simple simulation. Higher Education Research & Development, 7(2), 103-110. https://doi.org/10.1080/0729436880070201

Chickering, A., & Gamson, Z. (2006). Seven principles of good practice in undergraduate education. New Directions for Teaching and Learning, 1991, 63-69. https://doi.org/10.1002/tl.37219914708

Colthorpe, K., & Ainscough, L. (2021). Do-it-yourself physiology labs: Can hands-on laboratory classes be effectively replicated online? Advances in Physiology Education, 45(1), 95-102. https://doi.org/10.1152/advan.00205.2020

Cortright, R. N., Lujan, H. L., Blumberg, A. J., Cox, J. H., & DiCarlo, S. E. (2013). Higher levels of intrinsic motivation are related to higher levels of class performance for male but not female students. Advances in Physiology Education, 37(3), 227-232. https://doi.org/10.1152/advan.00018.2013

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555

Dohn, N. B., Fago, A., Overgaard, J., Madsen, P. T., & Malte, H. (2016). Students’ motivation toward laboratory work in physiology teaching. Advances in Physiology Education, 40(3), 313-318. https://doi.org/10.1152/advan.00029.2016

Hofstein, A., & Lunetta, V. N. (2004). The laboratory in science education: Foundations for the twenty‐first century. Science education, 88(1), 28-54. https://doi.org/10.1002/sce.10106

Lellis-Santos, C., & Abdulkader, F. (2020). Smartphone-assisted experimentation as a didactic strategy to maintain practical lessons in remote education: Alternatives for physiology education during the COVID-19 pandemic. Advances in Physiology Education, 44(4), 579-586. https://doi.org/10.1152/advan.00066.2020

Lewis, L. H., & Williams, C. J. (1994). Experiential learning: Past and present. New Directions for Adult and Continuing Education, 1994(62), 5-16. https://doi.org/10.1002/ace.36719946203

Low, I. C. C. (2021). Remote physiology practical: Viable alternative to in-person practical in health sciences education? [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.17170964

Neves, B. S., Altermann, C., Goncalves, R., Lara, M. V., & Mello-Carpes, P. B. (2017). Home-based vs. laboratory-based practical activities in the learning of human physiology: The perception of students. Advances in Physiology Education, 41(1), 89-93. https://doi.org/10.1152/advan.00018.2016

Randall, W. C., & Burkholder, T. (1990). Hands-on laboratory experience in teaching-learning physiology. Advances in Physiology Education, 259(6), S4-S7. https://doi.org/10.1152/advances.1990.259.6.s4

Ray, S., & Srivastava, S. (2020). Virtualization of science education: A lesson from the COVID-19 pandemic. Journal of proteins and proteomics, 11, 77-80. https://doi.org/10.1007%2Fs42485-020-00038-7

Stamov Roßnagel, C., Fitzallen, N., & Lo Baido, K. (2020). Constructive alignment and the learning experience: Relationships with student motivation and perceived learning demands. Higher Education Research & Development, 40(4), 838-851. https://doi.org/10.1080/07294360.2020.1787956

Vasiliadou, R. (2020). Virtual laboratories during coronavirus (COVID‐19) pandemic. Biochemistry and Molecular Biology Education, 48(5), 482-483. https://doi.org/10.1002/bmb.21407

*Ivan Cherh Chiet LOW
Department of Physiology,
Yong Loo Lin School of Medicine
Block MD9, 2 Medical Drive Level 4
Singapore 117593
Email: phsilcc@nus.edu.sg

Submitted: 6 April 2021
Accepted: 2 December 2021
Published online: 5 April, TAPS 2022, 7(2), 17-26
https://doi.org/10.29060/TAPS.2022-7-2/OA2510

Nicola Ngiam^1,2, Kristy X Fu^1,3 & Jacqueline SM Ong^1,3

¹Khoo Teck Puat- National University Children’s Medical Institute, National University Health System, Singapore; ²Centre for Healthcare Simulation, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Department of Paediatrics, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: Personal protection in aerosol-generating procedures is an important skill to safely deliver care to patients in the COVID-19 pandemic. This aim of this study was to evaluate the impact of “just-in-time” simulation training for airway management in a suspected COVID-19 patient.

Methods: This was a prospective mixed-method cohort study in a tertiary paediatric department. A mandatory “just-in-time” simulation training session for intubation of a suspected COVID-19 patient was conducted. Pre- and post-simulation questionnaires were administered. Participants were invited to attend focus group interviews to further delineate their experience. Quantitative and qualitative methods were employed to analyse the data.

Results: Thirty-three participants, including doctors, nurses and respiratory therapists attended the training. Self-confidence in intubation, managing and leading a resuscitation team and dealing with problems with intubation significantly improved. Simulation was valued for the experiential learning as well as for increasing confidence and awareness. Process improvement suggestions from both participants and trainers were raised.  There was a small signal of skill translation to real life scenarios.

Conclusion: Simulation-based training is a useful tool for infectious disease outbreak preparedness. Further research will need to be done to determine the impact on actual clinical practice in pandemics.

Keywords:           Simulation, COVID-19, Pandemic Preparedness, Training, Intubation

Practice Highlights

• The authors report their experience with “just-in-time” in situ simulation training for emergency preparedness in the face of an infectious disease outbreak.
• Simulation training was well received and improved the confidence as well as awareness of frontline staff in managing intubation and resuscitation in a suspected COVID-19 paediatric patient.
• Process improvement suggestions from participants and trainers was a useful by-product of the simulation training activity.

I. INTRODUCTION

Since the World Health Organization (WHO) declared the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) outbreak a public health emergency of international concern on 30 January 2020 (World Health Organization, 2020), the COVID-19 pandemic has now affected millions of people worldwide, with a wide range of case fatality rates amongst the 210 countries and territories affected (The Centre for Evidence-Based Medicine Nuffield Department of Primary Care Health Sciences University of Oxford, 2020). In Wuhan, China, one of the first epicentres of this international pandemic, out of 44672 confirmed cases of COVID-19, 1716 were healthcare workers (HCWs) (Wu & McGoogan, 2020). Of the confirmed cases among HCWs, 14.6% were classified as severe or critical, and 5 deaths were observed (The Novel Coronavirus Pneumonia Emergency Response Epidemiology Team, 2020). Early reports suggest that modes of transmission of SARS-CoV-2 include droplet and contact (via the fecal-oral route and fomites); airborne transmission may occur during aerosol-generating procedures (World Health Organization, 2020).

During the 2003 SARS-CoV outbreak, despite the presence of existing safety protocols, up to half of the SARS-CoV cases in some centers were HCWs as a result of transmission within hospital units (Booth et al., 2003). Critical care and emergency care providers are often involved in high-risk aerosol-generating procedures such as oronasal suctioning, bag-valve-mask ventilation, non-invasive positive pressure ventilation, and endotracheal intubation in patients with COVID-19 with respiratory failure, and so must be prepared (Caputo et al., 2006; Wax & Christian, 2020; Zuo et al., 2020). The challenge for providers would be to provide high-quality and timely care to infected patients, without compromising their own safety. Apart from having adequate supplies of personal protective equipment (PPE), a safe environment for HCWs requires the provision of up-to-date information regarding the disease, smooth dissemination of protocols, and easy accessibility to portals reinforcing education and training in infection control procedures. Lau et al. (2004) reported that HCWs who underwent more than 2 hours of training were far less likely to be infected with SARS-CoV during the 2003 SARS-CoV outbreak in Hong Kong. In addition to PPE training, we felt that a simulated airway management scenario in a suspected COVID-19 patient was a vital component of training in order for staff to experience the profound challenges of performing high risk aerosol-generating procedures while in PPE and in isolation facilities. During the recent 2014 to 2016 West Africa Ebola outbreak, Grillet et al. (2015) with the use of simulation, found that commonly performed procedures in the intensive care unit becomes more complicated, more stressful and less comfortable in appropriate PPE. We were fortunate to find a window of opportunity close to the beginning of the COVID-19 outbreak in Singapore to prepare our healthcare teams using “just-in-time” in-situ simulation. The main objective of this study was to evaluate the impact of this training on our resuscitation teams when managing a respiratory emergency of a suspected COVID-19 pediatric patient.

II. METHODS

This was a prospective mixed-method cohort study in a paediatric department of a tertiary university hospital. Residents were put through a mandatory “just-in-time” simulation training session to prepare them for intubation of a suspected COVID-19 patient. The doctors involved were senior paediatric residents who regularly manage emergencies on call. Nurses and respiratory therapists were rostered to participate whenever available on shift. Participants worked in teams of 5 to 6 to manage a simulation scenario involving a patient with bronchiolitis who was suspected to have COVID-19. Each team comprised of participants from each of the healthcare professional groups listed above.  Actual personal protective equipment (PPE) including powered air-purifying respirators (PAPR) were used in the simulation. Participants received training in the use of PPE and PAPR prior to the simulation session. The simulation was conducted in-situ in the paediatric intensive care unit (PICU) using a SimBaby (Laerdal). The scenario used can be found in the appendix. Each group participated in one scenario. Each scenario lasted 30 to 40 minutes. The end point was successful intubation of the patient. There were 2 instructors (paediatric intensive care clinicians) present, one in the anteroom and one in the patient’s room. Debriefing was conducted as a group by both instructors immediately after every simulation. Each debriefing session lasted 30 to 45 minutes and focused on technical and non-technical skills.

Anonymous pre-simulation and post-simulation questionnaires were administered. Questions focused on confidence levels with managing intubation in a COVID-19 suspect patient, using the PAPR, and anxiety levels. Responses were rated on a Likert scale of 1 to 5. Quantitative data was analysed using Microsoft EXCEL, 2016. Pre and post responses on the Likert scale were analysed using the paired t-test.

After the simulation and debriefing, participants were invited via email to attend focus group interviews to gain better insight into their responses and attitudes towards the simulation sessions. The format and logistics of the interviews were included in a participant information sheet in the email. Participation was voluntary. We aimed for maximal representation from the various groups of healthcare professionals who participated in the simulation. A semi-structured interview was conducted by 2 researchers (NN, JO) in groups of 3 to 5 and interviews were audio-recorded. Two focus group interviews were conducted. Participants were asked questions regarding how they felt, what they learnt and what the benefits of the simulation experience were. Each interview lasted 30 to 40 minutes and were conducted in a quiet room in the PICU. Audio recordings were transcribed verbatim and anonymised at the point of transcription. Participants were only identified by role in the transcript.

Thematic analysis of the transcripts was performed by the 2 interviewers using an inductive approach. Each interviewer coded the data independently, after which both interviewers discussed the codes and generated common themes together. Data was reviewed for commonality in responses, degree of specificity (detailed explanation), and extensiveness (number of different people who had similar responses). Where there was disagreement, review of the data and existing literature was used for resolution. Descriptive summaries were developed for each theme, and participant quotes provided further evidence for interpretation and recommendations made.

This study was approved by the institutional review board (National Healthcare Group, Domain Specific Review Board, NHG DSRB Ref. 2020/00234) and waiver of consent was obtained.

III. RESULTS

A. Pre- and Post-Simulation Questionnaire Responses

A total of 33 participants took part in the training and completed the pre- and post-simulation questionnaires. There were 19 doctors, 12 nurses and 2 respiratory therapists. Confidence, as assessed by the questionnaire, increased significantly after the simulation in the areas of intubation, use of the PAPR, airway management in a COVID-suspect patient, leading a team and dealing with problems that arise during resuscitation (p < 0.05). Regarding anxiety over intubating a COVID-suspect patient, most participants were less anxious after the simulation (p < 0.05). Interestingly, on looking more closely at the responses, 9% (n=3) of participants were more anxious after the simulation than before they started. 42% (n=14) reported no change in their level of anxiety. Table 1 shows the mean scores of the questions asked in the pre- and post-simulation questionnaire.

Table 1: Pre- and post-simulation responses

After the simulation training, 96% of participants agreed or strongly agreed that they were more aware of who to call for help and how to communicate effectively when wearing PPE or PAPR while in an isolation room and 93% agreed or strongly agreed that they were more able to anticipate problems. All participants agreed that simulation training was useful in preparing to deal with a similar situation while on call and in fact, 23% disagreed or strongly disagreed that 1 simulation session was sufficient. Figure 1 illustrates the perceived effectiveness of simulation training by participants.

Figure 1: Post-simulation responses on effectiveness of simulation training

B. Focus Group Interviews

Eight participants agreed to focus group interviews, 4 doctors, 3 nurses and 1 respiratory therapist. Comments from the focus group interviews helped to further delineate the benefits and lessons learnt from this simulation exercise. Key benefits were grouped into the following themes:

1) Experiential learning: Participants commented that getting to put theory into practice during the simulation was key to understanding what to expect. Being able to practice before an actual patient encounter helped them to be more prepared. Even simple things like setting an intravenous cannula could not be accomplished with ease. Participants stated that they had to rethink the way that things needed to be done as how they were done previously would not work in this situation. This led to a paradigm shift in the clinical processes and also the application of knowledge. For example, a lesson learnt was that there is a lower threshold for intubation in COVID-19 patients. In the words of a participant about his experience:

“But the fact that you actually go through it, firstly you realize that it takes 3 times the length of the duration of what you would do, and the difficulties in communicating with the people around. So I thought that was the most…an awakening…. the most beneficial part.”

Participants appreciated that this was a complex scenario that was high stakes for the healthcare team as well as for the patient. Getting to practice this, even though it was likely to be an infrequent occurrence, helped with familiarity with protocols and equipment used. Participants also felt that more people should attend this training and that they needed more practice.

2) Increasing confidence: Participants felt that the simulation training was a timely intervention in light of the developing COVID-19 situation. This experience took away the “first-time” feeling and made the participants feel more comfortable with the protective equipment. This took away some of the anxiety and fear about their own safety and the safety of their COVID-19 patients. One participant stated that more practice in simulations may reduce the resistance to wearing the PAPR, which may in itself pose a safety issue if PAPRs are not used when required. They felt that this experience made them a more valuable member to the resuscitation team.

3) Increasing awareness: Participants appreciated the feedback and debriefing that came after the simulation. Experiencing the simulation made taught them to anticipate problems, plan ahead and to prioritize as resources were less accessible than usual. They developed insight into a possible “unconscious incompetence” as they expressed that if they had not gone through the simulation, they would not have known what they did not know and would be inappropriately confident in a real situation. Exploring resource management and considering aspects of waste reduction were thought to be beneficial for future encounters with similar situations. Some of our participants said:

“It really gives you…. the best kind of idea of what to expect in a real life scenario.”

“I would feel quite…. in a sense, bad for the patient that I was doing it for the first time on that patient. So I do feel that it is a responsible thing to do, as healthcare providers that we… that this was actually undertaken.”

“If we had not had this, I probably won’t even have that fear of how terrible it could have been……… But if I had not known, I would still be confident not knowing what I’m expecting.”

Some participants mentioned developing more empathy for colleagues caring for COVID-19 patients through this experience. However, one participant found herself more frustrated after going through the simulation as the experience of managing a resuscitation in this situation was worse than she had thought it would be.

4) Process improvement: In the focus group discussions, participants mentioned the need for process improvement that was discovered while encountering difficulties during the simulation scenario. Communication challenges were brought up multiple times and participants suggested making laminated signs for communication with staff outside the room when assistance was required, having pen, paper and drug labels in the room, as well as using intercoms or walkie-talkies for more efficient communication.

5) Translation: A couple of participants encountered a real subsequent clinical situation which required them to apply skills that they had learnt during the simulation training. The fact that they had gone through the training made them less anxious and more able to take control of the emergency situation. The respiratory therapist in the group expressed more confidence in the nurses that had gone through the simulation training when compared to those who had not when managing a COVID-19 patient. Participants also commented that the skills learnt could be translated to a different institution in the future and perhaps also to a different infective agent. Participants expressed the following:

“And so it would make you, I think… more valuable as a team member in providing care for a COVD patient or any other sort of respiratory pathogen X anywhere.”

“Because I observe that those nurses that attended the simulation, in the actual scenario or in the actual patient handling, they are more confident and competent in doing their PPE and flow of the sequence inside.”

“Because for me, for example, I actually had to initiate and help…coordinate initiation of ECMO for a COVID suspect patient and so I think if not for that…that mock… I would probably have been much more nervous and unsure.”

Key challenges encountered during the simulation were related to the simulation scenario and to the process of simulation itself. The most emphasised challenges encountered during the simulation were cognitive overload and communication barriers. Not only did the team have to deal with a deteriorating patient who was suspected to have COVID-19, they also had to deal with infection control protocols and the inherent challenges that it posed to the resuscitation process. As many tasks needed to be coordinated in a stressful situation, prioritisation was key. Communication barriers came in the form of restricted movement in and out of the isolation room, not being able to use their mobile devices, hearing impairment with the PAPR on and not being able to be heard clearly while wearing the N95 mask. Other challenges raised relating to the scenario were unfamiliarity with the safety equipment as it was not used frequently and having to wait a long time for orders to be carried out. An interesting challenge brought up was a tension between the patient’s safety or well-being and healthcare worker’s own safety. Healthcare professionals frequently put their patients first and in this unique situation, the need for donning personal protective equipment before entering the patient’s environment does not allow for as rapid a response to a deteriorating patient as they are used to:

“I feel like I need to go in as soon as possible but donning the PAPR takes some…. more time than I thought.”

Logistical challenges in planning for this simulation training included the availability and timely attendance of all frontline healthcare workers. With the developing crisis, healthcare professionals were kept busy with their clinical responsibilities, therefore this simulation training was seen as an unwelcome interruption. There was also a perceived resistance to participating by the nurses although the reasons are unclear.

Participants also commented on the design of this simulation training. Prior training in the process of intubation and use of PPE and PAPR were deemed necessary to optimize the benefits of this exercise. Participants appreciated that there were 2 trainers, 1 in the ante-room and 1 in the patient’s room, who were observing different processes and able to give valuable feedback during the debriefing. They also appreciated that the simulation exercise was done in-situ, and therefore was realistic in design.

Suggestions for improvement included providing a variety of clinical scenarios, training junior physicians so that the whole medical team is familiar with the processes, mimicking the typical manpower available on-call in the simulation exercise, and increasing the complexity of the scenarios to address resource allocation issues faced in a pandemic.

IV. DISCUSSION

With the development of the COVID-19 outbreak and patients requiring intensive care, our unit thought it would be imperative to train our frontline staff to be prepared to manage airway emergencies and to be able to resuscitate infected patients. Simulation provides a safe environment for mistakes to be made without compromising patient safety (Ziv et al., 2000). It also provides a platform for deliberate practice (Ericsson, 2004) and not allow for a chance clinical encounter to develop expertise in an area. Simulation has also been utilised in high consequence infectious outbreak training, resulting in improved self-efficacy, reduced anxiety and improved inter-professional teamwork (Marrs et al., 2019; O’Keeffe et al., 2016). As the consequences for patient and individual safety were high in the COVID-19 outbreak, we felt it was prudent to use simulation-based education as a platform for upskilling our staff.

Despite training in the use of personal protection equipment (PPE), including the powered air-purifying respirator (PAPR), Watson et al found that there was an inadequate adherence to the use of PPE and resuscitation guidelines in simulated cardiac arrest in paediatric influenza (H1N1) patients (Watson et al., 2011). Intensive care procedures have been found to be more difficult, stressful and uncomfortable when performed in PPE (Grillet et al., 2015). Simulation training has also been able to detect breaches in infection control procedures (Seet et al., 2009) and potentially improve compliance with infection prevention measures (Tan et al., 2021).

For the above reasons, healthcare professionals who worked in the paediatric intensive care unit were put through a mandatory simulation-based training session on airway management of a deteriorating patient who was suspected to have COVID-19 infection. An in-situ model was chosen as it could be used to evaluate system competence and identify latent conditions that predispose to medical error (Patterson et al., 2013). In this study, in situ simulation provided a means to identify areas for process improvement and knowledge gaps. It provided the ability to test the actual clinical care system, including equipment, processes, and staff response. This form of “just-in-time” training takes place in close proximity to the clinical encounter in a focused concise manner (Itoh et al., 2019). This provides the participants with contextually relevant hands-on experience in dealing with an impending low-frequency event before it actually happens. This has shown to improve confidence levels and clinical skills (Sam et al., 2012).

As expected, there was improvement in self-efficacy as shown in the improvement in pre- and post-simulation responses for all questions relating to management of the patient and clinical team. This has been shown in many previous studies using simulation as a methodology (McLaughlin et al., 2019; Secheresse et al., 2020). Evidence does suggest that clinicians have a limited ability for self-assessment of competence (Davis et al., 2006) and self-assessment. From the Kirkpatrick levels of evaluation, this would be a level 2 evaluation of knowledge, skills and attitudes (Kirkpatrick & Kirkpatrick, 2006). From the qualitative data, the benefits that were emphasised where related to experiential learning and increasing confidence as well as awareness. Kolb’s framework of experiential learning includes the phases of concrete experience and reflective observation (Kolb, 1984). These phases were evident in the simulation experience. Participants reflected that theoretical knowledge does not guarantee perfect execution in real life. The cognitive load of managing a high consequence, low incidence event along with the concomitant risk of exposure to a highly infectious agent makes clinical decision making harder. Although the participants underwent prior PPE training, they appreciated the opportunity to put it all into practice.

There was a definite signal for increased self-efficacy and confidence. This is seen in the literature on simulation-based healthcare education across disciplines (Bragard et al., 2018; Cohen et al., 2013; Fisher et al., 2011; Fouilloux et al., 2019). A study on influenza pandemic preparedness showed that simulation improved staff confidence and also demonstrated that many tasks and procedures took longer to complete with the implementation of guidelines (Phin et al., 2009). This was similarly evident in our participants as they became more aware of the challenges and the additional time needed for most resuscitative actions due to infection control requirements. Going through the simulated scenario and debriefing made them feel more prepared for an actual emergency. In addition, participants highlighted that the simulation experience alleviated anxiety as it successfully removed the “first-time” feeling for them, and felt that it was the responsible thing to do for healthcare workers in the midst a public health crisis. Lessons learnt by the participants included anticipatory planning, prioritisation and resource management. We had a small signal that the skills learnt translated to real life as one participant had a clinical encounter that required initiating extracorporeal life support in a patient with suspected COVID-19. She reported feeling relieved that she had the simulation experience before the real-life clinical encounter, and felt she was less nervous because of her prior stimulated experience. A respiratory therapist also observed that the nurses who he worked with in the PICU who had gone through the simulation seemed more confident and more aware of the necessary processes when intubating suspected COVID-19 patients.

An interesting phenomenon was the signal that there was an increase in anxiety levels after participants went through the simulation as seen in the pre- and post-simulation response. A possible explanation could be what participants brought up in the interviews about the simulation and debriefing revealing their “unconscious incompetence”. This made them more aware of the complexity and so increased their anxiety with dealing with intubation in a COVID-19 patient. Other studies in the literature generally report a reduction in anxiety after simulation (Bragard et al., 2019; Shrestha et al., 2019, 2020). This may not be a negative impact arising from the simulation experience itself as it may reflect increased awareness in an uncommon, yet stressful and complex clinical situation for our healthcare workers. Anxiety levels in frontline healthcare workers in a pandemic may also be due to other factors such as being at higher risk of exposure to COVID-19 at work and the possibility of bringing the infection home to their family (Holmes et al., 2020; Shanafelt et al., 2020).

Another benefit derived from running this simulation-based training is the process improvement suggestions made by the participants. This is a known benefit of simulation (Paige et al., 2018) and was utilised during the 2003 SARS-CoV outbreak (Abrahamson et al., 2005). The main challenge faced by participants during this scenario was the communication barrier which arose from not being able to communicate with personnel outside the room and the presence of the PPE and PAPR physically obstructing hearing and clarity of speech. Also, restriction of movement in and out of the patient’s room experienced during simulation highlighted the need to rethink resuscitation practices when dealing with COIVD patients. These systemic issues surfaced allowed us to brainstorm for practical solutions as a unit, and some have been implemented in our PICU. We have trialled the use of infant monitors as a 2-way communication device. Pre-packed resuscitation drug kits containing intravenous adrenaline and intravenous atropine as well as pre-packed intravenous cannulation disposables have been put in every isolation room so that these would be easily accessible in an emergency. As suggested by the participants, we have also extended the simulation training to include all junior doctors in the department, more nurses and all respiratory therapists in the PICU to facilitate better teamwork. We are also exploring the provision of a dedicated COVID-19 crash cart to minimise waste and prevent cross-contamination.

There are several limitations to this study. Firstly, the sample size is small and selection bias is possible due to the study design. Next, focus group interviews were conducted with a small subset of the participants who agreed to participate, and therefore our findings may not have been representative of the entire cohort. However, we are reassured by the fact that each healthcare professional group that took part in the simulation training was represented in the focus groups. As this simulation exercise was designed to be “just in time” training and we were limited by the urgency of the situation as well as the need to train as many staff in the shortest amount of time, we could only conduct a one-time simulation scenario specific to the highest risk procedure in resuscitating a suspected COVID-19 patient. We were therefore not able to assess the impact of this training on the subsequent performance of the participants either in another simulated scenario or a real life one.

V. CONCLUSION

Simulation-based training is a useful tool for infectious disease outbreak preparedness for the healthcare team. It improves confidence and awareness around managing emergencies while maintaining personal protection through deliberate practice in a safe environment. It also provides an opportunity for process improvement in a new and evolving pandemic situation. It was well-received by the participants and perhaps more sessions are needed for adequate practice. This is limited by the resource-intensive nature of in-situ simulation and the heavy clinical workload at this time of crisis. Further research will need to be done to determine if simulation-based training has a significant impact on actual clinical practice

Notes on Contributors

Nicola Ngiam conceptualised and designed the study, collected data, analysed and interpreted it, drafted the manuscript and approved the final version submitted.

Kristy X Fu did the background work and literature review, assisted in drafting the manuscript and critically revised the manuscript for important intellectual content and approved the final version submitted.

Jacqueline SM Ong designed the study, collected data, interpreted it, critically revised the manuscript for important intellectual content and approved the final version submitted.

Ethical Approval

Ethical approval has been granted by National Healthcare Group Domain Specific Review Board (Ref: NHG DSRB Ref. 2020/00234)

Data Availability

Transcripts from the interviews are confidential and the authors do not have consent to upload onto a repository. Data from questionnaires can be made available on request.

Acknowledgement

We would like to thank Dr Dimple Rajgor for helping with formatting, and submission of the manuscript for publication.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Interest

The Authors declare that there is no conflict of interest.

References

Abrahamson, S. D., Canzian, S., & Brunet, F. (2005). Using simulation for training and to change protocol during the outbreak of severe acute respiratory syndrome. Critical Care, 10(1), R3. https://doi.org/10.1186/cc3916

Booth, C. M., Matukas, L. M., Tomlinson, G. A., Rachlis, A. R., Rose, D. B., Dwosh, H. A., Walmsley, S. L., Mazzulli, T., Avendano, M.,  Derkach, P., Ephtimios, I. E., Kitai, I., Mederski, B. D., Shadowitz, S. B., Gold, W. L., Hawryluck, L. A., Rea, E., Chenkin, J. S., Cescon, D. W., Poutenan, S.M. & Detsky, A. S. (2003). Clinical features and short-term outcomes of 144 patients with SARS in the greater Toronto area. JAMA, 289(21), 2801-2809. https://doi.org/10.1001/jama.289.21.JOC30885

Bragard, I., Farhat, N., Seghaye, M.-C., Karam, O., Neuschwander, A., Shayan, Y., & Schumacher, K. (2019). Effectiveness of a high-fidelity simulation-based training program in managing cardiac arrhythmias in children: A randomized pilot study. Pediatric Emergency Care, 35(6), 412-418. https://doi.org/10.1097/PEC.0000000000000931

Bragard, I., Seghaye, M.-C., Farhat, N., Solowianiuk, M., Saliba, M., Etienne, A.-M., & Schumacher, K. (2018). Implementation of a 2-day simulation-based course to prepare medical graduates on their first year of residency. Pediatric Emergency Care, 34(12), 857-861. https://doi.org/10.1097/PEC.0000000000000930

Caputo, K., Byrick, R., Chapman, M., Orser, B., & Orser, B. (2006). Intubation of SARS patients: Infection and perspectives of healthcare workers. Canadian Journal of Anaesthesia, 53(2), 122-129. https://doi.org/10.1007/BF03021815

Cohen, E. R., Barsuk, J. H., Moazed, F., Caprio, T., Didwania, A., McGaghie, W. C., & Wayne, D. B. (2013). Making July safer: Simulation-based mastery learning during intern boot camp. Academic Medicine, 88(2), 233-239. https://doi.org/10.1097/ACM.0b013e31827bfc0a

Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296(9), 1094-1102. https://doi.org/10.1001/jama.296.9.1094

Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 79(10), S70-S81. https://doi.org/10.1097/00001888-200410001-00022

Fisher, N., Eisen, L. A., Bayya, J. V., Dulu, A., Bernstein, P. S., Merkatz, I. R., & Goffman, D. (2011). Improved performance of maternal-fetal medicine staff after maternal cardiac arrest simulation-based training. American Journal of Obstetrics and Gynecology, 205(3), e1-e5. https://doi.org/10.1016/j.ajog.2011.06.012

Fouilloux, V., Gran, C., Guervilly, C., Breaud, J., El Louali, F., & Rostini, P. (2019). Impact of education and training course for ECMO patients based on high-fidelity simulation: A pilot study dedicated to ICU nurses. Perfusion, 34(1), 29-34. https://doi.org/10.1177/0267659118789824

Grillet, G., Marjanovic, N., Diverrez, J., Tattevin, P., Tadié, J., & L’Her, E. (2015). Intensive care medical procedures are more complicated, more stressful, and less comfortable with Ebola personal protective equipment: A simulation study. The Journal of Infection, 71(6), 703-706. https://doi.org/10.1016/j.jinf.2015.09.003

Holmes, E. A., O’Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., Ballard, C., Christensen, H., Silver, R. C., & Everall, I. (2020). Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. The Lancet Psychiatry, 7(6), 547-560. https://doi.org/10.1016/S2215-0366(20)30168-1

Itoh, T., Lee-Jayaram, J., Fang, R., Hong, T., & Berg, B. (2019). Just-in-Time Training for Intraosseous Needle Placement and Defibrillator Use in a Pediatric Emergency Department. Pediatric Emergency Care, 35(10), 712-715. https://doi.org/10.1097/PEC.0000000000001516

Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publishers.

Kolb, D. (1984). The experiential learning theory of development. Experiential Learning: Experience as the Source of Learning and Development (pp. 132-160). Prentice Hall, Englewood Cliffs, NJ, 132-160.

Lau, J., Fung, K., Wong, T., Kim, J., Wong, E., Chung, S., Ho, D., Chan, L., Lui, S., & Cheng, A. (2004). SARS transmission among hospital workers in Hong Kong. Emerging Infectious Diseases, 10(2), 280-286. https://doi.org/10.3201/eid1002.030534

Marrs, R., Horsley, T. L., Hackbarth, D., & Landon, E. (2019). High consequence infectious diseases training using interprofessional simulation and TeamSTEPPS. American Journal of Infection Control, 48(6), 615-620. https://doi.org/10.1016/j.ajic.2019.10.007

McLaughlin, C., Barry, W., Barin, E., Kysh, L., Auerbach, M. A., Upperman, J. S., Burd, R. S., & Jensen, A. R. (2019). Multidisciplinary simulation-based team training for trauma resuscitation: A scoping review. Journal of Surgical Education, 76(6), 1669-1680. https://doi.org/10.1016/j.jsurg.2019.05.002

O’Keeffe, D., Bradley, D., Evans, L., Bustamante, N., Timmel, M., Akkineni, R., Mulloy, D., Goralnick, E., & Pozner, C. (2016). Ebola Emergency Preparedness: Simulation Training for Frontline Health Care Professionals. MedEdPORTAL, 12, 10433-10433. https://doi.org/10.15766/mep_2374-8265.10433

Paige, J. T., Fairbanks, R. J. T., & Gaba, D. M. (2018). Priorities related to improving healthcare safety through simulation. Simulation in Healthcare, 13(3S), S41-S50. https://doi.org/10.1097/SIH.0000000000000295

Patterson, M., Geis, G., Falcone, R., LeMaster, T., & Wears, R. (2013). In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Quality & Safety, 22(6), 468-477. https://doi.org/10.1136/bmjqs-2012-000942

Phin, N., Rylands, A., Allan, J., Edwards, C., Enstone, J., & Nguyen-Van-Tam, J. (2009). Personal protective equipment in an influenza pandemic: a UK simulation exercise. Journal of Hospital Infection, 71(1), 15-21. https://doi.org/10.1016/j.jhin.2008.09.005

Sam, J., Pierse, M., Al-Qahtani, A., & Cheng, A. (2012). Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes. Paediatrics & Child Health, 17(2), e16-20. https://doi.org/10.1093/pch/17.2.e16

Secheresse, T., Pansu, P., & Lima, L. (2020). The impact of full-scale simulation training based on Kolb’s learning cycle on medical prehospital emergency teams: A multilevel assessment study. Simulation in Healthcare, 15(5), 335-340. https://doi.org/10.1097/SIH.0000000000000461

Seet, R., Lim, E., Oh, V., Ong, B., Goh, K., Fisher, D., Ho, K., & Yeoh, K. (2009). Readiness exercise to combat avian influenza. QJM : Monthly Journal of the Association of Physicians, 102(2), 133-137. https://doi.org/10.1093/qjmed/hcn159

Shanafelt, T., Ripp, J., & Trockel, M. (2020). Understanding and addressing sources of anxiety among health care professionals during the COVID-19 pandemic. JAMA, 323(21), 2133-2134. https://doi.org/10.1001/jama.2020.5893

Shrestha, R., Badyal, D., Shrestha, A., & Shrestha, A. (2020). In-situ simulation-based module to train interns in resuscitation skills during cardiac arrest. Advances in Medical education and Practice, 11, 271-285. https://doi.org/10.2147/AMEP.S246920

Shrestha, R., Shrestha, A. P., Shrestha, S. K., Basnet, S., & Pradhan, A. (2019). Interdisciplinary in situ simulation-based medical education in the emergency department of a teaching hospital in Nepal. International Journal of Emergency Medicine, 12(1), 19. https://doi.org/10.1186/s12245-019-0235-x

Tan, K. W., Ong, H. K., & Mok, U. S. (2021). Using simulation and inter-professional education to teach infection prevention during resuscitation. The Asia Pacific Scholar, 6(1), 93-108. https://doi.org/10.29060/TAPS.2021-6-1/OA2229

The Centre for Evidence-Based Medicine Nuffield Department of Primary Care Health Sciences University of Oxford. (2020). Global Covid-19 case fatality rates. https://www.cebm.net/covid-19/global-covid-19-case-fatality-rates/. Accessed April 26, 2020.

The Novel Coronavirus Pnuemonia Emergency Response Epidemiology Team. (2020). The epidemiological characteristics of an outbreak of 2019 novel coronavirus diseases (COVID-19)—China, 2020. China CDC Weekly, 2(8), 113-122. https://doi.org/10.46234/ccdcw2020.032

Watson, C., Duval-Arnould, J., McCrory, M., Froz, S., Connors, C., Perl, T., & Hunt, E. (2011). Simulated pediatric resuscitation use for personal protective equipment adherence measurement and training during the 2009 influenza (H1N1) pandemic. The Joint Commission Journal on Quality and Patient Safety, 37(11), 515-523. https://doi.org/10.1016/S1553-7250(11)37066-3

Wax, R. S., & Christian, M. D. (2020). Practical recommendations for critical care and anesthesiology teams caring for novel coronavirus (2019-nCoV) patients. Canadian Journal of Anesthesia/Journal canadien d’anesthésie, 67(5), 568–576. https://doi.org/10.1007/s12630-020-01591-x

World Health Organization. (2020). Statement on the second meeting of the International Health Regulations (2005) Emergency Committee regarding the outbreak of novel coronavirus (2019- nCoV). Retrieved 6 April, 2021, from https://www.who.int/news-room/detail/30-01-2020-statement-on-the-second-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-outbreak-of-novel-coronavirus-(2019-ncov)

Wu, Z., & McGoogan, J. (2020). Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: Summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA, 323(13), 1239-1242. https://doi.org/10.1001/jama.2020.2648

Ziv, A., Small, S. D., & Wolpe, P. R. (2000). Patient safety and simulation-based medical education. Medical Teacher, 22(5), 489-495. https://doi.org/10.1080/01421590050110777

Zuo, M., Huang, Y., Ma, W., Xue, Z., Zhang, J., Gong, Y., & Che, L. (2020). Expert recommendations for tracheal intubation in critically ill patients with Noval Coronavirus Disease 2019. Chinese Medical Sciences Journal, 35(2), 105–109. https://doi.org/10.24920/003724

*Nicola Ngiam
Centre for Healthcare Simulation,
National University of Singapore,
14 Medical Dr,
Singapore 117599
Email: nicola_ngiam@nuhs.edu.sg

Submitted: 5 July 2021
Accepted: 13 September 2021
Published online: 5 April, TAPS 2022, 7(2), 6-16
https://doi.org/10.29060/TAPS.2022-7-2/OA2654

Ming-Jung Ho^1,2, Yu-Che Chang^3,4,5 & Steven L. Kanter⁶

¹Center for Innovations and Leadership in Education, Georgetown University Medical Center (CENTILE), Washington, D.C., United States; ²Department of Family Medicine, Georgetown University School of Medicine, Washington D.C., United States; ³Chang Gung Medical Education Research Centre (CG-MERC), Chang Gung Medical Foundation, Taoyuan, Taiwan; ⁴Chang Gung University College of Medicine, Taoyuan, Taiwan; ⁵Department of Emergency Medicine, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan; ⁶Association of Academic Health Centers, Washington, D.C., United States.

Abstract

Introduction: The COVID-19 pandemic forced medical schools worldwide to transition online. While there are ample reports about medical education adaptations to this crisis, there are limited studies evaluating the impact.

Methods: This study includes a case study of how Taiwanese medical school deans maintained in-person education during the COVID-19 pandemic. Additionally, it demonstrates how SWOT analyses can help medical educators reflect on adaptations during the COVID-19 pandemic and future crises.  This study employed two online surveys and a semi-structured interview regarding curricular adaptations. Eligible participants were deans or associate deans of all medical schools in Taiwan.

Results: Through a SWOT analysis, this study identified Strengths as strong leadership, prior experience with SARS, and ability to promptly adapt curriculum; Weaknesses as lack of faculty confidence in online education, limited numbers of administrative staff, and inability to rapidly add new topics; Opportunities as centralised anti-epidemic policies, inter-institutional collaborations, and educational innovations; and Threats as concerns from parents, patients, and teaching hospitals. It is reported that the quality of education was maintained as students’ evaluations of courses, performances in written exams, clerkship supervisors’ assessments, national OSCE, and national board exams remained comparable to pre-COVID times. Strengths and Opportunities such as strong leadership and centralised anti-epidemic policies can overcome Weaknesses and Threats, such as lack of confidence in online education and familial pressure to suspend hospital-based education.

Conclusion: This study recommends that medical school leaders conduct SWOT analyses as early as possible to plan strategies to continue safe and quality medical education during COVID-19 and future crises.

Keywords:           SWOT Analysis, Adaptation, Medical School, COVID-19, Pandemic, National Survey, Medical Education, Taiwan

Practice Highlights

• Medical school deans in Taiwan documented critical observations made during a global pandemic.
• SWOT analysis can help us reflect on medical education adaptations during the pandemic and crises.
• Strong leadership and centralised anti-epidemic policies can aid in overcoming weaknesses/threats.
• Schools can benefit from using SWOT analyses to plan for continuing education during emergent crises.

Table 1. Characteristics of Survey Schools

^aOne medical school had both 6-year program for high school graduates and 4-year program for college graduates.

Table 2. Themes and quotes of curriculum adaptations

Table 3. SWOT Analysis^b

^b This table is created using the blank SWOT form from Topor et al.²¹

A. Curricular Adaptations

1) Pre-clinical curriculum adaptations:

Taiwanese medical schools rapidly adapted their curricula to comply with government policy on social distancing prohibiting indoor gatherings of more than 100 individuals. Whereas a school with a small student body and sufficient classroom capacity did not have to transition online, a school with a large student body and insufficient classroom capacity moved almost all pre-clinical classes online. Some schools went partially online, converting only lectures with more than 100 students to online learning and capping the number of students in the classroom, with live stream available for those not in the classroom. While schools adapted different modalities for lecture-based courses, all schools made efforts to keep laboratory courses face-to-face. Some schools shared how they adjusted laboratory sessions to maintain social distance, including simultaneously using more lab rooms or holding lab sessions in different time slots.

2) Clinical curriculum adaptations:

In contrast to approaches in the United States suspending clinical rotations in spring semester recommended by AAMC (Association of American Medical Colleges, 2021), all medical schools in Taiwan continued clinical rotations while protecting the safety of students. Several strategies were implemented rapidly: Orientation to clinical rotations articulated pandemic prevention and included online modules required by the hospital. Clinical students were provided with surgical masks. In the emergency room, students were given N95 respirators each day. Exposure to suspected and confirmed COVID-19 cases was avoided for students replacing rotations in specialties with high risk of COVID-19 exposure. To prevent cross infection, inter-institutional and overseas exchange programs were cancelled. Students rotated in only one of the affiliated hospitals. Social distancing is also mentioned in clinical rotation adjustments. Several respondents mentioned that the number of students in outpatient clinics, in classrooms, and in conferences was reduced. One school reported an increase of clinical teachers to meet social distancing demands.

3) Assessment adaptations:

Eight schools replied that they did not change the assessment of students by written examinations in classrooms but arranged empty seats between students to keep social distance and protected students from airborne droplet transmissions by requiring everyone to wear a mask. Some schools decreased the use of written exams in classrooms by substituting reports, reflections, online exams, online presentations, and other approaches. Some evaluations involving hands-on practices were cancelled or replaced by simulation. One school reported that their admission exams included tests in laboratories. To prevent face-to-face transmission of airborne droplets, transparent acrylic partitions were installed on the benches in front of each student.

B. Impact of curricular adaptations

Medical school deans tried to evaluate the impact of curricular adaptions by comparing student performance in the 2020 spring semester with previous years. Most deans reviewed the passing rate of courses and reported no statistical differences. They also reported monitoring students’ performances in national board exams in July and national OSCE clinical skills exam held before graduation. The deans were relieved to see that the national exam scores were comparable to previous years and felt that curricular adaptations didn’t impact student competencies negatively.

C. Strengths

The strengths of Taiwanese medical schools’ responses to COVID-19 can be summarised into three themes: strong leadership, prior experience with SARS, and ability to rapidly adapt the curriculum.

1) Strong leadership: The key strength most Taiwanese deans cited in their medical schools’ response to COVID-19 is the strong leadership in the early deployment of epidemic prevention measures. The guidelines from the Central Epidemic Command Center were implemented on each campus, including requiring students to wear masks, to maintain social distance, to monitor temperature and symptoms of COVID-19, and to quarantine international students in separate dormitory areas. Communication also was highlighted by respondents as manifesting strong leadership. Furthermore, two deans made videos in collaboration with students to promote anti-epidemic measures (Chang Gung University, 2020; Fu Jen Catholic University, 2020).

2) Prior experience with SARS: Another strength most respondents noted was the experience of SARS. When COVID-19 spread to Taiwan, most clinical teachers had experienced SARS and were able to share critical lessons to face new emerging infectious diseases with the younger generation. The experience of SARS also prepared medical educators to deal with professionalism and ethics issues such as distribution of limited medical resources. Many participants mentioned that SARS had a great impact on the medical education system in Taiwan because it exposed the lack of primary care workforce with medical school graduates entering specialty training directly after medical school. After SARS, a postgraduate year program was established with one year of required general medicine training prior to specialty training programs with the goal of producing more physicians competent in general medicine to respond to emergent infectious diseases.

3) Ability to rapidly adapt the curriculum: All schools reported rapid curricular adaptations when spring semester started one or two weeks later than the scheduled time. Different strategies were employed in pre-clinical and clinical curricula to prevent and control the spread of COVID-19 as described in 3.1 curricular adaptation.

D. Weaknesses

The weaknesses of Taiwanese medical schools’ responses to COVID-19 can be summarised into three themes: lack of faculty confidence in online education, limited number of administrative staff, and inability to rapidly add new topics.

1) Lack of faculty confidence in online education: All respondents mentioned online teaching as challenging to teachers. Many voiced concerns about the effectiveness of online learning and found assessment of online learning challenging. Online learning is also considered challenging to students. Some respondents mentioned that online learning requires self-directed learning, which is not a strength of Taiwanese students since didactic teaching and rote memorisation is the norm of the K-12 education system.

2) Limited number of administrative staff: In response to the question about weaknesses of their medical schools’ responses to the COVID-19 pandemic, administrative challenges were mentioned frequently. Although Taiwan did not experience high burdens of COVID-19, all schools were preparing for large-scale outbreaks with frequent scenario planning, describing these planning activities using the military metaphor of “military sand table planning” and “early deployment.”

3) Inability to rapidly add new topics: The pandemic also revealed the need for medical educators to engage in some topics traditionally not articulated in medical curricula. It was challenging to add new topics in the ongoing 2020 spring semester curricula but respondents suggested to add the following topics in the future: infectious disease control, emergent infectious disease, crisis management and communication, public policy and ethical issues related to pandemics such as distribution of medical resources, and government control and restrictions on human rights vs quarantine policy.

E. Opportunities

Taiwanese medical school deans highlighted three opportunities which enabled them to continue to open medical schools during the COVID-19 pandemic: centralised anti-epidemic policies, collaborations among medical schools, and educational innovations.

1) Centralised anti-epidemic policies: Taiwanese deans all reported that the policies from the central government are helpful in their implementation of anti-epidemic measures. Several deans shared the policy guidelines issued by the Ministry of Education with clear guidelines to follow the principles of infection containment. Although each university has the flexibility to decide how to comply with the guidelines, there was external oversight to ensure compliance. For example, when a teaching hospital had an intra-hospital transmission of COVID-19, the medical school had to work with the Central Epidemic Command Center and Ministry of Education in its response.

2) Inter-institutional collaborations: Collaborations among medical schools through the Taiwan Association of Medical Education (TAME) also helped Taiwanese medical school deans to adapt to COVID-19. For instance, the deans worked together at TAME to ensure that final-year students graduated on time. Every year, TAME organises a national OSCE examination required for graduation for final year Taiwanese medical students. Medical schools and teaching hospitals collaborated to administer these exams at OSCE centers in selected teaching hospitals with infection mitigating measures.

3) Educational innovations: All respondents mentioned that the COVID-19 pandemic provided opportunities to engage in innovations in medical education, especially online education.  Online education was quickly implemented in some schools because access to internet is prevalent in Taiwan and they utilised the technology developed before the pandemic. One school reported an innovation using the Interactive Response System (IRS) developed three years ago and is now applied to enhance online teaching amid COVID-19. Another school adapted communication software instead of face-to-face counseling meetings. One school reported using “LINE” (a messaging app widely used in Taiwan) for a group chat for students to reduce student anxiety related to COVID-19. 

F. Threats

Three themes were identified during the COVID-19 pandemic as threats to medical education, but external to medical schools: parental pressure to remove students from clinical settings, decrease in patient volume, and hospital infection control measures limited educational opportunities.

1) Parental pressure to remove students from clinical settings:  Several deans mentioned that students’ parents reached out to them, pressuring them to discontinue clinical rotations. One respondent stated, “Parents are more worried than teachers and students. They are concerned that clinical rotations in the hospitals are risky for students. However, parents accepted our explanation that clinical rotations will not proceed unless there is adequate PPE and infection control measures.”

2) Decrease in patient volume: Another threat commonly mentioned was the decrease in patient volume. A participant explained, “Our hospital experienced a 30% decrease in patient volume. It is inconvenient for patients to come to the hospital due to tightened entrance control. The restriction on patients’ family visits is another factor. We are not letting students rotate on wards with suspected COVID-19 patients.” The limited patient encounters are perceived as potential threat to students’ clinical competence.

3) Hospital infection control measures limited educational opportunities: Finally, another threat reported was associated with infection control measures by the teaching hospitals. Deans expressed concerns that students’ clinical learning opportunities were compromised during the pandemic due to cancelled bedside teaching in high-risk specialties, cancelled presentations in conferences, cancelled interdepartmental grand rounds, and suspension of exchange programs both domestically and internationally. Some mentioned that reduced clinical exposures might result in less role modeling.

IV. DISCUSSION

During this historical event, medical schools around the world are challenged to navigate a path forward to meet missions of education, research, and patient care during an ongoing pandemic. As many countries are loosening or tightening restrictions after their curves of infection have flattened or resurged, medical schools globally are at various stages of deliberating when and how to resume campus-based and hospital-based medical education (Blaaza et al., 2020; Cleland et al., 2020; Tolsgaard et al., 2020). A systematic rapid review of published reports of medical educational developments in response to the COVID-19 pandemic concluded that most papers lacked evaluation data and focused on transitioning online and removing students from the clinical contexts (Gordon et al., 2020). The review suggested that medical schools learn from a few postgraduate papers reporting how face-to-face patient contact could be maintained while mitigating risk using PPE and social distancing (G. Choi et al., 2020; Hanel et al., 2020; Kang et al., 2020). Our study of medical school deans in Taiwan on how they continued in-person medical education throughout the COVID-19 pandemic offers lessons for the medical education community to engage medical students in campus-based and workplace-based learning with evaluation of impacts.

This study offers a conceptual advance by demonstrating how SWOT analysis can help us reflect on medical education adaptations to COVID-19 pandemic and future crises. Our analysis of the strengths, weaknesses, opportunities and threats in Taiwanese medical school adaptations to COVID-19 highlights certain strategies to leverage strengths and opportunities to overcome weaknesses and threats. A critical factor in the decision to continue face-to-face medical education, clearly identified by the opportunity-strength strategy in Table 3, was the early deployment of epidemic prevention measures under centralised government policies and strong medical school leadership. Epidemic prevention measures were deployed at all levels of schools in Taiwan as students returned to campus for spring semester in February 2020. To enact anti-epidemic policies in different contexts, this study presented various preventive measures to keep the recommended social distance in classrooms, laboratories, and teaching hospitals. To implement these measures, transparent communication and administrative support from medical school leadership are essential.

SWOT analysis on opportunity-weakness and threat-weakness strategies offers lessons on how to overcome the weakness of lack of faculty confidence in online education. Although online education is an important curricular adaptation to maintain social distance in campus-based medical education, our survey shows that respondents found online teaching challenging to faculty and are concerned about the effectiveness of online education. Strategies identified to overcome this weakness is to take advantage of the educational technology readily available and to transform the threats from hospital infection control to suspend hospital-based education into opportunities for integrating innovative online education. It is important to leverage innovative online education in workplace learning in the context of reduced patient case exposures before the pandemic subsides (B. Choi et al., 2020; Lo et al., 2020). There is sufficient evidence in the education literature that well-designed online education is as effective as traditional classroom teaching and shows equivalent or better learning outcomes (George et al., 2014; Jung & Rha, 2000; Khalil et al., 2020). We hope that the evaluations reported in the fall semester survey showing equivalent outcomes of online adaptations during COVID-19 would encourage medical educators to take advantage of innovations reported by this study, existing literature on online education in general, and the growing literature about innovative online medical education (Ahmed et al., 2020; Medical Education, 2020).

This study highlighted that another enabling factor for Taiwanese medical educators to continue face-to-face medical education was the experience of SARS. The Taiwanese population became accustomed to anti-epidemic measures, including wearing masks after SARS, which facilitated the implementation of anti-epidemic measures in schools during COVID-19. Taiwanese health providers were severely challenged during the 2003 SARS epidemic. 103 of 318 confirmed cases  (Centers for Disease Control and Prevention, 2003) and 11 of 60 deaths (including two first-year residents) (Taiwan Centers for Disease Control, 2013) were healthcare workers. Medical education leaders had a difficult decision to make regarding the final year students who were working as interns in hospitals (Lai, 2020). Following closures of clinics and resignation of healthcare providers, there were heated debates about physician duties to treat patients while being at risk of exposure to a deadly infectious disease and of being stigmatised by society (Lee, 2003). After SARS, not only were curricula on ethics and infection control strengthened, but the medical education system was reformed to convert the final-year of medical school to a formal post-graduate year program (Ho et al., 2017). In addition, as articulated by the respondents of the survey, SARS prepared Taiwanese medical educators and clinical teachers to be confident and competent to perform their educational and clinical duties in the time of COVID-19.

Although this study has many lessons to offer both conceptually and practically, there are some limitations to be considered. We conducted the survey with open-ended questions to explore diverse perspectives. The reported theme frequencies should be considered with caution since some respondents record more comprehensive answers than others. While the most frequently mentioned themes might be most important to the respondents, the least frequently given answer might be valuable due to its innovative nature. Another limitation of an open-ended questionnaire is that respondents might not have the time to enter answers in detail. To overcome this limitation, we invited the respondents to participate in a follow-up interview. However, only about one-third of deans were available to be interviewed during the study period. The four interviews conducted nonetheless enriched the study with more context and details. Finally, this study only surveyed medical school deans. It would be interesting to conduct further studies comparing these results to the perspectives of students and faculty members.

V. CONCLUSION

This nationwide study of all medical schools in Taiwan documents critical observations of top medical school leaders during an active pandemic. This case study demonstrated that strengths and opportunities such as strong leadership and centralised anti-epidemic policies can overcome weaknesses and threats such as faculty lack of confidence in online education and parental pressure to stop hospital-based education. We recommend that medical school leaders conduct SWOT analyses as early as possible to plan strategies to continue safe and quality medical education during COVID-19 and forthcoming emergencies. Similar to the impact of SARS, which led to reform of the Taiwanese postgraduate training system, there is hope that thoughtful reflection on medical education adaptations to the COVID-19 pandemic will improve medical education in general and lead to better preparation for future crises.

Notes on Contributors

MH contributed to the conception, acquisition and analysis of data, drafted the article, and approved the final version; YC contributed to the design, acquisition and analysis of data, drafted the article, and approved the final version; SK contributed to analysis and interpretation of data, revised the article critically, and approved the final version.

Ethical Approval

This study was approved as exempt by the Georgetown University Institutional Review Board (STUDY00002812). Since the study poses no greater than minimal risk to subjects, a waiver of written consent was granted. . We emailed invitations with information about study purpose, data confidentiality, and a weblink to the online survey. A verbal consent was obtained before data collection began. Participation was voluntary without incentives offered. All methods were carried out in accordance with relevant guidelines and regulations.

Data Availability

It is not possible to share data since we did not get consent from participants to share all data collected. They agreed to have results reported in a summary format with selected quotes.

Acknowledgement

The authors wish to thank survey respondents and interviewees for providing insights on the research topic. Fred Hafferty and Carrie Chen provided useful feedback to improve the manuscript.

Funding

This study did not receive any funding.

Declaration of Interest

The authors declare that they have no competing interests.

References

Abbas, M., Dhane, M., Beniey, M., Meloche-Dumas, L., Eissa, M., Guerard-Poirier, N., El-Raheb, M., Lebel-Guay, F., Dubrowski, A., & Patocskai, E. (2020). Repercussions of the COVID-19 pandemic on the well-being and training of medical clerks: A pan-Canadian survey. BMC Medical Education, 20(1), 385. https://doi.org/10.1186/s12909-020-02293-0

Ahmed, S. A., Hegazy, N. N., Abdel Malak, H. W., Cliff Kayser, W., 3rd, Elrafie, N. M., Hassanien, M., Al-Hayani, A. A., El Saadany, S. A., Ai-Youbi, A. O., & Shehata, M. H. (2020). Model for utilizing distance learning post COVID-19 using (PACT)^TM a cross sectional qualitative study. BMC Medical Education, 20(1), 400. https://doi.org/10.1186/s12909-020-02311-1

Al-Balas, M., Al-Balas, H. I., Jaber, H. M., Obeidat, K., Al-Balas, H., Aborajooh, E. A., Al-Taher, R., & Al-Balas, B. (2020). Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: Current situation, challenges, and perspectives. BMC Medical Education, 20(1), 341. https://doi.org/10.1186/s12909-020-02257-4

Association for Medical Education in Europe. (2020). COVID-19. Retrieved April 1, 2021, from https://amee.org/covid-19

Association of American Medical Colleges. (2021). Coronavirus (COVID-19) resource hub. Retrieved April 1, 2021, from https://www.aamc.org/coronavirus-covid-19-resource-hub

Blaaza, M., Akhtar, H. N., & Tanzim, U. (2020). Returning to clinics amidst COVID-19: A student response to Compton et al. Medical Education, 54(12), 1193. https://doi.org/10.1111/medu.14282

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.

Centers for Disease Control and Prevention. (2003). Severe acute respiratory syndrome–Taiwan, 2003. Morbidity and mortality weekly report, 52(20), 461‐466.

Chang Gung University. (2020, March 16). Chang Gung University COVID-19 anti-epidemic advanced adaptation measures [Video]. YouTube. https://www.youtube.com/watch?v=WyoOoShXVa8

Choi, B., Jegatheeswaran, L., Minocha, A., Alhilani, M., Nakhoul, M., & Mutengesa, E. (2020). The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey. BMC medical education, 20(1), 206. https://doi.org/10.1186/s12909-020-02117-1

Choi, G., Wan, W., Chan, A., Tong, S., Poon, S., & Joynt, G. (2020). Preparedness for COVID-19: In situ simulation to enhance infection control systems in the intensive care unit. British Journal of Anaesthesia, 125(2), e236-e239. https://doi.org/10.1016/j.bja.2020.04.001

Cleland, J., McKimm, J., Fuller, R., Taylor, D., Janczukowicz, J., & Gibbs, T. (2020). Adapting to the impact of COVID-19: Sharing stories, sharing practice. Medical Teacher, 42(7), 772-775. https://doi.org/10.1080/0142159X.2020.1757635

Compton, S., Sarraf-Yazdi, S., Rustandy, F., & Radha Krishna, L. K. (2020). Medical students’ preference for returning to the clinical setting during the COVID-19 pandemic. Medical Education, 54(10), 943-950. https://doi.org/10.1111/medu.14268

Frieden, J. (2020, May 9). Medical schools grapple with challenges for the coming year— Three-quarters hope to have students back by the end of July. MedPage Today. Retrieved April 1 fromhttps://www.medpagetoday.com/publichealthpolicy/medicaleducation/86416

Fu Jen Catholic University. (2020, February 16). COVID-19: Fu Jen Catholic University anti-epidemic promotion video. [Video]. YouTube. https://youtu.be/-n3KvlVHpzY

Gardner, L. (2020, January 31). Update January 31: Modeling the spreading risk of 2019-nCoV. Johns Hopkins University Center for Systems Science and Engineering. Retrieved April 1 from https://systems.jhu.edu/research/public-health/ncov-model-2/

George, P. P., Papachristou, N., Belisario, J. M., Wang, W., Wark, P. A., Cotic, Z., Rasmussen, K., Sluiter, R., Riboli-Sasco, E., Tudor Car, L., Musulanov, E. M., Molina, J. A., Heng, B. H., Zhang, Y., Wheeler, E. L., Al Shorbaji, N., Majeed, A., & Car, J. (2014). Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. Journal of Global Health, 4(1), 010406. https://doi.org/10.7189/jogh.04.010406

Ghazinoorya, S., Abdia, M., & Azadegan-Mehr, M. (2011). SWOT methodology: A state-of-the-art review for the past, a framework for the future. Journal of Business Economics and Management, 12(1), 24-48. https://doi.org/10.3846/16111699.2011.555358

Gordon, L., & Cleland, J. A. (2021). Change is never easy: How management theories can help operationalise change in medical education. Medical Education, 55(1), 55-64. https://doi.org/10.1111/medu.14297

Gordon, M., Patricio, M., Horne, L., Muston, A., Alston, S. R., Pammi, M., Thammasitboon, S., Park, S., Pawlikowska, T., Rees, E. L., Doyle, A. J., & Daniel, M. (2020). Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher, 42(11), 1202-1215. https://doi.org/10.1080/0142159X.2020.1807484

Hanel, E., Bilic, M., Hassall, K., Hastings, M., Jazuli, F., Ha, M., Trotter, B., Fraser, C., & Rutledge, G. (2020). Virtual application of in situ simulation during a pandemic. Canadian Journal of Eemergency Medicine, 22(5), 563-566. https://doi.org/10.1017/cem.2020.375

Harries, A. J., Lee, C., Jones, L., Rodriguez, R. M., Davis, J. A., Boysen-Osborn, M., Kashima, K. J., Krane, N. K., Rae, G., Kman, N., Langsfeld, J. M., & Juarez, M. (2021). Effects of the COVID-19 pandemic on medical students: A multicenter quantitative study. BMC Medical Education, 21(1), 14. https://doi.org/10.1186/s12909-020-02462-1

Ho, M.-J., Shaw, K., Shih, J., & Chiu, Y.-T. (2017). Mission and modernity: The history and development of medical education in Taiwan. In L. C. Chen, M. R. Reich, & J. Ryan (Eds.), Medical Education in East Asia: Past and Future (pp. 84-111). Indiana University press.

International Association of Medical Science Educators. (2020, April 22). #IAMSECafe Archives & COVID-19 resources for medical science educators. Retrieved April 1 from http://www.iamse.org/covid-19-resources-for-medical-science-educators/

Journal of the Association of American Medical Colleges. (2021). COVID-19 and medical education. Retrieved April 1 from https://journals.lww.com/academicmedicine/pages/collectiondetails.aspx?TopicalCollectionId=68

Jung, I., & Rha, I. (2000). Effectiveness and cost-effectiveness of online education: A review of the literature. Educational Technology, 40(4), 57-60.

Kang, Y., Deng, L., Zhang, D., Wang, Y., Wang, G., Mei, L., Zhou, G., & Shu, H. (2020). A practice of anesthesia scenario design for emergency cesarean section in patients with COVID-19 infection based on the role of standard patient. BioScience Trends, 14(3), 222-226. https://doi.org/10.5582/bst.2020.03066

Keegan, D. A., & Bannister, S. L. (2021). More than moving online: Implications of the COVID-19 pandemic on curriculum development. Medical Education, 55(1), 101-103. https://doi.org/10.1111/medu.14389

Khalil, R., Mansour, A. E., Fadda, W. A., Almisnid, K., Aldamegh, M., Al-Nafeesah, A., Alkhalifah, A., & Al-Wutayd, O. (2020). The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: A qualitative study exploring medical students’ perspectives. BMC Medical Education, 20(1), 285. https://doi.org/10.1186/s12909-020-02208-z

Lai, C. (2020, March 20). 2003 SARS: Reflections on Taiwanese medical education. UDN. https://health.udn.com/health/story/6631/4421448

Lazarus, G., Findyartini, A., Putera, A. M., Gamalliel, N., Nugraha, D., Adli, I., Phowira, J., Azzahra, L., Ariffandi, B., & Widyahening, I. S. (2021). Willingness to volunteer and readiness to practice of undergraduate medical students during the COVID-19 pandemic: A cross-sectional survey in Indonesia. BMC Medical Education, 21(1), 138. https://doi.org/10.1186/s12909-021-02576-0

Lee, M. (2003). SARS 風暴的啟示[Learning from the SARS event] [Editorial]. Journal of Medical Education, 7(1), 1-2.

Lo, H.-Y., Lin, S.-C., Chaou, C.-H., Chang, Y.-C., Ng, C.-J., & Chen, S.-Y. (2020). What is the impact of the COVID-19 pandemic on emergency medicine residency training: An observational study. BMC Medical Education, 20(1), 348. https://doi.org/10.1186/s12909-020-02267-2

Medical Education. (2020, May 7). Medical education adaptations. Retrieved April 1, 2020, from https://onlinelibrary.wiley.com/doi/toc/10.1002/(ISSN)1365-2923.med.adaptations.vi

Reid, M. D., & Sam, A. H. (2021). Reflections on assessment in the wake of change from the COVID-19 pandemic. Medical Education, 55(1), 128-130. https://doi.org/10.1111/medu.14368

Soled, D., Goel, S., Barry, D., Erfani, P., Joseph, N., Kochis, M., Uppal, N., Velasquez, D., Vora, K., & Scott, K. W. (2020). Medical student mobilization during a crisis: Lessons from a COVID-19 medical student response team. Academic Medicine, 95(9), 1384-1387. https://doi.org/10.1097/ACM.0000000000003401

Taiwan Center for Disease Control. (2020). Prevention and control of COVID-19 in Taiwan. Retrieved April 1, 2021, from https://www.cdc.gov.tw/Category/Page/0vq8rsAob_9HCi5GQ5jH1Q

Taiwan Centers for Disease Control. (2013). Severe acute respiratory syndrome (SARS). Retrieved April 1, 2021 from https://www.cdc.gov.tw/File/Get/InG8jagjxffXBDW1UexnrA

Tolsgaard, M. G., Cleland, J., Wilkinson, T., & Ellaway, R. H. (2020). How we make choices and sacrifices in medical education during the COVID-19 pandemic. Medical Teacher, 42(7), 741-743. https://doi.org/10.1080/0142159X.2020.1767769

Topor, D. R., Dickey, C., Stonestreet, L., Wendt, J., Woolley, A., & Budson, A. (2018). Interprofessional health care education at academic medical centers: Using a SWOT analysis to develop and implement programming. MedEdPORTAL, 14, 10766. https://doi.org/10.15766/mep_2374-8265.10766

UNICEF. (2021). COVID-19 Education: From disruption to recovery. Retrieved April 1, 2021, from https://en.unesco.org/covid19/educationresponse

Wang, C. J., Ng, C. Y., & Brook, R. H. (2020). Response to COVID-19 in Taiwan: Big data analytics, new technology, and proactive testing. The Journal of the American Medical Association, 323(14), 1341-1342. https://doi.org/10.1001/jama.2020.3151

Zhu, N., Zhang, D., Wang, W., Li, X., Yang, B., Song, J., Zhao, X., Huang, B., Shi, W., Lu, R., Niu, P., Zhan, F., Ma, X., Wang, D., Xu, W., Wu, G., Gao, G. F., & Tan, W. (2020). A novel coronavirus from patients with pneumonia in China, 2019. The New England Journal of Medicine, 382(8), 727-733. https://doi.org/10.1056/NEJMoa2001017

*Yu-Che Chang
No. 5, Fuxing St.,
Guishan Dist.,
Taoyuan City 333012,
Taiwan (R.O.C.)
Tel: +886-33281200 ext 3807
Email: changyuche@gmail.com

Submitted: 21 April 2021
Accepted: 7 October 2021
Published online: 4 January, TAPS 2022, 7(1), 87-97
https://doi.org/10.29060/TAPS.2022-7-1/OA2519

Nisakorn Deesaen¹, Kongpop Sutantikorn¹, Punyanuch Phonngoenchai¹, Sakchai Chaiyamahapruk² & Patcharada Amatyakul³

¹Faculty of Medicine, Naresuan University, Thailand; ²Department of Community Medicine, Faculty of Medicine, Naresuan University, Thailand; ³Department of Obstetrics and Gynaecology, Faculty of Medicine, Naresuan University, Thailand

Abstract

Introduction: Pelvic examination of patients in the department of obstetrics and gynaecology (Ob–Gyn) is an important skill for medical students. Because it involves a physical assessment of the patients’ genitalia, patients may refuse medical students to participate in the examination, affecting the medical students’ clinical skills.

Methods: This cross–sectional study was conducted at Naresuan University Hospital to determine the factors that influence the acceptance of medical student participation in the pelvic examinations. A total of 198 out–patients from the Ob–Gyn department were included. A Likert scale questionnaire was designed which featured topics on patients’ attitudes and circumstances related to medical student involvement in gynaecological procedures.

Results: The majority of outpatients (71.7%) accepted the participation of medical students in pelvic examinations. Patients with prior experiences in physical and pelvic examination by medical students had a significant impact on the patients’ acceptance (P–value<0.001). The patients’ impressions had an influence on the decision to accept students in pelvic exam participation. Approximately 40% of patients were concerned about the breach of confidentiality. However, most patients strongly agreed that allowing medical students to perform pelvic examination would benefit their medical education.

Conclusion: Most of the participants permitted medical students to participate in pelvic examinations and preferred that the medical instructor be the one to request permission. The patients’ impressions of medical students were crucial factors that significantly influence their decision whether to allow or deny them to participate in the procedure. Disclosure of confidentiality was found to be matters of concern to most patients.

Keywords:           Pelvic Examination, Medical Students, Acceptance, Performance, Clinical Teaching

Practice Highlights

• Most of the patients consent to a medical student participating in a pelvic examination.
• Medical instructors should ask for patients’ permission.
• Confidentiality and privacy of patients are critical issues that must be addressed.
• Patients’ impressions and prior examination experiences by medical students influence patients’ decisions.

I. INTRODUCTION

Medical students should learn how to perform pelvic examinations on patients during clinical years. History taking, physical examination, and pelvic examination are taught during the fourth to sixth year of medical training in our curriculum. Pelvic examination is defined as the assessment of external genitalia, speculum examination of the vagina and cervix, bimanual palpation of the uterus, adnexa and sometimes rectovaginal examination. This procedure is used to screen for gynaecological diseases and cancers in asymptomatic women and to diagnose gynaecological diseases in symptomatic women. Pelvic examination differs from other physical examinations because it involves an inspection of the genitalia, which, according to studies, commonly causes anxiety, fear, embarrassment (10–80% of women), pain, and discomfort (11–60% of women) (Bloomfield et al., 2014). Some factors why patients feel uncomfortable during a physical examination by medical students are concerns about privacy, confidentiality, and embarrassment (Rizk et al., 2002). As a result, patients may refuse to allow medical students to participate in the pelvic examination, affecting the learning experience and clinical skills of medical students. Furthermore, factors related to age, race, religion, cross–cultural differences, marital status, and previous delivery may also influence the patients’ decision to allow medical students in conducting pelvic examination (Anfinan et al., 2014; McLean et al., 2010). Compared to other ethnic and religious groups, Muslim women had a higher rate of refusing medical students, particularly male students (Nicum & Karoo, 1998). During intrapartum care, approximately 50% of Hindu and Muslim patients refused to accept medical students. Sikh and Muslim patients accepted only female medical students, 41% and 40%, respectively, whereas 59% of Christian patients accepted both female and male medical students (Nicum & Karoo, 1998). However, patients’ reasons for accepting and refusing medical student participation in pelvic examination have not been established in Southeast Asia because of limited investigation.

According to the patient’s bill of rights, patients have a right to accept or refuse the medical students’ participation. They should have the opportunity to decide if they want to contribute to the medical education or decline care from medical students (Teunissen, 2018). Approximately 26% of the patients refused to have a medical student perform a pelvic examination (Fortier et al., 2006). Previous study reported that the refusal rate increased as the level of student involvement increased, from observation to history taking to examinations and procedure, particularly with digital vaginal and rectal examination (Salah et al., 2015). In contrast, some patients accepted the medical students’ participation because they wanted to contribute to the clinical training in medical education. Because of the training process, the patients felt more confident that they would receive appropriate treatments. From the literature review, most studies were conducted in developed and some Arab countries. There are limited data from women in Southeast Asia countries on the factors and attitudes that influence their decision whether they allow or refuse medical students to participate in a pelvic examination. This research aimed at studying the patients’ characteristics, influencing factors and correlation with the acceptance of medical students, which may increase the Ob–Gyn patients to participate in clinical education.

II. METHODS

This cross–sectional descriptive study was conducted at the out–patient unit of the Department (OPD) of Obstetrics and Gynaecology, Naresuan University Hospital. Patients who visited Ob–Gyn department between November 2018 and May 2019 were included in the study. Patients under 20 years old, mentally or critically ill and unable to understand the questionnaire in Thai language were excluded in this study. All the participants were able to complete the consent forms and questionnaires independently, and were guaranteed anonymity.

The questionnaire was developed to evaluate 4 domains, as follows: (1) demographic and socio–economic data (age, gender, educational level, occupation and parity), (2) patient preference, (3) factors influencing patient receptivity of medical student involvement in pelvic examination (gender, hygiene, manner and demeanour, patients’ impressions of the medical students, prior experience in physical and pelvic examination), and (4) patients’ attitudes toward accepting medical students to conduct pelvic examination under supervision of medical instructors. Influencing factors and attitudes included in the questionnaire were modified based on a literature review. A 5–point Likert scale was used to determine the level of agreement (1= strongly agree; 2= agree; 3= neutral; 4= disagree; and 5= strongly disagree). The questionnaire was initially tested for content validity using the item-objective congruence (IOC) index, and then used in a pilot study on 30 patients who were not included in the study to ensure clarity and reliability.

A. Statistical Analysis

A Microsoft Excel spreadsheet was created for data entry and statistical analysis. Descriptive data was presented in the form of frequency, mode, and percentage. Chi-square test was calculated for proportions. The p–value of less than 0.05 was considered statistically significant.

B. Ethical Considerations

All collected data was kept confidential, and the information was used for research only. This study was approved by Naresuan University Institutional Review Board in compliance with the principles of the Helsinki Declaration. 

III. RESULTS

During the study period, 236 participants were recruited from the out–patient department of Ob–Gyn, but only 198 completed the questionnaire. Most of them (99.0%) were aware that Naresuan University Hospital is a primary teaching hospital of the Faculty of Medicine, Naresuan University which provides clinical training for medical students and residents. The socio–demographic data of the study population are shown in Table 1. The data that supports the findings of this study are openly available at http://doi.org/10.17605/OSF.IO/HBV68 (Amatyakul, 2021).

Table 1. Demographics and characteristics of the study population

From 198 participants, 71.7% accepted the medical students to participate in pelvic examination. Fifty–seven percent of the participants in the acceptor group allowed both male and female medical students, while 42.9% in the same group allowed only female medical students. The correlation between patients’ acceptance and refusal for the medical students to participate in pelvic examination showed that age, occupation and parity of the patients were not statistically different (p> 0.05). Thirty–one percent of bachelor’s degree holders and 37% of postgraduate degree holders refused the participation of medical students. The higher the patient’s educational attainment, the more likely they are to refuse a pelvic examination performed by medical students, as observed in this study. In the conduct of pelvic exam procedure, 69.3% of the patients in the acceptor group and 53.6% of the patients in the non-acceptor group felt comfortable with students present as observers. Before the students participate in a pelvic examination, most of the participants (81.4%) preferred that the medical instructors (56.6%) ask permission rather than the medical students (21.7%) or nurses (21.7%).

This research recorded 52.7% of the patients with prior experience of physical examination by medical students from other departments in Naresuan University Hospital. Thirty–two percent of the patients previously received pelvic examination by medical students (from our department or other medical training hospitals). Patients who had their physical examinations performed by medical students from other departments or had pelvic examinations performed by medical students had a statistically significant increase in allowing medical students to participate in pelvic examinations under the supervision of a medical instructor (Table 2).

Table 2. The correlation between the acceptance of medical student participation in pelvic examination and prior experience of pelvic examination performed by medical students.

Table 3. The positive and negative attitudes of the participants about medical students’ participation in pelvic examination under the supervision of medical instructors.

IV. DISCUSSION

Our study demonstrated that 71.7% of the participants agreed to have their pelvic examinations performed by medical students. This result was comparable to the study of Western women that reported an acceptance rate ranging from 58 to 77% (Nicum & Karoo, 1998). Conversely, our acceptance rate was lower when compared to a study conducted in United Arab Emirates by Rizk et al. (2002), in which 87.1% of the out-patients in Ob-Gyn accepted the involvement of medical students. According to the results of our study, there were no statistically significant differences between acceptors and non-acceptors in terms of age, educational level, occupation, parity, or prior pelvic examination. Hartz and Beal (2000) also reported similar findings, stating that the age and education of the patients were not statistically different between the two groups.  However, Rizk et al. (2002) stated that the acceptance of the patients with older age, higher parity, and higher education was statistically significant. Interestingly, there was a trend discovered in our research that patients who are highly educated were more reluctant to allow medical students in performing pelvic examination, even when supervised by medical instructors. This reluctance could be because of a strong concern for their privacy, which should be investigated further through an in-depth interview.

Prior experience of the patients receiving physical examination by medical students from other departments, and prior experience of the patients receiving pelvic examination performed by medical students significantly increased the rate of acceptance. These findings are consistent with those of Ghobain et al. (2016) who reported that a positive prior experience with a medical student was significantly related to giving medical students permission to perform a physical examination. This can be explained by the fact that these patients were already aware of the medical student involvement in performing physical examinations. Therefore, they are more likely to accept medical student participation in subsequent Ob-Gyn consultations.

One interesting finding from our study was the positive patient receptivity of medical students acting as observers during pelvic examination. Patients in approximately 70% of the acceptor group allowed other medical students to observe the examination process. Remarkably, 53.6% in the non-acceptor group was comfortable with medical students observing a pelvic examination performed by medical instructor. This would imply that even if students cannot perform pelvic examinations, they can still gain clinical knowledge through observation, and clinical instructors can take advantage of this valuable opportunity to educate their students.

Other major concern of patients is the requirement for students to obtain permission before participating in any procedures. The majority of the participants prefer that medical instructors be in charge of obtaining patient permission to allow students to perform physical examinations on their behalf. This tendency may lead to a higher rate of acceptance of student participation.

The characteristics and performance of the medical students in our study had a significant impact on patients’ decision. The external part of the female reproductive organ is a sensitive and the most private area for every woman, and patients would only allow medical students who practice good hygiene and cleanliness to participate in the examination. Furthermore, the appropriate manner and demeanour, including respectfulness and politeness toward patients, and trustworthiness of the medical students during the clinical procedure may increase the patients’ receptivity of medical students to conduct pelvic examination. Professional appearance reflecting cultural backgrounds also impacts patient preference and acceptance; therefore, medical students should dress properly (Aljoudi et al., 2016).

Several studies, including ours, found that participants felt more at ease with female students than with male students (Salah et al., 2015; Subki et al., 2018). Chang et al. (2010) reported that male students were refused by patients to participate in clinical interviews and physical examinations, including pelvic examinations. In a study conducted at Taibah University in India (Shetty et al., 2021), women significantly preferred female students during abdominal and genital examinations. As a result, it is difficult for obstetrics and gynaecology educators to consider methods of encouraging patients to accept participation of medical students, regardless of their genders.

Patients’ positive attitudes toward medical students’ participation contributed to clinical teaching, which is an important process for professional development. The patients agreed that they would receive more accurate and comprehensive treatment if they had two pelvic examinations. These details are consistent with the findings of a study conducted by Rizk et al. (2002), which revealed that 69.7% of the participants were willing to contribute to the education of students. Most participants were satisfied with the overall service because they were well informed about their care by the health care team and were actively involved in the treatment decision-making process. Like other literature (Nicum & Karoo, 1998), medical students also provided patients with useful medical information and support. Furthermore, patients strongly supported the idea that real patient encounters and practices under clinical supervision are more effective methods for improving student’s clinical skills than just mere observation or skill laboratory practice (Subki et al., 2018).

Patients’ main reasons for refusing medical students’ participation in pelvic examination were concerns about breaching their confidentiality and privacy, which were similar to a study conducted in London. According to the findings, the common reasons for patients’ uneasiness with participation of the medical students were related to privacy, receiving lower standard of care, examinations, lack of control over the student’s level of involvement, and a longer consultation time (Ryder et al., 2005). The participants in our study were also concerned about taking a longer than usual routine visit. Not only applicable in Ob-Gyn department, patients’ perceptions of students’ professionalism and respect for privacy were significantly related to the acceptance of medical students’ participation in surgical ward (Ghobain et al., 2016). Thus, all medical students must be informed about the doctor-patient relationship and the importance of maintaining patient confidentiality. Medical instructors must inform and reassure patients about their confidentiality. Before participating in any clinical teachings, medical instructors should explain to students their roles and responsibilities. To minimise patients’ feeling of discomfort, these roles and responsibilities should be conveyed and explained to them before asking for their approval.

This paper has highlighted the significance of patients’ attitudes toward medical students’ manner and demeanor, which greatly influences patients’ decision-making process.  However, some limitations should be considered, such as the fact that all participants were Thai, and that the study was carried out in one of the Southeast Asia countries where data on the attitudes of female patients had not been thoroughly investigated. Since socio-demographic factors and cultural issues vary across Southeast Asia, the results of this research only represent the characteristics of the Thai population and not the entire region. Additionally, this cross-sectional study could not establish the reasons for patients’ negative attitudes toward student involvement in pelvic examination. Therefore, it is suggested that future research use in-depth interview methodology to gather more information from both the acceptor and non-acceptor groups.

V. CONCLUSION

Most of patients agreed to medical students participating in pelvic examinations and preferred medical instructors to be the persons to ask patients for permission. Essentially, patients’ confidentiality and privacy must always be safeguarded. The performance of medical students, and their observance of patient privacy and confidentiality are crucial factors in gaining the patient’s approval. Furthermore, the gender of the medical student influences the patient’s acceptance and comfort level in student’s involvement. Clinical instructors must effectively convince patients in gynaecology department to allow male medical students to perform gynaecologic procedures.

Notes on Contributors

Nisakorn Deesaen, Punyanuch Phonngoenchai, and Kongpop Sutantikorn contributed to the literature review, concept development, questionaire design, data collection, data analysis, and manuscript finalisation. Sakchai Chaiyamahapruk was involved in the study design, data analysis, and manuscript finalisation. Patcharada Amatyakul contributed to the literature review, concept development, study design, data analysis, and manuscript writing and finalisation.

Ethical Approval

Ethical approval for this study was obtained from the Naresuan University Institutional Review Board, Naresuan University, Thailand (Ethics approval number IRB 0653/60).

Data Availability

The data that support the findings of this study are publicly available in Open Science Framework repository, http://doi.org/10.17605/OSF.IO/HBV68.

Acknowledgement

The authors would like to express their gratitude to the nurses at the out-patient unit of the Department of Obstetrics and Gynaecology, Naresuan University Hospital, for their help in distributing and collecting questionnaires from the patients.

Funding

This research received no specific grant from any funding agency.

Declaration of Interest

The authors declare no conflict of interest.

References

Aljoudi, S. B., Alsolami, S. S., Farahat, F. M., Alsaywid, B., & Abuznadah, W. (2016). Patients’ attitudes towards the participation of medical students in clinical examination and care in Western Saudi Arabia. Journal of Family and Community Medicine, 23(3), 172‑178. https://doi.org/10.4103/2230-8229.189133

Amatyakul, P. (2021). Proposal for patients’ attitude and factors influencing the acceptance of medical students’ participation in pelvic examination. [Data set]. Open Science Framework. http://doi.org/10.17605/OSF.IO/HBV68

Anfinan, N., Alghunaim, N., Boker, A., Hussain, A., Almarstani, A., Basalamah, H., Sait, H., Arif, R., & Sait, K. (2014). Obstetric and gynecologic patients’ attitudes and perceptions toward medical students in Saudi Arabia. Oman Medical Journal, 29(2), 106-109. https://doi.org/10.5001/omj.2014.26

Bloomfield, H. E., Olson, A., Greer, N., Cantor, A., MacDonald, R., Rutks, I., & Wilt, T. J. (2014). Screening pelvic examinations in asymptomatic, average-risk adult women: An evidence report for a clinical practice guideline from the American College of Physicians. Annals of Internal Medicine, 161(1), 46–53. https://doi.org/10.7326/M13-2881

Chang, J. C., Odrobina, M. R., & McIntyre-Seltman, K. (2010). The effect of student gender on the obstetrics and gynecology clerkship experience. Journal of Women’s Health, 19(1), 87-92. https://doi.org/10.1089/jwh.2009.1357

Fortier, A. M., Hahn, P. M., Trueman, J., & Reid, R. L. (2006). The acceptance of medical students by women with gynaecology appointments. Journal of Obstetrics and Gynaecology Canada, 28(6), 526-530. https://doi.org/10.1016/S1701-2163(16)32179-X

Ghobain, M. A., Alghamdi, A., Arab, A., Alaem, N., Aldress, T., & Ruhyiem, M. (2016). Patients’ perceptions towards the participation of medical students in their care. Sultan Qaboos University Medical Journal, 16(2), 224–229. https://doi.org/10.18295/squmj.2016.16.02.014

Hartz, M. B., & Beal, J. R. (2000). Patients’ attitudes and comfort levels regarding medical students’ involvement in Obstetrics–Gynecology outpatient clinics. Academic Medicine, 75(10), 1010-1014. https://doi.org/10.1097/00001888-200010000-00018 

McLean, M., Al Ahbabi, S., Al Ameri, M., Al Mansoori, M., Al Yahyaei, F., & Bernsen, R. (2010). Muslim women and medical students in the clinical encounter. Medical Education, 44(3), 306-315. https://doi.org/10.1111/j.1365-2923.2009.03599.x.

Nicum, R., & Karoo, R. (1998). Expectations and opinions of pregnant women about medical students being involved in care at the time of delivery. Medical Education, 32(3), 320-324. https://doi.org/10.1046/j.1365-2923.1998.00205.x.

Rizk, D. E. E., Al-Shebah, A., El-Zubeir, M. A., Thomas, L. B., Hassan, M. Y., & Ezimokhai, M. (2002). Women’s perceptions of and experiences with medical student involvement in outpatient obstetric and gynecologic care in the United Arab Emirates. American Journal of Obstetrics and Gynecololgy, 187(4), 1091-1100. https://doi.org/10.1067/mob.2002.126284.

Ryder, N., Ivens, D., & Sabin, C. (2005). The attitude of patients towards medical students in a sexual health clinic. Sexually Transmitted Infection, 81(5), 437–439. https://doi.org/10.1136/sti.2004.014332.

Salah, A. B., Mhamdi, S. E., Bouanene, I., Sriha, A., & Soltani, M. (2015). Patients’ attitude towards bedside teaching in Tunisia. International Journal of Medical Education, 6, 201-207. https://doi.org/10.5116/ijme.5669.ea24.

Shetty, P. A., Magazine, R., & Chogtu, B. (2021). Patient outlook on bedside teaching in a medical school. Journal of Taibah University Medical Sciences, 16(1), 50-56. https://doi.org/10.1016/j.jtumed.2020.10.002

Subki, A. H., Algethami, M. R., Addas, F. A., Alnefaie, M. N., Hindi, M. M., & Abduljabbar, H. S. (2018). Women’s perception and attitude to medical students’ participation in obstetrics and gynecology care. Saudi Medical Journal, 39(9), 902-909.  https://doi.org/10.15537/smj.2018.9.22668

Teunissen, P. W. (2018). An inconvenient discussion. Medical education, 52(11), 1104-1110. https://doi.org/10.1111/medu.13689

*Patcharada Amatyakul
Department of Obstetrics and Gynaecology,
Faculty of Medicine, Naresuan University,
99 Thaphoe District, Muang,
Phitsanulok, 65000 Thailand
Tel: 66-86-397-3455
Email: pamatyakul@hotmail.com

Submitted: 3 June 2021
Accepted: 4 October 2021
Published online: 4 January, TAPS 2022, 7(1), 76-86
https://doi.org/10.29060/TAPS.2022-7-1/OA2547

Nadia Greviana^1,2, Dewi Anggraeni Kusumoningrum², Ardi Findyartini^1,2, Chaina Hanum¹ & Garry Soloan^1,3

¹Medical Education Center, Indonesian Medical Education & Research Institute (IMERI) Faculty of Medicine Universitas Indonesia, Jakarta, Indonesia; ²Department of Medical Education, Faculty of Medicine Universitas Indonesia, Jakarta, Indonesia; ³Undergraduate Program in Medicine, Faculty of Medicine Universitas Indonesia, Jakarta, Indonesia

Abstract

Introduction: As significant autonomy is given in a Massive Open Online Course (MOOC), online self-regulated learning (SRL) ability is crucial in such courses. We aim to measure the online SRL abilities of early-career medical doctors enrolled in a MOOC.

Methods: We performed a cross-sectional study using the Self-Regulated Online Learning Questionnaire-revised version (SOL-Qr). We conducted a three-stage cross-cultural validation of the SOL-Qr, followed by Confirmatory Factor Analysis (CFA). The online SRL ability of 5,432 medical doctors enrolled in a MOOC was measured using the validated SOL-Qr.

Results: The CFA of the cross-translated SOL-Qr confirmed its comparability to the original version, with excellent validity & reliability. Participants showed high levels of online SRL during their early careers. Despite high online SRL scores, MOOC completion rate was low. Male participants showed slightly better time management ability than female participants. Participants working in the primary epicentrum for COVID-19 in the country showed lower online SRL scores, while participants who graduated from higher accreditation levels showed better time management ability.

Conclusion: The SOL-Qr and its subscales are suitable and valid for measuring the online SRL abilities of medical doctors in a MOOC during their early-career period. Time management ability was associated with previous experience during the medical education period, while other online SRL subscales were mostly associated with workload. However, as the scores did not correlate with the time spent for learning in MOOC, the corresponding learning effort or time spent may be beyond just the commitment to the described MOOC.

Keywords:           Self-Regulated Learning, MOOC, Online Learning

Practice Highlights

• It is important to take into account learners’ online SRL ability in MOOCs as it is dynamic across online learning contexts.
• The use of the SOL-Qr is beneficial for providing learners’ online SRL profiles in MOOC during medical doctors’ early career period.
• Understanding online SRL abilities helps MOOC developers to evaluate learning activities in MOOC and support learners’ online SRL ability.

I. INTRODUCTION

Massive Open Online Courses (MOOCs) are open academic platforms in which students can access learning resources interactively. The self-paced nature of MOOCs provides time-flexibility, facilitating deeper learning (Bullock & De Jong, 2013). MOOCs are useful for replacing direct interaction for knowledge transfer and learning processes during the ongoing pandemic because they utilise various formats, such as video lectures, reading resources, assignments, tests, and asynchronous discussion within the platform (Sandars & Patel, 2020). Because MOOC’s aim to give learners useful resources and empower teachers to provide vital knowledge, curation of the platform, with its copious information, it is beneficial for achieving high-quality content that fits the learning objectives and learners’ characteristics (Asarbakhsh & Sandars, 2013). As demand for technological solutions in education rise during the COVID-19 pandemic, MOOCs have been promoted as forms of disruption that accelerate adaptation to balance safety with the achievement of competencies by medical students and graduates (Hall et al., 2020; Liang et al., 2020).

MOOCs have generally been designed as open in access, location, pace, and time of completion; therefore, learners must control their learning process. Learning goals are usually set less strictly in MOOCs than in other courses. Unlike traditional, face-to-face teaching, MOOCs require highly engaged & strategic students. Learners must plan their study, set goals, evaluate their knowledge related to the course material, adapt their learning strategies, and assess their performance. They are solely responsible for managing their time and study environment (Jansen et al., 2017).

With high enrollment rates, the majority of learners fail to complete MOOCs, for various reasons: lack of time, insufficient prior knowledge, inadequate supervision, and difficulties in understanding the course materials (Hew & Cheung, 2014). Time management, effort regulation, metacognition, and critical thinking as part of Self-Regulated Learning (SRL) relate to success in online academic activities (Broadbent & Poon, 2015). Because MOOCs give students significant autonomy in completing the course, SRL is crucial for successful completion of MOOCs (Chung, 2015; Wong et al., 2018).

SRL encompasses a student’s ability to actively and constructively control their thoughts, acts, and emotions towards learning objectives (Jouhari et al., 2015), using the cycle of forethought, performance, and self-reflection phases, which should be implemented in an online learning environment (Panadero, 2017). Various external factors may interplay and affect self-regulated learning among students. This underscores the importance of a supportive family, helpful peers, and motivational, feedback-centered instruction methods; together, these factors support SRL (Jouhari et al., 2015).

Virtanen and Nevgi (2010) recognised gender as a factor affecting how SRL is perceived by students, especially during the forethought stage; they found that male students scored slightly higher on the sub-scale for self-efficacy, while female students demonstrated greater help-seeking strategies, performance anxiety, and beliefs in the value of studying. Bembenutty (2009) also found that female students perceive learning as a more valuable task in SRL than male students do.

Several instruments were developed to measure SRL, including structured interviews like the Self-Regulation Interview Schedule (SRLIS), questionnaires, teachers’ judgments, think-aloud techniques, and performance observations (Magno, 2011). The Motivated Strategies for Learning Questionnaire (MSLQ) is a common measure of SRL. It assesses two aspects: motivational orientation—encompassing values, expectations, and affective components—and the use of different learning strategies, e.g. cognitive and metacognitive strategies or resource management strategies (Pintrich et al., 1993). Another instrument for measuring students’ SRL in higher education is the Academic Self-Regulated Learning Scale (A-SRL-S), evaluating memory strategy, goal-setting, self-evaluation, seeking assistance, environmental structuring, learning responsibility, and planning and organising (Magno, 2011).

In the context of online learning, several instruments have been developed to measure students’ SRL, such as the Online Self-regulated Learning Questionnaire (OSLQ) and the Self-regulated Online Learning Questionnaire (SOL-Q). The OSLQ consists of six subscales: environment structuring, goal-setting, time management, help-seeking, task strategies, and self-evaluation (Barnard et al., 2009). However, this questionnaire cannot measure SRL activities in the appraisal or self-reflection phase. Meanwhile, the SOL-Q was developed by Jansen et al. (2017), using other existing SRL questionnaires to measure SRL activities, specifically in MOOCs, for all three SRL phases: preparatory, performance, and appraisal. The SOL-Q consists of five sub-scales: metacognitive skills, environmental structuring, help-seeking, time management, and persistence (Jansen et al., 2017). In 2018, a revision was made to split the metacognitive skills scale into three subscales: metacognitive activities before, during, and after a learning task. The revised questionnaire (SOL-Qr) demonstrated improved validity, usability, and reliability (Jansen et al., 2018).

Recognising the importance of learners’ during the use of MOOCs—and that SRL is affected by various factors—we attempted to measure the online self-regulated learning of early-career medical doctors enrolled in a MOOC that provides essential knowledge about COVID-19 to support medical doctors’ early careers during the COVID-19 pandemic. We seek to answer the following questions in this study:

1. Is the Self-Regulated Online Learning Question (SOL-Qr) valid for use in our setting?
2. What is the profile of students’ SRL scores, and are there any relationships between the SRL score, course completion, gender, respondents’ former medical school, and internship location

 

II. METHODS

A. Context

With approximately 270 million inhabitants across 34 provinces, Indonesia is one of the largest archipelagos in the world. According to a recent report, there is one medical doctor per 2500 people across the country (National Ministry of Health, 2020). Recently graduated medical doctors must undergo a one-year compulsory internship program upon graduation, where they serve as front-liners in primary health care settings across the country to serve in societies in very diverse sociocultural contexts and ethnicities. Those who graduated in 2020, mostly finished high school and entered medical schools in 2013–2014, completed their clinical stages and graduated from different medical schools in Indonesia at the beginning of the COVID-19 pandemic, during which some of the clinical duties in teaching hospitals were suspended and amended for safety reasons (Findyartini et al., 2020). A total of 93 public and private medical schools are distributed across Indonesia, all accredited by the National Accreditation Agency for Higher Education in Health with A-level as the highest accreditation indicating that the medical school has reached an established quality in teaching-learning processes and faculty members.

Considering the need for the newly graduated medical doctors serving as front liners to learn about the current pandemic situation and the importance of safety for both patients and healthcare workers, the Ministry of Health equipped these new graduates with essential COVID-19 knowledge before their involvement in patient management during the ongoing pandemic. Given the geographical reality of the country’s archipelago and the nature of the workplace setting during the internship program, the use of a Massive Open Online Course (MOOC) was preferred.

Little was known about COVID-19 at the beginning of the pandemic. Guidelines created by organisations (such as the CDC and WHO) are mostly amended living documents based on newly published articles, data, and clinical expertise. Studies on COVID-19 are being conducted on a massive scale worldwide, which may create information overload and overwhelm learners, especially those serving as front–liners (Poonia & Rajasekaran, 2020). A MOOC approach would facilitate a prompt response for equipping medical and health students, professionals, and the broader community to learn about the pandemic situation and prepare them to contribute to the pandemic response in community and hospital settings (Ortiz-Martìnez et al., 2021).

B. The COVID-19 MOOC Description

All internship doctors in Indonesia batch 3 and 4 in 2020, were compulsorily enrolled in an open course on COVID-19 at the beginning of their internship period. The open course was sequenced into two, sequentially accessed parts. The mandatory part A consists of fundamental knowledge on COVID-19 (such as COVID-19 screening, triage, infection control, management of patients, preventive strategy, etc); Part B consists of supplemental knowledge about COVID-19 (such as management of patients with comorbidity, ethics, and medicolegal aspects, perioperative management, etc); it is optional for participants to choose which topics to learn based on their interests and needs. Participants were encouraged to complete Part A during the first two weeks of their internship program to ensure sufficient knowledge before their service. However, participants were given full access to revisit the MOOC for up to 9–12 months of their internship programs. More details on the online course are reported elsewhere (Findyartini et al., 2021).

C. Study Design and Instruments

Our cross-sectional study uses the SOL-Qr (Jansen et al., 2018), which was adapted to Bahasa Indonesia, to assess online self-regulated learning ability in a Massive Open Online Course. Secondary data were obtained from the Moodle-based MOOC platform, including the total number of respondents, gender, internship location, former medical school, and course completion.

SOL-Qr measures seven aspects of SRL as mentioned in Table 1. Respondents answer each item on a 7-point Likert scale (1 for “not at all true for me” and 7 for “very true for me”). This questionnaire was translated to Bahasa Indonesia and back-translated to English to ensure similarity in meaning. Cognitive interviews with four respondents resembling the study participants were also conducted to obtain clarity of the items. The two respondents in this phase were final year medical school students from the authors’ institution (prospective participants of the national internship program) and the other two respondents were medical doctors who have just completed the national internship program.

Table 1. Descriptions of SOL-Qr (Jansen et al., 2018)

Course completion, as analysed in this study, refers to the completion rate (in percentage) of the optional topics in part B that were accessed and completed by study participants in the open course.

D. Data Collection

Data were collected from the Moodle-based MOOC platform of the COVID-19 module from January to February 2021, two months after each course had started. This study uses a total sampling approach, with a minimum sample size of 204 respondents, calculated from standard deviation of 3.43 (Yen et al., 2016) and alpha (type I error) of 0.05 and beta (type II error) of 0.20 and 10% estimated drop out level.

All study participants were National Internship Medical Doctors in the year 2020 who were enrolled in the COVID-19 Open Course on the MOOC platform.

The SOL-Qr questionnaire was embedded in the evaluation questionnaire placed at the end of Part A, before participants moved forward to Part B. The questionnaire was completed voluntarily by participants who agreed to participate in this study. They were given adequate written information about the study and assured that there were no consequences of participation in regards to the course or the internship program evaluation. All data included in this study were collected from participants who signed and agreed upon the written consent embedded in the questionnaire. This study obtained ethical clearance from Faculty of Medicine Universitas Indonesia/ dr. Cipto Mangunkusumo General Hospital Research Ethics Committee Board (KET-1395/UN2.F1/ETIK/PPM.00.02/2020) in 2020.

E. Data Analysis

We conducted a three-stage validation process for the SOL-Qr, including the process of translation to Bahasa Indonesia by the research investigator, the back-translation process, and a review process by four newly graduated medical doctors who represented the characteristics of the study participants. This process ensured that the Bahasa Indonesia version of SOL-Qr was comparable to the original version. Furthermore, a Confirmatory Factor Analysis (CFA) was performed to confirm the model proposed by Jansen et al. (2018) as a fit model in the Bahasa Indonesia version compared to the original version. The fit model of CFA analysis determined whether normality, multicollinearity, residual values, and multivariate outliers were met. Furthermore, the Two-Index Presentation Strategy, the fit index combination of at least two indicators among the root mean square error of approximation (RMSEA), the standardised root mean square residual (SRMR), and the comparative fit index (CFI) all indicated the fit model of CFA analysis (Hu & Bentler, 1999; Schreiber et al., 2006). Internal consistency analysis of the Bahasa Indonesia version of SOL-Qr was also completed. Items were concluded as valid if the corrected item-total item correlation value was higher than 0.3. The questionnaire was considered reliable if alpha Cronbach ≥ 0.700; an alpha value higher than 0.900 indicates excellent internal consistency (Blunch, 2008).

All survey data obtained from the questionnaire were statistically analysed using IBM SPSS Statistic version 21. Demographic data were processed using descriptive statistics (proportion, mean, and standard deviation). Study participants were classified into two groups according to their internship location:

• Medical doctors who serve in Java- Bali Region, which was the primary epicenter of COVID-19 in the country in 2020.
• Medical doctors who serve in Outside Java- Bali Region.

We also classified participants according to their former medical school accreditation.

Bivariate analysis using the t-independent test was used to find relationships between the online self-regulated learning scores and gender, internship location, the former medical school, and the course completion. The Pearson correlation test was also used to analyse correlations between online SRL subscales and the course completion rate.

III. RESULTS

A. Validation of the SOL-Qr

The three stages of validation were conducted in the SOL-Qr instrument to ensure the content validity of the Bahasa Indonesia version of SOL-Qr. CFA was performed on the Bahasa Indonesia version of SOL-Qr, with the results showing the goodness-of-fit according to Hu and Bentler’s Two Index Presentation; the SRMR value was 0.056 (<0.08) while RMSEA value was 0.078 (<0.08). Meanwhile, the χ2/df value was < 0.001; the CFI value was 0.874 (Hu & Bentler, 1999). The model (Figure 1) also confirms the comparability of the subscales to the original SOL-Qr.

The Bahasa Indonesia version of SOL-Qr also shows excellent validity and reliability, with a Cronbach’s alpha of 0.974. The reliability of each subscale ranges from 0.971 to 0.975.

Figure 1. CFA results of the National Language Version of SOL-Qr (MB: Metacognitive Before, MD: Metacognitive During, MA: Metacognitive After, TM: Time Management, ES: Environmental Structuring, P: Persistence, and HS: Help Seeking)

B. Profile of Participants’ Online SRL Scores

A total of 5,846 internship doctors from all 34 provinces in Indonesia were enrolled and accessed the MOOC; as many as 5,432 participants, graduated from 74 medical schools (of which 46% are A-accredited while 54% are B-accredited), agreed to participate in the study (response rate of 92.9%). Details on the study participants are presented in Table 2.

Table 2. Study participants (N = 5,432)

C. Relationship Between Online SRL Score, Course Completion, Gender, and Internship Location

Data on the online SRL scores and course completion were analysed according to gender, internship location, and medical school accreditation. The data are described in means and standard deviations, as they were normally distributed, as shown in Table 3.

According to Table 3, the average total scores of participants’ online self-regulated learning in all groups show high levels of online SRL (SRL score > 5). When comparing the male and female participants, the finding suggests that only the Time Management subscale shows a significant difference (p = 0.001). Male participants show higher scores in time management than female participants. Participants from the Outside Java-Bali Region had significantly higher online SRL scores in all subscales, except for the Persistence subscale (p = 0.181), than participants from the Java-Bali Region, which was the primary epicenter of COVID-19 in the country.

Meanwhile, the online SRL scores of participants who graduated from A-level versus B-level accredited medical schools mostly show similar online self-regulated learning scores, except in the time management scale (p = 0.009).

Although Table 3 shows no significant difference regarding course completion across gender, internship region, or former internship location, it does show a low course completion rate. Participants only completed less than approximately 40% of all optional topics in Part B.

The results of the Pearson correlation test show significant differences between the course completion rate and several SRL subscales, with very low correlation values (r < 0.1 in all subscales) for the SOL-Qr in the whole samples, as well as between the internship location and former medical education groups. Only the Time Management score was correlated to Course Completion in the Outside Java-Bali group, with a very low correlation value (r = 0.102). Therefore, the course completion rate does not correlate with the online SRL scores.

IV. DISCUSSION

Self-regulated learning is a dynamic process and may change while learners undergo various learning contexts (Barnard et al., 2009). The rapidly changing pandemic has accelerated the adaptation of new learning approaches and methods worldwide, including MOOCs that had gained popularity. Therefore, the use of the SOL-Qr is beneficial for providing MOOC learners’ SRL profiles (Jansen et al., 2018).

This study represents the first successful attempt to cross-culturally validate the SOL-Qr and determine the suitability of all its subscales for profiling online SRL of medical doctors in their early career. The three-stage validation process for the SOL-Qr was conducted as a form of cross-cultural adaptation of the instrument to facilitate its use in measuring self-regulated learning abilities in an online setting in the context of medical education—specifically in the MOOC used for continuing medical education and professional development programs for early-career, newly graduated medical doctors (Hambleton, 2005). The CFA conducted in this study also demonstrates a good fit, with excellent internal consistencies of the SOL-Qr and its subscales; this demonstrates the comparability of the instrument with the original SOL-Qr (Jansen et al., 2018).

The use of the SOL-Qr in this study demonstrates online SRL abilities during the transition phase in the early careers of medical doctors, from medical students to medical interns. This study demonstrates the high average level of online SRL ability among newly graduated medical doctors (SRL score > 5). Participants’ online SRL may increase due to the positive online learning experience obtained, considering that the knowledge on COVID-19 provided in this particular MOOC was vital and timely knowledge for these recent graduates. While participant perceptions of MOOCs have been reported elsewhere, it is known that positive perceptions of an online learning experience and environments are significantly correlated with the online SRL score (Abouzeid et al., 2021; Findyartini et al., 2020; Liaw & Huang, 2013).

The high scores for online SRL in our study were found in total and in most of the subscales, except for the metacognitive skills before learning (MB) scale in participants from the Java-Bali region. A similar finding on the low level of metacognitive ability was reported during the transition phase from preclinical to clinical learning, which was associated with previous clinical experience (Cho et al., 2017). Albeit being assessed using a different instrument, the decreased level of metacognitive skill in this study may also be affected by a similar factor. Since the Java-Bali Region had been the primary epicenter of COVID-19 in the country, with the greatest number of COVID-19 cases (69.41%) out of all other regions (National Ministry of Health, 2021), most interns in the Java-Bali Region may have experienced being removed from their duties at the beginning of the pandemic for safety reasons during their last clinical rotations as medical students (Findyartini et al., 2020). This may have caused discomfort because they felt useless and unable to contribute to patient care, further affecting their sense of competence and motivation, despite previous clinical experiences. Hence, this may have affected the process of goal-setting and reflecting on their prior knowledge during the transition to becoming medical interns (Dornan et al., 2014; Dubé et al., 2015).

Findings of this study demonstrate no significant correlation between online SRL and the number of optional topics achieved by participants. This result was supported by the fact that, despite the high level of online SRL ability depicted in this study, the duration of which participants accessed the compulsory Part A was lower than the expected minimum duration as estimated by course developers although we did not explore whether participants learned about COVID-19 from any other learning resources (Findyartini et al., 2021). The current study also supports the results of a previous study with similar findings regarding online SRL scores and academic achievement, although this study does report a significant relationship between online discussion and academic achievement (Abouzeid et al., 2021). The MOOC examined in this study does not provide two-way interactions among learners or between learners and instructors, which may affect the low number of optional courses completed.

Our study also shows that the online SRL scores of participants in the Outside Java-Bali Region were significantly higher (p < 0.001) across all scales than those of participants in the Java-Bali Region, except on the Persistence scale (p = 0.181). This suggests that participants in the Outside Java-Bali Region also have better time management, environmental structuring, and help-seeking abilities, which may relate to the workload at the internship locations. Participants in the Java-Bali Region face a higher workload and stress as front-liners in managing patients during the COVID-19 pandemic (as health educators, contact tracers, vaccinators, etc.). This aligns with other findings suggesting that an optimal workload determines the quality of students’ SRL in the early transition and adaptation phases (Barbosa et al., 2018).

Furthermore, the MOOC was given to the participants at the beginning of their internship programs, in which several orientations and patient management also took place. Hence, it is confirmed that the participants’ workload affected the time allocated to learn the MOOC (Eriksson et al., 2017). This study also suggests that the use of the MOOC for knowledge provision would preferably occur with sufficient time before immersion in other workplace-based learning experiences.

Hew and Cheung (2014) report few challenges in MOOC completion, including a lack of time and having other priorities to fulfill the course; therefore, time management ability remains crucial. The results of this study indicate that male participants had slightly better time management abilities than female participants. Although the absolute difference of the scores between groups is small, this study involved 5,432 respondents (much more than the minimum sample size), thus small differences in results can be statistically significant. As newly graduated doctors, study participants were interns who worked in a new environment hence new challenges regarding workplace relationships and workload may be faced. Previous studies show that female and male physicians may perceive these challenges differently (Babaria et al., 2009). Female physicians report that they feel uncertain and stressed when facing different clinical environments. Because they tend to need more time to adapt than male physicians (Malau-Aduli et al., 2020), completing a MOOC may not be their priority. Moreover, with masculinity/femininity level of cultural determinant, Indonesians have a clear cut between gender roles, thus affecting roles of female participants in most settings (Mangundjaya, 2013). Female participants may culturally face different roles in their personal lives, such as the expectation to perform housework and childcare, in addition to their internship obligation resulting in conflicting time and higher stress levels. The conflicting time and higher stress level in both work and personal life (Isaac et al., 2013) may influence female participants’ ability to commit time to learn and work and may explain the lower time management scores among female participants.

This study also highlights that, overall, the online SRL scores of participants graduating from the A-level and B-level of accredited medical schools show similar results, except on the time management scale (p = 0.009), where the participants from the higher level accredited schools show higher scores than the participants from the lower level accredited schools. Prior online learning experience has been reported as an important determinant of online learning success (Vilkova, 2019). For newly graduated doctors completing an internship program, prior online learning experience may be largely attributed to the use of online learning formally in their medical school’s curriculum. Our study suggests that the higher accredited medical schools may provide more online learning experiences, leading to better time management skills among the participants from this group.  Furthermore, MOOC completion and the fulfillment of learning outcomes were determined by the forethought phase in the students’ SRL; thus, the goal-setting, self-efficacy, and task values should be emphasised by the participants and facilitated by the MOOC (Vilkova, 2019). Our study further indicates that time management in the use of the MOOC should be considered by learners in the forethought, performance, and self-reflection phases of their SRL; the planning stage of the MOOC development must determine the estimated completion time for the whole course and its sections about the course learning outcomes and the participant’s characteristics (Stracke et al., 2018).

Similar to the SRL ability in offline learning, our findings further imply the importance of accounting for learners’ online SRL abilities, which are dynamic across online learning contexts, including MOOCs. Certain characteristics of MOOCs, such as their open access and self-paced nature, stress the importance of online SRL ability, especially for MOOCs used in the transition phase in the early career of medicine. Therefore, using validated and reliable instruments, such as the SOL-Qr, to measure the online SRL abilities of MOOC participants would help course developers to identify whether the online learning context supports or hinders learners’ SRL abilities, thus helping course administrators further improve MOOC to provide further support for learners’ SRL (Barnard et al., 2009; Sandars & Patel, 2020).

V. LIMITATIONS OF THE STUDY

We identify several limitations of this study. As this study examined online SRL ability of participants using a self-reported questionnaire, it was unable to examine the actual online SRL ability, given the poor correlation with the course completion rate. Furthermore, as the evaluation of online SRL was only conducted once, it was not possible to observe changes in SRL ability throughout the use of the MOOC or in a longer period. With the data that we obtained, we were also unable to analyse whether participants used other online resources to learn about COVID-19 before the internship program or parallel with accessing the provided module as the marker of online SRL nor to explain any causal relationships between online SRL scores and the factors under investigation. However, measurement of online SRL ability using the cross-culturally validated SOL-Qr reveals that this instrument can be used for MOOCs on continuing medical education and professional development in the early-career context. 

VI. CONCLUSION

Our study demonstrates the cross-cultural validity of the SOL-Qr and the suitability of its subscales for use in the medical and health fields. The results portray the online SRL ability of medical doctors as participants in a MOOC during the transition phase in their early career. We found that the time management ability was associated with previous experience during the medical education period, while other subscales were mostly associated with workload during the transition phase. However, as the scores did not correlate with the completion rate of MOOC, it can be concluded that the questionnaire is a possible valid tool to assess self-regulated learning in the MOOC environment. Yet, the corresponding learning effort or completion rate may be beyond just the commitment to the described MOOC.

Notes on Contributors

NG designed and led the study, led data collection and analysis, and led manuscript development. AF, DAK, and CH contributed in data collection, completed data analysis, and contributed to manuscript development.  GS contributed in the data analysis and manuscript development. All authors approved the final version of the manuscript. 

Ethical Approval

This study obtained ethical clearance from Research Ethics Committee of Faculty of Medicine Universitas Indonesia/ dr. Cipto Mangunkusumo General Hospital (KET-1395/UN2.F1/ETIK/PPM.00.02/2020) in 2020.

Data Availability

Data will be available upon request to corresponding author due to conditions of consent provided by respondents in this study and that it should abide data sharing policy from the authors’ institution and the Republic of Indonesia Ministry of Health.

Acknowledgement

The authors would like to acknowledge the Ministry of Health Republic of Indonesia for the trust that has been given to develop and organise the MOOC for the national internship program participants. We would also like to thank all national internship program participants for participation in this study and to Vernonia Yora Saki for assisting the authors with statistical analysis of the study. The preliminary results of this study were presented in Niigata Meeting 2020.

Funding

The development of MOOC and data analysis has been supported by the Ministry of Health Republic of Indonesia through a direct appointment decree to our institution. 

Declaration of Interest

All authors state no possible conflicts of interest, including financial, consultant, institutional, and other relationships that might lead to bias or a conflict of interest.

References

Abouzeid, E., O’Rourke, R., El-Wazir, Y., Hassan, N., Ra’oof, R. A., & Roberts, T. (2021). Interactions between learner’s beliefs, behaviour and environment in online learning: Path analysis. The Asia Pacific Scholar, 6(2), 38-47. https://doi.org/10.29060/TAPS.2021-6-2/OA2338

Asarbakhsh, M., & Sandars, J. (2013). E-learning: The essential usability perspective. The Clinical Teacher, 10(1), 47-50. https://doi.org/10.1111/j.1743-498X.2012.00627.x

Babaria, P., Abedin, S., & Nunez-Smith, M. (2009). The effect of gender on the clinical clerkship experiences of female medical students: Results from a qualitative study. Academic Medicine, 84(7), 859-866. https://doi.org/10.1097/ACM.0b013e3181a8130c

Barbosa, J., Silva, Á., Ferreira, M. A., & Severo, M. (2018). Do reciprocal relationships between academic workload and self-regulated learning predict medical freshmen’s achievement? A longitudinal study on the educational transition from secondary school to medical school. Advances in Health Sciences Education, 23, 733-748. https://doi.org/10.1007/s10459-018-9825-2

Barnard, L., Lan, W. Y., To, Y. M., Paton, V. O., & Lai, S.-L. (2009). Measuring self-regulation in online and blended learning environments. The Internet and Higher Education, 12(1), 1-6. https://doi.org/10.1016/j.iheduc.2008.10.005

Bembenutty, H. (2009). Academic delay of gratification, self-regulation of learning, gender differences, and expectancy-value. Personality and Individual Differences, 46(3), 347-352. https://doi.org/10.1016/j.paid.2008.10.028

Blunch, N. J. (2008). Introduction to structural equation modelling using SPSS and AMOS. SAGE.

Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007

Bullock, A., & De Jong, P. G. M. (2013). Technology-enhanced learning. In T. Swanwick (Ed.), Understanding medical education: Evidence, theory, and practice (2nd ed., pp. 149-160). Wiley Blackwell. https://doi.org/10.1002/9781118472361.ch11

Cho, K. K., Marjadi, B., Langendyk, V., & Hu, W. (2017). Medical student changes in self-regulated learning during the transition to the clinical environment. BMC Medical Education, 17(59). https://doi.org/10.1186/s12909-017-0902-7

Chung, L.-Y. (2015). Exploring the effectiveness of self-regulated learning in massive open online courses on non-native English speakers. International Journal of Distance Education Technologies, 13(3), 61-73. https://doi.org/10.4018/ijdet.2015070105

Dornan, T., Tan, N., Boshuizen, H., Gick, R., Isba, R., Mann, K., Scherpbier, A., Spencer, J., & Timmins, E. (2014). How and what do medical students learn in clerkships? Experience based learning (ExBL). Advances in Health Sciences Education, 19(5), 721-749. https://doi.org/10.1007/s10459-014-9501-0

Dubé, T. V., Schinke, R. J., Strasser, R., Couper, I., & Lightfoot, N. E. (2015). Transition processes through a longitudinal integrated clerkship: A qualitative study of medical students’ experiences. Medical Education, 49(10), 1028-1037. https://doi.org/10.1111/medu.12797

Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29, 133-146. https://doi.org/10.1007/s12528-016-9127-8

Findyartini, A., Anggraeni, D., Husin, J. M., & Greviana, N. (2020). Exploring medical students’ professional identity formation through written reflections during the COVID-19 pandemic. Journal of Public Health Research, 9(s1). https://doi.org/10.4081/jphr.2020.1918

Findyartini, A., Greviana, N., Hanum, C., Husin, J. M., Sudarsono, N. C., Krisnamurti, D. G. B., & Rahadiani, P. (2021). Supporting newly graduate medical doctors in managing COVID-19: An evaluation of Massive Open Online Course in a limited-resource setting. PLoS ONE, 16(9), e0257039. https://doi.org/10.1371/journal.pone.0257039

Hall, A. K., Nousiainen, M. T., Campisi, P., Dagnone, J. D., Frank, J. R., Kroeker, K. I., Brzezina, S., Purdy, E., & Oswald, A. (2020). Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic. Medical Teacher, 42(7), 756-761. https://doi.org/10.1080/0142159X.2020.1766669

Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3–38). Lawrence Erlbaum.

Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. https://doi.org/10.1016/j.edurev.2014.05.001

Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Isaac, C., Petrashek, K., Steiner, M., Manwell, L. B., Carnes, M., & Byars-Winston, A. (2013). Male spouses of women physicians: Communication, compromise, and carving out time. The Qualitative Report, 18(52), 1-12. https://doi.org/10.46743/2160-3715/2013.1423

Jansen, R. S., van Leeuwen, A., Janssen, J., & Kester, L. (2018). Validation of the revised self-regulated online learning questionnaire. In V. Pammer-Schindler, M. Pérez-Sanagustín, H. Drachsler, R. Elferink, & M. Scheffel (Eds.), Lifelong Technology-Enhanced Learning (pp. 116-121). Springer. https://doi.org/10.1007/978-3-319-98572-5_9

Jansen, R. S., van Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2017). Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education, 29, 6-27. https://doi.org/10.1007/s12528-016-9125-x

Jouhari, Z., Haghani, F., & Changiz, T. (2015). Factors affecting self-regulated learning in medical students: A qualitative study. Medical Education Online, 20(1). https://doi.org/10.3402/meo.v20.28694

Liang, Z. C., Ooi, S. B. S., & Wang, W. (2020). Pandemics and their impact on medical training: Lessons from Singapore. Academic Medicine, 95(9), 1359-1361. https://doi.org/10.1097/ACM.0000000000003441

Liaw, S.–S., & Huang, H.–M. (2013). Perceived satisfaction, perceived usefulness and interactive learning environments as predictors to self-regulation in e-Learning environments. Computers & Education, 60(1), 14-24. https://doi.org/10.1016/j.compedu.2012.07.015

Magno, C. (2011). Validating the academic self-regulated learning scale with the Motivated Strategies for Learning Questionnaire (MSLQ) and Learning and Study Strategies Inventory (LASSI). The International Journal of Educational and Psychological Assessment, 7(2), 56-73.

Malau-Aduli, B. S., Roche, P., Adu, M., Jones, K., Alele, F., & Drovandi, A. (2020). Perceptions and processes influencing the transition of medical students from pre-clinical to clinical training. BMC Medical Education, 20, 279. https://doi.org/10.1186/s12909-020-02186-2

Mangundjaya, W. L. H. (2013). Is there cultural change in the national cultures of Indonesia? In Y. Kashima, E. S. Kashima, & R. Beatson (Eds.), Steering the cultural dynamics: Selected papers from the 2010 Congress of the International Association for Cross-Cultural Psychology. https://scholarworks.gvsu.edu/iaccp_papers/105/

National Ministry of Health. (2020, December 31). Health care workforce statistics database in Indonesia. Retrieved May 10, 2021, from http://bppsdmk.kemkes.go.id/info_sdmk/info/index?rumpun=101

National Ministry of Health. (2021, April 23). Current situations of the development of novel coronavirus (COVID-19). Retrieved May 10, 2021, from https://covid19.kemkes.go.id/situasi-infeksi-emerging/situasi-terkini-perkembangan-coronavirus-disease-covid-19-23-april-2021

Ortiz-Martìnez, Y., Castellanos-Mateus, S., Vergara-Retamoza, R., Gaines-Martìnez, B., & Vergel-Torrado, J. A. (2021). Online medical education in times of COVID-19 pandemic: A focus on Massive Open Online Courses (MOOCs), Educación Médica, 22, S40. https://doi.org/10.1016%2Fj.edumed.2020.12.001

Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8, Article 422. https://doi.org/10.3389/fpsyg.2017.00422

Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MLSQ). Educational and Psychological Measurement, 53(3), 801-813. https://doi.org/10.1177%2F0013164493053003024

Poonia, S. K., & Rajasekaran, K. (2020). Information overload: A method to share updates among frontline staff during the COVID-19 pandemic. Otolaryngology-Head and Neck Surgery, 163(1), 60-62. https://doi.org/10.1177%2F0194599820922988

Sandars, J., & Patel, R. (2020). The challenge of online learning for medical education during the COVID-19 pandemic. International Journal of Medical Education, 11, 169-170. https://doi.org/10.5116/ijme.5f20.55f2

Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modelling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323–338. https://doi.org/10.3200/JOER.99.6.323-338

Stracke, C. M., Tan, E., Moreira Texeira, A., do Carmo Pinto, M., Vassiliadis, B., Kameas, A., Sgouropoulou, C., & Vidal, G. (2018). Quality Reference Framework (QRF) for the Quality of Massive Open Online Courses (MOOCs): Developed by MOOQ in close collaboration with all interested parties worldwide. MOOQ. http://www.mooc-quality.eu/QRF

Vilkova, K. A. (2019, May 20-22). Self-regulated learning and successful MOOC completion [Europeaan MOOCs Stakeholders Summit 2019 Conference]. Proceedings of EMOOCs: Work in Progress Papers of the Research, Experience and Business Tracks.

Virtanen, P., & Nevgi, A. (2010). Disciplinary and gender differences among higher education students in self-regulated learning strategies. Educational Psychology, 30(3), 323-347. https://doi.org/10.1080/01443411003606391

Wong, J., Baars, M., Davis, D., Van Der Zee, T., Houben, G.-J., & Paas, F. (2018). Supporting self-regulated learning in online learning environments and MOOCs: A systematic review. International Journal of Human-Computer Interaction, 35(4-5), 356-373. https://doi.org/10.1080/10447318.2018.1543084

Yen, C.-J., Tu, C.-H., Sujo-Montes, L., & Sealander, K. (2016). A predictor for PLE management: Impacts of self-regulated online learning on students’ learning skills. Journal of Educational Technology Development and Exchange, 9(1), 29-48. https://doi.org/10.18785/jetde.0901.03

*Nadia Greviana
Faculty of Medicine Universitas Indonesia,
Jalan Salemba 6 Central Jakarta, Indonesia,
Email: nadiagreviana@ui.ac.id/ nadia.greviana@gmail.com

Submitted: 19 May 2021
Accepted: 26 August 2021
Published online: 4 January, TAPS 2022, 7(1), 66-75
https://doi.org/10.29060/TAPS.2022-7-1/OA2521

Lay Ling Tan¹, Pim W. Teunissen², Wee Shiong Lim³, Vanessa Wai Ling Mok¹ & Hwa Ling Yap¹

¹Department of Psychological Medicine, Changi General Hospital, Singapore; ²School of Health Professions Education (SHE), Maastricht University, Netherlands; ³Cognition and Memory Disorders Service, Tan Tock Seng Hospital, Singapore

Abstract

Introduction: Development of expertise and counselling skills in psychiatry can be mastered only with effective supervision and mentoring. The conceptualisations of educational supervision amongst supervisors and residents were explored in this study to understand how supervisory roles may have been affected by the adoption of competency-based psychiatry residency training.

Methods: A qualitative research approach with thematic analysis was adopted. Individual in-depth interviews using a semi-structured interview guide with a purposive sample of six supervisors and six newly graduated residents were conducted. Transcripts of the interview were analysed and coded using the Atlas Ti software.     

Results: Four major themes emerged from analysis of the transcripts: (1) Meaning and definition of supervision; (2) Expectations and responsibilities of the educational supervisor; (3) Elusiveness of mentoring elements in educational supervision and (4) Personal and professional development of residents in supervision. Supervisors and residents perceived educational supervision narrowly to be transactional with acquisition of knowledge and skills, but residents yearned for more relational interactions.

Conclusion: This study showed that the roles and functions of supervisors in educational supervision were unclear. It also highlighted the lack of a mentoring orientation in supervision in the psychiatry residency training program. An emphasis on assessment of competencies might have contributed to tension in the supervisory relationship and lack of a mentoring role, with concerns on residents’ personal and professional identity development in their psychiatry training. 

Keywords:           Psychiatry, Mentoring, Educational Supervision, Competency-Based Medical Education, Professional Identity Development

Practice Highlights

• Supervision in psychiatry has been described to encompass more than just a teaching and learning relationship but also a supportive and mentoring one.
• Educational supervision has been purported to offer the unique opportunity for effective mentoring within supervision.
• This qualitative study highlighted significant differences in definitions, roles and expectations of educational supervision.
• The conflict between mentoring and appraisal of competency needs to be addressed.
• Roles and expectations of the educational supervisor must be articulated clearly to both supervisors and residents.

Table 1: Faculty’s understanding of the roles and functions of the educational supervisor

2) Residents’ perspectives: Residents’ expected the educational supervisor to be “approachable and open”, “easy to talk to”, “relaxed”, “able to attend to personal growth”, “helping to reflect” and “build rapport”. Residents thus expected a more relational as opposed to transactional interaction with the supervisor.

“…apart from the bread and butter clinical aspects of work…talk to you a bit more about your personal stage in life, how things are coming along… It is this stuff that I find it hard to find in textbooks or anywhere along the clear training roadmap”

(R1)

Residents tended to see the educational supervisor’s role as conflictual in view of the attendant assessor function, and how boundaries between the two roles are often not clearly delineated.

“Ultimately is quite hierarchical in the system…. so if a supervisee has a lot of problems and come to the supervisor for advice…the supervisor might negatively evaluate the supervisee”

(R2)

C. Elusiveness of Mentoring Elements in Educational Supervision

1) Supervisor’s perspectives: Faculty viewed mentoring as “broader”, “longer term” and “beyond clinical and professional development”. “Trust” and “special” characterised a mentoring relationship but the focus was more on “professional development” rather than “personal development”. Faculty did not see themselves as fulfilling a mentoring role but generally agreed that there could be opportunities in offering elements of mentoring in educational supervision and to “contribute to the growth of residents beyond the short-term focus on clearing examinations”.

“…take about certain other aspects you know…mentorship if possible…beyond the pure clinical and professional development”

(S1)

Supervisors alluded to their roles in educational supervision as “facilitating the growth as a person”; “giving advice” and “role modelling” but they did not consider these as mentoring roles even though these were generally accepted as mentoring in nature.

“…never seen myself in a mentoring position…mentoring goes beyond just the supervisor-supervisee relationship…but I don’t think it has really extended beyond that”

(S3)

2) Residents’ perspectives: There was a similar reference to mentoring as “adopting a broader view of the development of the resident” over “a longer period of time”. Residents described mentoring as having a “deeper emotional connect” and “beyond the clinical development”. The evaluator role was viewed as potentially inhibiting the development of a mentoring relationship.

“So, it muddles the role…when they are so tied up to assess…they may not be able to mentor as much…”

(R1) 

D. Personal and Professional Development of Residents

1) Supervisor’s perspectives: Professional development was referred to as the “professional attitudes and motivation”, “duty of care”, “ethical boundaries” and the attributes which are more “formal” as contrasted to personal development. The latter being referred to as “one’s character”, “temperament”, “personality”; “the way we see things”; “development of self-awareness and self-actualisation” and the “need to know yourself and what problems you have in order to be able to help your patients struggling with personal problems”. Faculty also referred to personal development as “developing as a person and handling of stress”, “how to handle complaints”, and “how to juggle various roles to have a work-life balance”. Other views of personal development included “extracurricular activities outside of psychiatry” and “some hobbies” which some faculty opined as “more important in psychiatry because of the importance for self-care”.

However, some faculty held the opposing view that personal development should not be the focus of educational supervision. Lapses in personal development would only be brought up during supervision if there were “negative effects on professional roles and clinical practice and impeding progress” for risk of being accused of “prying into the residents’ personal lives and being nosy”.

“…but I don’t focus so much unless they hinder the work side of things. But maybe we should…”

(S3)

The reluctance of some residents to discuss issues of personal development were interpreted as an “Asian thing”, for instance being “uncomfortable” talking about personal struggles and “residents not expecting it”’.

2) Residents’ perspectives: Residents’ referred to personal developmental aspects as “be as a person”, “religious growth”, “personal well-being”, “how you are getting on with life as a whole”, “finding out about the person’s preferences”, “strengths and weaknesses” and “outside of the career”.

There was the fear of the lack of confidentiality and of being evaluated negatively if residents were to portray themselves as having personal struggles.

“though resident want to grow and develop but exposing these shortcomings could be very sensitive…”

(R1)

Residents considered personal aspects of their development to be “more private”; “should not be covered unless interferes with professional development”; “not so important”; “not fair for the supervisee and supervisor” and “something you should sort out on your own”. They perceived supervision to be formal and mainly moments of assessment for their professional development and so it would be inappropriate to discuss about personal struggles. Some residents also held the view that “personal and professional lives are separate” and the “supervisor may not be interested”.

“…to say that the supervisor should cover personal growth I don’t think that is very fair as well”

(S4)

IV. DISCUSSION

A. Conceptualisation of Educational Supervision

The concept of educational supervision was alien to both residents and supervisors. Supervision was seen mainly as teaching the residents to acquire knowledge and clinical skills with a focus on the transactional aspects. Our residents preferred a more relational supervisory interaction. The finding of psychiatry trainees valuing a supervisor’s emotional supportiveness more highly than clinical competence was also reported in another study (Chur-Hansen & McLean, 2007).

Our results showed that roles of teaching and assessing were more consistently endorsed by the supervisors. Relational roles like facilitating, role modelling, rapport building, and mentoring were considered important but cited less frequently by both groups. Participants tended to attribute this to the training’s emphasis on evaluation and assessment such that the focus of the supervision was more on the transactional rather than the relational components. This might have resulted in tension within the supervisory relationship as expectations for assessment of competencies take precedence (Julyan, 2009). This phenomenon was both ironic and worrying as the original intention for setting up the supervisory framework in the psychiatry residency program was to ensure that the training and learning would be supplemented by the presence of an educational supervisor with a mentoring role, entrusted with the fostering of the personal and professional components in clinical supervision (Kilminster et al., 2007).

B. Assessor Role of Supervisors

Although residency assessments were mainly formative, residents in our study still harbored concerns about the supervisor fulfilling the assessor role and determining their readiness to progress. If assessments were to be perceived as high stake in our examination-oriented training environment, the role of the teacher as helper might be compromised (van der Vleuten et al., 2012). As highlighted by the data in our study, combining the roles of helper and judge could confront the supervisor with a conflict of interest (Cavalcanti & Detsky, 2011). There might be situations either of inflation of judgement (Govaerts et al., 2007) or trivialisation of the assessment process (Dudek et al., 2005), which would potentially impair the professional development of the residents.

C. Personal and Professional Development

There was a common reference by both faculty and residents to the growth of the resident. Based on our data, there appeared to be some overlap of the concept of personal and professional development of residents in psychiatry training. Supervisors viewed the ability to handle stress and developing resilience not only as aspects of personal development but a reflection of the professional competence as well. But supervisors also opined that within the current supervisory framework, they would not be able to support the personal development of the residents. Although residents indicated their desires for supervisors to facilitate their holistic development and growth towards becoming a psychiatrist, they did not expect supervisors to be interested in their personal development but to focus more on the professional development of their clinical competencies.

There was avoidance by supervisors and residents to discuss personal struggles and frustrations in our study. This would be of concern if in the context of educational supervision in psychiatry, the personal aspects and development of better self-awareness could not be achieved, bearing in mind that this had been considered by some to be the fundamental goal of supervision in counselling (Kadushin, 1985). Concerns of boundary violations within the supervision relationship were raised by both groups of participants. In our study, the supervisors’ strict adherence to the boundaries as accustomed to in psychotherapeutic practice might deter self-disclosure. In contrast, our data suggested that sharing of clinical experiences by their supervisors were very much welcomed by the residents. Research has shown that there might be improvement in alliance when supervisors disclosed relevant past clinical experiences (Matazzoni, 2008). Self-disclosure by the supervisor usually normalised clinical struggles experienced by the residents and could enhance the supervision relationship (Knox et al., 2011). In our study, supervisors held fairly rigid boundaries within supervision, which had been shown to hinder the development of authentic emotional relationships or deeper mentoring relationships (Kozlowski et al., 2013).

Supervisors in our study reflected that it might have been cultural or an ‘Asian thing’ for residents to avoid discussion of personal struggles. Eastern cultures were noted to appreciate a larger power distance than Western cultures (Hofstede, 2011). Cultural value theory also opined that Eastern cultures tended to be more conservative and hierarchical and valued mastery to a lesser extent than the West (Schwartz, 1999). In our predominantly Asian context, the perception of a hierarchical training system might result in supervisors maintaining a stricter teacher-student boundary, or residents being more reluctant to share personal frustrations, particularly with the more senior supervisors. The evaluator role of the supervisor might also result in the resident erecting certain boundaries in the supervisory relationship. This would have implications for nurturing the personal and professional growth of the residents, which require guidance through mentorship and self-reflection within a trusting relationship (Holden et al., 2015).

D. Psychological Safety Within a Trusting Supervisory Relationship

Previous research has shown that any feedback which invoked the self potentially carried with it social judgements which might threaten the educational alliance (Pugh & Regehr, 2016; Telio et al., 2015) and there was a tendency for both supervisors and supervisees to interpret performance assessments as part of a judgement of personal worth (Hawe, 2003). The psychological safety within the supervisory relationship would be particularly important as awareness of the residents’ own inadequacies might be unpleasant and threatening as they faced their imperfect understanding and subjective theories (Vec et al., 2014). As such, the goals of supervision would be best attained with a trusting supervisor-supervisee relationship. However, our study showed that residents are unwilling to reveal too much of their inadequacies as this was too threatening for them, considering that their supervisors also evaluated their overall work performance and ability to deal with stress. This had resulted in tension in the supervisory relationship. The failure to pay heed to this, whether it was inherent to the training program or secondary to the supervisors’ lack of awareness, might further hinder and jeopardise the supervisory process.

The tension between assessment for support and assessment for high stakes decision-making will continue to challenge supervisors. The conflict between mentoring and appraisal of competency would need to be addressed. It would be important for residency training programs to create opportunities to allow the fostering of trusting supervisory relationships. Roles and expectations of the educational supervisor would need to be articulated clearly to both supervisors and residents. Supervisor training would need to focus not only on supervisor ability and competencies but more importantly, supervisor motivation. There should be the emphasis on instilling awareness of internal values and beliefs encompassing competency assessment, accountability, potential role conflicts, feedback delivery and “skills for establishing trusting, open and non-defensive yet problem–confronting relationships” (Govaerts et al., 2007).

E. Limitations of the Study

In our study, newly graduated residents who were agreeable to participate were recruited. The views of the ‘unwilling’ participants regarding supervision which might be more diverse and contentious might be inadvertently excluded. The content of the interview guide used was also not validated. Another limitation concerned the dual roles of researcher and APD. The view of the resident group might be subjected to biases and undue influence due to power differentials (Kotter, 2010). The researcher had minimised such potential biases by being reflexive and addressed concerns of imposing on the participants’ views openly during the interview (Creswell & Miller, 2000).

V. CONCLUSION

The mentoring role in supervision was found to be lacking in our current residency training. The residency program structure, with its focus on assessments of competencies and examinations, might have the unintended consequences of encouraging a transactional supervisory structure at the expense of a relational and mentoring relationship. This qualitative study highlighted significant differences in definitions, roles and expectations of educational supervision. It was our intention that this research endeavor contribute towards better appreciation of the dynamics within educational supervision in a competency-based residency training framework and further inform developments in the mentoring component of supervisory practices in the other training programs as well.

Notes on Contributors

Dr Lay Ling Tan formulated the research question and designed the research methodology. She conducted the semi-structured interviews and wrote the first draft of the manuscript.  

Prof Pim W. Teunissen provided guidance for the research methodology and data analysis. He was also involved with the revision of the manuscript drafts.

Dr Wee Shiong Lim provided guidance for the research methodology, assisted with the initial analysis of the first three interviews independently as well as the revision of the manuscript drafts.

Both Dr Vanessa Wai Ling Mok and Dr Hwa Ling Yap were involved with recruitment of participants and data analysis.

Ethical Approval

Ethics approval was sought from the Institutional Review Board (CIRB Ref: 2017/2319) and informed consent was received from all participants.

Data Availability

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.

Acknowledgement

We would like to acknowledge the contributions of the National Psychiatry Residency Program Supervisors and Residents for their willingness to participate.

Funding

We did not receive any funding for this research study.

Declaration of Interest

Dr Lay Ling Tan is the Associate Program Director and the PI of this research. Dr Hwa Ling Yap and Dr Vanessa Mok are both teaching faculty of the teaching site. They have a vested interest in ensuring the quality of supervision of residents. The other authors have no other conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Akhtar, S. (2009). Comprehensive dictionary of psychoanalysis. Routledge.

Carroll, M. (2006). Counselling supervision: Theory, skills and practice. SAGE.

Cavalcanti, R. B., & Detsky, A. S. (2011). The education and training of future physicians: Why coaches can’t be judges. JAMA, 306(9), 993-994. https://doi.org/10.1001/jama.2011.1232

Chay, O. M. (2019). Transformation of medical education over the years – A personal view. The Asia Pacific Scholar, 4(1), 59–61. https://doi.org/10.29060/TAPS.2019-4-1/PV1076

Chur-Hansen, A., & McLean, S. (2007). Trainee psychiatrists’ views about their supervisors and supervision. Australasian Psychiatry, 15(4), 269-272.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed method approaches (4th ed.). SAGE.

Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory Into practice, 39(3), 124-130. https://doi.org/10.1207/s15430421tip3903_2

Driessen, E. W., Overeem, K., & van der Vleuten, C. P. M. (2011). Get yourself a mentor. Medical Education, 45(5), 438-439. https://doi.org/10.1111/j.1365-2923.2011.03948.x

Dudek, N. L., Marks, M. B., & Regehr, G. (2005). Failure to fail: The perspectives of clinical supervisors. Academic Medicine, 80(10), S84-S87.

Falender, C. A., & Shafranske, E. P. (2007). Competence in competency-based supervision practice: Construct and application. Professional Psychology: Research and Practice, 38(3), 232–240. https://doi.org/10.1037/0735-7028.38.3.232

Frank, J. R., Snell, L., Englander, R., & Holmboe, E. S. (2017). Implementing competency-based medical education: Moving forward. Medical Teacher, 39(6), 568–573. https://doi.org/10.1080/0142159x.2017.1315069.

Govaerts, M. J., van der Vleuten, C. P., Schuwirth, L. W., & Muijtjens, A. M. (2007). Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Advances in Health Sciences Education Theory Practical, 12(2), 239-260. https://doi.org/10.1007/s10459-006-9043-1

Hansman, C. A. (2001). Mentoring as continuing professional education. Adult learning, 12(1), 7–8. https://doi.org/10.1177/104515950101200104

Hawe, E. (2003). “It’s Pretty Difficult to Fail”: The reluctance of lecturers to award a failing grade. Assessment & Evaluation in Higher Education, 28(4), 371–382. https://doi.org/10.1080/0260293032000066209

Hofstede, G. (2011). Dimensionalizing Cultures: The Hofstede Model in Context. Online Readings in Psychology and Culture, 2(1), 1–26. https://doi.org/10.9707/2307-0919.1014

Holden, M. D., Buck, E., Luk, J., Ambriz, F., Boisaubin, E. V., Clark, M. A., Mihalic, A. P., Sadler, J. Z., Sapire, K. J., Spike, J. P., Vince, A., & Dalrymple, J. L. (2015). Professional identity formation: Creating a longitudinal framework through TIME (Transformation in Medical Education). Academic Medicine, 90(6), 761–767. https://doi.org/10.1097/acm.0000000000000719

Julyan, T. E. (2009). Educational supervision and the impact of workplace-based assessments: a survey of psychiatry trainees and their supervisors. BMC Medical Education, 9(1), 51. https://doi.org/10.1186/1472-6920-9-51

Kadushin, A. (1985). Supervision in social work. Columbia University Press.

Kilminster, S., Cottrell, D., Grant, J., & Jolly, B. (2007). AMEE Guide No. 27: Effective educational and clinical supervision. Medical Teacher, 29(1), 2–19. https://doi.org/10.1080/01421590701210907

Knox, S., Edwards, L. M., Hess, S. A., & Hill, C. E. (2011). Supervisor self-disclosure: Supervisees’ experiences and perspectives. Psychotherapy, 48(4), 336–341. https://doi.org/10.1037/a0022067

Kolb, D. A. (2014). Experiential learning: Experience as the Source of Learning and Development (2nd ed.). Pearson Education, Inc. (Original work published 1984).

Kotter, J. P. (2010). Power and influence. Simon and Schuster.

Kozlowski, J. M., Pruitt, N. T., DeWalt, T. A., & Knox, S. (2013). Can boundary crossings in clinical supervision be beneficial? Counselling Psychology Quarterly, 27(2), 109–126. https://doi.org/10.1080/09515070.2013.870123

Launer, J. (2013). Supervision, mentoring and coaching. In Swanwick. T (Ed.), Understanding medical education. Evidence, theory and practice (2nd ed., pp 111-122). Wiley Blackwell. https://doi.org/10.1002/9781118472361.ch8

MacDonald, J. (2002). Clinical supervision: A review of underlying concepts and developments. Australian & New Zealand Journal of Psychiatry, 36(1), 92-98. https://doi.org/10.1046/j.1440-1614.2002.00974.x

Matazzoni, T. A. (2008). The influence of supervisor self-disclosure on the supervisory working alliance in beginning and advanced therapists-in-training [Doctoral dissertation]. ProQuest.

Passi, V. (2016). The importance of mentoring during educational supervision. Perspectives on Medical Education, 5, 195-196.

Patel, P. (2016). An evaluation of the current patterns and practices of educational supervision in postgraduate medical education in the UK. Perspectives on Medical Education, 5(4), 205-214. https://doi.org/10.1007/s40037-016-0280-6

Pront, L., Gillham, D., & Schuwirth, L. W. (2016). Competencies to enable learning-focused clinical supervision: a thematic analysis of the literature. Medical Education, 50(4), 485–495. https://doi.org/10.1111/medu.12854

Pugh, D., & Regehr, G. (2016). Taking the sting out of assessment: is there a role for progress testing? Medical Education, 50(7), 721-729.

‌Schon, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching learning in the professions. John Wiley.

Schwartz, S. H. (1999). A theory of cultural values and some implications for work. Applied psychology, 48(1), 23-47. https://doi.org/10.1111/j.1464-0597.1999.tb00047.x

Tebes, J. K., Matlin, S. L., Migdole, S. J., Farkas, M. S., Money, R. W., Shulman, L., & Hoge, M. A. (2011). Providing competency training to clinical supervisors through an international supervision approach. Research on Social Work Practice, 21(2), 190-199. https://doi.org/10.1177/1049731510385827

Telio, S., Ajjawi, R., & Regehr, G. (2015). The “educational alliance” as a framework for reconceptualizing feedback in medical education. Academic Medicine, 90(5), 609-614. https://doi.org/10.1097/acm.0000000000000560

Torby, A., Turner, M. B., Kinzie, J. M., & Usher, C. (2015). What we talk about when we talk about supervision: The clear and the confusing in graduate psychiatric education. Journal of the American Psychoanalytic Association, 63(5), NP7-NP12. https://doi.org/10.1177/0003065115609724

van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205-214. https://doi.org/10.3109/0142159X.2012.652239

Vec, T., Rupnik Vec, T., & Žorga, S. (2014). Understanding how supervision works and what it can achieve. In Watkins C. E. &. Milne D. I (Eds.) The Wiley International Handbook of Clinical Supervision (pp 103-127). Wiley. https://doi.org/10.1002/9781118846360.ch5

Wald, H. S. (2015). Professional identity (trans) formation in medical education: Reflection, relationship, resilience. Academic Medicine, 90(6), 701-706.

*Lay Ling Tan
2 Simei Street 3 S529889
Email: tan.lay.ling@singhealth.com.sg

Submitted: 23 January 2021
Accepted: 11 August 2021
Published online: 4 January, TAPS 2022, 7(1), 55-65
https://doi.org/10.29060/TAPS.2022-7-1/OA2475

Marcus A Henning¹, Vanamali Joseph¹, Roger J Booth², Christian U Krägeloh³ & Craig S Webster¹

¹School of Medicine, University of Auckland, New Zealand; ²Department of Medical Science, University of Auckland, New Zealand; ³Department of Psychology and Neuroscience, Auckland University of Technology, New Zealand

Abstract

Introduction: This study investigates psychological distress and quality of life (QoL) amongst first year premedical and health science students. The primary aim of this study was to investigate potential differences in QoL and psychological distress between students who sought entry into a medicine programme when compared to those opting for a non-medicine career. 

Methods: We examined participant responses to measures of QoL, psychological distress, and course preference (medicine or other). A structural equation model was conducted to consider the interrelationships among future course preference, gender, QoL, depression, anxiety and stress.

Results: Three hundred and sixty-five students completed the online survey. An a priori conceptual model was developed and then evaluated using a structural equation model. The values obtained for RMSEA (0.027), CFI (0.999), and SRMR (0.016) indicated an excellent model fit. The overall model fit statistic, chi-square (χ2 = 7.626, df=6, p= .267), confirmed a good model fit. Students aiming to enrol in medicine generated higher psychological health and environmental QoL scores compared to their non-medicine oriented peers. In addition, physical QoL and psychological health QoL scores significantly predicted psychological distress measures.

Conclusion: The study raises a potential debate regarding placing students with mixed career intentions into the same course and the potential implications this may have on teaching in interprofessional and large student groups in relation to wellbeing, pedagogy, equity, and expenditure. The findings clearly indicated that medical students are not as adversely impacted upon in terms of QoL and psychological distress compared with their non-medical peers.

Keywords:           Medical and Health Science Students, Quality of Life, Psychological Distress, Course Preference

Practice Highlights

• Students focusing on medicine report higher levels of psychological health and environmental QoL.
• Course preference is associated with psychological QoL and psychological distress.
• Issues with psychological QoL are more common among females.
• QoL and learning in large lecture theatres requires further investigation.

I. INTRODUCTION

At the University of Auckland, the Biomedical Common Year or Overlapping Year 1 (BCY) programme is a first-year syllabus and the gateway for many students aiming to enrol into either medicine or other healthcare disciplines (e.g., nursing, optometry, or pharmacy). The requirements for entry into medicine is highly competitive such that approximately 700 students who study in BCY aim to apply for medicine out of which only 250 students are chosen (Henning et al., 2017b). Entry into other healthcare disciplines (e.g., nursing, optometry, or pharmacy) is less competitive and selection is thus likely less distressing (Al Alwan et al., 2013). However, the large class sizes within the pre-medical and health science year (of approximately 1,300 students) containing students applying for either medicine or non-medicine programmes may also create a marginalising learning dynamic that may hinder the motivation and wellbeing of non-medicine students (Henning et al., 2017a). Hence, the aim of this research was to assess whether those students aiming for selection into medicine were more at risk of poor quality of life (QoL) and psychological distress relative to those students aiming for selection into other healthcare disciplines.

Numerous studies have researched the areas of QoL and psychological distress amongst medical students (Lovibond & Lovibond, 1995; Szabó, 2010; Wickramasinghe et al., 2019).  In their systematic review, Dyrbye et al. (2006) reported concerning rates of depression, anxiety and general psychological distress among medical students, citing for example in one report that 37% of medical students in their first year of study experienced mental health issues of some kind. The adverse impact of distress among students can contribute to problems associated with academic attainment, professional development, academic honesty, substance abuse, cynicism, humanitarian attitude, and development of empathy. Moreover, psychological distress appears to be more of a concern among female medical students. The implication underlying the research investigating QoL and psychological distress suggests that medical students are more compromised than their non-medical peers.

Our conceptual model for the present study was developed in accordance with self-determination theory (Ryan & Deci, 2000). According to this theory, the act of choosing can positively affect motivation, well-being and learning if the needs for autonomy, relatedness, and competence in the individual are high. In this study, students were free to choose their course of study and hence freedom of choice was perceived as equivalent to the notion of course preference. According to this theory, students can be intrinsically motivated (engaged in their studies because of inherent enjoyment) and extrinsically motivated with a focus on outcome (Vansteenkiste et al., 2006). Extrinsic motivation, however, can be differentiated into aspects of regulation dependent upon the degree of autonomy. Therefore, there are facets of autonomous motivation (which involves volition and choice) versus that of controlled motivation (feeling pressured or coerced) (Vansteenkiste et al., 2006). Hence, intrinsic motivation and well-intentioned aspects of extrinsic motivation are considered autonomous, whilst the poorly internalised form of extrinsic motivation are less well incorporated into an individual’s sense of self.  In our proposed model, we suggest features of choice with links to QoL.

We cannot assume that the medical or non-medical students differ in terms of their intrinsic or extrinsic motivation, but we do know that the majority of non-medical students at this university (e.g. nursing and pharmacy) have been given conditional entry into their courses on the proviso that they pass their courses in the BCY.  However, medical students do not have this luxury and are in a more competitive mode of action. They are, thus, likely intrinsically and extrinsically motivated, and their extrinsic motivation could be either autonomous or controlled.  Our assumption is that the non-medical students are also likely to be intrinsic and extrinsically motivated, but their extrinsic motivation is likely more autonomous as they have conditional entry. Therefore, non-medical students would likely experience more heightened levels of QoL than medical students who are experiencing higher levels of competition (Katz & Assor, 2007; Ryan & Deci, 2000). If this is not true another more pressing variable may be affecting the QoL outcome, such as mixing high numbers of students together from different disciplines into one course (Hornsby & Osman, 2014). In addition, given the established empirical evidence (Haist et al., 2000; Henning et al., 2013; Naylor et al., 2018; Ramsbottom-Lucier et al., 1995), we argued that gender would likely moderate choice and QoL. Therefore, the first stage of our conceptual model proposes that, once students have chosen a particular course of study (medicine or other) this will directly impact their QoL given the increased expectation associated with the decision process (Katz & Assor, 2007).  Therefore, those students aiming for a programme (i.e., medicine) requiring higher stakes will likely be more at risk of ill-health.

The next element of our argument proposes that QoL will likely impact psychological distress. For example, if a student experiences good health-related physical QoL they are then likely to have lowered risk of psychological distress (Burris et al., 2009). In their study of first year medical students, Verger et al. (2009) reported that psychological distress was moderated by gender and influenced by wellbeing protective factors, such as social support, coping with academic pressure, and socio-economic factors. Consequently, we suggest that there is likely a follow-on effect such that those students with compromised QoL will likely experience more psychological distress (Dyrbye et al., 2006). Nonetheless, the causes of poor QoL are likely complex and vary substantially between individuals given that some studies have shown that students aiming to study medicine may be more motivated and experience higher levels of QoL (Henning et al., 2017a).

The primary aim of this study was to empirically investigate potential differences in QoL and psychological distress between students who sought entry into a medicine programme when compared to those opting for a non-medicine career.  The issue of follow-on effects from QoL to psychological distress has not been studied within this learning context. To illustrate potential differences between the two groups, we scrutinised conceptual links between choice, gender, QoL and psychological distress in reference to students’ course preference (medicine or other) whilst studying in the BCY programme. 

II. METHODS

A. Participants

All students (N=1277) studying within the BCY programme were invited to participate in this survey. Students aiming to study in any health-related course at the University of Auckland first enrol in the BCY programme in preparation for formalised entry into degrees for Bachelor of Health Science, Bachelor of Pharmacy, Bachelor of Optometry or Bachelor of Medicine (The University of Auckland, 2019).

B. Procedure

Ethics approval for the collection and use of data was obtained from the University of Auckland Human Participants Ethics Committee (Ref. 021985).

Students were informed about the study by email in late July (2018), and two reminder emails were sent out. The anonymous survey was conducted prior to final course completion in Semester 2 (2018), but after grades for the first semester had been released. Informed consent was implied given that study participants received a participant information sheet prior to the start of the questionnaire.  Data were collected online using a Google Forms questionnaire (Google.com, 2015), which requires respondents to fully complete the survey before they are able to submit it.

C. Measures

Demographic and background details were obtained to assess the representativeness of the sample included: (1) Age in years; (2) Gender; (3) Future course preference (medicine or other); (4) Enrolment status (international or domestic); and (5) Ethnicity.

Quality of life was measured using the New Zealand version of the WHOQOL (NZ WHOQOL-BREF) questionnaire (Krägeloh et al., 2016). It has four domains (physical QoL, psychological health QoL, social relationships QoL and the environmental QoL) and contains 31 items. In this study, physical QoL had six items, psychological health QoL had 10 items, social relationships QoL had four items and environmental QoL had eight items (The NZ WHOQOL Group, 2017). Questions 1 and 2 are global QoL and health items and, thus, not included in the scoring of the four quality of life domains. Question 3 was deemed to have little relevance with respect to the research question and had been previously identified as having a ceiling effect in early career medical students (Krägeloh et al., 2011). All items utilised a 5-point Likert scale with varying anchors (the scores for two items were reversed). High scores indicated higher levels of QoL.

Psychological distress was measured using the Depression, Anxiety and Stress Scale (DASS-21), which captures respondents’ emotional states in reference to three domains: depression, anxiety and stress (within the past seven days). Each domain is comprised of seven items. The DASS-21 instrument has been developed from a previous 42-item version (Lovibond & Lovibond, 1995), and has undergone psychometric investigation among a variety of populations (Le et al., 2017; Medvedev et al., 2018; Szabó, 2010; Yıldırım et al., 2018). Each item utilises a 4-item Likert scale. Higher scores indicate more frequency and severity in the experience of negative emotions.

D. Data Analysis

First, the response rate (n/N) was calculated as a percentage measure, and the details of the participants were described. Second, descriptive statistics (means and standard deviations (SDs)) were computed for the two self-reported questionnaires measures (NZ WHOQOL-BREF and DASS-21). These measures were computed for each of the questionnaire domains with respect to course preference (medicine; other) and gender (male; female). A multivariate analysis of variance (ANOVA) approach was used to assess levels of significance for the independent variables, course preference and gender, with respect to the aforementioned dependent variables.

Lastly, one structural equation model (SEM) was developed to consider the interrelationships among course preference, gender, and QoL and DASS-21 domain measures. This SEM was developed using the software AMOS v25 (Arbuckle, 2017). The primary goodness-of-fit indices used to assess the model fit were inspection of the model chi-square value, root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardised root mean square residual (SRMR). Model fits were considered acceptable if they met the following recommendations: (1) RMSEA < 0.06; (2) SRMR < 0.08; and (3) CFI close to 0.95 or higher (Hu & Bentler, 1998). Additionally, using maximum likelihood bootstrapping with 1,000 samples, regression weights were computed, and bias-corrected 95% confidence intervals were determined (Byrne, 2010).

III. RESULTS

A. Response Rate and Participant Data

Three hundred and sixty-five students (n = 365) fully completed the online survey resulting in a response rate of 29%. The average age of participants was 19 years (SD = 1.84) with 75% of respondents being female. Table 1 provides the demographic profile for the sample, and upon examination, the distributions of the sample data appeared representative of the population profile. In this study, more than 75% of participants stated a preference to study medicine as opposed to other non-medicine health options, which is a rate higher than that found elsewhere (Henning et al., 2017b). The data that support the findings of this study are openly available in Figshare at https://figshare.com/articles/dataset/_/1405 0463 (Henning et al., 2021).

Table 1. Demographic and academic achievement details of the sample versus the population.

The means and standard deviations for the variables of interest are shown in Table 2. The NZ WHOQOL-BREF questionnaire means for these participants are marginally lower than those found elsewhere (Henning et al., 2017b). No comparable New Zealand medical student reference values for the DASS-21 are available for this population. In reference to internal consistency coefficients computed for each of the measures (Table 2), all Cronbach’s alpha values were equal to or greater than 0.70 thus meeting ‘acceptable’ requirements (Field, 2005).

Table 2. Means, standard deviations (SD) and internal consistency scores (Cronbach alpha) of the measures used in the study.

Note: Using a multivariate ANOVA approach, significant differences (p<.05) for course preference (medicine vs. other) were noted for psychological health QoL, environmental QoL, and depression. In addition, significant differences (p<.05) for gender (male vs. female) were noted for psychological health QoL, depression, anxiety and stress.

B. Structured Equation Modelling

Based on previous literature (Dyrbye et al., 2006) and self-determination theory (Katz & Assor, 2007; Ryan & Deci, 2000), an a priori conceptual model was developed and tested using a SEM framework. The primary criterion variable was the students’ course preference (medicine, scored as 1 vs. other, scored as 0). The values obtained for RMSEA, SRMR, and CFI indicated that the model fit was excellent, that is, RMSEA = 0.027, CFI = 0.999, and SRMR = 0.016. As a secondary check, the overall model fit determined by the chi-square (χ2 = 7.626, df=6. p= .267) also indicated the model fitted very well. We noted that some of the error variances within measures of the NZ WHOQOL-BREF and DASS-21 were required to be correlated.

Figure 1 shows the path diagram for all the data collected (n = 365). The path coefficients are shown for significant relationships only (p < .05). The bias-corrected 95% confidence intervals for the regression weights (path coefficients) obtained using the maximum likelihood bootstrapping method were generated (Table 3) to ensure the accuracy and stability of the parameter estimates (Byrne, 2010).

Figure 1. Path diagram of the tested structural equation model for all study participants.

Note. The path standardised regression weight estimates were shown only when statistically significant (p<.05). For non-significant relationships, directive arrows are shown with dashed lines (see Table 3 for details). Errors for all pertinent measures were removed to improve clarity of the figure. Error variances among the WHOQOL- NZ version and DASS-21 domian measures were were allowed to be correlated. No other error variances were correlated. Key fit indices and chi-square calculations were standardised RMR = .016, CFI = .999, RMSEA = .027, χ2 = 7.626, df=6. p= .267

Table 3. Standardised regression weights: Lower and upper ends of bias-corrected 95% confidence intervals of the regression weights of the directional paths of the SEM.

Note. * p<.05

When inspecting Table 2 and Figure 1, we can make certain assertions related to the differences between the two groups (medical and other students) with respect to QoL and psychological distress. Figure 1 indicates two significant associations between course preference and psychological health QoL and environmental QoL. Therefore, students indicating a preference for enrolling in a medical course were likely to experience greater psychological health QoL and environmental QoL. The findings in Table 2, clarify that students aiming to study medicine have higher levels of QoL in these two domains, compared to students aiming to study in non-medicine courses. In addition, the multivariate analyse indicate that depression is more of a concern for non-medical students than intending medicine students.

IV. DISCUSSION

The known health concerns associated with first year students (Naylor et al., 2018; Tinto, 2009) appear consistent with the current findings. The actual values (Table 2) for three of the four QoL domains from the NZ WHOQOL-BREF -psychological health QoL, social relationships QoL, and environmental QoL – are lower than previously published values for year 4 and 5 students at the same university (Henning et al., 2012), although the physical QoL score was higher. This indicates that year 1 students at this university appear to have relatively lower QoL than their year 4 and 5 peers and are consequently lower than the general population norms, particularly in the areas of psychological health QoL, social relationships QoL, and environmental QoL (Henning et al., 2012). In addition, the values for all domains are lower than those acquired from a 2015 first-year cohort (overall) studying in the same programme (Henning et al., 2017b).

Similarly, the DASS-21 findings (Table 2) show that this group scored higher on the domains of depression and anxiety compared to early career doctors also graduating from the same university (Henning et al., 2014). In comparison to a Malaysian student group, the mean values for the present study show distinct similarity in terms of stress and anxiety scores, but a marked lower score for depression (Yusoff et al., 2013). Therefore, we have provided good evidence to suggest that this group of students, irrespective of choosing either medicine or other as their preferred course option, studying in the BCY programme are experiencing relatively lower levels of QoL and higher levels of psychological distress compared to previous cohorts studying at the same university. However, their experiences may not differ greatly when compared to those students’ psychological distress perceptions in other universities.

In reference to the conceptual model, the findings also confirm that preference for course of study is linked with aspects of QoL. This in turn influences psychological distress, suggesting a feedback loop. Both of these associations were confirmed by the path analysis resulting in an excellent model fit for the SEM (Figure 1). The first part of the model considered how the preferred course option (medicine versus other) could directly impact QoL measures and the second part of the model considered the flow-on (indirect) effects to psychological distress (depression, anxiety and stress).

A. Choosing to Study medicine and Its Impact on Quality of Life

Figure 1 clearly shows that choosing to study medicine has an influence on aspects of QoL, although gender was not found to be significantly correlated with course preference. The decision to choose to study medicine was directly and significantly associated with psychological health QoL and environmental QoL (with higher scores noted for intending medicine students), but did not influence either physical QoL or social relationships QoL. In Table 2, we noted that psychological health QoL and environmental QoL for those vying for a place in the medical programme was higher than students’ scores for other health options (e.g., nursing, pharmacy, or optometry).  The direction of the arrows in the model implies that the act of deciding to study medicine is likely influencing QoL on these two domains. These findings can be interpreted in several ways.

First, it is likely that students who aim for admission to medicine may be more psychologically robust to begin with. The psychological health QoL domain of the New Zealand version of the WHOQOL encompasses 10 facets (Krägeloh et al., 2016; WHOQoL Group, 1998). The facets that likely emphasise the difference between those applying for medicine and those applying for another degree are self-esteem, perception of negative or positive feelings, concentration, ability to meet expectation, and management of personal circumstances. In a previous study (Henning et al., 2017b), perceived stress was found to be highly correlated with psychological QoL and also noted to be a strong predictor of course preference (medicine versus other), with potential medical students scoring higher than their other heath professional peers.

Second, the aspects of cognitive capacity, personal management, and mental wellbeing need to be weighed in comparison to environmental QoL (measured in perceived access to resources, i.e., finance, health care, transport, security, technology, and housing) (WHOQoL Group, 1998). The current findings confirm previous research linking course preference to environmental QoL, suggesting that those students who come from affluent and supportive backgrounds will likely have more interest in pursuing a medical degree (Henning et al., 2017b).

B. Impact of Quality of Life on Psychological Distress

The SEM also revealed an interesting flow-on effect suggesting that those students not choosing medicine will have more issues with psychological wellbeing than those choosing to pursue medicine, which translates to greater risk of psychological distress in the form of depression, anxiety and stress for non-medical students. Furthermore, intending medicine students generated significantly lower depression scores than their non-medicine peers. Even though physical QoL is also directly connected with psychological distress factors, this association was not influenced by course preference. Hence, the key issues linked to course preference are related to psychological health QoL, which is incidentally linked to gender. Environmental QoL appears to have no flow-on psychological distress effects.

These findings indicate that some perceptions of QoL are likely linked to choice and ultimately psychological distress factors. The flow-on effect suggests that those students who struggle with psychological health QoL tend to further struggle with psychological distress. The point of difference between the two groups (medical and other) is that the intending medical students appear less affected by depression than their non-medical peers. The concept of choice has been established as a key component in motivation and wellbeing, for example, the perceived expectancies and values regarding a task are influenced by weighing up the benefits and costs associated with the forthcoming task (Katz & Assor, 2007). Hence, in the medical context the preferred choice of medicine likely has a powerful impact on motivation, learning and wellbeing (Katz & Assor, 2007). The current findings indicate that students focussing on medicine have higher psychological health QoL and lower psychological distress (particularly in the area of depression) than non-medicine students in the same year. This may imply that the current programme is more suited to those aiming for a medical career than those heading for a non-medicine health-related vocation, although this proposition requires further investigation.

C. Implications for Learning and Wellbeing

There are two clear implications associated with these results.

First, the findings indicate that intending medical students are more likely to have good access to environmental resources. The elevation of environmental QoL scores for students intending to study medicine as compared to students in other disciplines is consistent with findings reported elsewhere (Collins et al., 1993; Henning et al., 2012; Ng et al., 2009), suggesting that these students likely have better access to the needed resources. Equity arguments regarding selection of medical students is a global concern and impacts areas related to communication, empathetic listening, and shared understandings between clinician and patient (Rao & Flores, 2007). The resources differential suggests a clear argument that students who prefer to study medicine may not equitably mirror the less-resourced population at large and thus may have difficulty relating to large proportions of society.  It is important to also emphasise that equity initiatives at the university in which this study was conducted are in place, although the complexity of the issues are multifaceted and may require further development (Curtis et al., 2015; Poole et al., 2009).

Second, the findings in this study clearly indicate that intending medical students are more likely to experience higher levels of psychological health QoL. Therefore, they are more likely to have a different wellbeing outcome than those students who aim for an alternative health sciences career, which may warrant investigating the rationale for placing students aiming for different health professions in one course of study.  An argument for placing all health-career students together is likely based on the idea of interprofessional learning (Salamonson et al., 2009). Some of the benefits include improved knowledge gains especially for those students preparing for non-medical courses and positive attitudinal shifts regarding each profession (Carpenter, 1995). The findings in this study revealed that one lecturer was able lecture to 1277 students at one time (using asynchronous technology), suggesting this is a cost effective teaching option. Nonetheless, the findings in this study do indicate that the implication of large classes (or ‘massification’) in reference to wellbeing gains requires further investigation (Hornsby & Osman, 2014).

It has been established that students preferring medicine to other health science options will lead to greater academic attainment. For example, Henning et al. (2017b) reported that those students preferring to pursue medicine attained a bioscience grade average of 6.77 which was significantly higher than the non-medical comparison cohort which attained a grade average of 4.42. Therefore, creating smaller, discipline-specific groups could create more equitable learning, as well as engendering more opportunities for staff-student engagement, more engagement with relevant content, and greater likelihood of motivation for learning (Hornsby & Osman, 2014). However, in the context of pre-medical courses, such as the BCY programme, even though there are clear differences in resulting grade achievement, more research is required to tease out the subtle learning and wellbeing interconnections and to further investigate whether one group is more disadvantaged in their learning than the other (in this case the non-medicine group).

A limitation of this study is the relatively low response rate, although this is comparable with other studies using online questionnaires (Nulty, 2008) and the participants engaged in this study had similar characteristics to the total population (Table 1). Moreover, the response rate is within the range considered acceptable for surveys of representatives of organisations (36%±13%) – a category arguably appropriate for students entering their respective professions (Baruch, 1999). A further limitation of the study is that the research design is cross-sectional suggesting that the links between variables cannot infer temporal causality. Lastly, we acknowledge that it was outside the scope of the study to fully explore some of the speculations raised in this paper, such as whether choice of course of study was influenced by either intrinsic or extrinsic factors (i.e., degree of autonomy).

V. CONCLUSION

The main implication of this study is the direct relationships between course preference (medicine versus other) and psychological health QoL and environmental QoL, as well as the further indirect or flow-on effects to psychological distress via the psychological QoL pathway. Thus, indicating that medical students are not as adversely impacted upon in terms of QoL and psychological distress compared with their non-medicine peers. Lastly, the study findings provide useful debate and scope for further research in the areas of pedagogy, educational equity, career aspiration, and financial cost related to interprofessional education and teaching in large student groups.

Notes on Contributors

Marcus A. Henning is an associate professor in the Centre for Medical and Health Sciences Education at the University of Auckland, New Zealand. MAH was involved in the conceptualisation of the idea, data curation, formal analysis, developing the design of the study, writing the original draft and ongoing editing.

Vanamali Joseph was, at the time of this study, a third year medical student studying in the Faculty of Medical and Health Sciences at the University of Auckland, New Zealand. VJ was involved in the conceptualisation of the idea, data curation, developing the design of the study and ongoing editing.

Roger J. Booth is an associate professor in molecular medicine and pathology, in the Faculty of Medical and Health Sciences at the University of Auckland, New Zealand. RB was involved in the conceptualisation of the idea, data curation, developing the design of the study and ongoing editing.

Christian U. Krägeloh is an associate professor in the Department of Psychology at AUT University, New Zealand. CUK was involved in the conceptualisation of the idea, formal analysis and ongoing editing.

Craig S. Webster is an associate professor in the Centre for Medical and Health Sciences Education at the University of Auckland, New Zealand. CSW was involved in the conceptualisation of the idea, data curation, formal analysis, developing the design of the study, writing the original draft and ongoing editing.

Ethical Approval

Ethics approval for the collection and use of data was obtained from the University of Auckland Human Participants Ethics Committee (Ref. 021985).

Data Availability

The anonymised data that supports the findings of this study are openly available in the Figshare repository, https://figshare.com/articles/dataset/_/14050463

Funding

No funding source required for this study.

Declaration of Interest

The authors declare no conflicts of interest.

References

Al Alwan, I., Al Kushi, M., Tamim, H., Magzoub, M., & Elzubeir, M. (2013). Health sciences and medical college preadmission criteria and prediction of in-course academic performance: A longitudinal cohort study. Advances in Health Sciences Education, 18(3), 427-438. https://doi.org/10.1007/s10459-012-9380-1

Arbuckle, J. L. (2017). IBM® SPSS® Amos™ 25 User’s Guide. https://usermanual.wiki/Document/IBMSPSSAmosUserGuide.2983928102/view

Baruch, Y. (1999). Response rate in academic studies- A comparative analysis. Human Relations, 52(4), 421-438. https://doi.org/10.1177/001872679905200401

Burris, J. L., Brechting, E. H., Salsman, J., & Carlson, C. R. (2009). Factors associated with the psychological well-being and distress of university students. Journal of American College Health, 57(5), 536-544. https://doi.org/10.3200/JACH.57.5.536-544

Byrne, B. M. (2010). Structural Equation Modeling with AMOS: Basic concepts, applications and programming (2nd ed.). Routledge.

Carpenter, J. (1995). Interprofessional education for medical and nursing students: Evaluation of a programme. Medical Education, 29(4), 265-272. https://doi.org/10.1111/j.1365-2923.1995.tb02847.x

Collins, J., Jones, J., & White, G. (1993). Demographic variables in Auckland medical students. New Zealand Medical Journal, 106(960), 306-308.

Curtis, E., Wikaire, E., Jiang, Y., McMillan, L., Loto, R., Airini., & Reid, P. (2015). A tertiary approach to improving equity in health: Quantitative analysis of the Māori and Pacific admission scheme (MAPAS) process, 2008–2012. International Journal for Equity in Health, 14(1), 1-15. https://doi.org/10.1186/s12939-015-0133-7

Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Academic Medicine, 81(4), 354-373. https://doi.org/10.1097/00001888-200604000-00009

Field, A. P. (2005). Discovering statistics using SPSS : And sex, drugs and rock’n’roll (2nd ed.). SAGE.

Google.com. (2015). Homepage. https://www.google.co.nz/forms/about/

Haist, S. A., Wilson, J. F., Elam, C. L., Blue, A. V., & Fosson, S. E. (2000). The effect of gender and age on medical school performance: An important interaction. Advances in Health Sciences Education: Theory and Practice, 5(3), 197-205. https://doi.org/10.1023/A:1009829611335

Henning, M. A., Joseph, V., Booth, R. J., Krageloh, C. U., & Webster, C. S. (2021). Biomedical students’ course preference and links with quality of life and psychological distress [Data set]. Figshare. https://figshare.com/articles/dataset/_/14050463

Henning, M. A., Krägeloh, C. U., Hawken, S. J., Zhao, Y., & Doherty, I. (2012). The quality of life of medical students studying in New Zealand: A comparison with non-medical students and a general population reference group. Teaching and Learning in Medicine, 24(4), 334-340.https://doi.org/10.1080/10401334.2012.715261

Henning, M. A., Krägeloh, C. U., Booth, R., Hill, E. M., Chen, J., & Webster, C. S. (2017a). Biomedical students in their first year of study: Factors explaining performance in a high stakes examination. Medical Science Educator, 27(4), 633–643.  https://doi.org/10.1007/s40670-017-0444-y

Henning, M. A., Krägeloh, C. U., Booth, R., Hill, E. M., Chen, J., & Webster, C. S. (2017b). Profiling potential medical students and exploring determinants of career choice. The Asia Pacific Scholar, 2(1), 7-15. https://doi.org/10.29060/TAPS.2017-2-1/OA1019

Henning, M. A., Krägeloh, C. U., Manalo, E., Doherty, I., Lamdin, R., & Hawken, S. J. (2013). Medical students in early clinical training and achievement motivation: Variations according to gender, enrolment status, and age. Medical Science Educator, 23(1), 6-15. https://doi.org/10.1007/BF03341798

Henning, M. A., Sollers, J., Strom, J. M., Hill, A. G., Lyndon, M. P., Cumin, D., & Hawken, S. J. (2014). Junior doctors in their first year: Mental health, quality of life, burnout and heart rate variability. Perspectives in Medical Education, 3, 136-143. https://doi.org/10.1007/s40037-013-0075-y

Hornsby, D. J., & Osman, R. (2014). Massification in higher education: Large classes and student learning. Higher Education Research & Development, 67(6), 711-719. https://doi.org/10.1007/s10734-014-9733-1

Hu, L.-t., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424-453. https://doi.org/10.1037/1082-989X.3.4.424

Katz, I., & Assor, A. (2007). When choice motivates and when it does not. Educational Psychology Review, 19(4), 429. https://doi.org/10.1007/s10648-006-9027-y

Krägeloh, C., Henning, M., Hawken, S., Zhao, Y., Shepherd, D., & Billington, R. (2011). Validation of the WHOQOL-BREF quality of life questionnaire for use with medical students. Eductaion for Health, 24(2), 545.

Krägeloh, C. U., Billington, D. R., Hsu, P. H.-C., Feng, X. J., Medvedev, O. N., Kersten, P., Landon, J., & Siegert, R. J. (2016). Ordinal-to-interval scale conversion tables and national items for the New Zealand version of the WHOQOL-BREF. PLOS ONE, 11(11), e0166065. https://doi.org/10.1371/journal.pone.0166065

Le, M. T. H., Tran, T. D., Holton, S., Nguyen, H. T., Wolfe, R., & Fisher, J. (2017). Reliability, convergent validity and factor structure of the DASS-21 in a sample of Vietnamese adolescents. PLOS ONE, 12(7), e0180557. https://doi.org/10.1371/journal.pone.0180557

Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335-343.

Medvedev, O. N., Krägeloh, C. U., Titkova, E. A., & Siegert, R. J. (2018). Rasch analysis and ordinal-to-interval conversion tables for the Depression, Anxiety and Stress Scale. Journal of Health 25(10–11), 1374–1383. https://doi.org/10.1177/1359105318755261

Naylor, R., Baik, C., & Arkoudis, S. (2018). Identifying attrition risk based on the first year experience. Higher Education Research & Development, 37(2), 328-342. https://doi.org/10.1080/07294360.2017.1370438

Ng, C. L., Tambyah, P. A., & Wong, C. Y. (2009). Cost of medical education, financial assistance and medical school demographics in Singapore. Singapore Medical Journal, 50(5), 462-467.

Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment and Evaluation in Higher Education, 33(3), 301-314. https://doi.org/10.1080/02602930701293231

Poole, P. J., Moriarty, H. J., Wearn, A. M., Wilkinson, T. J., & Weller, J. M. (2009). Medical student selection in New Zealand: looking to the future. New Zealand Medical Journal, 122(1306), 88-100.

Ramsbottom-Lucier, M., Johnson, M. M., & Elam, C. L. (1995). Age and gender differences in students’ preadmission qualifications and medical school performances. Academic Medicine, 70(3), 236-239. https://doi.org/10.1097/00001888-199503000-00016

Rao, V., & Flores, G. (2007). Why aren’t there more African-American physicians? A qualitative study and exploratory inquiry of African-American students’ perspectives on careers in medicine. Journal of the National Medical Association, 99(9), 986-993.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78. https://doi.org/10.1037/0003-066X.55.1.68

Salamonson, Y., Everett, B., Koch, J., Wilson, I., & Davidson, P. M. (2009). Learning strategies of first year nursing and medical students: A comparative study. International Journal of Nursing Studies, 46(12), 1541-1547. https://doi.org/10.1016/j.ijnurstu.2009.05.010

Szabó, M. (2010). The short version of the Depression Anxiety Stress Scales (DASS-21): Factor structure in a young adolescent sample. Journal of Adolescence, 33(1), 1-8. https://doi.org/10.1016/j.adolescence.2009.05.014

The NZ WHOQOL Group. (2017). WHOQOL tools. Auckland University of Technology. https://cpcr.aut.ac.nz/new-zealand-whoqol

The University of Auckland. (2019). Biomedical Science. https://www.auckland.ac.nz/en/study/study-options/find-a-study-option/biomedical-science.html

Tinto, V. (2009). Taking student retention seriously: Rethinking the first year of university. FYE curriculum design symposium. Queensland University of Technology, Australia.

Vansteenkiste, M., Lens, W., & Deci, E. L. (2006). Intrinsic versus extrinsic goal contents in self-determination theory: Another look at the quality of academic motivation. Educational Psychologist, 41(1), 19-31. https://doi.org/10.1207/s15326985ep4101_4

Verger, P., Combes, J.-B., Kovess-Masfety, V., Choquet, M., Guagliardo, V., Rouillon, F., & Peretti-Wattel, P. (2009). Psychological distress in first year university students: Socioeconomic and academic stressors, mastery and social support in young men and women. Social Psychiatry and Psychiatric Epidemiology, 44(8), 643-650. https://doi.org/10.1007/s00127-008-0486-y

WHOQoL Group. (1998). Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological Medicine, 28(3), 551-558. https://doi.org/10.1017/S0033291798006667

Wickramasinghe, D. P., Almeida, I. S., & Samarasekera, D. N. (2019). Depression and stressful life events among medical students during undergraduate career: Findings from a medical school in South Asia. The Asia Pacific Scholar, 4(1), 42-47. https://doi.org/10.29060/TAPS.2019-4-1/OA2025

Yıldırım, A., Boysan, M., & Kefeli, M. C. (2018). Psychometric properties of the Turkish version of the Depression Anxiety Stress Scale-21 (DASS-21). British Journal of Guidance & Counselling, 46(5), 582-595. https://doi.org/10.1080/03069885.2018.1442558

Yusoff, M. S. B., Rahim, A. F. A., Baba, A. A., Ismail, S. B., Pa, M. N. M., & Esa, A. R. (2013). Prevalence and associated factors of stress, anxiety and depression among prospective medical students. Asian Journal of Psychiatry, 6(2), 128-133. https://doi.org/10.1016/j.ajp.2012.09.012

*Marcus A. Henning
The University of Auckland,
Private Bag 92019,
Auckland 1142, New Zealand
Tel: 0064 923 7392
Email: m.henning@auckland.ac.nz