Online medical interview training in preclinical medical education: Educational outcomes comparable to face-to-face training

Submitted: 24 October 2024
Accepted: 5 July 2025
Published online: 7 October, TAPS 2025, 10(4), 26-34
https://doi.org/10.29060/TAPS.2025-10-4/OA3552

Shoko Horita1,2, Masashi Izumiya2, Satoshi Kondo2,3,4, Junki Mizumoto2,5,6, Hiroko Mori6,7 & Masato Eto2

1Department of Medical Education, School of Medicine, Teikyo University, Itabashi-ku, Tokyo, Japan; 2Department of Medical Education Studies, International Research Centre for Medical Education, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan; 3Department of Medical Education, Graduate School of Medicine, University of Toyama, Toyama, Japan; 4Center for Medical Education and Career Development, Graduate School of Medicine, University of Toyama, Toyama, Japan; 5Department of Family Practice, Ehime Seikyo Hospital, Matsuyama, Ehime, Japan; 6Center for General Medicine Education, School of Medicine, Keio University, Shinjuku, Tokyo, Japan; 7Professional Development Centre, The University of Tokyo Hospital, Bunkyo-ku, Tokyo, Japan

Abstract

Introduction: Conventionally, face-to-face education has been prevalent in medical education because it can help medical students learn interpersonal skills, including medical interviews and physical examination. However, because of the coronavirus disease 2019 pandemic, face-to-face education was suspended to prevent the spread of the infection. As face-to-face classes in Japan were discontinued when the pandemic began in the spring of 2020, we developed an online education program to develop medical interview skills. We were interested in determining the educational outcomes between face-to-face and online medical interview classes. Therefore, we compared them before and after the pandemic.

Methods: Fourth-year students of the University of Tokyo Medical School took medical interview classes. Under consent, the score of the medical interview area of the preclinical clerkship, Objective Structured Clinical Examination (OSCE), as a high-stakes examination, which falls at the top level of the Kirkpatrick’s model, was compared by year or before and after the pandemic.

Results: The online group showed higher item-wise scores of the medical interview of the preclinical clerkship OSCE than the face-to-face group. In terms of the global score, no significant difference was observed. In the computer-based test (CBT), the online group had higher scores compared with the face-to-face group.

Conclusion: The educational outcomes of online medical interview classes were not inferior to those of conventional face-to-face classes, as revealed by high-stakes examination preclinical clerkship OSCE. Similar to face-to-face education, online education is a viable option for developing interpersonal skills.

Keywords:          COVID-19 Pandemic, Medical Interview, OSCE, Educational Outcome, Online Education, Interpersonal Skills, Communication Skills

Practice Highlights

  • Due to the COVID-19 pandemic, we shifted medical interview classes from face-to-face to online.
  • The online group had interview global OSCE scores non-inferior to those of the face-to-face group.
  • The online group had higher interview elementary OSCE scores than the face-to-face group.

I. INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic severely restricted face-to-face teaching and affected almost all levels and fields of education, including undergraduate preclinical medical education (Bastos et al., 2022; Crawford et al., 2020). Moreover, it resulted in drastic changes in medical education. Globally, face-to-face learning was forcibly discontinued as part of infection control. Thus, to continue medical education, online or remote learning was rapidly introduced (Daniel et al., 2021; Gordon et al., 2020). Various instrumental trans communication devices, including video conferencing tools, simulation, virtual reality, and augmented reality, were used to facilitate online learning. However, this rather hasty shift from face-to-face to online learning brought some confusion into the field of medical education. In the UK, Dost et al. (2020) reported that medical students were unsatisfied with online classes compared with face-to-face classes.

Globally, tele-education is increasingly being encouraged around the world (American Medical Association, 2016). In the field of medical interview (Budakoğlu et al., 2021; Hammersley et al., 2019; Zaccariah et al., 2022), telemedicine is gradually becoming common, showing favourable results. However, because of technical problems, tele-education did not spread smoothly (Zaccariah et al., 2022). Additionally, the educational outcomes of both strategies have not been satisfactorily studied (Khamees et al., 2022). Recently, some reports showing that the educational outcome of online classes are equal or more effective than traditional face-to-face education, however, they are restricted mainly in knowledge-based education (Alshaibani et al., 2023; Basuodan, 2024; Saad et al., 2023). Furthermore, few studies have compared high-stakes examination, including the Objective Structured Clinical Examination (OSCE), and no study has compared the educational results between face-to-face classes and tele-education (online) using the top level of Kirkpatrick’s model (Kirkpatrick, 1996).

The OSCE (Harden et al., 1975) has been widely accepted as a form to assess clinical performance in medical education. Currently, OSCEs are used worldwide to appraise medical students’ communication and clinical skills. Various educational methods have been evaluated using OSCE as one of the indicators of educational outcomes (Guetterman et al., 2019). In Japan, passing the preclinical clerkship (pre-CC) OSCE has become legally obligatory as one of the elements for promotion to the CC course since the spring of 2023. In 2023, the pre-CC OSCE in Japan is conducted in at least eight areas, which are medical interview, “Basic Clinical Procedure”, “Basic Life Support”, and physical examinations of “head and neck”, “chest”, “vital signs”, “abdomen”, and “neurological examinations”.

In the present study, we aimed to determine the educational outcomes between face-to-face and online medical interview classes. We provided medical interview classes to fourth-year medical students before taking the pre-CC OSCE, face-to-face classes before 2019, and tele-education (online) after 2020. We decided to conduct research in medical interview, other than the other areas of the pre-CC OSCE, because of the importance of the medical interview as the basis of medical practice. Moreover, it was inevitable that the medical interview classes had to be implemented as online classes to protect the simulated patients form the risk of infection, which was another main reason for selecting medical interview for this research. In another point of view, medical interview classes were able to implement via online. As mentioned above, no prior studies have compared face-to-face and online medical interview training using both high-stakes OSCE score and Kirkpatrick’s top-level outcomes, our study would have significant importance.

II. METHOD

A. Participants

This study was approved in 2021 by the ethics committee of the University of Tokyo (UTokyo) Faculty of Medicine (Approval No. 2021005NI). All methods were performed in accordance with the relevant guidelines and regulations. Moreover, the data of students who provided consent for the secondary use of their data (Approval No. 11763) in another research approved in 2017 were included.

B. Sample Population

Students in the UTokyo Faculty of Medicine were asked if they were willing to participate in “A Study of the Educational Effectiveness of Online “Medical interviewing Practice” in the post-class reflection questionnaire of the “Online medical interview classes or the waiting period after the pre-CC OSCE. Out of 229 students (2021 and 2022), 87 students participated in this study. A summary of the annual participants is shown in Appendix 1. In early 2020 almost all the classes in UTokyo were stopped due to the COVID-19 pandemic, which made it difficult to contact students face-to-face and to obtain participants in the previous research (Approval No. 11763); and as this research started in 2021, it was practically difficult to obtain consent to participate in this study in 2020. In 2020 the online medical interview classes have just been launched, which significantly improved in 2021. Hence, we thought that it would be better to exclude the small participants of 2020 from the analysis to keep the validity of this study.

C. Details of Medical Interview Classes

Before 2019, the medical interview classes were performed as follows: Early in their fourth year, students joined classes introducing the outline of medical interview. A few days before the class, students watched an instructional video of a medical interview performed by the Common Achievement Tests Organization (CATO) (2005) in Japan. Afterward, students in a group of eight to nine faced the simulated patient in a classroom in the UTokyo and performed a medical interview roleplay. Thereafter, feedback about the technical factor of the medical interview as well as rapport status and nonverbal communications such as faces and gesture was provided by the students themselves, other students, simulated patients, and teachers. Since 2020, most face-to-face classrooms, including those in the present study, were closed because of the COVID-19 pandemic and were replaced with online classes. The present face-to-face class was also held online with the simulated patients and teachers using Zoom(R). Using the “Close-Up” function of Zoom(R), the simulated patient and student were faced with each other, whereas other participants (e.g., other students, other simulated patients, and the teacher) were not on the television (Appendix 2). After the roleplay was over, all students and the teacher came back on the television and provided feedback to the student, similar to face-to-face classes. Moreover, the class was recorded using the function of Zoom(R) and provided to students exclusively for review. After the class students reflected on the reflection sheet (until 2019) or the Learning Management System (from 2020) which was reviewed and commented on by teachers. The contents of the reflections were used for this study to investigate the impressions of the students.

D. Pre-CC OSCE and Computer-Based Test (CBT)

In Japan, medical students usually take the pre-CC OSCE in the fourth year, prior to the two-year CC course. Before 2022, the minimum assessment factors were medical interview, physical examinations (including head and neck, chest, abdomen, neurological examinations), basic clinical procedure, and basic life support. The examinations were administered by CATO. The evaluation criteria are not publicly available because of CATO policy. Two scores are used in the evaluation: global score (GS) which means the evaluation as a total performance and item-wise score (IS) which means scores by checklist. Before 2023, the borderline was set by each institute. At least one certified evaluator per area was responsible, and each evaluator was a faculty member. Moreover, CATO sent at least one external evaluator per area and an external supervisor. After each performance, each examinee was evaluated by two or three evaluators per room. The pre-CC OSCE is one of the examinations that students must pass to proceed to the CC course.

Aside from the pre-CC OSCE, students must also pass the CBT. The CBT corresponds to the assessment of medical knowledge prior to the CC (Horita et al., 2021). In 2023, the pre-CC OSCE and CBT have been made official, and students must pass both examinations before they can take the national board examination in Japan.

E. Data Analysis

The pre-CC OSCE scores were analysed using R, Rstudio, JMP version17.0 (SAS Institute, N.C., USA) and Microsoft Excel (Microsoft Corporation, W.A., USA). Non-paired T test, Mann-Whitney U test, and Steel-Dwass test were used respectively, for parametric or non-parametric comparisons.

III. RESULTS

A. Year-to-Year Comparison of the Pre-CC OSCE Results in the Medical Interview area and CBT Results

First, we compared the year-to-year results of the pre-CC OSCE in the medical interview area. Table 1 and Figure 1 shows a statistical summary of the pre-CC OSCE scores in 2022, 2021, 2019, and 2018. The results of non-parametric tests revealed that the p-values in the IS between 2022 and 2019, 2022 and 2018, and 2021 and 2018 were below 0.05, whereas no significant difference was observed in the GS.

Year

IS/GS

Average

SD

SE

Bottom 95

Upper 95

2022

IS

85.67

9.19

1.18

83.32

88.02

GS

4.41

0.68

0.09

4.24

4.58

2021

IS

82.69

9.71

1.90

78.77

86.61

GS

4.62

0.75

0.15

4.31

4.92

2019

IS

79.04

10.19

1.07

76.92

81.16

GS

4.33

0.89

0.09

4.14

4.51

2018

IS

73.63

9.94

1.10

71.43

75.83

GS

4.26

0.79

0.09

4.09

4.44

Table 1. Averages of IS and GS of the medical interview area per the pre-CC OSCE implementation year. IS, item-wise score; GS, global score; SD, standard deviation; SE, standard error

Figure 1. Average of IS and GS. The error bar shows standard error

Year

Average

SD

SE

Bottom 95

Upper 95

2022

566.62

121.03

15.50

535.63

597.62

2021

576.07

106.64

20.52

533.89

618.26

2019

565.02

119.91

12.71

539.76

590.28

2018

529.72

116.34

12.93

503.99

555.44

Table 2. Year-by-year score distribution of CBT (IRT score)

B. Comparison Before and After the Pandemic

The medical interview classes were held face-to-face before the pandemic (2018 and 2019) and online after the pandemic (2021 and 2022). We compared the results of pre-CC OSCE medical interview and CBT before and after the pandemic. A summary of the results is shown in Table 3 and Figure 2. The results of statistical analyses revealed a significant difference in the medical interview IS and CBT between the face-to-face group and the online group (p < 0.001 and 0.032 respectively). However, no significant difference in GS was observed.

 

Group

Number

Average

SE

Bottom 95

Upper 95

OSCE (medical interview)

IS

F-to-F

164

76.18

0.79

74.62

77.74

online

85

84.71

1.10

82.54

86.87

OSCE (medical interview)

GS

F-to-F

164

4.28

0.06

4.15

4.40

online

85

4.47

0.09

4.30

4.64

CBT (IRT score)

F-to-F

162

546.5

9.39

528.1

565.0

online

86

569.7

12.9

544.3

595.1

Table 3. Comparison of pre-CC OSCE (IS and GS respectively) and CBT results between the face-to-face (F-to-F) group and the online group

Figure 2. Comparison of pre-CC OSCE (IS and GS respectively) and CBT results between the face-to-face (F-to-F) group and the online group. The error bar shows standard error.

IV. DISCUSSION

We found no significant negative effects in some of the important educational outcomes in medical students’ scores of the medical interview due to online education caused by the COVID-19 pandemic. The quality of the medical interview after the emergence of the pandemic was no less than that before the pandemic. The same could be said for other indicators, including the CBT and other areas of the OSCE (data not shown).

One of the reasons why the scores of the online classes were not inferior to those of face-to-face classes might be because of the availability of each student to review the video recordings. We provided each student with a recording of their own performance in the class for self-review, which was not always provided in face-to-face classes. We also provided students with feedback from the teacher and other students during online classes. This is consistent with the findings of a previous study, which found that video reviewing of the OSCE performance is effective (Mookherjee et al., 2019). Moreover, the students accepted online classes well, and their motivation for learning was not affected despite the lack of face-to-face communication with simulated patients. During the reflection, some students noted that “I learned a lot in this class, though the class was held online” and that “I thought that online classes are not so bad” (data not shown). There were almost no complaints regarding online classes. We guess that in the environment that the face-to-face classes were restricted and the students experienced suspended classes the students felt satisfied for joining the classes even online. Further investigation will be needed regarding this point.

Recently, Khamees et al. (2022) pointed out the lack of control groups and poor transferability in numerous publications due to singularity of institution, department, and program. In the present study, the marks of students on high-stakes examinations before the pandemic were used as a comparison between face-to-face and online classes. Some studies have revealed that there are no significant differences in educational outcomes between face-to-face and online classes in basic medicine (Omole et al., 2023), and pharmacological education (Aoe et al., 2023). However, when it comes to high-stakes examinations, it remains unclear whether online education is not inferior to face-to-face education. Saad et al (2023) have showed that in some areas (clinical reasoning and history taking) of pre-clinical OSCE, students showed no less than comparable results, arguing that these skills are amenable to online learning in a medical school in Australia. Their results in some areas like medical interview in Japanese OSCE support our results. However, in their study, it is not clear about the details of the OSCE assessment, whether the assessment is by item-wise or global. In recent years, the pre-CC OSCE results have been recognized as an important educational outcome also for educational institutions (Hirsh et al., 2012). Our result, the educational outcome in the high-stakes examination, can be considered to fall in the top tier, the result, in the Kirkpatrick’s four-level model (Kirkpatrick, 1996). Moreover, our research is unique and important as few studies have directly compared the educational outcomes between face-to-face classes and online classes in high-stakes examination.

In Japan, the Medical Practitioners Act was revised in 2023, allowing medical students to perform some medical procedures under the supervision of a teaching physician after passing the pre-CC OSCE and CBT. This change also made the pre-CC OSCE a requirement for the national board examination. Hence, the pre-CC OSCE in Japan has become even more important, as much the responsibility for the education even greater. Our results show that online classes can contribute to the practice of “Medical interviews”.

It must be noted that online classes are not a complete alternative to face-to-face classes. Many studies have indicated that online education has some negative aspects (e.g., the need for infrastructure and devices, high cost, lack of personal interaction, etc.) (Arja et al., 2022; Mortazavi et al., 2021; Shaiba et al., 2023). One of the most significant elements that are difficult to teach in online classes is nonverbal communication. However, as Ishikawa et al. (2010) reported, although students are capable of understanding nonverbal communication despite struggling to change their performance through educational intervention, it is well recognized that nonverbal communication is difficult to teach even in face-to-face classes. Additionally, when it comes to procedural skills such as venipuncture, the educational outcomes in the online learning group were inferior to that of face-to-face learning group and students also felt that they were not taught satisfactorily (Dost et al., 2020; Saad et al., 2023). We should keep in mind that online education does not fully replace face-to-face education.

We saw a lack of significant differences in GS, both in year-by-year comparison and comparison between face-to-face and online groups. Although the tasks allocated to each university by CATO differ every year, the checkpoints are essentially common in quite a few areas; so, a comparison was made for both year by year and before and after the pandemic. GS usually reflects holistic assessment, which is difficult to produce results via technical education, whereas it might be easier for learners and teachers to deal with item-wise assessment (Govaerts et al., 2011; Jonsson & Svingby, 2007; Sadler, 2009). Moreover, in online classes, we used a checklist of the students’ performance (not shown to the students, but comments were given according to the checklist), which might have contributed to the improvement of IS. As to CBT, it is standardized by the accumulated examinations and the Item Response Theory and is assessed basically by knowledge base. The educational strategies that mainly should impact on the assessment of CBT, based on the lecture, have not changed before and after the pandemic, in the face-to-face classes or online classes. During the pandemic, the extra-curricular activities of the students were restricted, and several articles argue that self-studying time of the students increased (Barton et al., 2021; Guluma & Brandl, 2023). These might have contributed to the smaller elevation of CBT-IRT than the IS of pre-CC OSCE.

We need to take into consideration the confounding of several factors such as the curriculum changes, instructor training, student characteristics and students’ self-study time. While online classes have been a change in the curriculum, the rest remains unchanged. The instructors and students needed to become familiar with online classes, but there was no change in the educational goals of the class itself. Additionally, in the first year of the pandemic in 2020, we were unable to get enough data and the online class itself was implemented as “being built”. By 2021 and 2022, the class was almost stable. However, getting used to online classes of the instructors and students could be a confounding factor. The class tool (Zoom®) was continuously improved, which might be a minor confounding factor. Additionally, the students might have had excellent ITC skills, which might also be a confounding factor.

Some frameworks describing the evidence of online medical education outcome might contribute to generalizing our results (Martinengo et al., 2024; McGee et al., 2024; Wilcha, 2020). Needless to say, there are confrontations regarding the limitations of these generalizations, pointing out the context-depending factors, high heterogeneity among studies and “The Covid-19 Effects” (Abdull Mutalib et al., 2022; Liu et al., 2016; Martinengo et al., 2024; McGee et al., 2024). However, these frameworks will be applicable in generalizing our results; although there are some potential confounding factors such as the students, the instructors, the educational resources and the “pandemic era” itself, the online medical interview education could be an effective educational curriculum for educating medical interview skills as well as some interpersonal skills.

A. Limitations

One of the limitations of this study is that it was performed in a single institution. Hence, the generalizability of this study may be lower than that of multi-institutional studies. However, not much variation exists in the nature of students and in the educational curriculum they experience. Of course, to make the evidence more robust and further validate, multi-centred or multi-institutional studies are still needed. At the same time, these factors should not be too disparate as it is very difficult to find a suitable population for these factors. In this regard, our participants and classes can be considered as a reasonable population. 

Another limitation of this study is the number of participants. In 2020, we could not obtain enough participants because of the pandemic. After the pandemic, our students and staff shifted to online classes, and the number of face-to-face classes decreased. In 2021 and 2022, we decided to obtain consent for participation in this study in the waiting time after face-to-face OSCE examination as it was difficult to obtain consent only during online classes. In this context, the participants may have a positive view of various aspects of student life including studies, which may be a potential bias of sample population. Additionally, the waiting time after the OSCE examination was short for some students, which might have made it difficult to think about understanding the concept of this research and whether consent should be given.

In this study, qualitative analyses investigating if the students were positive about the classes are limited to some extent. During the pandemic, the psychological situation and the learning behaviour of the students might have differed from that of before the pandemic. To investigate this aspect, qualitative studies will be needed. 

Moreover, the evaluation criteria of the pre-CC OSCE are not open to the public due to CATO policy. This probably leads to a lack of transparency of the evaluation, causing another limitation of this study. However, Japanese OSCE evaluation criteria is similar to CANMED’s OSCE checklist (Kassam et al., 2016), which will support the validity of the results of Japanese OSCE and our results.

Finally, this study retrospectively compared the educational effect between students before and after the pandemic, which may limit the causal inferences of educational outcome effects of face-to-face versus online in medical interview OSCE. A randomized controlled trial will be needed to verify the results obtained in this study.

V. CONCLUSION

Because of the COVID-19 pandemic, we were forced to change our medical interview classes from face-to-face to be online. However, in high-stakes examinations such as the pre-CC OSCE and CBT, the results of the online group were not inferior to those of the face-to-face group. We consider this result extremely important because we directly compared the educational outcomes of high-stakes examinations between online and face-to-face groups who took the same medical interview classes, and because this evaluation falls in the top level of Kirkpatrick’s model. Our results suggest that online education provides a viable option in teaching interpersonal skills and support the integration of online medical interview training into preclinical curricula, particularly in resource-constrained settings. Randomized controlled trials and multi-institutional studies are needed to further validate our results.

Notes on Contributors

SH and ME conducted the whole research.

SH, MI, SK, JM, HM, and ME performed the classes and collected the data.

SH performed data analyses.

SH, MI, SK, JM, HM, and ME contributed to writing the manuscript.

Ethical Approval

This study was approved by the ethics committee of the UTokyo Faculty of Medicine (Approval No. 2021005NI). All methods were performed in accordance with the relevant guidelines and regulations. Moreover, the data of students who provided consent to the secondary use of their data in another research, given by the ethics committee of the UTokyo Faculty of Medicine (Approval No. 11763), were included.

Data Availability

The data in this study are not publicly available because of confidentiality agreements with the participants, conditions obligating CATO, and confidential nature of the data.

Acknowledgement

We thank the students of the UTokyo Medical School who participated in this study. We also thank the UTokyo Staff for their cooperation.

Funding

This study was funded by JSPS KAKENHI Grant Number 24K06092 and ACRO incubation grants of Teikyo University.

Declaration of Interest

The authors have no potential conflicts to disclose.

References

Abdull Mutalib, A. A., Md. Akim, A., & Jaafar, M. H. (2022). A systematic review of health sciences students’ online learning during the COVID-19 pandemic. BMC Medical Education22(1), 524. https://doi.org/10.1186/s12909-022-03579-1

Alshaibani, T., Almarabheh, A., Jaradat, A., & Deifalla, A. (2023). Comparing online and face-to-face performance in scientific courses: A retrospective comparative gender study of Year-1 students. Advances in Medical Education and Practice, 14, 1119–1127. https://doi.org/10.2147/amep.s408791

American Medical Association. (2016, June 15). AMA encourages telemedicine training for medical students, residents. AMA Press Releases.

Aoe, M., Esaki, S., Ikejiri, M., Ito, T., Nagai, K., Hatsuda, Y., Hirokawa, Y., Yasuhara, T., Kenzaka, T., & Nishinaka, T. (2023). Impact of different attitudes toward face-to-face and online classes on learning outcomes in Japan. Pharmacy, 11(1), 16. https://doi.org/10.3390/pharmacy11010016

Arja, S. B., Fatteh, S., Nandennagari, S., Pemma, S. S. K., Ponnusamy, K., & Arja, S. B. (2022). Is emergency remote (online) teaching in the first two years of medical school during the COVID-19 pandemic serving the purpose? Advances in Medical Education and Practice, 13, 199–211. https://doi.org/10.2147/amep.s352599

Barton, J., Rallis, K. S., Corrigan, A. E., Hubbard, E., Round, A., Portone, G., Kuri, A., Tran, T., Phuah, Y. Z., Knight, K., & Round, J. (2021). Medical students’ pattern of self-directed learning prior to and during the coronavirus disease 2019 pandemic period and its implications for Free Open Access Meducation within the United Kingdom. Journal of Educational Evaluation for Health Professions, 18, 5. https://doi.org/10.3352/jeehp.2021.18.5

Bastos, R. A., Carvalho, D. R. D. S., Brandão, C. F. S., Bergamasco, E. C., Sandars, J., & Cecilio-Fernandes, D. (2021). Solutions, enablers and barriers to online learning in clinical medical education during the first year of the COVID-19 pandemic: A rapid review. Medical Teacher, 44(2), 187–195. https://doi.org/10.1080/0142159x.2021.1973979

Basuodan, R. (2024). Comparisons of the academic performance of Medical and Health-Sciences students related to three learning methods: A cross-sectional study. Advances in Medical Education and Practice, 15, 1339–1347. https://doi.org/10.2147/amep.s493782

Budakoğlu, I. İ., Sayılır, M. Ü., Kıyak, Y. S., Coşkun, Ö., & Kula, S. (2021). Telemedicine curriculum in undergraduate medical education: a systematic search and review. Health and Technology, 11(4), 773–781. https://doi.org/10.1007/s12553-021-00559-1

Common Achievement Tests Organization (CATO). (2005). Common Achievement Tests Organization. https://www.cato.or.jp/index.html

Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., Magni, P. A., & Lam, S. (2020). COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, 3(1). https://doi.org/10.37074/jalt.2020.3.1.7

Daniel, M., Gordon, M., Patricio, M., Hider, A., Pawlik, C., Bhagdev, R., Ahmad, S., Alston, S., Park, S., Pawlikowska, T., Rees, E., Doyle, A. J., Pammi, M., Thammasitboon, S., Haas, M., Peterson, W., Lew, M., Khamees, D., Spadafore, M., … Stojan, J. (2021). An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME Guide No. 64. Medical Teacher, 43(3), 253–271. https://doi.org/10.1080/0142159x.2020.1864310

Dost, S., Hossain, A., Shehab, M., Abdelwahed, A., & Al-Nusair, L. (2020). Perceptions of medical students towards online teaching during the COVID-19 pandemic: A national cross-sectional survey of 2721 UK medical students. BMJ Open, 10(11), e042378. https://doi.org/10.1136/bmjopen-2020-042378

Gordon, M., Patricio, M., Horne, L., Muston, A., Alston, S. R., Pammi, M., Thammasitboon, S., Park, S., Pawlikowska, T., Rees, E. L., Doyle, A. J., & Daniel, M. (2020). Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher, 42(11), 1202–1215. https://doi.org/10.1080/0142159x.2020.1807484

Govaerts, M. J. B., Schuwirth, L. W. T., Van Der Vleuten, C. P. M., & Muijtjens, A. M. M. (2010). Workplace-based assessment: Effects of rater expertise. Advances in Health Sciences Education, 16(2), 151–165. https://doi.org/10.1007/s10459-010-9250-7  

Guetterman, T. C., Sakakibara, R., Baireddy, S., Kron, F. W., Scerbo, M. W., Cleary, J. F., & Fetters, M. D. (2019). Medical students’ experiences and outcomes using a virtual human simulation to improve communication skills: Mixed methods study. Journal of Medical Internet Research, 21(11), e15459. https://doi.org/10.2196/15459

Guluma, K. Z., & Brandl, K. (2022). Virtual learning allows for adaptation of study strategies in a cohort of U.S. medical students. Medical Teacher, 45(1), 89–96. https://doi.org/10.1080/0142159x.2022.2105690

Hammersley, V., Donaghy, E., Parker, R., McNeilly, H., Atherton, H., Bikker, A., Campbell, J., & McKinstry, B. (2019). Comparing the content and quality of video, telephone, and face-to-face consultations: A non-randomised, quasi-experimental, exploratory study in UK primary care. British Journal of General Practice, 69(686), e595–e604. https://doi.org/10.3399/bjgp19x704573

Harden, R. M., Stevenson, M., Downie, W. W., & Wilson, G. M. (1975). Assessment of clinical competence using objective structured examination. BMJ, 1(5955), 447–451. https://doi.org/10.1136/bmj.1.5955.447

Hirsh, D., Gaufberg, E., Ogur, B., Cohen, P., Krupat, E., Cox, M., Pelletier, S., & Bor, D. (2012). Educational outcomes of the Harvard Medical School–Cambridge integrated clerkship. Academic Medicine, 87(5), 643–650. https://doi.org/10.1097/acm.0b013e31824d9821

Horita, S., Park, Y., Son, D., & Eto, M. (2021). Computer-based test (CBT) and OSCE scores predict residency matching and National Board assessment results in Japan. BMC Medical Education, 21(1). https://doi.org/10.1186/s12909-021-02520-2

Ishikawa, H., Hashimoto, H., Kinoshita, M., & Yano, E. (2010). Can nonverbal communication skills be taught? Medical Teacher, 32(10), 860–863. https://doi.org/10.3109/01421591003728211

Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity, and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002

Kassam, A., Cowan, M., & Donnon, T. (2016). An objective structured clinical exam to measure intrinsic CanMEDS roles. Medical Education Online, 21(1), 31085. https://doi.org/10.3402/meo.v21.31085

Khamees, D., Peterson, W., Patricio, M., Pawlikowska, T., Commissaris, C., Austin, A., Davis, M., Spadafore, M., Griffith, M., Hider, A., Pawlik, C., Stojan, J., Grafton-Clarke, C., Uraiby, H., Thammasitboon, S., Gordon, M., & Daniel, M. (2022). Remote learning developments in postgraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 71. Medical Teacher, 44(5), 466–485. https://doi.org/10.1080/0142159x.2022.2040732

Kirkpatrick, D. (1996). Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick’s Four-Level model. Training & Development, 50(1), 54–59. https://eric.ed.gov/?id=EJ515660

Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., & Yan, W. (2016). The effectiveness of blended learning in health professions: Systematic review and meta-analysis. Journal of Medical Internet Research, 18(1), e2. https://doi.org/10.2196/jmir.4807

 Martinengo, L., Ng, M. S. P., De Rong Ng, T., Ang, Y., Jabir, A. I., Kyaw, B. M., & Car, L. T. (2024). Spaced digital education for health professionals: A systematic review and meta-analysis (Preprint). Journal of Medical Internet Research, 26, e57760. https://doi.org/10.2196/57760

McGee, R. G., Wark, S., Mwangi, F., Drovandi, A., Alele, F., & Malau-Aduli, B. S. (2024). Digital learning of clinical skills and its impact on medical students’ academic performance: A systematic review. BMC Medical Education, 24(1). https://doi.org/10.1186/s12909-024-06471-2

Mookherjee, S., Strujik, J., Cunningham, M., Kaplan, E., & Çoruh, B. (2018). Independent and mentored video review of OSCEs. The Clinical Teacher, 16(1), 23–29. https://doi.org/10.1111/tct.12755

Mortazavi, F., Salehabadi, R., Sharifzadeh, M., & Ghardashi, F. (2021). Students’ perspectives on the virtual teaching challenges in the COVID-19 pandemic. Journal of Education and Health Promotion, 10(1), 59. https://doi.org/10.4103/jehp.jehp_861_20

Omole, A. E., Villamil, M. E., & Amiralli, H. (2023). Medical education during COVID-19 pandemic: A comparative effectiveness study of face-to-face traditional learning versus online digital education of basic sciences for medical students. Cureus. https://doi.org/10.7759/cureus.35837

Saad, S., Richmond, C., King, D., Jones, C., & Malau-Aduli, B. (2023). The impact of pandemic disruptions on clinical skills learning for pre-clinical medical students: Implications for future educational designs. BMC Medical Education, 23(1). https://doi.org/10.1186/s12909-023-04351-9 

Sadler, D. R. (2008). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159–179. https://doi.org/10.1080/02602930801956059

Shaiba, H., John, M., & Meshoul, S. (2022). Female Saudi College students’ e-learning experience amidst COVID-19 pandemic: An investigation and analysis. Heliyon, 9(1), e12768. https://doi.org/10.1016/j.heliyon.2022.e12768

Wilcha, R. (2020). Effectiveness of virtual medical teaching during the COVID-19 Crisis: Systematic review. JMIR Medical Education, 6(2), e20963. https://doi.org/10.2196/20963

Zaccariah, Z. R., Irvine, A. W., & Lefroy, J. E. (2022). Feasibility study of student telehealth interviews. The Clinical Teacher, 19(4), 308–315. https://doi.org/10.1111/tct.13490

*Shoko Horita
2-11-1, Kaga, Itabashi-ku,
Tokyo 173-8605, Japan
Email: horitas-tky@umin.ac.jp

Submitted: 30 December 2024
Accepted: 5 July 2025
Published online: 7 October, TAPS 2025, 10(4), 63-72
https://doi.org/10.29060/TAPS.2025-10-4/OA3777

Chollada Sorasak1, Worayuth Nak-Ai2, Choosak Yuennan3 & Mansuang Wongsapai1

1Intercountry Centre for Oral Health, Department of Health, Thailand; 2Sirindhorn College of Public Health Chonburi, Praboromarajchanok Institute, Thailand; 3Boromarajonani College of Nursing Chiang Mai, Praboromarajchanok Institute, Thailand

Abstract

Introduction: Nutrition literacy represents a critical determinant of oral health outcomes. Guided by Social Cognitive Theory and the Nutrition Literacy Skills Framework, this study evaluated the implementation and effectiveness of a nutrition literacy programme for oral health promotion among village health volunteers (VHVs), key implementers in Thailand’s healthcare system, during January to December 2024.

Methods: A convergent parallel mixed-methods design was employed to address existing methodological gaps in nutrition literacy research. The quantitative component comprised a cross-sectional survey (N=60 VHVs trained in January 2024) and clinical outcome monitoring via electronic health records. The qualitative strand involved a multi-case study approach with purposive sampling (n=20) through in-depth interviews. Data collection occurred at 6-month post-implementation (July 2024), with clinical monitoring through December 2024. Analysis integrated descriptive and inferential statistics with thematic analysis.

Results: Post-implementation analysis revealed significantly enhanced nutrition literacy skills (M=4.14, SD=0.414), with notable improvements in communication (M=4.74, SD=0.511) and implementation (M=4.21, SD=0.440). All six nutrition literacy domains showed strong correlations (r=0.712-0.868, p<.01), supporting the framework’s interconnected nature. Clinical outcomes improved significantly: oral health check-up rates increased from 1.41% to 2.61% (p<.05), and functional teeth retention rose from 87.36% to 92.72% (p<.01). Qualitative findings revealed adaptive knowledge transfer methods and context-specific implementation strategies influenced by community readiness.

Conclusion: Through comprehensive mixed-methods evaluation, the 12-month implementation data demonstrated significant improvements in both VHVs’ nutrition literacy skills and clinical oral health outcomes. Success factors included theoretically-grounded implementation strategies and stakeholder engagement in resource-limited settings.

Keywords:         Convergent Parallel, Health Literacy, Mixed Methods, Nutrition, Oral Health, Thailand, Village Health Volunteer

Practice Highlights

  • Nutrition literacy among VHVs significantly improved across all six key domains.
  • Oral health check-up rates increased from 41% to 2.61% post-programme implementation.
  • Functional teeth retention rose from 36% to 92.72% over the 12-month period.
  • VHVs used context-specific strategies for community-based nutrition education.

I. INTRODUCTION

Oral health is fundamentally linked to nutrition and dietary behaviours, yet nutritional factors affecting oral health remain a significant public health challenge worldwide, particularly in low- and middle-income countries (Peres et al., 2019; Watt et al., 2019). In Thailand, the high prevalence of dental caries and periodontal diseases related to dietary habits (Chaianant et al., 2022), underscores the urgent need for effective nutrition education strategies for oral health promotion.

Understanding the relationship between nutrition literacy and oral health behaviours requires consideration of multiple theoretical perspectives. Social Cognitive Theory (Bandura, 2004) highlights how personal factors, dietary patterns, and environments interact to shape oral health behaviours, particularly relevant in Thailand’s family-based eating culture. The nutrition literacy skills Framework (Squiers et al., 2012) outlines how individuals develop and apply nutrition literacy competencies through interactions between dietary knowledge and social environments. Additionally, Ecological Systems Theory (Bronfenbrenner, 1979) demonstrates how family and societal systems influence health behaviours and programme implementation.

Within this theoretical context, nutrition literacy for oral health emerges as a critical determinant of oral health outcomes. While health literacy encompasses capacities for accessing and using health information (Sørensen et al., 2012), nutrition literacy for oral health specifically focuses on these competencies in oral healthcare. Evidence consistently shows that individuals with low nutrition literacy tend to exhibit poor oral health behaviours and outcomes (Berkman et al., 2011; Kickbusch et al., 2013). This relationship is particularly significant in reducing oral health disparities (Horowitz & Kleinman, 2012), with higher nutrition literacy correlating with improved oral hygiene practices and health outcomes (Baskaradoss, 2018).

Recent advances in nutrition literacy programmes for oral health promotion have revealed that culturally tailored, context-specific interventions can significantly enhance service accessibility and oral healthcare engagement (Macek et al., 2016). Various programme modalities have emerged, encompassing educational initiatives, community-based activities, and digital media interventions (Dickson-Swift et al., 2014). These approaches align well with Thailand’s dental public health policy, which emphasises proactive oral health promotion and community participation.  Systematic review (Firmino et al., 2017) identified several critical gaps in existing research: the absence of mixed-methods studies examining both programme effectiveness and change processes, limited analysis of community-level behavioural change mechanisms, and insufficient research in resource-constrained developing countries where success factors may differ substantially from developed nations.

To address these research gaps, this study aims to evaluate the effectiveness of a nutrition literacy programme for oral health promotion in Thailand’s context. Of particular interest is the role of VHVs as key implementation agents, given their established position in community health promotion (Kowitt et al., 2015). While previous research has demonstrated VHVs’ capacity to utilise technology for expanding health service coverage (Jandee et al., 2015), empirical evidence regarding their role in promoting nutrition literacy for oral health remains limited.

Guided by our theoretical framework, we employed a Convergent Parallel Mixed Methods design (Creswell & Plano Clark, 2017), enabling comprehensive assessment of both quantitative programme effectiveness and qualitative change mechanisms. This approach examines how social modelling, nutrition literacy skill development related to oral health, and environmental factors interact to influence programme outcomes. Ultimately, this study’s findings will contribute to developing contextually appropriate nutrition literacy strategies for oral health promotion in developing countries while aligning with Thailand’s dental public health policies.

II. METHODS

A. Study Design

This study employed a convergent parallel mixed methods design (Creswell & Plano Clark, 2017) to comprehensively evaluate the implementation and effectiveness of a nutrition literacy programme for oral health promotion. The design integrated quantitative outcomes with qualitative insights to achieve deeper understanding than single-method approaches. The quantitative component utilised a cross-sectional survey to assess nutrition literacy skills and clinical outcomes, while the qualitative component employed a multi-case study approach (Yin, 2018) to explore implementation experiences and contextual factors.

B. Population and Sampling

The quantitative phase included all VHVs who completed nutrition literacy training (N=60) in January 2024, with data collection occurring in July 2024. For the qualitative component, 20 VHVs were purposively selected using intensity sampling (Miles et al., 2013) based on four criteria: programme implementation experience exceeding six months, strong communication abilities, representation from varied performance areas, and voluntary informed consent. This sample size achieved theoretical saturation (Creswell, 2013; Guest et al., 2006). Gender distribution differed between samples (quantitative: 98.3% female; qualitative: 70% female) due to purposive sampling for diverse leadership perspectives. Sensitivity analysis confirmed no significant gender-based differences in primary outcomes (p > .05). The six-month assessment period aligned with established behaviour change evaluation timeframes (Glasgow et al., 2019), while monitoring through December 2024 captured seasonal variations and sustainability data.

C. Research Instruments

Two complementary instruments were developed and validated through pilot testing with 30 VHVs sharing similar characteristics with the target population, but excluded from the final sample. The questionnaire was designed according to Nutbeam’s health literacy framework (Nutbeam, 2000), operationalizing three literacy levels into six nutrition literacy components relevant to oral health promotion. Items utilised a five-point Likert scale (1 = “not confident at all” to 5 = “very confident”) for self-assessment of perceived competencies. A panel of five experts including community dentistry, nutrition, public health, health literacy, and health communication specialists assessed content validity, achieving a high IOC index of 0.96, while internal consistency demonstrated excellent reliability (Cronbach’s alpha = 0.929).

The structured interview guide explored knowledge application, teaching methods, implementation challenges, outcomes, and recommendations following established qualitative research principles (Jacob & Furgerson, 2012). Qualitative trustworthiness was ensured through member checking at two stages: during interviews for immediate verification and after preliminary analysis with eight selected participants for validation and refinement.

D. Data Collection

Baseline data was collected prior to programme implementation in January 2024, establishing pre-intervention metrics through public health service records. Following six-month implementation, parallel quantitative and qualitative assessments were conducted in July 2024. Self-assessment questionnaires were administered to all VHVs, followed by in-depth interviews (45-60 minutes) with 20 purposively selected participants until data saturation was achieved (Guest et al., 2006). In accordance with Consolidated Criteria for Reporting Qualitative Studies (COREQ) guidelines (Zachariah et al., 2024), participant confidentiality was maintained throughout the study, with written informed consent obtained after comprehensive briefing on study objectives and participant rights. Monthly data extraction from the Health Data Centre continued through December 2024 to capture sustained programme effects, with systematic collection on the 5th of each month ensuring complete and timely data acquisition.

E. Data Analysis

The analytical approach integrated multiple complementary methods for comprehensive understanding. Quantitative analysis included descriptive statistics (frequencies, percentages, means, standard deviations) with Shapiro-Wilk normality testing. Inferential analyses comprised paired t-tests for pre-post comparisons (α = 0.05), chi-square tests for categorical outcomes, and Pearson’s correlation coefficients examining relationships between nutrition literacy domains. Effect sizes were reported using Cohen’s d with bootstrap confidence intervals (1,000 resamples). Statistical analyses utilised IBM’s Statistical Package for the Social Sciences (SPSS) Statistics software. Missing data patterns were examined using Little’s Missing Completely at Random (MCAR) test, with multiple imputation (5 datasets) addressing missing values following Rubin’s guidelines (2004). Sensitivity analyses compared complete-case and imputed datasets (van Buuren, 2018).

Qualitative data underwent thematic analysis following established frameworks (Braun & Clarke, 2006), involving verbatim transcription, independent coding by two researchers, and iterative thematic framework development through consensus meetings. ATLAS.ti software facilitated systematic organisation and analysis. Quality assurance included investigator triangulation, member checking with eight participants, audit trail documentation, and researcher reflexivity journals.

F. Data Integration

A comprehensive integration strategy synthesised quantitative and qualitative findings through three interconnected phases (Cano & Lomibao, 2023). Joint displays facilitated systematic comparison of results, enabling identification of convergent and divergent patterns. Meta-inferences were constructed through iterative cross-method analysis, with attention to complementary insights. Pattern matching techniques examined alignments between quantitative outcomes and qualitative themes, developing integrated theoretical understandings. Conflicting findings were reconciled by contextualising quantitative results with qualitative explanations, while complementary data enriched overall interpretation, enhancing study rigor and validity.

III. RESULTS

All participants (N=60) completed quantitative assessments at baseline and a 6-month follow-up, with 20 VHVs participating in qualitative interviews. Clinical outcomes were monitored through December 2024 using complete Health Data Centre monthly data. Following convergent parallel design, quantitative and qualitative data streams were systematically merged to achieve comprehensive understanding of programme implementation and outcomes. The integrated analysis revealed that communication skills improvements were explained through qualitative evidence of adaptive teaching strategies, while regional outcome variations were illuminated by implementation challenges identified through qualitative inquiry. This systematic data merging approach provided richer insights than either quantitative or qualitative methods could offer independently.

A. Baseline Characteristics

1. Qualitative Sample (n = 20)

The qualitative sample achieved a full response rate (100%). Participants were predominantly female (70%), with males comprising 30%. The age distribution showed that 70% were between 50–60 years, while 15% each were aged 30–39 and 40–49 years. No participants were over 60. In terms of role, 65% served as Village Health Volunteers (VHVs), and 35% were Caregivers. None held dual roles.

2. Quantitative Sample (n = 60)

The quantitative sample also achieved a 100% response rate. Females constituted the vast majority (98.3%), with only one male respondent (1.7%). Most participants (70%) were aged 50–60 years, with smaller proportions aged 30–39 (11.7%), 40–49 (16.7%), and over 60 (1.7%). Regarding position, 85% were VHVs, 13.3% were Caregivers, and 1.7% held both roles.

B. Programme Implementation and Nutrition Literacy Skills for Oral Health

The intervention (Table 1) demonstrated significant improvements across all six nutrition literacy domains (p< 0.001) with large effect sizes. Communication skills showed the greatest improvement (d = 1.64, mean difference: 0.84 points, 95% CI: 0.66-1.02), followed by Decision Making (d = 0.90), Critical Inquiry (d = 0.88), Understanding (d = 0.85), Application (d = 0.77), and Access (d = 0.74). Other domains improved by 0.36-0.41 points.

C. Clinical Outcomes and Programme Effectiveness

Clinical outcomes significantly improved. Dental check-up rates increased from 1.41% to 2.61% (difference: 1.20 percentage points, 95% CI: 0.90-1.50, p=0.032). Participants with ≥20 functional teeth rose from 87.36% to 92.72% (difference: 5.36 percentage points, 95% CI: 3.38-7.34, p< 0.001), indicating substantial improvements in both knowledge and oral health behaviour.

Outcomes

Baseline

(mean±SD)

6-month

(mean±SD)

Mean difference

(95% CI)

p-value

Nutrition Literacy Skills
related to Oral Health

Access

3.80±0.50

4.16±0.47

0.36 (0.19, 0.53)

<0.001†

Understanding

3.75±0.48

4.15±0.46

0.40 (0.23, 0.57)

<0.001†

Critical Inquiry

3.70±0.47

4.11±0.46

0.41 (0.24, 0.58)

<0.001†

Decision Making

3.72±0.46

4.13±0.45

0.41 (0.25, 0.57)

<0.001†

Application

3.85±0.49

4.21±0.44

0.36 (0.19, 0.53)

<0.001†

Communication

3.90±0.52

4.74±0.51

0.84 (0.66, 1.02)

<0.001†

Clinical Outcomes

Dental check-up rate (%)

1.41

2.61

1.20 (0.90, 1.50)

0.032‡

Functional teeth (%) *

87.36

92.72

5.36 (3.38, 7.34)

<0.001‡

Note: *Defined as having ≥20 functional natural teeth
†Statistically significant at p< 0.001, Paired t-test
‡Statistically significant at p< .05 for dental check-up rate and p< 0.001 for functional teeth, Chi-square test
Data were retrieved from the Health Data Centre database (Ministry of Public Health, 2024).

Table 1. Changes in Nutrition Literacy Skills Related to Oral Health and Clinical Outcomes After a 6-Month Training Programme (N=60)

Health Literacy Domain

1. Access

2.
Understanding

3.
Critical Inquiry

4.
Decision Making

5. Application

6. Communication

    1.
    Access

    1

    2.
    Understanding

    .858**

    1

    3.
    Critical Inquiry

    .753**

    .712**

    1

    4.
    Decision Making

    .775**

    .817**

    .834**

    1

    5.
    Application

    .724**

    .770**

    .797**

    .797**

    1

    6.
    Communication

    .812**

    .820**

    .822**

    .868**

    .799**

    1

    Note: N = 60; **p < .01 (2-tailed) Pearson correlation coefficients are shown.

    Table 2. Correlation Analysis of Nutrition Literacy Domains Related to Oral Health

    Regional variations in dental check-up rates were substantial, ranging from 0.07% to 38.18% (p < 0.001) across participating health centres, suggesting the need to investigate factors contributing to different implementation outcomes despite similar geographical and healthcare delivery contexts.

    The findings support overall programme effectiveness, though the cross-sectional design indicates the need for longitudinal research to confirm long-term impacts. Future nutrition literacy programmes for oral health promotion should emphasize communication skills and context-specific implementation approaches. The qualitative analysis of 20 VHV interviews yielded four main themes (Figure 1).

    Figure 1. Qualitative final thematic map

    D. Implementation Process and Contextual Factors

    1. Knowledge Transfer Patterns

    VHVs utilised multiple communication channels and diverse pedagogical approaches. Individual consultations involved direct problem assessment, with participants noting “Face-to-face, asking what problems they have, like sensitive teeth” (P15). Digital platforms expanded reach through “Online communication and inviting others to join our Line group” (P5). Teaching methods included demonstrations, mnemonics, and hands-on practice.

    2. Audience Diversity

    VHVs encountered heterogeneous learning populations with varying engagement levels. Successful interactions were characterised by high comprehension rates: “Everyone understood and could practice, no problems as they all understood well” (P19). However, engagement challenges persisted, with some noting “One person at home is not very interested” (P16).

    3. Implementation Challenges

    Communication barriers emerged as significant obstacles. VHVs identified hearing difficulties: “The listener’s hearing, they can’t hear well” (P1), language barriers: “Don’t use too many English terms, some words are not understood” (P10), and content complexity issues: “Some content is difficult to understand, takes a long time and repeated study” (P14).

    4. Development Approaches

    VHVs suggested practical improvements emphasising “Should practice more than theory” (P1). They recommended age-appropriate strategies: “Elderly may have difficulty learning, but if we can make content easy to understand, they will gain knowledge too” (P15), and streamlined delivery: “Shorter courses might attract more participants” (P5).

     E. Integrated Results

    The convergent parallel design employed a merging data integration approach to synthesise quantitative and qualitative findings systematically, providing comprehensive understanding of programme effectiveness., as presented in Table 3.

    Major Themes

    Quantitative
    Results (n=60)

    Qualitative
    Evidence (n=20)

    Meta-inference

    Nutrition Literacy Skills Performance Related to Oral Health

    Overall implementation: M=4.14±0.41, p< 0.001; Highest in communication (M=4.74±0.51); Strong inter-skill correlations (r=.712-.868, p< 0.001)

    Demonstrated multiple teaching approaches: individual counselling, memory techniques, continuous monitoring

    Quantitative high scores validated by qualitative evidence of practical skill application

    Clinical Outcome Changes

    Dental check-up: 1.41% to 2.61% (p< .01); Functional teeth: 87.36% to 92.72% (p< .01); Regional variation: 0.07-38.18%

    Implementation variations: successful behaviour adoption, mixed community readiness, diverse response levels

    Outcome improvements linked to implementation quality and community readiness

    Implementation Challenges

    Highest in self-monitoring (M=4.25±0.44); Significant regional differences (p< .01)

    Identified barriers: technical language, age-related learning, practice compliance

    Statistical variations explained by specific implementation challenges identified qualitatively

    Support Systems

    Strong correlations between: decision-making and communication (r=.868); access and understanding (r=.858); all p< 0.001

    Multiple support channels: digital platforms, family networks, community groups

    Integrated support systems crucial for programme effectiveness

    Table 3. Integrated Analysis of Mixed Methods Results

    The systematic merging of quantitative and qualitative data through meta-inference analysis revealed four key dimensions of programme implementation and outcomes.

    1. Nutrition Literacy Skills and Clinical Outcomes

    Quantitative findings demonstrated high overall implementation levels (M=4.14±0.41, p< 0.001), with communication skills showing exceptional improvement (M=4.74±0.51). The strong correlation between communication and decision-making skills (r=.868, p< 0.001) was validated through qualitative evidence: “We adapted communication methods based on audience needs” (P15).

    Dental check-up rates increased significantly from 1.41% to 2.61% (p< .01), while functional dentition improved from 87.36% to 92.72% (p< .01). Qualitative insights revealed implementation quality influences: “Regular follow-ups and practical demonstrations helped maintain behaviour changes” (P8). Regional outcome variations (0.07-38.18%) aligned with identified barriers and facilitators.

    2. Implementation Dynamics and Support Systems

    Strong correlations between access and understanding (r=.858, p< 0.001) were complemented by contextual adaptation findings. VHVs balanced cultural factors: “We needed to balance traditional beliefs with modern dental care practices” (P13). Statistical associations among nutrition literacy domains (r=.712-.868, all p< 0.001) were substantiated by interconnected support mechanisms: “The combination of in-person support and online reminders helped maintain engagement” (P5).

    The meta-inference demonstrates programme effectiveness through synergy of enhanced nutrition literacy skills and context-sensitive implementation strategies, emerging through systematic integration of quantitative measurements with qualitative insights.

    IV. DISCUSSION

    A. Programme Effectiveness and Theoretical Framework

    This study demonstrates the effectiveness of a Village Health Volunteers (VHVs)-led nutrition literacy programme for oral health promotion in significantly improving nutrition literacy skills and clinical outcomes. The findings align with established empirical evidence at regional and international levels regarding healthcare personnel capacity development and relationships between nutrition literacy for oral health, oral health behaviours, and preventive service utilisation (Baskaradoss, 2018; Nutbeam, 2008; Samarasekera et al., 2024; Batista et al., 2017; Baskaradoss, 2016).

    The strong correlation between nutrition literacy components, particularly communication and decision-making (r = .868), reflects their interconnected nature and underscores comprehensive skill development importance (Kunathum, 2023). This finding aligns with Social Cognitive Theory (Bandura, 2004), emphasising behavioural, personal, and environmental factor interdependence in health promotion. Results support the Nutrition Literacy Skills Framework (Squiers et al., 2012), positioning communication and decision-making as essential mediators between nutrition literacy and oral health behavioural outcomes in diverse cultural contexts.

    B. Clinical Outcomes and Community Engagement

    The increase in dental check-up rates from 1.41% to 2.61%, while statistically significant, represents modest absolute change. However, within rural communities where oral health service access is severely limited and baseline utilisation extremely low, even small improvements may represent important community health engagement shifts (Petersen, 2009). This suggests early evidence of improved health literacy and behaviour change among participants, particularly VHVs who played critical implementation roles.

    Future interventions could incorporate community-based incentives, outreach dental services, and proactive VHV follow-up to reinforce preventive behaviours. Evidence demonstrates that community mobilisation and culturally tailored interventions effectively improve oral health behaviours in low-resource settings (Fisher-Owens et al., 2013; Watt, 2007).

    C. Domain-Specific Performance and Regional Variations

    Communication and skill application emerged as key behavioural change drivers in nutrition literacy for oral health (M = 4.74, SD = 0.51 and M = 4.21, SD = 0.44 respectively). The relatively lower scores in critical inquiry (M = 4.11, SD = 0.46) and decision-making (M = 4.13, SD = 0.45) skills align with identified community health worker limitations (Gall et al., 2023) and indicate the necessity of incorporating hybrid learning approaches to strengthen advanced nutrition literacy competencies (Lin et al., 2024).

    Regional analysis revealed significant outcome variations across implementation areas (0.07% to 38.18%, p < .01) (Watt et al., 2019), with stronger outcomes in communities with higher social capital. This pattern aligns with systematic reviews from low- and middle-income countries (Haldane et al., 2019) and documented disparities in Thailand’s healthcare systems (Chaianant et al., 2022). These findings support Asset-Based Community Development theory (Kretzmann & McKnight, 1993), emphasising the importance of leveraging existing community strengths for sustainable oral health improvements.

    D. Social Support Systems and Cultural Context

    Social support systems proved crucial for programme success, particularly in developing countries where social networks, family support systems, and community resources serve as primary health determinants (Kowitt et al., 2015). The strong correlation between communication and community participation (r = .799, p < .01) reflects these interconnections, aligning with Ecological Systems Theory (Bronfenbrenner, 1979), which emphasises how multiple environmental layers influence nutrition-related oral health behaviours in developing countries where community and cultural contexts play crucial roles.

    E. Gender Considerations and Methodological Considerations

    The quantitative sample exhibited significant gender imbalance (98.3% female participants), potentially influencing generalisability. In Northern Thailand, approximately 83% of VHVs are female, reflecting traditional social roles where women are often a group highly motivated to engage in volunteer work aimed at assisting others. Furthermore, women’s volunteer roles frequently involve healthcare and activities related to building community resilience (Sukhampha et al., 2023). Women typically exhibit higher health awareness and more proactive health behaviours than men, which may partly explain observed positive outcomes (Tan et al., 2021).

    The notably high correlations between nutrition literacy domains (r=0.712-0.868) reflect comprehensive skill development influenced by the holistic training programme and Thai VHVs’ cultural context where integrated health communication is traditionally emphasised. This finding aligns with studies in Asian contexts (Leung et al., 2020; Oh et al., 2022) suggesting important cultural influences on health literacy skill development.

     F. Study Strengths and Limitations

    This study demonstrates methodological strengths through its convergent parallel mixed-methods design with systematic data integration, enhancing understanding through integrated quantitative and qualitative insights. The qualitative component achieved theoretical saturation (Guest et al., 2006), while community-based implementation aligned with established nutrition literacy research practices (Kowitt et al., 2015).

    Key limitations include absence of factor analysis despite high internal consistency (Cronbach’s α = 0.929), pronounced gender imbalance restricting applicability, six-month follow-up potentially inadequate for capturing long-term changes (Baskaradoss, 2018), self-reported data risks and social desirability bias (Althubaiti, 2016), geographical specificity limiting generalisability given Thailand’s varied healthcare systems (Chaianant et al., 2022), and resource constraints precluding randomised controlled design. While the dental check-up rate increase was statistically significant (p=0.032), the modest improvement suggests need for more intensive interventions.

    V. CONCLUSION

    The VHVs-led nutrition literacy programme for oral health promotion demonstrates clear effectiveness through significant behavioural and clinical changes. Key success factors include local context adaptation and community engagement. For broader implementation, three policy directions are suggested: (1) integration with national health promotion policies, (2) inclusion of nutrition literacy indicators related to oral health in monitoring systems, and (3) development of standardised guidelines allowing local adaptation. Long-term VHVs capacity development should incorporate continuous professional development through structured mentoring programmes, nutrition literacy skill enhancement workshops for oral health promotion, and recognition systems for advanced competencies. Digital health integration should focus on mobile learning platforms, telemedicine support, and electronic health records, while sustainable monitoring mechanisms should include automated data collection, regular feedback loops, and community-based evaluations.

    Future studies should have follow-up periods of at least one year to confirm sustainability of nutrition-related oral health behaviour changes (Baskaradoss, 2018). Research priorities should analyse regional variations, conduct economic evaluations, and develop sustainability indicators while integrating diverse learning approaches to enhance effectiveness (Lin et al., 2024). This study confirms the programme’s effectiveness and provides insights into change mechanisms and success factors for future nutrition literacy programmes focused on oral health promotion and public health policy. A phased scaling approach with diverse pilot programmes is recommended to optimise outcomes through cross-regional learning and experience sharing.

    Notes on Contributors

    Chollada Sorasak led the research design, developed methodology, conducted formal analysis and investigation. She was responsible for writing the original manuscript draft and managing the revision process.

    Worayuth Nak-Ai provided expertise in validating the research design, research methodology and supervised the overall research implementation process. He was responsible for proof the original manuscript draft and managing the revision process.

    Choosak Yuennan managed the data curation process and provided supervision for data collection and analysis procedures.

    Mansuang Wongsapai coordinated resource allocation and managed project administration tasks throughout the study period.

    Ethical Approval

    This study was approved by the Human Research Ethics Committee of Sirindhorn College of Public Health, Chonburi (COA No. 2023/T07, dated 21 August 2023).

    Data Availability

    The data supporting the findings of this study, including four tables and one figure used in the analysis, are openly available in Figshare at http://dx.doi.org/10.6084/m9.figshare.28105718.

    The dataset includes the complete quantitative and qualitative analysis results, tables, and figures used in this study and can be accessed without restrictions for research purposes.

    Acknowledgement

    We express our gratitude to Dr. Kwanmuang Kaewdamkoeng, Mr. Songkat Duangkhamsawat, Ms. Jariyakorn Ditjinda, and Ms. Wilawan Tangsattayatistan for their expertise in health literacy. We thank Dr. Chalermpol Kongchit, Ms. Waenkaew Chaiararm from Chiang Mai University for communications guidance, and Ms. Umaporn Nimtrakul and the Health Centre Region 1 Chiang Mai team for networking support. We also acknowledge the institutional support from the Intercountry Centre for Oral Health, Department of Health, Thailand, Sirindhorn College of Public Health Chonburi, and Boromarajonani College of Nursing.

    Funding

    This research received no external funding. The Intercountry Centre for Oral Health, Department of Health provided in-kind support through equipment, materials, and transportation for data collection. The remaining expenses were self-funded by the corresponding author.

    Declaration of Interest

    The authors declare no conflicts of interest, financial, consultant, institutional or other relationships that might lead to bias or a conflict of interest.

    References

    Althubaiti, A. (2016). Information bias in health research: Definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare, 9, 211–217. https://doi.org/10.2147/JMDH.S104807

    Bandura, A. (2004). Health promotion by social cognitive means. Health Education & Behaviour, 31(2), 143–164. https://doi.org/10.1177/1090198104263660

    Baskaradoss, J. K. (2016). The association between oral health literacy and missed dental appointments. Journal of the American Dental Association, 147(11), 867–874. https://doi.org/10.1016/j.adaj.2016.05.011

    Baskaradoss, J. K. (2018). Relationship between oral health literacy and oral health status. BMC Oral Health, 18(1), 172. https://doi.org/10.1186/s12903-018-0640-1

    Batista, M. J., Lawrence, H. P., & Sousa, M. D. L. R. D. (2017). Oral health literacy and oral health outcomes in an adult population in Brazil. BMC Public Health, 18, 60. https://doi.org/10.1186/s12889-017-4443-0

    Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., & Crotty, K. (2011). Low health literacy and health outcomes: An updated systematic review. Annals of Internal Medicine, 155(2), 97–107. https://doi.org/10.7326/0003-4819-155-2-201107190-00005

    Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

    Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.

    Cano, J. C., & Lomibao, L. S. (2023). A mixed methods study of the influence of phenomenon-based learning videos on students’ mathematics self-efficacy, problem-solving and reasoning skills, and mathematics achievement. American Journal of Educational Research, 11(3), 97–115.

    Chaianant, N., Tussanapirom, T., Niyomsilp, K., & Gaewkhiew, P. (2022). Factors associated with oral health check-up in working adults: The 8th Thai National Oral Health Survey 2017. Thai Dental Public Health Journal, 27(2), 112–123.

    Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). SAGE Publications, Inc.

    Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE Publications, Inc.

    Dickson-Swift, V., Kenny, A., Farmer, J., Gussy, M., & Larkins, S. (2014). Measuring oral health literacy: A scoping review of existing tools. BMC Oral Health, 14, Article 148. https://doi.org/10.1186/1472-6831-14-148

    Duijster, D., Monse, B., Dimaisip-Nabuab, J., Djuharnoko, P., Heinrich-Weltzien, R., Hobdell, M., Kromeyer-Hauschild, K., Kunthearith, Y., Mijares-Majini, M. C., Siegmund, N., Soukhanouvong, P., & Benzian, H. (2017). “Fit for School” – A school-based water, sanitation and hygiene programme to improve child health: Results from a longitudinal study in Cambodia, Indonesia and Lao PDR. BMC Public Health, 17, Article 302. https://doi.org/10.1186/s12889-017-4203-1

    Firmino, R. T., Ferreira, F. M., Paiva, S. M., Granville-Garcia, A. F., Fraiz, F. C., & Martins, C. C. (2017). Oral health literacy and associated oral conditions: A systematic review. Journal of the American Dental Association, 148(8), 604–613. https://doi.org/10.1016/j.adaj.2017.04.012

    Fisher-Owens, S. A., Isong, I. A., Soobader, M. J., Gansky, S. A., Weintraub, J. A., Platt, L. J., & Newacheck, P. W. (2013). An examination of racial/ethnic disparities in children’s oral health in the United States. Journal of Public Health Dentistry, 73(2), 166–174. https://doi.org/10.1111/j.1752-7325.2012.00367.x

    Gall, M., Schroeder, F., & Heise, G. (2023). Community health worker use of smart devices for health promotion: Scoping review. JMIR mHealth and uHealth, 11(1), e42023. https://doi.org/10.2196/42023

    Gholami, M., Pakdaman, A., Montazeri, A., Jafari, A., & Virtanen, J. I. (2014). Assessment of periodontal knowledge following a mass media oral health promotion campaign: A population-based study. BMC Oral Health, 14, Article 31. https://doi.org/10.1186/1472-6831-14-31

    Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., Ory, M. G., & Estabrooks, P. A. (2019). RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review. Frontiers in Public Health, 7, 64. https://doi.org/10.3389/fpubh.2019.00064

    Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822X05279903

    Haldane, V., Chuah, F. L. H., Srivastava, A., Singh, S. R., Koh, G. C. H., Seng, C. K., & Legido-Quigley, H. (2019). Community participation in health services development, implementation, and evaluation: A systematic review of empowerment, health, community, and process outcomes. PLOS ONE, 14(5), e0216112. https://doi.org/10.1371/journal.pone.0216112

    Horowitz, A. M., & Kleinman, D. V. (2012). Oral health literacy: A pathway to reducing oral health disparities in Maryland. Journal of Public Health Dentistry, 72(s1), S26–S30. https://doi.org/10.1111/j.1752-7325.2012.00316.x

    Jacob, S. A., & Furgerson, S. P. (2012). Writing interview protocols and conducting interviews: Tips for students new to the field of qualitative research. The Qualitative Report, 17, 1–10.

    Jandee, K., Kaewkungwal, J., Khamsiriwatchara, A., Lawpoolsri, S., Wongwit, W., & Wansatid, P. (2015). Effectiveness of using mobile phone image capture for collecting secondary data: A case study on immunization history data among children in remote areas of Thailand. JMIR mHealth and uHealth, 3(3), e75. https://doi.org/10.2196/mhealth.4183

    Kickbusch, I., Pelikan, J. M., Apfel, F., & Tsouros, A. D. (2013). Health literacy: The solid facts. World Health Organization Regional Office for Europe. https://apps.who.int/iris/handle/10665/128703

    Kowitt, S. D., Emmerling, D., Fisher, E. B., & Tanasugarn, C. J. (2015). Community health workers as agents of health promotion: Analysing Thailand’s Village Health Volunteer Programme. Journal of Community Health, 40(4), 780–788. https://doi.org/10.1007/s10900-015-9999-y

    Kretzmann, J. P., & McKnight, J. L. (1993). Building communities from the inside out: A path toward finding and mobilizing a community’s assets. ACTA Publications.

    Kunathum, W. (2023). The relationship between health literacy and roles in chronic disease prevention among village health volunteers. Public Health Policy and Laws Journal, 9(3), 431–444.

    Leung, A. Y. M., Chau, P. H., & Leung, I. S. H. (2020). Health literacy experiences of multi‐ethnic patients and their health‐care providers in the management of type 2 diabetes in Malaysia. Health Expectations, 23(6), 1442–1452. https://doi.org/10.1111/hex.13095

    Lin, G. S. S., Tan, W. W., Ng, Y. S., & Afrashtehfar, K. I. (2024). Enhancing students’ academic performance through hybrid team-based case-based learning. The Asia Pacific Scholar, 9(3), 67–69. https://doi.org/10.29060/TAPS.2024-9-3/CS3189

    Macek, M. D., Atchison, K. A., Watson, M. R., Holtzman, J., Wells, W., Braun, B., … Richards, J. (2016). Assessing health literacy and oral health: Preliminary results of a multi-site investigation. Journal of Public Health Dentistry, 76(4), 303–313. https://doi.org/10.1111/jphd.12156

    McKinnon, M. A., Odoh, O., Taylor, P., Charlie, D., Morry, J., Mathu-Muju, K., & Donnelly, L. (2022). Developing a land-based oral health promotion project with an Indigenous community in northern British Columbia, Canada. Canadian Journal of Dental Hygiene, 56(3), 172–176.

    Miles, M. B., Huberman, A. M., & Saldaña, J. (2013). Qualitative data analysis: A methods sourcebook (3rd ed.). SAGE Publications, Inc.

    Ministry of Public Health. (2024). Health Data Centre Service [Dashboard].

    Nutbeam, D. (2000). Health literacy as a public health goal: A challenge for contemporary health education and communication strategies into the 21st century. Health Promotion International, 15(3), 259–267. https://doi.org/10.1093/heapro/15.3.259

    Nutbeam, D. (2008). The evolving concept of health literacy. Social Science & Medicine, 67(12), 2072–2078. https://doi.org/10.1016/j.socscimed.2008.09.050

    Oh, K. M., An, K., Lee, M., & Kreps, G. L. (2022). Cultural factors influencing health literacy, health care access, and health behaviours among Korean-Americans. Health Literacy Research and Practice, 6(2), e89–e97. https://doi.org/10.4324/9781003230243-2

    Peres, M. A., Macpherson, L. M. D., Weyant, R. J., Daly, B., Venturelli, R., Mathur, M. R., Listl, S., Celeste, R. K., Guarnizo-Herreño, C. C., Kearns, C., Benzian, H., Allison, P., & Watt, R. G. (2019). Oral diseases: A global public health challenge. The Lancet, 394(10194), 249–260. https://doi.org/10.1016/S0140-6736(19)31146-8

    Petersen, P. E. (2009). Global policy for improvement of oral health in the 21st century – Implications to oral health research of World Health Assembly 2007, World Health Organisation. Community Dentistry and Oral Epidemiology, 37(1), 1–8. https://doi.org/10.1111/j.1600-0528.2008.00448.x

    Rubin, D. B. (2004). Multiple imputation for nonresponse in surveys (Vol. 81). Wiley.

    Samarasekera, D. D., Lee, S. S., Yeo, S. P., Chen, J., Findyartini, A., Greviana, N., & Sherman, L. (2024). The state of continuing professional development in East and Southeast Asia among medical practitioners. The Asia Pacific Scholar, 9(3), 1–14. https://doi.org/10.29060/TAPS.2024-9-3/OA3045

    Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., & Brand, H. (2012). Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health, 12, 80. https://doi.org/10.1186/1471-2458-12-80

    Squiers, L., Peinado, S., Berkman, N., Boudewyns, V., & McCormack, L. (2012). The health literacy skills framework. Journal of Health Communication, 17(suppl 3), 30–54. https://doi.org/10.1080/10810730.2012.713442

    Sukhampha, R., Khwanriang, P., & Vaisamruat, K. (2023). Women on the frontline of COVID-19: Understanding local women village health volunteers in the northern province of Thailand. BMC Health Services Research, 23, Article 9305. https://doi.org/10.1186/s12913-023-09305-x

    Tan, J., Yoshida, Y., Ma, S.-K., & Mauvais-Jarvis, F. (2021). Gender differences in health protective behaviours during the COVID-19 pandemic in Taiwan: An empirical study. medRxiv [Preprint]. https://doi.org/10.1101/2021.04.14.21255448

    Tay, R. S. C., Wegner, D. R., Lim, L. S., Ting, J., & Ong, S. T. (2024). Enhancing teaching and learning of evidence-based practice via game-based learning. The Asia Pacific Scholar, 9(3), 41–49. https://doi.org/10.29060/TAPS.2024-9-3/SC3111

    van Buuren, S. (2018). Flexible imputation of missing data (2nd ed.). Chapman and Hall/CRC. https://doi.org/10.1201/9780429492259

    Walker, D., Tynan, A., Tucker, T., Fisher, B., & Fisher, T. (2023). Engaging with a rural Aboriginal community to identify strategies to improve oral health within their community. Australian Journal of Primary Health, 29(1), 38–46. https://doi.org/10.1071/PY22215

    Watt, R. G. (2007). From victim blaming to upstream action: Tackling the social determinants of oral health inequalities. Community Dentistry and Oral Epidemiology, 35(1), 1–11. https://doi.org/10.1111/j.1600-0528.2007.00348.x

    Watt, R. G., Daly, B., Allison, P., Macpherson, L. M. D., Venturelli, R., & Benzian, H. (2019). Ending the neglect of global oral health: Time for radical action. The Lancet, 394(10194), 261–272. https://doi.org/10.1016/S0140-6736(19)31133-X

    Yin, R. K. (2018). Case study research and applications: Design and methods. SAGE Publications, Inc.

    Zachariah, R., Kumar, A. M. V., Isaakidis, P., Sreenivas, A., Bissell, K., Van den Bergh, R., Satyanarayana, S., Van Henten, S., & Reid, A. J. (2024). Reporting guideline for global health qualitative research methods. Global Health Action, 17(1), 2350585. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11353496/

    *Worayuth Nak-Ai
    Praboromarajchanok Institute, Thailand
    668-3570-5859
    Email: worayuth@scphc.ac.th

    Submitted: 28 October 2024
    Accepted: 16 June 2025
    Published online: 7 October, TAPS 2025, 10(4), 35-43
    https://doi.org/10.29060/TAPS.2025-10-4/OA3559

    Reshma Mohamed Ansari1,2, Chan Choong Foong3, Hidayah Mohd Fadzil4 & Mohamad Nabil Mohd Noor3

    1Institute for Advanced Studies, Universiti Malaya, Malaysia; 2Department of Medical Education, International Medical School, Management and Science University, Malaysia; 3Medical Education and Research Development Unit (MERDU), Faculty of Medicine, Universiti Malaya, Malaysia; 4Department of Mathematics and Science Education, Faculty of Education, Universiti Malaya, Malaysia

    Abstract

    Introduction: Self-care agency, a core concept that helps alleviate the stressors of medical training, is postulated to be practiced by medical students who exhibit a growth mindset. Hence, this pilot study was designed to measure, compare, and correlate the self-care agency and mindsets of undergraduate medical students to assess the potential for scaling to a national survey.

    Methods: This cross-sectional study was conducted at one public and one private medical university using a revised version of the Appraisal of Self-Care Agency Scale to measure self-care agency and the Implicit Theories of Intelligence Scale to measure participants’ mindset. Data was analysed using IBM SPSS.

    Results: In total, 329 complete responses were obtained. Among the self-care domains, a higher capacity for self-care and a developing capacity for self-care, with a lower ability to indulge in self-care, were reported. Self-care agency showed a significant difference between pre-clinical and clinical students (p = .027; Cohen’s d = .2). Mindset measurements revealed a higher growth than a fixed mindset. The Pearson correlation coefficient showed a weak positive correlation, (r = 0.19) between the means of self-care agency and a growth mindset. Clinical students showed a higher tendency toward self-care than their pre-clinical counterparts in an independent-samples t-test, with no differences between genders and universities.

    Conclusion: This pilot study found a positive correlation between self-care agency and growth mindset among undergraduate medical students. Although limited by two prototype universities and response bias, this study provides a solid foundation for future nationwide or cross-country studies.

    Keywords:            Self-care Agency, Growth Mindset, Fixed Mindset, Medical Education, Undergraduate Medical Students, Pilot Study

    Practice Highlights

    • Undergraduate medical students acknowledge importance of self-care agency and growth mindset.
    • Growth mindset correlates with self-care agency necessitating inculcation of both.
    • Growth mindset combats the stressors of medical training by indulging in proactive self-care.

    I. INTRODUCTION

    Orem et al. (1995, p. 19) defined self-care as the “practice of activities that individuals start and perform for their benefit, for the maintenance of life, health, and well-being”. Self-care agency, a component of the self-care deficit nursing theory, is “the acquired, complex capacity to meet the requirements to take care of oneself, regulating life processes, maintaining or promoting integrity, structure, and functioning, as well as one’s development and promotion of well-being” (Oliveira et al., 2022, p. 20; Orem et al., 1995). Research has elucidated that the stressors of medical training, which can cause high rates of burnout, anxiety, depression and poor physical health in students (Ayala et al., 2017; Bostock et al., 2018), could be alleviated through self-care (Ayala et al., 2018).

    However, during medical training, students find little to no time to engage in self-care; factors such as increased pressure, poor time management, and a negative environment often acting as barriers (Ayala et al., 2017; Ayala et al., 2018). Guldner et al. (2020) suggested that mindset is a predictor of depression and burnout among medical residents, highlighting the relationship between mindset and poor psychological well-being, alleviated by self-care practices (Carter et al., 2025).

    Carol Dweck coined the terms “growth mindset” and “fixed mindset” as part of the implicit theories of intelligence, in which individuals could view intelligence as fixed (entity theory) or as a dimension that can be improved with effort (incremental theory) (Cook et al., 2018; Dweck, 2013). The mindset theory, when applied to the medical education context, suggests that a growth mindset views effort as a means to develop abilities (Theard et al., 2021). In contrast, a fixed mindset could lead to poorer outcomes in a high-pressure educational environment (Bostock et al., 2018) and resultant poor psychological well-being (Root Kustritz, 2017).

    Studies have measured participants’ mindsets and correlated them with scales measuring anxiety or overall well-being (Guldner et al., 2020; Root Kustritz, 2017; Wolcott et al., 2021). One of the reasons for a growth mindset to aid students’ well-being is by allowing them to place greater value on health and fitness and exhibit health-seeking behaviours (Orvidas et al., 2018; Thomas et al., 2019).

    Adding on, identification of stressors, the burnout caused by them, mental health issues, active self-care seeking behaviours including coping strategies are associated with a growth mindset (Burnette et al., 2020). Nursing students who were exposed to structured activities fostering a growth mindset, reported indulging in student-preferred strategies such as viewing obstacles from a newer perspective, working for competency, and indulging in self-care practices, thus directly relating a growth mindset to better self-care agency (Carter et al., 2025).

    Medical students in Malaysia, akin to their global counterparts have reported increased levels of stress and psychological distress (Masilamani et al., 2020), due to stringent admission processes, higher cognitive load, and an assessment-based curriculum (Yusoff et al., 2013). Studies across four public medical schools in Malaysia have shown that students tend to alleviate stress through religious activities, active coping strategies, positive reinterpretation, acceptance, and planning (Yusoff et al., 2011), which could be viewed as a subtle expression of a growth mindset. Despite the postulation that a growth mindset accentuates self-care agency, there is a dearth of studies on the self-care agency of Malaysian medical students and its correlation with mindset.

    Hence, this pilot study was conducted with the objective of measuring, comparing, and correlating self-care agency and the mindsets of Malaysian undergraduate medical students to gauge the feasibility of a nationwide survey (Lowe, 2019).

    II. METHODS

    This cross-sectional pilot study was conducted at a public and a private medical university in Malaysia. which were accessible to researchers. The selected public university is a prototype of Malaysian public universities characterised by highly qualified students with excellent examination results paying subsidised fees, as the operation of the university is funded by the government (Wan, 2007). The selected private university is representative of Malaysian private universities, which primarily provide an alternate pathway for the high school students who are not placed in public universities. Being self-funded, the fees are relatively higher, and the students are typically of paying capacity (Wan, 2007).

    Permission to conduct this study was obtained from the ethics committees of both the public and private universities (UM.TNC2/UMREC_2872 and RMC/SEP/2023/EC02, respectively). Generally, although every medical school in Malaysia is homegrown and has variations in its curriculum, both the universities chosen for this study had a similar integrated curriculum spanning over five years for the Bachelor of Medicine, Bachelor of Surgery (MBBS) program, with two years of pre-clinical and three years of clinical studies.

    The study instrument consisted of three sections. Data collection was anonymous, and the first section recorded the details of the participants’ age, gender, institution of study, and year of study. Section two included the revised version of the Appraisal of Self-care Agency Scale (ASAS-R), used to measure self-care agency (Oliveira et al., 2022). The ASAS is a 24-item scale developed by Evers et al. in 1986 (Evers, 1989) and modified by Sousa et al. (2010) into a 15-item scale that can be applied to adults aged over 18 years.

    The three domains of ASAS-R holistically measure the three types of personal trait components that characterise the concept of self-care agency (Oliveira et al., 2022). The responses were ranked on a 5-point Likert scale (1 = totally disagree; 5 = totally agree). The total scores ranged from 15 to 75, with higher scores indicating greater self-care agency (Oliveira et al., 2022).

    Sousa et al. (2010) divided the items into the following domains: Domain I: having the capacity for self-care (items 1, 2, 3, 5, 6, and 10; maximum domain score = 30); Domain II: developing the capacity to self-care (items 7, 8, 9, 12, and 13; maximum domain score = 25); and Domain III: inability to self-care (items 4, 11, 14, and 15; maximum domain score = 20), with acceptable Cronbach’s alpha values of 0.86, 0.83, and 0.79, respectively. The instrument has been validated among the general population in countries such as China, Spain, and Brazil, and among medical students in Portugal (Alhambra-Borrás et al., 2017; Guo et al., 2017; Oliveira et al., 2022; Yuan et al., 2021).  

    Section three consisted of the Implicit Theories of Intelligence Scale (ITIS), adopted from the published version of the ITIS by Cook et al. (2017), to measure the mindset of medical students. The ITIS is an eight-item instrument with four items related to incremental beliefs (1, 2, 3, and 4) and four items related to entity beliefs (5, 6, 7, and 8) regarding intelligence and ability. Responses were based on a 6-point Likert scale (1 = strongly disagree; 6 = strongly agree) (Cook et al., 2017). For each domain, the scores ranged from 4 to 24.

    A confirmatory factor analysis of the ITIS scores demonstrated an overall acceptable model fit; an exploratory factor analysis confirmed a two-factor structure (Cook et al., 2017), with the Cronbach’s alpha ≥ 0.77 for each domain (Cook et al., 2018).

    The sample size of the study was calculated using Raosoft online software (http://www.raosoft.com/samplesize.html).

    With an approximate total population of 1300 (public university = approximately 900; private university = approximately 400), based on a confidence level of 95%, a response distribution of 50%, and a margin of error of 5%, the sample size was determined to be 297. A quota sampling technique was applied to ensure adequate sample representation for each university: 204 from the public university and 93 from the private university.

    The questionnaire items were entered into a Google Form, and the link was distributed to all the students of both the universities where they were asked to answer the questionnaire after reading the participant information sheet and providing informed consent.

    The data were collected over four weeks, starting on November 1 – November 30, 2023, which was two months from the start of the new semester (September 2023) in both universities which gives ample time for the students to experience the learning environment and respond to the questionnaire. A reminder was provided two weeks after the start of data collection, and data collection ended two weeks after the reminder. The students were informed of the closure of data collection.

    The data were analysed using the IBM Statistical Package for Social Sciences (SPSS) v.26 via descriptive and inferential tests. The normality of the data distribution was screened. Pearson’s product-moment correlation was used to analyse the relationship between self-care agency and mindset, whereas independent-samples t-tests were used for comparative analyses between gender and years of study.

    III. RESULTS

    A. Participants’ Characteristics

    A total of 329 valid responses were received. The participants’ demographic characteristics are presented in Table 1.

    The distribution of the participants according to their year of study followed the same pattern in both universities, with more female than male respondents and more pre-clinical than clinical students, implying that the sample was representative of the population.

    Age

    Range (years)

    Mean

     

     

    18–28

    20.8

     

    Year of study

    Private university

    n (%)

    n=118

    (35%)

    Pre-clinical

    n (%)

    68

    57.63%

    Year 1

    n (%)

    n=31

    45.5%

    Year 2

    n (%)

    n=37

    54.5%

    Clinical

    n (%)

    50

    42.37%

    Year 3

    n (%)

    n=28

    56%

    Year 4

    n (%)

    n=12

    24%

    Year 5

    n (%)

    n=10

    20%

    Public university

    n (%)

    n=211

    (65%)

    Pre-clinical

    n (%)

    127

    60.2%

    Year 1

    n (%)

    n=78

    61.4%

    Year 2

    n (%)

    n=49

    38.6%

    Clinical

    n (%)

    84

    39.8%

    Year 3

    n (%)

    n=20

    23.8%

    Year 4

    n (%)

    n=8

    9.5%

    Year 5

    n (%)

    n=56

    66.7%

    Gender

    Male

    n (%)

     

    n=112

    34.1%

     

    Private university

    n (%)

    n=38

    33.9%

     

    Public university

    n (%)

    n=74

    66.1%

    Female

    n (%)

     

    n=217

    65.9%

    Private university

    n (%)

    n=80

    36.9%

    Public university

    n (%)

    n=137

    63.1%

    Table 1. Participants’ demographic characteristics (N = 329)

    B. Descriptive Statistics

    The key descriptive statistics and reliability indices of the ASAS-R are presented in Table 2.

    Item No.

    Itemsa

    Minimum

    Maximum

    Mean (SD)

     

     

    ASAS-R Cronbach’s alpha

    .841

    ASAS-R total score Mean (SD)

    55.74 (8.10)

    Domain I: Having the capacity for self-care (items 1, 2, 3, 5, 6 & 10 of ASAS-R)

     

    Domain total score (SD)

     

     

     

    24.14 (4.17)

     

    Domain Cronbach’s alpha

     

     

     

    .875

    Domain II: Developing the capacity for self-care (items 7, 8, 9, 12 & 13 of ASAS-R)

     

    Domain total score (SD)

     

     

     

    19.69 (3.61)

     

    Domain Cronbach’s alpha

     

     

     

    .762

    Domain III: Inability to self-care (items 4, 11, 14 & 15 of ASAS-R)

    4

    * I often lack energy to care for myself in the way that I know I should.

    1.00

    5.00

    3.09 (1.18)

     

    11

    * In my daily activities I seldom take time to care for myself.

    1.00

    5.00

    2.82 (1.19)

    14

    * I seldom have time for myself.

    1.00

    5.00

    2.82 (1.18)

    15

    * I am not always able to care for myself in a way I would like.

    1.00

    5.00

    3.15 (1.20)

    Domain total score (SD)

    11.90 (3.63)

    Domain Cronbach’s alpha

    .761

    *Negatively worded items; the answers were reversely scored.

    a5-point Likert scale: 1 (totally disagree) to 5 (totally agree).

     Table 2. ASAS-R and its scores

    The domain scores for Domains I (M = 24.14, SD = 4.17) and II (M = 19.69, SD = 3.61; maximum scores = 30 and 25, respectively) were higher than those for Domain III (M = 11.90, SD = 3.63; maximum score = 20). The items of Domains I and II showed mean scores varying from 3.7 to 4.1, indicating global positive scores for one’s self-perception of having and developing self-care abilities. The mean scores of the items of Domain III varied from 2.8 to 3.15, suggesting that the students acknowledged a lack of means to self-care (all four items were reversely worded) (Damásio & Koller, 2013).

    Table 3 presents the key descriptive statistics and reliability indices of the ITIS questionnaire. The maximum score for each domain was 24. The score for the incremental domain (Min: 1.00 and Max: 6.00; M = 18.49, SD = 4.13) was higher than that for the entity domain (Min: 1.00 and Max: 6.00; M = 12.33, SD = 5.24), indicating that the students were more aligned toward a growth mindset than a fixed mindset (Sun et al., 2021). The mean of the items indicating a fixed mindset (entity domain) showed a range of 2.8 to 3.3, compared with the mean of the items indicating a growth mindset (incremental domain), which showed a range of 4.5 to 4.7 (Hong et al., 1999).

    ITIS Cronbach’ alphaa

    .724

    Domain: Entity (items 1, 2 3 & 4 of ITIS)

    Domain total score Mean (SD)

    12.33 (5.24)

    Domain Cronbach’s alpha

    .930

    Domain: Incremental (items 5, 6, 7 & 8 of ITIS)

    Domain total score Mean (SD)

    18.49 (4.13)

    Domain Cronbach’s alpha

    .907

    a6-point Likert scale: 1 (strongly disagree) to 6 (strongly agree).

    Table 3. ITIS and its scores

    C. Correlational Analysis

    For inferential statistics, the total score of self-care agency and the mindset domains were subjected to tests of normality using the applied statistical methods of skewness and kurtosis; z-values were not considered as the sample size was greater than 300 (Kim, 2013). The resultant absolute skew values were less than 2, and the absolute kurtosis values were less than 7. Hence, the data were considered to be normally distributed, and parametric tests were applied (Hair et al., 2006; Kim, 2013). All values were considered significant if the p value was less than 0.05.

    The relationship between self-care agency (total score) and growth mindset (domain score) was investigated using Pearson’s product-moment correlation coefficient. There was a significant positive correlation between the two variables (r = .19, n = 329, p < .01 (two-tailed)), indicating that self-care agency behaviour is associated with a growth mindset. In contrast, the Pearson correlation between self-care agency (total score) and a fixed mindset (domain score) was not significant (r = .060, n = 329, p = .278 (two-tailed)). Pearson’s correlation did not show significant correlations between the domains of growth and a fixed mindset with the individual domains of capacity for self-care, developing capacity for self-care, and inability to perform self-care.

    D. Comparative Analysis

    An independent-samples t-test was performed to compare the self-care agency (total score) between pre-clinical and clinical students. The analysis showed a significant difference in the total scores of pre-clinical (M = 54.92, SD = 8.86) and clinical students (M = 56.93, SD = 6.71; t (329) = -2.22, p = .027, two-tailed). Though the magnitude of the differences in the means (mean difference = -2.00, 95% CI: -3.78 to -.23) was small (Cohen’s d = .2), it implicates that the factors that hinder self-care could be further explored and mindset interventions could be targeted for pre-clinical students. However, the independent-samples t-test performed to compare the domain scores of fixed and growth mindsets did not show any significant difference between pre-clinical and clinical students (t (329) = -1.668, p = .096, two-tailed) and (t (329) = .216, p = .829, two-tailed), indicating that their mindsets did not differ in this population. 

    An independent-samples t-test performed to compare the means of self-care agency, fixed mindset, and growth mindset among private and public university students exhibited no significant differences (t (329) = .772, p = .441, two-tailed), (t (329) = -.916, p = .360, two-tailed), (t (329) = -.252, p = .801, two-tailed) indicating that similar stressors and barriers to self-care agency existed in both types of institutions.

    An independent-samples t-test was performed to compare the means of self-care agency, fixed mindset, and growth mindset among genders; no significant differences were found (t (329) = -.58, p = .954 two-tailed), (t (329) = .936, p = .350, two-tailed), (t (329) = .052, p = .959, two-tailed) suggesting that both the genders had similar views on the problems encountered in medical schools which could hinder self-care despite possessing a growth mindset. A one-way ANOVA was performed to compare the means of self-care agency, growth mindset, and fixed mindset with respect to the year of study (Year 1–Year 5). The post-hoc Tukey test also did not reveal any significant differences.

    IV. DISCUSSION

    The total ASAS-R score in this pilot study is comparable to Guo et al.’s (2017) study of older Chinese individuals (55.29 ± 5.22) and Schönenberg et al.’s (2022) study of adults with at least one chronic disorder (52.81 ± 8.39). A study conducted in Brazil among fibromyalgia patients showed similar baseline total scores of 51.9 ± 9.7 and 53.5 ± 11.0 in two samples (Yuan et al., 2021). Oliveira et al. (2022) did not report the total ASAS-R score among Portuguese medical students, but the higher means of Domains I and II and the lower mean of Domain III in this pilot study resonated with their findings which could be further explored by qualitative research. Despite that, Portuguese medical students reported higher mean with Domain II rather than Domain I unlike participants of the current pilot study. This indicates that Portuguese students though lack self-care currently, are positive to be able develop self-care abilities in future. On the other hand, Malaysian students are quite satisfied with their current self-care abilities rather than develop the abilities (Oliveira et al., 2022). The factors for this difference of opinion could be that the Portuguese respondents were of higher age (Mean = 22) and possible cultural factors which could be explored by research as well.  

    Two items in Domain III (items 11 and 14) scored the lowest in this pilot study and can be likened to the study by Guo et al. (2017) who reported that item number 15 in Domain III “I am not always able to care for myself in the way I would like” scored the lowest (2.83 ± 0.93). The opinion that our students also agree that they lack time and energy for self-care are similar to a study conducted at Yale University, and in the United States of America (USA), where medical students neglected self-care and attributed it to lack of time and high stress levels in their academic journey (Ayala et al., 2018; Gold et al., 2015). Similar factors including organisation for self-care, attributable to an intense medical curriculum were voiced out by Portuguese and Malaysian medical students alike (Oliveira et al., 2022; Yusoff et al., 2013). A worldwide analogous opinion among medical students should be taken into account by medical educators to act on easing the academic journey through early detection and targeted interventions.

    In this pilot study, there was no difference in gender in the self-care agency domain, which differs from a study conducted by Ayala et al. (2018), who reported higher stress and self-care activities among female students. However, nursing students in Turkey reported a more optimistic approach to stress management by male students, which correlated with self-care agency among them as well (Komser & Özakgül, 2023). The difference in study results pertaining to gender necessitates a multi-institutional future study with a robust sample size to yield comparable outcomes.  Higher self-care agency among clinical students reported in this study is similar to another Malaysian study by Aida et al (2014). Clinical students initially sought predominantly informal ways (peers, friends, and parents) for health seeking rather than formal channels (counselling and psychiatrists) (Aida et al., 2014). Active coping strategies, reframing (focusing on the process not results, viewing failures as opportunities) and planning as means of self-care was also reported by clinical students (Masilamani et al., 2020).  Students have reported struggle in self-care behaviours within the first 12 weeks of medical school training regardless of their gender in USA as they struggle with poor sleep and physical activity (Boyd et al., 2023). Though it can be hypothesised that older students with more experience and maturity learn self-care behaviours along their student journey, it is still open to research. Since there is no difference in self-care agency between public and private universities similar to other studies, (Ayala et al. 2018), we can deduce that the factors for poor self-care are alike across different institutions.

    In this pilot study, more medical students were found to exhibit a growth mindset rather than a fixed mindset with comparable means with a study among international students in USA, where the growth mindset score was 19.51 ± 1.362, and the fixed mindset score was 14.34 ± 1.557 (Winfrey, 2020) comparable to clinical veterinary students (Guldner et al., 2020; Whittington et al., 2017). There was no difference between fixed and growth mindsets between genders in this pilot study. However, Bostock et al. (2018) reported that more females had a fixed mindset and poorer mental health than their male counterparts. There was no difference in mindset among the students of different years of study in this pilot study. This finding differs from the results of Root Kustritz (2017), who reported that yeartwo veterinary students showed a more growth-oriented mindset, while Bostock et al. (2018) reported that yearfour students had a higher growth-oriented mindset.

    Studies that establish a link between psychological distress and mindset predict overall psychological well-being and better mental health in people with a growth mindset due to their adjustment skills (Whittington et al., 2017). Although there have been limited studies directly correlating growth mindsets and self-care, Orvidas et al. (2018) reported that mindsets lead to self-regulatory actions to help people engage in exercise and healthy eating habits, which are attributed to two of the ten domains of self-care by Ayala et al. (2017). This is due to the fact that growth mindsets are important for increasing the capacity to engage in activities even when challenges arise, as it enables understanding of the value and self-relevance of such activities (Ayala et al., 2017). 

    Individuals with a growth mindset have intrinsic motivation and show resilience, which translates into self-care abilities during periods of distress (Alatorre et al., 2020; Root Kustritz, 2017). Additionally, growth-mindset individuals value personal growth, autonomy, purpose in life, and self-acceptance, which could explain their self-care abilities and psychological well-being (Whittington et al., 2017). Individuals with a fixed mindset are more prone to stress and unhealthy perfectionism, which are linked to suicidal behaviours (Dweck, 2013). A fixed mindset does not believe in change, and self-care to bring about positive change may not be appreciated (Root Kustritz, 2017). A meta-analysis conducted by Burnette et al. (2020) concluded that there is a negative relationship between growth mindset and psychological distress, but a positive correlation with active coping and treatment values, indicating that a growth mindset places value on self-care (Burnette et al., 2020).

    Selection, complexity, reliability and generalisability biases could occur with quota sampling employed in this study. Selection bias was mitigated by calculating the sample size based on the population in the respective medical schools and complexity was avoided by including two protype schools only. The alpha values for the data ensured reliability. Since this was a pilot study the findings were not generalised but were intended to gauge the logistic and statistical feasibility of a nationwide study. Although respondents were informed of their anonymity prior to answering the questionnaire to minimise bias, some respondents might have wanted to demonstrate their positive side and, therefore, present themselves as having self-care abilities and a growth mindset, giving rise to response bias.

    V. CONCLUSION

    The findings of this pilot study indicate that although students are in favour of self-care capacity and its development, they lack of self-care ability in practice, factors of which could be explored qualitatively in future research. The positive correlation between a growth mindset and self-care ability could be utilised in medical curricula by integrating mindset training to enhance self-care capacity among the students. This pilot study also provides justification for a nationwide, multi-institutional global research.

    Notes on Contributors

    RMA was involved in literature search, data acquisition and analysis, manuscript preparation and revision.

    FCC was involved in the study conception and design, data analysis, manuscript editing, and review.

    HMF was involved in the study conception and design and manuscript review.

    MNMN was involved in data acquisition and manuscript revision.

    All the authors approved the final version of the manuscript and its revised versions.

    Ethical Approval

    Ethical approval was obtained from the research ethics committees of both the public and private universities, (UM.TNC2/UMREC_2872 and RMC/SEP/2023/EC02, respectively) in accordance with the Declaration of Helsinki.

    Data Availability

    As per the requirements of the local ethics committees, data will be stored in an enclosed and dedicated facility in the faculty building. The datasets used and/or analysed in the current study are available from the corresponding author upon reasonable request. 

    Acknowledgement

    The authors express their sincere gratitude to all the students who participated in this study.  

    Funding

    The authors declare no sources of funding.

    Declaration of Interest

    The authors declare that they have no competing interests.

    References

    Aida, J., Hizlinda, T., Siti Fatimah, S., Mohd Muhaimin, A., Chai, S. Y., Sahrina, W., & Teo, B. H. (2014). Psychological disorders and help seeking behaviour among Malaysian medical students in their clinical years. Medicine & Health9(2), 114–23. https://www.researchgate.net/publication/272093951

    Alatorre, A. I., DePaola, R. V., & Haeffel, G. J. (2020). Academic achievement and depressive symptoms: Are fixed mindsets distinct from negative attributional style? Learning and Individual Differences, 77, 101811. https://doi.org/10.1016/j.lindif.2019.101811

    Alhambra-Borrás, T., Durá-Ferrandis, E., Garcés-Ferrer, J., & Sánchez-García, J. (2017). The Appraisal of Self-Care Agency Scale – Revised (ASA-R): Adaptation and validation in a sample of Spanish older adults. The Spanish Journal of Psychology, 20, E48. https://doi.org/10.1017/sjp.2017.52

    Ayala, E. E., Omorodion, A. M., Nmecha, D., Winseman, J. S., & Mason, H. R. C. (2017). What do medical students do for self-care? A student-centered approach to well-being. Teaching and Learning in Medicine, 29(3), 237–246. https://doi.org/10.1080/10401334.2016.1271334

     Ayala, E. E., Winseman, J. S., Johnsen, R. D., & Mason, H. R. C. (2018). U.S. medical students who engage in self-care report less stress and higher quality of life. BMC Medical Education, 18(1), 189. https://doi.org/10.1186/s12909-018-1296-x

    Bostock, R., Kinnison, T., & May, S. A. (2018). Mindset and its relationship to anxiety in clinical veterinary students. Veterinary Record, 183(20), 623–623. https://doi.org/10.1136/vr.104621

    Boyd, A., Mealand, K., Briggs Early, K., & Oestreich, E. (2023). Perceived stress, grit, and self-care behaviours in first-year medical students. American Journal of Lifestyle Medicine17(6), 803-812. https://doi.org/10.1177/15598276221124576

    Burnette, J. L., Knouse, L. E., Vavra, D. T., O’Boyle, E., & Brooks, M. A. (2020). Growth mindsets and psychological distress: A meta-analysis. Clinical Psychology Review, 77, 101816. https://doi.org/10.1016/j.cpr.2020.101816

    Carter, A. N., Evans, J., Seibert, S. A., & Weir, S. (2025). Growth mindset enhancement for new nursing students. Teaching and Learning in Nursing20(1), e258-e262. https://doi.org/10.1016/j.teln.2024.10.023

    Cook, D. A., Castillo, R. M., Gas, B., & Artino Jr., A. R. (2017). Measuring achievement goal motivation, mindsets and cognitive load: Validation of three instruments’ scores. Medical Education, 51(10), 1061–1074. https://doi.org/10.1111/medu.13405

    Cook, D. A., Gas, B. L., & Artino, Jr., A. R. (2018). Measuring mindsets and achievement goal motivation: A validation study of three instruments. Academic Medicine, 93(9), 1391–1399. https://doi.org/10.1097/ACM.0000000000002290

    Damásio, B. F., & Koller, S. H. (2013). The Appraisal of Self-Care Agency Scale – Revised (ASAS-R): Adaptation and construct validity in the Brazilian context. Cadernos de Saúde Pública, 29(10), 2071–2082. https://doi.org/10.1590/0102-311X00165312

    Dweck, C. S. (2013). Self-theories: Their role in motivation, personality, and development. Psychology Press.

    Evers, G. C. (1989). Appraisal of Self-Care Agency ASA-scale: Reliability and validity testing of the Dutch version of the ASA-scale measuring Orem’s concept’ Self-Care Agency’. [Doctoral thesis, Maastricht University]. https://doi.org/10.26481/dis.19890629ge

    Gold, J. A., Johnson, B., Leydon, G., Rohrbaugh, R. M., & Wilkins, K. M. (2015). Mental health self-care in medical students: A comprehensive look at help-seeking. Academic Psychiatry, 39(1), 37–46. https://doi.org/10.1007/s40596-014-0202-z

    Guldner, G., Siegel, J., Ellis, B., & Brafford, A. (2020, February 27–29). The role of mindset, impostorism, and irrational beliefs in resident wellness: Results from the HCA Nationwide Longitudinal Resident Wellness Study [Poster presentation]. ACGME 2020 Annual Educational Conference, San Diego, CA. https://scholarlycommons.hcahealthcare.com/teaching-learning/5/ 

    Guo, L., Zauszniewski, J. A., Ding, X., Zhang, L., Gao, H., Guo, Q., & Liu, K. (2017). The Appraisal of Self-Care Agency Scale – Revised (ASAS-R): Reliability and validity among older Chinese people. Western Journal of Nursing Research, 39(11), 1459–1476. https://doi.org/10.1177/0193945916672821

    Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). Pearson Prentice Hall.

    Hong, Y. Y., Chiu, C. Y., Dweck, C. S., Lin, D. M. S., & Wan, W. (1999). Implicit theories, attributions, and coping: A meaning system approach. Journal of Personality and Social Psychology, 77(3), 588–599. https://doi.org/10.1037/0022-3514.77.3.588

    Kim, H. Y. (2013). Statistical notes for clinical researchers: Assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics, 38(1), 52–54. https://doi.org/10.5395/rde.2013.38.1.52 

    Komser, N., & Özakgül, A. (2023). Attitudes of coping with stress and self-care agency levels of nursing students. Journal of Education and Research in Nursing, 20(1), 28–34. https://doi.org/10.5152/jern.2023.21004

    Lowe, N. K. (2019). What is a pilot study? Journal of Obstetric, Gynecologic & Neonatal Nursing, 48(2), 117–118. https://doi.org/10.1016/j.jogn.2019.01.005

    Masilamani, R., Jabbar, M. A., Liang, C. S., You, H. L. S., Jonathan, L. J. K., Pei-Suen, W., Yuan, Y.X. & Ling, Y. M. (2020). Stress, stressors, and coping strategies between pre-clinical and clinical medical students at Universiti Tunku Abdul Rahman. Malaysian Journal of Public Health Medicine20(1), 175-183. https://doi.org/10.37268/mjphm/vol.20/no.1/art.503

    Oliveira, L., Teixeira, A., & Duarte, I. (2022). The Appraisal of Self-Care Agency Scale – Revised (ASAS-R): Reliability and validity among Portuguese medical students. International Journal of Environmental Research and Public Health, 19(17), 10848. https://doi.org/10.3390/ijerph191710848

    Orem, D. E., Taylor, S. G., & Renpenning, K. M. (1995). Nursing: Concepts of practice. Mosby.

    Orvidas, K., Burnette, J. L., & Russell, V. M. (2018). Mindsets applied to fitness: Growth beliefs predict exercise efficacy, value and frequency. Psychology of Sport and Exercise, 36, 156–161. https://doi.org/10.1016/j.psychsport.2018.02.006

    Root Kustritz, M. V. (2017). Pilot study of veterinary student mindset and association with academic performance and perceived stress. Journal of Veterinary Medical Education, 44(1), 141–146. https://doi.org/10.3138/jvme.1115-181R1

    Schönenberg, A., Teschner, U., Prell, T., & Mühlhammer, H. M. (2022). Validation and psychometric analysis of the German translation of the appraisal of self-care agency scale-revised. Healthcare, 10(9), 1785. https://doi.org/10.3390/healthcare10091785

    Sousa, V. D., Zauszniewski, J. A., Bergquist-Beringer, S., Musil, C. M., Neese, J. B., & Jaber, A. F. (2010). Reliability, validity and factor structure of the Appraisal of Self‐Care Agency Scale – Revised (ASAS‐R). Journal of Evaluation in Clinical Practice, 16(6), 1031–1040. https://doi.org/10.1111/j.1365-2753.2009.01242.x

    Sun, X., Nancekivell, S., Gelman, S. A., & Shah, P. (2021). Growth mindset and academic outcomes: A comparison of US and Chinese students. NPJ Science of Learning, 6(1), 21. https://doi.org/10.1038/s41539-021-00100-z

    Theard, M. A., Marr, M. C., & Harrison, R. (2021). The growth mindset for changing medical education culture. eClinicalMedicine, 37, 100972. https://doi.org/10.1016/j.eclinm.2021.100972 

    Thomas, F. N., Burnette, J. L., & Hoyt, C. L. (2019). Mindsets of health and healthy eating intentions. Journal of Applied Social Psychology, 49(6), 372–380. https://doi.org/10.1111/jasp.12589

    Wan, C. D. (2007). Public and private higher education institutions in Malaysia: Competing, complementary or crossbreed as education providers. Kajian Malaysia, 25(1), 1–14. https://www.researchgate.net/publication/49583836

     Whittington, R. E., Rhind, S., Loads, D., & Handel, I. (2017). Exploring the link between mindset and psychological well-being among veterinary students. Journal of Veterinary Medical Education, 44(1), 134–140. https://doi.org/10.3138/jvme.1215-192R

    Winfrey, S. E. (2020). Validating the mindset scale for use with international students attending college in the United States [Doctoral dissertation, North Dakota State University]. https://hdl.handle.net/10365/32293  

    Wolcott, M. D., McLaughlin, J. E., Hann, A., Miklavec, A., Beck Dallaghan, G. L., Rhoney, D. H., & Zomorodi, M. (2021). A review to characterise and map the growth mindset theory in health professions education. Medical Education, 55(4), 430–440. https://doi.org/10.1111/medu.14381

    Yuan, S. L. K., Couto, L. A., & Marques, A. P. (2021). Effects of a six-week mobile app versus paper book intervention on quality of life, symptoms, and self-care in patients with fibromyalgia: A randomized parallel trial. Brazilian Journal of Physical Therapy, 25(4), 428–436. https://doi.org/10.1016/j.bjpt.2020.10.003

    Yusoff, M. S. B., Abdul Rahim, A. F., Baba, A. A., Ismail, S. B., Mat Pa, M. N., & Esa, A. R. (2013). The impact of medical education on psychological health of students: A cohort study. Psychology, Health & Medicine, 18(4), 420–430. https://doi.org/10.1080/13548506.2012.740162

    Yusoff, M. S. B., Yee, L. Y., Wei, L. H., Meng, L. H., Bin, L. X., Siong, T. C., & Rahim, A. F. A. (2011). A study on stress, stressors and coping strategies among Malaysian medical students. International Journal of Students’ Research, 1(2), 45–50. https://www.researchgate.net/publication/50944546_A_study_on_stress_stressors_and_coping_strategies_among_Malaysian_medical_students

    *Foong Chan Choong
    Medical Education and Research Development Unit,
    Faculty of Medicine, Universiti Malaya, Malaysia
    +0060 12-419 1248
    Email: foongchanchoong@um.edu.my

    Submitted: 1 January 2025
    Accepted: 12 August 2025
    Published online: 7 October, TAPS 2025, 10(4), 44-54
    https://doi.org/10.29060/TAPS.2025-10-4/OA3613

    Dinushee Atapattu-Bakmeewewa1, Bhagya Devagiri1, Gayanthi Kodituwakku1 & Madawa Chandratilake2

    1Department of Disability Studies, Faculty of Medicine, University of Kelaniya, Sri Lanka;  2Department of Medical Education, Faculty of Medicine, University of Kelaniya, Sri Lanka

    Abstract

    Introduction: Tele-clinical training is an effective approach, increasingly adopted post-pandemic and in resource-limited settings. However, it requires systematic development. This study details the first-time implementation of a tele-clinical training in an undergraduate Speech and Hearing Sciences programme, exploring student experiences and validating a novel evaluation tool, specifically designed for tele-clinical training programmes.

    Methods: The study used a mixed-method approach. Quantitative data were gathered from 128 students using the developed 23-item Kelaniya Tele-Clinical Evaluation Tool (KeTCET), which covers three broad areas: Learning Environment, Supervisory Attributes, and Telehealth Teaching Practices. Qualitative insights from 13 participants were thematically analysed. The tool was validated for reliability and psychometric robustness using expert feedback and statistical evidence.

    Results: Quantitative analysis showed high domain scores: Learning Environment 80.64%, Supervisory Attributes 81.67%, and Telehealth Teaching Practices 80.31%. Strong positive correlations between domains (r > 0.86, p < 0.001) indicated interconnectedness. The 23-item evaluation tool demonstrated high internal consistency (Cronbach Alpha = 0.98) and a single-factor structure (Eigenvalue = 17.12, 74.44% variance explained). Qualitative data highlighted strengths in supervisor interaction and resource availability, also noting challenges such as issues in connectivity and limited peer learning. Students appreciated structured feedback and supervisor presence during tele-clinical sessions.

    Conclusions: The successful development of a tele-clinical programme requires consideration of multiple elements broadly categorised as pedagogical environment, supervisory characteristics, and virtual teaching practices. Well-structured programmes can effectively meet training needs in resource-limited settings, although strengths and challenges may vary across learning environments. The validated 23-item KeTCET offers a reliable framework for evaluating and improving tele-clinical programmes.

    Keywords:           Tele-clinical Programmes, Tele-clinical Supervision, Speech and Language Therapy, Audiology, Tele-clinical Evaluation, Tool Development, Undergraduate Clinical Training, KeTCET

    Practice Highlights

    • Tele-Practice based clinical programmes can be delivered successfully even in countries with limited resources.
    • Robust planning is required to ensure that tele-clinical programmes address specific training needs and satisfy student expectations.
    • Incorporating virtual clinical training modules into the regular curriculum can effectively address some of the barriers students have identified in tele-clinical learning.
    • The unique features of tele-clinical training should guide the formulation of tailored guidelines and supervisory models for the virtual format.
    • The evaluation of tele-clinical programmes is most effective when customised tools are developed to incorporate elements of the virtual format of training.

    I. INTRODUCTION

    The COVID-19 pandemic necessitated a global shift to online platforms in healthcare and education. Higher education institutions implemented online teaching methods, including webinars and interactive sessions (Hameed et al., 2020), while healthcare education adopted tele-medicine as a platform to continue clinical training, ensuring graduate preparedness for professional practice. Speech and language therapy and audiology professionals swiftly utilised tele-therapy, a method previously endorsed by the American Speech-Language-Hearing Association (ASHA, n.d.). This model of service delivery employs synchronous, asynchronous, and hybrid methods, tailored to patient needs and available resources. Beyond the pandemic, tele-medicine programmes offer enduring benefits, including improved access to care for remote and underserved populations, enhanced patient satisfaction and cost-effectiveness (Car et al., 2020). The World Health Organization emphasises the role of tele-medicine in broadening healthcare access, especially for people in remote areas and underserved communities (WHO, 2022).

    Tele-clinical programmes, that is, clinical training conducted using tele-therapy have proven to be an effective method in clinical teaching and skills training. Considerable evidence demonstrates the tele-clinical programmes can be successfully used for clinical training, not limiting it to practical skills (Anderson et. al., 2023) but also addressing attitudinal changes (Wearne et. al., 2015) and clinical soft skills (Bramstedt et. al., 2014; Liu et. al., 2022).

    Clinical supervision, distinct from classroom teaching, involves case-based learning, critical thinking, and professional behaviour modelling (Council of Academic Programmes in Communication Sciences and Disorders (CAPCSD), 2013). Traditionally conducted face-to-face clinical supervision shifted to tele-clinical supervision during the pandemic, utilizing a variety of strategies (Shawwa, 2023). However, much of the available literature on tele-supervision does not clearly state whether important aspects of clinical teaching, such as supervisor–student relationships and feedback (Kilminster & Jolly, 2000), were considered during programme design.

    Evidence suggests that the effectiveness of tele-supervision depends significantly on the qualities of both the supervisor and the supervisee (Martin et al., 2017). Positive supervisory relationships, characterised by structure and support (Martin et al., 2014) are achievable in both virtual and face-to-face formats (Reese et al., 2009). Effective tele-supervision also depends on communication strategies, supervisor availability, and feedback models (Gibson et al., 2007; Snowdon et. al., 2019). Technological barriers, such as poor connectivity and equipment issues, can hinder outcomes, necessitating proactive solutions (Martin et al., 2017; Reese et al., 2009; Tarlow et al., 2020).

    Systematic evaluation of student experiences in tele-clinical programmes is essential to understand their effectiveness. Most studies that report on tele-clinical programmes in allied health sciences (e.g., Bacon et al., 2023; Snowdon et al., 2019) and medicine (e.g,. Pit et al., 2021; Wearne et al., 2015) rely primarily on qualitative interview data. Ideally, such qualitative data should be supplemented with the use of a validated and reliable tools specifically designed to evaluate a virtual model of clinical training. This ensures that tele-clinical learning meets the professional standards and training requirements of audiology and speech and language therapy. The objectives of this study were to evaluate student experiences in an undergraduate Speech and Hearing Sciences tele-clinical training programme, and to validate a feedback tool to evaluate similar tele-clinical programmes.

    II. METHODS

    A. Context and Setting

    The BSc (Hons) Speech and Hearing Sciences is a full-time, four-year undergraduate programme. The qualification allows graduates to practice as speech and language therapists or audiologists following registration at the national medical regulatory body. The programme consists of theoretical classroom-based sessions, synchronous to intensive supervised clinical training offered across the four years of study. Speech and Language Therapy (SLT) undergraduates are trained to work with communication and swallowing disorders while audiology undergraduates train in the detection and management of hearing loss, across the life span.

    The data gathered in this study reflects experiences from the pandemic period, when clinic and hospital-based teaching was significantly limited. For almost two and a half years since the commencement of the pandemic, SLT and audiology service provision shifted fully into a tele-therapy programme. Simultaneously, students were enrolled in a tele-clinical programme, offered two to three times per week, where they worked with the patient under the supervision of an academic or qualified clinician. All sessions took place using the Zoom platform, which was made freely available to students by the university. Tele-therapy for adult patients requiring SLT services was delivered primarily using a synchronous method and for paediatric clients using an asynchronous or a mixed method. Audiology clinical services were primarily synchronous. The tele-clinical training was designed to align with the method of tele-therapy (Table 1).

    Format

    Synchronous Method

    Asynchronous Method

    Hybrid Method

    Tele-therapy

    Conducted in real time using audio or video interactive sessions.

    Clinical management through stored images and captured data.

    Combines both synchronous and asynchronous methods.

    Tele-clinical Supervision

    Students conduct session. Supervisor joins in. Feedback is provided simultaneously and after session.

    Students join session. Supervisor shares recorded videos and relevant clinical information/documentation. Students are given time to reflect.

    Students conduct session. Supervisor joins in. Following a short real-time session, a recorded video is watched together.

    A discussion follows.

    Table 1. Modes of Tele-therapy and tele-supervision delivery

    The general arrangement of a synchronous session was that the patient, student and supervisor joined the session at a mutually agreed time, but from three separate locations. Material for the therapy session, if required, was developed and shared on the screen by the student. For paediatric clients, parents arranged the toys needed. In audiology, students gathered patient data through interviews and questionnaires with limited use of conventional hearing tests. Auditory verbal training  (AVT sessions) in audiology followed a similar format to SLT synchronous sessions. The supervisor remained a silent observer unless intervention was required. In all sessions, supervision concluded with an interactive patient discussion, facilitated by the supervisor using Zoom features such as whiteboard, break out rooms etc.

    The evaluation of the tele-clinic programme was conducted using mixed methods by collecting students’ perceptions quantitatively (Phase I) and qualitatively (Phase II).

    B. Phase 1- Development of Evaluation Tool and Quantitative Feedback

    The quantitative evaluation of the tele-clinical programme was conducted with the aim of developing and validating a standardised tool for evaluating similar programmes.

    1. Tool Development

    As there are no existing tools available to evaluate SLT or audiology clinical programmes, whether face-to-face or virtual, a new evaluation tool was developed based on the Clinical Learning Environment, Supervision and Nurse Teacher evaluation scale (CLES +T ) (Mikkonen et al., 2017) and the Nursing Clinical Facilitator Questionnaire (NCFQ) (Espeland & Indrehus, 2003). The resulting 23-item tool, named the Kelaniya Tele-Clinical Evaluation Tool (KeTCET), was designed to map onto three primary domains: pedagogical/learning environment (LE; 9 items), supervisory relationship (SA; 6 items), and telehealth teaching practices (TTP; 8 items). The stem question used here was, how often did you experience this aspect in the online clinical learning sessions provided for the SHS programme during university closure? (Table 2). Participants rated each item on a 5-point Likert scale (0 – never, 1 – rarely, 2 – sometimes, 3 – often and 4 – always).

    To enhance face and content validity, a panel of 10 experts in speech and language therapy and/or audiology  rated the tool items on a five-point scale for [a] content appropriateness, [b] relevance, and [c] technical accuracy (1 = Very Low, 5 = Very High). Experts could also provide comments to refine the items. The panel scored the items high across all three aspects (mean[a]=4.8; mean[b]= 4.8; mean[c]=4.7). A measure of item relevance, I-CVI (Item- Content validity Index) scores for all items (n=23) were > 0.9. Minor language adjustments suggested were incorporated. The tool was then translated into Sinhala and Tamil and pre-tested with five students (three Sinhala speakers and two Tamil speakers) to confirm clarity and translation accuracy.

    2. Study Participants

    All 155 SLT undergraduate students in the SLT and audiology programmes who had attended at least 80% of the tele-clinical training sessions were considered eligible for participation in phase I. All eligible students were invited to participate in the study. At the time of data collection, these students were in their second, third, and fourth years of study. The minimum sample size required was calculated based on the recommended item-to-response ratio of 1:5 for factor analysis (Bujang et al., 2012; Gorsuch, 1983), requiring at least 115 responses. A total of 128 students responded (82.6%) to phase I.

    Learning Environment

    1.     Professionalism and mutual trust

    2.     Enabling identity formation and promoting learning

    3.     Developing relationships with supervisor and peers

    4.     Optimised logistics and access to an interactive virtual learning platform

    5.     Mechanism for constructive and timely feedback

    6.     Encouraging autonomy in learning

    7.     Promoting teamwork

    8.     Equity and equal opportunity to participate and learn

    9.     Known session structure

    Supervisor Attribute

    1.     Expertise

    2.     Ability to integrate taught content with remote clinical learning

    3.     Supervision skills including timely feedback

    4.     Communication skills to suit virtual training

    5.     Preparation

    6.     Familiarity (knowing the supervisor through face-to-face contact priorly

    Telehealth Teaching Practices

    1.     Patient care and rapport building with the patient with a virtual space

    2.     Learning with virtual clinical encounters

    3.     Dedicated or adapted resources to suit virtual learning

    4.     Clinical documentation development and maintenance for virtual learning

    5.     Creating meaningful learning situations

    6.     Supervision and personalised attention

    7.     Peer learning

    8.     Competency marking for virtual learning/ Adapted assessment methods

    Table 2. List of 23 items included in the developed tool

    3. Data Collection

    The participant information sheet and the online-converted 23-item tool were disseminated to participants through a link shared by an independent assistant lecturer, who was not a teacher on the programme, in order to avoid bias and any undue pressure to participants. In the first section of the online response form, participants provided written, informed consent by clicking on the ‘I agree to participate’ icon. At the time of evaluation, all students had received a minimum of 18 months training through the tele-clinical programme.

    4. Data Analysis

    The reported frequency of student experience was dichotomised as ‘never to sometimes’ (0-2) and ‘often or always’ (3 and 4). The initial analysis involved generating item-wise dichotomised frequencies to identify the aspects most frequently experienced in the offered programme. Subsequently, evidence supporting the validity and reliability of the evaluation tool was obtained through responses, assessed using internal consistency (Cronbach’s alpha), correlations between subjectively identified domains, and exploratory factor analysis.

    C. Phase II – Qualitative evaluation of the Programme

    1. Participants

    In Phase II, 10% of the population (n= 13) who participated in Phase I of the study were purposively selected. These students represented the socio-demographic and educational characteristics of the population.

    2. Data Collection

    The selected participants were invited to participate in a focus group discussion, which was conducted by the researchers in native languages. The discussion lasted for approximately 65 minutes. It was audio-recorded and transcribed verbatim.

    3. Data Analysis

    Data analysis was guided by the procedure outlined by Braun and Clarke (2006). Transcripts were first coded by two team members (GK and BD) and reviewed by the third (DB). Data collection and analysis happened synchronously, where new codes were identified after each interview. Thematic analysis was inductively performed; themes were not identified a priori but emerged from the data.  These themes reflected the subjective domains of the questionnaire but were not limited to them, allowing for the exploration of novel insights.

    III. RESULTS

    The findings are reported in terms of participants’ characteristics, students’ perceptions about the programme, and the psychometric properties of the evaluation tool.

    A. Participants Characteristics

    A total of 128 responded to phase I of the study (82.6%); 122 females and 6 males. The mean age was 24.43 years (SD= l4.24). 98 students were from the Speech and Language Therapy programme and 30 were from the Audiology programme. Out of the respondents, 48 were in their second year, 49 in their third year and 31 in their final year. The composition of the 13 students who participated in the focus group discussion is as follows: 12 females and 1 male student; four students from the second year, four students from the third and five students from the final year.

    B. Perception about the Tele-clinical Programme

    Quantitative analysis showed that the tele-clinical programme achieved high average scores (>80%) across all domains. Teachers appeared to have fostered professionalism and equity in the virtual learning environment, prepared well, and brought in meaningful learning situations. However, they may need to focus on building better familiarity with the student, encouraging peer learning and reflecting on strategies to better develop clinical skills in the virtual learning environment (Table 3).

    Domain

    Max Domain
    Score

    Mean Score
    (SD)

    % Score

    High-Scoring
    Attributes

    Low-Scoring
    Attributes

    Learning
    Environment
    (LE)

    36

    29.03
    (7.6)

    80.64%

    Professionalism,
    equity

    Supervisor familiarity

    Supervisory
    Attributes (SA)

    24

    19.6
    (5.3)

    81.67%

    Supervisor
    preparation

    Facilitating peer
    learning

    Telehealth Teaching
    Practices (TTP)

    32

    25.7
    (7.04)

    80.31%

    Meaningful
    learning
    situations

    Clinical skill
    development via
    virtual encounters

    Table 3. Domains-level perception scores

    In the correlational analysis, a strong interconnectedness between the three domains was observed which suggests that improvements or strengths in one domain are likely to support and enhance the effectiveness of the others (Table 4).

    Domains

    Learning Environment
    (LE)

    Supervisory Attributes
    (SA)

    Telehealth Teaching
    Practices (TTP)

    Learning Environment
    (LE)

    1

    0.876

    (p < 0.001)

    0.881

    (p < 0.001)

    Supervisory Attributes
    (SA)

    0.876

    (p < 0.001)

    1

    0.863

    (p < 0.001)

    Telehealth Teaching
    Practices (TTP)

    0.881

    (p < 0.001)

    0.863 (p < 0.001)

    1

    Table 4. Correlation between subjective domains of the evaluation tool

    In the correlational analysis, a strong interconnectedness between the three domains; LE, SA and TTP was observed which suggests that improvements or strengths in one domain are likely to support and enhance the effectiveness of the others (Table 3).

    The qualitative data highlight both the strengths and challenges of the tele-clinical programme. Participants in the tele-clinical programme highlighted various experiences across the three subjective domains, Learning Environment (LE), Supervisory Attributes (SA), and Telehealth Teaching Practices (TTP). These findings help explain the pattern of rating of items observed in the qualitative analysis.

    Under LE, students appreciated the time supervisors took to interact with them, fostering a sense of connection. “The interaction with the lecturer was good. We had an opportunity for that” (P17). However, many noted that the lack of structure in sessions hindered effective task management. “If it were more structured, and if we had a better plan to submit documents within like two hours after the session, that would have been ideal” (P94). Virtual clinics also presented environmental challenges, with frequent disruptions due to background noise or technical issues. “Sometimes there was so much noise we couldn’t focus” (P52).  Additionally, students had to creatively adapt therapy methods for the virtual format, often requiring supervisor feedback. “We really had to think of different ways to test and manage hearing issues” (P49).

    In the SA domain, participants valued supervisors who provided context before and after sessions, which clarified the learning process. “Supervisors gave us a description about the client before they came into the session and then did the same after the session” (P3). Supervisory styles had a significant impact on student confidence. For example, students noted that when supervisors turned on their video cameras, their visible presence positively influenced their performance.

    “Some supervisors turned on their videos. It made us feel confident” (P23). Students also expressed a need for independent practise opportunities, even within the limitations of tele-clinics. “Supervisors allowed us to do exactly what we did in FTF sessions” (P19).

    For TTP, students appreciated resources like a shared material library, which facilitated session preparation. “The best part of it was the material library that the staff made for us” (P12). However, connectivity issues, such as poor internet connections and power outages, often disrupted sessions. “It was terrible when my clinical partner had a very poor connection” (P53). Technical limitations, such as using small phone screens or faulty laptops, further impeded learning. “Some didn’t have laptops and used phones. The screen is small so we can’t see” (P19). Patient-related factors, like poor camera positioning or noisy environments, added stress to students. “Parents kept the tab on a table, then sat on the floor to play. So, we couldn’t see anything” (P23).

    In summary, it appeared that while students valued interaction, feedback, and innovative resources, they faced issues with session structure, connectivity, and technical limitations. Supervisory presence and adaptability were crucial for building confidence and overcoming challenges.

    Item

    Component 1

    SA3

    Supervision skills including timely feedback

    .923

    SA2

    Ability to integrate taught content with remote clinical learning

    .900

    SA1

    Expertise

    .893

    LE5

    Mechanism for constructive and timely feedback

    .888

    SA4

    Communication skills to suit virtual training

    .884

    TTP6

    Supervision and personalised attention

    .880

    LE8

    Equity and equal opportunity to participate and learn

    .876

    TTP5

    Creating meaningful learning situations

    .874

    LE3

    Developing relationships with supervisor and peers

    .871

    SA5

    Preparation

    .871

    TTP1

    Patient care and rapport building with the patient with a virtual space

    .864

    TTP4

    Clinical documentation development and maintenance for virtual learning

    .861

    TTP2

    Learning with virtual clinical encounters

    .859

    LE2

    Enabling identity formation and promoting learning

    .858

    LE6

    Encouraging autonomy in learning

    .856

    TTP7

    Peer learning

    .856

    TTP3

    Dedicated or adapted resources to suit virtual learning

    .851

    LE7

    Promoting teamwork

    .851

    LE9

    Known session structure

    .850

    TTP8

    Competency marking for virtual learning/ Adapted assessment methods

    .842

    SA6

    Familiarity (knowing the supervisor through face-to-face contact priorly

    .836

    LE1

    Professionalism and mutual trust

    .812

    LE4

    Optimised logistics and access to an interactive virtual learning platform

    .775

    a Extraction Method: Principal Component Analysis only one component was extracted. Cannot be rotated.

    Table 5. The Principal Component Analysis of the 23 items of the evaluation tool

    C. The Psychometric Properties of the Tool

    The internal consistency of the 23 items, as measured by Cronbach’s alpha, was very high (α = 0.98). While a high alpha value may indicate internal consistency, it can also suggest item redundancy. To assess this, inter-item and item-total correlations were examined. All items showed acceptable item-total correlations (>0.3), suggesting minimal redundancy. Although only one factor was extracted in the principal component analysis (Eigenvalue = 17.12), varimax rotation was initially applied during the exploratory analysis phase to evaluate whether multiple factor structures might emerge. This step was performed prior to confirming the single-factor solution. Since all items loaded strongly (>0.7) onto a single component and no additional eigenvalues exceeded 1 (Table 5), the use of rotation was ultimately deemed unnecessary, and only the unrotated solution is reported. Although the tool was originally structured around three subdomains (Learning Environment, Supervisory Attributes, and Telehealth Teaching Practices), exploratory factor analysis revealed a single latent factor structure. This suggests that in the context of tele-clinical learning, these domains may not function as distinct constructs but rather as interrelated facets of a unified student experience. While this does not contradict theoretical expectations, it highlights the integrated nature of tele-clinical learning, where pedagogical, supervisory, and teaching practice components coalesce in a single virtual training environment.

    The data gathered for this study and analysed above can be accessed by readers for viewing purposes only, from the Figshare data repository at https://doi.org/10.6084/m9.figshare.28116863 (Atapattu-Bakmeewewa et. al., 2025).

    IV. DISCUSSION

    This study evaluated undergraduate SLT students’ experiences in a tele-clinical programme revealing positive outcomes with domain scores exceeding 80%. Qualitative insights highlighted professionalism, equity and meaningful learning to be the strengths of this programme. The validated 23-item tool demonstrated strong psychometric properties, with high reliability (α = 0.98) and a single-factor structure, supporting its adaptability.

    A. Student Experiences in a Tele-Clinical Programme

    Although prior studies have shown a preference for face-to-face clinical teaching (Bacon et al., 2023), findings from our study add to growing evidence that support a shift in thinking. Our tele-clinical programme was implemented over an extended period and was well-established at the time of evaluation. This may have contributed to higher acceptance scores reported. Evaluating the effectiveness of virtual clinical training has often relied on either qualitative research (e.g., Gammon et al., 1998; Gibson et al., 2007) or quantitative surveys (e.g., Heckner & Giard, 2005). This study employed a robust mixed-methods approach, analysing quantitative data from 128 participants and complementing it with qualitative insights from 13 randomly selected individuals. High ratings across the 23 evaluated items, with over 80% agreement, suggest that delivering an effective tele-clinical programme is feasible, even in resource-limited contexts. Qualitative findings, however, highlight the importance of thorough planning and holistic design, also the need to integrate elements from multiple domains.

    Our findings indicated that students had similar expectations in the virtual programme as those in face-to-face training, particularly support for developing online materials. Learning material such as scaffolds and scripts have been identified by students as enablers of tele-clinical learning (Bacon et al., 2023). Gracious et al. (2024) report that the versatility of virtual environments may at times lead to unrealistic expectations, such as improved grades or increased institutional support. Unmet expectations may in turn be associated with dissatisfaction with the virtual tele-clinical concept. We therefore comprehensively discussed student expectations before the programme delivery to enhance the acceptance and effectiveness of virtual clinical programmes. Technical disruptions, reduced reading of non-verbal cues, background noise, patient camera placement and limited IT literacy were shared challenges (See, Gibson et al., 2007; Tarlow et al., 2020). Training (Pit et al., 2021) and pre-session briefings (Heckner & Giard, 2005) are considered effective strategies to mitigate such barriers.

    There is evidence that supervisory familiarity, that is prior supervisor contact, improves outcomes in tele-clinical programmes (Martin et al., 2018). Supervisor familiarity was included as an item in our tool but was not a high-scoring attribute possibly because our tele-clinical programme was taught by permanent academic staff, already familiar to the students. As a result, students may have focused more on the other attributes. Participants, however, emphasised the need for supervisory traits that foster supervisor-student engagement. This aligns with findings from Reese et al. (2009), who reported no significant differences in supervisory satisfaction between virtual and face-to-face formats, if the supervisor maintained a supportive attitude.

    Students in our study not only advocated for equal participation and autonomy within the tele-clinical programme (Gracious et al., 2024; Tarlow et al., 2020) but also used it as a descriptor when differentiating between different supervisory styles. This reflects findings by Miller and Gibson (2004) who emphasised the importance of power balance and involvement in clinical supervision, which may hold even greater significance in virtual settings. The study further suggests that the successful delivery of tele-clinical programmes depends on trainee characteristics; more mature students or those with prior face-to-face experience, may adapt better (Martin et al., 2023; Reese et al., 2009). Integrating virtual clinical modules into undergraduate curricula presents a viable strategy for providing students with essential tele-clinical experience. This would additionally address evolving training demands, support the development of competencies among future therapists and contribute to a sustainable transformation in patient access to healthcare services (Iancu et al., 2020; Jeffries et al., 2022).

    Our findings indicate that educators overlooked certain elements, such as promoting peer learning. This highlights the importance of robust planning in tele-clinical practice. Without it, critical elements such as opportunities for continuing professional development (CPD), skills around ethics, concepts of multidisciplinary collaboration and patient and family advocacy may be inadvertently overlooked, especially in simulated environments (Jeffries et al., 2022). Such elements if missed, can lead to a potentially negative impact on the long-term professional growth of learners. Recognizing and addressing the pitfalls in tele-clinical practice, as applicable to the setting in which it is delivered, is a crucial step to optimizing its effectiveness. Tutors must assess training needs, patient suitability, human resources, and available infrastructure for both teachers and trainees, during programme development. Research shows that integrating synchronous (live) and asynchronous (self-paced) learning helps make tele-clinical programmes more effective (Perle & Zheng, 2024; Snowdon et al., 2019).

    The unique features of tele-clinical training should guide the formulation of context-specific guidelines and supervisory frameworks (Gibson et al., 2007), taking in to account the unique training requirements of allied-health professions such as speech and language therapy, audiology, occupational therapy, and physiotherapy (Bacon et al., 2023), all of which require a combination of direct and reflective supervision.

    B. Development of the Evaluation Tool

    Our findings suggest that the success of a tele-clinical programme relies on the integrated consideration of the pedagogical environment, supervisory attributes and virtual teaching practices. Data showed these aspects appeared to be highly complementary to each other as the statistical analyses strongly suggest that they are highly interconnected and strongly correlated. The 23-item single-domains tool, which we wish to name as KeTCET (Kelaniya Tele-Clinical Evaluation Tool), has provided basic but strong psychometric evidence as a tool for evaluating tele-clinical programmes. The KeTCET aligns closely with established practices in educational tool design and draws on the strengths of existing instruments in clinical education evaluation. Its 23-item structure is consistent with tools like the (CLES+T) scale, which features 34 items across subdomains such as pedagogical environment and supervisory attributes (Saarikoski et al., 2008), and the Manchester Clinical Supervision Scale (MCSS), which comprises 26 items to evaluate supervision quality (Winstanley & White, 2011). The compact structure of KeTCET balances comprehensiveness and practicality, making it an efficient yet thorough evaluation tool.

    KeTCET’s development followed a rigorous validation process involving item selection from already existing validated instruments and expert feedback for content appropriateness, cultural relevance, and technical accuracy. This aligns with recommended methodologies for reliable tool development (DeVellis & Xie, 2021). The tool underwent pilot testing to ensure clarity and relevance, a process comparable to the development of other notable tools like the Surgical Mini-CEX and the Physician Work Environment Survey (Friedberg et al., 2014; Norcini et al., 2003). KeTCET demonstrated high internal consistency (α = 0.98), surpassing the widely accepted reliability benchmark (Nunnally, 1978) and factor analysis confirmed a single-factor structure (Eigenvalue = 17.12, 74.44% variance explained), supporting its psychometric robustness (Kline, 1999).

    The domains assessed by KeTCET, pedagogical environment, supervisory traits, and virtual teaching practices, mirror the constructs of established tools but are uniquely tailored to address the challenges of tele-clinical education. By integrating these domains synchronously, KeTCET effectively evaluates the complexities of virtual supervision, bridging a gap left by tools primarily designed for face-to-face settings. Its strong psychometric properties establish it as a reliable and effective instrument for assessing tele-clinical programmes, particularly in speech and hearing sciences.

    While existing tools like CLES+T and MCSS are used successfully to evaluate traditional clinical education and supervision, KeTCET extends this utility to tele-clinical settings. Its tailored approach involving a synchronous integration of pedagogical elements, supervisory attributes, and virtual teaching practices positions it as a highly appropriate tool for evaluating tele-clinical programmes. The initial psychometric evidence supporting KeTCET underscores its potential to advance the evaluation of tele-clinical supervision, ensuring robust assessments that inform programme development and improvement.

    Beyond individual programme evaluation, KeTCET shows potential as a comparison tool for checking institutional programmes and as a starting point for changes in clinical education. Its organised framework could help make evaluation practices more uniform across institutions and different settings. This may lead to fairer and more consistent assessments in tele-clinical training.

    C. Future Directions

    While the study highlights the potential use of telehealth for medical education, further improvements could consider including supervisor experiences for a more comprehensive perspective. Future work can also expand to involve diverse programmes and evaluating long-term impacts of tele-clinical programmes. Validation in varied contexts and exploration of peer learning mechanisms would enhance its applicability and effectiveness in clinical training.

    V. CONCLUSION

    Tele-clinical supervision programmes offer a viable solution to train healthcare professionals, especially in resource-limited settings. This study shows their potential for high student acceptance and effectiveness when systematically designed. Addressing pedagogical environments, supervisor traits, and virtual teaching practices with synchronous and asynchronous elements is crucial. The validated 23-item tool (KeTCET) provides a strong framework for the evaluation of tele-clinical programmes, paving the way for future longitudinal research on long-term outcomes.

    Notes on Contributors

    Dinushee Atapattu-Bakmeewewa, Bhagya Devagiri, Gayanthi Kodituwakku and Madawa Chandratilake contributed to the conceptualization and implementation of this research and have also contributed to the writing of this manuscript.

    Ethical Approval

    This study was reviewed and approved by the Ethical Review Committee of the Faculty of Medicine, University of Kelaniya, Sri Lanka (Ref. no. P-84-08-2021).

    Data Availability

    The data set generated for the quantitative part of this study is available at the following URL:

    https://doi.org/10.6084/m9.figshare.28116863.

    Acknowledgement

    The authors wish to thank all students for their participation and Emeritus Prof. Pathmeswaran for his guidance in the analysis of data.

     Funding

    This project and manuscript did not receive any funding.

    Declaration of Interest

    None of the authors has any conflict of interest or financial interest to declare.

    References

    Anderson, H. K., Hayes, S., Smith, J. P., & Currie, S. (2023). Comparing two methods of clinical supervision. Internet Journal of Allied Health Sciences and Practice, 21(2), 10. https://doi.org/10.46743/1540-580X/2023.2319

    Atapattu Bakmeewewa, A. D. S., Devagiri, B., Kodituwakku, G. K., & Chandratilake, M. (2025). Tele-clinical item rating_dataset.xlsx (Version 1). figshare. https://doi.org/10.6084/m9.figshare.28116863

    Bacon, R., Hopkins, S., Georgousopoulou, E., Nahon, I., Hilly, C., Millar, C., Flynn, A., Smillie, L., Chapman, S., & Brown, N. (2023). While allied health students prefer face-to-face clinical placement, telehealth can support competency development: Results from a mixed-methods study. Frontiers in Medicine10, 1151980. https://doi.org/10.3389/fmed.2023.1151980

    Bramstedt, K. A., Prang, M., Dave, S., Shin, P. N. H., Savy, A., & Fatica, R. A. (2014). Telemedicine as an ethics teaching tool for medical students within the nephrology curriculum. Progress in Transplantation, 24(3), 294–297. https://doi.org/10.7182/PIT2014289

    Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

    Bujang, M. A., Ab Ghani, P., Soelar, S. A., & Zulkifli, N. A. (2012, September). Sample size guideline for exploratory factor analysis when using small sample: Taking into considerations of different measurement scales. International Conference on Statistics in Science, Business and Engineering (ICSSBE), (pp. 1-5). IEEE. https://doi.org/10.1109/ICSSBE.2012.6396605

    Car, J., Koh, G. C. H., Foong, P. S., & Wang, C. J. (2020). Video consultations in primary and specialist care during the COVID-19 pandemic and beyond. BMJ, 371. https://doi.org/10.1136/BMJ.M3945

    Council of Academic Programmes in communication sciences and disorders. (2013, April 11). White paper: Preparation of speech-language pathology clinical educators. American Speech-Language-Hearing Association. https://wordpressstorageaccount.blob.core.windows.net/wp-media/wp-content/uploads/sites/1023/2019/06/White-Paper-Prep-of-SLP-Clinical-Education-Final-April-11-2013.pdf

    DeVellis, R. F., & Xie, Y. (2021). Scale development: Theory and applications (5th ed.). SAGE Publications.

    Espeland, V., & Indrehus, O. (2003). Evaluation of students’ satisfaction with nursing education in Norway. Journal of Advanced Nursing, 42(3), 226–236. https://doi.org/10.1046/j.1365-2648.2003.02611.x

    Friedberg, M. W., Chen, P. G., Van Busum, K. R., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D. D., Brook, R. H., Crosson, F. J., & Tutty, M. (2014). Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Quarterly, 3(4), 1https://pubmed.ncbi.nlm.nih.gov/28083306

    Gammon, D., Sørlie, T., Bergvik, S., & Høifødt, T. S. (1998). Psychotherapy supervision conducted by videoconferencing: A qualitative study of users’ experiences. Journal of Telemedicine and Telecare, 4(SUPPL. 1), 33–35. https://doi.org/10.1258/1357633981931353

    Gibson, A., Miller, R., & King, R. (2007). Staying connected: Video conferenced supervision for rural provisional psychologists. InPsych: The Bulletin of the Australian Psychological Society Ltd29(5), 16.

    Gracious, P. D., Armah, J., Boateng, E. A., Bam, V., Dzomeku, V., Kyei-Dompim, J., Oklodu-Abbey, A. D., & Amponsah, A. K. (2024). Expectations, experiences and challenges of nursing students using the virtual learning medium during the COVID-19 pandemic: A descriptive phenomenological study. PLOS ONE, 19(3), e0299967–e0299967. https://doi.org/10.1371/JOURNAL.PONE.0299967

    Gorsuch, R. L. (1997). Exploratory factor analysis: Its role in item analysis. Journal of Personality Assessment68(3), 532–560. https://doi.org/10.1207/s15327752jpa6803_5

    Hameed, T., Husain, M., Jain, S. K., Singh, C. B., & Khan, S. (2020). Online medical teaching in COVID-19 era: Experience and Perception of Undergraduate Students. Mædica, 15(4), 440. https://doi.org/10.26574/MAEDICA.2020.15.4.440

    Heckner, C., & Giard, A. (2005). A comparison of on-site and telepsychiatry supervision. Journal of the American Psychiatric Nurses Association, 11(1), 35–38. https://doi.org/10.1177/1078390305275602

    Iancu, A. M., Kemp, M. T., & Alam, H. B. (2020). Unmuting medical students’ education: Utilizing telemedicine during the COVID-19 pandemic and beyond. Journal of Medical Internet Research, 22(7), e19667–e19667. https://doi.org/10.2196/19667

    Jeffries, P. R., Bushardt, R. L., DuBose-Morris, R., Hood, C., Kardong-Edgren, S., Pintz, C., Posey, L., & Sikka, N. (2022). The role of technology in health professions education during the COVID-19 pandemic. Academic Medicine Journal of the Association of American Medical Colleges, 97(3S), S104–S109. https://doi.org/10.1097/ACM.0000000000004523

    Kilminster, S. M., & Jolly, B. C. (2000). Effective supervision in clinical practice settings: A literature review. Medical Education34(10), 827–840. https://doi.org/10.1046/j.1365-2923.2000.00758.x

    Kline, R. B. (1999). [Review of the book Psychometric theory (3rd ed.), by J.C. Nunally & I. H. Bernstein]. Journal of Psychoeducational Assessment, 17(3), 275-280. https://doi.org/10.1177/073428299901700307

    Liu, Y., Guo, S., Yang, Z., Hirata, H., & Tamiya, T. (2022). A home-based tele-rehabilitation system with enhanced therapist-patient remote interaction: A Feasibility study. IEEE Journal of Biomedical and Health Informatics, 26(8), 4176–4186. https://doi.org/10.1109/JBHI.2022.3176276

    Martin, P., Copley, J., & Tyack, Z. (2014). Twelve tips for effective clinical supervision based on a narrative literature review and expert opinion. Medical Teacher, 36(3), 201–207. https://doi.org/10.3109/0142159X.2013.852166

    Martin, P., Hulme, A., Fallon, T., Kumar, S., McGrail, M., Argus, G., Gurney, T., & Kondalsamy-Chennakesavan, S. (2023). Impact of the COVID-19 pandemic on student supervision and education in health care settings: A state-wide survey of health care workers. Australian Journal of Rural Health, 31(3), 484–492. https://doi.org/10.1111/AJR.12969

    Martin, P., Kumar, S., & Lizarondo, L. (2017). Effective use of technology in clinical supervision. Internet Interventions, 8, 35–39. https://doi.org/10.1016/J.INVENT.2017.03.001

    Martin, P., Lizarondo, L., & Kumar, S. (2018). A systematic review of the factors that influence the quality and effectiveness of tele supervision for health professionals. Journal of Telemedicine and Telecare, 24(4), 271–281. https://doi.org/10.1177/1357633X17698868

    Mikkonen, K., Elo, S., Miettunen, J., Saarikoski, M., & Kääriäinen, M. (2017). Clinical learning environment and supervision of international nursing students: A cross-sectional study. Nurse Education Today, 52, 73–80. https://doi.org/10.1016/J.NEDT.2017.02.017

    Miller, R. J., & Gibson, A. M. (2004). Supervision by videoconference with rural probationary psychologists. International Journal of Innovation in Science and Mathematics Education, 11(1). https://openjournals.library.sydney.edu.au/CAL/article/view/6065

    Nunnally, J. C. (1978). An overview of psychological measurement. In M. M. Katz, J.e. Barrett, & F. F. Williams (Eds.), Clinical diagnosis of mental disorders: A Handbook (pp. 97-146). Springer. https://doi.org/10.1007/978-1-4684-2490-4_4

    Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003). The mini-CEX: A method for assessing clinical skills. Annals of Internal Medicine, 138(6), 476–481. https://doi.org/10.7326/0003-4819-138-6-200303180-00012

    Perle, J. G., & Zheng, W. (2024). A primer for understanding and utilizing tele supervision with healthcare trainees. Journal of Technology in Behavioural Science, 9(1), 46–52. https://doi.org/10.1007/S41347-023-00322-5

    Pit, S. W., Velovski, S., Cockrell, K., & Bailey, J. (2021). A qualitative exploration of medical students’ placement experiences with telehealth during COVID-19 and recommendations to prepare our future medical workforce. BMC Medical Education, 21(1). https://doi.org/10.1186/s12909-021-02719-3

    Reese, R. J., Aldarondo, F., Anderson, C. R., Lee, S. J., Miller, T. W., & Burton, D. (2009). Telehealth in clinical supervision: A comparison of supervision formats. Journal of Telemedicine and Telecare, 15(7), 356–361. https://doi.org/10.1258/JTT.2009.090401

    Saarikoski, M., Isoaho, H., Warne, T., & Leino-Kilpi, H. (2008). The nurse teacher in clinical practice: developing the new sub-dimension to the clinical learning environment and supervision (CLES) scale. International Journal of Nursing Studies, 45(8), 1233–1237. https://doi.org/10.1016/J.IJNURSTU.2007.07.009

    Shawwa, L. (2023). The use of telemedicine in medical education and patient care. Cureus, 15(4). https://doi.org/10.7759/cureus.37766

    Snowdon, D. A., Sargent, M., Williams, C. M., Maloney, S., Caspers, K., & Taylor, N. F. (2019). Effective clinical supervision of allied health professionals: A mixed methods study. BMC Health Services Research, 20(1). https://doi.org/10.1186/S12913-019-4873-8

    Tarlow, K. R., McCord, C. E., Nelon, J. L., & Bernhard, P. A. (2020). Comparing in-person supervision and tele supervision: A multiple baseline single-case study. Journal of Psychotherapy Integration, 30(2), 383–393. https://doi.org/10.1037/INT0000210

    Wearne, S. M., Teunissen, P. W., Dornan, T., & Skinner, T. (2015). Physical isolation with virtual support: Registrars’ learning via remote supervision. Medical Teacher, 37(7), 670–676. https://doi.org/10.3109/0142159X.2014.947941

    Winstanley, J., & White, E. (2011). The MCSS-26: Revision of the Manchester clinical supervision scale using the rasch measurement model. Journal of Nursing Measurement, 19(3), 160–178. https://doi.org/10.1891/1061-3749.19.3.160

    World Health Organization (WHO). (2022, November 10). WHO issues consolidated guide to running effective telemedicine services. WHO Newshttps://www.who.int/news/item/10-11-2022-who-issues-new-guide-to-running-effective-telemedicine-services

    *D Atapattu Bakmeewewa
    Department of Disability Studies, Faculty of Medicine
    PO Box 6, Thalagolla road, Ragama, 11010 Sri Lanka
    Email: dinushee@kln.ac.lk

    Submitted: 10 March 2025
    Accepted: 5 July 2025
    Published online: 7 October, TAPS 2025, 10(4), 55-62
    https://doi.org/10.29060/TAPS.2025-10-4/OA3690

    Aaron Tigor Sihombing1,2, Antonia Kartika3,4 & Anglita Yantisetiasti2,5

    1Department of Surgery, Division of Urology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia; 2Hasan Sadikin General Hospital, Bandung, Indonesia; 3National Eye Center-Cicendo Eye Hospital, Bandung, Indonesia; 4Department of Ophthalmology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia; 5Department of Anatomical Pathology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia

    Abstract

    Introduction: Music preferences have been linked to personality traits, which in turn may influence career choices. This study explores the potential relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University, Indonesia.

    Methods: A cross-sectional survey was conducted among all residents trained in urology, ophthalmology, and anatomic pathology. Data collected included demographic characteristics, music genre preferences, and work habits related to music. Independent t-tests are used when data are normally distributed, whereas Kruskal-Wallis tests are used when data are not normally distributed.

    Results: The study included 125 residents (19 pathology anatomy, 33 urology, 73 ophthalmology). Pop was the most preferred music genre across all specialties (48% in urology, 61% in ophthalmology, 52% in pathology anatomy). However, secondary preferences varied: rock (21%) was the second most popular among urology residents, jazz (10.9%) and indie (10.9%) among ophthalmology residents, and classical music (26%) among pathology anatomy residents. Demographic differences were noted, with pathology anatomy residents being older and urology residents having a higher proportion of males. Ethnic distribution was relatively consistent across specialties, primarily mixed ethnicity, Sundanese, and Javanese.

    Conclusion: While pop music was the predominant preference across all specialties, secondary music preferences varied, potentially reflecting different personality traits associated with each specialty. The study’s findings are limited by its single-institution sample and cross-sectional design, necessitating further research with larger, more diverse populations to explore the underlying mechanisms linking music preferences to medical specialisation choices.

    Keywords:           Music Preferences, Medical Specialty Selection, Personality Traits

    Practice Highlights

    • Personality traits could predict specialty preferences among medical students.
    • Music genre preferences are associated with personality traits, thus are drawn to particular type of music.
    • Residents in different specialties distributed music genres differently.

    I. INTRODUCTION

    The notion that music genres reflect and influence personality traits is a topic of much debate and interest. Research has shown a correlation between musical preferences and personality traits, with certain genres being associated with specific characteristics (Andrews et al., 2022; Wang et al., 2024). For example, music in slow tempo and music in minor keys were significantly predicted by emotional stability and optimism, whereas music in fast tempo and music in major keys was significantly predicted by openness to experiences, introversion, and gender (Dobrota & Reić, 2014; Upadhyay et al., 2017). Some evidence suggests that individuals drawn to certain music genres may exhibit personality profiles that align with specific career paths. For instance, individuals who prefer classical music tend to score higher in openness and introversion—traits associated with analytical or solitary professions—while those who enjoy rock or pop may display extraversion and sensation-seeking behaviors, often linked to high-energy or interactive professions (Rentfrow & Gosling, 2003; Schäfer & Mehlhorn, 2017). Moreover, individuals with high levels of achievement tend to prefer music that reflects their professional identity, indicating a potential association between occupational roles and musical preferences (Knox & McDonald, 2017).

    Similarly, in the field of medicine, there is a growing interest in understanding how personality traits may influence medical residency preferences. Studies suggest that certain personality types may be drawn to specific medical specialties, and this alignment could impact their satisfaction and performance within that field. This intersection of music, personality, and medical career choices highlights the complex ways in which our preferences and personalities shape and reflect our identities and professional paths.

    Culture and environment have a profound effect on a person’s personality (Smaldino et al., 2019; Triandis & Suh, 2002). Culture provides a framework within which individuals develop beliefs, values, and behaviours that are in harmony with their societal norms. It shapes personality through language, customs, and social norms, which guide an individual’s actions and interactions. Environment, encompassing family dynamics, social relationships, and educational experiences, also plays a pivotal role in molding personality traits. As individuals navigate through different environmental contexts, their personalities adapt and evolve, reflecting the dynamic interplay between their innate dispositions and external influences. This complex interconnection suggests that personality is not a static entity but a fluid construct that changes over time, influenced by the cultural and environmental landscapes we inhabit.

    Indonesia’s rich of cultures is indeed a testament to its diversity, with over 300 ethnic groups calling it home. This multicultural environment offers a unique blend of traditions, languages, and beliefs, which can significantly shape an individual’s personality. Exposure to diverse cultural practices and values can foster open-mindedness, adaptability, and a complex worldview. Research suggests that multicultural experiences can enhance creativity, reduce stereotypes, and provide a broader perspective on life. Moreover, individuals who navigate multiple cultures may develop a multicultural identity, integrating different cultural influences into their personal identity, which can contribute to their overall well-being and social harmony.

    The exploration of a potential correlation between musical preferences and medical specialty choice in Indonesia is indeed a fascinating subject. Since music has been noted to affect personality, which in turn influences the choice of medical specialisation, Indonesia is a multicultural country, and this correlation is intriguing. As a result, this study aims to determine if there is a distribution difference between music genre preferences and healthcare specialisations in Indonesia.

    II. METHODS

    This research employed a cross-sectional study design, with total sampling of all Padjadjaran University resident utilising a comprehensive survey administered to all residents. The study population comprised residents specialising in three distinct medical disciplines: urology, ophthalmology, and anatomic pathology. All residents in urology, ophthalmology, and anatomic pathology were eligible to be included. Exclusion criteria included incomplete responses or refusal to provide consent. This selection allowed for a comparative analysis across specialties with varying degrees of procedural and cognitive demands. In this study we utilised total sampling, all eligible residents during the study period were invited to participate. The survey instrument was designed to collect a range of demographic and preference-based data. Participants were asked to provide information regarding their age, current year of residency training, and their preferred music genres. Crucially, the survey also explored the residents’ work habits related to music, specifically inquiring whether they typically worked with or without background music and their preferred music genre. The survey also collected information on spouse employment status, which was divided into two groups: physicians and non-physicians.

    Statistical analysis was performed to assess the difference between groups within the collected data. The independent t-test was employed to analyse continuous data that demonstrated a normal distribution. For continuous data that did not meet the assumptions of normality, the non-parametric Kruskal-Wallis test was utilised. In all statistical analyses, a significance level (p-value) of 0.05 was established as the threshold for statistical significance, indicating a 5% risk of concluding a relationship exists when it does not. All participants have given informed consent before any data were collected.

    III. RESULTS

    A total of 125 residents participated in this study. Nineteen pathology anatomy residents, 33 urology resident and 73 ophthalmology residents were included. The descriptive statistic of age and sex are presented in Table 1. Marital status and ethnicity are presented in Appendix 1.

     

     

    Pathology Anatomy (n = 19)

    Urology (n = 33)

    Ophthalmology (n = 73)

    p-value

    Age

     

     

     

     

     

     

    Mean ± SD

    33.16 ± 3.11

    30.09 ± 2.11

    30.51 ± 2.55

    0.002*

     

    Median (Range)

    34.00 (29-39)

    30.00 (27-37)

    30.00 (26-36)

     

    Sex

     

     

     

     

     

    Male

    4 (21.1%)

    26 (79.8%)

    22 (30.1%)

    <0.01

     

    Female

    15 (78.9%)

    7 (21.2%)

    51 (69.9%)

     

    *Kruskal Wallis Test

    Table 1. Age, sex, ethnicity and marital status of the residents

    In male resident population, distribution between married and not married is quite equal between specialty (Figure 1). In urology, married male residents is 58%, while it is 68% and 50% in ophthalmology and pathology, respectively. There is sharp difference in female urology resident compared to other specialties. There are only 14% female urology residents who is married, while in ophthalmology and pathology is 63% and 67%, respectively.

    Figure 1. Marital status proportion in male (A) and female (B) residents

    Figure 2. Proportion of married residents with physician spouse

    In urology, 81% of residents also married to physicians, while it is only 55% in ophthalmology and 42% in pathology (Figure 2).

    Most residents in urology (69.70%) and pathology (73.68%) reported that they were listening to musical background while working (Figure 3). While only 38.36% in ophthalmology that worked with musical background.

    Figure 3. Comparison between residents working with musical background and those without

    In those three specialties, most of the residents prefer pop music compared to others. A total of 48.48%, 52.63%, and 61.64% residents in urology, pathology and ophthalmology prefers pop music (Figure 4). Rock music was the 2nd most popular music among urology residents (21.21%), while it is classical music in pathology residents (26.32%).

    Figure 4. Residents’ music genre preference (in percentage)

    IV. DISCUSSION

    This study explored the relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University. There is a higher median age among pathology anatomy residents than among urology and ophthalmology residents, and there is a greater proportion of males in the urology department (Table 1). In spite of this, all residents from the three departments belong to the same generation (Juekiewicz, 2023). As residents in the same generation (generation Y), they are influenced by similar external influences, values, and ethical principles which influence their music genre preferences (Juekiewicz, 2023; Krumhansl, 2017).

    Given that Indonesia is composed of multiple ethnic groups, and ethnicity could influence music genre preference, we found that the top three ethnic groups that reside in the three departments are somewhat similar, namely mixed ethnicity, Sundanese, and Javanese (Table 1). This percentage ranking differs from that of the Badan Pusat Statistik (BPS), which indicates that the top three groups by population are Java (40.22%), Sundanese (15.5%), and Batak (3.58%) (Badan Pusat Statistik [BPS], 2010).

    Since music serves as a connection function between people, we evaluate the marital status of the residents (Bamford et al., 2024). In urology, 52% of residents are married, while in pathology and anatomy, 37% and 36% are married, respectively (Table 1). While females comprise only 21% of urology residents, the majority of them are unmarried (86%), which is compared to only 33% and 37% of female pathology anatomy and ophthalmology residents who are unmarried, respectively (Figure 1).

    Eighty-one percent of urology residents are married to a physician compared to 55% of ophthalmology residents and 42% of pathology anatomy residents (Figure 2). Study by Dutta RR, et all showed that only 26.1% of physician married with physician (Dutta et al., 2024). However, the study did not compare the percentage of physicians who are married to other physicians in each specialty.

    Compared with urology residents (69.7%) and pathology anatomy residents (73.68%), only 38.36% of ophthalmology residents listen to music while working (Figure 3).

    Pop genre is the most preferred genre among three groups of residents, comprising 48% of residents in the urology and 61% of residents in the ophthalmology groups, as well as 52% of residents in the pathology anatomy groups (Figure 4). This finding is similar to a study by Krumhansl that the pop genre is the most preferred genre for individuals born between 1940 and 1999 (Krumhansl, 2017). However, the second most preferred genre among urology residents is rock (21%), while jazz (10.9%) and indie (10.9%) are the second most preferred genres in ophthalmology, and classical music (26%) is the second most preferred genre in pathology anatomy. The difference in genre music preference can also be observed in the third to last rank on the list (Figure 4).

    The variation in secondary music genre preferences among specialties may be partially explained by psychological theories of personality, particularly the Five-Factor Model (FFM), which posits five broad domains of personality: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Prior research has shown that individuals who prefer classical music tend to score high in openness and introversion—traits often linked to reflective and analytical disciplines such as pathology. In contrast, rock and pop fans tend to exhibit higher levels of extraversion and openness to experience, traits more commonly observed in high-intensity, procedurally driven fields like urology (Rentfrow & Gosling, 2003; Schäfer & Mehlhorn, 2017). However, in our study, most residents prefer pop music.

    From a sociological perspective, medical specialties may be seen as “occupational subcultures” (Becker, 1963), each with distinct values, stress profiles, and interaction patterns. These subcultures attract individuals whose identities align with the implicit norms of the field (Light, 1979), potentially explaining the alignment between music preference and specialty. For example, the high-paced, team-intensive nature of urology might attract residents who identify with more energetic and expressive music like rock. Conversely, fields like pathology, which involve solitary analysis, may appeal to those who appreciate structured and introspective genres like classical music.

    This study has several limitations that affect generalisability of the findings. While the study used a total sample of residents from three departments, the number of residents from each specialty varies, with 19 respondents from pathology anatomy, 33 respondents from urology, and 73 respondents from ophthalmology. In addition, the study’s sample was drawn exclusively from residents of one university.

    Despite its limitations, this study presents several strengths that contribute to its value. The study explores a unique and interesting relationship between music genre preferences and medical specialty selection, particularly within the Indonesian cultural context. This area is relatively understudied and by focusing on medical residents, a specific population facing unique career pressures, the research addresses a relevant and understudied group. Furthermore, we acknowledgment of Indonesia’s diverse cultural influences, highlighting the potential impact of environment on both musical tastes and professional choices. We hope that this study will open avenues for further research and raising awareness about the potential influence of external factors like music on medical professionals.

    V. CONCLUSION

    This cross-sectional study explored the relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University in Indonesia. While the study revealed variations in music preferences distribution across different specialties, pop being the most favored genre and different preference in second to last rank.

    We found that there are differences in demographic distributions, particularly age and gender, among the resident groups, it is important to acknowledge the limitations of the study’s design. The use of a single-institution sample and the cross-sectional methodology limit the generalisability of the findings and preclude the establishment of causal relationships. Nevertheless, the study offers a preliminary investigation into the potential interplay between musical tastes and career choices within the medical field, raising intriguing questions for future research.

    Further studies with larger and more diverse samples, are needed to expand upon these findings, and to explore the underlying mechanisms that might link music preferences to medical specialisation.

    Notes on Contributors

    ATS contributed to the conceptualisation, data curation, methodology, formal analysis, project administration, validation, investigation, funding acquisition, resources, visualisation, software, supervision, writing of the original draft, review and editing.

    AK contributed to the methodology, formal analysis, resources, visualisation, software, supervision, writing of the original draft, review and editing.

    AY contributed to the investigation, visualisation, software, supervision, writing of the original draft, review and editing.

    Ethical Approval

    This study was performed under the ethical approval from Hasan Sadikin Hospital Ethical Committee (Approval Number: DP.04.03/D.XXIV.16/14527/2024). This study is in line with the 1964 Declaration of Helsinki and existing ethical standards.

    Data Availability

    The data supporting this study are available upon reasonable request to Corresponding Author.

    Funding

    The authors received no financial support for the research, authorship, and/or publication of this article.

    Declaration of Interest

    The authors declare no relevant financial or non-financial competing interest from any party.

    References

    Andrews, C., Gardiner, K., Jain, T. K., Olomi, Y., & North, A. C. (2022). Culture, personal values, personality, uses of music, and musical taste. Psychology of Aesthetics, Creativity, and the Arts, 16(3), 468–486. https://doi.org/10.1037/aca0000318

    Badan Pusat Statistik. (2010). Kewarganegaraan, suku bangsa, agama, dan bahasa sehari-hari penduduk Indonesia (Ch. 3, Komposisi penduduk, p. 9). Badan Pusat Statistik. https://www.bps.go.id/id/publication/2012/05/23/55eca38b7fe0830834605b35/kewarganegaraan-suku-bangsa-agama-dan-bahasa-sehari-hari-penduduk-indonesia.html

    Bamford, J. S., Vigl, J., Hämäläinen, M., & Saarikallio, S. H. (2024). Love songs and serenades: A theoretical review of music and romantic relationships. Frontiers in Psychology, 15, 1302548. https://doi.org/10.3389/fpsyg.2024.1302548

    Becker, H. S. (1963). Outsiders: Studies in the sociology of deviance. Free Press.

    Dobrota, S., & Reić, E. I. (2014). The relationship between music preferences of different mode and tempo and personality traits – Implications for music pedagogy. Music Education Research, 17(2), 234–247. https://doi.org/10.1080/14613808.2014.933790

    Dutta, R. R., Wu, A. T., Picton, B., Shah, S., Chernyak, M., Bauer, K., Solomon, S., Chang, J., Nguyen, B., Jiang, M., & Hurria, A. (2024). Physician marriage survey reveals sex and specialty differences in marital satisfaction factors. Scientific Reports, 14(1), 5159. https://doi.org/10.1038/s41598-024-55437-3

    Jurkiewicz, C. L. (2023). Age diversity and generational ethics. In L. C. Becker & C. B. Becker (Eds.), Encyclopedia of ethics (pp. 54–62). Springer. https://doi.org/10.1007/978-3-030-22767-8

    Knox, D., & MacDonald, R. (2016). Broadcasting personalities: The relationship between occupation and music preferences in the BBC radio programme Desert Island Discs. Psychology of Music, 45(5), 645–664. https://doi.org/10.1177/0305735616670497

    Krumhansl, C. L. (2017). Listening niches across a century of popular music. Frontiers in Psychology, 8, 431. https://doi.org/10.3389/fpsyg.2017.00431

    Light, D. W. (1979). Becoming psychiatrists: The professional transformation of self. W. W. Norton & Company.

    Rentfrow, P. J., & Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology, 84(6), 1236–1256. https://doi.org/10.1037/0022-3514.84.6.1236

    Schäfer, T., & Mehlhorn, C. (2017). Can personality traits predict musical style preferences? A meta-analysis. Personality and Individual Differences, 116, 265–273. https://doi.org/10.1016/j.paid.2017.04.061

    Smaldino, P. E., Lukaszewski, A., von Rueden, C., & Gurven, M. (2019). Niche diversity can explain cross-cultural differences in personality structure. Nature Human Behaviour, 3, 1276–1283. https://doi.org/10.1038/s41562-019-0730-3

    Triandis, H. C., & Suh, E. M. (2002). Cultural influences on personality. Annual Review of Psychology, 53, 133–160. https://doi.org/10.1146/annurev.psych.53.100901.135200

    Upadhyay, D. K., Shukla, R., Tripathi, V. N., & Agrawal, M. (2017). Exploring the nature of music engagement and its relation to personality among young adults. International Journal of Adolescence and Youth, 22(4), 484–496. https://doi.org/10.1080/02673843.2016.1245150

    Wang, Y., Qu, W., & Ge, Y. (2024). Relationship between personality and music preference: Psychometric adaptation of the music preference scales in a Chinese sample. Musicae Scientiae, 29(1), 109-130. https://doi.org/10.1177/10298649241287072

    *Aaron Tigor Sihombing
    Jl. Raya Jatinangor, Cikeruh, Kec. Jatinangor,
    Kabupaten Sumedang, Jawa Barat 45363
    +62 813-2132-9126
    Email: aarontigor@gmail.com

    Submitted: 6 November 2024
    Accepted: 11 April 2025
    Published online: 1 July, TAPS 2025, 10(3), 58-64
    https://doi.org/10.29060/TAPS.2025-10-3/OA3567

    Nathania Nishida Tan1, Daniel Ardian Soeselo2,3, Natalia Puspadewi2, V. Dwi Jani Juliawati2 & Gisella Anastasia2

    1Faculty of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Indonesia;  2Medical Education Unit and 3Department of Surgery, Faculty of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Indonesia

    Abstract

    Introduction: Career selection is a critical decision influenced by various factors, including media exposure. As a popular genre among adolescents, medical dramas may shape perceptions of the medical profession. This study explores the impact of medical dramas on medical students’ motivation to pursue a career in medicine.

    Methods: This study was conducted at the School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, between February and March 2024, using a mixed-method approach. Quantitative data were gathered through a questionnaire, followed by qualitative insights from in-depth interviews. A two-stage sampling strategy was employed, initially identifying first-year students influenced by medical dramas through screening, followed by purposive sampling to select participants for in-depth interviews.

    Results: Analysis identified six key motivational aspects through which medical dramas influence students’ decision to pursue medicine. These include the detailed portrayal of doctors’ roles, their daily lives, and the reinforcement of a positive perception of the profession. Notably, emotional engagement- such as inspiring doctor characters- and the realistic depiction of challenges, including the dedication and hardships of becoming a doctor, emerged as significant factors.

    Conclusion: Medical dramas play a significant role in shaping students’ motivation and perception of the medical profession. This study highlights six key motivational aspects influencing career decisions, offering new insights into media-driven career choices and their potential implications for medical education.

    Keywords:           Medical Dramas, Medical Education, Medical Students, Student Motivation, Career Choice

    Practice Highlights

    • Medical dramas serve as informal educational tools through entertainment for the public.
    • Television influences adolescents’ career aspirations by shaping their interests.
    • Medical dramas shape public perceptions of the medical profession and may inspire viewers.

    I. INTRODUCTION

    Medical dramas have remained a widely popular television genre since their emergence in the 1960s, continuously evolving to portray healthcare professionals’ professional and personal lives within hospital settings (14th Media Mutations International Conference, 2023; Sonego & Rocchi, 2024).

    Communication theorist Marshall McLuhan had predicted the success of medical dramas as early as 1964 in his book Understanding Media: The Extensions of Man, attributing their appeal to the immersive experience viewers have when witnessing medical procedures, which fosters an “obsession with bodily well-being” (Aboud, 2012; Levine & McLuhan, 1964).

    Beyond entertainment, medical dramas serve as informal educational tools illustrating medical practices, bioethical dilemmas, and professional conduct (Cambra-Badii et al., 2021; Zago et al., 2024). Research suggests that these dramas reinforce key principles of medical ethics and professionalism, including communication skills, patient confidentiality, and empathetic bedside manner (Aboud, 2012). By portraying both the challenges and rewards of medical practice, these narratives may also shape viewers’ perceptions of the profession, inspiring people to pursue careers in healthcare (Alahmari, 2023).

    Research indicates a correlation between specific television programs and career aspirations, suggesting that the content adolescents consume can shape vocational interests and goals (Gehrau et al., 2016). As an accessible informal educational resource, television provides insight into professional responsibilities and challenges (Mendick & Williams, 2022). Within this context, medical dramas may play a significant role in shaping students’ motivation to enter the medical field. Unlike prior studies that broadly examine media influence over career perceptions, this study focuses on first-year medical students, analysing their motivational drive. By integrating quantitative and qualitative data, this study provides deeper insight into how medical dramas influence medical students’ motivation to pursue a career in medicine.  

    II. METHODS

    This study was conducted at the School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, from February to March 2024, using a mixed-method approach. A two-stage sampling method was employed, beginning with a questionnaire to identify eligible participants, followed by purposive sampling to select individuals for in-depth interviews. The target population consisted of first-year medical students enrolled in the 2023 academic year who acknowledged that medical dramas had influenced their motivation to pursue a medical degree.

    A. Participant Selection

    Out of 210 first-year students, 114 reported watching medical dramas. These students completed a demographic questionnaire, which gathered information on gender, parents’ profession, and their viewing habits related to medical dramas. Additionally, they completed the Strength of Motivation for Medical School Revised (SMMS-R) Questionnaire, a validated self-report instrument consisting of 15 items rated on a 5-point Likert scale (ranging from “strongly disagree” to “strongly agree”), with possible scores ranging from 15 to 75, indicating the strength of motivation for medical school.

    B. Qualitative Phase Sampling

    Purposive sampling was used to ensure an accurate representation of the study population based on key demographic factors, specifically the profession of participants’ parents. The selection process identified respondents from diverse parental backgrounds in the medical field: those with both parents as physicians, those with one parent in medicine, and those with no parental medical background. To ensure gender diversity, one male and one female were selected from each category, all of whom had high motivation scores based on their SMMS-R results. Six participants were selected to achieve thematic saturation, as preliminary analysis showed no new themes emerging beyond this point. Although not statistically representative, this sample captures diverse perspectives on medical career motivation based on different childhood backgrounds.

    C. Data Collection

    The in-depth interviews, conducted by the researcher (NNT), explored participants’ perceptions of how medical dramas influenced their motivation to pursue a career in medicine. The interview guide addressed various aspects of this influence on their decision-making process. One key question asked was “How have medical dramas influenced your decision to pursue medicine?” All participants provided written informed consent, including permission for audio recording. The interviews were then transcribed verbatim for further analysis.

    D. Data Analysis

    The transcribed interviews were analysed using thematic analysis with an inductive coding approach, facilitated by Atlas.Ti software. Initial coding was conducted by NNT, who identified key patterns and concepts. These were then systematically categorised into themes and subthemes to capture broader meanings and insights emerging from the interviews. To ensure coding reliability, DAS and NP reviewed the preliminary codes and analysis results. Through thorough discussions and iterative revisions, the team resolved discrepancies and achieved near-total agreement on the coding structure. To maintain participant confidentiality, each transcript was assigned a unique identification code. 

    III. RESULTS

    A total of 114 respondents participated in this study, consisting of 26 males and 88 females, reflecting a possible gender disparity in medical drama viewership. Among them, 93 students (81.6%) reported that medical dramas significantly impacted their decision to pursue a medical career, while 21 students indicated no such impact.

    Characteristic

    Respondents (n)

    Percentage (%)

    Parental Background

     

     

     

    Both parents are doctors

    8

    7.0

     

    Only one parent is a doctor

    18

    15.8

     

    Neither parent is a doctor

    88

    77.2

     

     

      Reports Motivation to Enter Medical Field Due to Medical Dramas

     

    Yes

    93

    81.6

     

    No

    21

    18.4

     

    Total Respondents

    114

    100.0

    Table 1. Distribution of Respondent Demographic Characteristics

    A. Various Dimensions of the Impact of Medical Dramas on Motivation

    The following table presents the themes and subthemes derived from the in-depth interview analysis, which has been discussed and approved by each inter-rater.

    No.

    Theme

    Subtheme

    1.

    Medical dramas provide an in-depth portrayal of the roles and responsibilities of doctors.

    Medical dramas shape public perceptions of the medical profession and doctors’ roles.

    Medical dramas depict the challenges and rewards of doctors’ daily work.

    Medical dramas highlight doctors’ lives beyond their profession.

    2.

    Medical dramas depict both the professional and personal aspects of doctors’ lives.

    Medical dramas reveal lesser-known aspects of doctors’ personal experiences.

    Medical dramas provide an authentic portrayal of doctors’ daily lives.

    Medical dramas depict ethical dilemmas, emphasising doctors’ professional integrity.

    3.

    Medical dramas contribute to a positive public perception of the medical profession.

    Medical dramas portray impressive doctors, evoking admiration, and inspiration in viewers.

    Doctors’ life-saving actions in medical dramas evoke deep emotions and motivation.

    Medical dramas illustrate doctors’ perseverance and resilience in their profession.

    Medical dramas spark curiosity about medical procedures and treatments.

    4.

    Medical dramas evoke emotional responses that may inspire viewers to pursue a medical career.

    Medical dramas evoke empathy by portraying patients in distress.

    5.

    Medical dramas showcase inspiring doctor characters, motivating viewers to follow in their footsteps.

     

    6.

    Medical dramas illustrate the challenges and dedication required to become a doctor.

     

    Table 2. Table of Themes and Subthemes

    B. Medical Dramas Provide an In-Depth Portrayal of the Roles and Responsibilities of Doctors

    Medical dramas frequently portray various medical scenarios using specialised terminology, offering viewers insight into the daily responsibilities of doctors. As medical settings are generally inaccessible to the public, these dramas provide a unique perspective on the profession. Among the key influences explored in this study, this aspect resonated most with respondents, with four out of six students emphasising its significance.

    One participant explained how watching medical dramas sparked their interest in the field, as they previously had limited information about medicine.

    Grey’s Anatomy offered me insight into the medical field and sparked my interest, as reliable information is not easily accessible to those outside the profession” (V1_H3_B19)

    Another respondent shared how their perception of doctors shifted from fear to admiration after watching medical dramas, recognising the profession’s vital role in society:

    “As a child, I perceived doctors as intimidating due to painful medical visits, such as dental appointments. However, after watching Grey’s Anatomy, I realised that medicine extends beyond that and saw how doctors can positively impact many lives.” (V1_H2_B27)

    Beyond hospital settings, medical dramas also depict the role of doctors in broader societal contexts. Through diverse narratives, these dramas offer multiple perspectives on the medical profession. For example, Doc Martin illustrates the role of a doctor in a rural community, showcasing their involvement in local issues, such as water problems and education (V2_H4_B21). This highlights how doctors contribute beyond clinical practice, reinforcing their role in public health and community development.

    Additionally, medical dramas influence students’ motivation to pursue medicine by portraying the realities of medical work, including complex procedures and life-saving interventions. More than half of the respondents identified this as a significant factor in their decision to study medicine. As one participant stated:

    “The scenes depicting doctors helping others, particularly during surgeries, inspired me to study medicine. I wanted to experience performing surgeries and learn how to help others.” (V4_H7_B19)

    These findings suggest that medical dramas not only enhance public understanding of the profession but also play a role in inspiring future medical students.

    C. Medical Dramas Depict Both the Professional and Personal Aspects of Doctors’ Lives

    Medical dramas depict not only medical procedures but also the personal lives of doctors, including their relationships, friendships, and personal struggles. By presenting doctors as multidimensional individuals beyond their professional roles, these narratives contribute to a more relatable and humanised portrayal of the profession. Notably, four out of six highlighted the importance of this aspect in shaping their motivation. One respondent noted that their interest in medical dramas stemmed primarily from the blend of romantic elements, rather than the medical content alone (V3_H6_B1).

    The public often has limited insight into the personal experiences of doctors, making it difficult to relate to them. However, medical dramas help bridge this gap by shedding light on aspects of doctors’ lives that are not commonly shared. This increased visibility fosters a sense of closeness and enhances a deeper understanding of the profession, which may, in turn, inspire career interest. As one participant expressed:

    “Medical dramas reveal stories that doctors seldom share, providing an effective way to convey these experiences to the general public.” (V5_H11_B4)

    By offering a representation of doctors’ daily lives, medical dramas provide viewers with a deeper understanding of what a career in medicine entails. This additional perspective can be a key factor in the decision-making process for those considering entering the field (V2_H4_B28).

    D. Medical Dramas Contribute to a Positive Public Perception of the Medical Profession

    Three out of six respondents emphasised that medical dramas portray the ethical dilemmas faced by doctors, reinforcing professional values that shape viewers’ perceptions of the medical field. One respondent noted that the depiction of doctors’ strong work ethic in these dramas increased their interest in pursuing a career in medicine (V3_H6_B4).

    Additionally, medical dramas present compelling and admirable doctor characters, which can serve as a source of inspiration for viewers. As one respondent shared:

    “The dramatised portrayal made it inspiring for my younger self, leading me to think, ‘Doctors are cool.’” (V5_H11_B2)

    The depiction of medical procedures, particularly surgeries, also left a strong impression on some respondents, reinforcing their motivation to enter the medical field. One participant stated:

    “Medical dramas provide insight into a doctor’s work, including various surgeries, highlighting the fascinating ability to help many people.”  (V4_H7_B33)

    Furthermore, the portrayal of doctors saving lives in medical dramas can evoke strong emotional responses, fostering admiration and a desire to pursue medicine. One respondent expressed how these scenes encouraged them to consider a career in medicine (V5_H11_B2). Medical dramas also emphasise the perseverance and resilience of doctors in the face of various pressures and stress, further reinforcing the appeal of the profession for aspiring medical students (V5_H9_B11).

    E. Medical Dramas Evoke Emotional Responses that may Inspire Viewers to Pursue a Medical Career

    Medical dramas can spark viewers’ curiosity about medical procedures, motivating them to explore the medical field further and increase their interest in the profession. One respondent noted that medical dramas played a role in their initial motivation for pursuing medicine, as the cases depicted often align with real-world medical scenarios, piquing their interest to learn more:

    “I believe the medical procedures shown in Grey’s Anatomy exist in real life and are up to date. For example, an episode featured an abdominal wall transplant, which piqued my curiosity. After researching it, I discovered that the procedure had only recently been developed, showing how the show connects to world medicine.” (V1_H2_B14)

    Beyond fostering curiosity, medical dramas also evoke strong emotional responses, such as sadness and empathy, particularly when portraying patients in need of medical care. One respondent shared that such portrayals heightened their awareness of the number of people requiring medical assistance. The emotions they experienced, combined with this awareness, ultimately served as an inspiration to pursue medicine (V1_H2_B31).

    F. Medical Dramas Showcase Inspiring Doctor Characters, Motivating Viewers to Follow in their Footsteps

    Medical dramas can inspire viewers through uplifting scenes and compelling portrayals of doctor characters. These representations serve as role models, fostering admiration and desire to follow in their footsteps. Notably, three out of six respondents indicated that such portrayals influenced their decision to pursue a medical career. As one participant shared:

    “In ‘New Amsterdam’, there is a doctor who stands out from others – someone who wants to create change and has a unique vision. I found him quite inspiring.” (V3_H6_B5)

    G. Medical Dramas Illustrate the Challenges and Dedication Required to Become a Doctor

    Medical dramas not only portray the professional responsibilities of doctors but also illustrate the journey of becoming one, highlighting the challenges and obstacles along the way. Two out of six students emphasised the significance of this theme, noting that the depiction of medical interns’ experience in these dramas played a crucial role in their motivation to pursue medicine. As one participant shared:

    “Medical dramas often depict doctors under immense stress, highlighting the challenges of the profession. However, rather than discouraging me, witnessing their struggles made the career even more appealing.” (V5_H10_B28)

    IV. DISCUSSION

    The in-depth interviews revealed that medical students felt more mentally prepared and motivated to pursue medicine after watching medical dramas. These shows provided insight into both the professional and personal lives of doctors, reinforcing previous studies on their educational influence in shaping viewers’ perceptions of the medical field. This aligns with the Media Effect Theory, introduced by Gerbner in 1960, which suggests that television shapes individuals’ perceptions of reality, particularly among frequent viewers (Stacks et al., 2015). Hoffman and colleagues further applied this theory to medical dramas, highlighting their role in influencing viewers’ perception of the medical profession (Arias, 2018).

    Beyond depicting technical medical procedures, medical dramas also emphasise ethical principles and professional values, contributing to a favourable image of the profession. Research indicates that medical and nursing students often use these shows as supplementary learning tools for bioethics and professionalism (Cambra-Badii et al., 2021). Similarly, these portrayals may inspire high school students to consider careers in medicine by offering an engaging introduction to the field.

    Emotional engagement is another key factor in the influence of medical dramas. Strong emotional responses- such as curiosity, empathy, and excitement- can enhance motivation, as emotions play a crucial role in learning, achievement, and psychological well-being, particularly in adolescents (Pekrun, 2017). Positive emotions make tasks more appealing, fostering inner motivation that arises after the initial external excitement provided by medical dramas (Legault, 2016).

    Furthermore, medical dramas often depict the challenges of medical training, including long working hours and emotional strain, providing a more nuanced perspective on the profession. While these portrayals may initially present an idealised view of medicine, they also serve as a reality check, reinforcing the dedication required to pursue a medical career. This dual representation may deter less committed individuals while strengthening the resolve of those deeply passionate about the field (Tedeschi, 2024).

    However, the potential for misinformation in medical dramas must also be considered. The oversimplifications or dramatisation of medical practice may create discrepancies between fictional portrayals and real-world medicine, leading to misconceptions among viewers (Alismail et al., 2018).  One notable bias is the tendency to depict doctors as infallible heroes performing miraculous saves amidst dramatic emergencies, often intertwined with personal achievements and romantic subplots. This portrayal fosters unrealistic expectations, potentially leading to disillusionment when aspiring medical students encounter the demanding and less glamorous realities of medical education and practice (Tedeschi, 2024).

    These findings highlight the complex role of medical dramas in shaping career motivation, offering both inspiration and potential misconceptions. Their influence appears particularly significant for students without a familial background in medicine, as most respondents came from non-medical families. This trend may suggest that medical dramas serve as an accessible source of insight into the profession for those without direct exposure. However, the possibility of sampling bias should be considered, as students already interested in medicine may be more likely to engage in medical dramas. While these dramas may serve as informal educational tools, their impact should be critically assessed to ensure they provide a balanced and realistic portrayal of the medical profession.

    V. CONCLUSION

    Medical dramas play a significant role in shaping students’ motivation to pursue a medical career by providing an engaging portrayal of the profession. These shows offer insight into the roles and responsibilities of doctors, their daily challenges, and the emotional rewards of the profession. Additionally, they influence public perceptions of medicine and evoke strong emotional responses through inspiring characters and narratives. By depicting both the struggles and triumphs of medical training, medical dramas contribute to students’ understanding of the dedication required to become a doctor. This study highlights six key aspects through which medical dramas impact students’ decision to pursue medical education, highlighting their role as both an informational and motivational tool.

    Notes on Contributors

    Nathania Nishida Tan participated in data collection, data analysis, review, the writing of the paper, and the formatting for publication.

      dr. Daniel Ardian Soeselo, Sp.B, MSi.Med. participated in the data analysis, review, and direction of the paper.

      dr. Natalia Puspadewi, M.Med.Ed, Ph.D participated in the data analysis, review, and direction of the paper.

      dr. Gisella Anastasia, MHPE participated in the review of the paper.

      dr. V. Dwi Jani Juliawati, M.PD, Sp.KKLP participated in the review of the paper.

      Ethical Approval

      This study received ethical clearance from the Research Ethics Commission of the Faculty of Medicine and Health Sciences Atma Jaya Catholic University of Indonesia under the ethical approval code 13/02/KEP-FKIKUAJ/2024. To ensure confidentiality and data security, all interview transcripts were anonymised and securely stored in password-protected files, accessible only to the research team.

      Data Availability

      The data of this qualitative study are not publicly available due to confidentiality agreements with the participants.

      Acknowledgement

      The authors would like to thank all the students who participated in this study.

      Funding

      There was no funding for this article.

      Declaration of Interest

      The authors have no conflicts of interest in connection with this article.

      References

      Aboud, K. A. (2012). Medical dramas – The pros and the cons. Dermatology Practical & Conceptual, 2(1), 75–77. https://doi.org/10.5826/dpc.0201a14

      Alahmari, A. A. (2023). Professionalism, ethics, and realism of television medical dramas as perceived by Saudi medical students. Journal of Medical Education and Curricular Development, 10. https://doi.org/10.1177/23821205231175037

      Alismail, A., Meyer, N. C., Almutairi, W., & Daher, N. S. (2018). CPR in medical TV shows: Non-health care student perspective. Advances in Medical Education and Practice, 9, 85–91. https://doi.org/10.2147/AMEP.S146149

      Arias, E. (2018). How does media influence social norms? Experimental evidence on the role of common knowledge. Political Science Research and Methods, 7(3), 561–578. https://doi.org/10.1017/psrm.2018.1

      Cambra-Badii, I., Moyano, E., Ortega, I., Baños, J., & Sentí, M. (2021). TV medical dramas: Health sciences students’ viewing habits and potential for teaching issues related to bioethics and professionalism. BMC Medical Education, 21, Article 509. https://doi.org/10.1186/s12909-021-02947-7

      Gehrau, V., Brüggemann, T., & Handrup, J. (2016). Media and occupational aspirations: The effect of television on career aspirations of adolescents. Journal of Broadcasting & Electronic Media, 60(3), 465–483. https://doi.org/10.1080/08838151.2016.1203319

      14th Media Mutations International Conference. (2023). Investigating medical drama TV series: Approaches and perspectives.

      Legault, L. (2016). Intrinsic and extrinsic motivation. Springer eBooks. https://doi.org/10.1007/978-3-319-28099-8_1139-1

      Levine, S., & McLuhan, M. (1964). Understanding media: The extensions of man. American Quarterly, 16(4), 646. https://doi.org/10.2307/2711172

      Mendick, H., & Williams, K. (2022). Does television influence young people’s career choices? Journal of the National Institute for Career Education and Counselling, 21(1), 18–22. https://doi.org/10.20856/jnicec.2103

      Pekrun, R. (2017). Emotion and achievement during adolescence. Child Development Perspectives, 11(3), 215–221. https://doi.org/10.1111/cdep.12237

      Sonego, A., & Rocchi, M. (2024). Medical drama TV series: A semi-systematic literature review. Online Journal of Communication and Media Technologies, 14(4), Article e202459. https://doi.org/10.30935/ojcmt/15268

      Stacks, D. W., Li, Z. C., & Spaulding, C. (2015). Media effects. Elsevier eBooks. https://doi.org/10.1016/B978-0-08-0970868.95045-1

      Tedeschi, R. (2024). The paradox of medical dramas: Global aspirations versus realities. BJPsych Advances, 31(1), 62–64. https://doi.org/10.1192/bja.2024.56

      Zago, D., Cautero, P., Scarpis, E., Pompili, E., Voglino, G., Siliquini, R., Brusaferro, S., & Brunelli, L. (2024). TV medical dramas: Assessing the portrayal of public health in primetime. Frontiers in Public Health, 12. https://doi.org/10.3389/fpubh.2024.1432528

      *Daniel Ardian Soeselo
      Medical Education Unit and Department of Surgery,
      Faculty of Medicine and Health Sciences,
      Atma Jaya Catholic University of Indonesia
      Jl. Pluit Raya No.2 21, RT.21/RW.8,
      Penjaringan, Kec. Penjaringan, Jakarta Utara,
      Daerah Khusus Ibukota Jakarta, Indonesia 14440
      Email: daniel.ardian@atmajaya.ac.id

      Submitted: 16 August 2024
      Accepted: 23 December 2024
      Published online: 1 July, TAPS 2025, 10(3), 49-57
      https://doi.org/10.29060/TAPS.2025-10-3/OA3495

      Shuh Shing Lee1, Shefaly Shorey2, Tang Ching Lau3 & Dujeepa D. Samarasekera1

      1Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; 2Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; 3Dean’s Office, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

      Abstract

      Introduction: Numerous studies have been conducted on COVID-19, with the majority focusing on interventions involving students and teachers. However, limited research has delved into the pandemic’s impact on the wellness of various stakeholders and how they have adapted to the challenges it presented. This study aims to fill this gap by exploring these neglected areas.

      Methods: This study employs a sequential mixed-method approach to study these areas. The quantitative data collection was carried out using a combination of validated surveys (ranging between 63-88 items) for students, faculty and administrators.  Subsequently, qualitative data collection was gathered via semi-structured interview using a convenient sampling method.

      Results: Seventeen faculty, 18 administrators and 369 students responded to the survey. The quantitative data indicated faculty (teachers) exhibited the lowest stress levels and the highest resilience during the pandemic. In comparison, administrators and students experienced moderate levels of stress, with students scoring slightly higher on the stress level. The themes that emerged from the qualitative data were personal endurance, emotional reaction, cognitive-behavioural reaction and social support.

      Conclusion: Our study highlighted that, apart from personal endurance, the tension arises from emotional and cognitive-behavioural responses of students, teachers, and administrators can be mitigated based on the presence or absence of support mechanisms.

      Keywords:           Wellbeing, Change, Stakeholders, Educational Environment, Culture

      Practice Highlights

      • Students experienced the highest stress levels compared to administrators and teachers.
      • However, students and administrators demonstrated resilience, bouncing back quickly after challenging times.
      • Students and administrators tolerated for uncertainty and displayed cognitive flexibility to enable them to adapt and seek opportunities.
      • Teachers and administrators initially experienced negative emotions, but their emotional resilience facilitated quick recovery.
      • Coming from a culture emphasising collectivism, the sense of belonging and social connection served as a protective factor against psychological distress.

      I. INTRODUCTION

      The foundation of any education system rests upon the harmonious collaboration of three essential elements: teachers, students, and administrators. Each of these components play a vital role in ensuring the smooth functioning of the educational ecosystem and this symbiotic relationship becomes even more evident during challenging times, such as the COVID-19 pandemic. Together, they navigated the complexities of remote learning, ensuring that the pursuit of knowledge remained uninterrupted. In essence, it is the collaborative synergy of these three integral components that propels the educational journey forward. The strength of an education system lies in the seamless interplay of these elements, fostering a holistic and empowering learning experience for all.

      Nevertheless, numerous studies have studied the impact of pandemic such as SARS, COVID-19, with a predominant focus on students and teachers. A significant portion of these studies, approximately 50%, has highlighted the insights and innovations from health professions educators in response to the pandemic, particularly at the undergraduate level (Daniel et al., 2021; Eva & Anderson, 2020; Gordon et al., 2020). The majority of these investigations have primarily collected data on student reactions, satisfaction levels, shifts in attitudes, and changes in knowledge and skills. The review conducted by Best Evidence Medical Education (BEME) revealed that almost half of the studies centred on the transition from traditional in-person teaching to online education, only a meagre 6% of the research primarily focused on aspects related to well-being, mental health, or learner support (Daniel et al., 2021). Amid the widespread concern about the well-being of individuals during the pandemic, much attention has been given to medical students (Jia et al., 2022; Paz et al., 2022; Wilcha, 2020) and frontline healthcare workers (Danet, 2021; Muller et al., 2020; Xiong et al., 2022) in the published articles. The reactions of teachers and administrators to the changes brought about by the pandemic, and how these changes have impacted their well-being, have been largely overlooked in the existing literature.

      Hence, the principal objective of this research is to investigate the impact of the initiatives implemented during the pandemic on the well-being of students, teachers, and administrators. This study aims to explore how these key stakeholders reacted and adapted to the changes, shedding light on a vital aspect that has been underrepresented in the current body of literature.

      II. METHODS

      We employed a sequential explanatory mixed-methods design to assess the adaptation and impact of the pandemic on the well-being of administrators, teachers, and students within the specific context of the Yong Loo Lin School of Medicine (YLLSoM) at the National University of Singapore. This design involved collecting and analysing both quantitative and qualitative data in two consecutive phases within a single study. In the quantitative phase, data were gathered through a comprehensive survey/questionnaire, allowing us to capture a broad spectrum of responses from the participants. Subsequently, in the qualitative phase, we employed the phenomenological approach, conducting in-depth interviews with participants representing various categories. Phenomenology, as an approach in qualitative research, enables us to delve deeply into the shared experiences within a specific group. The primary objective of this approach is to develop a detailed description of the nature of the phenomenon under investigation (Creswell, 2013). The details of this methodological approach are elaborated in the subsequent sections.

      A. Phase I Quantitative Data Collection

      The quantitative data collection was carried out using a survey.  The survey was adapted from Landis’s and Bradley’s (2003) work on The Impact of the 2003 SARS Outbreak on Medical Students at the University of Toronto, The Brief Resilience Scale (Smith et al., 2008), Perceived Stress Scale (Cohen et al., 1983) and Teachers’/Students’ Self-Efficacy towards Technology Integration (Kiili et al., 2016). Table 1 shows the sections of the surveys for administrators, teachers and students.

      Section

      Items in each section

      Student

      Teacher

      Administrator

      A: Demographic Information

      5

      8

      7

      B: The psychological impact of COVID-19

      7

      C: Perception of medical students on the restriction of clinical activities and the impact of COVID-19 on their medical/nursing education

      15

      (13 5-point likert scale items & 2 open-ended questions)

      2

      (open-ended questions)

      2

      (open-ended questions)

      D: Perceived quality of information received by respondents about COVID-19 from specific groups

      8

      E: The source and level of psycho-social support that medical students rely on during the COVID-19 outbreak

      26

      19

      F: Brief Resilience Scale

      6

      G: Perceived Stress   Scale

      10

      H: Teachers’/Students’ Self-Efficacy towards Technology Integration

      11

      4

      Total Items

      88

      71

      63

      Table 1. Sections of the Surveys for Administrators, Teachers and Students

      The survey was validated by 10 medical educators from various departments (Paediatrics, Surgery, Centre for Medical Education, Nursing). After the validation, the survey was administered to medical (Year 1 – 5) and nursing (Year 1 – 4) students, administrators and faculty members in Yong Loo Lin School of Medicine and Alice Lee Centre for Nursing Studies using convenient sampling. It took about 20-30 minutes to complete the survey and the data was collected between Jan – June 2021.

      B. Phase II Qualitative Data Collection

      The qualitative data collection was gathered via semi-structured interview. The interview was conducted for about 60-90 minutes among the medical/nursing students, administrators and faculty members (teachers). Followed up from the data collected from the quantitative data, the questions were revolved around teaching and learning, content, assessment, policies, guidelines, communication, environment (safety)/support and wellness.

      From July 2021 – Nov 2022, we used convenient sampling method to recruit of students, administrators and faculty members. The interviews were carried out by 2 trained interviewers with no power relationship with the interviewees. Interviews were carried out after getting consent from the volunteer interviewees. All digital audio recordings made during the interviews were transcribed and member-checked with the interviewees to ensure transparency and trustworthiness of the data.

      Data collection ceased when the data reached saturation stage.

      C. Data Analysis

      The quantitative data was analysed using descriptive statistics (such as mean, frequency and percentage) using Microsoft Excel for the data collected from students, administrators and teachers.

      The interviews were thematically analysed by 2 researchers in the team. The two researchers coded the transcripts independently and came together to resolve any discrepancy or disagreement on the coding.  Subsequently, they continued to code and form categories and eventually themes. There were multiple discussions that took place among the researchers and the team before the themes were crystalised.

      III. RESULTS

      A. Phase I Quantitative Data

      The demographic information was illustrated in Table 2. Majority of the participants were not quarantined during the pandemic and more than 80% of them did not have a family member tested positive for COVID when this study was conducted. The teachers from the school of medicine were mainly from Family Medicine, Paediatrics, Physiology, Pathology, Public Health, Medicine, Anatomy and Anaesthesia departments. They are educators for postgraduate and undergraduate students. As for administrators, their roles in the departments are educational related such as instructional design, learning analytics, planning and execution of education, managing project and training.

       

       

       

      Teachers

      Administrators

      Students

      0

      Finished

      Completed

      17

      65.4%

      18

      54.5%

      369

      73.8%

      Did not complete

      9

      34.6%

      15

      45.5%

      131

      26.2%

      Total Responses

      26

      33

       

      500

       

      1

      Faculty

      Medicine

      12

      70.6%

      16

      88.9%

      305

      83.0%

      Nursing

      5

      29.4%

      2

      11.1%

      64

      17.0%

      2

      Gender

      Male

      6

      35.3%

      3

      16.7%

      140

      37.9%

      Female

      11

      64.7%

      15

      83.3%

      229

      62.1%

      6

      Living arrangement

      Alone

      1

      5.9%

      1

      5.6%

      17

      4.6%

      With Parents

      2

      11.8%

      7

      38.9%

      338

      91.6%

      With Partners (married, common-law, etc.)

      14

      82.4%

      9

      50.0%

      1

      0.3%

      With Room-mates

      0

      0.0%

      1

      5.6%

      13

      3.5%

      7

      Status during COVID-19 Outbreak

      Non-Quarantined

      17

      100.0%

      18

      100.0%

      355

      96.2%

      Quarantined

      0

      0.0%

      0

      0.0%

      7

      1.9%

      Stay-Home-Notice (SHN)

      0

      0.0%

      0

      0.0%

      7

      1.9%

      8

      I have family member(s), relative or friend(s) who tested positive for COVID-19

      Yes

      2

      11.8%

      1

      5.6%

      29

      7.9%

      No

      15

      88.2%

      17

      94.4%

      340

      92.1%

      Table 2. Demographic Information of the Respondents

      For each section, the summary was illustrated in Table 3. The mean for different sections was quite close for the 3 groups. Likewise, the items that were scored low and high were quite similar for all the sections. For example, Section B The psychological impact of COVID-19, The sleep quality and concentration in all three groups were not affected by the pandemic, but they are more worries about their family members contracted with COVID-19.

      Section

      Administrator

      Teacher

      Student

      B: The psychological impact of COVID-19  (7 items)

      Mean ranging between 2.39-3.67

      Mean ranging between 1.71 – 3.29

      Mean ranging between 2.14 – 3.85

      C: Perception of medical students on the restriction of clinical activities and the impact of COVID-19 on their medical/nursing education (15 items)

      Mean ranging between 1.77 – 3.78

      Not relevant

      D: Perceived quality of information received by respondents about COVID-19 from specific groups

      Mean ranging between 3.33 – 4.17

      Mean ranging between 3.47 – 4.19

      Mean ranging between 3.17 – 4.14

       

      E: The source and level of psycho-social support that medical students rely on during the COVID-19 outbreak

      Mean ranging between 2.78 – 4.11

      Mean ranging between 2.47 – 4.47

      Mean ranging between 2.65 – 4.36

       

      F: Brief Resilience Scale

      Mean 3.4

      Mean 4.01

      Mean 3.3

      G: Perceived Stress Scale

      Mean:  16.6 (Moderately stress)

      Mean 11.7

      (Low stress)

      Mean 18.7 (Moderately stress)

      H: Teachers’/Students’ Self-Efficacy towards Technology Integration

      Mean ranging between 3.83-4.06

      Mean ranging between 3.47 – 4.12

      Mean ranging between 3.68 – 4.22

      Table 3. Summary of the Mean for Different Sections for the 3 Groups

      During the pandemic, students expressed significant concerns about the adequacy of their training, particularly due to reduced patient contact, raising apprehensions about their preparedness for exams. This concern will be elaborated upon in the qualitative data section, shedding light on the depth of their worries. All three groups shared the view that information originating from the government and hospitals was the most reliable, with friends and family scoring the lowest mean among all sources. Despite this, the participants unanimously agreed that the support from friends and family, in terms of both source and level of assistance, was the most substantial. Conversely, organisational support from entities such as the University Wellness Centre, Dean’s office, community, and social media was perceived as unreliable and lacking during the pandemic.

      Furthermore, our observations revealed that teachers exhibited the lowest stress levels and the highest resilience during the pandemic, showcasing their ability to cope effectively. In comparison, administrators and students experienced moderate levels of stress, with students scoring slightly higher on the Perceived Stress Scale. Although students acknowledged challenges, as indicated by their agreement with statements such as “I have a hard time making it through stressful events” (mean: 2.93), they also exhibited resilience, agreeing with the statement “I tend to bounce back quickly after hard times” (mean: 3.76). Additionally, concerning self-efficacy towards technology integration, students reported the highest mean score, indicating confidence in their ability to navigate various Internet applications. While teachers felt competent in using technology for teaching and learning, their confidence wavered when it came to resolving technical issues, as reflected in their mean score of 3.47. This nuanced understanding underscores the complex interplay of stress, resilience, and technological proficiency among the different groups during the challenging circumstances of the pandemic.

      B. Phase II Qualitative Data

      As for the qualitative data collection, we have recruited 7 administrators, 17 teachers (12 from Yong Loo Lin School of Medicine and 5 from Alice Lee Centre for Nursing Studies) and 9 undergraduate students (6 from Yong Loo Lin School of Medicine and 3 from Alice Lee Centre for Nursing Studies). The themes and subthemes that emerged were depicted in the Figure 1.

      Figure 1. Themes and Subthemes of the Qualitative Data

      1) Theme 1: Personal Endurance

      Personal endurance depends on perceived ability and tolerance to uncertainty. For administrators, they felt that it was quite stressful and frustrated during the pandemic as the situation was unclear. However, they were able to manage and there was a sense of relief after they had gone through the critical phase.

      “It was very intense, stressful but looking back now, it is not that bad. We have gone through the worse” (Admin 5)

      While administrators’ contribution to the education system is crucial, some of the administrators perceived their contribution was minor as compared to medical front liner.

      “We are not front liner, our contribution is limited.” (Admin 3)

      Although the situation was stressful in the beginning, we noticed a positive endurance among the teachers and perceived the pandemic as an opportunity instead of a threat.

      “Overall, I think the predominant mood was of a challenge that needs to be overcome and that brought a certain amount of excitement.” (Teacher 7)

      On the other hand, the students felt that they were being too protected and perceived themselves as having the ability to manage the situation themselves.

      “I understood that they wanted to protect us but I felt that eventually, they can’t protect us anymore” (Student 3)

      The students also perceived that the teachers lacked ability in using technology in teaching and learning especially in remote learning.

      “A lot of professors are not familiar with the technology.” (Student 5)

      2) Theme 2: Emotional Reaction

      There were a lot of negative emotions illustrated by the students, administrators and teachers due to various reasons. Students were worried, frustrated and anxious that they may not learn since the contact with the patients was less during the pandemic. Too much protection from the school and the system put in place has heightened these negative emotions.

      “We feel quite unconfident because we feel we have not seen enough patients”/ “..fear that we are not as good as the previous batch” (Students 1 & 2)

      Administrators were frustrated mainly because they need to manage the family and work at the same time when working from home system was implemented. However, some of them shared that they are able to regulate and get used to the situation after a while.

      “Everybody was under pressure at that time…while I have to juggling with work, my kids were at home because school close.” (Admin 4)

      “I usually regulate my own emotion.” (Admin 7)

      While there were some positive emotions state in Theme 1 for teachers, they did feel stressful in the early stage of the pandemic due to the change of the approaches in teaching and learning and they are unsure of the outcomes when the teaching was entirely online.

      “…stressful in the beginning… I even have nightmares…dreaming students get lost in the virtual room.” (Teacher 17)

      “It was a bit stressful in the beginning because you did not know how is going to turn out…” (Teacher 5)

      3) Theme 3: Cognitive-Behavioural Reaction

      Amidst the challenges posed by the pandemic, administrators, students, and teachers made concerted efforts to adapt their cognition and behavior in response to various initiatives, including social distancing measures, reduced patient contact, and a shift to virtual teaching environments. Throughout this period, interviewees shared both positive and negative reactions to these changes.

      Administrators and teachers found themselves navigating the delicate balance between the need to innovate and the need to maintain productivity (ambidexterity). Administrators, in particular, faced the challenge of fostering creativity in coordinating and delivering the curriculum, which involved tasks such as timetabling, resource management, and providing IT support for online learning. These adjustments were made within a short timeframe, reflecting their resilience and adaptability. However, amid these innovative efforts, administrators were also keen on upholding the quality of their work, highlighting the complexity of their role in managing these rapid changes.

      “We have to deliver in a short time but also the content has to be rigorous” (Administrator 3)

      Similarly, teachers tried to be creative in an online teaching environment and ensure the student learned at the same time especially in small group teaching. However, they find it challenging.

      “…there is an urgency to find a way around this small group teaching…we kind of lose the whole power of collaboration.”  (Teacher 5)

      There is also a tension arose among the students for being too protected by the school and compromised with their learning as shared in the quotes below. This was repeatedly mentioned by the students, and they felt they have to face the situation eventually.

      “I am not very interested in surgery, but this is like once in a lifetime and after I go out of medical school I won’t have the chance to see surgery” (Students 3)

      “There’s a culture… in the society in general…protect my child from COVID. But once day we are going to deal with COVID” (Students 6)

      Notwithstanding the aforementioned tensions, it’s worth noting that administrators, students, and teachers exhibited remarkable innovativeness and adaptability during the pandemic. All three groups demonstrated evidence of both Operational Capabilities, which encompass the efficient and effective use of resources, and Dynamic Capabilities, which involve the continuous development of competencies to align with the evolving environment.

      With the predominant shift in communication from face-to-face to virtual platforms, administrators found themselves assuming the role of intermediaries responsible for conveying information to various stakeholders. In this new virtual setting, where body language cues were less apparent, administrators recognised the need to be more attuned and sensitive to subtle nuances in communication compared to traditional face-to-face interactions. This adaptation reflected their ability to pivot and operate effectively within the changing landscape of remote communication.

      “We play the middleman role because we have to speak administrative language to certain people and be sensitive when communicate with faculty members” (Admin 3)

      “We have to start thinking about (what kind of information needed) before the faculty member even ask those questions” (Admin 7)

      Teachers utilised different resources to innovate in their teaching as well as learning from different others.

      “I break it up my lectures into smaller bits and disperse it with PollEverywhere” (Teacher 5)

      “We formed a group we called a brown bag meeting – basically we meet at lunchtime with technologically savvy administrators to introduce to the staff on how to make online learning more interactive.”  (Teacher 10)

      Likewise, since there was less patient contact time, students tried to make use of their time for other learning sessions.

      “Since there’s little time in the hospital, I had read up a lot” (Students 1)

      “It allows us to have some processing time and have time to consolidate our knowledge” (Students 5)

      4) Theme 4: Social Support Mechanism

      Social support mechanism has been mentioned by the 3 groups as one of the prominent mechanisms in adapting the changes during pandemic. It includes transparent communication, team collaboration and support from various stakeholders. For example, administrators shared that they all came together and supported each other during the hard times.

      “All different teams come together, I think that was very precious” (Administrator 2)

      Students sought seniors’ help to provide additional sessions to compensate their learning.

      “What my group would do is we call our seniors to give us extra tutorials” (Student 2)

      However, there was also lack of support mechanism brought up by the teachers which led to negative emotion (such as frustration).

      “Educational technology team are overworked…if the school would really want to be the best or world class, I think we need a very good support from the IT.” (Teacher 9)

      IV. DISCUSSION

      The impact of the COVID-19 pandemic on both our educational systems and personal lives has been profound. This unprecedented disruption has been keenly observed by various stakeholders, including administrators, teachers, many of whom were also frontline workers, and students in medical schools. Swift adaptation to the ever-changing situation became imperative, particularly in response to the government’s new guidelines. The abrupt alterations in social interactions and extracurricular activities routines compelled a shift towards a heightened emphasis on family life, accompanied by the necessity to work and learn from home due to lockdown measures. These changes had profound physical and psychological effects on our lives.

      Our study revealed that students experienced the highest stress levels compared to administrators and teachers, a finding consistent with previous research indicating that medical students often have higher baseline anxiety than their peers studying other disciplines (Dyrbye et al., 2006; Lasheras et al., 2020). Qualitative data highlighted that students’ stress levels were primarily attributed to the lack of patient contact and inadequate training, potentially impacting their future practice. Additionally, students expressed feelings overly protected due to initiatives like stay-at-home learning. The altered learning environment, combined with a lack of guidance on learning strategies and interpersonal relationships, left students vulnerable to intense emotional fluctuations and strained family relationships (Zhang et al., 2020). Similarly, for administrators, the shift to remote work and social isolation policies posed challenges in balancing work and family responsibilities, as evident from their qualitative comments.

      However, students and administrators demonstrated resilience, bouncing back quickly after challenging times. According to Del Carmen Pérez Fuentes et al.’s (2020) Adaptability to Change framework, a sense of control, tolerance for uncertainty, and cognitive flexibility are crucial in coping with adverse situations. Despite feeling anxious and frustrated due to the inability to control the study-from-home or working-from-home policy, student and administrator tolerated for uncertainty and display cognitive flexibility to enable them to adapt and seek opportunities. Emotional resilience, the ability to generate positive emotions and recover swiftly from negative emotional experiences, played a pivotal role in psychological resilience (Zhang et al., 2020). This emotional resilience led to diverse emotional responses, influencing the cognitive processing of emotional information. Teachers and administrators initially experienced negative emotions, but their emotional resilience facilitated quick recovery, evident from their transcripts.

      Emotional and cognitive-behavioural responses were further shaped by social support mechanisms within peer groups, colleagues, organisational leaders, and the government. While studies have shown that social media can heighten anxiety due to misinformation and distressing news (Gao et al., 2020), our research indicated that students, teachers, and administrators placed significant trust in information provided by the government and institutions. This trust in government intentions and capabilities fosters adherence to health regulations, essential in crisis management (Siegrist & Zingg, 2014).

      Coming from a culture emphasising collectivism, our society values interdependence and family connections highly. This sense of belonging and social connection served as a protective factor against psychological distress, aligning with previous research findings (Xiao, 2021; Yu et al., 2020) . Conversely, a lack of social support within a collectivist culture, as reported by teachers and students, contributed to psychological distress. Our qualitative and quantitative data in this study support this observation, emphasising the significance of social support structures in mitigating the adverse effects of challenging circumstances. The importance of fostering a supportive environment, both within institutions and at a societal level, cannot be overstated in times of crisis.

      V. LIMITATIONS

      This study has some potential limitations. The study was carried out in a single medical school; hence, the results can only be transferable to the same context. The number of respondents was quite small (especially for nursing respondents) despite multiple reminders sent to the various groups. Therefore, they may not be representative of the entire student, teachers and administrator’s population. Third, the survey was a self-reported survey and may have inherent biases while answering the questions. While rigor is more challenging to achieve in qualitative data collection and analysis, the researchers adhere to the trustworthiness principles as much as possible in analysing and presenting the results in this paper.

      VI. CONCLUSION

      In conclusion, achieving wellness during a pandemic is indeed possible, but it hinges not only on the resources that organisations and governments can marshal but also on individual resilience in navigating uncertainty, cultural factors, trust, and support systems. Our study highlights the importance of familial and peer connections within our cultural context, underscoring how these bonds facilitate adaptation and innovation amid the challenges posed by the pandemic. The emotional and cognitive-behavioural responses of students, teachers, and administrators are depending on their personal endurance. However, the tension that arises in these individuals can be mitigated or exacerbated based on the presence or absence of adequate support mechanisms. Sufficient support can act as a buffer, helping individuals cope effectively with the challenges they face. Conversely, insufficient support can exacerbate stress and strain, hindering their ability to adapt and respond positively to the situation at hand.

      Therefore, fostering a strong support network, both within organisations and communities, is crucial. This support not only alleviates the immediate challenges faced by individuals but also empowers them to build emotional resilience, enabling them to navigate uncertainties and adversities with greater ease. In this way, the collective endurance of individuals, coupled with robust support systems, becomes the cornerstone of achieving wellness and fostering positive responses in the face of a pandemic.

      Notes on Contributors

      DDS developed the research idea and design with SSL, SR & LTC. The data collection was performed by SSL. The data were analysed by SSL & DDS. DDS, SSL, SR & LTC performed the data interpretation. DDS, SSL & SR wrote the article with revision by LTC. All the authors read and agreed with the final manuscript.

      Ethical Approval

      Ethics approval was sought from the National University of Singapore (NUS) Institutional Review Board (NUS-IRB-2020-216). Written informed consent was obtained from all participants.

      Data Availability

      The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available as the participants of this study did not give written consent for their data to be shared publicly.

      Acknowledgement

      We would like to express our heartfelt gratitude to Jillian Yeo and Lilusha Kaludewa for helping in data collection and analysis.

      Funding

      No funding is available for this research.

      Declaration of Interest

      The authors report that there are no conflict of interests to declare.

      References

      Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385. https://doi.org/10.2307/2136404

      Creswell, J. W. (2013, January 1). Steps in conducting a scholarly mixed methods study [Slide show]. https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=dberspeakers

      Danet, A. D. (2021). Psychological impact of COVID-19 pandemic in Western frontline healthcare professionals. A systematic review. Medicina Clínica (English Edition), 156(9), 449–458. https://doi.org/10.1016/j.medcle.2020.11.003

      Daniel, M., Gordon, M., Patricio, M., Hider, A., Pawlik, C., Bhagdev, R., Ahmad, S., Alston, S., Park, S., Pawlikowska, T., Rees, E., Doyle, A. J., Pammi, M., Thammasitboon, S., Haas, M., Peterson, W., Lew, M., Khamees, D., Spadafore, M., . . . Stojan, J. (2021). An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME guide no. 64. Medical Teacher, 43(3), 253–271. https://doi.org/10.1080/0142159x.2020.1864310

      Del Carmen Pérez-Fuentes, M., Del Mar Molero Jurado, M., Martínez, Á. M., Fernández-Martínez, E., Valenzuela, R. F., Herrera-Peco, I., Jiménez-Rodríguez, D., Mateo, I. M., García, A. S., Del Mar Simón Márquez, M., & Linares, J. J. G. (2020). Design and validation of the adaptation to change questionnaire: New realities in times of COVID-19. International Journal of Environmental Research and Public Health, 17(15), 5612. https://doi.org/10.3390/ijerph17155612

      Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Academic Medicine, 81(4), 354–373. https://doi.org/10.1097/00001888-200604000-00009

      Eva, K. W., & Anderson, M. B. (2020). Medical education adaptations: Really good stuff for educational transition during a pandemic. Medical Education, 54(6), 494. https://doi.org/10.1111/medu.14172

      Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020). Mental health problems and social media exposure during COVID-19 outbreak. PLoS ONE, 15(4), Article e0231924. https://doi.org/10.1371/journal.pone.0231924

      Gordon, M., Patricio, M., Horne, L., Muston, A., Alston, S. R., Pammi, M., Thammasitboon, S., Park, S., Pawlikowska, T., Rees, E. L., Doyle, A. J., & Daniel, M. (2020). Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME guide no. 63. Medical Teacher, 42(11), 1202–1215. https://doi.org/10.1080/0142159x.2020.1807484

      Jia, Q., Qu, Y., Sun, H., Huo, H., Yin, H., & You, D. (2022). Mental health among medical students during COVID-19: A systematic review and meta-analysis. Frontiers in Psychology, 13, Article 846789. https://doi.org/10.3389/fpsyg.2022.846789

      Kiili, C., Kauppinen, M., Coiro, J., & Utriainen, J. (2016). Measuring and supporting pre-service teachers’ self-efficacy towards computers, teaching, and technology integration. The Journal of Technology and Teacher Education, 24(4), 443–469. https://jyx.jyu.fi/bitstream/123456789/53087/1/kiilikauppinencoiroutriainen.pdf

      Lasheras, I., Gracia-García, P., Lipnicki, D., Bueno-Notivol, J., López-Antón, R., De La Cámara, C., Lobo, A., & Santabárbara, J. (2020). Prevalence of anxiety in medical students during the COVID-19 pandemic: A rapid systematic review with meta-analysis. International Journal of Environmental Research and Public Health, 17(18), 6603. https://doi.org/10.3390/ijerph17186603

      Mark, L., & John, B. (2003). The impact of the 2003 SARS outbreak on medical students at the University of Toronto. University of Toronto Medical Journal, 82, 158–164. https://api.semanticscholar.org/CorpusID:59193656

      Muller, A. E., Hafstad, E. V., Himmels, J. P. W., Smedslund, G., Flottorp, S., Stensland, S. Ø., Stroobants, S., Van De Velde, S., & Vist, G. E. (2020). The mental health impact of the COVID-19 pandemic on healthcare workers, and interventions to help them: A rapid systematic review. Psychiatry Research, 293, Article 113441. https://doi.org/10.1016/j.psychres.2020.113441

      Paz, D. C., Bains, M. S., Zueger, M. L., Bandi, V. R., Kuo, V. Y., Cook, K., & Ryznar, R. (2022). COVID-19 and mental health: A systematic review of international medical student surveys. Frontiers in Psychology, 13, Article 1028559. https://doi.org/10.3389/fpsyg.2022.1028559

      Siegrist, M., & Zingg, A. (2014). The role of public trust during pandemics. European Psychologist, 19(1), 23–32. https://doi.org/10.1027/1016-9040/a000169

      Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008). The brief resilience scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine, 15(3), 194–200. https://doi.org/10.1080/10705500802222972

      Wilcha, R. (2020). Effectiveness of virtual medical teaching during the COVID-19 Crisis: Systematic review. JMIR Medical Education, 6(2), Article e20963. https://doi.org/10.2196/20963

      Xiao, W. S. (2021). The role of Collectivism–Individualism in attitudes toward compliance and psychological responses during the COVID-19 Pandemic. Frontiers in Psychology, 12, Article 600826. https://doi.org/10.3389/fpsyg.2021.600826

      Xiong, N., Fritzsche, K., Pan, Y., Löhlein, J., & Leonhart, R. (2022). The psychological impact of COVID-19 on Chinese healthcare workers: A systematic review and meta-analysis. Social Psychiatry and Psychiatric Epidemiology, 57(8), 1515–1529. https://doi.org/10.1007/s00127-022-02264-4

      Yu, H., Li, M., Li, Z., Xiang, W., Yuan, Y., Liu, Y., Li, Z., & Xiong, Z. (2020). Coping style, social support and psychological distress in the general Chinese population in the early stages of the COVID-19 epidemic. BMC Psychiatry, 20, Article 426. https://doi.org/10.1186/s12888-020-02826-3

      Zhang, Q., Zhou, L., & Xia, J. (2020). Impact of COVID-19 on emotional resilience and learning management of middle school students. Medical Science Monitor, 26, Article e924994. https://doi.org/10.12659/msm.924994

      *Lee Shuh Shing
      Centre for Medical Education,
      Yong Loo Lin School of Medicine,
      National University of Singapore, Singapore
      10 Medical Dr, Singapore 117597
      Email: medlss@nus.edu.sg

      Submitted: 31 July 2024
      Accepted: 24 February 2025
      Published online: 1 July, TAPS 2025, 10(3), 37-48
      https://doi.org/10.29060/TAPS.2025-10-3/OA3473

      Kevin Xuan Hong Tang1, Koon Kee Teo1 & Kye Mon Min Swe2

      1Department of Medicine, Faculty of Medicine, Newcastle University Medicine Malaysia (NUMed), Malaysia; 2Department of Research, Faculty of Medicine, Newcastle University Medicine Malaysia (NUMed), Malaysia

      Abstract

      Introduction: Every medical graduate is expected to fulfil the teaching responsibilities stated by the General Medical Council (GMC). It is beneficial to nurture both teaching motivation and skills early in the undergraduate program. This study aims to evaluate the outcomes of final-year medical students as near-peer teachers in a student-led near-peer teaching program and their fulfilment of the educational responsibilities stated by the GMC.

      Methods: A cross-sectional study was conducted among the year 5 medical students who participated in the Peer Teaching Program. A structured post-participation 6-point Likert scale questionnaire with written consent was distributed to the near-peer teachers to assess their perspectives on skills enhancement, motivation, and career direction. Additionally, the Peer Tutor Assessment Instrument questionnaires were distributed to the near-peer students to evaluate the performance of the near-peer teachers in five areas: responsibility and respect, information processing, communication, critical analysis, and self-awareness.

      Results: There were 28 near-peer teachers, and 49 near-peer students participated in the study.  The near-peer teachers score the highest in skills (5.36 ± 0.53), followed by motivation (5.16 ± 0.60) and career direction (4.79 ± 0.82). Three quarters of the near-peer teachers considered teaching to be their future primary career path after experiencing this teaching experience (4.36 ±1.34). Generally, the near-peer teachers were highly evaluated by the near-peer students across all domains (5.06 ± 0.51).

      Conclusion: Overall, the near-peer teaching programme likely improved the final-year medical students in fulfilling the “Doctors as Teachers” responsibilities outlined by the GMC.

      Keywords:           Near-peer Teaching, Medical Students, Undergraduate Medical Education, General Medical Council, Doctors as Teachers

      Practice Highlights

      • Near-peer teachers are likely improved in skills enhancement, motivation and career direction.
      • Sex and students’ background are not associated with the perceived outcomes of near-peer teachers.

      I. INTRODUCTION

      One of the aspects of Good Medical Practice outlined by the General Medical Council (GMC) for all medical professionals is to “be willing to offer professional support to colleagues, including students, through teaching” (General Medical Council, 2023). The role of doctors as teachers has been widely recognised as they need to teach and educate juniors, students and even patients (General Medical Council, 2011, 2015, 2023). On average, junior doctors spend around 80 minutes per day teaching medical students (Busari et al., 2002). This task is daunting for every new medical graduate who has just begun their UK foundation programme. They need to assume this responsibility with minimal formal training and preparations (Pierce et al., 2024; Qureshi et al., 2013).

      Therefore, it is beneficial to motivate medical graduates to teach and equip them with appropriate teaching skills as early as their undergraduate programme (General Medical Council, 2011; Knobloch et al., 2018). Near-peer teaching involves students one or more academic years ahead teaching their peers or junior students (De Menezes & Premnath, 2016; Ten Cate & Durning, 2007; Yu et al., 2011). This has long been thought of as a programme to be incorporated into the medical curriculum to optimise teaching qualities and to produce more competent and knowledgeable doctors in the future (Botelho et al., 2022; Burgess et al., 2014; Zheng & Wang, 2022). Generally, medical schools provide a safe space for medical students to practice, correct and improve their teaching and pedagogical skills (Hardie et al., 2022). Most medical students feel less daunted and more supported involved in teaching their near-peer students (Yu et al., 2011).

      To address this gap of insufficient teaching opportunities, most medical schools provide near-peer teaching programmes for their medical student (Frearson & Gale, 2017). However, most of the near-peer teaching programmes are carried out formally with structured guidance and training (General Medical Council, 2011), be it in the form of the Peer Assisted Learning Scheme (PALS) student-selected components (SSC) (Furmedge et al., 2014; Hettle & Morgan, 2019; Ross & Cameron, 2007; Ten Cate & Durning, 2007) or Doctors as Teachers and Educators training course (Cook et al., 2010; General Medical Council, 2011). Little is known about the outcomes of the student-led, student-run, near-peer teaching (NPT) programme in medical schools.

      In the academic year 2023/2024, the Newcastle University Medicine Malaysia (NUMed) final-year medical students were involved as near-pear teachers in an NPT programme. This study thus aims to evaluate the outcomes of participation of the final year medical students as near-peer teachers in the student-led near-peer teaching programme and to determine whether the soon-to-be medical graduates can fulfil the “Doctors as Teachers” responsibilities stated by the GMC.  

      II. METHODS

      The NPT Programme was a purely student-led, student-run 3-month teaching programme which provided additional focus on the learning outcomes of the third-year medical curriculum. This programme functioned as an adjunct to the formal curriculum and provided precious opportunities for final-year medical students to improve their teaching skills.

      Before the academic year started, invitation email was sent out to recruit final-year medical students to participate voluntarily as the near-peer teachers and the year 3 medical students as the near-peer students. A total of 51 final-year medical students and 100 year 3 medical students signed up for this programme. The near-peer students were randomly assigned to groups of 5 to 6 each, and each group was guided by 3 near-peer teachers. Before the programme commenced, all near-peer teachers were required to attend a mandatory online training course conducted by the lecturers to enhance their presentation and teaching skills.

      This programme comprised a total of twelve teaching sessions spanning twelve weeks, covering four sessions of Essential Clinical Placement teaching sessions, four sessions of Case-Based Discussion, one surgical teaching topic and case-based session, one Single Best Answer practice session, one Written Prescribing Exam (WRISKE) session and one Objective Structural Clinical Examination (OSCE) session. Most sessions were delivered virtually (Zoom) or physically, depending on students’ preferences, except for the OSCE session, which was always conducted physically. The teaching materials were prepared by the near-peer teachers beforehand and distributed to the students after each teaching session. The NPT programme coordinator supervised and provided necessary support to both near-peer teachers and near-peer students throughout the entire programme.

      The near-peer teachers and near-peer students consented to participate in this study via written consent. A structured post-participation Likert 6-point scale “Peer Tutors Own Assessment” questionnaire with written consent, which was adopted from Liew et al. (2015), was sent to the near-peer teachers after this Near-Peer Teaching Programme via Google form to explore their perceived benefits in three components which are 1) Skills Enhancement, 2) Motivation and 3) Career Direction [Expectation]. This questionnaire (Appendix 1) contains 14 items, with responses scale from strongly disagree (1) to strongly agree (6). Cronbach’s alpha was 0.801, 0.714, and 0.814 for the domains of skills enhancement, motivation, and career direction.

      Similarly, all the near-peer students who participated in this near-peer teaching programme were given the Peer Tutor Assessment questionnaire adopted from Liew et al. (2015), to fill in via Google form (Appendix 2). This questionnaire is to assess the acceptability of the teachings of the near-peer teachers. It contains 16 items that evaluate five domains: (1) Responsibility and Respect, (2) Information Processing, (3) Communication, (4) Critical Analysis, and (5) Self-Awareness. Subgroup analyses were conducted to evaluate whether sex and student background affect self-perceived outcomes of near-peer teachers using independent T-test. Each participant was given 3 weeks to complete the questionnaire. Several reminders were sent via email throughout these 3 weeks to each participant to encourage them to fill in the questionnaire. Both near-peer teachers and near-peer students’ data were checked for normality. The asymmetry fell between -1 and +1 and assumed relatively symmetrical and mesokurtic.

      III. RESULTS

      A total of 51 medical students in their final year signed up as near-peer teachers. Of these, 28 near-peer teachers completed the questionnaire (response rate 54.9%), while 49 out of 100 near-peer students who joined this NPT programme responded in this study (response rate 49.0%). Among those near-peer teachers who responded, there were 9 (32.1%) males and 19 (67.9%) females. More local students responded in this study than international students (75% vs 25%). The overall mean age ± SD of the near-peer teachers is 23.75 ±1.21 years old.  For the near-peer students, the overall mean age ± SD is 21.69 ± 0.74 years old. The number of international and local near-peer students who responded was similar. The data of the responses of both near-peer teachers and near-peer students that supports the findings of this study  is openly available at Figshare https://doi.org/10.6084/m9.figshare.26886517.v1 (Tang et al., 2024a) and https://doi.org/10.6084/m9.figshare.26886514.v1 (Tang et al., 2024b).

      Table 1 showed peer review findings to ensure their voices were represented. To ensure the trustworthiness of the findings, actions were taken to address credibility, dependability, confirmability, transferability, and reflexitivity are outlined in Table 2. Figure 1 indicates the near-peer teachers view of the benefits of involving in the student-led near-peer teaching program.

      Demographic

      Near-peer Teachers (n,%)

      Near-peer Students (n,%)

      Sex

      Male

      9 (32.1)

      27 (55.1)

      Female

      19 (67.9)

      22 (44.9)

      Age (Mean ± SD)

      23.75 ±1.21

      21.69 ± 0.74

      Student background

      Local

      21 (75)

      28 (57.1)

      International

      7 (25)

      21 (42.9)

      Table 1. Demographic data of the near-peer teachers

      Skills Enhancement  

      Mean score +/- SD 

      Improved own learning skills  

      5.21 ± 0.63 

      Improved practical teaching skills 

      5.39 ± 0.69 

      Improved understanding of educational principles 

      5.14 ± 0.93 

      Increased confidence in speaking to groups 

      5.54 ± 0.58 

      Improved organisation/planning skills 

      5.50 ± 0.64 

      Total Mean Score  

      5.36 ± 0.53 

      Motivation 

      Increased my desire to help fellow students 

      4.96 ± 1.14 

      Helped me to focus on practical skills 

      5.07 ± 0.60 

      Revised my own clinical skills 

      5.36 ± 0.56 

      Increased my desire to emulate good teaching I have had 

      5.32 ± 0.61 

      Encouraged me to do more 

      5.11 ± 0.92 

      Total Mean Score  

      5.16 ± 0.60 

      Career Direction (Expectation) 

      Motivated me to undertake more teaching trainings 

      4.96 ± 1.04 

      Helped me to decide on my career direction 

      4.71 ± 0.94 

      Teaching will be a major part of my career 

      4.36 ± 1.34 

      Enhanced my curriculum vitae 

      5.14 ± 0.65 

      Total Mean Score  

      4.79 ± 0.82 

       Total mean score for all domains

       5.11± 0.58

      Table 2. The mean score (± SD) for the self-evaluation of near-peer teachers in relation to (1) skills enhancement, (2) motivation and (3) career direction (expectation)

      Figure 1. Near-peer teachers view of the benefits of involving in the student-led NPT program: Analysis of post-participation questions responses (n=28) in Likert 6-point scale in relation to (a) skills enhancement, (b) motivation and (c) career direction (expectation)

      Based on the responses to the questionnaire, the near-pear teachers widely agreed that the NPT programme positively impacted them (5.11 ± 0.58), with the highest score in the domain of skills enhancement (5.36 ± 0.53), followed by motivation (5.16 ± 0.60) and career direction (4.79 ± 0.82). Most of the near-peer teachers considered this programme improved their skills in terms of teaching, organising, communicating and understanding educational principles (Table 2). 100% of them agreed that this NPT programme increased their confidence in speaking to groups and enhanced their planning and organising skills.  Furthermore, all 28 respondents (100%) reported being more motivated to revise their own clinical skills and focus more on their practical skills after teaching near-peer students to address the

      knowledge gap. A significant proportion of near-peer teachers felt more encouraged to participate in more teaching programmes in the future (n=27, 96.43%) and more inclined to help fellow students next time (n=25, 89.28%).  It is noteworthy that 96.43% of near-peer teachers agreeing that this NPT programme helped in deciding their career direction and motivated them to undertake more trainings. Besides, three quarters of them would consider teaching as their major career pathway in the future, with a mean score of 4.36 ± 1.34. Lastly, all 28 respondents (100%) agreed that joining the NPT programme had enhanced their curriculum vitae (100%).

      Table 3 shows the mean score of near-peer students’ evaluation of near-peer teachers in five domains after joining the near-peer teaching programme.

      Responsibility and Respect

      Mean Score +/- SD

      Completed all assigned tasks to the appropriate level

      5.27 ± 0.73

      Completed all assigned tasks on time

      5.08 ± 0.67

      Participated actively in the session

      5.14 ± 0.76

      Showed behaviour and input that facilitated learning

      5.16 ± 0.66

      Was punctual to the session

      5.08 ± 0.89

      Listened and showed respect for the opinions of others

      5.16 ± 0.66

      Total Mean Score

      5.15 ± 0.57

      Information Processing

       

      Brought in new information to share with the group

      5.16 ± 0.71

      Provided information that was relevant and helpful

      5.10 ± 0.77

      Seemed to use a variety of resources to obtain the information

      5.10 ± 0.82

      Total Mean Score

      5.12 ± 0.66

      Communication

       

      Was able to communicate ideas clearly

      5.10 ± 0.68

      Made comments and responses that were not confusing

      4.92 ± 0.84

      Total Mean Score

      5.01 ± 0.65

      Critical analysis

       

      Gave input that was focused and relevant to the case

      4.94 ± 0.75

      Gave a summary of the session

      4.90 ± 0.82

      Gave a summary of the session that showed evidence of reflection and evaluation

      4.94 ± 0.83

      Total Mean Score

      4.93 ± 0.66

      Self-awareness

       

      Appeared to be able to acknowledge his/her own strengths and weaknesses

      5.12 ± 0.73

      Accepted and responded to criticism gracefully

      5.10 ± 0.74

      Total Mean Score

      5.11 ± 0.62

      Total mean score for all domains 

      5.06 ± 0.51

      Table 3. The mean score (± SD) for the near-peer students’ evaluation of near-peer teachers after the NPT programme in relation to (1) responsibility and respect, (2) information processing, (3) communication, (4) critical analysis and (5) self-awareness

      When asked to evaluate the teaching of their near-peer teachers, the near-peer students considered the near-peer teachers demonstrating positive outcomes in all five domains (5.06 ± 0.51). The near-peer teachers were thought to have a high degree of professionalism in terms of responsibility and respect (5.15 ± 0.57) and self-awareness (5.11 ± 0.62). The most outstanding attribute demonstrated was the ability to complete assigned tasks appropriately (5.27 ± 0.73). Besides, the near-peer teachers performed satisfactorily to process information (5.12± 0.66), communicate (5.01± 0.65) and analyse critically (4.93 ± 0.66). However, the near-peer teachers were identified to score slightly lower in making non-confusing comments and responses (4.92 ± 0.84) as well as giving a summary of the session (4.90 ± 0.82).

      Variables

      Independent T- test

      Mean difference (95% CI)

      P-value

      Sex (Male vs Female)

      Skills

      -0.101 (-0.549, 0.348)

      0.649

      Motivation

      0.118 (-0.385, 0.621)

      0.634

      Career Direction (Expectation)

      0.507 (-0.154, 1.169)

      0.127

      Students’ background (Local vs International)

      Skills

      -0.210 (-0.688, 0.268)

      0.376

      Motivation

      -0.276 (-0.810, 0.258)

      0.297

       Career Direction (Expectation)

      0.280 (-0.439, 1.034)

      0.414

      Table 4. Comparison of self-perceived outcomes of near-peer teachers between male and female, local and international students (Independent t-test)

      The independent t-test were performed to find out the association between sex and perceived outcomes of the near-peer teachers. Our study revealed that it is statistically insignificant between male and female in the perceived benefits for skills (P = 0.649), motivation (P = 0.549) and career direction (P = 0.127).

      Besides, there is no correlation between students’ background and the three measured outcomes. There is statistically insignificant between local and international students in term of skills (P = 0.376), motivation (P = 0.397) and career direction (P = 0.414).

      IV. DISCUSSION

      This study provides valuable insights into the background and characteristics of the final-year medical students who voluntarily participated in a student-led, student-run NPT programme. The outcomes of their participation concerning their perceived benefits in terms of skills, motivation, and expectations were investigated. Overall, the near-peer teachers reported that this NPT programme helped them tremendously to improve their skills in terms of learning and teaching, which might be driven by their primary motive for joining this programme. This finding was similar to previous studies, which showed skills enhancement in volunteer near-peer teachers (Buckley & Zamora, 2007; Liew et al., 2015). Our study further reaffirms the plausibility of a student-run NPT programme to enhance teaching and learning skills. However, due to the voluntary nature of participation in this near-peer teaching programme, the students who are likely most in need of skill enhancement may have been omitted from this programme, and they might be less equipped to teach after they graduate. Some studies recommended more incentives to be given to such students to encourage them to make use of the opportunities offered (Buckley & Zamora, 2007).

      The motivation evaluated includes both self-actualising inner motivations to improve their clinical and practical skills and the external, tangible desire to help fellow students. The high motivation score suggests the reinforcement of a desirable attitude towards future educational and teaching responsibilities, which matches the GMC’s emphasis on the teaching role of doctors (General Medical Council, 2015). A couple of reasons may explain this: firstly, the near-peer teachers are final-year medical students, who will sit for their final examinations very soon and are desperately finding ways to improve their learning. The process of teaching, which requires extensive preparation, a comprehensive understanding of the content, dynamic synthesis, and anticipation of the questions that may be asked of them, forms an efficient learning strategy. Secondly, the near-peer teachers, inspired by the excellent teaching they once had, wish to impart good teaching to the near-peer students going through the same journey.

      Although many near-peer teachers are more motivated to be involved in more teaching and even take up teaching training courses in the future, the influence is not apparent in the long-term career direction. This could be explained well by the brief intervention of this NPT programme that lasted 3 months. However, the lucrative income opportunities in other medical specialities and the limited exposure to medical education pathways in undergraduate medical schools are some factors that may sway them away from considering medical education as their primary career pathway (Puri et al., 2021; Sarikhani et al., 2021). Therefore, more effort should be directed to increase teaching opportunities and to raise awareness of medical education career options in the undergraduate medical school programme. This includes developing Student Selected Components focusing on medical education and giving opportunities to shadow clinical teaching staff (Liew et al., 2015; Wilson et al., 2008).

      After participating in this study, the near-peer students evaluated the near-peer teachers highly in all the domains. This provides a strong indication of the recognition and acceptance of the teaching skills of the near-peer teachers. In addition, the ability of the near-peer teachers to demonstrate responsibility and respect throughout this programme shows their preparedness to work under the GMC with desirable attitudes and professionalism. Previous studies have also shown that near-peer teachers gain more subjectively and objectively than students (Liew et al., 2015; Ten Cate & Durning, 2007). This can be related to the underpinning of the psychological and social theories behind the dynamics between near-peer teachers and near-peer students (Loda et al., 2019). The theoretical model of cognitive and social congruence explains the positive evaluation of the near-peer teachers (Loda et al., 2019; Rollmann et al., 2023). The proximity of age between the near-peer teachers and the near-peer students enables them to share similar knowledge frameworks, language and social roles.  Besides, near-peer teachers are perceived to be more approachable and understanding of the needs and struggles of the near-peer students. This may be because the near-peer teachers have had similar experiences themselves. Therefore, near-peer teachers are better able to process difficult concepts and frameworks, emphasize the key points, and communicate the information using familiar and non-confusing language to ensure that near-peer students comprehend better (Loda et al., 2019; Loda et al., 2020). The perceived barrier to providing feedback to near-peer teachers is also lower compared to faculty-led staff, as the age difference between them is much smaller. This might suggest why near-peer teachers feel less offended by criticism and are more likely to accept and respond to criticism gracefully (Loda et al., 2019). Nevertheless, the near-peer students thought that some near-peer teachers experienced some difficulties in giving relevant input and summary of the sessions, thus necessitating more structured pedagogical training for near-peer teachers in this aspect. Mastering these teaching skills would allow the near-peer students to appreciate better the big picture and key takeaway points of each lesson (Khaw & Raw, 2016).

      One of the focuses of this study is to analyse the sex-specific difference of the perceived outcomes of the near-peer teachers. Although there is an appropriate twice female near-peer teachers who responded to this study compared to male near-peer teachers, the results shows that the sex-specific difference in the perceived improvement in motivation, skills and career direction is not significant. There is no sex-specific difference in term of enthusiasm and motivation to involve in near-peer teaching (Messerer et al., 2021).  Throughout the whole process, they receive similar gender-equitable support and guidance without any discrimination.

      V. LIMITATIONS AND RECOMMENDATIONS

      As few studies have reported outcomes for a purely student-led student-run NPT programme, this study offers valuable insights concerning the perceived benefits for near-peer teachers. However, this study has several limitations. Firstly, given the relatively small sample size of the near-peer teachers and the subjective nature of the self-reported questionnaire, the results may benefit from further objective testing, such as correlation with examination results. Secondly, this study is only carried out in a single medical school, with a slight variation in the implementation of the NPT compared to other institutions. Verification of these results across various medical schools would strengthen these findings. This study thus calls for more structured student-led peer teaching programmes to be implemented in more medical schools and to be assessed longitudinally to evaluate the association between the student-led peer teaching programme and the outcomes of participation of the near-peer teachers. It may also be worthwhile to investigate and assess the perspective of near-pear teachers who demonstrated interest in medical education, as well as to evaluate the long-term outcomes in career direction for medical graduates who once participated in near-peer teaching programmes.

      VI. CONCLUSION

      In conclusion, this purely student-led, student-run near-peer teaching programme likely improved the final-year medical students in fulfilling the “Doctors as Teachers” responsibilities outlined by the GMC. Besides, the near-peer teachers also reported having positive outcomes in their skills and career direction.  Likewise, from the perspective of the near-peer students, the near-peer teachers demonstrated outstanding skills and professionalism in all five domains: responsibility and respect, information processing, communication, critical analysis and self-awareness. Possessing the skills and professionalism fulfils the expectations of GMC for healthcare professionals to provide the right care at the right time with the right skills for the good of patients.

      Notes on Contributors

      Kevin Xuan Hong Tang was a final year medical student who conceptualised and designed the study, reviwed the literature, conducted the data collection and analysis, prepared the figures and wrote the manuscript.

      Koon Kee Teo was a final year medical student who reviewed the literature, collected and analysed the data, prepared the figures and helped in writing the manuscript.

      Kye Mon Min Swe is an Associate Professor in Education Research in NUMed. She participated in conceptualising the study, performed statistical analysis and drafted, reviewed and edited the manuscript. All the authors have read and approved the final manusript.

      Ethical Approval

       This study titled “Embracing Doctors as Teachers: Evaluating the Student-led Near-Peer Teaching at Transnational Campus” was approved by the Research Management Committee and the Newcastle University Ethics Committee (Approval number 45070/2023).

      Data Availability

      The data that support the findings of this study are openly available in Figshare repository, as below,

      https://doi.org/10.6084/m9.figshare.26886517.v1 (Tang et al., 2024a) and

      https://doi.org/10.6084/m9.figshare.26886514.v1 (Tang et al., 2024b).

      Acknowledgement

      The authors would like to thank the NUMed Medical Education Society for carrying out this programme. The authors are grateful for Professor Vishna Devi Nadarajah for her support and feedback in this research.

      Funding

      The authors declare that there is no funding received in this study.

      Declaration of Interest

      The authors declare that there is no conflict of interest.

      References

      Botelho, M. G., Boubaker, B., & Wong, I. B. (2022). Near‐peer teaching for learning clinical photography skills: Perceptions of students. European Journal of Dental Education, 27(3), 477–488. https://doi.org/10.1111/eje.12831

      Buckley, S., & Zamora, J. (2007). Effects of participation in a cross year peer tutoring programme in clinical examination skills on volunteer tutors’ skills and attitudes towards teachers and teaching. BMC Medical Education, 7(1), Article 20. https://doi.org/10.1186/1472-6920-7-20

      Burgess, A., McGregor, D., & Mellis, C. (2014). Medical students as peer tutors: A systematic review. BMC Medical Education, 14(1), Article 115. https://doi.org/10.1186/1472-6920-14-115

      Busari, J. O., Prince, K. J., Scherpbier, A. J., Van Der Vleuten, C. P., & Essed, G. G. (2002). How residents perceive their teaching role in the clinical setting: A qualitative study. Medical Teacher, 24(1), 57-61. https://doi.org/10.1080/00034980120103496

      Cook, V., Fuller, J. H., & Evans, D. E. (2010). Helping students become the medical teachers of the future – The Doctors as Teachers and Educators (DATE) programme of Barts and the London School of Medicine and Dentistry, London. Education for Health, 23(2), 415. https://doi.org/10.4103/1357-6283.101487

      De Menezes, S., & Premnath, D. (2016). Near-peer education: A novel teaching program. International Journal of Medical Education, 7, 160–167. https://doi.org/10.5116/ijme.5738.3c28

      Frearson, S., & Gale, S. (2017). Educational opportunities on a ward round; utilising near-peer teaching. Future Healthcare Journal, 4(1), 19–22. https://doi.org/10.7861/futurehosp.4-1-19

      Furmedge, D. S., Iwata, K., & Gill, D. (2014). Peer-assisted learning – Beyond teaching: How can medical students contribute to the undergraduate curriculum? Medical Teacher, 36(9), 812–817. https://doi.org/10.3109/0142159x.2014.917158

      General Medical Council. (2011, February). Developing teachers and trainers in undergraduate medical education. Retrieved June 2, 2024, from https://www.gmc-uk.org/-/media/documents/developing-teachers-and-trainers-in-undergraduate-medical-education—guidance-0815_pdf-56440721.pdf

      General Medical Council. (2015, July). Outcomes for graduate (Tomorrow’s doctors). Retrieved June 2, 2024, from  https://www.gmcuk.org//media/documents/Outcomes_for_graduates_ Jul_15_1216.pdf_61408029.pdf

      General Medical Council. (2023, August 22). Good medical practice. Retrieved June 2, 2024, from https://www.gmc-uk.org/-/media/documents/good-medical-practice-2024—english-102607294.pdf

      Hardie, P., O’Donovan, R., Jarvis, S., & Redmond, C. (2022). Key tips to providing a psychologically safe learning environment in the clinical setting. BMC Medical Education, 22(1), Article 816. https://doi.org/10.1186/s12909-022-03892-9

      Hettle, D., & Morgan, J. (2019 December 2-3). “Teaching and Training for the future”: Final-year students’ perspectives on their role as near-peer teachers” [Conference presentation abstract]. Developing Excellence in Medical Education, Manchester. https://www.researchgate.net/publication/344439487_Teaching_and_Training_for_the_future_Finalyear_students’_perspectives_on_their_role_as_near-peer_teachers

      Khaw, C., & Raw, L. (2016). The outcomes and acceptability of near-peer teaching among medical students in clinical skills. International Journal of Medical Education, 7, 189–195. https://doi.org/10.5116/ijme.5749.7b8b

      Knobloch, A. C., Ledford, C. J., Wilkes, S., & Saperstein, A. K. (2018). The impact of near-peer teaching on medical students’ transition to clerkships. Family Medicine, 50(1), 58–62. https://doi.org/10.22454/fammed.2018.745428

      Liew, S., Sow, C., Sidhu, J., & Nadarajah, V. D. (2015). The near-peer tutoring programme: embracing the ‘doctors-to-teach’ philosophy – A comparison of the effects of participation between the senior and junior near-peer tutors. Medical Education Online, 20(1), 27959. https://doi.org/10.3402/meo.v20.27959

      Loda, T., Erschens, R., Loenneker, H., Keifenheim, K. E., Nikendei, C., Junne, F., Zipfel, S., & Herrmann-Werner, A. (2019). Cognitive and social congruence in peer-assisted learning – A scoping review. PloS One, 14(9), e0222224. https://doi.org/10.1371/journal.pone.0222224

      Loda, T., Erschens, R., Nikendei, C., Giel, K., Junne, F., Zipfel, S., & Herrmann-Werner, A. (2020). A novel instrument of cognitive and social congruence within peer-assisted learning in medical training: construction of a questionnaire by factor analyses. BMC Medical Education, 20(1), Article 214. https://doi.org/10.1186/s12909-020-02129-x

      Messerer, D. A. C., Kraft, S. F., Horneffer, A., Messerer, L. A. S., Böckers, T. M., & Böckers, A. (2021). What factors motivate male and female Generation Z students to become engaged as peer teachers? A mixed‐method study among medical and dental students in the gross anatomy course. Anatomical Sciences Education, 15(4), 650–662. https://doi.org/10.1002/ase.2114

      Pierce, H., Kermack, A., & Foley, V. (2024). What training do foundation doctors feel they require to develop as clinical teachers? Journal of Education and Training Studies12(2), 106113. https://doi.org/10.11114/jets.v12i2.6714

      Puri, P., Landman, N., Smoldt, R. K., & Cortese, D. (2021). Quantifying the financial value of clinical specialty choice and its association with competitiveness of admissions. Curēus, 13(2), e13272. https://doi.org/10.7759/cureus.13272

      Qureshi, Z., Gibson, K., Ross, M., & Maxwell, S. (2013). Perceived tutor benefits of teaching near peers: Insights from two near peer teaching programmes in South East Scotland. Scottish Medical Journal, 58(3), 188–192. https://doi.org/10.1177/0036933013496935

      Rollmann, I., Lauter, J., Kuner, C., Herrmann-Werner, A., Bugaj, T. J., Friederich, H., & Nikendei, C. (2023). Tutors ́ and students’ agreement on social and cognitive congruence in a Sonography peer-assisted-learning scenario. Medical Science Educator, 33(4), 903–911. https://doi.org/10.1007/s40670-023-01814-y

      Ross, M. T., & Cameron, H. S. (2007). Peer assisted learning: A planning and implementation framework: AMEE Guide no. 30. Medical Teacher, 29(6), 527–545. https://doi.org/10.1080/01421590701665886

      Sarikhani, Y., Ghahramani, S., Bayati, M., Lotfi, F., & Bastani, P. (2021). A thematic network for factors affecting the choice of specialty education by medical students: A scoping study in low-and middle-income countries. BMC Medical Education, 21(1), Article 99. https://doi.org/10.1186/s12909-021-02539-5

      Tang, K. X. H., Teo, K. K., & Swe, K. M. M. (2024a). The evaluation of near-peer teachers in five domains completed by near-peer students after joining peer teaching program (Version 1) [Data set].  Figshare. https://doi.org/10.6084/m9.figshare.26886517.v1

      Tang, K. X. H., Teo, K. K., & Swe, K. M. M. (2024b). Near-peer teachers self-evaluation data after joining peer teaching program (Version 1) [Data set]. Figshare. https://doi.org/10.6084/m9.fig share.26886514.v1

      Ten Cate, O., & Durning, S. (2007). Dimensions and psychology of peer teaching in medical education. Medical Teacher, 29(6), 546–552. https://doi.org/10.1080/01421590701583816

      Wilson, S., Denison, A. R., & McKenzie, H. (2008). A survey of clinical teaching fellowships in UK medical schools. Medical Education, 42(2), 170–175. https://doi.org/10.1111/j.1365-2923.2007.02933.x

      Yu, T. C., Wilson, N. C., Singh, P. P., Lemanu, D. P., Hawken, S. J., & Hill, A. G. (2011). Medical students-as-teachers: A systematic review of peer-assisted teaching during medical school. Advances in Medical Education and Practice, 2, 157–172. https://doi.org/10.2147/amep.s14383

      Zheng, B., & Wang, Z. (2022). Near-peer teaching in problem-based learning: Perspectives from tutors and tutees. PloS One, 17(12), e0278256. https://doi.org/10.1371/journal.pone.0278256

      *Kye Mon Min Swe
      1, Jalan Sarjana 1, Educity,
      79200 Iskandar Puteri, Johor, Malaysia
      Email: kye-mon.min-swe@newcastle.edu.my

      Submitted: 13 June 2024
      Accepted: 4 February 2025
      Published online: 1 July, TAPS 2025, 10(3), 26-36
      https://doi.org/10.29060/TAPS.2025-10-3/OA3439

      Humairah Zainal1, Xin Xiaohui1, Julian Thumboo1,2,3 & Fong Kok Yong2,3

      1Health Services Research Unit, Singapore General Hospital, Singapore; 2Department of Rheumatology and Immunology, Singapore General Hospital, Singapore, 3Duke-NUS Medical School, National University of Singapore, Singapore

      Abstract

      Introduction: As healthcare becomes increasingly digital, undergraduate medical students receive limited formal training in digital healthcare technologies (DHTs). Additionally, stakeholders’ perspectives on essential digital health competencies (DHCs) across generations remain underexplored. This comparative study aims to identify knowledge gaps, inform evolving expectations, and promote continuous learning by comparing medical students’ and senior clinicians’ views on essential DHCs in Singapore.

      Methods: Individual semi-structured interviews were conducted with medical students, clinical educators, and clinical teachers. Clinical-year medical students from all three local medical schools were recruited using convenience and snowball sampling, while purposive sampling ensured diverse representation of clinicians across Singapore’s public healthcare organisations, focusing on those engaged in education-related activities. Data were collected from September 2020 to February 2023 until thematic saturation was achieved, and analysed using qualitative thematic analysis.

      Results: Sixty-three participants took part, including 30 medical students, 12 clinical educators and 21 clinical teachers. All clinicians had more than 10 years of clinical experience and came from 24 different disciplines and 12 healthcare institutions. The findings showed unanimous support for integrating electronic health records in patient communication, and training in DHTs like telemedicine and point-of-care ultrasound. Discrepancies emerged; clinicians prioritised physical examination, while students viewed it as potentially replaceable by DHTs. Furthermore, while students valued healthcare-related smartphone applications in patient care, few clinicians shared this view, citing privacy and security concerns.

      Conclusion: Evaluating stakeholders’ perspectives highlights evolving clinical skills and essential DHCs for medical students, potentially informing global DHC training programmes.

      Keywords:           Technology, Medical Education, Curriculum, Clinical Competence, Digital Competence, Qualitative, Singapore

      Practice Highlights

      • An inter-generational perspective in medical education leverages the strengths of both medical students and senior clinicians, ensuring a balanced approach that keeps pace with technological advancements while maintaining the core principles of patient care.
      • The digital health competencies deemed essential for future clinical practice are consistent across all stakeholders.
      • The study highlights the emerging need for additional competencies among medical trainees so as to meet the evolving demands of healthcare delivery.
      • Regularly comparing the perspectives of various stakeholders in the healthcare system enables educational institutions to adapt and refine their curricula continuously.

      I. INTRODUCTION

      Digital healthcare, which leverages advanced technologies to enhance, complement, or replace traditional healthcare services, is becoming increasingly common in clinical practice (Alkire (née Nasr) et al., 2020). Digital healthcare technologies (DHTs) such as electronic health records (EHR), telemedicine, and Artificial Intelligence (AI) have significantly improved data management, diagnostics, and patient care (Clay-Williams et al., 2023; Food and Drug Administration, 2019; Welcher et al., 2018;). However, many studies and reviews have highlighted a global lack of formal and systematic training in DHTs for undergraduate medical students (Aungst & Patel, 2020; Edirippulige et al., 2018; Tudor Car et al., 2021). While recommendations for integrating digital health education (DHE) exist, they often propose standalone courses focused on specific DHTs rather than embedding DHE into the core medical curriculum (Tudor Car et al., 2021; Zainal et al., 2023a; Zainal et al., 2023b).  Barriers to consistently incorporating DHE into medical curricula, as reported in countries such as Canada, Germany, the United Kingdom (U.K.), and the United States (U.S.) include limited faculty expertise, curriculum overload and fragmented implementation efforts (Gillissen et al., 2022; Hurley et al., 2011; Machleid et al., 2020; Sit el al., 2020; Sorg et al., 2022).

      Furthermore, there is limited research exploring the perspectives of both medical students and senior clinicians on the relevant digital health competencies (DHCs) required for clinical care. DHCs encompass the knowledge, skills, and attitudes essential for assessing, diagnosing, and treating patients in the digital age. An inter-generational comparison of these perspectives is crucial for identifying gaps in the existing curricula and ensuring medical education aligns with the rapidly evolving healthcare landscape. Medical students can offer fresh insights into emerging tools and platforms while senior clinicians can contribute perspectives on clinical skills given their wealth of clinical experience.

      Despite Singapore’s advanced digitalisation, challenges remain in standardising DHE across its three medical schools: Yong Loo Lin School of Medicine at National University of Singapore (NUS), Lee Kong Chian School of Medicine at Nanyang Technological University (NTU) and Duke-NUS Medical School (Duke-NUS) (Zainal et al., 2023a). Although these institutions boast state-of-the-art facilities, a disconnect persists between medical training and clinical practice due to various institutional and structural barriers (Zainal et al., 2023a). While efforts to integrate DHE, such as virtual reality courses, have been initiated, there are variations in content and duration across institutions, underscoring the lack of a standardised approach (Zainal et al., 2023a).

      This study uses Singapore as a case study to explore the perceptions of medical students, clinical educators, and teachers regarding DHCs, with the aim of improving DHE integration into undergraduate medical curricula. Specifically, it compares the views of these stakeholders in identifying any digital competencies that may be relevant for clinical practice in an increasingly digital healthcare landscape. By focusing on Singapore, this study not only sheds light on gaps in the local curricula but also provides insights that can inform global efforts to strengthen DHE integration in medical education.

      II. METHODS

      A. Setting and Sample

      A qualitative study was conducted using individual semi-structured interviews with medical students, clinical educators, and clinical teachers. Participants were identified by Principal Investigator (P.I.) FKY, based on their year of study and clinical roles respectively, ensuring relevant knowledge and experience.

      The study employed a combination of convenience sampling and snowballing technique to recruit students from all three local medical schools. Clinical-year students (third to fifth years) were included due to their potential exposure to DHTs during clinical rotations, while pre-clinical students were excluded. Convenience sampling was initially used for ease of access, and snowballing technique was subsequently employed to expand participation and capture diverse perspectives.

      To ensure diverse representation among clinicians, purposive sampling was applied across Singapore’s three public healthcare organisations. Clinical educators dedicating over 20 per cent of their time to education-related activities (e.g., teaching, curriculum planning, research, and administration) and clinical teachers contributing less than 20 per cent of their time to such roles were included. Clinicians not involved in educational roles were excluded from the study.

      This study was classified as a quality improvement (QI) project on medical education curricula by the SingHealth Research Integrity, Compliance, and Ethics (RICE) committee. As such, it did not meet the criteria for human subject research and was granted an ethical waiver by SingHealth Institutional Review Board (Reference Number: 2020/2880). Nonetheless, this research adhered strictly to the ethical principles of the Declaration of Helsinki and institutional guidelines.

      Data collection occurred from September 2020 to February 2023. Participants were invited by the P.I. via email detailing the study’s purpose, procedures, potential risks, and benefits, with consent obtained before the interviews. To address potential biases due to the P.I.’s professional relationships, interviews were conducted by a research fellow without prior connections to participants. Verbal consent was recorded at the start of each Zoom session, and participants were reminded of their right to withdraw, with pre-withdrawal data retained for comprehensive analysis.

      To ensure anonymity, participants were assigned coded identifiers (‘MS’ for medical students, ‘CL’ for clinicians). Identifying information and audio recordings were securely stored and separated from the main dataset. Results were reported with care to avoid identifying individuals, and the data was exclusively used for curriculum development.

      B. Data Collection

      The interview guide followed Kallio et al.’s (2016) framework, including pre-requisites for semi-structured interviews, literature review, expert consultation, preliminary guide development, pilot testing with five participants, and finalisation. Questions addressed clinical competencies, their relevance to digital healthcare, and ways medical schools can better prepare students for digitalisation (Tables 1 and 2). Interviews included medical students and clinicians across specialties to ensure diverse perspectives.

      The study involved 63 participants, with sample size guided by theoretical and practical considerations. Data collection continued until thematic saturation was reached, ensuring all relevant perspectives were captured. Practical constraints like time and resources influenced the participant number, but the focus remained on data richness and diversity for a comprehensive exploration of the research questions.

      Sixty interviews were conducted via Zoom due to COVID-19 restrictions, with three in-person interviews held under local health guidelines. Each session lasted about 40 minutes and was audio-recorded. Transcriptions were generated using Otter AI and reviewed for accuracy by the P.I. and research fellow.

      No.

      Topics

      Questions and prompts

      1.

      Clinical skills that doctors should have

      In general, what are the clinical skills that a medical doctor should have?

      Are there any clinical skills that have become less essential or relevant in this digital age?

      Are there any other clinical skills that have been fulfilled or transformed by digital technology?

      What new skills, clinical or otherwise, should a medical doctor have today and in the future especially in light of the digital age?

      2.

      Medical school curriculum

      Do you think the clinical skills taught in your medical school have adequately prepared you for clinical practice, especially in today’s modern setting?

      Prompts:

      Are there any modules in your formal curriculum that might have exposed you to the digital aspects of medicine, such as telemedicine, AI or healthcare informatics?

      Are you aware of any elective courses or programmes that might have done the same?

      3.

      Exposure to digital technologies in clinical practice

      Have you ever adopted or come across any forms of digital technology such as robotics surgery, AI or big data in your clinical practice yet?

      Prompts:

      If yes, where did you encounter this?

      – What digital healthcare technologies would you be interested in learning if given the opportunity, and why?

      4.

      Experience with digital technologies

      Have you ever encountered any recurring challenges in clinical practice that you foresee could be overcome by digital technologies?

      5.

      Role of various stakeholders

      What do you think medical schools can do to help doctors optimise the use of digital healthcare technologies in the future?

      What do you think professional bodies such as Singapore Medical Association (SMA) and Academy of Medicine can do to help doctors make full use of digital healthcare technologies in the future?

      What do you think the government can do to help doctors optimise the use of digital healthcare technologies in the future?

      6.

      Other thoughts on how the medical school curriculum can better prepare students

      Do you have any other comments on how the medical school curriculum can better prepare you for future clinical practice?

      Any other comments on digital transformations of medicine or healthcare before we end this interview?

      Table 2. Interview questions for clinical educators and teachers

      C. Data Analysis

      Thematic analysis followed Braun and Clarke’s six-step framework to explore curricular gaps (Braun & Clarke, 2006). Two researchers independently coded the data, resolving discrepancies through discussion to refine the codebook and ensure consistency. Credibility was enhanced by triangulating findings across participants from various medical schools and healthcare clusters, capturing diverse perspectives and minimising bias.

      To contextualise the findings, comparisons were made with studies from other high-income countries undergoing similar healthcare digitalisation. Additionally, recent data from stakeholders, including medical school leaders, were analysed to understand the digital competencies needed for future clinical practice. Reporting adhered to the Standards for Reporting Qualitative Research (O’Brien et al. 2014).

      III. RESULTS

      Sixty-three participants took part in our study. Thirty of them were medical students who were undergoing clinical rotations in various disciplines, with 10 coming from each school. Twelve were clinical educators and 21 were clinical teachers. The clinicians were aged between 44 and 70 years. All had more than 10 years of clinical experience and came from 24 different disciplines and 12 healthcare institutions. A summary of their demographics is reported in Table 3 and illustrative excerpts are found in Table 4.

      Profile

      Number of participants

      Gender

       

      Medical students:

      Male

      Female

       

      15 (50%)

      15 (50%)

      Clinical educators and teachers:

      Male

      Female

       

      26 (78.8%)

      7 (21.2%)

      Year of study/ specialty

       

      Medical students:

      Third year

      Fourth year

      Fifth year

       

      8 (26.7%)

      14 (46.7%)

      8 (26.7%)

      Clinical educators and teachers:

      Roles

      Clinical educator

      Clinical teacher**

      Number of participants

      12

      21

      1 interviewee per discipline

      2 interviewees per discipline

      3 interviewees per discipline

      Cardiology

      Dermatology

      Ear, Nose and Throat Surgery

      Endocrinology

      Gastroenterology and Hepatology

      Hepatobiliary Surgery

      Infectious Diseases Medicine

      Internal Medicine

      Medical Oncology

      Neurology

      Obstetrics and Gynaecology

      Ophthalmology

      Pathology

      Public Health

      Radiology

      Renal Medicine

      Respiratory and Critical Care Medicine

      Vascular Surgery

      Anaesthesiology

      Geriatrics Medicine

      Paediatrics Medicine

      Emergency Medicine

      Family Medicine

      Rheumatology

       

       

       

       

       

       

       

       

       

       

       

       

       

       

       

      Table 3. Demographics of participants

      Four major themes were derived from the data.

      A. Limited and Inconsistent Exposure to DHTs

      Students reported limited exposure to DHTs in core medical curricula, with training mainly through electives, select programmes, and student-driven initiatives. Exposure to health informatics and telemedicine was brief, with telemedicine training primarily prompted by the COVID-19 pandemic. Other DHTs like robotic surgery, virtual reality, and POCUS were inconsistently incorporated across medical schools. Some clinicians also reiterated the need for DHT exposure in public healthcare institutions, as doing so could inspire medical students to engage in innovation, research, and development in cutting-edge fields like robotics.

      B. Unified Views among Clinicians and Students on Essential Competencies

      Clinicians and students shared unified perspectives on incorporating key DHCs like EHR, telemedicine, and POCUS into medical curricula. These competencies are viewed as crucial for preparing students for contemporary clinical practice.

      Participants emphasised the need to balance EHR use with effective patient interaction. M25, a third-year student, highlighted the importance of maintaining empathy and humanistic values, while CL27, a rheumatologist, stressed integrating EHR data into patient discussions to improve engagement and understanding of health conditions.

      All groups also agreed on the importance of telemedicine training. Students expressed interest in learning how to identify conditions suitable for virtual consultations, build patient rapport, and conduct comprehensive assessments via tele-platforms. MS5 noted the challenges of teleconsultation, such as interpreting body language and addressing legal responsibilities. Clinicians like CL14 underscored the need to teach students how to select patients for teleconsultation and identify red flags effectively.

      Additionally, both clinicians and students strongly supported incorporating POCUS into the curriculum. Students acknowledged its utility in bedside diagnostics, while clinicians like CL9 and CL18 highlighted its value in resource-limited settings and its role in enhancing diagnostic accuracy. They advocated for routine training in ultrasound, considering its increasing use in general practice and as a critical tool in diverse healthcare settings. These unified views suggest the need to prioritise these DHCs in medical education to meet the evolving demands of healthcare delivery.

      C. Divergent Opinions on Traditional Clinical Skills

      Discrepancies in opinions emerged regarding traditional clinical skills, particularly physical examination. Students often viewed diagnostic imaging and tests as superior to these skills, perceiving physical exams as less critical. For instance, M11, a Year 4 student, argued that ultrasonography provides more precise diagnoses, making physical examination less relevant.

      Clinicians, however, emphasised preserving traditional skills like history-taking and physical examination, especially for resource-limited settings. CL3, an ophthalmologist, expressed concern that reliance on diagnostic tools has led to the erosion of these essential skills.

      D. Emerging Technologies and Concerns

      The perceived competencies for medical education varied between students and senior clinicians. Students valued wearables and healthcare apps for patient empowerment, personalised care, and strengthening doctor-patient trust. For instance, MS30, a fourth-year student, suggested that wearable data could help create personalised care plans and enhance clinical practice.

      Clinicians, however, expressed concerns about the privacy and security of patient data in these apps. They emphasised the responsibility to protect patient information and were cautious about using apps that did not meet security standards. These differing opinions highlight the need for a DHE curriculum that addresses both student interests and clinician concerns.

      In our sample selection, we included both clinical educators and clinical teachers to capture a range of perspectives on the integration of DHCs into medical education. However, our analysis did not reveal significant differences in the views expressed by these two groups. Both shared similar opinions on the importance of teaching DHCs such as EHR, telemedicine and POCUS. This alignment may be attributed to their shared roles in preparing medical students for clinical practice, regardless of their specific titles or responsibilities. As such, the findings reflect a unified perspective among clinicians involved in medical education, despite the initial distinction made in the sample selection.

      Themes

      Quotes from Participants

      Limited and inconsistent exposure to DHTs

      In private practice, particularly in Family Medicine, telemedicine is fairly common. But I don’t think it is a model that has been adopted in public hospitals or tertiary settings yet… Telemedicine is a new thing that only came up because of COVID. (MS10, Year 4)

      In Korea, medical students are already exposed to advanced medical robots, with 40 robots used in operating theaters. Introducing a robotics module for medical students could spark interest and lead to innovations in robotics and research. (CL20, Otorhinolaryngology)

      Essential competencies for clinical practice

      EHR:

      A skill that should be taught to students is the ability to strike a balance between facing the screen and interacting with patients. In clinics, you need to face the computer to scribe your notes while talking to the patient. (MS8, Year 5).

      In the United States, students are trained in communication skills that incorporate electronic medical records into patient interactions. This training is lacking here, which is why patients often complain that doctors focus too much on the computer screen during consultations. (CL27, Rheumatology)

      Telemedicine:

      I wish to be adept at performing teleconsultation, as this seems to be the up-and-coming practice. But with this comes many ambiguities such as reading body language, building rapport, and with that comes legal liabilities and responsibilities, like what exactly are the conditions that are suitable for teleconsultation, what are the ways to circumvent issues with teleconsultation and how to best integrate it into a patient’s individualized care. (MS5, Year 5)

      A skill students need to learn is how to interact well in a non-face-to-face context, like in a tele-setting. They need to learn how to select patients appropriately and spot red flags. (CL14, Respiratory Medicine)

      POCUS:

      Ultrasound is currently not taught routinely to students. It is important to know ultrasound because a lot of GPs are now using it in their clinics as an adjunct to diagnosing conditions and confirm clinical examinations. (CL9, General Surgery)

      Every medical student should learn ultrasound. It’s a useful skill for those who may work in developing countries, where they may need to provide treatment directly at the point of care. (CL18, Orthopaedic Surgery)

      Divergent opinions on traditional clinical skills

      Students’ perspective:

      When it comes to clinical signs, if a patient has a mildly enlarged liver, if we are able to feel it, we can actually do tests like ultrasound or further testing to know exactly the length, size or span of the liver. So, it doesn’t actually matter whether you feel it on the physical examination or not. (M11, Year 4)

      When assessing heart murmurs in patients with heart conditions, it is not always important to precisely diagnose the type of murmur, as an echocardiogram will ultimately provide the exact diagnosis. Similarly, for cases like detecting abdominal masses, even if physical examination is challenging, such as with obese patients, an ultrasound can be readily ordered to confirm the findings. (MS11, Year 4)

      Clinicians’ perspective:

      As doctors become more dependent on investigation findings and diagnostic tests, I do see basic skills such as history-taking and physical examination skills have eroded. (CL3, Ophthalmology)

      Imaging has somewhat replaced traditional clinical examination skills, but these skills remain important and relevant. I still emphasise teaching clinical examination as a fundamental component before turning to imaging. However, most schools now default to imaging, which might be a drawback of technological advances. We need to strike a balance by ensuring clinical examination skills are adequately taught alongside imaging. (CL9, General Surgery)

      Emerging technologies and concerns

      · Wearables and healthcare apps

      We should empower patients by improving their access to reliable medical knowledge. In this modern era, trustworthy and accurate apps can be valuable tools for disseminating information, such as red flags patients should be aware of. Apps offering verified and relevant information could greatly benefit patients by providing better guidance than what they might find through general online searches. (MS17, Year 3)

      Health trackers, like wearable devices for cardiovascular or gynaecological health, hold great potential. If doctors are equipped with the knowledge to interpret the data generated by these devices—many of which patients already use voluntarily—it could significantly enhance clinical practice. (MS30, Year 4)

      Table 4. Illustrative quotes from interviews with participants

      IV. DISCUSSION

      This study contributes to the literature by comparing the views of medical students, clinical educators, and clinical teachers on essential competencies for medical school curricula. It highlights a consensus on the need for training in EHR, telemedicine, and POCUS, while also identifying emerging needs like interpreting data from healthcare devices and engaging patients with this data. Unlike previous studies that focused on either clinicians or students (Blacketer et al., 2021; Foadi et al., 2021; Gillissen et al., 2022; Hersh et al., 2017; Liu et al., 2022; Machleid et al., 2020; Pontefract & Wilson, 2019), this research explores inter-generational differences, revealing that younger trainees prioritise patient-centred care and digital health literacy. This emphasis on inclusivity and data literacy is crucial given the global challenge of inadequate health literacy, particularly among older populations (Sørensen et al., 2021; Wittink & Oosterhaven, 2018). Integrating these competencies into curricula can foster ongoing improvement in medical education.

      In comparing the views of medical students and senior clinicians regarding DHCs, it is important to acknowledge that the latter group represents a diverse range of roles within the clinical environment. While this study focused specifically on clinicians involved in medical education, senior clinicians may also take on roles as clinical researchers, administrators, or leaders in healthcare innovation. These differing responsibilities could shape their perspectives on the DHCs that should be emphasised in medical curricula. The results of this study should be interpreted within the context of the clinical educators’ primary role in teaching medical students, which may prioritise practical and immediate clinical skills over broader administrative or research-based digital competencies.

      The integration of EHR systems into medical education has been recognised as essential. Successful models in the U.S. and U.K. have embedded EHR systems into training, such as allowing students to write notes at primary clinical sites or use de-identified patient data for practice (Pereira et al., 2018; Pontefract & Wilson, 2019; Welcher et al., 2018). These approaches ensure hands-on learning and prepare students for modern healthcare environments. For broader applicability, partnerships with local healthcare institutions and technology providers can enable the development of localised EHR modules tailored to different healthcare contexts. However, over-reliance on EHR systems may reduce critical thinking or direct patient interaction. Mitigation strategies include embedding reflective practices into training and balancing EHR use with patient engagement activities. When successfully implemented, EHR training enhances data management skills and promotes patient interaction.

      Telemedicine training varies globally, often focusing on rural healthcare placements, as seen in Australia and the U.K. (Rienits et al., 2016; Wootton, 1999). In Singapore, telemedicine exposure arose mainly during the COVID-19 pandemic’s social distancing requirements, highlighting its potential in urban healthcare. To ensure comprehensive telemedicine training, curricula should include simulated consultations, role-playing exercises, and coverage of diverse scenarios, such as breaking bad news over digital platforms. To address the challenge of interpreting body language, medical schools should incorporate modules that focus on interpreting non-verbal cues, such as facial expressions and gestures, which are essential for building rapport and understanding patients’ concerns. Courses that address the legal responsibilities in telemedicine, including patient confidentiality, informed consent, and documentation requirements, should also be integrated to ensure students are aware of the regulatory frameworks guiding virtual care. Case studies and real-life examples that discuss the legal implications of telemedicine can also help students navigate potential legal challenges. With regard to patient selection, it is crucial for professional bodies to develop frameworks that guide clinicians in determining when in-person care is more appropriate, especially for complex or high-risk cases. Additionally, the curricula should provide guidance on how to recognise red flags such as signs of mental health distress, urgent medical conditions, or patient non-compliance with digital tools, and escalate them appropriately. Providing continuing education opportunities for practising clinicians is also important to keep them updated on telemedicine advancements. The healthcare system can then pair students with experienced telemedicine practitioners who can provide the necessary mentorship and guidance.

      POCUS has become an essential component of undergraduate medical education (Glass et al., 2021; Sheppard et al., 2023). However, its application is often limited to specific specialties, such as emergency medicine, obstetrics and gynaecology, and radiology (Glass et al., 2021). In contrast, our study advocates for a longitudinal POCUS curriculum spanning all years and specialties. Integrating POCUS training into medical curricula improves diagnostic accuracy, enhances bedside decision-making, and builds clinical confidence. A structured approach, such as Kern’s six-step model, provides a robust framework for designing and implementing a standardised curriculum with consistent competency assessments (Kern, 1998). Innovative teaching methods, including handheld devices and near-peer instruction, have been shown to improve ultrasound proficiency and student performance, as evidenced by successful implementations at the University of Connecticut School of Medicine (Boivin et al., 2022). While improper use of POCUS can result in diagnostic errors, these risks can be mitigated through routine competency evaluations and guided practice, ensuring students develop the necessary skills safely and effectively.

      Addressing differing opinions regarding DHC priorities in schools requires collaboration among medical educators, technology developers, and healthcare providers. Medical curricula should include training on the responsible use of DHTs, and evaluating the reliability of apps. Developers should design apps that address clinicians’ data security concerns. Moreover, healthcare organisations should establish guidelines and protocols for healthcare-related app use to ensure patient safety and privacy. Integrating these competencies into medical curricula will better prepare future healthcare professionals for evolving demands of healthcare delivery.

      Implementing DHC also requires consideration of diverse healthcare systems and resource availability. Local adaptations, coupled with ongoing evaluation and feedback, ensure curricula remain relevant and scalable. Training programmes must balance global standards with local needs, enabling students to navigate evolving healthcare landscapes effectively. By addressing the implementation strategies, risks, and benefits of DHTs, this study provides actionable insights for advancing DHE.

      A. Strengths and Limitations

      This qualitative study highlights the DHCs needed in the core medical curricula from the perspectives of students, clinical educators and teachers. The diverse sample of institutions and specialties enriched the data. Student feedback provided up-to-date information on curricula gaps, while senior clinicians’ experience enhanced credibility. Interviewing participants from different generations ensured medical education remains relevant to current and future healthcare needs.

      A perceived limitation of this study is its small sample size, which may affect its generalisability, validity and reliability. With a small sample, the findings might not represent the broader population and could miss the population’s diversity (Sandelowski, 1995). Additionally, small sample sizes may not adequately capture the diversity within the population of interest (Sandelowski, 1995). Moreover, the unique socio-political, cultural and economic context of Singapore may limit the generalisability of our findings to other settings. Singapore’s relatively small population and centralised governance create conditions that may differ from other countries. Consequently, while the insights from our study provide valuable lessons, they should be interpreted with caution when applying them to contexts with different governance structures or cultural dynamics.

      Despite the limitations, this qualitative study provides a contextualised understanding of participants’ expectations and concerns through in-depth narrative analysis. While not entirely generalisable, the findings have significant implications for medical curricula in other developed countries. They highlight valued competencies, informing curriculum design, training programmes and continuing education initiatives.

      Furthermore, this study focused on clinicians from public healthcare institutions, which may influence their perspectives on DHCs. Public sector clinicians often work within more structured systems that prioritise certain competencies differently than in the private sector. While this focus provides insights specific to public healthcare education, it is important to note that the views of private-sector clinicians, who may face different digital health challenges, were not captured in this study. Future work comparing clinicians across both sectors could provide a more comprehensive view. The perspectives of clinicians from the private healthcare sector may also differ due to the distinct operational and financial frameworks in which they practise. 

      Moreover, future research could explore the perspectives of clinicians in various roles beyond medical education, such as clinical researchers, healthcare administrators, or those involved in clinical governance. By comparing these viewpoints with those of clinical educators, a more comprehensive understanding of the DHCs required across different professional pathways in healthcare can be developed. Such studies could further inform curriculum development to ensure that medical students are adequately prepared not only for clinical practice but also for the evolving demands of digital health in research, administration, and leadership.

      V. CONCLUSION

      In conclusion, an inter-generational perspective in medical education leverages the strengths of both medical students and senior clinicians, ensuring a balanced approach that keeps pace with technological advancements while maintaining the core principles of patient care. This collaborative model prepares future healthcare professionals to navigate the evolving landscape of healthcare technology effectively. As the findings have shown, despite their varying levels of experience and diverse medical backgrounds, the DHCs deemed essential for future clinical practice are consistent across all stakeholders. While all stakeholder groups considered most basic clinical skills important, medical students viewed physical examination as replaceable by DHTs. In contrast, the clinical educators and teachers thought otherwise. To address the needs and concerns of all groups, DHCs should complement instead of replace traditional clinical skills.

      We acknowledge that the DHTs deemed important in other countries may vary from those reported in this study. However, the type of DHTs to be introduced should not distract us from the main goal of ensuring that the curricula of medical schools around the world remain updated and relevant to current and future healthcare needs. Regularly comparing the perspectives of various stakeholders in the healthcare system is crucial, as it enables educational institutions to adapt and refine their curricula continuously. This iterative process ensures that the curricula remain responsive to the dynamic nature of clinical work.

      Notes on Contributors

      Humairah Zainal collected and analysed the data, wrote the initial drafts, revised the manuscript for publication and approved the submitted copy of the manuscript.

      Xin Xiaohui reviewed, edited, read and approved the submitted copy of the manuscript.

      Julian Thumboo reviewed, edited, read and approved the submitted copy of the manuscript.

      Fong Kok Yong was involved in the conception and design of the study, reviewed, edited, read and approved the submitted copy of the manuscript.

      Ethical Approval

      Waiver for ethical approval was granted by SingHealth Institutional Review Board (Reference Number: 2020/2880). 

      Data Availability

      The data that support the findings of this study are available from the first author upon reasonable request. The data cannot be uploaded to the repository as it contains some confidential views of participants.

      Acknowledgements

      The authors extend their gratitude to the reviewers for their insightful feedback on the manuscript drafts. Special thanks go to Anisa Muhammad for transcribing the interviews, as well as to Dr. Warren Fong Weng Seng and Dr. Ng Qin Xiang for their assistance in recruiting some of the medical students. The authors also deeply appreciate the invaluable contributions of the study participants.

      Funding Statement

      This study was supported by SingHealth Duke-NUS Medicine Academic Clinical Programme under Seah Cheng Siang Distinguished Professorship in Medicine.

      Declaration of Interest

      There are no relevant financial or non-financial competing interests to report.

      References

      Alkire (née Nasr), L., O’Connor, G. E., Myrden, S., & Köcher, S. (2020). Patient experience in the digital age: An investigation into the effect of generational cohorts. Journal of Retailing and Consumer Services, 57, Article 102221. https://doi.org/10.1016/j.jretconser.2020.102221

      Aungst, T. D., & Patel, R. (2020). Integrating digital health into the curriculum – Considerations on the current landscape and future developments. Journal of Medical Education and Curriculum Development, 7, Article 2382120519901275. https://doi.org/10.1177/2382120519901275

      Blacketer, C., Parnis, R., B. Franke, K., Wagner, M., Wang, D., Tan, Y., Oakden-Rayner, L., Gallagher, S., Perry, S. W., Licinio, J., Symonds, I., Thomas, J., Duggan, P., & Bacchi, S. (2021). Medical student knowledge and critical appraisal of machine learning: A multicentre international cross-sectional study. Internal Medicine Journal, 51(9), 1539-1542. https://doi.org/10.1111/imj.15479

      Boivin, Z., Carpenter, S., Lee, G., Chimileski, B., Harrison, J., Choudhary, D., & Herbst, M. (2022). Evaluation of a required vertical point-of-care ultrasound curriculum for undergraduate medical students. Cureus, 14(10), e30002. https://doi.org/10.7759/cureus.30002

      Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

      Clay-Williams, R., Hibbert, P., Carrigan, A., Roberts, N., Austin, E., Fajardo Pulido, D., Meulenbroeks, I., Nguyen, H. M., Sarkies, M., Hatem, S., Maka, K., Loy, G., & Braithwaite, J. (2023). The diversity of providers’ and consumers’ views of virtual versus inpatient care provision: A qualitative study. BMC Health Services Research, 23(1), 724. https://doi.org/10.1186/s12913-023-09715-x

      Edirippulige, S., Brooks, P., Carati, C., Wade, V. A., Smith, A.C., Wickramasinghe, S., & Armfield, N. (2018). It’s important, but not important enough: eHealth as a curriculum priority in medical education in Australia. Journal of Telemedicine and Telecare, 24(10), 697-702. https://doi.org/10.1177/1357633×18793282

      Food and Drug Administration. (2019). Digital health center of excellence. https://www.fda.gov/medical-devices/digital-health-center-excellence

      Foadi, N., Koop, C., Mikuteit, M., Paulmann, V., Steffens, S., & Behrends, M. (2021). Defining learning outcomes as a prerequisite of implementing a longitudinal and transdisciplinary curriculum with regard to digital competencies at Hannover Medical School. Journal of Medical Education and Curricular Development, 8, 23821205211028347. https://doi.org/10.1177/23821205211028347

      Gillissen, A., Kochanek, T., Zupanic, M., & Ehlers, J. (2022). Medical students’ perceptions towards digitization and Artificial Intelligence: A mixed-methods study, Healthcare (Basel), 10(4), 723-737. https://doi.org/10.3390/healthcare10040723

      Glass, C., Sarwal, A., Zavitz, J., Nitsche, J., Joyner, J., Johnson, L. L., Garcia-Vargas, J., & O’Brien, M. C. (2021). Scoping review of implementing a longitudinal curriculum in undergraduate medical education: The wake forest experience. The Ultrasound Journal, 13(1), 23. https://doi.org/10.1186/s13089-021-00206-w

      Hersh, W., Biagioli, F., Scholl, G., Gold, J., Mohan, V., Kassakian, S., Kerns, S., & Gorman, P. (2017). From competencies to competence: Model, approach, and lessons learned from implementing a clinical informatics curriculum for medical students. In Shachak, A., Borycki, E.M., & Reis, S.P. (Eds.),  Health professionals’ education in the age of clinical information systems, mobile computing and social networks (pp. 269-287). Academic Press. https://doi.org/10.1016/B978-0-12-805362-1.00013-9    

      Hurley, K. F., Taylor, B., Postuma, P., & Paterson, G. (2011). What are Canadian medical students learning about health informatics. Electronic Journal of Health Informatics, 6(4), np.

      Kallio, H., Pietilä, A.-M., Johnson, M., & Kangasniemi, M. (2016). Systematic methodological review: Developing a framework for a qualitative semi-structured interview guide. Journal of Advanced Nursing., 72(12), 2954–2965. https://doi.org/10.1111/jan.13031

      Kern, D. E. (1998). Overview: A six-step approach to curriculum development. In P.A., Thomas, D.E., Kern, M.T., Hughes & B.Y. Chen (Eds.), Curriculum Development for Medical Education: A Six-Step Approach (pp. 5-10). Johns Hopkins University Press.

      Liu, D. S., Sawyer, J., Luna, A., Aoun, J., Wang, J., Boachie, L., Halabi S., & Joe B. (2022). Perceptions of US medical students on Artificial Intelligence in medicine: Mixed methods survey study. JMIR Medical Education, 8(4), e38325. https://doi.org/10.2196/38325

      Machleid, F., Kaczmarczyk, R., Johann, D., Balčiūnas, J., Atienza-Carbonell, B., von Maltzahn, F., & Mosch, L. (2020). Perceptions of digital health education among european medical students: Mixed methods survey. Journal of Medical Internet Research, 22(8), e19827. https://doi.org/10.2196/19827

      O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/ACM.0000000000000388

      Pereira, A. G., Kim, M., Seywerd, M., Nesbitt, B., & Pitt, M. B. (2018). Collaborating for competency – A model for single electronic health record onboarding for medical students rotating among separate health systems. Applied Clinical Informatics, 9(1), 199-204. https://doi.org/10.1055/s-0038-1635096

      Pontefract, S. K., & Wilson, K. (2019). Using electronic patient records: Defining learning outcomes for undergraduate education. BMC Medical Education, 19(1), 30. https://doi.org/10.1186/s12909-019-1466-5

      Rienits, H., Teuss, G., & Bonney, A. (2016). Teaching telehealth consultation skills. The Clinical Teacher, 13(2), 119–123. https://doi.org/10.1111/tct.12378

      Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18(2), 179–183. https://doi.org/10.1002/nur.4770180211

      Sheppard, G., Williams, K.-L., Metcalfe, B., Clark, M., Bromley, M., Pageau, P., Woo, M., Yi, Y., Devasahayam, A. J., & Dubrowski, A. (2023). Using Kane’s framework to build an assessment tool for undergraduate medical student’s clinical competency with point of care ultrasound. BMC Medical Education, 23(1), 43. https://doi.org/10.1186/s12909-023-04030-9

      Sørensen, K., Levin-Zamir, D., Duong, T. V., Okan, O., Brasil, V. V., & Nutbeam, D. (2021). Building health literacy system capacity: A framework for health literate systems. Health Promotion International, 36(Supp 1), i13–i23. https://doi.org/10.1093/heapro/daab153

      Tudor Car, L., Kyaw, B. M., Panday, R. S. N., van der Kleij, R., Chavannes, N., Majeed, A., & Car, J. (2021). Digital health training programs for medical students: Scoping review. JMIR Medical Education, 7(3), e28275. https://doi.org/10.2196/28275

      Welcher, C. M., Hersh, W., Takesue, B., Stagg Elliott, V., & Hawkins, R. E. (2018). Barriers to medical students’ electronic health record access can impede their preparedness for practice. Academic Medicine, 93(1), 48–53. https://doi.org/10.1097/ACM.0000000000001829

      Wittink, H., & Oosterhaven, J. (2018). Patient education and health literacy. Musculoskeletal Science and Practice, 38, 120–127. https://doi.org/10.1016/j.msksp.2018.06.004

      Wootton, R. (1999). Telemedicine and isolated communities: A UK perspective. Journal of Telemedicine and Telecare, 5(2), 27–34. https://doi.org/10.1258/1357633991933495

      Zainal, H., Xiaohui, X., Thumboo, J., & Kok Yong, F. (2023a). Digital competencies for Singapore’s national medical school curriculum: A qualitative study. Medical Education Online, 28(1), 2211820. https://doi.org/10.1080/10872981.2023.2211820

      Zainal, H., Xiaohui, X., Thumboo, J., & Kok Yong, F. (2023b). Exploring the views of Singapore junior doctors on medical curricula for the digital age: A case study. PLOS ONE, 18(3), e0281108. https://doi.org/10.1371/journal.pone.0281108

      *Fong Kok Yong
      10 Hospital Boulevard
      Singapore 168582
      +65 6908 8949
      Email: fong.kok.yong@singhealth.com.sg

      Submitted: 29 February 2024
      Accepted: 5 February 2025
      Published online: 1 July, TAPS 2025, 10(3), 15-25
      https://doi.org/10.29060/TAPS.2025-10-3/OA3259

      Zaw Phyo¹, Titi Savitri Prihatiningsih2, Ye Phyo Aung¹ & Tun Tun Naing¹

      1Department of Medical Education, Defence Services Medical Academy, Yangon, Myanmar; 2Public Health and Nursing Universitas Gadjah Mada, Yogyakarta, Indonesia

      Abstract

      Introduction: The effective implementation of changes in medical school curricula requires modifications to assessments alongside teaching strategies. The World Federation for Medical Education emphasises the need for assessment policies to align with educational outcomes. In Myanmar, the Defence Services Medical Academy (DSMA) has adopted an outcome-based curriculum since 2017, but the standard setting in assessments remains pending. This study explores concerns and challenges for enhancing assessment practices.

      Methods: An explanatory sequential design was used. For a quantitative approach, 36 assessment committee members were surveyed using questionnaires, and pre-and post-test analyses were conducted using SPSS Statistics Version 25. Qualitatively, eight committee members were selected through purposive sampling for in-depth interviews using interpretative methodology with thematic analysis by MAXQDA Software 2022.

      Results: It shows that most committee members acknowledge the advantages of standard settings, such as improved teaching and learning defect identification, staff knowledge application, enhanced educational programs, and personal contributions to education quality. Half of the committee members expressed concerns about the increased workload and administrative burden, while another half-supported standard-setting implementation. They all appreciate the integration of standard setting in the outcome-based curriculum, recognising its positive impact on student quality and accreditation standards. Challenges include limited human resources, resistance to change, time-consuming, and an increased workload.

      Conclusion: Committee members suggested that targeted interventions are needed to improve awareness, collaboration, and successful implementation. These should consider perspectives, enhance understanding, define roles, and address resources and resistance. This will optimise standard setting, ensure educational quality, and meet accreditation standards.

      Keywords:           Assessment, Standard Setting, Integrated Curriculum, Awareness, Perception, Attitude, Lifelong Learning

      Practice Highlights

      • Members of the committee think that having standards in DSMA’s curriculum is essential for raising the quality of students and the school’s standards.
      • Most members know how important it is to set standards, especially for pass/fail exams, and stress how important it is to consider culture and institutional factors.
      • Setting standards is thought to help with aligning the curriculum, giving students relevant tests, and meeting accreditation requirements.
      • For implementation to go well, all stakeholders must be involved, the process must be in line with real evaluation principles, and teachers must continue to learn and grow.
      • Not enough resources, resistance to change, limited time, and more work to do are problems that need solutions based on educational ideas.

      I. INTRODUCTION

        In the last 30 years, there have been many suggestions for changing the curriculum, but only a few medical schools have made significant changes (Supe & Burdick, 2006). Without altering assessments, modifying curricula or teaching strategies will be ineffective. In the global standards for quality improvement area of the World Federation for Medical Education (WFME), there are clear rules about primary medical education in chapter 3.1 of Assessment Policy and System:

        “An assessment policy with a centralised system that guides and supports its implementation will entail using multiple summative and formative methods to acquire the knowledge, clinical skills, and behaviors needed to be a doctor. The policy and the system should be responsive to the school’s mission, specified educational outcomes, available resources, and the context” (World Federation for Medical Education, 2020).

        In current Medical Education, with competency-based education becoming more popular, healthcare educational institutions and assessment bodies are being looked at more closely than ever before to make sure they are fair, transparent, and defensible when setting the expected level of performance and rating students’ performance to make decisions about their next steps (Lane et al., 2020). With competency-based assessment, the way the test is done must align with what the student has learned (Crespo et al., 2010).

        Regularly setting, keeping, and evaluating assessment standards is essential in medical education. It is not an exact science to set standards. Most educational test validity arguments focus on how well the test matches the curriculum and how valuable the test’s content is (Barman, 2008).

        Standards are an expression of values, so the ways to set them are systematic ways to gather people’s opinions on values, make agreements, and show that agreement as a single score on a test (Norcini, 2003). “Standard Setting” is a process used to make precise lines between things like “pass” and “fail” or “honors,” “proficient,” and “needs improvement.” Setting standards is “central to giving meaning to test results and, as such, is at the heart of validity arguments” (Wiliam, 1996).

        There are two primary methods in which standards are set: (1) criterion-referenced or absolute methods, in which standards setting is not based on test results, and (2) norm-referenced or relative methods, in which setting standards is based on test results. No golden rule exists (Cohen-Schotanus & Van Der Vleuten, 2010).  In health professions education, it can be challenging to develop credible, defendable, and acceptable passing or cut-off scores for exams (Norcini & Guille, 2002).

        In Myanmar, there are currently no private medical schools, although there are five public medical schools and one military medical school (Defence Services Medical Academy). Myanmar medical schools have used traditional lecture-based teaching methods, and many eminent physicians have emerged from teacher-centered educational programs (Myint, 2020).

        Defence Services Medical Academy (DSMA), listed in the World Directory of Medical Schools, was founded in Mingalardon, Yangon, on November 19, 1992 (Naing et al., 2022). Since 2017, DSMA has been the first medical school in Myanmar to implement an outcome-based, integrated curriculum. According to the Dundee three circle models, the graduates of DSMA are supposed to be scholars and scientists, as well as practitioners and practitioners professionals (Harden et al., 1999).  The new DSMA curriculum is meant to encourage students to keep learning and give them more responsibility for their learning. It is presently focusing on meeting the criteria of WFME accreditation.

        In DSMA, the assessment methods (written, practical, oral, and structured and unstructured observations) include accurate and standardised patients, objective structured clinical examination, patient management problems or simulations, etc. The assessments include continuous assessment, formative assessment, and summative assessment. However, there is no Standard Setting in DSMA, and it will be implemented soon (after training standard setting experts) to set the required basic minimum standards.

        Even though accrediting organisations advise medical schools in general what the curriculum should cover and how it should be tested, most medical schools decide how and what to test (Epstein, 2007). There are five ways to figure out how valuable a method of assessment is: reliability (how accurate and repeatable the measurement is), validity (whether the assessment measures what it says it measures), impact on future learning and practice, acceptability to students and teachers, and costs (Van Der Vleuten, 1996). Therefore, the Assessment Committee’s DSMA meetings establish the process and procedures for conducting assessments throughout the undergraduate medical program. The Assessment committee must collect and evaluate feedback regularly, and faculty feedback revealed issues with the assessment process in DSMA.

         For a successful, acceptable, and feasible assessment to implement an outcome-based, integrated curriculum, it is essential to explore how students, faculty, and assessment committee members perceive the implication of the standard setting for the assessment. Therefore, this study investigated what the Assessment Committee members think about setting standards in medical education and the most significant problems.

        II. METHODS

        A. Research Design

        Figure 1 shows explanatory sequential mixed methods design (also called a two-phase model (Creswell & Clark, 2017)): Quantitative followed by qualitative was used to explore the views, opinions, beliefs, and feelings of the assessment committee members (Ivankova & Stick, 2007). For the quantitative, all assessment committee members were asked using questionnaires (Likert scale), and qualitatively, Individual In-depth Interviews (IDI) were asked of the selected assessment committee members.

        Figure 1. Explanatory sequential mixed methods design

        The main benefit of mixed methods research is that they make a potent mix when quantitative and qualitative data are combined (Miles & Huberman, 1994). In addition, a “complex” image of a social phenomenon might be created, for instance, by evaluating both the quantitative and qualitative aspects of a study’s results (Greene, 2006).

        B. Data Collection

        For Phase 1 (Quantitative): The survey consists of four parts: (1) Collecting demographic information; (2) Investigating people’s perception of standard-setting procedures; (3) Examining attitudes towards these procedures; and (4) Evaluating these procedures using five-point Likert scale items ranging from 5 (strongly agree) to 1 (strongly disagree).

        For Phase 2 (Qualitative): An in-depth interview (IDI) with semi-structured questions was used to collect the data. The researcher made an interview guide, which helped shape the interview and get detailed comments from the assessment committee members. The IDIs lasted 30-45 minutes, and the researcher informed participants that their names and information would be kept private.

        The researcher performed all the interviews in the participants’ local language (Myanmar). The IDI explored the themes of the conceptual framework. A video recording was made for continuous recording in a quiet, private section of the medical education department office. The researcher conducted open discussions with the assessment committee members and kept them confidential, especially the students. Each interviewer uses a different set of questions (10–12), and the sessions last between 45 and 60 minutes. The researcher kept doing interviews until it became clear that the stories, themes, and problems had been covered to saturation point. The researchers utilised constructivist reflexivity throughout the entire study procedure, as advised by Alvesson and Sköldberg (2017).

        C. Sampling Procedure

        In DSMA, there are 38 members of the assessment committee. The assessment committee comprises three groups: Discussant members, voting members, and secretariat.

        For Phase 1 (Quantitative): All the assessment committee members of DSMA were recruited. Among the 38 eligible persons who met eligibility, approximately 95% of the population (36 participants) responded to the evaluation survey.

        For Phase 2 (Qualitative): By using the purposes sampling methods, individual in-depth interviews (IDIs) with semi-structured questions were given to the selected eight members of the assessment committee who are information-rich persons (two discussant Members, five voting members, and one secretariat) to find out what they thought, what they had done, and how they did it.

        D. Data Analysis

        The survey data was stored in an Excel file and processed using SPSS software (version 25). Descriptive analysis explored assessment committee members’ perceptions, attitudes, and practices.

        The MAXQDA 2022 program extracted themes and sub-themes (Côté & Turgeon, 2005). The recorded interviews were transcribed to do content analysis. First, the data were summarised, proofread, and translated into English. Second, for the data analysis, themes, sub-themes, and clusters were identified (Cresswell, 2012). Finally, conceptualisation and explanation throughout the later stages led to the development of concrete abstracts. Analytic coding was done while reading the transcripts, and emerging issues were thought to be included in the thematic statements.

        E. Ethical Considerations

        The study was done according to the guidelines published by the Ethical and Research Committee of DSMA. Participation in this study is entirely voluntary.

        F. Informed Consent

        Before conducting the study, informed verbal consent was obtained for respondents’ autonomy to participate in this study.

        III. RESULTS

        Quantitatively, thirty-seven questions were asked about each participant’s background, knowledge, perception, attitude, and practices regarding the standard-setting. The survey received responses from 36 participants. The data supporting this study’s findings are openly available in the Figshare repository, https://doi.org/10.6084/m9.figshare.25657875.v1 (Phyo et al., 2024).

        A. Demographic Characteristics

        According to an analysis of the committee’s demography (see Table 1), two-thirds of the members were older than 25, and males dominated. Faculty members (64%) were primarily doctoral holders on the evaluation committee. More than ten years of teaching experience can be found among half of the assessment committee members.

        No

        Personal Background Information

        Total (n = 36)

        Frequency

        Percentage
        (%)

        1

        Age Group

        less than 25

        10

        27.8

        26-40

        4

        11.1

        above 40

        22

        61.1

        2

        Gender

        Male

        26

        72.2

        Female

        10

        27.8

        3

        Level of Education

        Undergraduate

        10

        27.7

        Master’s degree

        3

        8.3

        Doctoral

        23

        63.8

        4

        Current Position

        Assistant Lecturer

        3

        8.3

        Lecturer

        9

        25.0

        Assistant Professor

        4

        11.1

        Professor

        9

        25.0

        Students

        10

        27.8

        Others

        1

        2.8

        5

        Time in the teaching profession

        less than five years

        12

        33.3

        5 – 10 years

        6

        16.7

        More than ten years

        18

        50.0

        6

        Served in Current Position

        less than five years

        20

        55.6

        5 – 10 years

        11

        30.6

        More than ten years

        5

        13.9

        Table 1. Personal background information of the assessment committee members

        B. Awareness of Standard Setting

        In Table 2, most assessment committee members believe that DSMA has been implementing a standard setting in the assessment process, but it is still in the process of being fully implemented. However, 75% know the purpose behind establishing standard settings in DSMA. Furthermore, faculty members are debating whether DSMA is a precise standard-setting approach for standardisation and how DSMA intends to implement whatever assessment model.

        No

        Question

        Answers

        Total (n=36)

        Frequency

        Percentage

        1

        Does DSMA have a standard Setting in the MBBS Programme?

        Yes

        23

        63.9

        No

        11

        30.6

        Uncertain

        2

        5.6

        2

        Who Is the most Responsible person in the Implementation standard setting within DSMA?

        Rector

        9

        25.0

        Assessment Committee

        6

        16.7

        All Staff

        20

        55.6

        Uncertain

        1

        2.8

        3

        Why did DSMA introduce a Standard setting?

        Rector’s foresight and planning

        8

        22.2

        Requirement of MMCAC guideline

        27

        75.0

        Uncertain

        1

        2.8

        4

        How many types of standard settings?

        One

        6

        16.7

        Two

        22

        61.1

        Three

        8

        22.2

        5

        The assessment system of the DSMA is designed by applying a ready-made model like programmatic assessment.

        Yes

        12

        33.3

        No

        12

        33.3

        Uncertain

        12

        33.3

        6

        Does DSMA have an exact standard-setting method to standardise the assessment system?

        Yes

        17

        47.2

        No

        10

        27.8

        Uncertain

        9

        25.0

        Table 2. Awareness of standard setting

        C. Perception, Attitude, and Practice towards Standard Setting

        Table 3 shows the mean scores above 4.0 indicate a positive perspective on the effects of standard setting, suggesting that it can aid in problem identification, improve educational programs, and raise the standard of education. Lower scores (3.6) and agreement percentages (71%–72%) on items related to the regularity and straightforwardness of the process in regular process implementation suggest improving how standard setting is carried out and perceived.

        No

        Questions

        Total (n=36)

        Mean ± SD

        Agreement percent

        Perception towards Standard Setting

        1

        Standard setting stimulates the identification of defects in the teaching and learning process.

        4.0 ± 0.8

        80

        2

        The standard-setting process helps staff members apply their academic lives.

        4.1 ± 0.6

        82

        3

        Standard setting can improve the development of educational programs.

        4.3 ± 0.9

        85

        4

        The standard setting enables staff members to contribute to the quality of education.

        4.1 ± 0.6

        82

        5

        Standard setting can ensure borderline candidates to make pass/fail decisions.

        3.9 ± 0.6

        79

        6

        In DSMA, standard-setting development is processed regularly.

        3.6 ± 0.7

        71

        7

        DSMA has been initiating a straightforward standard-setting process for the undergraduate program.

        3.6 ± 0.9

        72

        8

        In DSMA, standard-setting processes are planned by the Assessment Committee.

        4.0 ± 0.7

        79

        9

        Standard-setting development is regularly monitored and improved.

        3.8 ± 0.9

        76

        10

        There is a process for monitoring individual students’ progression.

        3.6 ± 0.9

        72

        Attitude towards Standard Setting

        1

        I often have negative attitudes about Standard setting.

        2.0 ± 0.7

        39

        2

        The standard setting is the job of institution administrators only.

        2.1 ± 0.8

        43

        3

        Implementation of Standard setting makes me an uncomfortable environment for academic activities

        1.9 ± 0.7

        38

        4

        I am still determining the work of standard setting and its outcomes.

        2.0 ± 0.7

        41

        5

        Implementation of Standard setting contributes to the increased workload.

        3.0 ± 1.0

        59

        6

        I get nervous that I am not able to handle changes introduced by standard setting

        2.2 ± 0.7

        44

        7

        Implementation of Standard setting contributes to administrative burden.

        2.8 ± 0.8

        56

        8

        Overall, the process of standard setting is effective.

        4.0 ± 0.7

        81

        9

        Implementing the standard setting can improve pass/fail decisions for the progression of next year.

        4.0 ± 0.7

        79

        Practice towards Standard Setting

        1

        I support the implementation of Standard setting.

        4.4 ± 0.7

        88

        2

        I have been involved in some discussions about Standard setting.

        3.8 ± 1.0

        76

        3

        I am encouraging my colleague to adopt the Standard setting.

        3.9 ± 0.9

        79

        4

        I am doubtful that I will work longer hours to implement standards set standard-setting successfully

        3.8 ± 1.0

        76

        5

        I have been involved in setting up processes in my department

        4.0 ± 1.3

        82

        6

        I have been involved in the standard-setting process at my institution.

        4.3 ± 1.1

        85

        Table 3. Perception, attitude, and practice towards standard setting

        The attitudes section indicates lower ratings (2.0–3.0) with worries about increasing workload (59% agreement) and administrative burden (56% agreement), revealing resistance or uncertainty among some staff members towards the practical implications of standard setting. They also feel that implementing standards creates an uncomfortable academic environment, but 81% think the standard-setting process is successful (4.0 ± 0.7).

        The high mean scores (4.0–4.4) and agreement percentages (up to 88%) in sections on perceptions and practices indicate strong support for standard setting among respondents. This suggests a recognition of its value in improving educational quality.

        The data suggests that standard settings are supported, but there’s a need for more engagement and clear communication about its processes. Apprehension about workload and administrative tasks may stem from a lack of understanding or poor implementation strategies. Institutions could benefit from transparent, inclusive, and efficient management of standard-setting processes, including regular updates, training sessions, and support systems. A balanced approach is needed to address apprehensions and ensure positive outcomes for educational quality and student success.

        D. Thematic Analysis Outcomes

        The participants were Assessment Committee Members (n=8) from the Assessment Committee of the DSMA who were involved in developing the Standard setting for the Assessment.  The data collected from IDIs were organised into five themes and 19 sub-themes in Figure 2.

        Figure 2. Showing the main themes and sub-themes

        1) Perceptions of the assessment committee members: The assessment committee members expressed appreciation for the incorporation of standard setting in the Outcome-based Integrated curriculum at our university. Most committee members showed enthusiasm for using this method, which they believed was fitting for assessing students in this program. Additionally, the committee aimed to enhance the assessment techniques used in the Outcome-based Integrated curriculum. They were convinced that implementing standard settings would improve student and DSMA quality.

        “An outcome-based integrated curriculum has replaced DSMA’s old curriculum. The product of an outcome-based curriculum is not produced. So, we must prepare for that product to be assessed using the standard setting. It is a transitional period for our DSMA from the old curriculum to the outcome-based curriculum. So, the state holders and teachers from DSMA must understand the standard setting. We must set the standard for the assessment of DSMA, but slowly and steadily.”

        (Participant 1)

        “If the integrated curriculum is changed and outcomes are improved, the quality of the university will improve. Local or international qualifications will be enhanced, not only by students but also by the quality of the DSMA.” 

        (Participant 3)

        2) Understanding of standard settings: It was evident that members emphasised the importance of standard setting, particularly in the context of Pass/Fail exams for students. Furthermore, committee members demonstrated knowledge regarding various types of standard setting. They were able to identify the most suitable method for DSMA, taking into consideration factors such as cultural context and institutional requirements.

        “The standard setting is a method designed to determine a student’s pass or fail scores, and it is also a technique for determining whether students will pass or fail a test.”

        (Participant 4)

        “DSMA has to set standards for the assessment according to our culture, values, and conditions. We have taken the standard setting of another prestigious university as a reference. It is not suitable to apply the ready-made model.”

        (Participant 1)

        3) Benefits of implementing standard settings: The assessment committee members recognised Standard Settings are essential for effective curriculum alignment, meaningful student assessment, and accreditation. DSMA can demonstrate its commitment to delivering high-quality education and meeting accreditation standards by aligning the curriculum, teaching practices, and assessments through Standard Settings. The committee members believe Standard Settings play a crucial role in enhancing the quality of education DSMA provides.

        “It is beneficial for constructive alignment. So, we adopted and prepared the standard setting for assessing DSMA for accreditation. It is also essential for the teaching and learning process and method. We can check the teaching and learning process, and if errors occur, we can fix them and circle again for the following year.”

        (Participant 1)

        “I think the outcomes that come out improve not only the student’s quality but also the DSMA’s quality. If the integrated curriculum is changed and outcomes are improved, the quality of the university will improve. Local or international qualifications will be enhanced, not only by students but also by the quality of the DSMA.”

        (Participant 3)

        4) Roles and responsibilities while implementing standard setting in DSMA: The Assessment Committee members recognised the critical roles and responsibilities of implementing standard setting. They emphasised the need for an inclusive approach involving all stakeholders. Furthermore, they highlighted the importance of aligning standard setting with authentic assessment principles and providing continuous professional development for faculty members. By considering these factors, the committee members aimed to enhance the quality of assessment and promote meaningful learning outcomes for students.

        “The standard setting must be built with support; all teachers at DSMA are responsible for it because only the medical education department knows about the standard setting, which is ineffective. Teachers in all departments, including professors, must know about the standard setting. That’s why all the teachers at the university need to understand when setting standards.”

        (Participant 2)

        “The assessment committee is in charge of the standard setting in DSMA, including the medical education committee, the curriculum committee, the quality assurance committee, the assessment committee, etc. The assessment committee can emphasise its role and responsibility, including implementing the standard setting. Thus, it is more reasonable and practical.”

        (Participant 6)

        5) Challenges and weaknesses of implementing the standard-setting: The Assessment committee members identified several challenges in implementing standard setting. These challenges include limited human resources, resistance to change, time-consuming processes, and increased workload. Understanding these challenges through the lens of educational theories can guide the development of strategies to address them effectively and ensure the successful implementation of standard-setting practices.

        “Human resources are our main weakness. There are very few human resources in DSMA to apply the standard setting. Time-consuming is another area for improvement.”

        (Participant 1)

        “Most importantly, students don’t know what a “standard” setting is, and some teachers don’t know either. They need to know what a “standard” setting is.”

        (Participant 7)

        IV. DISCUSSION

        The outcomes from the study at the Defence Service Medical Academy (DSMA) assessment committee shed light on essential aspects of standard-setting in medical education. In institutions like DSMA, which implement the outcome-based integrated curriculum, medical students must remember what they learned in their preclinical training to make sound clinical decisions based on reliable scientific principles (Lazić et al., 2006). In medical education, it is crucial to establish, maintain, and frequently review assessment criteria (Senanayake & Mettananda, 2005).

        The study revealed a predominantly male composition within the assessment committee, highlighting the need for diversity and expertise among committee members. While increasing the diversity of academic medical leadership to address the underrepresentation of minorities and women in medicine has been proposed, there has been limited investigation into this topic (Meadows et al., 2023).  Encouraging diversity can contribute to robust discussions and decision-making processes related to standard-setting. While committee members demonstrate a solid grasp of standard-setting, the results indicate a requirement for continual education to translate this knowledge into practical application (Norcini & McKinley, 2007). Therefore, targeted interventions and professional development programs must enhance their knowledge and awareness.

        Committee members show favorable attitudes towards standard-setting, but there is a discrepancy between their perceptions and actual practices. Implementing open communication strategies consistent with Downing’s (2005) emphasis on the value of clarity in assessment practices could close this gap. Moreover, the lack of active participation in the implementation process may indicate a more significant engagement problem within academic committees (Van der Vleuten et al., 2010).

        The study highlights issues that align with existing literature on medical education reform, citing resource limitations and reluctance to change as typical obstacles to adopting innovative instructional approaches (Swing, 2007). The committee’s alignment with constructivist and cognitive theories advocates for a transition to more interactive and participatory learning methods, which a parallel change in evaluation methods should accompany (Harden & Crosby, 2000). Committee members emphasised the need for an inclusive approach involving all stakeholders and aligning standard-setting with authentic assessment principles, resonating with sociocultural theories emphasising social interaction and collaboration in the learning process.

        The findings offer crucial insights into the committee’s perceptions and experiences with standard-setting in the integrated outcome-based curriculum. Aligned with educational theories, these insights underscore the importance of lifelong learning and can guide future research and targeted interventions to continuously enhance standard-setting practices.

        V. CONCLUSION

        In conclusion, the findings from both the quantitative and qualitative research provide valuable insights into the assessment committee’s knowledge, perceptions, attitudes, and practices regarding standard-setting in medical education at the Defence Service Medical Academy (DSMA). While there is a positive attitude towards the potential benefits of standard-setting, there are also concerns and doubts among some committee members.

        The study emphasises the importance of targeted interventions in medical education to improve awareness, address concerns, foster collaboration, and support the successful implementation of standard-setting. The Assessment Committee Members appreciate the incorporation of standard setting in the outcome-based integrated curriculum and recognise its importance during the transitional period. They demonstrate a strong understanding of standard settings, particularly in Pass/Fail exams, and emphasise the need to consider cultural and institutional requirements. Standard settings can enhance curriculum alignment, meaningful student assessment, and accreditation purposes.

        The committee members emphasise the roles and responsibilities associated with implementing standard setting, including an inclusive approach, continuous professional development for faculty members, and addressing challenges such as limited human resources, resistance to change, time-consuming processes, and increased workload. Strategies informed by educational theories are needed to improve human resource capacity, provide training on standard setting, and create awareness among students and teachers.

        These findings underscore the importance of supporting the assessment committee in their understanding, implementation, and continuous improvement of standard-setting practices. By addressing concerns, providing necessary resources, and promoting collaboration, DSMA can optimise the benefits of standard-setting in medical education and ensure the quality of its educational programs. Incorporating lifelong learning fosters continuous professional development, keeping DSMA at the forefront of educational excellence and adaptability.

        Notes on Contributors

        Dr. Zaw Phyo contributed significantly to the manuscript development process, including reviewing existing literature, identifying research gaps, and addressing questions and goals. He helped draft and revise the manuscript, emphasising its importance for medical education.

        Titi Savitri Prihatiningsih contributed to shaping the research question and framework, ensuring a systematic approach. She applied her expertise in evaluation to refine the research tools and meticulously reviewed the thesis and manuscript for accuracy.

        Dr. Ye Phyo Aung engaged in the study’s methodology phase, endorsing the chosen research design, sampling strategies, and data collection techniques, adding valuable insights to the research execution.

        Dr. Tun Tun Naing assisted in the meticulous proofreading of the article, ensuring clarity, coherence, and compliance with academic standards.

        Ethical Approval

        The Ethical Review Committee of the Defence Services Medical Academy,  Yangon, Myanmar, granted ethical approval. (2 / Ethics/ 2021).

        Data Availability

        The data supporting this study’s findings are openly available in the Figshare repository, https://doi.org/10.6084/m9.figshare.25657875.v1 (Phyo et al., 2024).

        Acknowledgment

        I would also like to thank the FAIMER Regional Institute of Indonesia for Educational Development and Leadership (FRIENDSHIP) for helping me finish this research project at the Faculty of Medicine, Public Health, and Nursing at Universitas Gadjah Mada (Indonesia).

        I would also like to thank my supervisor, the head of the medical education department (DSMA), and my colleagues. Without their help, I would not have been able to survey my research. I am grateful to all the assessment committee members from DSMA, Myanmar, who helped me complete my survey by giving me their valuable opinions and experiences.

        Funding

        The research was carried out during the study period at the FRIENDSHIP-FAIMER Regional Institute of Indonesia for Educational Development and Leadership. No financing is associated with this publication.

        Declaration of Interest

        There is no conflict of interest in the current research.

        References

        Alvesson, M., & Sköldberg, K. (2017). Reflexive methodology: New vistas for qualitative research. Sage publications.

        Barman, A. (2008). Standard setting in student assessment: Is a defensible method yet to come? Annals Academy of Medicine Singapore, 37(11), 957-963. https://doi.org/10.47102/annals-acadmedsg.V37N11p957

        Cohen-Schotanus, J., & Van Der Vleuten, C. P. (2010). A standard-setting method with the best-performing students as a point of reference: Practical and affordable. Medical Teacher, 32(2), 154-160. https://doi.org/10.3109/01421590903196979

        Côté, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education.  Medical Teacher, 27(1), 71–75. https://doi.org/10.1080/01421590400016308

        Crespo, R. M., Najjar, J., Derntl, M., Leony, D., Neumann, S., Oberhuemer, P., Totschnig, M., Simon, B., Gutierrez, I., & Delgado Kloos, C. (2010). Aligning assessment with learning outcomes in outcome-based education.  Proceedings of the IEEE International Engineering Education Conference (EDUCON) 2010. https://doi.org/10.1109/EDUCON.2010.5492385 

        Cresswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Pearson.

        Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research (3rd ed.). Sage.

        Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education, 10(2), 133-143. https://doi.org/10.1007/s10459-004-4019-5

        Epstein, R. M. (2007). Assessment in medical education. New England Journal of Medicine, 356(4), 387-396. https://doi.org/10.1056/nejmra054784

        Greene, J. C. (2006). Toward a methodology of mixed methods social inquiry. Research in the Schools, 13(1), 93-98. https://www.researchgate.net/publication/228968099

        Harden, R. M., & Crosby, J. (2000). AMEE Guide No 20: The good teacher is more than a lecturer – The twelve roles of the teacher. Medical Teacher22(4), 334–347. https://doi.org/10.1080/014215900409429

        Harden, R. M., Crosby, J., Davis, M., & Friedman, R. M. (1999). AMEE Guide No. 14: Outcome-based education: Part 5-From competency to meta-competency: A model for the specification of learning outcomes. Medical Teacher, 21(6), 546–552. https://doi.org/10.1080/01421599978951

        Ivankova, N. V., & Stick, S. L. (2007). Students’ persistence in a distributed doctoral program in educational leadership in higher education: A mixed methods study. Research in Higher Education, 48(1), 93-135. https://doi.org/10.1007/s11162-006-9025-4

        Lane, A. S., Roberts, C., & Khanna, P. (2020). Do we know who the person with the borderline score is, in standard-setting and decision-making. Health Professions Education, 6(4), 617-625. http://doi.org/10.1016/j.hpe.2020.07.001

        Lazić, E., Dujmović, J., & Hren, D. (2006). Retention of basic sciences knowledge at clinical years of medical curriculum. Croatian medical journal, 47(6), 880-882. https://pubmed.ncbi. nlm.nih.gov/17167861

        Meadows, A. M., Skinner, M. M., Hazime, A. A., Day, R. G., Fore, J. A., & Day, C. S. (2023). Racial, ethnic, and sex diversity in academic medical leadership. JAMA Network Open, 6(9), e2335529. https://doi.org/10.1001/jamanetworkopen.2023.35529

        Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.) Sage Publications.

        Myint, K. M. (2020). Curriculum reform and teaching methodologies: A survey on teachers’ perspectives in the university of medicine 1, Yangon. Myanmar Medical Journal, 62(1), 27-33.

        Naing, T. T., Minamoto, Y., Aung, Y. P., & Than, M. (2022). Faculty development of medical educators: Training evaluation and key challenges. The Asia Pacific Scholar, 7(3), 23-32. https://doi.org/10.29060/TAPS.2022-7-3/OA2742

        Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37(5), 464-469. https://doi.org/10.1046/j.1365-2923.2003.01495.x

        Norcini, J. J., & Guille, R. (2002). Combining tests and setting standards. In G. R. Norman, C. P. M. van der Vleuten, D. I. Newble, D. H. J. M. Dolmans, K. V. Mann, A. Rothman, & L. Curry (Eds.), International Handbook of Research in Medical Education (pp. 811-834). Springer. https://doi.org/10.1007/978-94-010-0462-6_30

        Norcini, J. J., & McKinley, D. W. (2007). Assessment methods in medical education. Teaching and teacher education, 23(3), 239-250. https://doi.org/10.1016/j.tate.2006.12.021

        Phyo, Z., Aung, Y, P., Naing, T. T., & Prihatiningsih, T. S. (2024). Perception of the assessment committee on the standard setting in medical education [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.25657875.v1

        Senanayake, M. P., & Mettananda, D. S. (2005). Standards medical students set for themselves when preparing for the final MBBS examination. Annals of the Academy of Medicine, Singapore, 34(8), 483–485.

        Supe, A., & Burdick, W. (2006). Challenges and issues in medical education in India. Academic Medicine, 81(12), 1076-1080. https://doi.org/10.1097/01.acm.0000246699.94234.ab

        Swing, S. R. (2007). The ACGME outcome project: Retrospective and prospective. Medical Teacher, 29(7), 648–654. https://doi.org/10.1080/01421590701392903

        Van der Vleuten, C., Schuwirth, L., Scheele, F., Driessen, E., & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice & Research Clinical Obstetrics & Gynaecology, 24(6), 703-719. https://doi.org/10.1016/j.bpobgyn.2010.04.001

        Van Der Vleuten, C. P. M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education, 1(1), 41-67. https://doi.org/10.1007/bf00596229

        Wiliam, D. (1996). Meanings and consequences in standard setting. Assessment in Education: Principles, Policy & Practice, 3(3), 287-308. https://doi.org/10.1080/0969594960030303

        World Federation for Medical Education. (2020). Basic medical education WFME global standards for quality improvement – The 2020 revision. https://wfme.org/wp-content/uploads/2022/03/WFME-BME-Standards-2020.pdf  

        *Zaw Phyo
        No.94, Pyay Road, Mingaladon Township,
        Postal code – 1102
        Yangon, Myanmar
        Phone: 95 92032754
        Email: dr.zawphyoo@gmail.com

        Announcements