Submitted: 28 February 2025
Accepted: 24 June 2025
Published online: 6 January, TAPS 2026, 11(1), 32-43
https://doi.org/10.29060/TAPS.2026-11-1/OA3683

Sei Keng Koh^1,2, Li Wen Loo^1,2, Zhining Goh^1,3, Dujeepa D. Samarasekera⁴, Carolyn Ching Ching Ho¹, Paul John Gallagher⁵, Wai Keung Chui⁵ & Camilla Ming Lee Wong^1,6

¹Chief Pharmacist’s Office, Ministry of Health, Singapore; ²Division of Pharmacy, Singapore General Hospital, Singapore; ³Department of Pharmacy, Ng Teng Feng General Hospital, Singapore; ⁴Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ⁵Department of Pharmacy and Pharmaceutical Sciences, National University of Singapore, Singapore; ⁶Division of Allied Health & Pharmacy, Sengkang Health, Singapore

Abstract

Introduction: An action research-guided national pre-registration pharmacist (pre-reg) training programme was implemented in two phases: July 2023 to January 2024, and May 2024 to January 2025. The programme is based on professional activities (PAs) required for independent practice, aligning with the Singapore Development Framework for Pharmacists’ competency standards. Workplace-based assessments (WBAs) graded using a supervision scale form the assessment framework.

Objective: This study aims to evaluate the appropriateness of PAs for training and assessment while gathering feedback on user experience.

Methods: Pre-regs and preceptors were selected through purposive sampling with randomisation. Data was collected via online surveys and group interviews. Interviews were conducted separately for pre-regs and preceptors using similar question guides, and audio-recorded, transcribed, then thematically analysed.

Results: Responses from 14 pre-regs and 16 preceptors during Phase One highlighted the strengths, challenges, and recommendations for improving PAs and WBA templates. Pre-regs felt the framework supports a seamless transition to post-course training, while preceptors recognised their role in ensuring that pre-regs attain appropriate supervision levels. The framework was positively received, with well-performing elements retained and areas for improvement identified. Insights gained from action research informed refinements to the framework. Action research for Phase Two is ongoing, with results forthcoming.

Conclusion: The study’s findings led to framework modifications for Phase Two implementation from May 2024. Adjustments were made to individual PAs and WBA forms, with consolidated guidance provided in a user toolkit for dissemination.

Keywords:           Action Research, Pre-registration Pharmacist Training, Entrustable Professional Activities

Practice Highlights

• Professional Activities (PA) for direct and indirect patient care (pharmaceutical companies and regulatory authority), were developed for the national pre-registration pharmacists (pre-regs) training and assessment framework and implemented in phases.
• Action research approach was used to identify areas of improvement for framework enhancement.
• Key areas of enhancement encompass three main aspects: improving the clarity of PA documents, validity of the indirect patient care PAs, and assessment tools and administrative processes; strengthening preceptors’ skills in conducting workplace-based assessments (WBAs), and enhancing pre-registration pharmacists’ engagement with PAs and WBAs.
• A Pre-Registration Training Advisory Committee will be appointed to support the full implementation of the framework in May 2025.

I. INTRODUCTION

In Singapore, pharmacy graduates must complete 12 months of pre-registration training to be eligible for registration with the Singapore Pharmacy Council (SPC). Training for local graduates is divided into two segments: Pre-reg 1 (PR1), a 22-week programme within the National University of Singapore (NUS) Bachelor of Pharmacy (Honours) course and Pre-reg 2 (PR2), a 30-week post-course programme (Figure 1). PR1 rotations are completed in primary care (i.e. retail pharmacy or polyclinic) and either an indirect patient care setting (i.e. pharmaceutical company or regulatory authority) or community hospital. The training was guided by the SPC Competency Standards Framework which articulates 301 competency standards across nine functional areas (Singapore Pharmacy Council, 2010).

Figure 1. Overview of a Pre-Registration Pharmacist’s Journey

In 2020, the Chief Pharmacist’s Office introduced the Development Framework for Pharmacists (DFP), defining competency continuum from foundation to advanced practice across eight key roles aligned with NUS educational outcomes (Ministry of Health, Chief Pharmacist’s Office, 2024) (Figure 2).

Copyright © Ministry of Health, Singapore (2024). Version 2.0: June 2024

Figure 2. Continuum of Competency for Development of Pharmacists Towards 8 Key Roles

To facilitate a seamless transition from school to workplace, the Pre-registration Competency Standards Framework Review Committee (PRCSFRC) was appointed in 2021 to align educational outcomes and pre-registration training with the DFP. The committee developed a training and assessment framework with key features including:

1. Competency blueprint for Day-1 Pharmacists adapted from DFP standards
2. 10 Professional Activities (PAs) proposed as macro-outcomes for training (Table 1)
3. Workplace-based assessments as main assessment modality
4. A 5-level supervision scale to guide progression towards independent practice (Figure 3)

 

Table 1. Professional Activities for Phase-1 Pre-Registration Pharmacists Training

This framework uses PAs adapted from Entrustable Professional Activities (EPAs) (Cate, 2018). EPAs are units of professional work that can be fully entrusted to an individual trainee, once they have demonstrated the needed ability to execute them unsupervised, and this requires the integration of multiple competencies (Cate, 2005). The impetus to adopt PAs for training and assessment included the potential of an EPA-based curriculum to link clinical training and assessment to the actual work that pharmacists do on a day-to-day basis (Hennus et al., 2022). This approach also provides scaffolding for workplace-based training by offering a safe and evidence-based way to facilitate the development of skills and knowledge (Richardson, 2024). EPAs are gaining attention as a practical method to assess competencies in pharmacy practice (Abeyaratne & Galbraith, 2024). Countries like USA, UK and Australia have developed core EPAs for their pharmacy graduates and for pre-registration training (Abeyaratne & Galbraith, 2023; Abeyaratne & Galbraith, 2024; Haines et al., 2017; Richardson, 2024). EPAs were also implemented locally in medical, nursing and allied health professions (Goh et al., 2015; Lau et al., 2020; Zainuldin & Tan, 2023a; Zainuldin & Tan, 2023b; Zhou et al., 2022).

The framework was implemented in phases, with Phase 1 involving 15 pre-registration pharmacists undergoing their PR1 rotations in July to September 2023 and November 2023 to January 2024. Phase 2 followed up with Phase 1 participants and included the entire second cohort of BPharm (Hons) students entering PR1. Action Research was used to identify problems, design solutions, and evaluate the impact of these solutions.

The research aims were to:

1. Review the appropriateness of entry-to-practice pharmacists’ scope of practice as described in the PAs and
2. Gather feedback on the assessment framework from preceptors and pre-registration pharmacists.

 

II. METHODS

A. Research Design

Practical Action Research was employed as a research strategy, coupled with both qualitative and quantitative content analysis. The research model is operational (technical), adopting an iterative approach of “Plan-Act-Observe-Reflect” (George, 2024). For the quantitative analysis, online surveys were disseminated to both pre-registration pharmacists (pre-reg) and their preceptors at the end of each rotation. The survey questions focused on three broad areas of (i) Content validity and reliability, (ii) Process and (iii) Comfort and Confidence. The qualitative analysis included group interviews with the pre-regs and their preceptors at the end of each rotation.

The survey and interview questions were developed based on the objectives of the action research cycle, informed by literature on EPAs. These were then validated by sharing with content experts from the pre-registration training and development committee and piloted with the first cycle collection of data. The questions relevant items were then enhanced for the subsequent cycles. This pretesting of the interview questions helped to identify unclear or ambiguous statements (Castillo-Montoya, 2016; Dikko, 2016).

B. Setting

The study was conducted in multiple sites across Singapore. Participating sites for indirect patient care included three pharmaceutical multinational companies. Participating sites for direct patient care included three retail pharmacy chains and one polyclinic (primary care) group.

C. Sample and Sampling Method

Pre-regs were posted to the training sites by the university. Fifteen pre-regs posted to the study sites for both rotations were invited via email to participate in the study. Nine female and six male pre-regs (n=15) were recruited. Twenty-four preceptors trained in the new framework were assigned to supervise these 15 pre-regs and invited to participate in the study. Each pre-reg is assigned one preceptor per rotation block, while each preceptor may precept up to two pre-regs per rotation block. Nineteen preceptors participated in the study.

D. Data Collection and Analysis

Data for this study was collected via an anonymous electronic survey, with responses analysed using descriptive statistics. Following the survey, group interviews were conducted both face-to-face and virtually. Qualitative data were obtained from free-text responses within the survey as well as from in-depth, audio-recorded group interviews, which were transcribed verbatim. Group interviews were held separately for pre-registration pharmacists (PRPs) and preceptors, utilising the same semi-structured interview guide to ensure consistency across cohorts. Each interview was facilitated by two trained pharmacist interviewers. To enhance credibility and trustworthiness, all transcripts underwent member-checking and peer debriefing. Coding was conducted inductively, with two researchers independently coding the transcripts before reconciling any discrepancies through consensus meetings. Thematic analysis was undertaken to interpret the qualitative data, enabling the identification of nuanced themes that complemented the broader trends observed in the survey findings.

E. Ethical Considerations

This study was approved by both the Singapore Health Services (SingHealth) and NUS Institution Review Boards. Informed consent was obtained from the pre-regs and preceptors. Participation in the study was purely voluntary.

III. RESULTS

A. Online Survey Results

Fourteen (93%) pre-regs and 16 (84%) preceptors responded to the online survey. One pre-reg did not respond despite multiple reminders. Majority of the pre-regs (13/14, 92.9%) agreed that the framework prepared them well for their future role as a pharmacist. They felt that the PAs adequately described the scope of work of a pharmacist and the supervision levels were pegged appropriately for entry-to-practice. Preceptors assessed them fairly and they had the resources and support that they needed from the institutions. There was uncertainty about the ease and efficiency of the system for submission of assessment outcomes and inputs; as well as pre-regs being able to pass rotations under the framework. Survey results are in Table 2.

No	Question	Rating*	Pre-Registration Pharmacists (n=14) (%)	Question	Rating*	Preceptor (n=16) (%)
1	I feel that the revised training and assessment framework will prepare me well for my future role as a pharmacist.	SA	2 (14.3)	I feel that the revised training and assessment framework will prepare pre-registration pharmacists well for their future role as a pharmacist.	SA	3 (18.8)
		A	11 (78.6)		A	13 (81.2)
		UN	1 (7.1)		UN	0
		DA	0		DA	0
		SDA	0		SDA	0
2	The Professional Activities define the scope of work of an entry-level pharmacist well.	SA	1 (7.1)	The Professional Activities adequately define the scope of work of an entry-level pharmacist well.	SA	3 (18.8)
		A	12 (85.7)		A	11 (68.8)
		UN	1 (7.1)		UN	2 (12.5)
		DA	0		DA	0
		SDA	0		SDA	0
3	I believe the Professional Activities will be assessed appropriately and sufficiently to reflect my actual performance ability.	SA	1 (7.1)	The Professional Activities are pegged at an appropriate level for entry to pharmacy practice.	SA	2 (12.5)
		A	11 (78.6)		A	13 (81.3)
		UN	2 (14.3)		UN	1 (6.3)
		DA	0		DA	0
		SDA	0		SDA	0
4	I believe the preceptors will be fair when assessing me.	SA	4 (28.6)	I believe I will be able to assess my pre-registration pharmacists fairly.	SA	3 (18.8)
		A	10 (71.4)		A	13 (81.3)
		UN	0		UN	0
		DA	0		DA	0
		SDA	0		SDA	0
5	I have a good working knowledge of how the revised training and assessment framework works.	SA	1 (7.1)	I have a good working knowledge of how the revised training and assessment framework works.	SA	1 (6.3)
		A	8 (57.1)		A	14 (87.5)
		UN	5 (35.7)		UN	1 (6.3)
		DA	0		DA	0
		SDA	0		SDA	0
6	I believe the training site will be adequately resourced to deliver pre-reg training using the revised framework (e.g. time given for assessments, sufficient number of preceptors).	SA	1 (7.1)	My training site is adequately resourced to deliver pre-registration training using the revised framework.	SA	3 (18.8)
		A	10 (71.4)		A	8 (50.0)
		UN	3 (21.5)		UN	2 (12.5)
		DA	0		DA	3 (18.8)
		SDA	0		SDA	0
7	I am confident that I will get adequate help and guidance from my preceptors to manage the workload with the revised training and assessment framework.	SA	2 (14.3)	I am confident that my pre-registration pharmacists will get adequate help and guidance to manage the case loads with the revised assessment framework.	SA	2 (12.5)
		A	8 (57.1)		A	12 (75.0)
		UN	4 (28.6)		UN	2 (12.5)
		DA	0		DA	0
		SDA	0		SDA	0
8	There is an easy and efficient system of submission online, after assessments and evaluations are completed.	SA	2 (14.3)	There is an easy and efficient system of submissions after assessments and evaluations are completed.	SA	1 (6.3)
		A	5 (35.7)		A	4 (25.0)
		UN	6 (42.9)		UN	10 (2.5)
		DA	0		DA	1 (6.3)
		SDA	1 (7.1)		SDA	0
9	I am confident about passing this rotation under the revised training and assessment framework.	SA	1 (7.1)	I am confident about my pre-registration pharmacists passing this rotation under the revised training and assessment framework.	SA	2 (12.5)
		A	7 (50.0)		A	11 (68.8)
		UN	6 (42.9)		UN	3 (18.8)
		DA	0		DA	0
		SDA	0		SDA	0

Rating*: Strongly Agreed (SA), Agree (A), Unsure/Neutral (UN), Disagreed (DA), Strongly disagreed (SDA)

Table 2. Results for Online Survey (Pre-Registration Pharmacists and Preceptors)

Qualitative comments were sought from pre-regs and preceptors on their perceived benefits and concerns of the PA-based framework compared to the previous competency-based one. For benefits, pre-regs felt that there was more interaction with patients and preceptors. They appreciated the use of real-life case scenarios and discussions for assessment. The PAs encouraged critical thinking and are comprehensive in defining the scope of work of a pharmacist. The PAs also helped to reflect their real competency level and areas for improvement. The preceptors were supportive of the training and assessment framework. The supervision scale was clearer and more intuitive than the previous rating scale of “Competent” or “Not Competent”. Furthermore, as pre-regs were trained to do the activities commonly performed by pharmacists, preceptors felt that pre-regs would be better prepared for their future roles as registered pharmacists. The training and assessment framework was noted to be better structured compared to the previous framework.

Some pre-regs were concerned about their ability to perform and complete training requirements. The training workload was perceived to be higher, and more preparation work was required to collate learning logs and WBAs. There were also concerns whether they could complete all assignments in a timely manner. Preceptors were concerned with inter-rater variability as WBAs were perceived to be subjective. They found the training framework to be more resource-intensive in terms of manpower and time. There were more documentation requirements. There was confusion on the various WBA tools, types of cases and complexities of cases that could be used for assessment. Preceptors also shared that implementation of the revised framework varied across institutions and the process for document submission to SPC was unclear.

B. Group Interviews

Thirteen interview sessions with 34 participants were conducted between September 2023 and January 2024.  Four group interviews were conducted with 15 pre-regs (100%), and six were conducted with 19 preceptors (19/24, 79.2%). Two preceptors and one pre-reg had individual interviews due to conflicting schedules.

C. Theme 1: Validity and Reliability

1) Professional Activities for Direct Patient Care:

Both preceptors and pre-regs agreed that the PAs for direct patient care were clear and appropriate. The six PAs accurately represented the competencies of a Day-1 pharmacist, and no revisions were suggested. However, there were differing views on the following:

a) The core categories under PA 2 (Accurate Supply of Medications) were not applicable to all settings.

“Medicines, quite easy…but the wound management part…Very limited chance, almost sometimes none, depends on the luck, that there will be wound dressing questions.” – Preceptor1

b) There were differing opinions on appropriateness of the passing criteria of Level 3C for PA3 (Educate Patients).

“A 3C and a 4 requires a lot of trust… so I feel that (3C) is a bit too high for PR1”- Preceptor9

“Because I can’t expect them to go into PR2 without a 3C level or so… I did have an expectation of them to be at this level.”- Preceptor7

“I feel…the gradings were quite tough (to achieve)” – Pre-reg13

c) The number of evidence required e.g. 10 case notes for PA1 (Develop and Implement a Care Plan) may not be sufficient. However, preceptors mitigated the concern with WBAs that were formally introduced with the framework.

“I think 10 (SOAP notes) is definitely not enough for me to do a very good evaluation”- Preceptor1

“But we have the new system (WBAs). I’ll argue that you get a better understanding of the student’s actual scope of knowledge…It’s one thing when you have all of the notes on your side and you just craft out the best (written) answer (SOAP), it’s another to be able to test that on the spot to see if they actually know more”- Preceptor6

“For direct patient care.. I feel that because of the new framework, I actually learn more.. like you get to apply what you learn in real life” – Pre-reg5

2) Professional Activities for Indirect Patient Care:

The five PAs for indirect patient care demonstrated poor validity due to the diverse functions pharmacists perform in the industry sector. PAs 7 and 8 were focused on regulatory affairs and restricted some pre-regs’ exposure to other significant industry roles such as marketing and pharmacovigilance. This narrow focus could also limit the number of companies eligible to serve as training sites. Furthermore, PA8 which addressed the roles of the Responsible Person lacked clarity, particularly in its supporting tasks and assessment criteria.

The value proposition of having an indirect patient care rotation as part of registration requirement was raised.

“Are we expecting that all of them would be doing regulatory…to be doing commercial…what is that functional competency we want to train the (pre-regs) on?” – Preceptor18

“…we really need to scope this pharma industry attachment… because if this is the trend moving forward…to scope them as more of an introduction of indirect (patient care)… How indirect patient care actually is important role in your direct patient care”- Preceptor19

“Basically, I feel it’s too streamlined already…like we are all funnelled into the RA role only…although there might be people like me who want to do other stuffs like commercial marketing, medical affairs etc” – Pre-reg10

3) Professional Activity on Developing a Continuous Professional Development (CPD) Plan:

PA6 (Developing a CPD Plan) required greater clarity in terms of its conduct and the template to be used. Several preceptors were unsure how to assess the CPD plans and to guide their pre-regs in developing one. Preceptors also proposed to reduce the number of CPD reviews over the 22-week training period.

“…the CPD, which (my pre-reg) needs to do in the beginning, in the mid and then at the end. It seems to be just repeating only.”- Preceptor10

“ For my student, she doesn’t really know what a CPD entails… she didn’t really know exactly how to fill it in. So when she did fill it in, understandably it was quite general. So she just copied paste…doesn’t really know what she needs to know”- Preceptor1

“Can I ask if CPD is really necessary?” – Pre-reg1

“Actually I am thinking for the final CPD, it can be better formatted…in the sense that we know there is a part on future learning plan and action plan…but it’s already our final week, so we don’t know how applicable it is to us – like we draft a learning plan, but when will we get to execute it” – Pre-reg6

4) Assessments and Supervision Scale:

The supervision scale was found to be intuitive and descriptive as it reflected how independence and supervision is accorded for work. It was preferred over the “Competent or Not Yet Competent” (C/NYC) rating in the previous assessment framework.

“I think C/NYC is like there’s no gray (area)…Most time…it’s not really to the extent that it’s NYC, so yeah, you just end up with a C…I think (the supervision scale) is real…you’ll be managing them like that in the workplace”- Preceptor7

“ I think [the supervision scale] is better …clearer to the PRPs as well…”“I feel my student has more progress …she’s at 3A…Let’s try to progress her to 3B…there was no such articulation [in the previous framework]” – Preceptor1

There was varied interpretation on the “minimum number for WBAs to be completed satisfactorily”. Some preceptors performed the minimum number regardless of assessment outcomes while others aimed to achieve the required passing criteria for all WBAs. There were also concerns about assessor variability affecting the assessment outcomes. Some pre-regs felt that the ratings were not reflective of their actual competencies as the preceptors would just give them the minimum rating for them to pass.

“For the same assessment, one preceptor may grade me as 3B, but to another preceptor, I may be a 3A” – Pre-reg8

“Because no matter how we perform, I feel eventually our preceptor will just pass us by giving us the bare minimum score required”- Pre-reg10

Mini-CEXes and Case-based discussions were conducted in a variety of ways due to a lack of understanding of the purpose of different assessment methods. Multi-source feedback (MSF) contributed to the administrative load for indirect patient care preceptors.

“How do you decide if this is a mini-CEX and this one is a CBD… in the end what I did was like CBD is (for) more complex cases then mini-CEX is (for) easier (cases)?” – Preceptor9

5) Mindset Shift:

Preceptors were unclear of the mindset shift required for prospective decision making that integrated learner attributes – agency, reliability, integrity, humility and capability (ARICH), when using the supervision scale under the revised framework.

“The new framework is actually very content(-based), it doesn’t focus a lot on attitude actually. So if they can get the content done, but their attitude is very poor, I think…we can still pass them actually, but their attitude will be reflected in the comments”- Preceptor12

D. Theme 2: Process

Both preceptors and pre-regs welcomed the change. Pre-regs felt that the revised framework would facilitate a smoother transition to post-course pre-registration training. The preceptors found the revised framework rewarding as it allowed them to be part of the pre-regs’ developmental journey. Pre-regs were viewed as a key success factor for the implementation of the revised framework to be successful and should be empowered to exercise their agency with proper preparation.

The initial implementation process was described as “messy”, and some participants felt “stressful” due to the perceived increase in workload and documentation requirements. Better clarity was needed in terms of the assessment requirements and different types of WBAs to be used.

E. Theme 3: Comfort and Confidence

1) Framework:

Pre-regs were comfortable with the framework and were confident that it would prepare them well for post-course pre-registration training. Preceptors welcomed the change and felt that the framework was learner-centric.

“Compared to the previous (framework)…good change yes…there’s really a lot more feedback…that is now in black and white…For the preceptor, more time-consuming. But I guess if we think in terms of learner-centric kind of role it’s good for the learners.”- Preceptor11

However, they were concerned about scalability due to resourcing concerns.

“Now (phased implementation) is maybe one or two (PRPs on the revised framework per rotation). It’s still fine, but I think really eight at one go…is gonna be pretty overwhelming… And then I think the volume might dilute the effort to give feedback after a while.”- Preceptor13

Preceptors further suggested for cross-learning amongst institutions on framework and WBA implementation, and for more resource materials to be included in the implementation toolkit.

2) Defensibility of Training Decisions:

Preceptors highlighted concerns on the defensibility of training decisions and felt compelled to pass pre-regs due to potential implications related to remediation and its consequences on training capacity and delay in graduation.

“It’s difficult for us not to pass them because it means we need to extend them. Then what are we going to do? Logistically, I think you all know it is very difficult” – Preceptor2

“Once bitten, twice shy.  (I document) like everything… if they are going to have issues then I have to bring up the details.” – Preceptor5

IV. DISCUSSION

The revised training and assessment approach uses professional activities (PAs) to frame the assessment, akin to the Entrustable Professional Activities (EPA) approach developed by Cate (2018). In a scoping review conducted by Hennus et al. (2002), the authors found that most programmes used a mix of logics to establish a framework of EPAs. These logics could be categorised into four broad areas: i) service provision, ii) procedures, iii) disease or patient group, and iv) non-EPAs that are unsuitable for summative entrustment decisions but are important as they describe the abilities physicians require to effectively meet healthcare needs (e.g., developing a CPD plan, PA6).

We chose a mixed logic approach to develop our PAs as pharmacists can practise in various settings. The “mixed logic approach” refers to the use of multiple design logics (e.g., task-based, setting-based, developmental) to develop professional activities. We link this to Hennus et al.’s (2022) framework on EPA design and align it with the iterative, participatory principles of action research (Plan-Act-Observe-Reflect). In Singapore, slightly more than half (approximately 55%) of our locally registered pharmacists work in patient care areas (tertiary hospitals, polyclinics, and retail pharmacies), while the remaining work in indirect patient care areas such as pharmaceutical industries and regulatory bodies (data provided by SPC). The Committee thus envisioned a training system whereby pharmacists from both settings could collaborate and co-develop outcomes for a holistic training programme for our local graduates. For harmonisation across all training sites, PAs for indirect patient care areas were developed. As not all professional activities fit the descriptors of an EPA as described by Cate and Taylor (2020), we have chosen to describe them as professional activities instead of EPAs. PAs 1 to 6 were adapted from those used for new pharmacy graduates in USA (Haines et al., 2017) and the general level training of junior pharmacists in a local academic medical centre since 2019, whereas PAs 7 to 10 were developed by local domain experts from the pharmaceutical industry and regulatory authority.

The results of our study showed that PAs for direct patient care were valid and mostly pegged appropriately to entry to practice level pharmacists. There were concerns about the supervision level for PA3 which some preceptors felt that it was pegged beyond an entry level pharmacist. Preceptors felt that it would be difficult to achieve the suggested level within a short duration of 11 weeks especially when this is the first pre-licensing assessment in their undergraduate days. To the best of our knowledge, there have been no PAs developed for pharmacists working in pharmaceutical companies. It was challenging to develop PAs due to the diverse roles pharmacists can play in the indirect patient care areas. Hence, preceptors felt that PAs 7 and 8 were too restrictive. Furthermore, the value preposition of having an indirect patient care rotation was unclear. Henceforth, there were varied opinions on what would constitute indirect patient care PAs. As SPC has not implemented CPD yet, PA6 will need to be refined further for clarity.

As WBAs were not widely used in all participating training institutions, there were varying comfort levels and understanding amongst preceptors and pre-regs in using mini-CEX and CBD for assessments. Resourcing for manpower, contact time with pre-regs and documentation were cited as key concerns among the preceptors. Similar concerns were raised by Massie and Ali (2015), who suggested approaches to alleviate these issues, including clarifying the purpose of WBA, mandatory training to reduce inter-rater variability and increasing trainees’ engagement with the WBA tools, building training time into preceptors’ responsibilities, as well as improving accessibility to WBA forms with tablet/smartphone applications (Massie & Ali, 2015; So et al., 2024).

The pre-regs’ concerns about inter-rater variability were valid concerns as it may impact the reliability of the revised training and assessment framework. This was subsequently mitigated with additional training sessions for preceptors and several preceptor townhalls to address queries and concerns. Further Phase 2 research is required to understand the preceptors’ concerns with ‘failure-to-fail’ and ensuring defensibility of their decisions as these may potentially compromise the quality of pre-regs who exit the programme and contribute to unpleasant training experience for preceptors through an onerous documentation workload.

The initial implementation process was highlighted as “messy”. Submission of training documents to SPC was done manually as the online registration system was not updated for the revised framework. Moving forward, the registration system will be renewed to support the submission of the training documents in a seamless manner. To mitigate the initial confusion and understanding of PA requirements, SPC will collaborate with NUS to conduct detailed briefing sessions for pre-regs in the subsequent phases of implementation. This is to better enable the students to drive their own learning during rotations, as highlighted by preceptors.

Overall, preceptors and PRPs welcomed the change and are supportive of the revised framework. Many felt that the change was a clearer reflection of the daily work as a pharmacist. The Committee had reviewed the key study findings and proposed the following recommendations and action plans which were implemented prior to Phase 2:

1. The Indirect Patient Care PAs were further refined for clarity and to broaden the scope beyond regulatory affairs.
2. The number of CPD plan reviews for PR1 were reduced from six to four.
3. The number of documents for SPC submission was streamlined to ease administrative burden.
4. The toolkit was updated to reflect changes made and included frequently asked questions, providing both pre-regs and preceptors with comprehensive information.
5. Emphasis on WBAs was increased at the national faculty development workshops that were conducted to upskill existing preceptors.
6. SPC worked with NUS to conduct more in-depth briefing sessions prior to the start of their PR1 training, to better prepare pre-regs to navigate the training requirements.
7. SPC established a Pre-Registration Training Advisory Committee to better support preceptors in the implementation of the framework.
8. SPC will commence an annual faculty development event for preceptors to share best-practices.

 

A. Limitation

This study is limited to the context settings of the participating PR1 rotation sites and has a small sample size, thereby limiting the generalisability of findings.

V. CONCLUSION

A robust training and assessment framework, incorporating meaningful placement experiences, is essential for pre-regs to develop and demonstrate their competence for professional practise. This study shares our experience of implementing PAs in pre-registration training across both direct and indirect care settings, and how action research guided our phased framework implementation. The findings may benefit other international pharmacy institutions implementing EPA frameworks. As this framework is newly established, longitudinal monitoring of PRPs’ post-training performance will be necessary to validate its effectiveness in developing the required workforce and to inform further programme refinements.

Notes on Contributors

Sei Keng Koh was involved in the conceptualisation of this paper, writing and revision.
Li Wen Loo was involved in the conceptualisation of this paper, interviewing, data analysis, writing and revision.
Zhining Goh was involved in the conceptualisation of this paper, interviewing, data analysis and revision of manuscript.
Dujeepa D. Samarasekera was involved in the design of the research methodology and in the revision of the manuscript.
Carolyn Ching Ching Ho was involved in the revision of manuscript.
Paul John Gallanger was involved in the revision of the manuscript.
Wai Keung Chui was involved in the revision of the manuscript.
Camilla Ming Lee Wong was involved in the revision of the manuscript.

Ethical Approval

Ethical approval was obtained from the Institutional Review Boards of the Singapore Health Services (CIRB ref: 2023/2393) and the National University of Singapore.

Data Availability

No data was given to share transcript data and analysis. 

Acknowledgement

We thank the Singapore Pharmacy Council, the Pre-Registration Training Standards Committee, the preceptors and pre-registration pharmacists for their support in this research study.

We thank Mr Richmond Ng from the Singapore Pharmacy Council for his assistance with the data extraction.

Funding

There is no funding support for this study.

Declaration of Interest

All authors have no potential conflicts of interest.

References

Abeyaratne, C., & Galbraith, K. (2023). A review of entrustable professional activities in pharmacy education. American Journal of Pharmaceutical Education, 87(3), ajpe8871. https://doi.org/10.5688/ajpe8872

Abeyaratne, C., & Galbraith, K. (2024). Implementation and evaluation of entrustable professional activities for a pharmacy intern training program in Australia. American Journal of Pharmaceutical Education, 88(12), 101308. https://doi.org/10.1016/j.ajpe.2024.101308

Castillo-Montoya, M. (2016). Preparing for interview research: The interview protocol refinement framework. The Qualitative Report, 21(5), 811–831. https://doi.org/10.46743/2160-3715/2016.2337

Cate, O. T. (2005). Entrustability of professional activities and competency-based training. Medical Education, 39(12), 1176–1177. https://doi.org/10.1111/j.1365-2929.2005.02341.x

Cate, O. T. (2018). A primer on entrustable professional activities. Korean Journal of Medical Education, 30(1), 1–10. https://doi.org/10.3946/kjme.2018.76

Cate, O. T., & Taylor, D. R. (2020). The recommended description of an entrustable professional activity: AMEE Guide No. 140. Medical Teacher, 43(10), 1106–1114. https://doi.org/10.1080/0142159x.2020.1838465

Dikko, M. (2016). Establishing construct validity and reliability: Pilot testing of a qualitative interview for research in Takaful (Islamic Insurance). The Qualitative Report, 21(3), 521–528. https://doi.org/10.46743/2160-3715/2016.2243

George, T. (2024). What is action research? Definitions and examples. Retrieved February 9, 2025, from https://www.scribbr.com/methodology/action-research/

Goh, D. L., Samarasekera, D. D., & Jacobs, J. L. (2015). Implementing entrustable professional activities at Yong Loo Lin School of Medicine Singapore undergraduate medical education program. South‐East Asian Journal of Medical Education, 9(1), 5. https://doi.org/10.4038/seajme.v9i1.90

Haines, S. T., Pittenger, A. L., Stolte, S. K., Plaza, C. M., Gleason, B. L., Kantorovich, A., McCollum, M., Trujillo, J. M., Copeland, D. A., Lacroix, M. M., Masuda, Q. N., Mbi, P., Medina, M. S., & Miller, S. M. (2017). Core entrustable professional activities for new pharmacy graduates. American Journal of Pharmaceutical Education, 81(1), S2. https://doi.org/10.5688/ajpe811s2

Hennus, M. P., Van Dam, M., Gauthier, S., Taylor, D. R., & Cate, O. T. (2022). The logic behind entrustable professional activity frameworks: A scoping review of the literature. Medical Education, 56(9), 881–891. https://doi.org/10.1111/medu.14806

Lau, S. T., Ang, E., Samarasekera, D. D., & Shorey, S. (2020). Development of undergraduate nursing entrustable professional activities to enhance clinical care and practice. Nurse Education Today, 87, 104347. https://doi.org/10.1016/j.nedt.2020.104347

Massie, J., & Ali, J. M. (2015). Workplace-based assessment: A review of user perceptions and strategies to address the identified shortcomings. Advances in Health Sciences Education, 21(2), 455–473. https://doi.org/10.1007/s10459-015-9614-0

Ministry of Health, Chief Pharmacist’s Office. (2024). Development Framework for Pharmacists (DFP).

Richardson, C. L. (2024). Entrustable professional activities: A new approach to supervising trainee pharmacists on clinical placements. Retrieved February 9, 2025, from https://pharmaceutical-journal.com/article/ld/entrustable-professional-activities-a-new-approach-to-supervising-trainee-pharmacists-on-clinical-placements

Singapore Pharmacy Council. (2010, March). Competency standards for Singapore pharmacists.

So, H., Choi, Y., Chan, P., Chan, A. K., Ng, G. W., & Wong, G. K. (2024). Workplace-based assessments: What, why, and how to implement? Hong Kong Medical Journal, 30(3), 250–254. https://doi.org/10.12809/hkmj2311361

Zainuldin, R., & Tan, H. Y. (2023a). Entrustable professional activities developed for allied health entry-level programs across a national level training standards framework in Singapore. PubMed, 52(2), 104–112. https://pubmed.ncbi.nlm.nih.gov/37269028

Zainuldin, R., & Tan, H. S. K. (2023b). Entrustable professional activities implementation in undergraduate allied health therapy programs. The Asia Pacific Scholar, 9(1), 42–48. https://doi.org/10.29060/taps.2024-9-1/sc2997

Zhou, W., Poh, C. L., Chan, H. L., & Shorey, S. (2022). Development of entrustable professional activities for advanced practice nurses education. Nurse Education Today, 116, 105462. https://doi.org/10.1016/j.nedt.2022.105462

*Sei Keng Koh
Division of Pharmacy, Singapore General Hospital
Level 9, SingHealth Tower
10 Hospital Blvd, Singapore 168582
+65-81251901
Email: koh.sei.keng@sgh.com.sg

Submitted: 6 December 2024
Accepted: 14 May 2025
Published online: 6 January, TAPS 2026, 11(1), 44-54
https://doi.org/10.29060/TAPS.2026-11-1/OA3596

Anke van der Merwe & Corlia Janse van Vuuren

School of Health and Rehabilitation Sciences, Faculty of Health Sciences, University of the Free State, South Africa

Abstract

Introduction: Although effective healthcare communication is widely acknowledged as essential, a significant gap remains in practical communication training for healthcare students. The Theory of Planned Behaviour (TPB) provides a framework for designing communication activities that positively influence healthcare students’ attitudes towards and perceived control over their communication skills. TPB also incorporates societal expectations around healthcare communication, encouraging students to apply their learned skills effectively. This study aimed to explore students’ experiences of interprofessional healthcare communication activities structured around the TPB within an interprofessional healthcare curriculum.

Methods: A qualitative descriptive study design was used to assess communication skill development among first- to third-year healthcare students participating in an interprofessional curriculum. Data were collected via an annual institutional module evaluation and anecdotal feedback and analysed thematically under the TPB’s key constructs: attitudes, subjective norms, and perceived behavioural control.

Results: Students reported positive experiences with communication-based activities, particularly those that involved real-world applications. Feedback indicated that interprofessional group activities and community engagement significantly improved students’ perceived communication efficacy, although some challenges in managing group dynamics were noted.

Conclusion: Intentionally integrating communication skills training within an interprofessional curriculum to create behavioral intention positively impacts students’ attitudes, motivation, and ability to communicate effectively in diverse healthcare settings. The TPB framework supports the purposeful design of communication-based activities, fostering a more patient-centered approach among future healthcare professionals. Further research should investigate the long-term effects of this curricular approach on professional communication practice.

Keywords:           Healthcare Communication, Interprofessional Education, Theory of Planned Behaviour, Curriculum Development

Practice Highlights

• Real-world communication activities enhance students’ confidence and skills through application.
• Interprofessional group work fosters communication skills related to teamwork and adaptability.
• TPB-guided communication activities increase students’ motivation for patient-centred communication.
• Continuous feedback supports the sustainable development of communication skills.

I. INTRODUCTION

Despite the recognised importance of effective healthcare communication, a substantial gap persists in the practical training of communication skills among healthcare students and professionals. Approximately half of the participants in a recent study reported limited exposure to health communication training, highlighting deficits in both skills and awareness of barriers affecting interpersonal health communication (Zota et al., 2023). Current curricula emphasise theoretical knowledge but often neglect practical applications, limiting healthcare providers’ ability to navigate diverse linguistic and cultural patient needs, especially in multilingual societies where language barriers can lead to miscommunication, misdiagnoses, and treatment errors (Mangal & Farmer, 2023; Organi et al., 2024). Compounded by limited access to interpreter services in resource-constrained settings, these barriers increase healthcare costs, delay treatments, and compromise patient safety (Organi et al., 2024).

While healthcare providers play a vital role in promoting informed decision-making through the informed consent process for assessment and treatment interventions, they often struggle to simplify complex medical language to accommodate varying patient health literacy levels- sometimes their own (Guo et al., 2020; Mirza & Tabani, 2018). Effective communication is essential for ensuring that patients fully understand their healthcare situation, which is crucial not only for fostering a strong patient-provider relationship but also for enabling truly informed decision-making (Mirza & Tabani, 2018). Literature thus highlights the critical role of communication skill development in healthcare training (Gilligan et al., 2021; Hagiwara et al., 2019; Medendorp et al., 2021; Stamer et al., 2023). Simulation-based learning is particularly emphasised as an effective method to address gaps in communication skills, especially in areas such as obtaining informed consent (Baby et al., 2018; Medendorp et al., 2021). The rise of digital platforms, however, introduces a need for additional training in digital health communication, considering ethical and legal risks (Etheredge & Fabian, 2022). In response, the practical development of a variety of communication skills is critical for future healthcare professionals (Del Riccio et al., 2023; Fliorent et al., 2023; Neuhauser & Kreps, 2003; Pang et al., 2023).

Research further indicates that fit-for-purpose communication positively impacts patient satisfaction, trust, and treatment adherence (Bennet & Lyons, 2011; Houbby et al., 2020). Structured communication training has been associated with improved patient interactions and better management of challenging interactions (Baby et al., 2018; Bachmann et al., 2022; Hagiwara et al., 2019). Effective communication is also instrumental in advancing health literacy, essential for meeting the United Nations Sustainable Development Goals (UN, 2024). Nevertheless, barriers to healthcare access persist, further complicated by an aging population that challenges global efforts to alleviate healthcare professional shortages (Pang et al., 2023). These complexities, in addition to the reported decline in communication skills, as healthcare professionals progress in their studies (Ha & Longnecker, 2010), highlight the need for the early and longitudinal integration of communication skills development into health curricula to empower students to engage effectively with diverse patient populations (Gilligan et al., 2021).

Despite the growing emphasis on communication in healthcare education, practical application is still limited, particularly in pre-clinical years (Madhavanprabhakaran et al., 2015; Nicolaides et al., 2018). To address this gap, the School of Health and Rehabilitation Sciences (SoHRS) at the University of the Free State (UFS) in South Africa embarked on an interprofessional curriculum initiative, comprising one module a year, that spans the four undergraduate years (van der Merwe & Janse van Vuuren, 2024). Students from the following departments participate in the interprofessional curriculum: Biokinetics, Dietetics, Occupational therapy, Optometry, and Physiotherapy. The first- to third-year modules specifically explore various aspects related to being a healthcare professional including but not limited to a structured progression of communication skills training. Students enrolled in this interprofessional curriculum are from the departments of biokinetics, dietetics, occupational therapy, optometry, and physiotherapy allowing varied perspectives within the curriculum. The detailed process of curriculum development has been described previously (van der Merwe & Janse van Vuuren, 2024) whereas this article will focus only on the intentionally scaffolded and implemented communication-based content from the first- to third-year of study through the lens of the Theory of planned behaviour (TPB).

A. Theoretical Framework: Theory of Planned Behaviour

This literature review explores the significance of healthcare communication skills, and their intentional curricular integration grounded in the Theory of Planned Behaviour (TPB). The TPB posits that an individual’s behavioural intention is influenced by their attitude towards the behaviour, the expectation and attitude of the significant other or group of people toward the behaviour known as the subjective norms, and the individual’s perceived behavioural control namely how difficult or easy they view the performance of the behaviour (Ajzen, 1991; Asare, 2015). The TPB has been previously applied in healthcare research to understand and predict behaviours, such as diagnostic radiographers’ adherence to radiation protection best practices (Lewis et al., 2021) as well as predicting nurses’ intentions and actual use of oxygen therapy in COVID-19 (Dehsorkhi et al., 2023). The TPB highlights the importance of attitudes, subjective norms, and perceived behavioural control in potentially fostering effective training transfer to clinical practice. Within the context of healthcare communication, TPB may help understand and predict how students’ attitudes, the influence of their peers and lecturers, and their confidence in their communication abilities may affect their communication practices.

1) Subjective Norm:

In South Africa, as in many other countries, healthcare professionals are held to high standards by society and regulatory bodies (Hompashe et al., 2021; HPCSA, 2025). The Health Professions Council of South Africa (HPCSA) sets minimum training standards for institutions offering healthcare programmes and requires that all registered professionals adhere to a strict code of conduct. Central to these standards is the expectation of effective patient communication (Hompashe et al., 2021; HPCSA, 2025). Higher education institutions such as the one where the study was performed also emphasise the importance of developing communication skills in their graduates. Coupled with the societal expectation of a healthcare professional who can professionally, but also effectively and clearly communicate with a diverse patient population (Amirthalingam et al., 2022; Hagiwara et al., 2019; Hompashe et al., 2021), the importance of and expectations regarding healthcare communication is clear. Aligned with the TPB, these professional and societal expectations would act as a strong motivator for healthcare students to adjust their communication behaviour through intentional skill development to ensure they satisfy societal and regulatory body expectations.

2) Perceived Behavioural Control:

The authors viewed the student’s perceived control over their communication abilities, as well as their perceived difficulty in demonstrating effective communication to be directly impacted by their educational environment. The impact of the educator is therefore situated within the intentional design of transformational learning spaces where students are guided and supported to develop the required communication skills (Mahdy et al., 2020; Pillay et al., 2019). Within the described interprofessional curriculum (van der Merwe & Janse van Vuuren, 2024), opportunities for gaining theoretical knowledge, the provision of both educator and peer feedback as well as the practical application of learnt skills in various settings were created to allow students to manage and mitigate any perceived barriers, they might experience towards developing their communication skills (Figure 1).

3) Attitude Towards Behaviour:

Student’s expectations and evaluation of their applied communication abilities were explored through the intentional application of the learned communication skills within a community setting (Figure 1). Additionally, students explored their changing attitudes toward their communication abilities through interprofessional class engagement, closely simulating the interprofessional nature of their future healthcare environment (Pillay et al., 2019). Healthcare students acknowledge the importance of collaborative teamwork in achieving optimal patient outcomes (Pillay et al., 2019), while their commitment to improving the well-being of individuals under their care serves as a key motivator for their engagement with provided educational content and learning opportunities (Kassab et al., 2022). By directly engaging with both their peers and the community they will serve during their later study years, students are afforded the opportunity of real-life implementation of their learned skills for the betterment of society.

B. Conceptual Framework

Applying the TPB in communication training involves addressing students’ attitudes toward communication, reinforcing positive subjective norms, and enhancing their perceived control over performing communication tasks. In line with the TPB, the authors considered the interplay between the various TPB elements and intentionally selected and designed activities, content, and assessments to better prepare students for real-world communication challenges. The authors mapped communication-based activities throughout the interprofessional modules from the first- to third year of study (Figure 1).

With an initial first step of any behavioural change being that of the participant’s motivation to change (Ajzen, 1991; Asare, 2015), it was deemed essential to ensure students were introduced to the importance and value of effective communication within the healthcare field from their first year of study. Additionally, to minimise potential cognitive overload the content was scaffolded, as illustrated in Figure 1, to gradually expose students to increasingly challenging concepts and activities.

HC: Healthcare
HPCSA: Health Professions Council of South Africa

Figure 1. Conceptual framework of the curricular mapping of communication-based activities

Table 1. Student enrolments 2022 to 2024

Qualitative data were collected from all students enrolled in the respective modules through two processes namely an annual institutional module evaluation (2022 and 2023) as well as voluntarily provided anecdotal feedback (2024). The survey included open-ended questions to gather narrative accounts of student participants’ module experiences and to capture any provided suggestions. Verbal feedback was provided by students to the researcher directly following activity engagement answering the question “How did you experience the communication-based activity?”.

The primary author (AvM) repeatedly reviewed both the typed responses to open-ended survey questions and the verbatim transcriptions of students’ verbal feedback to identify and extract aspects related to students’ experiences of communication-based activities presented in the interprofessional modules. All data were thematically analysed by the AvM through deductive In vivo coding (Saldaña, 2018) under the predetermined themes of the TPB namely subjective norm, attitude towards behaviour, and perceived behavioural control, and subsequently checked by the second author (CvV).

III. RESULTS

Module evaluation response rates, although varied between years and student groups (Table 2), remained low. Additional verbal feedback from students following communication activities supplemented module evaluation data.

Table 2. Response rates on module evaluation survey (2022, 2023)

Figure 2. Thematic analysis of qualitative data

Quotes supporting the identified themes and categories (Figure 2) are presented in Table 3.

Table 3. Supporting quotes per theme

A. Subjective Norm

Students reported that, through practical engagement, they were allowed to develop essential professional skills expected of them in their future work environments (Table 3).

The interprofessional nature of the module and module groups allowed students the opportunity to get to develop teamwork and interprofessional communication skills (Table 3).

B. Attitude Towards Behaviour

The real-world application of communication skills in a community setting was generally described as a positive experience, though it was accompanied by some uncertainty and revealed areas for improvement (Table 3).

C. Perceived Behavioural Control

Activities presented by or including external guests were mentioned to have a positive impact on students through providing a real-world perspective (Table 3).

The allowance for students to, both in class and in assessment activities, practically apply learned communication skills was mentioned repeatedly as a positive aspect in the development of these essential skills (Table 3).

As shown in Table 3 the inclusion of group work activities, although fostering the development of a supportive environment, did challenge some students, most notably in finding a common understanding during class activities.

An essential aspect highlighted in this study was that the communication extended beyond the taught skills, but also to the lecturer-student communication relationship (Table 3).

IV. DISCUSSION

This study aimed to explore the experiences of students participating in purposefully designed communication-based activities hosted within an interprofessional healthcare curriculum. The TPB framework used in this curriculum allowed the authors to purposefully design communication-based teachings and activities to foster behavioural intention on the part of undergraduate healthcare students to best incorporate learned communication skills and appreciate the value thereof within their future practice (Bennett & Lyons, 2011). The findings reveal that by integrating TPB elements – attitudes, subjective norms, and perceived behavioural control—into a healthcare education framework, students reported increased confidence, skill retention, and adaptability in real-world interactions. These results align with prior studies demonstrating the positive effects of structured communication training on healthcare students’ self-efficacy and readiness for patient-centred care (Bachmann et al., 2022; Vogel, Meyer, & Harendza, 2018).

Healthcare professionals are expected to be increasingly adaptable, driven by both the evolving demands of the healthcare environment and societal expectations for effective communication within the profession. Even though the healthcare landscape has changed dramatically over the past few years, the importance of effective and clear healthcare communication within a diverse patient population remains central to optimal patient management (Paget et al., 2011; Skolnik & Butler, 2014). Previous studies have noted challenges related to language and cultural barriers in healthcare often leading to miscommunication, delays in treatment, and poorer health outcomes (Organi et al., 2024; Zota et al., 2023). Not only limited to this study’s context but also from a global perspective, effective intercultural engagement is, therefore, an essential skill expected from healthcare professionals, necessitating the development thereof during the undergraduate study years (Madhavanprabhakaran et al., 2015; Nicolaides et al., 2018; Pillay et al., 2019). This does not imply mastery of all cultural nuances but rather an understanding of the principles necessary for managing diverse patient interactions effectively.

Effective health communication is grounded in a shared decision model, which emphasises the patient’s understanding of proposed management options and aligns with society’s expectation that healthcare communication be collaborative, fostering a partnership between healthcare professionals and patients (Ha and Longnecker, 2010; Paget et al., 2011; Zota et al., 2023). However, healthcare professionals’ reliance on an information-based approach where information on symptoms and treatment options is provided with limited consideration of the patient’s perspective, understanding, or emotional response remains a challenge (Ha & Longnecker, 2010; Zota et al., 2023). Clear, accessible information through, amongst others, employing lay terms in healthcare communication significantly enhances patient understanding by making information clearer, facilitating informed decision-making, reducing anxiety, building trust, and preventing misunderstandings (Guo et al., 2022; Paget et al., 2011; Skolnik & Butler, 2014). However, participants in this study noted struggling to communicate healthcare information in a simplified manner, highlighting the need for additional attention being given to the development of this essential skill.

Considering the collaborative nature of healthcare, this study highlighted the value of interprofessional communication-based engagement in preparing students for the world of work. Participants reported that working alongside peers from diverse health disciplines provided valuable exposure to different perspectives, reinforcing the importance of clear communication, teamwork, and adaptability. Interprofessional learning experiences have been reported to foster a shared understanding of patient needs and enhance collaborative problem-solving, skills that are vital in patient-centred models of care (Madhavanprabhakaran et al., 2015; Nicolaides et al., 2018). Allowing students the opportunity to engage with and learn from one another further ensures exposure to varied personal, professional, and cultural perspectives and managing the received input accordingly. The value of interprofessional engagement has further been highlighted in promoting resilience and adaptability (Gilligan et al., 2021; Pillay et al., 2019), by allowing students to communicate effectively across different professional contexts, aligning with the professional and societal expectations of qualified healthcare professionals. The importance of the early integration of interprofessional activities and community engagement spanning across study years has also been emphasised (Bennett & Lyons, 2011; Thibault, 2020) with this current study’s participants expressing the value of learning how to co-operate and respectfully communicate with others. The authors believe that positive experiences in interprofessional communication will lead healthcare students to engage more confidently and positively in future interprofessional interactions.

The study’s findings indicated that participants’ attitudes towards communication were positively influenced through practical community-based interactions, potentially translating to feelings of perceived behavioural control. Although some students reported uncertainty when engaging in real-world communication activities, most expressed increasing confidence and were able to identify areas for improvements related to their communication abilities. Aligned with previous research, it is suggested that if students perceive positive communication outcomes and consider those positive outcomes to be more likely, they will place more value on working toward achieving those outcomes (Bennett & Lyons, 2011; Dehsorkhi et al., 2023; Lewis et al., 2021). Consequently, it is hypothesised that students will be more inclined to engage in improved communication behaviours, with potentially improved skills transferred to clinical settings.

A scaffolded teaching and learning approach aimed to assist students in gradually developing their communication skills by providing opportunities to continuously build knowledge and allow engagement (Van de Pol et al., 2010). This scaffolded learning approach also aimed to prevent cognitive overload by breaking complex communication tasks and skill acquisition into manageable steps, allowing students to focus on mastering each element gradually (Sweller, 2010; Van Merriënboer & Kirschner, 2018). The intentional transition from theoretical understanding to real-world application also allowed students to engage in hands-on practice optimising their learning and potentially effecting behavioural change (Kurtz, Silverman, Benson, & Draper, 2003). Berkhof et al. (2011) underscore this by showing that repeated exposure to communication training enhances students’ skills in empathy and active listening, while also equipping students to manage emotionally charged patient interactions more effectively.

Through establishing a curricular space that encouraged the exchange of diverse perspectives and incorporated both peer and educator feedback, students were empowered to address and overcome perceived barriers to their communication-based learning, potentially enhancing their behavioural control over time. Additionally, exposure to varied viewpoints—through content presenters, assessors, and feedback providers—aimed to foster the development of critical thinking skills and a deeper understanding of the subject matter (Bedford & Barnes, 2024). Aligned with published recommendations, communication-based activities in this curriculum also emphasised diverse forms of interaction such as simulation and role-play, aiming to further assist students in overcoming communication barriers by providing practical experience in effective person-centred communication (Bennett & Lyons, 2011; De Sousa Mata et al., 2021). This practical approach encourages active student participation, enabling them to practice key skills and fostering behaviour change (De Sousa Mata et al., 2021).

The inclusion of group work activities was mostly experienced as positive in developing a different communication skill set. However, some students expressed frustration with groupwork activities especially when navigating diverse opinions and communication styles. These challenges reflect findings by Vacheishvili (2021) and Wilson et al. (2018) who note that while group work fosters teamwork skills, it can also be difficult to manage if group work activities are not selected with care. Considering the collaborative nature of healthcare practice, developing communication skills within a group setting may contribute to the resilience and flexibility required as students develop the skills to mediate conflicts and balance differing viewpoints, ultimately enhancing their adaptability in future professional settings (Paget et al., 2011; Vacheishvili, 2021; Wilson et al., 2018).

The perceived impact of educator-student communication as a model for professional behaviour and academic support was also highlighted in this study. Positive educator-student communication and feedback were frequently cited by participants as contributing to their confidence and clarity in the learning process, a finding corroborated by Kassab et al. (2022) linking educator engagement with increased student motivation and self-efficacy in healthcare training. Educator feedback provided students with practical guidance, while the supportive learning environment encouraged open dialogue, both of which are key for developing reflective healthcare practitioners (van der Merwe et al., 2024).

V. CONCLUSION

This study highlights the value of, through consideration of the TPB, embedding interprofessional, practical communication-based activities in healthcare education to facilitate behavioural intention towards patient-centred communication. Findings suggest that practical, real-world communication activities helped students develop the expected healthcare communication skills. Additionally, results indicated that the intentional communication-based activities positively influenced students’ attitudes toward patient-centred communication. By incorporating a range of communication-based activities paired with continuous feedback, students were consistently supported across multiple study years, helping them to develop and sustain essential communication skills. This ongoing reinforcement positively impacted participants’ sense of behavioural control, as they gained confidence in their ability to perform these skills effectively in practice. The study underscores that to ensure healthcare graduates can communicate in an adaptable, culturally sensitive, and collaborative way, purposefully integrating communication-based activities to affect potential behavioural change is essential. Future research into the long-term effects of such training on professional performance and patient outcomes is recommended.

Notes on Contributors

AvM coordinates the curricular implementation. All authors contributed equally to the manuscript conceptualisation, data checking, and proofreading of the manuscript. All authors have read and approved the manuscript.

Ethical Approval

Before the commencement of the study approval was granted by the General/Human Research Ethics Committee (UFS-HSD2021/0921/21) at the University of the Free State, ensuring that all guidelines for research including human participants were adhered to. Informed consent was obtained from all individual participants included in the study.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Acknowledgement

The authors wish to acknowledge the participating students for their valuable feedback. The authors acknowledge the use of OpenAI’s ChatGPT to provide language and grammar feedback in the discussion section during the preparation of the final manuscript.

Funding

No funding was received for this study.

Declaration of Interest

The authors have no conflict of interest to declare.

References

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-t

Amirthalingam, S. D., Ramasamy, S., & Aznal, S. S. H. S. (2023). Gamification through collaborative learning in medical education. The Asia Pacific Scholar, 8(3), 45–49. https://doi.org/10.29060/taps.2023-8-3/sc2921

Asare, M. (2015). Using the theory of planned behavior to determine the condom use behavior among college students. American Journal of Health Studies, 30(1), 43-50. https://pubmed.ncbi.nlm.nih.gov/26512197/

Baby, M., Gale, C., & Swain, N. (2018). Communication skills training in the management of patient aggression and violence in healthcare. Aggression and Violent Behaviour, 39, 67-82. https://doi.org/10.1016/j.avb.2018.02.004

Bachmann, C., Pettit, J., & Rosenbaum, M. (2022). Developing communication curricula in healthcare education: An evidence-based guide. Patient Education and Counseling, 105(2), 2320-2327. https://doi.org/10.1016/j.pec.2021.11.016

Bedford, A., & Barnes, N. (2024). A call for different perspectives. Historical Thinking, Culture, and Education, 1(1), 110-114. https://doi.org/10.12685/htce.1317

Bennett, K. S., & Lyons, Z. (2011). Communication skills in medical education: An integrated approach. Education Research and Perspectives, 38(2), 45-56. https://www.researchgate.net/publication/267977840_Communication_Skills_in_Medical_Education_An_Integrated_Approach

Berkhof, M., van Rijssen, H. J., Schellart, A. J., Anema, J. R., & van der Beek, A. J. (2011). Effective training strategies for teaching communication skills to physicians: An overview of systematic reviews. Patient Education and Counseling, 84(2), 152-162. https://doi.org/10.1016/j.pec.2010.06.010

Dehsorkhi, H. F., Shamami, M. A., Hayat, A. A., & Rokni, J. (2023). Predicting training transfer of oxygen therapy during COVID-19: Theory of planned behavior application. Journal of Health and Nursing Management, 33(4), 250-258. https://doi.org/10.32598/jhnm.33.4.2468

Del Riccio, M., Costantini, L., Guasconi, M., Casella, G., Fanfani, A., Cosma, C., Mindrican, P., Bonaccorsi, G., Corradini, E., Artioli, G., Sarli, L., Laverack, G., Rondini, E., & Martucci, G. (2023). Role and challenges to digital technologies in community health promotion programs in Italy during the COVID-19 pandemic: A multiple embedded case study protocol. PubMed, 94(1), e2023019. https://doi.org/10.23750/abm.v94i1.14109

De Sousa Mata, Á. N., De Azevedo, K. P. M., Braga, L. P., De Medeiros, G. C. B. S., De Oliveira Segundo, V. H., Bezerra, I. N. M., Pimenta, I. D. S. F., Nicolás, I. M., & Piuvezam, G. (2021). Training in communication skills for self-efficacy of health professionals: A systematic review. Human Resources for Health, 19(1). https://doi.org/10.1186/s12960-021-00574-3

Etheredge, H., & Fabian, J. (2022). Communication in healthcare: Global challenges in the 21st century. Hamostaseologie, 42(1), 29-35. https://doi.org/10.1055/a-1685-7096

Fliorent, R., Mistry, S., Javaid, K., et al. (2023). Skin of color is underrepresented on Instagram: Assessing South Jersey Skin Talk as an educational tool to increase awareness. Cureus, 15(10), e47388. https://doi.org/10.7759/cureus.47388

Gilligan, C., Powell, M., Lynagh, M. C., Ward, B. M., Lonsdale, C., Harvey, P., James, E. L., Rich, D., Dewi, S. P., Nepal, S., Cro, H. A., & Silverman, J. (2021). Interventions for improving medical students’ interpersonal communication in medical consultations. Cochrane Database of Systematic Reviews, 2021(2), CD012418. https://doi.org/10.1002/14651858.CD012418.pub2

Guo, Y., Qiu, W., Wang, Y., & Cohen, T. (2021). Automated lay language summarization of biomedical scientific reviews. Proceedings of the Thirty-Fifth Association for the Advancement of Artificial Intelligence Conference. 35(1), 160-168. https://doi.org/10.1609/aaai.v35i1.16089

Ha, J. F., & Longnecker, N. (2010). Doctor-patient communication: A review. Ochsner Journal, 10(1), 38–43. https://pubmed.ncbi.nlm.nih.gov/21603354/

Hagiwara, N., Elston Lafata, J., Mezuk, B., Vrana, S. R., & Fetters, M. D. (2019). Detecting implicit racial bias in provider communication behaviors to reduce disparities in healthcare: Challenges, solutions, and future directions for provider communication training. Patient Education and Counseling, 102(9), 1738-1743. https://doi.org/10.1016/j.pec.2019.04.023

Health Professions Council of South Africa. (2025, June 13). New and updated ethical guidelines. https://www.hpcsa-blogs.co.za/new-and-updated-ethical-guidelines/

Hompashe, D. M., Gerdtham, U. G., Christian, C. S., Smith, A., & Burger, R. (2021). ‘The nurse did not even greet me’: How informed versus non-informed patients evaluate health systems responsiveness in South Africa. BMJ Global Health, 6(4), e004360. https://doi.org/10.1136/bmjgh-2020-004360

Houbby, N., Abdelwahed, A., & Kumar, S. (2020). A reflection on Imperial College London’s Community Action Project. Education for Primary Care. 31(3), 186-187. https://doi.org/10.1080/14739879.2020.1744191

Kassab, S. E., Taylor, D., & Hamdy, H. (2022). Student engagement in health professions education: AMEE Guide No. 152. Medical Teacher, 45(9), 949-965. https://doi.org/10.1080/0142159X.2022.2137018

Kurtz, S. M., Silverman, J. D., Benson, J., & Draper, J. (2003). Marrying content and process in clinical method teaching: Enhancing the Calgary–Cambridge guides. Academic Medicine, 78(8), 802-809. https://doi.org/10.1097/00001888-200308000-00011

Lewis, S., Downing, C., & Hayre, C. M. (2021). Using the theory of planned behaviour to determine radiation protection among South African diagnostic radiographers: A cross-sectional survey. Journal of Medical Radiation Sciences, 69(1), 47-55. https://doi.org/10.1002/jmrs.537

Madhavanprabhakaran, G., Al-Khasawneh, E., & Wittmann, L. (2015). Perceived benefits of preclinical simulation-based training on clinical learning outcomes among Omani undergraduate nursing students. Sultan Qaboos University Medical Journal, 15(1), e105-e111. https://pubmed.ncbi.nlm.nih.gov/25685368/

Mahdy, Z. A., Maaya, M., Atan, I. K., et al. (2020). Simulation in healthcare in the realm of Education 4.0. Sains Malaysiana, 49(8), 1987-1993. https://doi.org/10.17576/jsm-2020-4908-21

Mangal, J. P., & Farmer, B. S. (2023). Language barriers in healthcare. In Advances in medical education, research, and ethics (AMERE) book series (pp. 113–129). https://doi.org/10.4018/978-1-6684-5493-0.ch007

Medendorp, N. M., van den Heuvel, L. M., Han, P. K. J., Hillen, M. A., & Smets, E. M. A. (2021). Communication skills training for healthcare professionals in providing genetic counseling: A scoping literature review. Patient Education and Counseling, 104(1), 27-39. https://doi.org/10.1016/j.pec.2020.07.018

Mirza, S. N., & Tabani, U. (2018). The rhetoric of healthcare professionals: Infectious communication and its effects. Madridge Journal of Internal and Emergency Medicine, 2(1), 45-47. https://doi.org/10.18689/mjiem-1000108

Neuhauser, L., & Kreps, G. L. (2003). Rethinking communication in the e-health era. Journal of Health Psychology, 8(1), 7-23. https://doi.org/10.1177/1359105303008001426

Nicolaides, M., Cardillo, L., Theodoulou, I., Hanrahan, J., Tsoulfas, G., Athanasiou, T., Papalois, A., & Sideris, M. (2018). Developing a novel framework for non-technical skills learning strategies for undergraduates: A systematic review. Annals of Medicine and Surgery, 36, 29-40. https://doi.org/10.1016/j.amsu.2018.10.005

Paget, L., Han, P., Nedza, S., Kurtz, P., Racine, E., Russell, S., Santa, J., Schuman, M. J., Simha, J., & Von Kohorn, I. (2011). Patient clinician communication: Basic principles and expectations – NAM. (n.d.). NAM. https://nam.edu/perspectives-2011-patient-cliniciancommunication-basic-principles-and-expectations/

Pang, T. Y., Lee, T. K., & Murshed, M. (2023). Towards a new paradigm for digital health training and education in Australia: Exploring the implication of the fifth industrial revolution. Applied Sciences, 13(11), 6854. https://doi.org/10.3390/app13116854

Pillay, J. D., Ally, F., & Govender, N. (2019). Exploring opportunities for embedding graduate attributes in a first-year undergraduate anatomy course for allied health students. BMC Medical Education, 19(1). https://doi.org/10.1186/s12909-019-1777-6

Saldaña, J., & Omasta, M. (2018). Analyzing interviews: Condensing and coding. In Qualitative research: Analysing life (pp. 117-140).

Organi, Z. K., Nazarenia, M., & Aghaee, F. (2024). Understanding the challenges of language barriers in healthcare. Advances in Medical Education, Research, and Ethics, 3(3), 28-35. https://doi.org/10.61838/kman.isslp.3.3.5

Skolnik, P. J., & Butler, N. (2014). The patient’s and patient family’s perspective. In Patient Safety in Surgery (pp. 403–408). https://doi.org/10.1007/978-1-4471-4369-7_28

Stamer, T., Steinhäuser, J., & Flägel, K. (2023). Artificial intelligence supporting the training of communication skills in the education of health care professions: Scoping review. Journal of Medical Internet Research, 25, e43311. https://doi.org/10.2196/43311

Sweller, J. (2010). Cognitive load theory: Recent theoretical advances. In Plass, J. L., Moreno, R., & Brünken, R. (Eds.), Cognitive load theory. Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.004

Thibault, G. E. (2020). The future of health professions education: Emerging trends in the United States. FASEB BioAdvances, 2, 685-694. https://doi.org/10.1096/fba.2020-00061

United Nations. (2024). Goal 3 | Department of Economic and Social Affairs. (n.d.). https://sdgs.un.org/goals/goal3

Vacheishvili, N. (2021). Collaborative communication as an effective strategy in education. Proceedings of the Strategic Communications Today: Existing Challenges and Opportunities Conference. https://www.researchgate.net/publication/353606348_Collaborative_Communication_as_an_Effective_Strategy_in_Education#fullTextFileContent

van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher–student interaction: A decade of research. Educational Psychology Review, 22(3), 271-296. https://doi.org/10.1007/s10648-010-9127-6

van der Merwe, A., & Janse van Vuuren, C. (2024). Creating transformational learning experiences for 21st century healthcare students through pre-clinical skills training at a South African university. BMC Medical Education, 24(198). https://doi.org/10.1186/s12909-024-05177-9

van der Merwe, A., Serekoane, M., & du Plooy, S. (2024). Situating pluralism in biomedical healthcare practices: Lessons from the ethnomedical approach. SoTL in the South, 8(2), 49-65. https://doi.org/10.36615/sotls.v8i2.397

Van Merriënboer, J. J. G., Kirschner, P. A., & Frèrejean, J. (2024). Ten steps to complex learning. https://doi.org/10.4324/9781003322481

Vogel, D., Meyer, M., & Harendza, S. (2018). Verbal and non-verbal communication skills including empathy during history taking of undergraduate medical students. BMC Medical Education, 18, 157. https://doi.org/10.1186/s12909-018-1260-9

Wilson, K. J., Brickman, P., & Brame, C. J. (2018). Group work. CBE—Life Sciences Education, 17(1), fe1. https://doi.org/10.1187/cbe.17-12-0258

Zota, D., Diamantis, D. V., Katsas, K., Karnaki, P., Tsiampalis, T., Sakowski, P., Christophi, C. A., Ioannidou, E., Darias-Curvo, S., & Batury, V.-L. (2023). Essential skills for health communication, barriers, facilitators and the need for training: Perceptions of healthcare professionals from seven European countries. Healthcare, 11(14), 2058. https://doi.org/10.3390/healthcare11142058

*Anke van der Merwe
205 Nelson Mandela Drive,
Park West, Bloemfontein, 9301
Email: gonzalesa@ufs.ac.za

Submitted: 16 October 2024
Accepted: 16 June 2025
Published online: 6 January, TAPS 2026, 11(1), 55-68
https://doi.org/10.29060/TAPS.2026-11-1/OA3543

Jie Kie Phang^1,2, Yu Heng Kwan^1,2, Sungwon Yoon² & Warren Fong^1,3,4

¹Department of Rheumatology and Immunology, Singapore General Hospital, Singapore; ²Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore; ³Office of Education, Duke-NUS Medical School, Singapore; ⁴Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: There are limited medical professionalism assessment tools that are adapted for Singapore. We aimed to evaluate the validity, reliability and feasibility of the Singapore version of the Professionalism Mini-Evaluation Exercise (P-MEX) for assessment of professionalism of residents at a teaching hospital in Singapore.

Methods: We collected assessment data from residents from July 2019 to January 2020. Content validity was evaluated through cognitive debriefing interviews (CDIs). Construct validity was evaluated using a priori hypothesis with another comparator instrument. Structural validity was assessed through both confirmatory factor analysis (CFA) using the original four-factor model and exploratory factor analysis (EFA). Internal consistency was assessed using Cronbach’s alpha.

Results: CDIs conducted with nine faculty supported content validity of the Singapore version of P-MEX. The cross-sectional data demonstrated satisfactory construct validity (83% of hypotheses met) and good internal consistency (Cronbach’s alpha ranged from 0.72 to 0.93). The CFA of the original four-factor model of the P-MEX did not obtain meet all the acceptable fit of indexes. EFA suggested a two-factor model—the first factor had an eigenvalue of 8.52 and explained 73.8% of the variance, while the second factor had an eigenvalue of 1.90 and explained 26.2% of the variance.

Conclusion: This study supports the content validity, construct validity, internal consistency of the Singapore version of P-MEX in assessing medical professionalism in the local residency training programme. However, there was no conclusive support for the original four-factor model of the P-MEX. EFA identified 2 main factors, which appeared to encompass “Patient care and inter-professional relationship skills” and “Reflective skills”.

Keywords:           Professionalism, Singapore, Validation, Assessment

Practice Highlights

• This study supports the content validity, construct validity, internal consistency of the Singapore version of P-MEX in assessing medical professionalism in the local residency training programme.
• Future research should examine the structural validity of the modified P-MEX in different samples of medical trainees as confirmatory factor analysis (CFA) of the original four-factor model of the P-MEX did not obtain meet all the acceptable fit of indexes.
• Medical educators in the region with similar multi-cultural backgrounds can consider using the Singapore version of P-MEX to assess medical professionalism among trainees.

I. INTRODUCTION

Listed as one of the core competencies in the Accreditation Council for Graduate Medical Education (ACGME) curriculum (Swing, 2007), medical professionalism is being increasingly emphasised in medical training for both undergraduates and postgraduates as part of their development of a professional identity (Cruess & Cruess, 2006; Cruess et al., 2015; Swick et al., 1999). Unprofessional behaviour during residency training has been shown to predict future disciplinary actions of doctors (Papadakis et al., 2008). Fortunately, there is evidence to suggest that remediation of unprofessional behaviour has been effective (Guerrasio et al., 2014; Zbieranowski et al., 2013). Various training programmes have been developed in the hope to teach and train professionalism among the residents (Lumlertgul et al., 2009). The ability to evaluate the level of medical professionalism is imperative to determine the effectiveness of the training programmes aiming to instil medical professionalism among residents (Hawkins et al., 2009). Furthermore, assessment supports the development of the professional identity and allows for early remediation of residents displaying unprofessional behaviour (Cruess et al., 2015; Cruess et al., 2019). 

Previous studies have identified professionalism as a complex multi-dimensional social construct, thus making geographical location and culture important considerations in the assessment of professionalism (Cruess et al., 2010; Ho et al., 2011). Importantly, there is evidence on geographical and regional variations in the attitudes, beliefs and behaviours constituting medical professionalism (Chandratilake et al., 2012). Following the Ottawa consensus conference, it has been advised that re-validation of assessment tools be performed when used in new cultural contexts (Hodges et al., 2011). The Professionalism Mini- Evaluation Exercise (P-MEX), developed in Canada (Cruess et al., 2006), is one of the promising observer-based tools that can be used for assessment of medical professionalism -in the workplace (Hawkins et al., 2009; Kwan et al., 2018). This allows us to assess the Does of Millers pyramid (Miller, 1990). Since its conception, the P-MEX has been piloted in Japan, Finland, Iran, Turkey and Switzerland (Amirhajlou et al., 2019; Bajwa et al., 2021; Karukivi et al., 2015; Taşçı et al., 2023; Tsugawa et al., 2009), but there are no studies validating the P-MEX in the multi-ethnic Southeast Asia region. Singapore is a multi-ethnic country, comprising of predominantly Chinese (74%), followed by Malays (13%) and Indians (9%) (Noor & Leong, 2013). Previous study has shown that the definition of medical professionalism differs even among two Chinese cultural contexts (Ho et al., 2014), thus highlighting the need to adapt the P-MEX when using it in a new socio-cultural context.

The current assessment tool in the Singapore residency programme includes the MOHH C1 form, which contains 6 general questions on medical professionalism such as (1) Accepts responsibility and follows through on tasks, (2) Responds to patient’s unique characteristics and needs equitably, (3) Demonstrates care and concern for the patient/family, (4) Communicates effectively with patient/family, (5) Communicates and works effectively with other healthcare professionals, and (6) Demonstrates integrity and ethical behaviour. However, the P-MEX, consisting of 24 observable behaviours that were distilled from the original 142 behaviours, allowed for assessors to evaluate as many attributes of a professional doctor with the smallest number of behaviours (Cruess et al., 2006). The explicit observable behaviours allow for more objective assessments of professionalism as compared to the more general items of the MOHH C1 form. Previously, our Delphi study had demonstrated consensus regarding the relevance of the Singapore version of P-MEX with 23 items to be used in Singapore for evaluation of medical professionalism among residents (Fong et al., 2020). The aim of the present study was to evaluate the validity and reliability of the Singapore version of the P-MEX for assessment of medical professionalism of residents at a teaching hospital in Singapore.

II. METHODS

A. Study Settings and Participants

We conducted two studies examining- (1) content validity of the Singapore version of the P-MEX using cognitive debriefing interviews (CDIs) and (2) construct validity, reliability and feasibility of the Singapore version of the P-MEX in a cross-sectional study among residents in the SingHealth residency programme in the Singapore General Hospital. Within the SingHealth residency programme, foreign medical graduates constitute 13.7% of the residents (unpublished data). SingHealth Residency offers 30 programmes, of which 21 are accredited by the Accreditation Council of Graduate Medical Education – International (ACGME-I), and 9 are accredited by the Joint Committee of Specialist Training (JCST) (SingHealth, 2024). ACGME-I is the extension of the accreditation model of ACGME internationally (Arrighi et al., 2024).

All study protocols were approved by the SingHealth Centralised Institutional Review Board (IRB) (Reference number: 2016/3009 and 2019/2431). For the first study investigating content validity of the Singapore version of the P-MEX using CDIs, written consent was obtained from all participants before study commencement. For the second study investigating construct validity, reliability as well as feasibility of the Singapore version of the P-MEX in a cross-sectional study, waiver of documentation of informed consent was granted.

B. Content Validity

We conducted CDIs to assess the content validity of the 23-item Singapore version of P-MEX from June 2019 to July 2019. Nine faculty who were core faculty in the residency programme were recruited using convenience sampling while ensuring representation of both genders and a range of duration of experience in the assessment of medical professionalism. Faculty chosen had to have performed assessments of medical professionalism of residents within the past one year. After viewing a video showing a resident displaying unprofessional behaviours, faculty were asked to complete the modified P-MEX assessment form in the presence of an interviewer unaided and were invited to comment on the necessity and relevance of items, instructions and response format through interview. During the interview, the faculty explained the reasons for their answers to each item, and commented on any ambiguities in the items, instructions, and response options. For example, for the question on “listened actively to patients”, we probed on (i) the meaning of the phrase ‘listened actively’, (ii) any different way to reword the phrase ‘listened actively’ based on the description, and (iii) the reason for choosing the response option (i.e., Exceeded expectations, Met expectations, Below expectations, Unacceptable, Not Applicable). The faculty were also asked whether any important issues had been omitted. We iteratively modified the items based on findings in CDIs via discussions with the study team until faculty reported no significant difficulty.

C. Cross-Sectional Study for Validation

Potential assessors (core faculty in residency programs and senior residents) in Endocrinology, Gastroenterology, Geriatric Medicine, Internal Medicine, Obstetrics & Gynaecology, Oncology, Rehabilitation Medicine, Rheumatology, and Respiratory Medicine specialties were sent an invitation via email introducing the study objectives and the study procedures. Depending on the faculty’s preference, hardcopy or electronic assessment forms were provided upon agreement to participate in the study. Prior to the study, assessors were given a short briefing on the use of the assessment forms, with descriptors of the items explained to the assessors. Residents were evaluated using the 23-item Singapore version of P-MEX forms from July 2019 to January 2020. Repeated assessments on the same resident were allowed as there was no identifiers collected to track the residents assessed. The P-MEX was conducted using direct observation of the resident by faculty as part of their workplace-based assessments. Immediate feedback was not compulsory, but it was encouraged especially if there were critical incidents observed.  The online forms were made available to faculty through an app and could be accessed conveniently when performing the direct observations.

Responses for selected items from comparator instruments were collected. Faculty were also asked to provide feedback after completing the assessment using the Singapore version of P-MEX through hardcopy or online survey, depending on faculty’s preference.

D. Singapore Version of P-MEX

The P-MEX assessed four different domains of medical professionalism: doctor-patient relationship, reflective skills, time management, and inter-professional relationship (Cruess et al., 2006). Each category contained 3 to 8 items, with a total of 23 items. The score for each item was calculated based on a four-point rating scale (4=Exceeded expectations, 3= Met expectations, 2= Below expectations, 1= Unacceptable). There was also a “Not applicable” option for each item. There is also a global question assessing the overall professional performance during the specific encounter at the end of the assessment form. Demographics information such as the level of the trainee (e.g. Medical officer, Year 1-3 Resident, Year 1-3 Senior resident) was collected as part of the P-MEX.

E. Comparator Instrument

There is no dedicated instrument to assess medical professionalism in residency programmes in Singapore. Therefore, selected questions which were deemed to encompass medical professionalism from the current assessment tool used in residency programmes (MOHH C1 form) was included as comparator instrument. The questions included- (1) Accepts responsibility and follows through on tasks, (2) Responds to patient’s unique characteristics and needs equitably, (3) Demonstrates care and concern for the patient/family, (4) Communicates effectively with patient/family, (5) Communicates and works effectively with other healthcare professionals, and (6) Demonstrates integrity and ethical behaviour. The questions were evaluated using a 9-point Likert scale (1-3: Unsatisfactory, 4-6: Satisfactory, 7-9: Superior).

F. Feedback from Assessors

The assessors’ feedback on the Singapore version of P-MEX was evaluated using the following questions using a 5-point Likert scale- (1) “Is this assessment tool (P-MEX) better than current tools used in your programme in terms of assessing medical professionalism?” (1=Very much better, 2= A little better, 3=Neutral, 4= A little worse, 5=Very much worse); (2) “Is this assessment tool (P-MEX) easy to complete?” (1=Very easy, 2= Fairly easy, 3=Neutral, 4= Fairly hard, 5=Very hard); and (3) “Are the behaviours assessed in the P-MEX reasonable and appropriate for the assessment of medical professionalism?” (1=Very reasonable and appropriate, 2= Fairly reasonable and appropriate, 3=Neutral, 4, Fairly unreasonable and inappropriate, 5=Very unreasonable and inappropriate).

G. Data Analysis

Construct validity was tested using the following a priori hypothesis:

1. Total score of the Singapore version of P-MEX items in the doctor-patient relationship domain is positively and highly correlated with “Responds to patient’s unique characteristics and needs equitably”
2. Total score of the Singapore version of P-MEX items in the doctor-patient relationship domain is positively and highly correlated with “Demonstrates care and concern for the patient/family”
3. Total score of the Singapore version of P-MEX items in the doctor-patient relationship domain is positively and highly correlated with “Communicates effectively with patient/family”
4. Total score of the Singapore version of P-MEX items in the time management domain is positively and moderately correlated with “Accepts responsibility and follows through on tasks”
5. Total score of the Singapore version of P-MEX items in the inter-professional relationship domain is positively and highly correlated with “Communicates and works effectively with other healthcare professionals”
6. The Singapore version of P-MEX global question score is positively and highly correlated with “Demonstrates integrity and ethical behaviour”

The correlation between P-MEX and comparator instrument was evaluated using Spearman’s rank correlation coefficient (r_s). High (r_s = 0.5-0.8) and moderate (r_s = 0.3-0.5) correlation coefficients suggest that the scores from 2 instruments are correlated, whereas low (r_s ≤ 0.3) correlation coefficients suggest that the PROMs are measuring different constructs (Cohen, 2016). Convergent validity is supported when at least 75% of the results are in accordance with the hypotheses (Prinsen et al., 2018). We adjusted the p values using Bonferroni’s correction to reduce the likelihood of Type I errors (Curtin & Schulz, 1998), and p values < 0.00833 were considered significant.

Structural validity was determined by first carrying out confirmatory factor analysis (CFA) through structural equation modelling to test model fit of the original four-factor model as observed in previous study on development of the P-MEX (Cruess et al., 2006). Modification indices (MIs) indicate how much the model fit will be improved after adding a new path to the model. MIs > 10 warranted covariance of error terms (Cole et al., 2005). Various goodness-of-fit indices were used to estimate the model fit, namely (1) x², (2) Comparative-of-Fit Index (CFI), (3) Tucker-Lewis Index (TLI), (4) Root mean square error of approximation (RMSEA). A model is considered to fit the data well when the x² value is not significant; CFI and TLI >0.9; RMSEA < 0.10³⁰. Standardised factor loadings were also examined.

In the case of less than acceptable model fit with the original four-domain model in CFA, exploratory factor analysis (EFA) would be conducted using the principal axis/factors method to extract factors, allowing for oblique rotation, without specifying the number of factors a priori. Value of the eigenvalues, proportion of variance explained, and cumulative variance explained would be examined. Factors with eigenvalue >1 or factors that explained >10% of the proportion of variance or cumulative variance of >75% would be retained. Factor loadings would be examined after determining the appropriate number of factors. In this step, any item with a factor loading less than 0.5 would be removed (Arifin & Yusoff, 2016).

Reliability refers to the overall consistency of the measure (Mokkink et al., 2018). In this study, reliability was assessed using internal consistency, assuming all items within a single sub-domain measured a single construct. Internal consistency is supported when Cronbach’s alpha ≥0.70.

We conducted the data analysis and factor analysis using Stata SE15.0 (Stata-Corp, College Station, TX, USA).

III. RESULTS

A. Content Validity

We conducted CDIs with nine faculty (33% male, median age 35 years old, age range from 31 to 38 years old, 89% Chinese, median year as faculty 5 years, years as faculty ranging from 1 to 10 years). The faculty were from rheumatology (n=2), obstetrics and gynaecology (n=2), respiratory and critical care medicine (n=2), renal (n=1), gastroenterology (n=1), advanced internal medicine (n=1) specialties. Overall, faculty found the P-MEX to be relevant in assessing medical professionalism without any missing concepts. However, changes to the description for 10 items were made after 7 CDIs. The main issues raised during the CDIs could be categorised into 2 main categories:

• Difficulty understanding/ misinterpretation of the intent of the question. To address this, we provided alternative terms, some of which were elicited from the faculty.
• Inappropriate examples in description of behaviour: To address this, we provided alternative examples derived from the faculty (Table 1).

The above-mentioned modifications were tested on two subsequent faculty. The faculty reported ease of comprehension with the modified P-MEX, and thus no further modifications were made.

B. Cross-Sectional Study for Validation

We received a total of 208 P-MEX assessment forms, of which 86% (n=178) assessed junior residents, and 14% (n=30) assessed senior residents. The mean (SD) number of years post-graduation for the resident was 3.4 (2.3). For all the 23 items, “met expectations” was the most frequent rating (Table 2). 6.3% (n=13) of the assessments identified behaviours that were unacceptable or below expectations, particularly in the item “solicited feedback” (n=7).  94% (n=195) of the evaluations were done based on ward activity, with the rest performed in the clinic, operating theatre or emergency room setting. Items with more than 10% ratings as “not applicable” were ‘solicited feedback’ (18.3%) and ‘admitted errors’ (13.0%).

C. Construct Validity

83% (5 out of 6) of the a priori hypotheses were met. There were significant associations between the P-MEX and the comparator instrument (all p values <0.01), with Spearman’s rank correlation coefficient ranging from 0.472 to 0.633 (Table 3).

D. Structural Validity

The CFA of the original four-factor model of the P-MEX did not meet all the acceptable fit of indexes, with x2= 573.349, CFI=0.856, TLI=0.824, and RMSEA=0.093 (Figure 1).

Given the poor fit of our data to the four-factor model, we decided to investigate the factor structure of the Singapore version of the P-MEX in the present sample by means of an EFA. EFA strongly suggested a two-factor model—the first factor had an eigenvalue of 8.52 and explained 73.8% of the variance, while the second factor had an eigenvalue of 1.90 and explained 26.2% of the variance. Three items (6. Advocated on behalf of a patient; 13. Maintained composure in a difficult situation; 22. Used health resources appropriately) had factor loadings <0.5 in the EFA. Table 4 presents the two-factor model for the present study. Factor 1 consisted of seventeen items with loadings of 0.5 or above: Listened actively to patients (factor loading 0.76); Showed interest in patients as a person (factor loading 0.62); Recognised and met patient needs (factor loading 0.72); Extended his/herself to meet patient needs(factor loading 0.66); Ensured continuity of patient care(factor loading 0.57); Maintained appropriate boundaries(factor loading 0.56); Communicated effectively with patient (factor loading 0.74); Demonstrated awareness of limitations(factor loading 0.77); Was on time(factor loading 0.53); Completed tasks in a reliable fashion(factor loading 0.71); Was available to colleagues(factor loading 0.57); Maintained appropriate appearance(factor loading 0.67); Addressed own gaps in knowledge and skills(factor loading 0.77); Demonstrated respect for colleagues(factor loading 0.83); Avoided derogatory language(factor loading 0.62); Maintained patient confidentiality(factor loading 0.65); Demonstrated collegiality(factor loading 0.73). Factor 2 consisted of three items with loadings of 0.5 or above: Admitted errors/omissions (factor loading 0.67); Solicited feedback (factor loading 0.55); Accepted feedback (factor loading 0.64).

E. Reliability

Cronbach’s alpha for the doctor-patient relationship, reflective, time management, and inter-professional relationship domains were 0.84, 0.72, 0.81 and 0.82 respectively, which implies that the original four domains of the P-MEX had satisfactory internal consistency.

Cronbach’s alpha for the 2-factor model were 0.93 and 0.72 respectively, which implies that the 2-factor model of the Singapore version of the P-MEX obtained from EFA had satisfactory internal consistency.

F. Feedback from Assessors

A total of 81 assessors (median number of years as faculty=4, range from 2 to 39) provided feedback on the Singapore version of P-MEX. Among the 81 assessors, 56 (69%) of the faculty considered the Singapore version of P-MEX to be better than current tools in terms of assessing medical professionalism. For the question regarding the ease of completion of the Singapore version of P-MEX, 57 (70%) deemed that the P-MEX to be very easy or fairly easy to complete. Lastly, 71 (88%) of the faculty deemed that the behaviours assessed in the Singapore version of P-MEX to be reasonable and appropriate for the assessment of medical professionalism.

Item	Original description	Issues raised (number of faculty who reported difficulty)	Modifications	Modified description
Domain: Doctor-patient relationship skills
Ensured continuity of patient care	Ensured proper hand-over of care of patient, e.g. at handover rounds in the evening or when discharging the patient to the care of another doctor	Unsure if the question was only applicable during transfer of care (n=1)	Added “proper follow-up of patient”	Ensured proper hand-over of care of patient (e.g. at handover rounds in the evening or when discharging the patient to the care of another doctor) or proper follow-up of patient
Advocated on behalf of a patient	Spoke up in support of the patient – e.g. made an appeal on behalf of the patient to get subsidy for medical care	Unsure whether the making an appeal on behalf of the patient would be applicable in local context, suggested that better example would be standing by the patient during a conflict in the treatment plan between the patient and family (n=5)	Added the example “making known the wishes of the patient when there is a conflict of opinion in the treatment plan between family members and patient”	Spoke up in support of the patient’s opinion, interests, priorities and concerns – e.g. made an appeal on behalf of the patient to get subsidy for medical care; making known the wishes of the patient when they is a conflict of opinion in the treatment plan between family members and patient
Maintained appropriate boundaries	Aware of and behaved appropriately according to social, religious and cultural norms in our multi-ethnic and multi-religious country. E.g. neither overly friendly or unfriendly to patients and colleagues	Commented that the example provided may be inappropriate as being unfriendly may be keeping appropriate distance (n=5)	Removed the example “neither overly friendly or unfriendly to patients and colleagues”	Aware of and behaved appropriately according to social, religious and cultural norms in our multi-ethnic and multi-religious country
Communicated effectively with patient	Explained with empathy the diagnosis of the disease, treatment options (including alternatives), risks and benefits, and ensured that patients understood	Commented that clarity is more important than empathy in communication (n=1)	Changed “explained with empathy” to “explained with empathy and clarity”	Explained with empathy and clarity the diagnosis of the disease, treatment options (including alternatives), risks and benefits, and ensured that patients understood
Domain: Reflective skills
Solicited feedback	Actively asked for feedback on his/her actions or management of patients	Unsure if the question was asking about soliciting feedback from who (n=4)	Added “from colleagues”	Actively asked for feedback from colleagues on his/her actions or management of patients
Accepted feedback	Open to criticisms and comments and had the humility to learn from feedback	Unsure if the question was asking about accepting feedback from who (n=2)	Added “from patients and/ or colleagues”	Open to criticisms and comments and had the humility to learn from feedback from patients and/ or colleagues
Domain: Inter-professional relationship skills
Maintained appropriate appearance	Maintained appropriate appearance Appeared clean, neat, groomed and dressed modestly	Unsure about the meaning of “modestly” (n=2)	Changed “modestly” to “appropriately (i.e convey a professional image)”	Appeared clean, neat, groomed and dressed appropriately (i.e convey a professional image)
Addressed own gaps in knowledge and skills	Kept up to date in medical knowledge and skills. E.g. by reading journals, attending courses and conferences	Commented that it would be good if the trainee can “understand gaps in knowledge and skills” (n=2)	Added “Recognised gaps in his/her knowledge”	Recognised gaps in his/her knowledge and kept up to date in medical knowledge and skills. E.g. by reading journals, attending courses and conferences
Demonstrated respect for colleagues	Listened to their professional opinion and did not oppose colleagues in public	Commented that “did not oppose colleagues in public” was inappropriate as doctors should be speaking up when there are differing opinion (n=5)	Changed “did not oppose colleagues in public” to “resolved difference of opinions in a professional and non-confrontational manner”	Listened to professional opinion of healthcare team and resolved difference of opinions in a professional and non-confrontational manner
Avoided derogatory language	Did not ridicule, insult or use foul language when talking about patients or colleagues	Commented that avoiding derogatory language was more important when talking to patients and colleagues (n=2)	Changed “when talking about patients or colleagues” to “when talking to patients or colleagues”	Did not ridicule, insult or use foul language when talking to patients or colleagues

Table 1. Issues raised during the CDIs and modifications made to the P-MEX

Item	Not applicable	Unacceptable	Below expectations	Met expectations	Exceeded expectations
Domain: Doctor-patient relationship skills
Listened actively to patients	0 (0)	0 (0)	0 (0)	151 (72.6)	57 (27.4)
Showed interest in patients as a person	1 (0.5)	0 (0)	1 (0.5)	145 (69.7)	61 (29.3)
Recognised and met patient needs	0 (0)	0 (0)	3 (1.4)	147 (70.7)	58 (27.9)
Extended his/herself to meet patient needs	0 (0)	0 (0)	2 (1.0)	154 (74.0)	52 (25.0)
Ensured continuity of patient care	1 (0.5)	0 (0)	3 (1.4)	131 (63.0)	73 (35.1)
Advocated on behalf of a patient	5 (2.4)	0 (0)	3 (1.4)	155 (74.5)	45 (21.6)
Maintained appropriate boundaries	6 (3.0)	0 (0)	2 (1.0)	151 (72.6)	49 (23.6)
Communicated effectively with patient	0 (0)	0 (0)	3 (1.4)	159 (76.4)	48 (23.1)
Domain: Reflective skills
Demonstrated awareness of limitations	0 (0)	0 (0)	1 (0.5)	159 (76.4)	48 (23.1)
Admitted errors/omissions	27 (13.0)	0 (0)	3 (1.4)	142 (68.3)	36 (17.3)
Solicited feedback	38 (18.3)	0 (0)	7 (3.4)	139 (66.8)	24 (11.5)
Accepted feedback	15 (7.2)	0 (0)	3 (1.4)	159 (76.4)	31 (14.9)
Maintained composure in a difficult situation	17 (8.2)	0 (0)	4 (1.9)	147 (70.7)	40 (19.2)
Domain: Time management
Was on time	0 (0)	0 (0)	1 (0.5)	137 (65.9)	70 (33.7)
Completed tasks in a reliable fashion	1 (0.5)	0 (0)	2 (1.0)	115 (55.3)	90 (43.3)
Was available to colleagues	2 (1.0)	0 (0)	3 (1.4)	131 (63.0)	72 (34.6)
Domain: Inter-professional relationship skills
Maintained appropriate appearance	0 (0)	0 (0)	1 (0.5)	141 (67.8)	66 (31.7)
Addressed own gaps in knowledge and skills	2 (1.0)	0 (0)	4 (1.9)	167 (80.3)	35 (16.8)
Demonstrated respect for colleagues	0 (0)	0 (0)	2 (1.0)	133 (63.9)	73 (35.1)
Avoided derogatory language	2 (1.0)	0 (0)	0 (0)	142 (68.3)	64 (30.8)
Maintained patient confidentiality	1 (0.5)	0 (0)	1 (0.5)	152 (73.1)	54 (26.0)
Used health resources appropriately	4 (1.9)	0 (0)	1 (0.5)	173 (83.2)	30 (14.4)
Demonstrated collegiality	0 (0)	0 (0)	1 (0.5)	134 (64.4)	73 (35.1)

Table 2. Distribution of score (n%) for the 208 P-MEX assessment forms received

P-MEX components	Question from comparator instrument	Hypothesis	Spearman correlation	Confirmation
Doctor-patient relationship domain	Responds to patient’s unique characteristics and needs equitably	High (+)	0.633*	Yes
Doctor-patient relationship domain	Demonstrates care and concern for the patient/family	High (+)	0.580*	Yes
Doctor-patient relationship domain	Communicates effectively with patient/family	High (+)	0.617*	Yes
Time management domain	Accepts responsibility and follows through on tasks	Moderate (+)	0.472*	Yes
Inter-professional relationship domain	Communicates and works effectively with other healthcare professionals	High (+)	0.498*	No
Global score	Demonstrates integrity and ethical behaviour	High (+)	0.545*	Yes

*p<0.00833 as corrected using Bonferroni’s correction; (+) and (-) indicate the direction of correlations.

Table 3. Construct validity of the P-MEX

Item	Factor 1	Factor 2	Uniqueness
Listened actively to patients	0.76		0.42
Showed interest in patients as a person	0.62		0.61
Recognised and met patient needs	0.72		0.47
Extended his/herself to meet patient needs	0.66		0.56
Ensured continuity of patient care	0.57		0.67
Advocated on behalf of a patient	a		0.82
Maintained appropriate boundaries	0.56		0.68
Communicated effectively with patient	0.74		0.45
Demonstrated awareness of limitations	0.77		0.41
Admitted errors/omissions		0.67	0.53
Solicited feedback		0.55	0.62
Accepted feedback		0.64	0.55
Maintained composure in a difficult situation	a		0.62
Was on time	0.71		0.42
Completed tasks in a reliable fashion	0.57		0.63
Was available to colleagues	0.67		0.50
Maintained appropriate appearance	0.77		0.37
Addressed own gaps in knowledge and skills	0.53		0.57
Demonstrated respect for colleagues	0.83		0.26
Avoided derogatory language	0.62		0.59
Maintained patient confidentiality	0.65		0.57
Used health resources appropriately	a		0.85
Demonstrated collegiality	0.73		0.43

^a represent loading <0 .5

Table 4. Factor loadings (pattern matrix) and unique variances of the P-MEX items in exploratory factor analysis

Figure 1. Confirmatory factor analysis (CFA) through structural equation modelling of the P-MEX. Latent variables are shown in ellipses, while observed variables are shown in rectangles. Directed arrows represent causal relationship among variables, while double-headed arrows represent correlations between variables. The numbers beside the arrows indicate the standard path coefficients.

IV. DISCUSSION

This study is the first to validate the psychometric properties of the Singapore version of P-MEX. The CDIs indicate that the Singapore version of P-MEX was generally deemed by faculty to be easy to understand, relevant and comprehensive. Feedback by assessors on the Singapore P-MEX were largely positive, with majority of assessors saying that the Singapore version of P-MEX was better than current assessment tools used in their respective programmes for the assessment of medical professionalism. The study also involved faculty and senior residents as assessors to reflect real-life situations that took place in the assessment of residents. This is important as 360-degree assessments and workplace-based assessments often involve both faculty and senior residents.

The four-factor CFA had a poor fit. Based on the EFA with a two-factor structure, three items (Advocated on behalf of a patient; Maintained composure in a difficult situation; Used health resources appropriately) had factor loadings less than 0.5, thus these 3 items could be removed in future assessments. Two of the items (Advocated on behalf of a patient; Used health resources appropriately) also did not reach consensus for either inclusion or exclusion into the questionnaire in a previous Delphi study (Fong et al., 2020). In the two-factor structure, factor 1 appeared to encompass “Patient care and inter-professional relationship skills” while factor 2 appeared to encompass “Reflective skills”. Previously, the structural validity of the four-factor model of the P-MEX has been supported in various studies (Tsugawa et al., 2011), and this divergence in structural validity observed in our sample may imply that the measure of medical professionalism may differ  even among Asian countries. It might be beneficial for medical organisations with different cultural backgrounds to conduct a factor analysis with their data independently prior to use. The lack of fit to the four-factor structure could also be because psychometric properties of some items measured more than one latent construct. For example, while ‘Demonstrated awareness of limitations’ showed reflective skills, it could also be an important marker of appropriate patient care and inter-professional relationship, where the resident sought help from more experienced colleagues when needed. Items under time management, inter-professional relationship and doctor-patient relationship also came together to measure the same construct, which appeared to encompass “Patient care and inter-professional relationship skills”. With Singapore’s rapidly aging population, the rise in numbers of patients with chronic diseases and complex conditions warrant healthcare professionals across different specialties to work in multidisciplinary teams and contribute their unique expertise to improve the quality of patient care (Tan et al., 2021; Zheng et al., 2016). As such, items measuring time management, inter-professional relationship and doctor-patient relationship might overlap and be viewed as one entity instead of individual segments.

Our study had identified professional lapses in 6.3% of the residents. This was comparable to the study by Papadakis et al (Papadakis et al., 2008), where 4.8% of the residents were noted to have low professionalism ratings in their assessments across all years of residency training. This was also consistent with the overall incidence of unprofessional behaviour that has been flagged up by faculty in disciplinary meetings, ranging between 1% to 3% of the cohort of residents from 2015 to 2019 (unpublished data). Also, only 13 cases (4.1%) of Singapore Medical Council (SMC) disciplinary tribunals involved junior doctors from 1979 to 2017 (Norman et al., 2021).

Medical unprofessionalism may be associated with suboptimal patient care. Therefore it is imperative for healthcare organizations to invest in efforts to improve medical professionalism. There is currently no validated instrument to assess medical professionalism in residency programmes in Singapore. The Singapore version of P-MEX validated in this study may be useful to bridge this gap, and allows provision of feedback to residents for improvement in medical professionalism (Hawkins et al., 2009). Assessment may also help to measure the effectiveness of educational programs that aim to instil medical professionalism (Hawkins et al., 2009), which are increasingly common in medical education (Lee et al., 2007; Macneill et al., 2020).

This study has several limitations. Firstly, we were unable to determine the reproducibility of the average score of the residents using the Singapore version of P-MEX by the generalisability theory as the IRB did not allow for any identifiers to be collected. Therefore, we were unable to track the residents assessed. Future studies will need to be performed to determine the number of evaluations needed to obtain a satisfactory dependability coefficient for the Singapore version of P-MEX. Secondly, as we did not collect data after remediation on medical professionalism, we were unable to determine the responsiveness of the Singapore version of P-MEX to remediation in our population. Thirdly, the participants in the CDIs and cross-sectional study were recruited using convenience sampling and the participants were mainly from medical disciplines. Coupled with the possibility that faculty members who are more interested in medical professionalism may be more likely to participate, this could skew the results because the sample may not be representative of the entire faculty and resident population. Moreover, the narrow distribution of the P-MEX scores has a halo effect on all items which limits variability and co-variability. However, the concentration of the P-MEX scores near the ceiling suggests that the most of the participants are professional, which is supported by previous study (Papadakis et al., 2008). Lastly, we used the same sample for CFA and EFA due to the small sample size, which may lead to overfitting. Future research should consider larger and more diverse samples involving more specialties to improve generalizability and robustness of results.

V. CONCLUSION

This study supports the content validity, construct validity, internal consistency and feasibility of the Singapore version of P-MEX in assessing medical professionalism in the local residency training programme. However, this study did not provide conclusive support for the original four-factor model of the P-MEX, and the EFA identified 2 main factors in our sample, which appeared to encompass “Patient care and inter-professional relationship skills” and “Reflective skills”. Future research should examine the structural validity of the P-MEX in different samples of medical trainees. Medical educators in the region with similar multi-cultural backgrounds can consider using the Singapore version of P-MEX to assess medical professionalism among trainees.

Notes on Contributors

WF, YHK, SWY conceptualised the study design, and drafted the survey questionnaire. WF, YHK and JKP acquired the data. WF, YHK, SWY and JKP wrote the initial manuscript draft. All authors critically reviewed and approved the final manuscript.

Ethical Approval

The SingHealth Centralised Institutional Review Board approved this study (Reference Number: 2016/3009 and 2019/2431).

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Data is not deposited in repository due to privacy concerns.

Acknowledgement

The authors thank all the participants for their time and contribution to the study.

Funding

This research was supported by SingHealth Duke-NUS Medicine Academic Clinical Programme Education Support Programme Grant (Ref No: 03/FY2017/P2/03-A47). Funder was not involved in the design, delivery or submission of the research.

Declaration of Interest

The authors declare that they have no competing interests.

References

Amirhajlou, L., Bidari, A., Alipour, F., Yaseri, M., Vaziri, S., Rezai, M., Tavakoli, N., Farsi, D., Yasinzadeh, A. R., Mosaddegh, R., & Hashemi, A. (2019). Validity, reliability and acceptability of Professionalism Mini-Evaluation Exercise (P-MEX) for emergency medicine residency training. Journal of Medical Ethics and History of Medicine, 12. https://doi.org/10.18502/jmehm.v12i12.1641

Arifin, W. N., & Yusoff, M. S. B. (2016). Confirmatory factor analysis of the Universiti Sains Malaysia emotional quotient inventory among medical students in Malaysia. Sage Open, 6(2), 2158244016650240. https://doi.org/10.1177/2158244016650240

Arrighi, J. A., Lewis, L. C., & Nasca, T. J. (2024). The Role of ACGME International in enhancing opportunities of residents for advanced subspecialty training. Journal of Graduate Medical Education, 16(2), 237-240. https://doi.org/10.4300/jgme-d-24-00150.1

Bajwa, N. M., Nendaz, M. R., Posfay-Barbe, K. M., Yudkowsky, R., & Park, Y. S. (2021). A meaningful and actionable professionalism assessment: Validity evidence for the Professionalism Mini-Evaluation Exercise (P-MEX) across 8 years. Academic Medicine, 96(11S), S151-S157. https://doi.org/10.1097/ACM.0000000000004286

Chandratilake, M., McAleer, S., & Gibson, J. (2012). Cultural similarities and differences in medical professionalism: A multi‐region study. Medical Education, 46(3), 257-266. https://doi.org/10.1111/j.1365-2923.2011.04153.x

Cohen, J. (2016). A power primer. https://doi.org/10.1037/14805-018

Cole, J. C., Motivala, S. J., Khanna, D., Lee, J. Y., Paulus, H. E., & Irwin, M. R. (2005). Validation of single‐factor structure and scoring protocol for the Health Assessment Questionnaire‐Disability Index. Arthritis Care & Research: Official Journal of the American College of Rheumatology, 53(4), 536-542. https://doi.org/10.1002/art.21325

Cruess, R., McIlroy, J. H., Cruess, S., Ginsburg, S., & Steinert, Y. (2006). The professionalism mini-evaluation exercise: A preliminary investigation. Academic Medicine, 81(10), S74-S78.

Cruess, R. L., & Cruess, S. R. (2006). Teaching professionalism: General principles. Medical Teacher, 28(3), 205-208.  https://doi.org/10.1080/01421590600643653

Cruess, R. L., Cruess, S. R., Boudreau, J. D., Snell, L., & Steinert, Y. (2015). A schematic representation of the professional identity formation and socialization of medical students and residents: A guide for medical educators. Academic Medicine, 90(6), 718-725. https://doi.org/10.1097/ACM.0000000000000700

Cruess, S. R., Cruess, R. L., & Steinert, Y. (2010). Teaching professionalism across cultural and national borders: Lessons learned from an AMEE workshop. Medical Teacher, 32(5), 371-374. https://doi.org/10.3109/01421591003692730

Cruess, S. R., Cruess, R. L., & Steinert, Y. (2019). Supporting the development of a professional identity: General principles. Medical Teacher, 41(6), 641-649. https://doi.org/10.1080/0142159X.2018.1536260

Curtin, F., & Schulz, P. (1998). Multiple correlations and Bonferroni’s correction. Biological Psychiatry, 44(8), 775-777. https://doi.org/10.1016/S0006-3223(98)00043-2

Fong, W., Kwan, Y. H., Yoon, S., Phang, J. K., Thumboo, J., Leung, Y. Y., & Ng, S. C. (2020). Assessment of medical professionalism using the Professionalism Mini Evaluation Exercise (P-MEX) in a multi-ethnic society: A Delphi study. BMC Medical Education, 20, 1-9. https://doi.org/10.1186/s12909-020-02147-9

Guerrasio, J., Garrity, M. J., & Aagaard, E. M. (2014). Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006–2012. Academic Medicine, 89(2), 352-358. https://doi.org/10.1097/ACM.0000000000000122

Hawkins, R. E., Katsufrakis, P. J., Holtman, M. C., & Clauser, B. E. (2009). Assessment of medical professionalism: Who, what, when, where, how, and… why? Medical Teacher, 31(4), 348-361. https://doi.org/10.1080/01421590902887404

Ho, M. J., Yu, K. H., Hirsh, D., Huang, T. S., & Yang, P. C. (2011). Does one size fit all? Building a framework for medical professionalism. Academic Medicine, 86(11), 1407-1414. https://doi.org/10.1097/ACM.0b013e31823059d1

Ho, M. J., Yu, K. H., Pan, H., Norris, J. L., Liang, Y. S., Li, J. N., & Hirsh, D. (2014). A tale of two cities: Understanding the differences in medical professionalism between two Chinese cultural contexts. Academic Medicine, 89(6), 944-950. https://doi.org/10.1097/ACM.0000000000000240

Hodges, B. D., Ginsburg, S., Cruess, R., Cruess, S., Delport, R., Hafferty, F., Ho, M. J., Holmboe, E., Holtman, M., Ohbu, S., Rees, C., Cate, O. T., Tsugawa, Y., Mook, W.V., Wilkinson, T., & Wade, W. (2011). Assessment of professionalism: Recommendations from the Ottawa 2010 Conference. Medical Teacher, 33(5), 354-363. https://doi.org/10.3109/0142159X.2011.577300

Karukivi, M., Kortekangas-Savolainen, O., Saxén, U., & Haapasalo-Pesu, K. M. (2015). Professionalism Mini-Evaluation Exercise in Finland: A preliminary investigation introducing the Finnish version of the P-MEX instrument. Journal of Advances in Medical Education & Professionalism, 3(4), 154.

Kwan, Y. H., Png, K., Phang, J. K., Leung, Y. Y., Goh, H., Seah, Y., Thumboo, J., Ng, S. C., Fong, W., & Lie, D. (2018). A systematic review of the quality and utility of observer-based instruments for assessing medical professionalism. Journal of Graduate Medical Education, 10(6), 629-638. https://doi.org/10.4300/JGME-D-18-00086.1

Lee, A. G., Beaver, H. A., Boldt, H. C., Olson, R., Oetting, T. A., Abramoff, M., & Carter, K. (2007). Teaching and assessing professionalism in ophthalmology residency training programs. Survey of Ophthalmology, 52(3), 300-314. https://doi.org/10.1016/j.survophthal.2007.02.003

Lumlertgul, N., Kijpaisalratana, N., Pityaratstian, N., & Wangsaturaka, D. (2009). Cinemeducation: A pilot student project using movies to help students learn medical professionalism. Medical Teacher, 31(7), e327-e332. https://doi.org/10.1080/01421590802637941

Macneill, P., Joseph, R., Lysaght, T., Samarasekera, D. D., & Hooi, S. C. (2020). A professionalism program in medical education and training – From broad values to specific applications: YLL School of Medicine, Singapore. Medical Teacher, 42(5), 561-571. https://doi.org/10.1080/0142159X.2020.1714021

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63-67.

Mokkink, L. B., De Vet, H. C., Prinsen, C. A., Patrick, D. L., Alonso, J., Bouter, L. M., & Terwee, C. B. (2018). COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27, 1171-1179. https://doi.org/10.1007/s11136-017-1765-4

Noor, N. M., & Leong, C.-H. (2013). Multiculturalism in Malaysia and Singapore: Contesting models. International Journal of Intercultural Relations, 37(6), 714-726. https://doi.org/10.1016/j.ijintrel.2013.09.009

Norman, N. B. M., Soo, J. M. P., Lam, M. Y. K., & Thirumoorthy, T. (2021). Unprofessional behaviour of junior doctors: A retrospective analysis of outcomes by the Singapore Medical Council disciplinary tribunals. Singapore Medical Journal, 62(3), 120. https://doi.org/10.11622/smedj.2020021

Papadakis, M. A., Arnold, G. K., Blank, L. L., Holmboe, E. S., & Lipner, R. S. (2008). Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Annals of Internal Medicine, 148(11), 869-876. https://doi.org/10.7326/0003-4819-148-11-200806030-00009

Prinsen, C. A., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., De Vet, H. C., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27, 1147-1157. https://doi.org/10.1007/s11136-018-1798-3

SingHealth. (2024). Why SingHealth Residency.

Swick, H. M., Szenas, P., Danoff, D., & Whitcomb, M. E. (1999). Teaching professionalism in undergraduate medical education. Jama, 282(9), 830-832. https://doi.org/10.1001/jama.282.9.830

Swing, S. R. (2007). The ACGME outcome project: Retrospective and prospective. Medical Teacher, 29(7), 648-654. https://doi.org/10.1080/01421590701392903

Tan, C. C., Lam, C. S., Matchar, D. B., Zee, Y. K., & Wong, J. E. (2021). Singapore’s health-care system: Key features, challenges, and shifts. The Lancet, 398(10305), 1091-1104.

Taşçı, A. İ., Akdeniz, E., Gülpınar, M. A., Danacıoğlu, Y. O., Sarı, E. E., Yaşar, L., Karandere, F., & Ferahman, S. (2023). Adaptation of the professionalism mini-evaluation exercise instrument into Turkish: A validity and reliability study. BMC Medical Education, 23(1), 698. https://doi.org/10.1186/s12909-023-04675-6

Tsugawa, Y., Ohbu, S., Cruess, R., Cruess, S., Okubo, T., Takahashi, O., Tokuda, Y., Heist, B., Bito, S., Itoh, T., Aoki, A., Chiba, T., & Fukui, T. (2011). Introducing the Professionalism Mini-Evaluation Exercise (P-MEX) in Japan: Results from a multicenter, cross-sectional study. Academic Medicine, 86(8), 1026-1031. https://doi.org/10.1097/ACM.0b013e3182222ba0

Tsugawa, Y., Tokuda, Y., Ohbu, S., Okubo, T., Cruess, R., Cruess, S., Ohde, S., Okada, S., Hayashida, N., & Fukui, T. (2009). Professionalism mini‐evaluation exercise for medical residents in Japan: A pilot study. Medical Education, 43(10), 968-978. https://doi.org/10.1111/j.1365-2923.2009.03437.x

Zbieranowski, I., Takahashi, S. G., Verma, S., & Spadafora, S. M. (2013). Remediation of residents in difficulty: A retrospective 10-year review of the experience of a postgraduate board of examiners. Academic Medicine, 88(1), 111-116. https://doi.org/10.1097/ACM.0b013e3182764cb6

Zheng, R. M., Sim, Y. F., & Koh, G. C.-H. (2016). Attitudes towards interprofessional collaboration among primary care physicians and nurses in Singapore. Journal of Interprofessional Care, 30(4), 505-511. https://doi.org/10.3109/13561820.2016.1160039

*Warren Fong
Academia, Level 4, 20 College Road, Singapore 169856
Email: gmsfwsw@nus.edu.sg

Submitted: 24 October 2024
Accepted: 5 July 2025
Published online: 7 October, TAPS 2025, 10(4), 26-34
https://doi.org/10.29060/TAPS.2025-10-4/OA3552

Shoko Horita^1,2, Masashi Izumiya², Satoshi Kondo^2,3,4, Junki Mizumoto^2,5,6, Hiroko Mori^6,7 & Masato Eto²

¹Department of Medical Education, School of Medicine, Teikyo University, Itabashi-ku, Tokyo, Japan; ²Department of Medical Education Studies, International Research Centre for Medical Education, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan; ³Department of Medical Education, Graduate School of Medicine, University of Toyama, Toyama, Japan; ⁴Center for Medical Education and Career Development, Graduate School of Medicine, University of Toyama, Toyama, Japan; ⁵Department of Family Practice, Ehime Seikyo Hospital, Matsuyama, Ehime, Japan; ⁶Center for General Medicine Education, School of Medicine, Keio University, Shinjuku, Tokyo, Japan; ⁷Professional Development Centre, The University of Tokyo Hospital, Bunkyo-ku, Tokyo, Japan

Abstract

Introduction: Conventionally, face-to-face education has been prevalent in medical education because it can help medical students learn interpersonal skills, including medical interviews and physical examination. However, because of the coronavirus disease 2019 pandemic, face-to-face education was suspended to prevent the spread of the infection. As face-to-face classes in Japan were discontinued when the pandemic began in the spring of 2020, we developed an online education program to develop medical interview skills. We were interested in determining the educational outcomes between face-to-face and online medical interview classes. Therefore, we compared them before and after the pandemic.

Methods: Fourth-year students of the University of Tokyo Medical School took medical interview classes. Under consent, the score of the medical interview area of the preclinical clerkship, Objective Structured Clinical Examination (OSCE), as a high-stakes examination, which falls at the top level of the Kirkpatrick’s model, was compared by year or before and after the pandemic.

Results: The online group showed higher item-wise scores of the medical interview of the preclinical clerkship OSCE than the face-to-face group. In terms of the global score, no significant difference was observed. In the computer-based test (CBT), the online group had higher scores compared with the face-to-face group.

Conclusion: The educational outcomes of online medical interview classes were not inferior to those of conventional face-to-face classes, as revealed by high-stakes examination preclinical clerkship OSCE. Similar to face-to-face education, online education is a viable option for developing interpersonal skills.

Keywords:          COVID-19 Pandemic, Medical Interview, OSCE, Educational Outcome, Online Education, Interpersonal Skills, Communication Skills

Practice Highlights

• Due to the COVID-19 pandemic, we shifted medical interview classes from face-to-face to online.
• The online group had interview global OSCE scores non-inferior to those of the face-to-face group.
• The online group had higher interview elementary OSCE scores than the face-to-face group.

I. INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic severely restricted face-to-face teaching and affected almost all levels and fields of education, including undergraduate preclinical medical education (Bastos et al., 2022; Crawford et al., 2020). Moreover, it resulted in drastic changes in medical education. Globally, face-to-face learning was forcibly discontinued as part of infection control. Thus, to continue medical education, online or remote learning was rapidly introduced (Daniel et al., 2021; Gordon et al., 2020). Various instrumental trans communication devices, including video conferencing tools, simulation, virtual reality, and augmented reality, were used to facilitate online learning. However, this rather hasty shift from face-to-face to online learning brought some confusion into the field of medical education. In the UK, Dost et al. (2020) reported that medical students were unsatisfied with online classes compared with face-to-face classes.

Globally, tele-education is increasingly being encouraged around the world (American Medical Association, 2016). In the field of medical interview (Budakoğlu et al., 2021; Hammersley et al., 2019; Zaccariah et al., 2022), telemedicine is gradually becoming common, showing favourable results. However, because of technical problems, tele-education did not spread smoothly (Zaccariah et al., 2022). Additionally, the educational outcomes of both strategies have not been satisfactorily studied (Khamees et al., 2022). Recently, some reports showing that the educational outcome of online classes are equal or more effective than traditional face-to-face education, however, they are restricted mainly in knowledge-based education (Alshaibani et al., 2023; Basuodan, 2024; Saad et al., 2023). Furthermore, few studies have compared high-stakes examination, including the Objective Structured Clinical Examination (OSCE), and no study has compared the educational results between face-to-face classes and tele-education (online) using the top level of Kirkpatrick’s model (Kirkpatrick, 1996).

The OSCE (Harden et al., 1975) has been widely accepted as a form to assess clinical performance in medical education. Currently, OSCEs are used worldwide to appraise medical students’ communication and clinical skills. Various educational methods have been evaluated using OSCE as one of the indicators of educational outcomes (Guetterman et al., 2019). In Japan, passing the preclinical clerkship (pre-CC) OSCE has become legally obligatory as one of the elements for promotion to the CC course since the spring of 2023. In 2023, the pre-CC OSCE in Japan is conducted in at least eight areas, which are medical interview, “Basic Clinical Procedure”, “Basic Life Support”, and physical examinations of “head and neck”, “chest”, “vital signs”, “abdomen”, and “neurological examinations”.

In the present study, we aimed to determine the educational outcomes between face-to-face and online medical interview classes. We provided medical interview classes to fourth-year medical students before taking the pre-CC OSCE, face-to-face classes before 2019, and tele-education (online) after 2020. We decided to conduct research in medical interview, other than the other areas of the pre-CC OSCE, because of the importance of the medical interview as the basis of medical practice. Moreover, it was inevitable that the medical interview classes had to be implemented as online classes to protect the simulated patients form the risk of infection, which was another main reason for selecting medical interview for this research. In another point of view, medical interview classes were able to implement via online. As mentioned above, no prior studies have compared face-to-face and online medical interview training using both high-stakes OSCE score and Kirkpatrick’s top-level outcomes, our study would have significant importance.

II. METHOD

A. Participants

This study was approved in 2021 by the ethics committee of the University of Tokyo (UTokyo) Faculty of Medicine (Approval No. 2021005NI). All methods were performed in accordance with the relevant guidelines and regulations. Moreover, the data of students who provided consent for the secondary use of their data (Approval No. 11763) in another research approved in 2017 were included.

B. Sample Population

Students in the UTokyo Faculty of Medicine were asked if they were willing to participate in “A Study of the Educational Effectiveness of Online “Medical interviewing Practice” in the post-class reflection questionnaire of the “Online medical interview classes or the waiting period after the pre-CC OSCE. Out of 229 students (2021 and 2022), 87 students participated in this study. A summary of the annual participants is shown in Appendix 1. In early 2020 almost all the classes in UTokyo were stopped due to the COVID-19 pandemic, which made it difficult to contact students face-to-face and to obtain participants in the previous research (Approval No. 11763); and as this research started in 2021, it was practically difficult to obtain consent to participate in this study in 2020. In 2020 the online medical interview classes have just been launched, which significantly improved in 2021. Hence, we thought that it would be better to exclude the small participants of 2020 from the analysis to keep the validity of this study.

C. Details of Medical Interview Classes

Before 2019, the medical interview classes were performed as follows: Early in their fourth year, students joined classes introducing the outline of medical interview. A few days before the class, students watched an instructional video of a medical interview performed by the Common Achievement Tests Organization (CATO) (2005) in Japan. Afterward, students in a group of eight to nine faced the simulated patient in a classroom in the UTokyo and performed a medical interview roleplay. Thereafter, feedback about the technical factor of the medical interview as well as rapport status and nonverbal communications such as faces and gesture was provided by the students themselves, other students, simulated patients, and teachers. Since 2020, most face-to-face classrooms, including those in the present study, were closed because of the COVID-19 pandemic and were replaced with online classes. The present face-to-face class was also held online with the simulated patients and teachers using Zoom(R). Using the “Close-Up” function of Zoom(R), the simulated patient and student were faced with each other, whereas other participants (e.g., other students, other simulated patients, and the teacher) were not on the television (Appendix 2). After the roleplay was over, all students and the teacher came back on the television and provided feedback to the student, similar to face-to-face classes. Moreover, the class was recorded using the function of Zoom(R) and provided to students exclusively for review. After the class students reflected on the reflection sheet (until 2019) or the Learning Management System (from 2020) which was reviewed and commented on by teachers. The contents of the reflections were used for this study to investigate the impressions of the students.

D. Pre-CC OSCE and Computer-Based Test (CBT)

In Japan, medical students usually take the pre-CC OSCE in the fourth year, prior to the two-year CC course. Before 2022, the minimum assessment factors were medical interview, physical examinations (including head and neck, chest, abdomen, neurological examinations), basic clinical procedure, and basic life support. The examinations were administered by CATO. The evaluation criteria are not publicly available because of CATO policy. Two scores are used in the evaluation: global score (GS) which means the evaluation as a total performance and item-wise score (IS) which means scores by checklist. Before 2023, the borderline was set by each institute. At least one certified evaluator per area was responsible, and each evaluator was a faculty member. Moreover, CATO sent at least one external evaluator per area and an external supervisor. After each performance, each examinee was evaluated by two or three evaluators per room. The pre-CC OSCE is one of the examinations that students must pass to proceed to the CC course.

Aside from the pre-CC OSCE, students must also pass the CBT. The CBT corresponds to the assessment of medical knowledge prior to the CC (Horita et al., 2021). In 2023, the pre-CC OSCE and CBT have been made official, and students must pass both examinations before they can take the national board examination in Japan.

E. Data Analysis

The pre-CC OSCE scores were analysed using R, Rstudio, JMP version17.0 (SAS Institute, N.C., USA) and Microsoft Excel (Microsoft Corporation, W.A., USA). Non-paired T test, Mann-Whitney U test, and Steel-Dwass test were used respectively, for parametric or non-parametric comparisons.

III. RESULTS

A. Year-to-Year Comparison of the Pre-CC OSCE Results in the Medical Interview area and CBT Results

First, we compared the year-to-year results of the pre-CC OSCE in the medical interview area. Table 1 and Figure 1 shows a statistical summary of the pre-CC OSCE scores in 2022, 2021, 2019, and 2018. The results of non-parametric tests revealed that the p-values in the IS between 2022 and 2019, 2022 and 2018, and 2021 and 2018 were below 0.05, whereas no significant difference was observed in the GS.

Submitted: 30 December 2024
Accepted: 5 July 2025
Published online: 7 October, TAPS 2025, 10(4), 63-72
https://doi.org/10.29060/TAPS.2025-10-4/OA3777

Chollada Sorasak¹, Worayuth Nak-Ai², Choosak Yuennan³ & Mansuang Wongsapai¹

¹Intercountry Centre for Oral Health, Department of Health, Thailand; ²Sirindhorn College of Public Health Chonburi, Praboromarajchanok Institute, Thailand; ³Boromarajonani College of Nursing Chiang Mai, Praboromarajchanok Institute, Thailand

Abstract

Introduction: Nutrition literacy represents a critical determinant of oral health outcomes. Guided by Social Cognitive Theory and the Nutrition Literacy Skills Framework, this study evaluated the implementation and effectiveness of a nutrition literacy programme for oral health promotion among village health volunteers (VHVs), key implementers in Thailand’s healthcare system, during January to December 2024.

Methods: A convergent parallel mixed-methods design was employed to address existing methodological gaps in nutrition literacy research. The quantitative component comprised a cross-sectional survey (N=60 VHVs trained in January 2024) and clinical outcome monitoring via electronic health records. The qualitative strand involved a multi-case study approach with purposive sampling (n=20) through in-depth interviews. Data collection occurred at 6-month post-implementation (July 2024), with clinical monitoring through December 2024. Analysis integrated descriptive and inferential statistics with thematic analysis.

Results: Post-implementation analysis revealed significantly enhanced nutrition literacy skills (M=4.14, SD=0.414), with notable improvements in communication (M=4.74, SD=0.511) and implementation (M=4.21, SD=0.440). All six nutrition literacy domains showed strong correlations (r=0.712-0.868, p<.01), supporting the framework’s interconnected nature. Clinical outcomes improved significantly: oral health check-up rates increased from 1.41% to 2.61% (p<.05), and functional teeth retention rose from 87.36% to 92.72% (p<.01). Qualitative findings revealed adaptive knowledge transfer methods and context-specific implementation strategies influenced by community readiness.

Conclusion: Through comprehensive mixed-methods evaluation, the 12-month implementation data demonstrated significant improvements in both VHVs’ nutrition literacy skills and clinical oral health outcomes. Success factors included theoretically-grounded implementation strategies and stakeholder engagement in resource-limited settings.

Keywords:         Convergent Parallel, Health Literacy, Mixed Methods, Nutrition, Oral Health, Thailand, Village Health Volunteer

Practice Highlights

• Nutrition literacy among VHVs significantly improved across all six key domains.
• Oral health check-up rates increased from 41% to 2.61% post-programme implementation.
• Functional teeth retention rose from 36% to 92.72% over the 12-month period.
• VHVs used context-specific strategies for community-based nutrition education.

I. INTRODUCTION

Oral health is fundamentally linked to nutrition and dietary behaviours, yet nutritional factors affecting oral health remain a significant public health challenge worldwide, particularly in low- and middle-income countries (Peres et al., 2019; Watt et al., 2019). In Thailand, the high prevalence of dental caries and periodontal diseases related to dietary habits (Chaianant et al., 2022), underscores the urgent need for effective nutrition education strategies for oral health promotion.

Understanding the relationship between nutrition literacy and oral health behaviours requires consideration of multiple theoretical perspectives. Social Cognitive Theory (Bandura, 2004) highlights how personal factors, dietary patterns, and environments interact to shape oral health behaviours, particularly relevant in Thailand’s family-based eating culture. The nutrition literacy skills Framework (Squiers et al., 2012) outlines how individuals develop and apply nutrition literacy competencies through interactions between dietary knowledge and social environments. Additionally, Ecological Systems Theory (Bronfenbrenner, 1979) demonstrates how family and societal systems influence health behaviours and programme implementation.

Within this theoretical context, nutrition literacy for oral health emerges as a critical determinant of oral health outcomes. While health literacy encompasses capacities for accessing and using health information (Sørensen et al., 2012), nutrition literacy for oral health specifically focuses on these competencies in oral healthcare. Evidence consistently shows that individuals with low nutrition literacy tend to exhibit poor oral health behaviours and outcomes (Berkman et al., 2011; Kickbusch et al., 2013). This relationship is particularly significant in reducing oral health disparities (Horowitz & Kleinman, 2012), with higher nutrition literacy correlating with improved oral hygiene practices and health outcomes (Baskaradoss, 2018).

Recent advances in nutrition literacy programmes for oral health promotion have revealed that culturally tailored, context-specific interventions can significantly enhance service accessibility and oral healthcare engagement (Macek et al., 2016). Various programme modalities have emerged, encompassing educational initiatives, community-based activities, and digital media interventions (Dickson-Swift et al., 2014). These approaches align well with Thailand’s dental public health policy, which emphasises proactive oral health promotion and community participation.  Systematic review (Firmino et al., 2017) identified several critical gaps in existing research: the absence of mixed-methods studies examining both programme effectiveness and change processes, limited analysis of community-level behavioural change mechanisms, and insufficient research in resource-constrained developing countries where success factors may differ substantially from developed nations.

To address these research gaps, this study aims to evaluate the effectiveness of a nutrition literacy programme for oral health promotion in Thailand’s context. Of particular interest is the role of VHVs as key implementation agents, given their established position in community health promotion (Kowitt et al., 2015). While previous research has demonstrated VHVs’ capacity to utilise technology for expanding health service coverage (Jandee et al., 2015), empirical evidence regarding their role in promoting nutrition literacy for oral health remains limited.

Guided by our theoretical framework, we employed a Convergent Parallel Mixed Methods design (Creswell & Plano Clark, 2017), enabling comprehensive assessment of both quantitative programme effectiveness and qualitative change mechanisms. This approach examines how social modelling, nutrition literacy skill development related to oral health, and environmental factors interact to influence programme outcomes. Ultimately, this study’s findings will contribute to developing contextually appropriate nutrition literacy strategies for oral health promotion in developing countries while aligning with Thailand’s dental public health policies.

II. METHODS

A. Study Design

This study employed a convergent parallel mixed methods design (Creswell & Plano Clark, 2017) to comprehensively evaluate the implementation and effectiveness of a nutrition literacy programme for oral health promotion. The design integrated quantitative outcomes with qualitative insights to achieve deeper understanding than single-method approaches. The quantitative component utilised a cross-sectional survey to assess nutrition literacy skills and clinical outcomes, while the qualitative component employed a multi-case study approach (Yin, 2018) to explore implementation experiences and contextual factors.

B. Population and Sampling

The quantitative phase included all VHVs who completed nutrition literacy training (N=60) in January 2024, with data collection occurring in July 2024. For the qualitative component, 20 VHVs were purposively selected using intensity sampling (Miles et al., 2013) based on four criteria: programme implementation experience exceeding six months, strong communication abilities, representation from varied performance areas, and voluntary informed consent. This sample size achieved theoretical saturation (Creswell, 2013; Guest et al., 2006). Gender distribution differed between samples (quantitative: 98.3% female; qualitative: 70% female) due to purposive sampling for diverse leadership perspectives. Sensitivity analysis confirmed no significant gender-based differences in primary outcomes (p > .05). The six-month assessment period aligned with established behaviour change evaluation timeframes (Glasgow et al., 2019), while monitoring through December 2024 captured seasonal variations and sustainability data.

C. Research Instruments

Two complementary instruments were developed and validated through pilot testing with 30 VHVs sharing similar characteristics with the target population, but excluded from the final sample. The questionnaire was designed according to Nutbeam’s health literacy framework (Nutbeam, 2000), operationalizing three literacy levels into six nutrition literacy components relevant to oral health promotion. Items utilised a five-point Likert scale (1 = “not confident at all” to 5 = “very confident”) for self-assessment of perceived competencies. A panel of five experts including community dentistry, nutrition, public health, health literacy, and health communication specialists assessed content validity, achieving a high IOC index of 0.96, while internal consistency demonstrated excellent reliability (Cronbach’s alpha = 0.929).

The structured interview guide explored knowledge application, teaching methods, implementation challenges, outcomes, and recommendations following established qualitative research principles (Jacob & Furgerson, 2012). Qualitative trustworthiness was ensured through member checking at two stages: during interviews for immediate verification and after preliminary analysis with eight selected participants for validation and refinement.

D. Data Collection

Baseline data was collected prior to programme implementation in January 2024, establishing pre-intervention metrics through public health service records. Following six-month implementation, parallel quantitative and qualitative assessments were conducted in July 2024. Self-assessment questionnaires were administered to all VHVs, followed by in-depth interviews (45-60 minutes) with 20 purposively selected participants until data saturation was achieved (Guest et al., 2006). In accordance with Consolidated Criteria for Reporting Qualitative Studies (COREQ) guidelines (Zachariah et al., 2024), participant confidentiality was maintained throughout the study, with written informed consent obtained after comprehensive briefing on study objectives and participant rights. Monthly data extraction from the Health Data Centre continued through December 2024 to capture sustained programme effects, with systematic collection on the 5^th of each month ensuring complete and timely data acquisition.

E. Data Analysis

The analytical approach integrated multiple complementary methods for comprehensive understanding. Quantitative analysis included descriptive statistics (frequencies, percentages, means, standard deviations) with Shapiro-Wilk normality testing. Inferential analyses comprised paired t-tests for pre-post comparisons (α = 0.05), chi-square tests for categorical outcomes, and Pearson’s correlation coefficients examining relationships between nutrition literacy domains. Effect sizes were reported using Cohen’s d with bootstrap confidence intervals (1,000 resamples). Statistical analyses utilised IBM’s Statistical Package for the Social Sciences (SPSS) Statistics software. Missing data patterns were examined using Little’s Missing Completely at Random (MCAR) test, with multiple imputation (5 datasets) addressing missing values following Rubin’s guidelines (2004). Sensitivity analyses compared complete-case and imputed datasets (van Buuren, 2018).

Qualitative data underwent thematic analysis following established frameworks (Braun & Clarke, 2006), involving verbatim transcription, independent coding by two researchers, and iterative thematic framework development through consensus meetings. ATLAS.ti software facilitated systematic organisation and analysis. Quality assurance included investigator triangulation, member checking with eight participants, audit trail documentation, and researcher reflexivity journals.

F. Data Integration

A comprehensive integration strategy synthesised quantitative and qualitative findings through three interconnected phases (Cano & Lomibao, 2023). Joint displays facilitated systematic comparison of results, enabling identification of convergent and divergent patterns. Meta-inferences were constructed through iterative cross-method analysis, with attention to complementary insights. Pattern matching techniques examined alignments between quantitative outcomes and qualitative themes, developing integrated theoretical understandings. Conflicting findings were reconciled by contextualising quantitative results with qualitative explanations, while complementary data enriched overall interpretation, enhancing study rigor and validity.

III. RESULTS

All participants (N=60) completed quantitative assessments at baseline and a 6-month follow-up, with 20 VHVs participating in qualitative interviews. Clinical outcomes were monitored through December 2024 using complete Health Data Centre monthly data. Following convergent parallel design, quantitative and qualitative data streams were systematically merged to achieve comprehensive understanding of programme implementation and outcomes. The integrated analysis revealed that communication skills improvements were explained through qualitative evidence of adaptive teaching strategies, while regional outcome variations were illuminated by implementation challenges identified through qualitative inquiry. This systematic data merging approach provided richer insights than either quantitative or qualitative methods could offer independently.

A. Baseline Characteristics

1. Qualitative Sample (n = 20)

The qualitative sample achieved a full response rate (100%). Participants were predominantly female (70%), with males comprising 30%. The age distribution showed that 70% were between 50–60 years, while 15% each were aged 30–39 and 40–49 years. No participants were over 60. In terms of role, 65% served as Village Health Volunteers (VHVs), and 35% were Caregivers. None held dual roles.

2. Quantitative Sample (n = 60)

The quantitative sample also achieved a 100% response rate. Females constituted the vast majority (98.3%), with only one male respondent (1.7%). Most participants (70%) were aged 50–60 years, with smaller proportions aged 30–39 (11.7%), 40–49 (16.7%), and over 60 (1.7%). Regarding position, 85% were VHVs, 13.3% were Caregivers, and 1.7% held both roles.

The intervention (Table 1) demonstrated significant improvements across all six nutrition literacy domains (p< 0.001) with large effect sizes. Communication skills showed the greatest improvement (d = 1.64, mean difference: 0.84 points, 95% CI: 0.66-1.02), followed by Decision Making (d = 0.90), Critical Inquiry (d = 0.88), Understanding (d = 0.85), Application (d = 0.77), and Access (d = 0.74). Other domains improved by 0.36-0.41 points.

C. Clinical Outcomes and Programme Effectiveness

Clinical outcomes significantly improved. Dental check-up rates increased from 1.41% to 2.61% (difference: 1.20 percentage points, 95% CI: 0.90-1.50, p=0.032). Participants with ≥20 functional teeth rose from 87.36% to 92.72% (difference: 5.36 percentage points, 95% CI: 3.38-7.34, p< 0.001), indicating substantial improvements in both knowledge and oral health behaviour.

Submitted: 28 October 2024
Accepted: 16 June 2025
Published online: 7 October, TAPS 2025, 10(4), 35-43
https://doi.org/10.29060/TAPS.2025-10-4/OA3559

Reshma Mohamed Ansari^1,2, Chan Choong Foong³, Hidayah Mohd Fadzil⁴ & Mohamad Nabil Mohd Noor³

¹Institute for Advanced Studies, Universiti Malaya, Malaysia; ²Department of Medical Education, International Medical School, Management and Science University, Malaysia; ³Medical Education and Research Development Unit (MERDU), Faculty of Medicine, Universiti Malaya, Malaysia; ⁴Department of Mathematics and Science Education, Faculty of Education, Universiti Malaya, Malaysia

Abstract

Introduction: Self-care agency, a core concept that helps alleviate the stressors of medical training, is postulated to be practiced by medical students who exhibit a growth mindset. Hence, this pilot study was designed to measure, compare, and correlate the self-care agency and mindsets of undergraduate medical students to assess the potential for scaling to a national survey.

Methods: This cross-sectional study was conducted at one public and one private medical university using a revised version of the Appraisal of Self-Care Agency Scale to measure self-care agency and the Implicit Theories of Intelligence Scale to measure participants’ mindset. Data was analysed using IBM SPSS.

Results: In total, 329 complete responses were obtained. Among the self-care domains, a higher capacity for self-care and a developing capacity for self-care, with a lower ability to indulge in self-care, were reported. Self-care agency showed a significant difference between pre-clinical and clinical students (p = .027; Cohen’s d = .2). Mindset measurements revealed a higher growth than a fixed mindset. The Pearson correlation coefficient showed a weak positive correlation, (r = 0.19) between the means of self-care agency and a growth mindset. Clinical students showed a higher tendency toward self-care than their pre-clinical counterparts in an independent-samples t-test, with no differences between genders and universities.

Conclusion: This pilot study found a positive correlation between self-care agency and growth mindset among undergraduate medical students. Although limited by two prototype universities and response bias, this study provides a solid foundation for future nationwide or cross-country studies.

Keywords:            Self-care Agency, Growth Mindset, Fixed Mindset, Medical Education, Undergraduate Medical Students, Pilot Study

Practice Highlights

• Undergraduate medical students acknowledge importance of self-care agency and growth mindset.
• Growth mindset correlates with self-care agency necessitating inculcation of both.
• Growth mindset combats the stressors of medical training by indulging in proactive self-care.

I. INTRODUCTION

Orem et al. (1995, p. 19) defined self-care as the “practice of activities that individuals start and perform for their benefit, for the maintenance of life, health, and well-being”. Self-care agency, a component of the self-care deficit nursing theory, is “the acquired, complex capacity to meet the requirements to take care of oneself, regulating life processes, maintaining or promoting integrity, structure, and functioning, as well as one’s development and promotion of well-being” (Oliveira et al., 2022, p. 20; Orem et al., 1995). Research has elucidated that the stressors of medical training, which can cause high rates of burnout, anxiety, depression and poor physical health in students (Ayala et al., 2017; Bostock et al., 2018), could be alleviated through self-care (Ayala et al., 2018).

However, during medical training, students find little to no time to engage in self-care; factors such as increased pressure, poor time management, and a negative environment often acting as barriers (Ayala et al., 2017; Ayala et al., 2018). Guldner et al. (2020) suggested that mindset is a predictor of depression and burnout among medical residents, highlighting the relationship between mindset and poor psychological well-being, alleviated by self-care practices (Carter et al., 2025).

Carol Dweck coined the terms “growth mindset” and “fixed mindset” as part of the implicit theories of intelligence, in which individuals could view intelligence as fixed (entity theory) or as a dimension that can be improved with effort (incremental theory) (Cook et al., 2018; Dweck, 2013). The mindset theory, when applied to the medical education context, suggests that a growth mindset views effort as a means to develop abilities (Theard et al., 2021). In contrast, a fixed mindset could lead to poorer outcomes in a high-pressure educational environment (Bostock et al., 2018) and resultant poor psychological well-being (Root Kustritz, 2017).

Studies have measured participants’ mindsets and correlated them with scales measuring anxiety or overall well-being (Guldner et al., 2020; Root Kustritz, 2017; Wolcott et al., 2021). One of the reasons for a growth mindset to aid students’ well-being is by allowing them to place greater value on health and fitness and exhibit health-seeking behaviours (Orvidas et al., 2018; Thomas et al., 2019).

Adding on, identification of stressors, the burnout caused by them, mental health issues, active self-care seeking behaviours including coping strategies are associated with a growth mindset (Burnette et al., 2020). Nursing students who were exposed to structured activities fostering a growth mindset, reported indulging in student-preferred strategies such as viewing obstacles from a newer perspective, working for competency, and indulging in self-care practices, thus directly relating a growth mindset to better self-care agency (Carter et al., 2025).

Medical students in Malaysia, akin to their global counterparts have reported increased levels of stress and psychological distress (Masilamani et al., 2020), due to stringent admission processes, higher cognitive load, and an assessment-based curriculum (Yusoff et al., 2013). Studies across four public medical schools in Malaysia have shown that students tend to alleviate stress through religious activities, active coping strategies, positive reinterpretation, acceptance, and planning (Yusoff et al., 2011), which could be viewed as a subtle expression of a growth mindset. Despite the postulation that a growth mindset accentuates self-care agency, there is a dearth of studies on the self-care agency of Malaysian medical students and its correlation with mindset.

Hence, this pilot study was conducted with the objective of measuring, comparing, and correlating self-care agency and the mindsets of Malaysian undergraduate medical students to gauge the feasibility of a nationwide survey (Lowe, 2019).

II. METHODS

This cross-sectional pilot study was conducted at a public and a private medical university in Malaysia. which were accessible to researchers. The selected public university is a prototype of Malaysian public universities characterised by highly qualified students with excellent examination results paying subsidised fees, as the operation of the university is funded by the government (Wan, 2007). The selected private university is representative of Malaysian private universities, which primarily provide an alternate pathway for the high school students who are not placed in public universities. Being self-funded, the fees are relatively higher, and the students are typically of paying capacity (Wan, 2007).

Permission to conduct this study was obtained from the ethics committees of both the public and private universities (UM.TNC2/UMREC_2872 and RMC/SEP/2023/EC02, respectively). Generally, although every medical school in Malaysia is homegrown and has variations in its curriculum, both the universities chosen for this study had a similar integrated curriculum spanning over five years for the Bachelor of Medicine, Bachelor of Surgery (MBBS) program, with two years of pre-clinical and three years of clinical studies.

The study instrument consisted of three sections. Data collection was anonymous, and the first section recorded the details of the participants’ age, gender, institution of study, and year of study. Section two included the revised version of the Appraisal of Self-care Agency Scale (ASAS-R), used to measure self-care agency (Oliveira et al., 2022). The ASAS is a 24-item scale developed by Evers et al. in 1986 (Evers, 1989) and modified by Sousa et al. (2010) into a 15-item scale that can be applied to adults aged over 18 years.

The three domains of ASAS-R holistically measure the three types of personal trait components that characterise the concept of self-care agency (Oliveira et al., 2022). The responses were ranked on a 5-point Likert scale (1 = totally disagree; 5 = totally agree). The total scores ranged from 15 to 75, with higher scores indicating greater self-care agency (Oliveira et al., 2022).

Sousa et al. (2010) divided the items into the following domains: Domain I: having the capacity for self-care (items 1, 2, 3, 5, 6, and 10; maximum domain score = 30); Domain II: developing the capacity to self-care (items 7, 8, 9, 12, and 13; maximum domain score = 25); and Domain III: inability to self-care (items 4, 11, 14, and 15; maximum domain score = 20), with acceptable Cronbach’s alpha values of 0.86, 0.83, and 0.79, respectively. The instrument has been validated among the general population in countries such as China, Spain, and Brazil, and among medical students in Portugal (Alhambra-Borrás et al., 2017; Guo et al., 2017; Oliveira et al., 2022; Yuan et al., 2021).  

Section three consisted of the Implicit Theories of Intelligence Scale (ITIS), adopted from the published version of the ITIS by Cook et al. (2017), to measure the mindset of medical students. The ITIS is an eight-item instrument with four items related to incremental beliefs (1, 2, 3, and 4) and four items related to entity beliefs (5, 6, 7, and 8) regarding intelligence and ability. Responses were based on a 6-point Likert scale (1 = strongly disagree; 6 = strongly agree) (Cook et al., 2017). For each domain, the scores ranged from 4 to 24.

A confirmatory factor analysis of the ITIS scores demonstrated an overall acceptable model fit; an exploratory factor analysis confirmed a two-factor structure (Cook et al., 2017), with the Cronbach’s alpha ≥ 0.77 for each domain (Cook et al., 2018).

The sample size of the study was calculated using Raosoft online software (http://www.raosoft.com/samplesize.html).

With an approximate total population of 1300 (public university = approximately 900; private university = approximately 400), based on a confidence level of 95%, a response distribution of 50%, and a margin of error of 5%, the sample size was determined to be 297. A quota sampling technique was applied to ensure adequate sample representation for each university: 204 from the public university and 93 from the private university.

The questionnaire items were entered into a Google Form, and the link was distributed to all the students of both the universities where they were asked to answer the questionnaire after reading the participant information sheet and providing informed consent.

The data were collected over four weeks, starting on November 1 – November 30, 2023, which was two months from the start of the new semester (September 2023) in both universities which gives ample time for the students to experience the learning environment and respond to the questionnaire. A reminder was provided two weeks after the start of data collection, and data collection ended two weeks after the reminder. The students were informed of the closure of data collection.

The data were analysed using the IBM Statistical Package for Social Sciences (SPSS) v.26 via descriptive and inferential tests. The normality of the data distribution was screened. Pearson’s product-moment correlation was used to analyse the relationship between self-care agency and mindset, whereas independent-samples t-tests were used for comparative analyses between gender and years of study.

III. RESULTS

A. Participants’ Characteristics

A total of 329 valid responses were received. The participants’ demographic characteristics are presented in Table 1.

The distribution of the participants according to their year of study followed the same pattern in both universities, with more female than male respondents and more pre-clinical than clinical students, implying that the sample was representative of the population.

Submitted: 1 January 2025
Accepted: 12 August 2025
Published online: 7 October, TAPS 2025, 10(4), 44-54
https://doi.org/10.29060/TAPS.2025-10-4/OA3613

Dinushee Atapattu-Bakmeewewa¹, Bhagya Devagiri¹, Gayanthi Kodituwakku¹ & Madawa Chandratilake²

¹Department of Disability Studies, Faculty of Medicine, University of Kelaniya, Sri Lanka;  ²Department of Medical Education, Faculty of Medicine, University of Kelaniya, Sri Lanka

Abstract

Introduction: Tele-clinical training is an effective approach, increasingly adopted post-pandemic and in resource-limited settings. However, it requires systematic development. This study details the first-time implementation of a tele-clinical training in an undergraduate Speech and Hearing Sciences programme, exploring student experiences and validating a novel evaluation tool, specifically designed for tele-clinical training programmes.

Methods: The study used a mixed-method approach. Quantitative data were gathered from 128 students using the developed 23-item Kelaniya Tele-Clinical Evaluation Tool (KeTCET), which covers three broad areas: Learning Environment, Supervisory Attributes, and Telehealth Teaching Practices. Qualitative insights from 13 participants were thematically analysed. The tool was validated for reliability and psychometric robustness using expert feedback and statistical evidence.

Results: Quantitative analysis showed high domain scores: Learning Environment 80.64%, Supervisory Attributes 81.67%, and Telehealth Teaching Practices 80.31%. Strong positive correlations between domains (r > 0.86, p < 0.001) indicated interconnectedness. The 23-item evaluation tool demonstrated high internal consistency (Cronbach Alpha = 0.98) and a single-factor structure (Eigenvalue = 17.12, 74.44% variance explained). Qualitative data highlighted strengths in supervisor interaction and resource availability, also noting challenges such as issues in connectivity and limited peer learning. Students appreciated structured feedback and supervisor presence during tele-clinical sessions.

Conclusions: The successful development of a tele-clinical programme requires consideration of multiple elements broadly categorised as pedagogical environment, supervisory characteristics, and virtual teaching practices. Well-structured programmes can effectively meet training needs in resource-limited settings, although strengths and challenges may vary across learning environments. The validated 23-item KeTCET offers a reliable framework for evaluating and improving tele-clinical programmes.

Keywords:           Tele-clinical Programmes, Tele-clinical Supervision, Speech and Language Therapy, Audiology, Tele-clinical Evaluation, Tool Development, Undergraduate Clinical Training, KeTCET

Practice Highlights

• Tele-Practice based clinical programmes can be delivered successfully even in countries with limited resources.
• Robust planning is required to ensure that tele-clinical programmes address specific training needs and satisfy student expectations.
• Incorporating virtual clinical training modules into the regular curriculum can effectively address some of the barriers students have identified in tele-clinical learning.
• The unique features of tele-clinical training should guide the formulation of tailored guidelines and supervisory models for the virtual format.
• The evaluation of tele-clinical programmes is most effective when customised tools are developed to incorporate elements of the virtual format of training.

I. INTRODUCTION

The COVID-19 pandemic necessitated a global shift to online platforms in healthcare and education. Higher education institutions implemented online teaching methods, including webinars and interactive sessions (Hameed et al., 2020), while healthcare education adopted tele-medicine as a platform to continue clinical training, ensuring graduate preparedness for professional practice. Speech and language therapy and audiology professionals swiftly utilised tele-therapy, a method previously endorsed by the American Speech-Language-Hearing Association (ASHA, n.d.). This model of service delivery employs synchronous, asynchronous, and hybrid methods, tailored to patient needs and available resources. Beyond the pandemic, tele-medicine programmes offer enduring benefits, including improved access to care for remote and underserved populations, enhanced patient satisfaction and cost-effectiveness (Car et al., 2020). The World Health Organization emphasises the role of tele-medicine in broadening healthcare access, especially for people in remote areas and underserved communities (WHO, 2022).

Tele-clinical programmes, that is, clinical training conducted using tele-therapy have proven to be an effective method in clinical teaching and skills training. Considerable evidence demonstrates the tele-clinical programmes can be successfully used for clinical training, not limiting it to practical skills (Anderson et. al., 2023) but also addressing attitudinal changes (Wearne et. al., 2015) and clinical soft skills (Bramstedt et. al., 2014; Liu et. al., 2022).

Clinical supervision, distinct from classroom teaching, involves case-based learning, critical thinking, and professional behaviour modelling (Council of Academic Programmes in Communication Sciences and Disorders (CAPCSD), 2013). Traditionally conducted face-to-face clinical supervision shifted to tele-clinical supervision during the pandemic, utilizing a variety of strategies (Shawwa, 2023). However, much of the available literature on tele-supervision does not clearly state whether important aspects of clinical teaching, such as supervisor–student relationships and feedback (Kilminster & Jolly, 2000), were considered during programme design.

Evidence suggests that the effectiveness of tele-supervision depends significantly on the qualities of both the supervisor and the supervisee (Martin et al., 2017). Positive supervisory relationships, characterised by structure and support (Martin et al., 2014) are achievable in both virtual and face-to-face formats (Reese et al., 2009). Effective tele-supervision also depends on communication strategies, supervisor availability, and feedback models (Gibson et al., 2007; Snowdon et. al., 2019). Technological barriers, such as poor connectivity and equipment issues, can hinder outcomes, necessitating proactive solutions (Martin et al., 2017; Reese et al., 2009; Tarlow et al., 2020).

Systematic evaluation of student experiences in tele-clinical programmes is essential to understand their effectiveness. Most studies that report on tele-clinical programmes in allied health sciences (e.g., Bacon et al., 2023; Snowdon et al., 2019) and medicine (e.g,. Pit et al., 2021; Wearne et al., 2015) rely primarily on qualitative interview data. Ideally, such qualitative data should be supplemented with the use of a validated and reliable tools specifically designed to evaluate a virtual model of clinical training. This ensures that tele-clinical learning meets the professional standards and training requirements of audiology and speech and language therapy. The objectives of this study were to evaluate student experiences in an undergraduate Speech and Hearing Sciences tele-clinical training programme, and to validate a feedback tool to evaluate similar tele-clinical programmes.

II. METHODS

A. Context and Setting

The BSc (Hons) Speech and Hearing Sciences is a full-time, four-year undergraduate programme. The qualification allows graduates to practice as speech and language therapists or audiologists following registration at the national medical regulatory body. The programme consists of theoretical classroom-based sessions, synchronous to intensive supervised clinical training offered across the four years of study. Speech and Language Therapy (SLT) undergraduates are trained to work with communication and swallowing disorders while audiology undergraduates train in the detection and management of hearing loss, across the life span.

The data gathered in this study reflects experiences from the pandemic period, when clinic and hospital-based teaching was significantly limited. For almost two and a half years since the commencement of the pandemic, SLT and audiology service provision shifted fully into a tele-therapy programme. Simultaneously, students were enrolled in a tele-clinical programme, offered two to three times per week, where they worked with the patient under the supervision of an academic or qualified clinician. All sessions took place using the Zoom platform, which was made freely available to students by the university. Tele-therapy for adult patients requiring SLT services was delivered primarily using a synchronous method and for paediatric clients using an asynchronous or a mixed method. Audiology clinical services were primarily synchronous. The tele-clinical training was designed to align with the method of tele-therapy (Table 1).

Format	Synchronous Method	Asynchronous Method	Hybrid Method
Tele-therapy	Conducted in real time using audio or video interactive sessions.	Clinical management through stored images and captured data.	Combines both synchronous and asynchronous methods.
Tele-clinical Supervision	Students conduct session. Supervisor joins in. Feedback is provided simultaneously and after session.	Students join session. Supervisor shares recorded videos and relevant clinical information/documentation. Students are given time to reflect.	Students conduct session. Supervisor joins in. Following a short real-time session, a recorded video is watched together. A discussion follows.

Table 1. Modes of Tele-therapy and tele-supervision delivery

The general arrangement of a synchronous session was that the patient, student and supervisor joined the session at a mutually agreed time, but from three separate locations. Material for the therapy session, if required, was developed and shared on the screen by the student. For paediatric clients, parents arranged the toys needed. In audiology, students gathered patient data through interviews and questionnaires with limited use of conventional hearing tests. Auditory verbal training  (AVT sessions) in audiology followed a similar format to SLT synchronous sessions. The supervisor remained a silent observer unless intervention was required. In all sessions, supervision concluded with an interactive patient discussion, facilitated by the supervisor using Zoom features such as whiteboard, break out rooms etc.

The evaluation of the tele-clinic programme was conducted using mixed methods by collecting students’ perceptions quantitatively (Phase I) and qualitatively (Phase II).

B. Phase 1- Development of Evaluation Tool and Quantitative Feedback

The quantitative evaluation of the tele-clinical programme was conducted with the aim of developing and validating a standardised tool for evaluating similar programmes.

1. Tool Development

As there are no existing tools available to evaluate SLT or audiology clinical programmes, whether face-to-face or virtual, a new evaluation tool was developed based on the Clinical Learning Environment, Supervision and Nurse Teacher evaluation scale (CLES +T ) (Mikkonen et al., 2017) and the Nursing Clinical Facilitator Questionnaire (NCFQ) (Espeland & Indrehus, 2003). The resulting 23-item tool, named the Kelaniya Tele-Clinical Evaluation Tool (KeTCET), was designed to map onto three primary domains: pedagogical/learning environment (LE; 9 items), supervisory relationship (SA; 6 items), and telehealth teaching practices (TTP; 8 items). The stem question used here was, how often did you experience this aspect in the online clinical learning sessions provided for the SHS programme during university closure? (Table 2). Participants rated each item on a 5-point Likert scale (0 – never, 1 – rarely, 2 – sometimes, 3 – often and 4 – always).

To enhance face and content validity, a panel of 10 experts in speech and language therapy and/or audiology  rated the tool items on a five-point scale for [a] content appropriateness, [b] relevance, and [c] technical accuracy (1 = Very Low, 5 = Very High). Experts could also provide comments to refine the items. The panel scored the items high across all three aspects (mean[a]=4.8; mean[b]= 4.8; mean[c]=4.7). A measure of item relevance, I-CVI (Item- Content validity Index) scores for all items (n=23) were > 0.9. Minor language adjustments suggested were incorporated. The tool was then translated into Sinhala and Tamil and pre-tested with five students (three Sinhala speakers and two Tamil speakers) to confirm clarity and translation accuracy.

2. Study Participants

All 155 SLT undergraduate students in the SLT and audiology programmes who had attended at least 80% of the tele-clinical training sessions were considered eligible for participation in phase I. All eligible students were invited to participate in the study. At the time of data collection, these students were in their second, third, and fourth years of study. The minimum sample size required was calculated based on the recommended item-to-response ratio of 1:5 for factor analysis (Bujang et al., 2012; Gorsuch, 1983), requiring at least 115 responses. A total of 128 students responded (82.6%) to phase I.

Submitted: 10 March 2025
Accepted: 5 July 2025
Published online: 7 October, TAPS 2025, 10(4), 55-62
https://doi.org/10.29060/TAPS.2025-10-4/OA3690

Aaron Tigor Sihombing^1,2, Antonia Kartika^3,4 & Anglita Yantisetiasti^2,5

¹Department of Surgery, Division of Urology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia; ²Hasan Sadikin General Hospital, Bandung, Indonesia; ³National Eye Center-Cicendo Eye Hospital, Bandung, Indonesia; ⁴Department of Ophthalmology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia; ⁵Department of Anatomical Pathology, Faculty of Medicine, Universitas Padjadjaran, Bandung, Indonesia

Abstract

Introduction: Music preferences have been linked to personality traits, which in turn may influence career choices. This study explores the potential relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University, Indonesia.

Methods: A cross-sectional survey was conducted among all residents trained in urology, ophthalmology, and anatomic pathology. Data collected included demographic characteristics, music genre preferences, and work habits related to music. Independent t-tests are used when data are normally distributed, whereas Kruskal-Wallis tests are used when data are not normally distributed.

Results: The study included 125 residents (19 pathology anatomy, 33 urology, 73 ophthalmology). Pop was the most preferred music genre across all specialties (48% in urology, 61% in ophthalmology, 52% in pathology anatomy). However, secondary preferences varied: rock (21%) was the second most popular among urology residents, jazz (10.9%) and indie (10.9%) among ophthalmology residents, and classical music (26%) among pathology anatomy residents. Demographic differences were noted, with pathology anatomy residents being older and urology residents having a higher proportion of males. Ethnic distribution was relatively consistent across specialties, primarily mixed ethnicity, Sundanese, and Javanese.

Conclusion: While pop music was the predominant preference across all specialties, secondary music preferences varied, potentially reflecting different personality traits associated with each specialty. The study’s findings are limited by its single-institution sample and cross-sectional design, necessitating further research with larger, more diverse populations to explore the underlying mechanisms linking music preferences to medical specialisation choices.

Keywords:           Music Preferences, Medical Specialty Selection, Personality Traits

Practice Highlights

• Personality traits could predict specialty preferences among medical students.
• Music genre preferences are associated with personality traits, thus are drawn to particular type of music.
• Residents in different specialties distributed music genres differently.

I. INTRODUCTION

The notion that music genres reflect and influence personality traits is a topic of much debate and interest. Research has shown a correlation between musical preferences and personality traits, with certain genres being associated with specific characteristics (Andrews et al., 2022; Wang et al., 2024). For example, music in slow tempo and music in minor keys were significantly predicted by emotional stability and optimism, whereas music in fast tempo and music in major keys was significantly predicted by openness to experiences, introversion, and gender (Dobrota & Reić, 2014; Upadhyay et al., 2017). Some evidence suggests that individuals drawn to certain music genres may exhibit personality profiles that align with specific career paths. For instance, individuals who prefer classical music tend to score higher in openness and introversion—traits associated with analytical or solitary professions—while those who enjoy rock or pop may display extraversion and sensation-seeking behaviors, often linked to high-energy or interactive professions (Rentfrow & Gosling, 2003; Schäfer & Mehlhorn, 2017). Moreover, individuals with high levels of achievement tend to prefer music that reflects their professional identity, indicating a potential association between occupational roles and musical preferences (Knox & McDonald, 2017).

Similarly, in the field of medicine, there is a growing interest in understanding how personality traits may influence medical residency preferences. Studies suggest that certain personality types may be drawn to specific medical specialties, and this alignment could impact their satisfaction and performance within that field. This intersection of music, personality, and medical career choices highlights the complex ways in which our preferences and personalities shape and reflect our identities and professional paths.

Culture and environment have a profound effect on a person’s personality (Smaldino et al., 2019; Triandis & Suh, 2002). Culture provides a framework within which individuals develop beliefs, values, and behaviours that are in harmony with their societal norms. It shapes personality through language, customs, and social norms, which guide an individual’s actions and interactions. Environment, encompassing family dynamics, social relationships, and educational experiences, also plays a pivotal role in molding personality traits. As individuals navigate through different environmental contexts, their personalities adapt and evolve, reflecting the dynamic interplay between their innate dispositions and external influences. This complex interconnection suggests that personality is not a static entity but a fluid construct that changes over time, influenced by the cultural and environmental landscapes we inhabit.

Indonesia’s rich of cultures is indeed a testament to its diversity, with over 300 ethnic groups calling it home. This multicultural environment offers a unique blend of traditions, languages, and beliefs, which can significantly shape an individual’s personality. Exposure to diverse cultural practices and values can foster open-mindedness, adaptability, and a complex worldview. Research suggests that multicultural experiences can enhance creativity, reduce stereotypes, and provide a broader perspective on life. Moreover, individuals who navigate multiple cultures may develop a multicultural identity, integrating different cultural influences into their personal identity, which can contribute to their overall well-being and social harmony.

The exploration of a potential correlation between musical preferences and medical specialty choice in Indonesia is indeed a fascinating subject. Since music has been noted to affect personality, which in turn influences the choice of medical specialisation, Indonesia is a multicultural country, and this correlation is intriguing. As a result, this study aims to determine if there is a distribution difference between music genre preferences and healthcare specialisations in Indonesia.

II. METHODS

This research employed a cross-sectional study design, with total sampling of all Padjadjaran University resident utilising a comprehensive survey administered to all residents. The study population comprised residents specialising in three distinct medical disciplines: urology, ophthalmology, and anatomic pathology. All residents in urology, ophthalmology, and anatomic pathology were eligible to be included. Exclusion criteria included incomplete responses or refusal to provide consent. This selection allowed for a comparative analysis across specialties with varying degrees of procedural and cognitive demands. In this study we utilised total sampling, all eligible residents during the study period were invited to participate. The survey instrument was designed to collect a range of demographic and preference-based data. Participants were asked to provide information regarding their age, current year of residency training, and their preferred music genres. Crucially, the survey also explored the residents’ work habits related to music, specifically inquiring whether they typically worked with or without background music and their preferred music genre. The survey also collected information on spouse employment status, which was divided into two groups: physicians and non-physicians.

Statistical analysis was performed to assess the difference between groups within the collected data. The independent t-test was employed to analyse continuous data that demonstrated a normal distribution. For continuous data that did not meet the assumptions of normality, the non-parametric Kruskal-Wallis test was utilised. In all statistical analyses, a significance level (p-value) of 0.05 was established as the threshold for statistical significance, indicating a 5% risk of concluding a relationship exists when it does not. All participants have given informed consent before any data were collected.

III. RESULTS

A total of 125 residents participated in this study. Nineteen pathology anatomy residents, 33 urology resident and 73 ophthalmology residents were included. The descriptive statistic of age and sex are presented in Table 1. Marital status and ethnicity are presented in Appendix 1.

		Pathology Anatomy (n = 19)	Urology (n = 33)	Ophthalmology (n = 73)	p-value
Age
	Mean ± SD	33.16 ± 3.11	30.09 ± 2.11	30.51 ± 2.55	0.002*
	Median (Range)	34.00 (29-39)	30.00 (27-37)	30.00 (26-36)
Sex
	Male	4 (21.1%)	26 (79.8%)	22 (30.1%)	<0.01
	Female	15 (78.9%)	7 (21.2%)	51 (69.9%)

*Kruskal Wallis Test

Table 1. Age, sex, ethnicity and marital status of the residents

In male resident population, distribution between married and not married is quite equal between specialty (Figure 1). In urology, married male residents is 58%, while it is 68% and 50% in ophthalmology and pathology, respectively. There is sharp difference in female urology resident compared to other specialties. There are only 14% female urology residents who is married, while in ophthalmology and pathology is 63% and 67%, respectively.

Figure 1. Marital status proportion in male (A) and female (B) residents

Figure 2. Proportion of married residents with physician spouse

In urology, 81% of residents also married to physicians, while it is only 55% in ophthalmology and 42% in pathology (Figure 2).

Most residents in urology (69.70%) and pathology (73.68%) reported that they were listening to musical background while working (Figure 3). While only 38.36% in ophthalmology that worked with musical background.

Figure 3. Comparison between residents working with musical background and those without

Figure 4. Residents’ music genre preference (in percentage)

IV. DISCUSSION

This study explored the relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University. There is a higher median age among pathology anatomy residents than among urology and ophthalmology residents, and there is a greater proportion of males in the urology department (Table 1). In spite of this, all residents from the three departments belong to the same generation (Juekiewicz, 2023). As residents in the same generation (generation Y), they are influenced by similar external influences, values, and ethical principles which influence their music genre preferences (Juekiewicz, 2023; Krumhansl, 2017).

Given that Indonesia is composed of multiple ethnic groups, and ethnicity could influence music genre preference, we found that the top three ethnic groups that reside in the three departments are somewhat similar, namely mixed ethnicity, Sundanese, and Javanese (Table 1). This percentage ranking differs from that of the Badan Pusat Statistik (BPS), which indicates that the top three groups by population are Java (40.22%), Sundanese (15.5%), and Batak (3.58%) (Badan Pusat Statistik [BPS], 2010).

Since music serves as a connection function between people, we evaluate the marital status of the residents (Bamford et al., 2024). In urology, 52% of residents are married, while in pathology and anatomy, 37% and 36% are married, respectively (Table 1). While females comprise only 21% of urology residents, the majority of them are unmarried (86%), which is compared to only 33% and 37% of female pathology anatomy and ophthalmology residents who are unmarried, respectively (Figure 1).

Eighty-one percent of urology residents are married to a physician compared to 55% of ophthalmology residents and 42% of pathology anatomy residents (Figure 2). Study by Dutta RR, et all showed that only 26.1% of physician married with physician (Dutta et al., 2024). However, the study did not compare the percentage of physicians who are married to other physicians in each specialty.

Compared with urology residents (69.7%) and pathology anatomy residents (73.68%), only 38.36% of ophthalmology residents listen to music while working (Figure 3).

Pop genre is the most preferred genre among three groups of residents, comprising 48% of residents in the urology and 61% of residents in the ophthalmology groups, as well as 52% of residents in the pathology anatomy groups (Figure 4). This finding is similar to a study by Krumhansl that the pop genre is the most preferred genre for individuals born between 1940 and 1999 (Krumhansl, 2017). However, the second most preferred genre among urology residents is rock (21%), while jazz (10.9%) and indie (10.9%) are the second most preferred genres in ophthalmology, and classical music (26%) is the second most preferred genre in pathology anatomy. The difference in genre music preference can also be observed in the third to last rank on the list (Figure 4).

The variation in secondary music genre preferences among specialties may be partially explained by psychological theories of personality, particularly the Five-Factor Model (FFM), which posits five broad domains of personality: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Prior research has shown that individuals who prefer classical music tend to score high in openness and introversion—traits often linked to reflective and analytical disciplines such as pathology. In contrast, rock and pop fans tend to exhibit higher levels of extraversion and openness to experience, traits more commonly observed in high-intensity, procedurally driven fields like urology (Rentfrow & Gosling, 2003; Schäfer & Mehlhorn, 2017). However, in our study, most residents prefer pop music.

From a sociological perspective, medical specialties may be seen as “occupational subcultures” (Becker, 1963), each with distinct values, stress profiles, and interaction patterns. These subcultures attract individuals whose identities align with the implicit norms of the field (Light, 1979), potentially explaining the alignment between music preference and specialty. For example, the high-paced, team-intensive nature of urology might attract residents who identify with more energetic and expressive music like rock. Conversely, fields like pathology, which involve solitary analysis, may appeal to those who appreciate structured and introspective genres like classical music.

This study has several limitations that affect generalisability of the findings. While the study used a total sample of residents from three departments, the number of residents from each specialty varies, with 19 respondents from pathology anatomy, 33 respondents from urology, and 73 respondents from ophthalmology. In addition, the study’s sample was drawn exclusively from residents of one university.

Despite its limitations, this study presents several strengths that contribute to its value. The study explores a unique and interesting relationship between music genre preferences and medical specialty selection, particularly within the Indonesian cultural context. This area is relatively understudied and by focusing on medical residents, a specific population facing unique career pressures, the research addresses a relevant and understudied group. Furthermore, we acknowledgment of Indonesia’s diverse cultural influences, highlighting the potential impact of environment on both musical tastes and professional choices. We hope that this study will open avenues for further research and raising awareness about the potential influence of external factors like music on medical professionals.

V. CONCLUSION

This cross-sectional study explored the relationship between music genre preferences and medical specialty selections among residents at Padjadjaran University in Indonesia. While the study revealed variations in music preferences distribution across different specialties, pop being the most favored genre and different preference in second to last rank.

We found that there are differences in demographic distributions, particularly age and gender, among the resident groups, it is important to acknowledge the limitations of the study’s design. The use of a single-institution sample and the cross-sectional methodology limit the generalisability of the findings and preclude the establishment of causal relationships. Nevertheless, the study offers a preliminary investigation into the potential interplay between musical tastes and career choices within the medical field, raising intriguing questions for future research.

Further studies with larger and more diverse samples, are needed to expand upon these findings, and to explore the underlying mechanisms that might link music preferences to medical specialisation.

Notes on Contributors

ATS contributed to the conceptualisation, data curation, methodology, formal analysis, project administration, validation, investigation, funding acquisition, resources, visualisation, software, supervision, writing of the original draft, review and editing.

AK contributed to the methodology, formal analysis, resources, visualisation, software, supervision, writing of the original draft, review and editing.

AY contributed to the investigation, visualisation, software, supervision, writing of the original draft, review and editing.

Ethical Approval

This study was performed under the ethical approval from Hasan Sadikin Hospital Ethical Committee (Approval Number: DP.04.03/D.XXIV.16/14527/2024). This study is in line with the 1964 Declaration of Helsinki and existing ethical standards.

Data Availability

The data supporting this study are available upon reasonable request to Corresponding Author.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Interest

The authors declare no relevant financial or non-financial competing interest from any party.

References

Andrews, C., Gardiner, K., Jain, T. K., Olomi, Y., & North, A. C. (2022). Culture, personal values, personality, uses of music, and musical taste. Psychology of Aesthetics, Creativity, and the Arts, 16(3), 468–486. https://doi.org/10.1037/aca0000318

Badan Pusat Statistik. (2010). Kewarganegaraan, suku bangsa, agama, dan bahasa sehari-hari penduduk Indonesia (Ch. 3, Komposisi penduduk, p. 9). Badan Pusat Statistik. https://www.bps.go.id/id/publication/2012/05/23/55eca38b7fe0830834605b35/kewarganegaraan-suku-bangsa-agama-dan-bahasa-sehari-hari-penduduk-indonesia.html

Bamford, J. S., Vigl, J., Hämäläinen, M., & Saarikallio, S. H. (2024). Love songs and serenades: A theoretical review of music and romantic relationships. Frontiers in Psychology, 15, 1302548. https://doi.org/10.3389/fpsyg.2024.1302548

Becker, H. S. (1963). Outsiders: Studies in the sociology of deviance. Free Press.

Dobrota, S., & Reić, E. I. (2014). The relationship between music preferences of different mode and tempo and personality traits – Implications for music pedagogy. Music Education Research, 17(2), 234–247. https://doi.org/10.1080/14613808.2014.933790

Dutta, R. R., Wu, A. T., Picton, B., Shah, S., Chernyak, M., Bauer, K., Solomon, S., Chang, J., Nguyen, B., Jiang, M., & Hurria, A. (2024). Physician marriage survey reveals sex and specialty differences in marital satisfaction factors. Scientific Reports, 14(1), 5159. https://doi.org/10.1038/s41598-024-55437-3

Jurkiewicz, C. L. (2023). Age diversity and generational ethics. In L. C. Becker & C. B. Becker (Eds.), Encyclopedia of ethics (pp. 54–62). Springer. https://doi.org/10.1007/978-3-030-22767-8

Knox, D., & MacDonald, R. (2016). Broadcasting personalities: The relationship between occupation and music preferences in the BBC radio programme Desert Island Discs. Psychology of Music, 45(5), 645–664. https://doi.org/10.1177/0305735616670497

Krumhansl, C. L. (2017). Listening niches across a century of popular music. Frontiers in Psychology, 8, 431. https://doi.org/10.3389/fpsyg.2017.00431

Light, D. W. (1979). Becoming psychiatrists: The professional transformation of self. W. W. Norton & Company.

Rentfrow, P. J., & Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology, 84(6), 1236–1256. https://doi.org/10.1037/0022-3514.84.6.1236

Schäfer, T., & Mehlhorn, C. (2017). Can personality traits predict musical style preferences? A meta-analysis. Personality and Individual Differences, 116, 265–273. https://doi.org/10.1016/j.paid.2017.04.061

Smaldino, P. E., Lukaszewski, A., von Rueden, C., & Gurven, M. (2019). Niche diversity can explain cross-cultural differences in personality structure. Nature Human Behaviour, 3, 1276–1283. https://doi.org/10.1038/s41562-019-0730-3

Triandis, H. C., & Suh, E. M. (2002). Cultural influences on personality. Annual Review of Psychology, 53, 133–160. https://doi.org/10.1146/annurev.psych.53.100901.135200

Upadhyay, D. K., Shukla, R., Tripathi, V. N., & Agrawal, M. (2017). Exploring the nature of music engagement and its relation to personality among young adults. International Journal of Adolescence and Youth, 22(4), 484–496. https://doi.org/10.1080/02673843.2016.1245150

Wang, Y., Qu, W., & Ge, Y. (2024). Relationship between personality and music preference: Psychometric adaptation of the music preference scales in a Chinese sample. Musicae Scientiae, 29(1), 109-130. https://doi.org/10.1177/10298649241287072

*Aaron Tigor Sihombing
Jl. Raya Jatinangor, Cikeruh, Kec. Jatinangor,
Kabupaten Sumedang, Jawa Barat 45363
+62 813-2132-9126
Email: aarontigor@gmail.com

Submitted: 6 November 2024
Accepted: 11 April 2025
Published online: 1 July, TAPS 2025, 10(3), 58-64
https://doi.org/10.29060/TAPS.2025-10-3/OA3567

Nathania Nishida Tan¹, Daniel Ardian Soeselo^2,3, Natalia Puspadewi², V. Dwi Jani Juliawati² & Gisella Anastasia²

¹Faculty of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Indonesia;  ²Medical Education Unit and ³Department of Surgery, Faculty of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, Indonesia

Abstract

Introduction: Career selection is a critical decision influenced by various factors, including media exposure. As a popular genre among adolescents, medical dramas may shape perceptions of the medical profession. This study explores the impact of medical dramas on medical students’ motivation to pursue a career in medicine.

Methods: This study was conducted at the School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, between February and March 2024, using a mixed-method approach. Quantitative data were gathered through a questionnaire, followed by qualitative insights from in-depth interviews. A two-stage sampling strategy was employed, initially identifying first-year students influenced by medical dramas through screening, followed by purposive sampling to select participants for in-depth interviews.

Results: Analysis identified six key motivational aspects through which medical dramas influence students’ decision to pursue medicine. These include the detailed portrayal of doctors’ roles, their daily lives, and the reinforcement of a positive perception of the profession. Notably, emotional engagement- such as inspiring doctor characters- and the realistic depiction of challenges, including the dedication and hardships of becoming a doctor, emerged as significant factors.

Conclusion: Medical dramas play a significant role in shaping students’ motivation and perception of the medical profession. This study highlights six key motivational aspects influencing career decisions, offering new insights into media-driven career choices and their potential implications for medical education.

Keywords:           Medical Dramas, Medical Education, Medical Students, Student Motivation, Career Choice

Practice Highlights

• Medical dramas serve as informal educational tools through entertainment for the public.
• Television influences adolescents’ career aspirations by shaping their interests.
• Medical dramas shape public perceptions of the medical profession and may inspire viewers.

I. INTRODUCTION

Medical dramas have remained a widely popular television genre since their emergence in the 1960s, continuously evolving to portray healthcare professionals’ professional and personal lives within hospital settings (14^th Media Mutations International Conference, 2023; Sonego & Rocchi, 2024).

Communication theorist Marshall McLuhan had predicted the success of medical dramas as early as 1964 in his book Understanding Media: The Extensions of Man, attributing their appeal to the immersive experience viewers have when witnessing medical procedures, which fosters an “obsession with bodily well-being” (Aboud, 2012; Levine & McLuhan, 1964).

Beyond entertainment, medical dramas serve as informal educational tools illustrating medical practices, bioethical dilemmas, and professional conduct (Cambra-Badii et al., 2021; Zago et al., 2024). Research suggests that these dramas reinforce key principles of medical ethics and professionalism, including communication skills, patient confidentiality, and empathetic bedside manner (Aboud, 2012). By portraying both the challenges and rewards of medical practice, these narratives may also shape viewers’ perceptions of the profession, inspiring people to pursue careers in healthcare (Alahmari, 2023).

Research indicates a correlation between specific television programs and career aspirations, suggesting that the content adolescents consume can shape vocational interests and goals (Gehrau et al., 2016). As an accessible informal educational resource, television provides insight into professional responsibilities and challenges (Mendick & Williams, 2022). Within this context, medical dramas may play a significant role in shaping students’ motivation to enter the medical field. Unlike prior studies that broadly examine media influence over career perceptions, this study focuses on first-year medical students, analysing their motivational drive. By integrating quantitative and qualitative data, this study provides deeper insight into how medical dramas influence medical students’ motivation to pursue a career in medicine.  

II. METHODS

This study was conducted at the School of Medicine and Health Sciences, Atma Jaya Catholic University of Indonesia, from February to March 2024, using a mixed-method approach. A two-stage sampling method was employed, beginning with a questionnaire to identify eligible participants, followed by purposive sampling to select individuals for in-depth interviews. The target population consisted of first-year medical students enrolled in the 2023 academic year who acknowledged that medical dramas had influenced their motivation to pursue a medical degree.

A. Participant Selection

Out of 210 first-year students, 114 reported watching medical dramas. These students completed a demographic questionnaire, which gathered information on gender, parents’ profession, and their viewing habits related to medical dramas. Additionally, they completed the Strength of Motivation for Medical School Revised (SMMS-R) Questionnaire, a validated self-report instrument consisting of 15 items rated on a 5-point Likert scale (ranging from “strongly disagree” to “strongly agree”), with possible scores ranging from 15 to 75, indicating the strength of motivation for medical school.

B. Qualitative Phase Sampling

Purposive sampling was used to ensure an accurate representation of the study population based on key demographic factors, specifically the profession of participants’ parents. The selection process identified respondents from diverse parental backgrounds in the medical field: those with both parents as physicians, those with one parent in medicine, and those with no parental medical background. To ensure gender diversity, one male and one female were selected from each category, all of whom had high motivation scores based on their SMMS-R results. Six participants were selected to achieve thematic saturation, as preliminary analysis showed no new themes emerging beyond this point. Although not statistically representative, this sample captures diverse perspectives on medical career motivation based on different childhood backgrounds.

C. Data Collection

The in-depth interviews, conducted by the researcher (NNT), explored participants’ perceptions of how medical dramas influenced their motivation to pursue a career in medicine. The interview guide addressed various aspects of this influence on their decision-making process. One key question asked was “How have medical dramas influenced your decision to pursue medicine?” All participants provided written informed consent, including permission for audio recording. The interviews were then transcribed verbatim for further analysis.

D. Data Analysis

The transcribed interviews were analysed using thematic analysis with an inductive coding approach, facilitated by Atlas.Ti software. Initial coding was conducted by NNT, who identified key patterns and concepts. These were then systematically categorised into themes and subthemes to capture broader meanings and insights emerging from the interviews. To ensure coding reliability, DAS and NP reviewed the preliminary codes and analysis results. Through thorough discussions and iterative revisions, the team resolved discrepancies and achieved near-total agreement on the coding structure. To maintain participant confidentiality, each transcript was assigned a unique identification code. 

III. RESULTS

A total of 114 respondents participated in this study, consisting of 26 males and 88 females, reflecting a possible gender disparity in medical drama viewership. Among them, 93 students (81.6%) reported that medical dramas significantly impacted their decision to pursue a medical career, while 21 students indicated no such impact.

Characteristic		Respondents (n)	Percentage (%)
Parental Background
	Both parents are doctors	8	7.0
	Only one parent is a doctor	18	15.8
	Neither parent is a doctor	88	77.2
Reports Motivation to Enter Medical Field Due to Medical Dramas
	Yes	93	81.6
	No	21	18.4
	Total Respondents	114	100.0

Table 1. Distribution of Respondent Demographic Characteristics

A. Various Dimensions of the Impact of Medical Dramas on Motivation

The following table presents the themes and subthemes derived from the in-depth interview analysis, which has been discussed and approved by each inter-rater.

No.	Theme	Subtheme
1.	Medical dramas provide an in-depth portrayal of the roles and responsibilities of doctors.	Medical dramas shape public perceptions of the medical profession and doctors’ roles.
		Medical dramas depict the challenges and rewards of doctors’ daily work.
		Medical dramas highlight doctors’ lives beyond their profession.
2.	Medical dramas depict both the professional and personal aspects of doctors’ lives.	Medical dramas reveal lesser-known aspects of doctors’ personal experiences.
		Medical dramas provide an authentic portrayal of doctors’ daily lives.
		Medical dramas depict ethical dilemmas, emphasising doctors’ professional integrity.
3.	Medical dramas contribute to a positive public perception of the medical profession.	Medical dramas portray impressive doctors, evoking admiration, and inspiration in viewers.
		Doctors’ life-saving actions in medical dramas evoke deep emotions and motivation.
		Medical dramas illustrate doctors’ perseverance and resilience in their profession.
		Medical dramas spark curiosity about medical procedures and treatments.
4.	Medical dramas evoke emotional responses that may inspire viewers to pursue a medical career.	Medical dramas evoke empathy by portraying patients in distress.
5.	Medical dramas showcase inspiring doctor characters, motivating viewers to follow in their footsteps.
6.	Medical dramas illustrate the challenges and dedication required to become a doctor.

Table 2. Table of Themes and Subthemes

B. Medical Dramas Provide an In-Depth Portrayal of the Roles and Responsibilities of Doctors

Medical dramas frequently portray various medical scenarios using specialised terminology, offering viewers insight into the daily responsibilities of doctors. As medical settings are generally inaccessible to the public, these dramas provide a unique perspective on the profession. Among the key influences explored in this study, this aspect resonated most with respondents, with four out of six students emphasising its significance.

One participant explained how watching medical dramas sparked their interest in the field, as they previously had limited information about medicine.

“Grey’s Anatomy offered me insight into the medical field and sparked my interest, as reliable information is not easily accessible to those outside the profession” (V1_H3_B19)

Another respondent shared how their perception of doctors shifted from fear to admiration after watching medical dramas, recognising the profession’s vital role in society:

“As a child, I perceived doctors as intimidating due to painful medical visits, such as dental appointments. However, after watching Grey’s Anatomy, I realised that medicine extends beyond that and saw how doctors can positively impact many lives.” (V1_H2_B27)

Beyond hospital settings, medical dramas also depict the role of doctors in broader societal contexts. Through diverse narratives, these dramas offer multiple perspectives on the medical profession. For example, Doc Martin illustrates the role of a doctor in a rural community, showcasing their involvement in local issues, such as water problems and education (V2_H4_B21). This highlights how doctors contribute beyond clinical practice, reinforcing their role in public health and community development.

Additionally, medical dramas influence students’ motivation to pursue medicine by portraying the realities of medical work, including complex procedures and life-saving interventions. More than half of the respondents identified this as a significant factor in their decision to study medicine. As one participant stated:

“The scenes depicting doctors helping others, particularly during surgeries, inspired me to study medicine. I wanted to experience performing surgeries and learn how to help others.” (V4_H7_B19)

These findings suggest that medical dramas not only enhance public understanding of the profession but also play a role in inspiring future medical students.

C. Medical Dramas Depict Both the Professional and Personal Aspects of Doctors’ Lives

Medical dramas depict not only medical procedures but also the personal lives of doctors, including their relationships, friendships, and personal struggles. By presenting doctors as multidimensional individuals beyond their professional roles, these narratives contribute to a more relatable and humanised portrayal of the profession. Notably, four out of six highlighted the importance of this aspect in shaping their motivation. One respondent noted that their interest in medical dramas stemmed primarily from the blend of romantic elements, rather than the medical content alone (V3_H6_B1).

The public often has limited insight into the personal experiences of doctors, making it difficult to relate to them. However, medical dramas help bridge this gap by shedding light on aspects of doctors’ lives that are not commonly shared. This increased visibility fosters a sense of closeness and enhances a deeper understanding of the profession, which may, in turn, inspire career interest. As one participant expressed:

“Medical dramas reveal stories that doctors seldom share, providing an effective way to convey these experiences to the general public.” (V5_H11_B4)

By offering a representation of doctors’ daily lives, medical dramas provide viewers with a deeper understanding of what a career in medicine entails. This additional perspective can be a key factor in the decision-making process for those considering entering the field (V2_H4_B28).

D. Medical Dramas Contribute to a Positive Public Perception of the Medical Profession

Three out of six respondents emphasised that medical dramas portray the ethical dilemmas faced by doctors, reinforcing professional values that shape viewers’ perceptions of the medical field. One respondent noted that the depiction of doctors’ strong work ethic in these dramas increased their interest in pursuing a career in medicine (V3_H6_B4).

Additionally, medical dramas present compelling and admirable doctor characters, which can serve as a source of inspiration for viewers. As one respondent shared:

“The dramatised portrayal made it inspiring for my younger self, leading me to think, ‘Doctors are cool.’” (V5_H11_B2)

The depiction of medical procedures, particularly surgeries, also left a strong impression on some respondents, reinforcing their motivation to enter the medical field. One participant stated:

“Medical dramas provide insight into a doctor’s work, including various surgeries, highlighting the fascinating ability to help many people.”  (V4_H7_B33)

Furthermore, the portrayal of doctors saving lives in medical dramas can evoke strong emotional responses, fostering admiration and a desire to pursue medicine. One respondent expressed how these scenes encouraged them to consider a career in medicine (V5_H11_B2). Medical dramas also emphasise the perseverance and resilience of doctors in the face of various pressures and stress, further reinforcing the appeal of the profession for aspiring medical students (V5_H9_B11).

E. Medical Dramas Evoke Emotional Responses that may Inspire Viewers to Pursue a Medical Career

Medical dramas can spark viewers’ curiosity about medical procedures, motivating them to explore the medical field further and increase their interest in the profession. One respondent noted that medical dramas played a role in their initial motivation for pursuing medicine, as the cases depicted often align with real-world medical scenarios, piquing their interest to learn more:

“I believe the medical procedures shown in Grey’s Anatomy exist in real life and are up to date. For example, an episode featured an abdominal wall transplant, which piqued my curiosity. After researching it, I discovered that the procedure had only recently been developed, showing how the show connects to world medicine.” (V1_H2_B14)

Beyond fostering curiosity, medical dramas also evoke strong emotional responses, such as sadness and empathy, particularly when portraying patients in need of medical care. One respondent shared that such portrayals heightened their awareness of the number of people requiring medical assistance. The emotions they experienced, combined with this awareness, ultimately served as an inspiration to pursue medicine (V1_H2_B31).

F. Medical Dramas Showcase Inspiring Doctor Characters, Motivating Viewers to Follow in their Footsteps

Medical dramas can inspire viewers through uplifting scenes and compelling portrayals of doctor characters. These representations serve as role models, fostering admiration and desire to follow in their footsteps. Notably, three out of six respondents indicated that such portrayals influenced their decision to pursue a medical career. As one participant shared:

“In ‘New Amsterdam’, there is a doctor who stands out from others – someone who wants to create change and has a unique vision. I found him quite inspiring.” (V3_H6_B5)

G. Medical Dramas Illustrate the Challenges and Dedication Required to Become a Doctor

Medical dramas not only portray the professional responsibilities of doctors but also illustrate the journey of becoming one, highlighting the challenges and obstacles along the way. Two out of six students emphasised the significance of this theme, noting that the depiction of medical interns’ experience in these dramas played a crucial role in their motivation to pursue medicine. As one participant shared:

“Medical dramas often depict doctors under immense stress, highlighting the challenges of the profession. However, rather than discouraging me, witnessing their struggles made the career even more appealing.” (V5_H10_B28)

IV. DISCUSSION

The in-depth interviews revealed that medical students felt more mentally prepared and motivated to pursue medicine after watching medical dramas. These shows provided insight into both the professional and personal lives of doctors, reinforcing previous studies on their educational influence in shaping viewers’ perceptions of the medical field. This aligns with the Media Effect Theory, introduced by Gerbner in 1960, which suggests that television shapes individuals’ perceptions of reality, particularly among frequent viewers (Stacks et al., 2015). Hoffman and colleagues further applied this theory to medical dramas, highlighting their role in influencing viewers’ perception of the medical profession (Arias, 2018).

Beyond depicting technical medical procedures, medical dramas also emphasise ethical principles and professional values, contributing to a favourable image of the profession. Research indicates that medical and nursing students often use these shows as supplementary learning tools for bioethics and professionalism (Cambra-Badii et al., 2021). Similarly, these portrayals may inspire high school students to consider careers in medicine by offering an engaging introduction to the field.

Emotional engagement is another key factor in the influence of medical dramas. Strong emotional responses- such as curiosity, empathy, and excitement- can enhance motivation, as emotions play a crucial role in learning, achievement, and psychological well-being, particularly in adolescents (Pekrun, 2017). Positive emotions make tasks more appealing, fostering inner motivation that arises after the initial external excitement provided by medical dramas (Legault, 2016).

Furthermore, medical dramas often depict the challenges of medical training, including long working hours and emotional strain, providing a more nuanced perspective on the profession. While these portrayals may initially present an idealised view of medicine, they also serve as a reality check, reinforcing the dedication required to pursue a medical career. This dual representation may deter less committed individuals while strengthening the resolve of those deeply passionate about the field (Tedeschi, 2024).

However, the potential for misinformation in medical dramas must also be considered. The oversimplifications or dramatisation of medical practice may create discrepancies between fictional portrayals and real-world medicine, leading to misconceptions among viewers (Alismail et al., 2018).  One notable bias is the tendency to depict doctors as infallible heroes performing miraculous saves amidst dramatic emergencies, often intertwined with personal achievements and romantic subplots. This portrayal fosters unrealistic expectations, potentially leading to disillusionment when aspiring medical students encounter the demanding and less glamorous realities of medical education and practice (Tedeschi, 2024).

These findings highlight the complex role of medical dramas in shaping career motivation, offering both inspiration and potential misconceptions. Their influence appears particularly significant for students without a familial background in medicine, as most respondents came from non-medical families. This trend may suggest that medical dramas serve as an accessible source of insight into the profession for those without direct exposure. However, the possibility of sampling bias should be considered, as students already interested in medicine may be more likely to engage in medical dramas. While these dramas may serve as informal educational tools, their impact should be critically assessed to ensure they provide a balanced and realistic portrayal of the medical profession.

V. CONCLUSION

Medical dramas play a significant role in shaping students’ motivation to pursue a medical career by providing an engaging portrayal of the profession. These shows offer insight into the roles and responsibilities of doctors, their daily challenges, and the emotional rewards of the profession. Additionally, they influence public perceptions of medicine and evoke strong emotional responses through inspiring characters and narratives. By depicting both the struggles and triumphs of medical training, medical dramas contribute to students’ understanding of the dedication required to become a doctor. This study highlights six key aspects through which medical dramas impact students’ decision to pursue medical education, highlighting their role as both an informational and motivational tool.

Notes on Contributors

Nathania Nishida Tan participated in data collection, data analysis, review, the writing of the paper, and the formatting for publication.

dr. Daniel Ardian Soeselo, Sp.B, MSi.Med. participated in the data analysis, review, and direction of the paper.

dr. Natalia Puspadewi, M.Med.Ed, Ph.D participated in the data analysis, review, and direction of the paper.

dr. Gisella Anastasia, MHPE participated in the review of the paper.

dr. V. Dwi Jani Juliawati, M.PD, Sp.KKLP participated in the review of the paper.

Ethical Approval

This study received ethical clearance from the Research Ethics Commission of the Faculty of Medicine and Health Sciences Atma Jaya Catholic University of Indonesia under the ethical approval code 13/02/KEP-FKIKUAJ/2024. To ensure confidentiality and data security, all interview transcripts were anonymised and securely stored in password-protected files, accessible only to the research team.

Data Availability

The data of this qualitative study are not publicly available due to confidentiality agreements with the participants.

Acknowledgement

The authors would like to thank all the students who participated in this study.

Funding

There was no funding for this article.

Declaration of Interest

The authors have no conflicts of interest in connection with this article.

References

Aboud, K. A. (2012). Medical dramas – The pros and the cons. Dermatology Practical & Conceptual, 2(1), 75–77. https://doi.org/10.5826/dpc.0201a14

Alahmari, A. A. (2023). Professionalism, ethics, and realism of television medical dramas as perceived by Saudi medical students. Journal of Medical Education and Curricular Development, 10. https://doi.org/10.1177/23821205231175037

Alismail, A., Meyer, N. C., Almutairi, W., & Daher, N. S. (2018). CPR in medical TV shows: Non-health care student perspective. Advances in Medical Education and Practice, 9, 85–91. https://doi.org/10.2147/AMEP.S146149

Arias, E. (2018). How does media influence social norms? Experimental evidence on the role of common knowledge. Political Science Research and Methods, 7(3), 561–578. https://doi.org/10.1017/psrm.2018.1

Cambra-Badii, I., Moyano, E., Ortega, I., Baños, J., & Sentí, M. (2021). TV medical dramas: Health sciences students’ viewing habits and potential for teaching issues related to bioethics and professionalism. BMC Medical Education, 21, Article 509. https://doi.org/10.1186/s12909-021-02947-7

Gehrau, V., Brüggemann, T., & Handrup, J. (2016). Media and occupational aspirations: The effect of television on career aspirations of adolescents. Journal of Broadcasting & Electronic Media, 60(3), 465–483. https://doi.org/10.1080/08838151.2016.1203319

14^th Media Mutations International Conference. (2023). Investigating medical drama TV series: Approaches and perspectives.

Legault, L. (2016). Intrinsic and extrinsic motivation. Springer eBooks. https://doi.org/10.1007/978-3-319-28099-8_1139-1

Levine, S., & McLuhan, M. (1964). Understanding media: The extensions of man. American Quarterly, 16(4), 646. https://doi.org/10.2307/2711172

Mendick, H., & Williams, K. (2022). Does television influence young people’s career choices? Journal of the National Institute for Career Education and Counselling, 21(1), 18–22. https://doi.org/10.20856/jnicec.2103

Pekrun, R. (2017). Emotion and achievement during adolescence. Child Development Perspectives, 11(3), 215–221. https://doi.org/10.1111/cdep.12237

Sonego, A., & Rocchi, M. (2024). Medical drama TV series: A semi-systematic literature review. Online Journal of Communication and Media Technologies, 14(4), Article e202459. https://doi.org/10.30935/ojcmt/15268

Stacks, D. W., Li, Z. C., & Spaulding, C. (2015). Media effects. Elsevier eBooks. https://doi.org/10.1016/B978-0-08-0970868.95045-1

Tedeschi, R. (2024). The paradox of medical dramas: Global aspirations versus realities. BJPsych Advances, 31(1), 62–64. https://doi.org/10.1192/bja.2024.56

Zago, D., Cautero, P., Scarpis, E., Pompili, E., Voglino, G., Siliquini, R., Brusaferro, S., & Brunelli, L. (2024). TV medical dramas: Assessing the portrayal of public health in primetime. Frontiers in Public Health, 12. https://doi.org/10.3389/fpubh.2024.1432528

*Daniel Ardian Soeselo
Medical Education Unit and Department of Surgery,
Faculty of Medicine and Health Sciences,
Atma Jaya Catholic University of Indonesia
Jl. Pluit Raya No.2 21, RT.21/RW.8,
Penjaringan, Kec. Penjaringan, Jakarta Utara,
Daerah Khusus Ibukota Jakarta, Indonesia 14440
Email: daniel.ardian@atmajaya.ac.id

Submitted: 16 August 2024
Accepted: 23 December 2024
Published online: 1 July, TAPS 2025, 10(3), 49-57
https://doi.org/10.29060/TAPS.2025-10-3/OA3495

Shuh Shing Lee¹, Shefaly Shorey², Tang Ching Lau³ & Dujeepa D. Samarasekera¹

¹Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Dean’s Office, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: Numerous studies have been conducted on COVID-19, with the majority focusing on interventions involving students and teachers. However, limited research has delved into the pandemic’s impact on the wellness of various stakeholders and how they have adapted to the challenges it presented. This study aims to fill this gap by exploring these neglected areas.

Methods: This study employs a sequential mixed-method approach to study these areas. The quantitative data collection was carried out using a combination of validated surveys (ranging between 63-88 items) for students, faculty and administrators.  Subsequently, qualitative data collection was gathered via semi-structured interview using a convenient sampling method.

Results: Seventeen faculty, 18 administrators and 369 students responded to the survey. The quantitative data indicated faculty (teachers) exhibited the lowest stress levels and the highest resilience during the pandemic. In comparison, administrators and students experienced moderate levels of stress, with students scoring slightly higher on the stress level. The themes that emerged from the qualitative data were personal endurance, emotional reaction, cognitive-behavioural reaction and social support.

Conclusion: Our study highlighted that, apart from personal endurance, the tension arises from emotional and cognitive-behavioural responses of students, teachers, and administrators can be mitigated based on the presence or absence of support mechanisms.

Keywords:           Wellbeing, Change, Stakeholders, Educational Environment, Culture

Practice Highlights

• Students experienced the highest stress levels compared to administrators and teachers.
• However, students and administrators demonstrated resilience, bouncing back quickly after challenging times.
• Students and administrators tolerated for uncertainty and displayed cognitive flexibility to enable them to adapt and seek opportunities.
• Teachers and administrators initially experienced negative emotions, but their emotional resilience facilitated quick recovery.
• Coming from a culture emphasising collectivism, the sense of belonging and social connection served as a protective factor against psychological distress.

Table 1. Sections of the Surveys for Administrators, Teachers and Students

The survey was validated by 10 medical educators from various departments (Paediatrics, Surgery, Centre for Medical Education, Nursing). After the validation, the survey was administered to medical (Year 1 – 5) and nursing (Year 1 – 4) students, administrators and faculty members in Yong Loo Lin School of Medicine and Alice Lee Centre for Nursing Studies using convenient sampling. It took about 20-30 minutes to complete the survey and the data was collected between Jan – June 2021.

B. Phase II Qualitative Data Collection

The qualitative data collection was gathered via semi-structured interview. The interview was conducted for about 60-90 minutes among the medical/nursing students, administrators and faculty members (teachers). Followed up from the data collected from the quantitative data, the questions were revolved around teaching and learning, content, assessment, policies, guidelines, communication, environment (safety)/support and wellness.

From July 2021 – Nov 2022, we used convenient sampling method to recruit of students, administrators and faculty members. The interviews were carried out by 2 trained interviewers with no power relationship with the interviewees. Interviews were carried out after getting consent from the volunteer interviewees. All digital audio recordings made during the interviews were transcribed and member-checked with the interviewees to ensure transparency and trustworthiness of the data.

Data collection ceased when the data reached saturation stage.

C. Data Analysis

The quantitative data was analysed using descriptive statistics (such as mean, frequency and percentage) using Microsoft Excel for the data collected from students, administrators and teachers.

The interviews were thematically analysed by 2 researchers in the team. The two researchers coded the transcripts independently and came together to resolve any discrepancy or disagreement on the coding.  Subsequently, they continued to code and form categories and eventually themes. There were multiple discussions that took place among the researchers and the team before the themes were crystalised.

III. RESULTS

A. Phase I Quantitative Data

The demographic information was illustrated in Table 2. Majority of the participants were not quarantined during the pandemic and more than 80% of them did not have a family member tested positive for COVID when this study was conducted. The teachers from the school of medicine were mainly from Family Medicine, Paediatrics, Physiology, Pathology, Public Health, Medicine, Anatomy and Anaesthesia departments. They are educators for postgraduate and undergraduate students. As for administrators, their roles in the departments are educational related such as instructional design, learning analytics, planning and execution of education, managing project and training.

Table 2. Demographic Information of the Respondents

For each section, the summary was illustrated in Table 3. The mean for different sections was quite close for the 3 groups. Likewise, the items that were scored low and high were quite similar for all the sections. For example, Section B The psychological impact of COVID-19, The sleep quality and concentration in all three groups were not affected by the pandemic, but they are more worries about their family members contracted with COVID-19.

Table 3. Summary of the Mean for Different Sections for the 3 Groups

During the pandemic, students expressed significant concerns about the adequacy of their training, particularly due to reduced patient contact, raising apprehensions about their preparedness for exams. This concern will be elaborated upon in the qualitative data section, shedding light on the depth of their worries. All three groups shared the view that information originating from the government and hospitals was the most reliable, with friends and family scoring the lowest mean among all sources. Despite this, the participants unanimously agreed that the support from friends and family, in terms of both source and level of assistance, was the most substantial. Conversely, organisational support from entities such as the University Wellness Centre, Dean’s office, community, and social media was perceived as unreliable and lacking during the pandemic.

Furthermore, our observations revealed that teachers exhibited the lowest stress levels and the highest resilience during the pandemic, showcasing their ability to cope effectively. In comparison, administrators and students experienced moderate levels of stress, with students scoring slightly higher on the Perceived Stress Scale. Although students acknowledged challenges, as indicated by their agreement with statements such as “I have a hard time making it through stressful events” (mean: 2.93), they also exhibited resilience, agreeing with the statement “I tend to bounce back quickly after hard times” (mean: 3.76). Additionally, concerning self-efficacy towards technology integration, students reported the highest mean score, indicating confidence in their ability to navigate various Internet applications. While teachers felt competent in using technology for teaching and learning, their confidence wavered when it came to resolving technical issues, as reflected in their mean score of 3.47. This nuanced understanding underscores the complex interplay of stress, resilience, and technological proficiency among the different groups during the challenging circumstances of the pandemic.

B. Phase II Qualitative Data

As for the qualitative data collection, we have recruited 7 administrators, 17 teachers (12 from Yong Loo Lin School of Medicine and 5 from Alice Lee Centre for Nursing Studies) and 9 undergraduate students (6 from Yong Loo Lin School of Medicine and 3 from Alice Lee Centre for Nursing Studies). The themes and subthemes that emerged were depicted in the Figure 1.

Figure 1. Themes and Subthemes of the Qualitative Data

1) Theme 1: Personal Endurance

Personal endurance depends on perceived ability and tolerance to uncertainty. For administrators, they felt that it was quite stressful and frustrated during the pandemic as the situation was unclear. However, they were able to manage and there was a sense of relief after they had gone through the critical phase.

“It was very intense, stressful but looking back now, it is not that bad. We have gone through the worse” (Admin 5)

While administrators’ contribution to the education system is crucial, some of the administrators perceived their contribution was minor as compared to medical front liner.

“We are not front liner, our contribution is limited.” (Admin 3)

Although the situation was stressful in the beginning, we noticed a positive endurance among the teachers and perceived the pandemic as an opportunity instead of a threat.

“Overall, I think the predominant mood was of a challenge that needs to be overcome and that brought a certain amount of excitement.” (Teacher 7)

On the other hand, the students felt that they were being too protected and perceived themselves as having the ability to manage the situation themselves.

“I understood that they wanted to protect us but I felt that eventually, they can’t protect us anymore” (Student 3)

The students also perceived that the teachers lacked ability in using technology in teaching and learning especially in remote learning.

“A lot of professors are not familiar with the technology.” (Student 5)

2) Theme 2: Emotional Reaction

There were a lot of negative emotions illustrated by the students, administrators and teachers due to various reasons. Students were worried, frustrated and anxious that they may not learn since the contact with the patients was less during the pandemic. Too much protection from the school and the system put in place has heightened these negative emotions.

“We feel quite unconfident because we feel we have not seen enough patients”/ “..fear that we are not as good as the previous batch” (Students 1 & 2)

Administrators were frustrated mainly because they need to manage the family and work at the same time when working from home system was implemented. However, some of them shared that they are able to regulate and get used to the situation after a while.

“Everybody was under pressure at that time…while I have to juggling with work, my kids were at home because school close.” (Admin 4)

“I usually regulate my own emotion.” (Admin 7)

While there were some positive emotions state in Theme 1 for teachers, they did feel stressful in the early stage of the pandemic due to the change of the approaches in teaching and learning and they are unsure of the outcomes when the teaching was entirely online.

“…stressful in the beginning… I even have nightmares…dreaming students get lost in the virtual room.” (Teacher 17)

“It was a bit stressful in the beginning because you did not know how is going to turn out…” (Teacher 5)

3) Theme 3: Cognitive-Behavioural Reaction

Amidst the challenges posed by the pandemic, administrators, students, and teachers made concerted efforts to adapt their cognition and behavior in response to various initiatives, including social distancing measures, reduced patient contact, and a shift to virtual teaching environments. Throughout this period, interviewees shared both positive and negative reactions to these changes.

Administrators and teachers found themselves navigating the delicate balance between the need to innovate and the need to maintain productivity (ambidexterity). Administrators, in particular, faced the challenge of fostering creativity in coordinating and delivering the curriculum, which involved tasks such as timetabling, resource management, and providing IT support for online learning. These adjustments were made within a short timeframe, reflecting their resilience and adaptability. However, amid these innovative efforts, administrators were also keen on upholding the quality of their work, highlighting the complexity of their role in managing these rapid changes.

“We have to deliver in a short time but also the content has to be rigorous” (Administrator 3)

Similarly, teachers tried to be creative in an online teaching environment and ensure the student learned at the same time especially in small group teaching. However, they find it challenging.

“…there is an urgency to find a way around this small group teaching…we kind of lose the whole power of collaboration.”  (Teacher 5)

There is also a tension arose among the students for being too protected by the school and compromised with their learning as shared in the quotes below. This was repeatedly mentioned by the students, and they felt they have to face the situation eventually.

“I am not very interested in surgery, but this is like once in a lifetime and after I go out of medical school I won’t have the chance to see surgery” (Students 3)

“There’s a culture… in the society in general…protect my child from COVID. But once day we are going to deal with COVID” (Students 6)

Notwithstanding the aforementioned tensions, it’s worth noting that administrators, students, and teachers exhibited remarkable innovativeness and adaptability during the pandemic. All three groups demonstrated evidence of both Operational Capabilities, which encompass the efficient and effective use of resources, and Dynamic Capabilities, which involve the continuous development of competencies to align with the evolving environment.

With the predominant shift in communication from face-to-face to virtual platforms, administrators found themselves assuming the role of intermediaries responsible for conveying information to various stakeholders. In this new virtual setting, where body language cues were less apparent, administrators recognised the need to be more attuned and sensitive to subtle nuances in communication compared to traditional face-to-face interactions. This adaptation reflected their ability to pivot and operate effectively within the changing landscape of remote communication.

“We play the middleman role because we have to speak administrative language to certain people and be sensitive when communicate with faculty members” (Admin 3)

“We have to start thinking about (what kind of information needed) before the faculty member even ask those questions” (Admin 7)

Teachers utilised different resources to innovate in their teaching as well as learning from different others.

“I break it up my lectures into smaller bits and disperse it with PollEverywhere” (Teacher 5)

“We formed a group we called a brown bag meeting – basically we meet at lunchtime with technologically savvy administrators to introduce to the staff on how to make online learning more interactive.”  (Teacher 10)

Likewise, since there was less patient contact time, students tried to make use of their time for other learning sessions.

“Since there’s little time in the hospital, I had read up a lot” (Students 1)

“It allows us to have some processing time and have time to consolidate our knowledge” (Students 5)

4) Theme 4: Social Support Mechanism

Social support mechanism has been mentioned by the 3 groups as one of the prominent mechanisms in adapting the changes during pandemic. It includes transparent communication, team collaboration and support from various stakeholders. For example, administrators shared that they all came together and supported each other during the hard times.

“All different teams come together, I think that was very precious” (Administrator 2)

Students sought seniors’ help to provide additional sessions to compensate their learning.

“What my group would do is we call our seniors to give us extra tutorials” (Student 2)

However, there was also lack of support mechanism brought up by the teachers which led to negative emotion (such as frustration).

“Educational technology team are overworked…if the school would really want to be the best or world class, I think we need a very good support from the IT.” (Teacher 9)

IV. DISCUSSION

The impact of the COVID-19 pandemic on both our educational systems and personal lives has been profound. This unprecedented disruption has been keenly observed by various stakeholders, including administrators, teachers, many of whom were also frontline workers, and students in medical schools. Swift adaptation to the ever-changing situation became imperative, particularly in response to the government’s new guidelines. The abrupt alterations in social interactions and extracurricular activities routines compelled a shift towards a heightened emphasis on family life, accompanied by the necessity to work and learn from home due to lockdown measures. These changes had profound physical and psychological effects on our lives.

Our study revealed that students experienced the highest stress levels compared to administrators and teachers, a finding consistent with previous research indicating that medical students often have higher baseline anxiety than their peers studying other disciplines (Dyrbye et al., 2006; Lasheras et al., 2020). Qualitative data highlighted that students’ stress levels were primarily attributed to the lack of patient contact and inadequate training, potentially impacting their future practice. Additionally, students expressed feelings overly protected due to initiatives like stay-at-home learning. The altered learning environment, combined with a lack of guidance on learning strategies and interpersonal relationships, left students vulnerable to intense emotional fluctuations and strained family relationships (Zhang et al., 2020). Similarly, for administrators, the shift to remote work and social isolation policies posed challenges in balancing work and family responsibilities, as evident from their qualitative comments.

However, students and administrators demonstrated resilience, bouncing back quickly after challenging times. According to Del Carmen Pérez Fuentes et al.’s (2020) Adaptability to Change framework, a sense of control, tolerance for uncertainty, and cognitive flexibility are crucial in coping with adverse situations. Despite feeling anxious and frustrated due to the inability to control the study-from-home or working-from-home policy, student and administrator tolerated for uncertainty and display cognitive flexibility to enable them to adapt and seek opportunities. Emotional resilience, the ability to generate positive emotions and recover swiftly from negative emotional experiences, played a pivotal role in psychological resilience (Zhang et al., 2020). This emotional resilience led to diverse emotional responses, influencing the cognitive processing of emotional information. Teachers and administrators initially experienced negative emotions, but their emotional resilience facilitated quick recovery, evident from their transcripts.

Emotional and cognitive-behavioural responses were further shaped by social support mechanisms within peer groups, colleagues, organisational leaders, and the government. While studies have shown that social media can heighten anxiety due to misinformation and distressing news (Gao et al., 2020), our research indicated that students, teachers, and administrators placed significant trust in information provided by the government and institutions. This trust in government intentions and capabilities fosters adherence to health regulations, essential in crisis management (Siegrist & Zingg, 2014).

Coming from a culture emphasising collectivism, our society values interdependence and family connections highly. This sense of belonging and social connection served as a protective factor against psychological distress, aligning with previous research findings (Xiao, 2021; Yu et al., 2020) . Conversely, a lack of social support within a collectivist culture, as reported by teachers and students, contributed to psychological distress. Our qualitative and quantitative data in this study support this observation, emphasising the significance of social support structures in mitigating the adverse effects of challenging circumstances. The importance of fostering a supportive environment, both within institutions and at a societal level, cannot be overstated in times of crisis.

V. LIMITATIONS

This study has some potential limitations. The study was carried out in a single medical school; hence, the results can only be transferable to the same context. The number of respondents was quite small (especially for nursing respondents) despite multiple reminders sent to the various groups. Therefore, they may not be representative of the entire student, teachers and administrator’s population. Third, the survey was a self-reported survey and may have inherent biases while answering the questions. While rigor is more challenging to achieve in qualitative data collection and analysis, the researchers adhere to the trustworthiness principles as much as possible in analysing and presenting the results in this paper.

VI. CONCLUSION

In conclusion, achieving wellness during a pandemic is indeed possible, but it hinges not only on the resources that organisations and governments can marshal but also on individual resilience in navigating uncertainty, cultural factors, trust, and support systems. Our study highlights the importance of familial and peer connections within our cultural context, underscoring how these bonds facilitate adaptation and innovation amid the challenges posed by the pandemic. The emotional and cognitive-behavioural responses of students, teachers, and administrators are depending on their personal endurance. However, the tension that arises in these individuals can be mitigated or exacerbated based on the presence or absence of adequate support mechanisms. Sufficient support can act as a buffer, helping individuals cope effectively with the challenges they face. Conversely, insufficient support can exacerbate stress and strain, hindering their ability to adapt and respond positively to the situation at hand.

Therefore, fostering a strong support network, both within organisations and communities, is crucial. This support not only alleviates the immediate challenges faced by individuals but also empowers them to build emotional resilience, enabling them to navigate uncertainties and adversities with greater ease. In this way, the collective endurance of individuals, coupled with robust support systems, becomes the cornerstone of achieving wellness and fostering positive responses in the face of a pandemic.

Notes on Contributors

DDS developed the research idea and design with SSL, SR & LTC. The data collection was performed by SSL. The data were analysed by SSL & DDS. DDS, SSL, SR & LTC performed the data interpretation. DDS, SSL & SR wrote the article with revision by LTC. All the authors read and agreed with the final manuscript.

Ethical Approval

Ethics approval was sought from the National University of Singapore (NUS) Institutional Review Board (NUS-IRB-2020-216). Written informed consent was obtained from all participants.

Data Availability

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available as the participants of this study did not give written consent for their data to be shared publicly.

Acknowledgement

We would like to express our heartfelt gratitude to Jillian Yeo and Lilusha Kaludewa for helping in data collection and analysis.

Funding

No funding is available for this research.

Declaration of Interest

The authors report that there are no conflict of interests to declare.

References

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385. https://doi.org/10.2307/2136404

Creswell, J. W. (2013, January 1). Steps in conducting a scholarly mixed methods study [Slide show]. https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=dberspeakers

Danet, A. D. (2021). Psychological impact of COVID-19 pandemic in Western frontline healthcare professionals. A systematic review. Medicina Clínica (English Edition), 156(9), 449–458. https://doi.org/10.1016/j.medcle.2020.11.003

Daniel, M., Gordon, M., Patricio, M., Hider, A., Pawlik, C., Bhagdev, R., Ahmad, S., Alston, S., Park, S., Pawlikowska, T., Rees, E., Doyle, A. J., Pammi, M., Thammasitboon, S., Haas, M., Peterson, W., Lew, M., Khamees, D., Spadafore, M., . . . Stojan, J. (2021). An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME guide no. 64. Medical Teacher, 43(3), 253–271. https://doi.org/10.1080/0142159x.2020.1864310

Del Carmen Pérez-Fuentes, M., Del Mar Molero Jurado, M., Martínez, Á. M., Fernández-Martínez, E., Valenzuela, R. F., Herrera-Peco, I., Jiménez-Rodríguez, D., Mateo, I. M., García, A. S., Del Mar Simón Márquez, M., & Linares, J. J. G. (2020). Design and validation of the adaptation to change questionnaire: New realities in times of COVID-19. International Journal of Environmental Research and Public Health, 17(15), 5612. https://doi.org/10.3390/ijerph17155612

Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Academic Medicine, 81(4), 354–373. https://doi.org/10.1097/00001888-200604000-00009

Eva, K. W., & Anderson, M. B. (2020). Medical education adaptations: Really good stuff for educational transition during a pandemic. Medical Education, 54(6), 494. https://doi.org/10.1111/medu.14172

Gao, J., Zheng, P., Jia, Y., Chen, H., Mao, Y., Chen, S., Wang, Y., Fu, H., & Dai, J. (2020). Mental health problems and social media exposure during COVID-19 outbreak. PLoS ONE, 15(4), Article e0231924. https://doi.org/10.1371/journal.pone.0231924

Gordon, M., Patricio, M., Horne, L., Muston, A., Alston, S. R., Pammi, M., Thammasitboon, S., Park, S., Pawlikowska, T., Rees, E. L., Doyle, A. J., & Daniel, M. (2020). Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME guide no. 63. Medical Teacher, 42(11), 1202–1215. https://doi.org/10.1080/0142159x.2020.1807484

Jia, Q., Qu, Y., Sun, H., Huo, H., Yin, H., & You, D. (2022). Mental health among medical students during COVID-19: A systematic review and meta-analysis. Frontiers in Psychology, 13, Article 846789. https://doi.org/10.3389/fpsyg.2022.846789

Kiili, C., Kauppinen, M., Coiro, J., & Utriainen, J. (2016). Measuring and supporting pre-service teachers’ self-efficacy towards computers, teaching, and technology integration. The Journal of Technology and Teacher Education, 24(4), 443–469. https://jyx.jyu.fi/bitstream/123456789/53087/1/kiilikauppinencoiroutriainen.pdf

Lasheras, I., Gracia-García, P., Lipnicki, D., Bueno-Notivol, J., López-Antón, R., De La Cámara, C., Lobo, A., & Santabárbara, J. (2020). Prevalence of anxiety in medical students during the COVID-19 pandemic: A rapid systematic review with meta-analysis. International Journal of Environmental Research and Public Health, 17(18), 6603. https://doi.org/10.3390/ijerph17186603

Mark, L., & John, B. (2003). The impact of the 2003 SARS outbreak on medical students at the University of Toronto. University of Toronto Medical Journal, 82, 158–164. https://api.semanticscholar.org/CorpusID:59193656

Muller, A. E., Hafstad, E. V., Himmels, J. P. W., Smedslund, G., Flottorp, S., Stensland, S. Ø., Stroobants, S., Van De Velde, S., & Vist, G. E. (2020). The mental health impact of the COVID-19 pandemic on healthcare workers, and interventions to help them: A rapid systematic review. Psychiatry Research, 293, Article 113441. https://doi.org/10.1016/j.psychres.2020.113441

Paz, D. C., Bains, M. S., Zueger, M. L., Bandi, V. R., Kuo, V. Y., Cook, K., & Ryznar, R. (2022). COVID-19 and mental health: A systematic review of international medical student surveys. Frontiers in Psychology, 13, Article 1028559. https://doi.org/10.3389/fpsyg.2022.1028559

Siegrist, M., & Zingg, A. (2014). The role of public trust during pandemics. European Psychologist, 19(1), 23–32. https://doi.org/10.1027/1016-9040/a000169

Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008). The brief resilience scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine, 15(3), 194–200. https://doi.org/10.1080/10705500802222972

Wilcha, R. (2020). Effectiveness of virtual medical teaching during the COVID-19 Crisis: Systematic review. JMIR Medical Education, 6(2), Article e20963. https://doi.org/10.2196/20963

Xiao, W. S. (2021). The role of Collectivism–Individualism in attitudes toward compliance and psychological responses during the COVID-19 Pandemic. Frontiers in Psychology, 12, Article 600826. https://doi.org/10.3389/fpsyg.2021.600826

Xiong, N., Fritzsche, K., Pan, Y., Löhlein, J., & Leonhart, R. (2022). The psychological impact of COVID-19 on Chinese healthcare workers: A systematic review and meta-analysis. Social Psychiatry and Psychiatric Epidemiology, 57(8), 1515–1529. https://doi.org/10.1007/s00127-022-02264-4

Yu, H., Li, M., Li, Z., Xiang, W., Yuan, Y., Liu, Y., Li, Z., & Xiong, Z. (2020). Coping style, social support and psychological distress in the general Chinese population in the early stages of the COVID-19 epidemic. BMC Psychiatry, 20, Article 426. https://doi.org/10.1186/s12888-020-02826-3

Zhang, Q., Zhou, L., & Xia, J. (2020). Impact of COVID-19 on emotional resilience and learning management of middle school students. Medical Science Monitor, 26, Article e924994. https://doi.org/10.12659/msm.924994

*Lee Shuh Shing
Centre for Medical Education,
Yong Loo Lin School of Medicine,
National University of Singapore, Singapore
10 Medical Dr, Singapore 117597
Email: medlss@nus.edu.sg