Submitted: 11 February 2022
Accepted: 21 July 2022
Published online: 4 April, TAPS 2023, 8(2), 4-13
https://doi.org/10.29060/TAPS.2023-8-2/OA2755

Tri Nur Kristina¹, Fatikhu Yatuni Asmara², Sudaryanto Sudaryanto¹, Nuryanto Nuryanto³ & Saekhol Bakri¹

¹Department of Medicine, Faculty of Medicine, Universitas Diponegoro, Indonesia; ²Department of Nursing, Faculty of Medicine, Universitas Diponegoro, Indonesia; ³Department of Nutrition, Faculty of Medicine, Universitas Diponegoro, Indonesia

Abstract

Introduction: This study aimed to examine the usefulness of Community Based-Education (CBE) and Interprofessional Education (IPE) to community health outcomes.

Methods: The design was a mixed-method study. Each small group worked together to identify family health problems, implement interventions, and evaluate the results. The Readiness for Interprofessional Learning Scale (RIPLS) questionnaire was applied to compare students’ perceptions before and after the implementation. In addition, qualitative data were acquired from students’ comments on the questionnaire, interviews with community representatives, and FGDs with instructors at the end of the program.

Results: Three hundred and sixty-seven out of 465 students returned the completed paired questionnaire (78.9 % response rate). Paired t-test showed that student perception of responsibility slightly increased, whereas teamwork and collaboration, negative and positive professional identity decreased somewhat. However, most pre-post students’ responses already trended to the positive side (scales 4 and 5). Moreover, the Chi-square test showed that pre-post students’ confidence levels significantly increased. Most small groups of students successfully empowered the community to solve health problems. Students, instructors, and representatives of community members appreciated the program. However, several students felt their instructors were not motivated to mentor their tasks, and sometimes they had difficulty conducting home visits together. 

Conclusion: This study showed that the combination of CBE-IPE contributes to more favourable community health outcomes. However, it was challenging in several aspects of preparation, including highly motivated instructors. Nevertheless, pre and post-implementation students’ attitudes are still positive.

Keywords:           Interprofessional Education (IPE), Community Based Education (CBE), Health Profession Students, Readiness

Submitted: 28 July 2022
Accepted: 12 October 2022
Published online: 3 January, TAPS 2023, 8(1), 25-32
https://doi.org/10.29060/TAPS.2023-8-1/OA2850

Choon Peng Jeremy Wee¹, Mingwei Ng¹ & Pim W. Teunissen²

¹Department of Emergency Medicine, Singapore General Hospital, Singapore; ²School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands

Abstract

Introduction: This study was performed to understand how fixed clinical teams affected informal learning in Emergency Medicine Residents. Better understanding the effects of team dynamics on informal learning may help to optimise learning and improve performance.

Methods: From 8^th February 2020 till 27^th September 2020, the Singapore General Hospital Emergency Department adopted a fixed team system. Zoom interviews were carried out amongst Emergency Medicine Residents who worked in the fixed team system using a semi-structured iterative interview guide. A qualitative content analysis was used for this exploratory study. The interviews were transcribed verbatim, anonymised and coding via template analysis performed. Data collection and analysis were performed until data sufficiency.

Results: The themes identified centred around relationship dynamics, team composition and motivation for learning. The first was how improved relationships led to improved trust, communications and camaraderie among team members. This improved peer learning and clinical supervision and provided a more personalised learning experience. A balanced team composition allowed learners to be exposed to experts in various subspecialties. Finally, there was an initial increase in motivation, followed by a decrease with time.

Conclusion: In postgraduate medical education, working in a fixed team system with balanced members had positive effects on informal learning by strengthening relationships and communications.

Keywords:           Informal Learning, Workplace Learning, Fixed Teams, Medical Education, Postgraduate

Practice Highlights

• Fixed teams can strengthen relationships between members through better trust, familiarity and communication.
• A balanced fixed team with members having different areas of expertise allows a variety of perspectives.
• Rotation of team members may achieve a balance between the stronger relationship and familiarity of the members of fixed teams and the greater variance in perspectives from a non-fixed team system.

I. INTRODUCTION

Fixed teams (FT) and non-fixed teams (NFT) exist in medicine because of differing service requirements and manpower resources. Examples of FTs can exist in ward round teams and operating surgical teams (Eddy et al., 2016; Stepaniak et al., 2012) where personnel stay within the same work team for long periods. In other areas of healthcare like the Emergency Departments (ED), a NFT system is usually employed where teams are formed according to the personnel rostered to work on that shift and team members change every shift. This allows a more flexible system for the team members as they can request off days and leave according to their personal schedule and yet allows for 24-hour coverage in the ED. 

High levels of performance are required of medical teams, both fixed and non-fixed, to achieve good patient outcomes. Therefore, team members should learn how to work effectively together to deliver the best patient care. There are studies, both within healthcare and other industries, which showed mixed results when FT were compared with NFT with regards to team performance. A systematic review on minimal invasive surgical teams found that the FTs had better teamwork and had reduced rates of technical mistakes compared to NFTs (Gjeraa et al., 2016). However, an aviation study showed that FTs made more minor errors compared to NFTs due to FT members being too familiar with each other and overlooking errors (Barker et al., 1996). Although it is unclear how transferable specific research findings from non-healthcare domains are, what is clear is that FTs and NFTs are different in the way teams were formed and the amount of time team members spend working together. There is a lack of understanding if and how these differences affect the way learning takes place in FT and NFTs; which could translate to the performance of the team and its members. 

Workplace learning occurs through informal learning by experiencing work challenges and via interpersonal relationships. Informal learning can be supported through learner engagement by encouraging active participation in work activities and aligning learners’ interests with that of the organisation’s objectives towards improving the individuals’ and organisation’s capabilities (Billett, 2007). Informal learning is now widely accepted as a form of workplace learning that occurs out of a formal planned teaching program. It usually occurs during work activities which are not primarily aimed towards education, with learning objectives not planned beforehand (Callanan et al., 2011; Rogoff et al., 2016; Watkins et al., 2018). Although informal learning had been studied, there are very few studies looking at how being in FTs affects informal learning. A review on the involvement of employees in workplace learning (Kyndt & Baert, 2013) revealed that there was a paucity of literature on whether any team system improves the involvement of employees in informal learning. Thus, it remains to be studied what effects working in a FT system has on informal learning especially of the more junior team members. 

An integrative literature review on informal learning found that some of the important components of informal learning within members of a team and between teams included interpersonal relationships, feedback, networking and leadership (Jeong et al., 2018). Therefore, there may be differences in interpersonal relationships and feedback between the different team systems. It is known that good interpersonal relationships include good supervisor and peer support and both affect whether what is learnt is applied at the workplace (Burke & Hutchins, 2016), there was little published data on whether supervisor and peer support or even the supervisory relationship were affected by the amount of time spent together. Within some healthcare systems both FTs and NFTs consists of members with varying levels of experience, differing expertise and roles. In a FT, the learners are only supervised by supervisors within that team; therefore, each learner’s supervised time is divided within a small group of supervisors as compared to a NFT system where each learner’s supervised time is spread amongst a larger number of supervisors. Thus, despite this increased time spent together in a FT, it is not clear if working in a FT impacts supervisory or interpersonal relationship. 

Having a good trusted learner-supervisor relationship can result in the establishment of an “Educational Alliance” (Telio et al., 2015). This is because feedback from a credible trusted source was more readily accepted and feedback is another important component of effective informal learning within a team (Jeong et al., 2018).

Furthermore, a study among social work students and their supervisors in a rotational placement model, found that the longer the amount of time they spent with each other the greater the trust between them (Vassos et al., 2017). On the other hand, being in a FT could restrict networking and socialisation to a smaller group of people as contact with other teams’ members could be reduced however it is not known how this could affect informal learning. 

Understanding how informal learning takes place within FT and NFT may allow optimisation of learning within each and perhaps even configure teams to enhance learning and thus ultimately improve performance. Our study aimed to fill this gap in the literature by exploring how fixed clinical teams affected the experience of informal learning for Emergency Medicine (EM) Residents. By doing so we hope to understand how informal learning can be supported via the appropriate implementation of team systems especially where high performance is expected from the teams.

II. METHODS

To study how being in fixed clinical teams affected the experience of informal learning for EM Residents we conducted an exploratory qualitative study based on a constructivism research paradigm using content analysis of individual interviews. This is because informal learning could not be quantified with specific learning outcomes.

A. Setting

EDs teams manage a large number of critically ill patients who may need time sensitive interventions. These teams would comprise of experienced Emergency Physicians (EPs) and more junior Medical Officers (MOs) and Residents. The Residents are postgraduate doctors who are training to graduate as EPs; therefore, informal workplace learning is a crucial part of their training. Hence the residents would be good study subjects to investigate the effect of team systems on informal learning. 

The Singapore General Hospital (SGH) ED functioned via NFTs where the composition and number of members in the team differed with each 8 hour shifts accordingly to the anticipated patient load. The COVID-19 pandemic provided a naturalistic setting where the effect of a FT system can be compared to a NFT system which had existed before hand. After COVID-19 was declared a pandemic by the World Health Organisation (WHO) on the 11^th March 2020 (World Health Organisation, 2020). There was an emphasis on infection control to contain the pandemic. Many countries had instituted social distancing measures which included curfew-like measures and travel restrictions (Lake, 2020). Similarly, the Singapore government had instituted legislative measures to limit face-to-face interactions. In the ED of SGH, measures were put in place to limit the spread of COVID-19.

Thus, from 8^th February till 27^th September 2020, as part of infection control measures, doctors were organised into 5 FTs, each having between 5-7 Consultants, 3-5 EM Senior Residents, 2-3 EM Junior Residents and 7-9 non-EM Medical Officers (MO) (Liu et al., 2020; Quah et al., 2020). The Senior Emergency Physicians (SEPs) consisted of certified specialists in EM (Associate Consultants, Consultants and Senior Consultants); they played supervisory and educational roles to the junior doctors (JD) which included MOs, Junior Residents and Senior Residents. Each FT worked 12-hour shifts. Interactions between teams were kept to a minimum and members from different teams were not allowed to mingle. Thus, the residents were only supervised by their team’s SEPs. Informal learning would now occur within these FTs. 

Formal learning was converted to a remote online platform because of infection control measures. Lectures and tutorials were held and recorded using software which enabled online asynchronous access e.g. Zoom (Zoom, 2016) as not all residents could be given protected learning time together. Sessions which could not be transferred onto an online platform (e.g. hands on simulation and procedure skills training) were cancelled. Formal summative examinations were also cancelled.

B. Interviews

Interviews were conducted and recorded via Zoom (Zoom, 2016) to maintain social distancing. The primary investigator performed 11 interviews and a coinvestigator performed the remaining 4 out of a total of 15 interviews. A semi-structured iterative interview guide was developed based on Eraut’s typology on informal learning which included team activities, tasks and enabling/disabling factors (Eraut, 2010) was used, and the interviews were audio-recorded and transcribed verbatim and anonymised. The interviews allowed positive and negative aspects to be explored and being semi structured the questions asked varied according to the interviewees’ responses. This helped to focus the interviewees to what informal learning was with examples when it could occur within team activities. The guide was iteratively amended with each interview to enhance clarity which helped to obtain more in-depth data in later interviews. 

C. Participants

Twenty-four ED residents were working in the FT system in the ED of SGH during COVID-19. Fifteen were Junior Residents and 9 were Senior Residents. Purposive sampling was carried out with at least two Residents from each team being sampled. This is to ensure that there was good representation for all of the fixed clinical teams. All 24 residents were invited to participate via email and WhatsApp messaging platform with written consent being obtained. Fifteen individual interviews were conducted before data sufficiency was achieved where no new data would change the outcome of the study, thus no further interviews were conducted beyond data sufficiency (Varpio et al., 2017). Eleven interviewees were Junior Residents (4 females and 7 males) and 4 were Senior Residents (3 females and 1 male).

D. Data Analysis

Data analysis was performed via template analysis by the primary and coinvestigator independently (Brooks et al., 2015). Eraut’s typology developed from his research on informal learning was used as a priori themes in the initial coding template (Eraut, 2010). Coding template modifications were made as the analysis of the transcripts continued. Themes were categorised into hierarchical clusters and relationships between them were studied and defined. After final modifications, the coding template was applied to the entire data set. Coding themes were compared and discussed between the primary investigator and the coinvestigator until consensus was reached. 

E. Ethics

Waiver for approval was granted by the Singhealth Institutional Ethics Board. The primary investigator was a core faculty within the Singhealth Emergency Medicine Residency Program and although the interview was conducted among EM residents the primary investigator did not conduct the interviews when the interviewees were from the same team as the primary investigator. These were conducted by the coinvestigator. The coinvestigator was an EM Senior Resident who was not involved in the FT system. A reflexivity diary was kept, and peer debrief was done.

III. RESULTS

Three main themes emerged on how working in FTs affected informal learning amongst our participants (Figure 1). These included changes in relationship dynamics between members, effect of FT composition on informal learning of the participants and influence on motivation.

Figure 1. Themes and sub-themes

Theme 1: Changes in Relationship Dynamics between FT Members

From the interviews, the participants felt that the FT system resulted in more familiarity, trust, teamwork and improved communications between team members including SEPs, Residents and MOs. Interviewees felt that this strengthened the relationship dynamics between FT members as compared to a NFT. This meant that FT members were able to coordinate and exchange information better. It led to an increase in familiarity in knowing each member’s style of practice and way of thinking. The team members could understand each other better and how they reasoned.

The strengthened relationship between residents and SEPs changed with dynamics. Having a “closer rapport” and “deeper bond” allowed the residents to “tag along” with SEPs “more often” and gave the residents more insight as to why the SEPs behaved in a certain way as to “how they had practiced medicine” and the rationale behind each step was “more easily communicated to the residents who were tagging along” (Resident A), resulting in residents having a deeper understanding of why things were done in addition to how things were done. This strengthened relationship was also present between the residents and their peers. Therefore, peer learning improved within the FTs as junior residents reported feeling less reservation in asking each other questions. 

Resident D felt that peer learning was better within FTs because of the improved relationship, there were less reservations which had prevented him from asking his peers questions in a NFT setting. 

He elaborated:

“Fixed team [was] definitely better for peer learning. For the same reasons, because you know each other better, you’re more familiar. We don’t only talk to each other about work… after a while, when you go for meals together… or rather like resting together and no cases around you sit and talk. So there’s a lot more familiarity with the person you’re working with, and… you’re just more comfortable with asking questions… you don’t feel like this is somebody who’s going to judge you if [you] asked a stupid question”

This was not just amongst the residents but also with SEPs. Resident G felt that it was easier for the residents to speak to the SEPs because of familiarity and resulted in less workplace stress:

“Over the time as we knew each other better … the workplace stress was much lower… so I could… work with less stress at workplace… Because if you didn’t really know the consultant you tend to be afraid to talk to them; then of course your stress levels will be higher. But if you know that consultant and you know what kind of person, he or she is then you could be more comfortable to talk to them…”

“…It is more comfortable to… approach the senior because you know every day… we have a fixed team so naturally we feel that our relationship is closer…”

“… so, I won’t be too afraid to speak out or to talk to them to discuss with them.”

Contributing to additional ways of informal learning, communication within FTs even during work took on a more “friendly [and] social” form with greater congeniality and via more “communication platforms” (WhatsApp and Tiger Text) which continued even “outside of work” (Resident H). These platforms were also used as learning tools to facilitate case discussions, share learning points and experiences. This was not previously present in the NFTs. The residents felt that learning was more customised because of the change in relationship dynamics. In FTs, there was closer supervision of residents by SEPs. The residents “spent more time” with the same group of supervisors, thus the supervisors were able to better “assess both strengths, weaknesses and address any particular loopholes” of the residents (Resident A). 

However, there were some adverse effects of this change in relationship dynamics. Some residents felt that with a closer relationship between team members, supervisors sometimes were more tolerant of the learners’ shortcomings and be less likely to point it out because of not wanting to affect the relationship. This closer relationship could result in residents taking “shortcuts” and “flying under the radar” because they knew the SEPs could tolerate or would not scrutinise the learners closely once “trust” was established (Resident L). Resident H gave an example of how familiarity could lead to less critical thinking by the learner:

“If… the senior always does like… [Rapid Sequence Intubation] … even though I question the first time I saw him do that… subsequently … every time I work with him I will do [it] this way. I won’t really think does the patient really need this way [of management] or will the patient benefit in a different way… if you are working with different bosses then every case you need to restart your thinking…”

Theme 2: Effect of Composition of the FT

All interviewees felt that the composition of the FT had affected informal learning, and that having a balanced team in terms of a wide range of years of practice amongst both the SEPs as well as the JDs would help improve informal learning. Having a team where the JDs were of differing seniorities of practice could help with peer learning because the senior ones could help the junior ones more. This also applied to the SEPs as that provided a wider perspective to clinical issues due to having different clinical experience and expertise in different subspecialties. Furthermore, if the JDs in a FT were of a similar level of practice, Resident C expressed that they could be “competing with each other for cases and procedures” which adversely affected informal learning with fewer opportunities to perform procedures. In a NFT the members would be constantly changing and it would be unlikely its JD would be always of the same level of practice.

The interviewees expressed concern that within a FT system that, although residents had close contact with a fixed group of supervisors, they lacked contact with the other teams’ SEPs. Many residents felt that this had adversely impacted informal learning because the SEPs were experts in different subspecialties (e.g., Trauma, Toxicology, Ultrasound, etc). By not interacting with many SEPs, residents were unable to learn from them. Furthermore, different supervisors had different perspectives and approaches to patients which the residents may not be exposed to if they were not in the same team as these seniors. However, this was mitigated by having a team with a balanced variation in the areas of specialty of the seniors. Resident F summarised this:

“…a team with… people from different seniorities are essential… (even) juniors can teach seniors… the way that my team was composed… it was a good mix… there are people from different… specialties… from different seniorities even within the juniors … like first posting to… [senior post graduate years]… offers different perspectives, learning of different things… people from different [subspecialties] can offer insight into the systems-based learning or component from other parts of the institution…”

Theme 3: Influence of a FT System on Motivation

Many residents felt that having FTs increased their motivation to learn. This resulted from their supervisors being able to inspire them and follow up with their learning progress more closely. Resident M was motivated to learn because his “friends (FT members) were very motivating” and “enthusiastic”; this encouraged him to learn more. Furthermore, resident C felt more motivated to learn in a FT because he “always sees the same senior” and this more frequent contact results in him being “more likely to take their feedback and opinion more seriously and work harder”

However, “after some time everyone is comfortable with each other” and some participants feared that their motivation “might go down” (Resident N). This was because there was a feeling of complacency as time went on within a FT, hence the motivation to learn started to dwindle after an initial increase.

Other reasons for this decline were related to COVID-19, the focus was more on facing the threat rather than learning and the priority to learn was secondary. The motivation to learn “was a bit less” as “the mood was more to survive than to learn”; Resident L was “less driven to learn” because there was a “general bleakness in the whole situation of [COVID-19] which made his “inner desire to learn… wane a bit”

IV. DISCUSSION

This study explored how working in fixed clinical teams affected informal learning for EM Residents. There are many pros and cons to fixed team rostering however the focus of this study is on informal learning. The findings highlighted the importance of having a balanced team composition where team members were able to establish trust and a strong bidirectional relationship because of the longer time spent working together. Motivation to learn increased initially; however after some time, some felt a decrease. This was consistent with prior work which highlighted team dynamics and commitment and that feedback which was given often and in a socially interactive environment were factors which helped to enable effective work-based learning (Attenborough et al., 2019; Jeong et al., 2018; Kyndt et al., 2016). Unlike in Attenborough’s work where team leadership was one of the factors identified; our respondents did not mention the effect of team leadership on informal learning. Their focus was more on the relationships between the different team members. From this study the predominant factors which positively affected informal learning included teamwork, collaborative task performance, where good communication was needed between different people, and personal development especially in building interpersonal relationships and group decision making.

Limited studies were done on how FTs affect informal learning. Our study found that FTs resulted in more (informal) communication channels (e.g. WhatsApp) being formed which was not present in NFTs, resulting in more learning activities including sharing ideas, resources and experiences. These sharing activities were some of the major forms of informal learning activities identified in the literature (Lohman, 2006). FTs resulted in open communication and quality feedback which was well received, and were found to be beneficial towards informal learning (Jeong et al., 2018). Our study showed that working in a FTs led to more customised learning. Findings of improved communications and strengthened relationships in a bidirectional manner involving teacher and learner alike, supports a shift from a predominantly teacher to learner type of dynamics to a team learning dynamics where all team members can learn with and from one another. This is important because informal learning takes place effectively when learning from past mistakes and feedback exchange occurs, involving both cognitive and social interactions (Jeong et al., 2018).

FTs had negative effects on informal learning as well. Familiarity resulted in supervisors being more tolerant of shortcomings and FTs limited learners’ contact with other teams’ supervisors and adversely affected informal learning. This was because informal learning also takes place when there is sharing of ideas, expertise and experience (Lohman, 2006) and limiting the number of supervisors limits the variance of shared viewpoints.

Our study has limitations. Firstly, interviewee recall bias was possible because 6 months had passed after the FT system was stopped before the interviews. Therefore, some details may not have been accurately recalled given this period which could affect the trustworthiness of results. Secondly, the participants were likely to be comparing their experiences in the FT system during COVID-19 to a NFT system without a pandemic. Thus, some of the experienced changes may be because of the pandemic rather than purely due to the FT system. Thirdly, there are many pros and cons to FTs however the focus of this study is on informal learning thus other factors not investigated with this study may affect the feasibility of FT.  Lastly there could be power differential effects between the interviewers and the interviewees because the interviewers performed supervisory and roles to the residents. However, to mitigate this, a reflexivity diary was kept, and peer debrief between the two interviewers was performed. Furthermore, the interviewers did not interview members who had been in the same team as them.

V. CONCLUSION

In conclusion, FTs impact informal learning by building strong relationships with improved team communications and adding a social dimension for learning. A balance of team members as well as rotating the residents across different FTs may be beneficial for improving informal learning for EM Residents.

Notes on Contributors

Dr Wee Choon Peng Jeremy submitted the CIRB application, (with the help of the last author) conceptualised the study and its design. He performed the literature review, recruited and interviewed the participants, collected and transcribed the data, performed the thematic analysis of the data and wrote the manuscript. 

Dr Ng Mingwei helped to recruit and interview some of the participants, transcribed and collected the data. Dr Ng helped perform the thematic analysis of the data and helped edit the manuscript.  

Prof. Dr. Pim Teunissen was central to the conceptualisation of the study, advised on the design of the study and gave critical feedback to the writing of the manuscript and edited the manuscript extensively

All the authors have read and approved the final manuscript. 

Ethical Approval

We have included the letter for waiver of CIRB via email. Ethics approval for waiver of written informed consent was obtained from the Singhealth Institutional Review Board (CIRB Ref: 2020/3114).

Data Availability

As the data set is qualitative in nature, we are not able to upload that in any public repository.

Funding

There is no funding for this paper/study.

Declaration of Interest

The authors report there are no competing interests to declare. 

References

Attenborough, J., Abbott, S., Brook, J., & Knight, R. A. (2019). Everywhere and nowhere: Work-based learning in healthcare education. Nurse Education in Practice, 36, 132-138. https://doi.org/10.1016/j.nepr.2019.03.004

Barker, J. M., Clothier, C. C., Woody, J. R., McKinney, E. H., Jr., & Brown, J. L. (1996). Crew resource management: A simulator study comparing fixed versus formed aircrews. Aviation, Space, and Environmental Medicine, 67(1), 3-7.

Billett, S. (2007). Constituting the workplace curriculum. Journal of Curriculum Studies, 38(1), 31-48. https://doi.org/10.1080/00220270500153781

Brooks, J., McCluskey, S., Turley, E., & King, N. (2015). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology, 12(2), 202-222. https://doi.org/10.1080/14780887.2014.955224

Burke, L. A., & Hutchins, H. M. (2016). Training transfer: An integrative literature review. Human Resource Development Review, 6(3), 263-296. https://doi.org/10.1177/1534484307303035

Callanan, M., Cervantes, C., & Loomis, M. (2011). Informal learning. Wiley Interdisciplinary Reviews Cognitive Science, 2(6), 646-655. https://doi.org/10.1002/wcs.143

Eddy, K., Jordan, Z., & Stephenson, M. (2016). Health professionals’ experience of teamwork education in acute hospital settings: a systematic review of qualitative literature. JBI Database of Systematic Reviews Implementation Reports, 14(4), 96-137. https://doi.org/10.11124/JBISRIR-2016-1843

Eraut, M. (2010). Informal learning in the workplace. Studies in continuing education, 26(2), 247-273. https://doi.org/10.1080/158037042000225245

Gjeraa, K., Spanager, L., Konge, L., Petersen, R. H., & Ostergaard, D. (2016). Non-technical skills in minimally invasive surgery teams: A systematic review, Surgical Endoscopy, 30(12), 5185-5199. https://doi.org/10.1007/s00464-016-4890-1

Jeong, S., Han, S. J., Lee, J., Sunalai, S., & Yoon, S. W. (2018). Integrative literature review on informal learning: Antecedents, conceptualizations, and future directions. Human Resource Development Review, 17(2), 128-152. https://doi.org/10.1177/1534484318772242

Kyndt, E., & Baert, H. (2013). Antecedents of employees’ involvement in work-related learning. Review of Educational Research, 83(2), 273-313. https://doi.org/10.3102/0034654313478021

Kyndt, E., Vermeire, E., & Cabus, S. (2016). Informal workplace learning among nurses. Journal of Workplace Learning, 28(7), 435-450. https://doi.org/10.1108/JWL-06-2015-0052

Lake, M. A. (2020). What we know so far: COVID-19 current clinical knowledge and research. Clinical Medicine, 20(2), 124-127. https://doi.org/10.7861/clinmed.2019-coron

Liu, Z., Teo, T. L., Lim, M. J., Nadarajan, G. D., Segaram, S. S. C., Thangarajoo, S., Wee, L. E., Wee, J. C. P., & Tan, K. B. K. (2020). Dynamic emergency department response to the evolving COVID-19 pandemic: The experience of a tertiary hospital in Singapore. Journal of the American College of Emergency Physicians Open, 1(6),1395-1403. https://doi.org/10.1002/emp2.12264

Lohman, M. C. (2006). Factors influencing teachers’ engagement in informal learning activities. Journal of Workplace Learning, 18(3), 141-156. https://doi.org/10.1108/13665620610654577

Quah, L. J. J., Tan, B. K. K., Fua, T. P., Wee, C. P. J., Lim, C. S., Nadarajan, G., Zakaria, N. D., Chan, S. J., Wan, P. W., Teo, L. T., Chua, Y. Y., Wong, E., & Venkataraman, A. (2020). Reorganising the emergency department to manage the COVID-19 outbreak. International Journal of Emergency Medicine, 13(1), Article 32. https://doi.org/10.1186/s12245-020-00294-w

Rogoff, B., Callanan, M., Gutiérrez, K. D., & Erickson, F. (2016). The organization of informal learning. Review of Research in Education, 40(1), 356-401. https://doi.org/10.3102/0091732×16680994

Stepaniak, P. S., Heij, C., Buise, M. P., Mannaerts, G. H., Smulders, J. F., & Nienhuijs, S. W. (2012). Bariatric surgery with operating room teams that stayed fixed during the day: A multicenter study analyzing the effects on patient outcomes, teamwork and safety climate, and procedure duration. Anesthesia and Analgesia, 115(6), 1384-1392. https://doi.org/10.1213/ANE.0b013e31826c7fa6

Telio, S., Ajjawi, R., & Regehr, G. (2015). The “Educational Alliance” as a framework for reconceptualizing feedback in medical education. Academic Medicine, 90(5), 609-614. https://doi.org/10.1097/ACM.0000000000000560

Varpio, L., Ajjawi, R., Monrouxe, L. V., O’Brien, B. C., & Rees, C. E. (2017). Shedding the cobra effect: Problematising thematic emergence, triangulation, saturation and member checking. Medical Education, 51(1), 40-50. https://doi.org/10.1111/medu.13124

Vassos, S., Harms, L., & Rose, D. (2017). Supervision and social work students: Relationships in a team-based rotation placement model. Social Work Education, 37(3), 328-341. https://doi.org/10.1080/02615479.2017.1406466

Watkins, K. E., Marsick, V. J., Wofford, M. G., & Ellinger, A. D. (2018). The evolving Marsick and Watkins (1990) theory of informal and incidental learning. New Directions for Adult and Continuing Education, 2018(159), 21-36. https://doi.org/10.1002/ace.20285

World Health Organisation. (2020). World Health Organisation emergencies press conference on coronavirus disease outbreak – 11 March 2020. https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020

Zoom. (2016). Security guide. Zoom Video Communications Inc.

*Wee Choon Peng Jeremy
Outram Road,
Singapore 169608
Email: jeremy.wee.c.p@singhealth.com.sg

Submitted: 26 November 2021
Accepted: 21 July 2022
Published online: 3 January, TAPS 2023, 8(1), 33-42
https://doi.org/10.29060/TAPS.2023-8-1/OA2712

Jaime Maria Tan¹, Junaidah Binte Badron¹ & Sashikumar Ganapathy^1,2

¹Department of Emergency Medicine, KK Women’s & Children’s Hospital, Singapore; ²Duke-NUS Medical School, Singapore

Abstract

Introduction: Perceptions towards the working and learning environment as well as coping mechanisms have been studied across different healthcare sectors. They have shown to reduce stress and burnout. However, perceptions of the work environment in the Emergency Department (ED) setting have not been studied in depth. The literature surrounding coping mechanisms also mostly focuses on their impacts rather than the mechanisms utilised. In addition, these were often investigated using surveys. This study aimed to use a phenomenological approach to explore the perceptions and coping strategies of junior doctors working in a paediatric ED.

Methods: Sixteen junior doctors working in the Paediatric ED were recruited. Semi-structured interviews were conducted after conducting literature reviews. Data was collected until saturation point. All interviews were recorded and transcribed verbatim manually and subsequently analysed.

Results: The greatest fears of junior doctors starting their paediatric emergency posting were lack of knowledge due to inexperience in the subspecialty; fear of the work environment due to unfamiliarity as well as workload and the intrinsic high-stress environment. The main coping strategies were ensuring clinical safety, obtaining psychosocial support from loved ones and colleagues, and placing focus on spirituality and wellbeing.

Conclusion: In this study, the perceptions and coping strategies of the junior doctors in the Paediatric ED were explored. The findings from this study will help to structure and improve the support given to future junior doctors who rotate to the department as well as better orientate them to allay their pre-conceived notions.

 Keywords:          Coping Behaviours, Perceptions, Paediatric Emergency Department, Stressors, Interviews

Practice Highlights

• Participants worried about knowledge, workload and responsibilities prior to starting their posting.
• Perceptions were mostly of an anticipatory nature, influenced by seniors’ past experiences.
• In work, support from senior staff was helpful in allaying their fears and increasing patient safety.
• Participants felt psychosocial support, spirituality and wellness were useful coping strategies.
• Maladpative coping strategies did not come up as a main theme in our study.

I. INTRODUCTION

Perceptions toward the work and learning environment can strongly impact experiences and even lead to large amounts of stress (Chan et al., 2016). A poorer perception of the learning environment is also associated with greater levels of burnout (Chew et al., 2019; Sum et al., 2019). Conversely, a positive perception of the work environment helps to alleviate stress (Abraham et al., 2018). Workers’ perception of their work environment contributes significantly to their overall experiences.

Main factors contributing to stress in the ED include heavy workload and critically ill patients. Workplace violence, trauma, abuse and morbidity also add to the stress and burnout experienced (Burbeck et al., 2002; Copeland & Henry, 2018; Healy & Tyrrell, 2011; Xu et al., 2019). In the paediatric setting, added stressors include dealing with sexual abuse and non-accidental injury as well as death and the inability to provide optimal care for children (Alomari et al., 2021; Basu et al., 2016; Durand et al., 2019; Greenslade et al., 2019; Shanafelt et al., 2012; Watson et al., 2019).

Given these significant stressors, individuals utilise different coping mechanisms to mediate these experiences (Howlett et al., 2015).

Some coping strategies discussed in previous studies include socialising with friends and family (Gribben et al., 2019). Focusing on physical wellbeing, clinical variety, reflectivity, and organizational activities were also helpful in alleviating burnout in other areas of healthcare (Barham et al., 2019; Koh et al., 2015).

Several studies also found that the use of maladaptive coping mechanisms, such as alcohol use and self-blame increased with the frequency of burnout (Jackson et al., 2016; Oreskovich et al., 2015; Ryali et al., 2018; Talih et al., 2018).

While many studies studied stressors and the effectiveness of the coping mechanisms used, the actual components of coping mechanisms were not well studied. In studies that did look at coping mechanisms and their effectiveness, these studies were also often done via the survey method and were only evaluated on the surface.

Most studies looked at healthcare workers in general. Few studies looked solely at the doctor population. This makes conclusively evaluating the doctor component of coping mechanisms and their effects difficult.

While other studies looked at an adult emergency perspective, there were also few studies looking at the paediatric ED. It has been reported that dealing with paediatric emergencies causes more stress compared to their adult equivalents. Some of the contributing factors are related to the nature of working with children. These, in itself, are unmodifiable (Guise et al., 2017). Therefore, it is important to study how the paediatric context can affect the experiences of the doctors who care for them.

In our study, we studied the perceptions of junior doctors at the beginning of their posting. We subsequently explored their coping mechanisms in the Singaporean context.

The element of stress in the ED among junior doctors is significant as the ED is often part of many specialist training pathways (Mason et al., 2015). During the time of training, the doctors are still learning and developing. Hence, many doctors experience sharp learning curves during their postings. This brings about more stress (McPherson et al., 2003). In some cases, the stress can even lead to doctors thinking about leaving clinical practice altogether (Degen et al., 2014).

In the Singaporean context, paediatric emergency postings are part and parcel of speciality training for junior doctors (especially for those in emergency medicine and family medicine training). Because of this, junior doctors spend the majority of their paediatric postings in the paediatric emergency. As such, a Singapore-specific context would give light on the challenges of this sizable group.

The nature of the healthcare system in Singapore is unique. Up to 60% of the consultations in the paediatric ED were for nonurgent conditions due to the overall perception of the severity of symptoms and parental preference towards paediatric specialist facilities (Ganapathy et al., 2015). This would lead to an increased workload for the paediatric ED. The distribution in workload may also differ compared to a global perspective, with the load of severe paediatric trauma in Singapore being low (Pek et al., 2019).

These subtleties in the paediatric ED in Singapore can influence the experiences of junior doctors differently. With these key differences in mind, we aimed to investigate the perceptions of junior doctors towards their paediatric ED posting in Singapore and how they subsequently coped with the challenges faced.

II. METHODS

A. Design

In this study, we examined the experiences of doctors in their paediatric ED rotation and how their thoughts and actions influenced their stress during their rotation. We deemed the phenomenological approach to be the most appropriate for this study. Phenomenology is defined as the study of how individuals see and experience a phenomenon and what this means to the individuals in their own experience (Neubauer et al., 2019; Smith, 2021).

The approach we chose was that of an interpretive phenomenological analysis in which we aimed to investigate the experience through the participants’ own experiences and perceptions. With the help of the various participants’ accounts, themes and ideas bound by their experiences were explored (Tuffour, 2017).

B. Methods

The members of the study performed a preliminary literature review on the topic and explored plausible methods of data collection. The study team decided on semi-structured interviews as it promotes sharing and would allow for sufficient privacy.

The team members included a senior consultant, a staff physician and a medical officer. Together, after discussions about concepts that the team was keen to explore, an interview guide was drawn up.

Subsequently, a proposal was submitted to the Hospital Centralised Institutional Review Board for approval.

One-on-one interviews were conducted with the participants by investigator A, a medical officer who was rotating within the department at the time of the study. This was done to reduce the power differential. Interviews were conducted at a location and time convenient to the participant.

Prior to the interviews, consent was sought and all interviews were recorded and subsequently transcribed verbatim. The interviews were conducted over a 1-month period in December 2019.

Questions were open-ended and allowed participants to share ideas that they were keen to raise with no restrictions to the topics brought up. Interview questions were tweaked alongside subsequent interviews so that they were easier to understand and would encourage sharing. Additional questioning in subsequent interviews was adopted to improve clarity. For example, one of the questions that featured early in the interviews was ‘What are some of the coping mechanisms you use?’ During subsequent interviews we noticed some participants utilised coping mechanisms before work to prepare themselves, some used other strategies during work to cope with the stress, while others dealt with their stressors after getting off work. We tweaked the question to include ‘during the shift or outside of the shift’ to help participants widen their perspective about certain coping methods they may have used but were not immediately conscious of when answering the questions. No new questions referring to particular themes were inserted although interviewers were aware of the themes that had been highlighted in previous interviews. This was done in addition to the initial interview guide and ensured the broad nature of questioning was not compromised and the breadth of interviews was maintained.

Themes were identified from the interviews until data saturation was reached. Data saturation was noted at the 12th interview. The team continued to learn from subsequent interviews, with interviews contributing additional depth to the issues explored. Further interviews were conducted to confirm that no new theme was being identified.

The interviews were then transcribed and de-identified. They were subsequently reviewed by 2 reviewers (Investigator A and Investigator B). Data was analysed using a step-by-step thematic analysis method (Braun & Clarke, 2006). Investigators A and B independently analysed the transcripts, identified themes and later reported the  common themes. These themes were discussed for concurrence. When any differences in opinion arose, these would be reviewed by investigator C to resolve any disagreement.

C. Setting

The research was conducted within the Paediatric ED in KK Women’s and Children’s Hospital, a tertiary paediatric hospital.

The Children’s Emergency of KK Women’s and Children’s Hospital is the largest paediatric emergency unit in Singapore. During the time of the interviews, the department treated over 400 patients daily. The Children’s Emergency sees all children under the age of 18 years for all medical complaints.

The department is staffed by over 60 junior doctors at a single time. These junior doctors come from various backgrounds and pass through the department for varying amounts of time. Thus, their experience can be very heterogeneous.

The job scope and responsibilities of all the junior doctors are primarily the same despite the different levels of experience. They are expected to treat the patients that present to the ED. These doctors can seek advice from the senior doctors who are on the ground. However, for the majority of the time, they would be tasked to treat patients on their own.

D. Participants

Participants were recruited through an email that was circulated to all junior doctors in the department. Participation in the study was voluntary and participants were not remunerated.

A total of 16 junior doctors were recruited and interviewed over a 1-month period. Due to the busyness of the ED and the limited time frame in which the interviews were conducted, only 16 interviews were conducted. Convenience sampling was chosen for the sampling method. The first 16 volunteers who had volunteered were interviewed. However, it was noted that saturation point was reached prior to the conclusion of the interview process.

The variety within the ED was well represented. The details of participant breakdown are elaborated on in Table 1.

Submitted: 19 May 2022
Accepted: 21 September 2022
Published online: 3 January, TAPS 2023, 8(1), 3-12
https://doi.org/10.29060/TAPS.2023-8-1/OA2817

Junji Haruta^1,2, Ayumi Takayashiki², Ryohei Goto², Takami Maeno², Sachiko Ozone² & Tetsuhiro Maeno²

¹Medical Education Center, School of Medicine, Keio University, Japan; ²Department of Primary Care and Medical Education, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan

Abstract

Introduction: The pandemic caused by the novel coronavirus (COVID-19) has produced dramatic changes in the learning environment for clinical practice in medical education, and the impact on medical students has been significant. However, few program evaluation studies involving professional identity, now emphasised in medical education, have compared current with pre-COVID-19 programs. Here, we compared the professional identity recognised by medical students in the years before and during COVID-19.

Methods: Medical students who participated in clinical practice were evaluated using the Japanese version of the Professional Self-Identity Questionnaire (PSIQ), a 9-item inventory rated using 7-point Likert scales. They answered on the first day of a 4-week clinical practice module within a community-based medical education (CBME) program and at graduation in 2018-2019 (pre-COVID-19 pandemic) and in 2019-2020 (during-COVID-19 pandemic). We compared the mean difference in total PSIQ score of the 2019-2020 students to those of the 2018-2019 students as a historical cohort using an unpaired t-test.

Results: Participants were 104 medical students in the 2018-2019 academic year and 92 in the 2019-2020 academic year. PSIQ was increased at graduation. Mean difference in total PSIQ score was statistically higher for the 2019-2020 students (13.5 ± 9.4) than for the 2018-2019 students (10.3 ± 8.1) (t =2.6, df =195, p = 0.01).

Conclusion: Although clinical practice was restricted by COVID-19, this had some positive impact on the strengthening of professional identity recognised by medical students. This finding may have been influenced by the imprinting of professional norms and changes in society.

Keywords:           Novel Coronavirus Infection, Professional Identity, Clinical Practice, Questionnaire Survey, Program Evaluation

Practice Highlights

• Professional identity recognised by medical students in Japan was assessed by questionnaire.
• Changes during clinical practice were compared between two grades using a historical cohort.
• Medical students’ recognised professional identity scores improved after one year in both cohorts.
• Medical students experienced the COVID-19 pandemic showed some positive impact on professional identity.
• Medical students internalised their role as healthcare professionals owing to COVID-19 pandemic.

I. INTRODUCTION

The pandemic caused by the novel coronavirus (COVID-19) has caused unprecedented disruption to medical education systems around the world (Kaul et al., 2021; Khasawneh et al., 2020; Lucey & Johnston, 2020; Papapanou et al., 2021). In particular, the pandemic has made it difficult to continue medical educational programs, including regular lectures, and face-to-face lectures, experiments, and clinical practice involving patients (Al Samaraee, 2020; Emanuel, 2020; Shankar & Wilson, 2020). The Association of American Medical Colleges took the unprecedented decision to suspend clinical practice and issued guidance instructing medical students to avoid clinical practice involving direct patient contact (Association of American Medical Colleges, 2020). The many university hospitals in the U.S. have made it a priority to utilise students in controlling the spread of COVID-19 infection. Other countries followed this precedent. Medical students accordingly lost the opportunity to learn through patients in clinical practice. In Japan also, clinical practice was suspended at many universities.

Commonly implemented alternative teaching methods were online interactive discussions and video lectures (Chiodini, 2020; Mian & Khan, 2020). To prevent the spread of COVID-19 from medical students in clinical practice (Alsoghair et al., 2021), students were required to socially distance not only from patients but also from healthcare providers and faculty members. However, implementing the requisite changes to curricula, such as interrupting or drastically postponing clinical practice, was difficult, forcing faculties to develop curricula that provided students with opportunities to supplement their ongoing clinical practice as early as possible, while making use of online devices (Ross, 2020). These changes in the learning environment affected medical students’ learning. For example, a survey of 2nd-6th grade students at the University of Geneva found that 2/3 of the medical students experienced decreased motivation and concentration in distracting learning environments such as home, whereas some medical students who were exposed to clinical practice during the COVID-19 pandemic showed an increased sense of belonging to the profession (Wurth et al., 2022). A change in perception with remote learning in response to the COVID-19 pandemic among Canadian undergraduate students also revealed a significant decrease in students’ achievement goals, engagement, and perceptions of success, and an increase in perceptions of cheating (Daniels et al., 2021). Perceptions of learning due to changes in the curriculum due to COVID-19 have been investigated and controversially reported, but few reports have investigated professional identity as a foundation for becoming a physician.

To be effective clinicians, it is important that medical students acquire not only knowledge and skills but also professional identity formation (Cruess et al., 2014; Hafferty & Franks, 1994). Professional identity is defined as “attitudes, values, knowledge, beliefs, and skills shared with others in a professional group” (Crossley & Vivekananda-Schmidt, 2009). Professional identity formation is noted as an ongoing process influenced by several factors, including practice experience and professional socialisation (Adams et al., 2006). In this study, we adopted a theoretical framework which is based on a concept that illustrates the interplay of professional identity formation and socialisation through a complex conscious and unconscious process centered on role models, mentors, and the accumulation of individual experiences (Cruess et al., 2015). Because clinical practice is a period in which medical students meet role models and mentors and gain their first experiences as a healthcare provider, clinical practice is major contributor to the development of professional identity in medical students (Haruta et al., 2020).

Although studies have reported that medical education programs altered by the COVID-19 pandemic have affected medical students’ perceptions of the role of the physician and their learning strategies (Findyartini et al., 2020; Wurth et al., 2022), little is known about the influence of curricula changes on medical students with regard to their professional identity formation as physicians affected by role models, mentors, and individual experience (Stetson et al., 2020).

Here, we aimed to compare professional identity as a physician as perceived by medical students in clinical practice before and during COVID-19 using the Professional Self Identity Questionnaire (PSIQ).                                                                                                     

II. METHODS

A. Setting

Japan first faced the challenge of the COVID-19 pandemic in January 2020, and a state of emergency was declared for all regions of Japan on April 16, 2020 (Prime Minister’s Office of Japan, 2020). During that time, some medical universities suspended clinical training and implemented online education instead. The COVID-19 Outline of Measures for COVID-19 formulated by the Japanese Ministry of Education, Culture, Sports, Science and Technology was sent to all higher education institutions, including medical schools, on June 5, 2020 (Hayashi et al., 2022), with the main advice that universities should operate in a resilient manner for experiential educational practices such as clinical practice. Under these circumstances, leaders of Japanese medical schools were asked to consult various stakeholders and make decisions on whether or not to continue clinical practice, often leaving detailed instructions to local operations managers in each field (Hayashi et al., 2022).  In contrast, primary care physicians were forced to adapt to local conditions (Haruta et al., 2021).

Students enter medical school in Japan after graduating from high school, and the curriculum is six years long. Clinical training usually takes place in the fourth to sixth years. This schedule is also implemented at the University of Tsukuba in Japan: clinical training is divided into a first half from October of the fourth year to September of the fifth year and a second half from October of the fifth year to June of the sixth year, for a total of 78 weeks. After clinical training, medical students spend time studying for graduation and national exams, in addition to attending lectures.

The first phase of this clinical practice consists of a four-week rotation in basic departments such as internal medicine, surgery, obstetrics and gynecology, pediatrics, and emergency medicine, mainly at affiliated hospitals. The second phase incorporates some elective subjects and includes 4 weeks’ mandatory clinical practice in a CBME curriculum and a further 4 weeks’ clinical practice in a department of interest. Typically, 15 to 17 students rotate every 4 weeks through clinical practice in the CBME curriculum. This clinical training was conducted before the pandemic, but was discontinued in response to it at the end of March 2020. In March, medical students continued clinical training while all elementary and junior high schools were closed. All clinical training was then cancelled in April 2020 and replaced with report assignments only. These were replaced in May and June with online training and on-demand assignments. Sixth-year students in the class of 2020, and subsequently also in 2021 and 2022, were required to adhere to health observation recording and infection prevention behaviors in accordance with the University of Tsukuba Hospital staff behavior guidelines, which had not been previously followed. Similarly, they were also restricted from participating in training hospital tours and extracurricular activities, which were usually a part of their training.

B. Study Design

Training in clinical practice at the University of Tsukuba consists of a first phase for 4th and 5th year students and a second phase for 5th and 6th year students. CBME is included in the latter phase. Data from 5th-year medical students who participated in clinical practice in a 4-week community-based medical education (CBME) program from October 2019 to March 2020 and who graduated in March 2021 were compared to those of the previous year’s students (2018-2019) as a historical cohort. This cohort included 6th-year medical students who graduated in March 2020 and who had participated in a similar clinical practice program the previous year, from October 2018 to May 2019. As in the 2018-2019 academic year, a questionnaire was administered using the PSIQ in the 2019-2020 academic year. The PSIQ, conducted as part of program evaluation of the CBME program, was administered on the first day of the 4-week clinical practice of the program and at graduation. Study participants included 118 fifth- and sixth-year medical students who participated in clinical practice in the CBME program from October 2018 to May 2019 (pre-COVID-19 pandemic) and 100 fifth- and sixth-year medical students who participated from October 2019 to March 2020 (during-COVID-19 pandemic). The 35 participants from April-May 2020, when the 4-week clinical practice in the CBME program was switched to online practice due to COVID-19, were excluded because they could not be pre-evaluated.

C. Instrument: Japanese Version of the Professional Self Identity Questionnaire (PSIQ)

The Professional Self-Identity Questionnaire (PSIQ) is a 9-item inventory which is rated using 7-point Likert scales (Crossley & Vivekananda-Schmidt, 2009). This instrument measures the sense of students in identifying their current position on a continuum between ‘first-day student’ and ‘qualified doctor’ when engaged in each of nine domains of professional activity. These nine domains include “Teamwork,” “Communication,” “Conducting assessment,” “Cultural awareness,” “Ethical awareness,” “Using records,” “Dealing with emergencies,” “Reflection,” and “Teaching”. Each item is rated on a 7-point Likert scale. For this research, we set 1 point as “equivalent to the first day of clinical practice” and 7 points as “equivalent to the first day of initial residency”. The PSIQ is an assessment sheet form originally designed in English. The original study was validated in a sample of 496 medical students across multiple phases of education, and the overall internal reliability (Cronbach’s alpha) was reported to be 0.93 (Crossley & Vivekananda-Schmidt, 2009). Previously, we developed a Japanese version of the PSIQ and operationally defined PSIQ scores as professional identity as a physician as perceived by medical students. Cronbach’s alpha of the Japanese PSIQ was reported to be 0.93 (Haruta et al., 2021), which allows students to measure changes in their own professional identity by tracking each of the nine professional activities over time. We set the total score on this scale as the main outcome of this study. The study showed that this was a reliable and valid tool for program evaluation during clinical practice (Haruta et al., 2021).

D. Participants and Data Collection

In 2018-2019 (pre-COVID-19 pandemic) and 2019-2020 (during-COVID-19 pandemic), we assessed the professional identity of medical students from October of their fifth year to May of the sixth year using PSIQ. The survey was administered on the first day of clinical practice in the CBME program as PSIQ 1 (September 2018 to May 2019 and September 2019 to March 2020) and at graduation as PSIQ 2 (2020 January, and January 2021) (Figure 1).

Figure 1. Timing in clinical practice and the PSIQ in 2018-2019 and 2019-2020

	2018-2019 (pre-COVID-19 pandemic) (n=104)	2019-2020 (during-COVID-19 pandemic) (n=92)
Age, mean ± standard deviation	20.6±1.5	21.2±3.4
Gender, number of persons (%)
Male	66 (63.5)	51 (55.4)
Female	33 (31.7)	33 (35.9)
Unknown	5 (4.8)	8 (8.7)

Table 1. Characteristics of medical students included in the analysis

Table 2 shows that the means of total PSIQ scores and each PSIQ item for 2018-2019 and 2019-2020 were increased at graduation. A comparison of the mean for the difference using the unpaired t-test in total PSIQ score for 2018-2019 and 2019-2020 revealed that it was statistically higher for the 2019-2020 students (13.5 ± 9.4) than for the 2018-2019 students (10.3 ± 8.1) (t =2.6, df =195, p = 0.01). However, there were no statistically significant differences using the unpaired t-test with Bonferronni correction at a 0.005 level of significance in mean scores for 9 items of the PSIQ, namely “Teamwork”  (t=0.96 df=195, p=0.341),  “Communication” (t=1.84, df=195, p=0.068),  “Conducting assessment” (t=2.31, df=195, p=0.24) “Cultural awareness” (t=1.41, df=195, p=0.022),  “Ethical awareness” (t=2.25, df=195, p=0.161), “Using records”  (t=2.67, df=195, p=0.026),  “Dealing with emergencies”  (t=2.33, df=195, p=0.008),  “Reflection” (t=2.32, df=195, p=0.021) and “Teaching” (t=2.19, df=195, p=0.030).  The effect size (Cohen’s d) for total PSIQ scores from the first day of clinical practice in the CBME program to graduation was Δ1.3 in 2018-2019 and Δ1.4 in 2019-2020. The effect size (Cohen’s d) for each item of the PSIQ was Δ1.4 in 2018-2019. Range for items was Δ0.8-1.1 in 2018-2019 and Δ0.9-1.2 in 2019-2020.

	Fiscal year (usually April 1 to March 31)	First day of the CBME program (1)	At graduation　(2)	Mean difference in PSIQ　(2)-(1)	t score	Df	p-value	Cohen’s d
Total score 　(Main outcome)	2018-2019 (pre-COVID-19 pandemic)	28.5±7.6	38.8±8.5	10.3±8.1	2.60	195	0.01	1.3
	2019-2020 (during-COVID-19 pandemic)	28.0±6.6	41.6±9.2	13.5±9.4				1.4
Sub-analysis
Teamwork	2018-2019	3.0±1.2	4.4±1.0	1.4±1.2	0.96	195	0.341	1.1
	2019-2020	3.0±1.1	4.6±1.1	1.6±1.3				1.2
Communication	2018-2019	3.7±1.1	4.7±1.1	1.0±1.2	1.84	195	0.068	0.8
	2019-2020	3.5±1.1	4.9±1.1	1.4±1.3				1
Conducting assessment	2018-2019	3.2±1.0	4.4±1.1	1.2±1.2	2.27	195	0.024	1
	2019-2020	3.1±1.0	4.7±1.1	1.6±1.3				1.2
Cultural awareness	2018-2019	2.9±1.1	4.2±1.3	1.3±1.3	2.31	195	0.022	1
	2019-2020	2.8±1.1	4.5±1.3	1.8±1.5				1.2
Ethics awareness	2018-2019	3.4±1.2	4.5±1.1	1.1±1.3	1.41	195	0.161	0.9
	2019-2020	3.5±1.2	4.9±1.1	1.4±1.4				1
Using record entries	2018-2019	3.4±1.0	4.5±1.1	1.0±1.1	2.25	195	0.026	0.9
	2019-2020	3.4±0.9	4.8±1.1	1.4±1.3				1.1
Dealing with emergencies	2018-2019	2.5±1.1	3.8±1.3	1.3±1.2	2.67	195	0.008	1.1
	2019-2020	2.3±1.2	4.1±1.4	1.9±1.7				1.1
Reflection	2018-2019	3.3±1.1	4.3±1.1	1.0±1.3	2.32	195	0.021	0.8
	2019-2020	3.2±0.9	4.6±1.3	1.4±1.5				1
Teaching	2018-2019	3.1±0.9	4.1±1.1	0.9±1.1	2.19	195	0. 030	0.8
	2019-2020	3.2±1.0	4.5±1.2	1.3±1.4				1

IV. DISCUSSION

We found that the means of total and each PSIQ score of medical students who graduated during the COVID-19 pandemic were statistically increased compared to those of students graduating before the pre-COVID-19 pandemic. These findings indicate that, compared with students in the preceding year, the COVID-19 pandemic had a certain positive impact on professional identity formation in medical students with regard to clinical practice.

Considering the framework we used, in which the interaction and reflection of role models, mentors, and individual experiences are key, we believe that all participants were influenced by COVID-19. In addition, professional identity formation is reported to be strongly influenced by environment, context, and other personal variables. (Jarvis-Selinger et al., 2012) The increase in total PSIQ score from the previous year’s medical students may have been due not only to changes in clinical practice, but also to changes in what is expected of medical students by the university hospital and society. In light of these factors, medical students had fewer opportunities to acquire the knowledge and skills required of physicians in the clinical setting directly from role models and mentors, but may nevertheless have been affected by compliance with COVID-19 control measures conducted by the university hospital, and thereby followed the norms of medical professionals communicated through intentional and unintentional messaging. In addition, with the all-engrossing impact of the COVID-19 pandemic throughout society, we speculate that at least some of the students in the COVID-19 group were consulted by family and social contacts as experts and authorities on a daily or more frequent basis, and that such constant reinforcement might also have encouraged PIF. That is, medical students under the COVID-19 pandemic may have internalised the values and norms of the medical professional (Kinnear et al., 2021). Since professional identity formation involves social and relational factors, it is possible that the students’ life and social changes due to COVID-19 reinforced their recognised professional identity (Goldie, 2012).

The more uncertain and ambiguous the period, the more important reflection becomes in PIF. To cope with such uncertainties as the transition from the classroom curriculum to online platforms, rapid changes in the clinical practice environment, and the isolation caused by social distancing, medical students need a well-formed professional identity and socialisation practices (Costello, 2005).  Poor PIF is reported to manifest in burnout (Abedini et al., 2018), and depression and anxiety disorders have increased in COVID-19 (Daniels et al., 2021). Since a core component of PIF is the acceptance of uncertainty and ambiguity, medical educators may have provided reflection opportunities for medical students to consciously consider the social factors manifested in COVID-19, such as individual and professional duties, resource priorities, and health disparities, all of which have a positive impact on PIF and socialisation (Stetson et al., 2020). Alternatively, emotion-focused coping strategies (e.g., arts engagement) and problem-solving strategies (e.g., volunteering) to reduce medical students’ anxiety, fear, and stress may be effective (Moula et al., 2022). Students require time and space to reflect on their emergent identity as professionals in these crisis situations (Moula et al., 2022). The extended time for independent learning may have allowed the students to subconsciously reflect on their professional identity, taking seriously the changes in society caused by COVID-19. PIF scores may have accordingly increased in this study.  

In general, medical teachers should have made efforts to promote medical students’ professional identity formation even during the COVID-19 pandemic (Goldhamer et al., 2020). Professional identity formation is not a process that should be left to chance; it should be explicitly supported through systematic support and curriculum initiatives (Cruess et al., 2015; Schumacher et al., 2012). In reality, it is necessary to require rapid changes in clinical practice to prevent infection caused by COVID-19, and it has been difficult to promote systematic support for professional identity formation for medical students. However, COVID-19 might not have negatively affected the professional identity recognised by medical students who were in clinical practice immediately after the pandemic. The efforts of Japanese medical educators to implement online and other types of learning to replace clinical practice, and the well-considered efforts of faculty to make clinical practice as feasible as possible may have served as role models or mentors, and influenced medical students’ professional identity formation.

This study has three limitations. First, because there was a difference of 8-17 months between the first day of clinical practice in the CBME program and graduation, given that more clinical practice experience may strengthen the professional identity recognised by medical students, it is possible that differences in clinical practice experience may have affected PSIQ scores. Second, extrapolation from this study should be done with care, since it is a comparison of a one-year questionnaire survey at a single university. Third, a degree of response bias may be present due to the self-reporting design, with inclusion of student ID number. Fourth, evaluation at three or more points may allow a more precise comparison of changes, given that professional identity formation is a dynamic process. Even allowing for these limitations, our findings provide meaningful feedback to medical students and faculty, particularly given the paucity of studies evaluating and comparing changes in professional identity formation – which consists of multiple factors – before and during the COVID-19 pandemic. For the future of medical education, it is imperative to track the professional identity recognised by physicians whose student experience in clinical practice was restricted under the COVID-19 pandemic, and was accordingly significantly changed compared with other years.

V. CONCLUSION

Comparison of the professional identity recognised by medical students whose experience of clinical practice was affected by the COVID-19 pandemic to that of students in the previous year showed some positive impact on professional identity. This lack of negative difference in recognised professional identity even under restricted clinical practice may have been influenced by the imprinting of professional norms and changes in society.

Notes on Contributors

JH, AT, SO, TaM and TeM were involved in the conception and design of this study. JH, AT, and TaM collected the data. JH and RG conducted data analysis. JH mainly wrote the paper. AT, SO, TaM and TeM gave critical feedback on the manuscript. All authors have read and approved the final manuscript.

Ethical Approval

This study was approved by the Ethics Committee of the University of Tsukuba’s Faculty of Medicine and Medical Sciences (No. 1329). All participants have given written consent for their data to be used in the research and for publication.

Data Availability

It is not possible to share data since we did not obtain consent from participants to share all data collected.

Acknowledgement

We thank the faculty staff members Shoji Yokoya, Hisashi Yoshimoto, Shogen Suzuki, Naoto Sakamoto, Yu Yamamoto, Yoshihiro Kataoka, Shoichi Masumoto, Haruka Kuno, Takashi Inaba, Shuhei Hamada, Shogo Kawada, and Sayaka Nin of the Department of Primary Care and Medical Education, Faculty of Medicine, University of Tsukuba. Additionally, we thank the healthcare staff who taught the medical students at the CBME program sites in clinical practice.

Funding

This study was funded by the education/research fund of Department of Primary Care and Medical Education, Faculty of Medicine, University of Tsukuba.

Declaration of Interest

There are no companies or other entities with which the authors have conflict of interest relationships that should be disclosed in relation to the content of the paper.

References

Abedini, N. C., Stack, S. W., Goodman, J. L., & Steinberg, K. P. (2018). “It’s not just time off”: A framework for understanding factors promoting recovery from burnout among internal medicine residents. Journal of Graduate Medical Education, 10(1), 26–32. https://doi.org/10.4300/JGME-D-17-00440.1

Adams, K., Hean, S., Sturgis, P., & Clark, J. M. (2006). Investigating the factors influencing professional identity of first-year health and social care students. Learning in Health and Social Care, 5(2), 55–68. https://doi.org/10.1111/j.1473-6861.2006.00119.x

Al Samaraee, A. (2020). The impact of the COVID-19 pandemic on medical education. British Journal of Hospital Medicine, 81(7), 1-4. https://doi.org/10.12968/HMED.2020.0191

Alsoghair, M., Almazyad, M., Alburaykan, T., Alsultan, A., Alnughaymishi, A., Almazyad, S., Alharbi, M., Alkassas, W., Almadud, A., & Alsuhaibani, M. (2021). Medical Students and COVID-19: Knowledge, preventive behaviors, and risk perception. International Journal of Environmental Research and Public Health, 18(2), 842. https://doi.org/10.3390/ijerph18020842

Association of American Medical Colleges. (2020). Guidance on medical students’ participation in direct in-person patient contact activities. https://www.aamc.org/system/files/2020-08/meded-August-14-Guidance-on-Medical-Students-on-Clinical-Rotations.pdf

Chiodini, J. (2020). Online learning in the time of COVID-19. Travel Medicine and Infectious Disease, 34, Article 101669. https://doi.org/10.1016/j.tmaid.2020.101669

Costello, C. Y. (2005). Professional identity crisis: Race, class, gender, and success at professional schools. Vanderbilt University Press.

Crossley, J., & Vivekananda-Schmidt, P. (2009). The development and evaluation of a professional self identity questionnaire to measure evolving professional self-identity in health and social care students. Medical Teacher, 3(12), e603–e607. https://doi.org/10.3109/01421590903193547

Cruess, R. L., Cruess, S. R., Boudreau, J. D., Snell, L., & Steinert, Y. (2014). Reframing medical education to support professional identity formation. Academic Medicine, 89(11), 1446–1451. https://doi.org/10.1097/ACM.0000000000000427

Cruess, R. L., Cruess, S. R., Boudreau, J. D., Snell, L., & Steinert, Y. (2015). A schematic representation of the professional identity formation and socialization of medical students and residents: A guide for medical educators. Academic Medicine, 90(6), 718–725. https://doi.org/10.1097/ACM.0000000000000700

Emanuel, E. J. (2020). The inevitable reimagining of medical education. JAMA, 323(12), 1127–1128. https://doi.org/10.1001/jama.2020.1227

Findyartini, A., Anggraeni, D., Mikhael Husin, J., & Greviana, N. (2020). Exploring medical students’ professional identity formation through written reflections during the COVID-19 pandemic. Journal of Public Health Research, 9(s1), 1918. https://doi.org/10.4081/jphr.2020.1918

Goldhamer, M. E. J., Pusic, M. V., Co, J. P. T., & Weinstein, D. F. (2020). Can covid catalyze an educational transformation? Competency-based advancement in a crisis. New England Journal of Medicine, 383(11), 1003–1005. https://doi.org/10.1056/NEJMP2018570

Goldie, J. (2012). The formation of professional identity in medical students: Considerations for educators. Medical Teacher, 34(9), e641–e648. https://doi.org/10.3109/0142159X.2012.687476

Hafferty, F. W., & Franks, R. (1994). The hidden curriculum, ethics teaching, and the structure of medical education. Academic Medicine, 69(11), 861–871. https://doi.org/10.1097/00001888-199411000-00001

Haruta, J., Maeno, T., Takayashiki, A., Goto, R., Ozone, S., & Maeno, T. (2021). Validation of the professional self-identity questionnaire for medical students during clinical practice in Japan. International Journal of Medical Education, 12, 160–165. https://doi.org/10.5116/ijme.610d.104b

Haruta, J., Ozone, S., & Hamano, J. (2020). Doctors’ professional identity and socialisation from medical students to staff doctors in Japan: Narrative analysis in qualitative research from a family physician perspective. BMJ Open, 10(7), e035300. https://doi.org/10.1136/bmjopen-2019-035300

Hayashi, M., Saiki, T., Kanter, S. L., & Ho, M. J. (2022). Leaders’ perspectives and actions to manage challenges in medical education presented by the COVID-19 pandemic: A nationwide survey of Japanese medical colleges. BMC Medical Education, 22(1), 144. https://doi.org/10.1186/s12909-022-03193-1

Jarvis-Selinger, S., Pratt, D. D., & Regehr, G. (2012). Competency is not enough: Integrating identity formation into the medical education discourse. Academic Medicine, 87(9), 1185–1190. https://doi.org/10.1097/ACM.0b013e3182604968

Kaul, V., Gallo de Moraes, A., Khateeb, D., Greenstein, Y., Winter, G., Chae, J. M., Stewart, N. H., Qadir, N., & Dangayach, N. S. (2021). Medical education during the COVID-19 pandemic. CHEST, 159(5), 1949–1960. https://doi.org/10.1016/j.chest.2020.12.026

Khasawneh, A. I., Humeidan, A. A., Alsulaiman, J. W., Bloukh, S., Ramadan, M., Al-Shatanawi, T. N., Awad, H. H., Hijazi, W. Y., Al-Kammash, K. R., Obeidat, N., Saleh, T., & Kheirallah, K. A. (2020). Medical students and COVID-19: Knowledge, attitudes, and precautionary measures. A descriptive study from Jordan. Frontiers in Public Health, 8, 253. https://doi.org/10.3389/fpubh.2020.00253

Kinnear, B., Zhou, C., Kinnear, B., Carraccio, C., & Schumacher, D. J. (2021). Professional identity formation during the COVID-19 pandemic. Journal of Hospital Medicine, 16(1), 44–46. https://doi.org/10.12788/jhm.3540

Lee, S., & Lee, D. K. (2018). What is the proper way to apply the multiple comparison test? Korean Journal of Anesthesiology, 71(5), 353-360. https://doi.org/10.4097/KJA.D.18.00242

Lucey, C. R., & Johnston, S. C. (2020). The transformational effects of COVID-19 on medical education. JAMA, 324(11), 1033–1034. https://doi.org/10.1001/jama.2020.14136

Mian, A., & Khan, S. (2020). Medical education during pandemics: A UK perspective. BMC Medicine, 18(1), 100. https://doi.org/10.1186/s12916-020-01577-y

Moula, Z., Horsburgh, J., Scott, K., Rozier-Hope, T., & Kumar, S. (2022). The impact of COVID-19 on professional identity formation: An international qualitative study of medical students’ reflective entries in a global creative competition. BMC Medical Education, 22(1), 545. https://doi.org/10.1186/s12909-022-03595-1

Papapanou, M., Routsi, E., Tsamakis, K., Fotis, L., Marinos, G., Lidoriki, I., Karamanou, M., Papaioannou, T. G., Tsiptsios, D., Smyrnis, N., Rizos, E., & Schizas, D. (2021). Medical education challenges and innovations during COVID-19 pandemic. Postgraduate Medical Journal, 98, 321-327. https://doi.org/10.1136/postgradmedj-2021-140032

Prime Minister’s Office of Japan. (2020). [COVID-19] Declaration of a state of emergency in response to the Novel Coronavirus Disease. https://japan.kantei.go.jp/ongoingtopics/_00020.html

Ross, D. A., & the members of National Neuroscience Curriculum Initiative “Quarantine Curriculum” Committee (2020). Creating a “Quarantine Curriculum” to enhance teaching and learning during the COVID-19 pandemic. Academic Medicine, 95(8), 1125–1126. https://doi.org/10.1097/ACM.0000000000003424

Schumacher, D. J., Slovin, S. R., Riebschleger, M. P., Englander, R., Hicks, P. J., & Carraccio, C. (2012). Perspective: Beyond counting hours: The importance of supervision, professionalism, transitions of care, and workload in residency training. Academic Medicine, 87(7), 883–888. https://doi.org/10.1097/ACM.0b013e318257d57d

Shankar, P. R., & Wilson, I. G. (2020). The COVID-19 pandemic and undergraduate medical student teaching-learning and assessment. Canadian Medical Education Journal, 12(1), e190–e191. https://doi.org/10.36834/cmej.70800

Stetson, G. V., Kryzhanovskaya, I. V., Lomen-Hoerth, C., & Hauer, K. E. (2020). Professional identity formation in disorienting times. Medical Education, 54(8), 765–766. http://doi.org/10.1111/medu.14202

Wurth, S., Sader, J., Cerutti, B., Broers, B., Nadia Bajwa, M., Carballo, S., Escher, M., Galetto-Lacour, A., Grosgurin, O., Lavallard, V., Savoldelli, G., Serratrice, J., Nendaz, M., & Audétat-Voiro, M. C. (2022). Correction to: Medical students’ perceptions and coping strategies during the first wave of the COVID-19 pandemic: Studies, clinical implication, and professional identity. BMC Medical Education, 22(1), Article 232. https://doi.org/10.1186/s12909-022-03307-9

*Junji Haruta
35 Shinanomachi Shinjukuku Tokyo
160-8582, Japan
Email: junharujp@keio.jp

Submitted: 12 May 2022
Accepted: 3 August 2022
Published online: 3 January, TAPS 2023, 8(1), 13-24
https://doi.org/10.29060/TAPS.2023-8-1/OA2810

Audrey Lim¹, Vicki Xafis² & Clare Delany³

¹Health and Social Sciences Cluster, Singapore Institute of Technology (SIT), Singapore; ²Graduate School of Humanities and Social Sciences, University of Melbourne, Melbourne, Australia; ³Department of Medical Education, School of Medicine, University of Melbourne, Melbourne, Australia

Abstract

Introduction: Workplace contexts, including political and sociocultural systems influence health professions’ perception and experience of ethical issues. Although established health ethics principles are relevant guiding values, they may be experienced and interpreted differently within different health contexts. How should ethics education account for this? This paper presents ethical dilemmas and concerns encountered by physiotherapists practicing in Singapore and discusses the implications for ethics education.

Methods: Qualitative methods informed by interpretivism and phenomenology were employed. In-depth interviews with 42 physiotherapists from different workplace settings in Singapore were conducted. Participants described everyday ethical challenges they encountered. Inductive content analysis was used to analyse the interview transcript data.

Results: Ethical issues occurred within and across three spheres of ethics:  micro, meso and macro. Ethical issues at the micro sphere centered around physiotherapist-patient relationships, interactions with colleagues, and therapists’ feelings of moral distress. In the meso sphere, ethical challenges related to influences arising from the organizational resources or systems. In the macro sphere, ethical challenges developed or were influenced by sociocultural, religious, economic, and political factors.

Conclusion: The findings reflect current literature indicating that context can influence ethical situations, as experienced and perceived by physiotherapists in their unique settings. Such empirical data might inform the development of ethics curricula to ensure that universal ethical principles are situated within the realities of clinical practice. Locally relevant and realistic ethical case studies will better enable students to recognise and address these situations.

Keywords:           Ethics, Physiotherapy, Health Professions Education, Ethics Education, Asian Context, Singapore, Healthcare Principles, Health Ethics Principles

Practice Highlights

• Context can influence the ethical situations experienced and interpreted by healthcare professionals.
• Contextualised cases studies need to be developed to make ethics real and relevant to students.
• Ethics education should incorporate local context and not focus only on ethics epistemic knowledge.
• Ethics education should incorporate the dynamic influence of macro, meso and micro factors.

I. INTRODUCTION

The established health ethics principles articulated by Beauchamp and Childress (2001): autonomy, beneficence, non-maleficence, and justice guide healthcare practice, including physiotherapy (Carpenter & Richardson, 2008; Edwards, Delany, et al., 2011). Although these principles were originally proposed as universally relevant and foundational to health ethics, health professionals interpret them differently, depending on their professional and personal background, their values, and the culture of the community and settings in which they work (Fuscaldo et al., 2013). In physiotherapy ethics scholarship, there is growing recognition that the universal nature of principlism as a supporting ethical decision-making framework, may not be a sufficient guide for physiotherapists in their ethical decision-making, because of the  plurality of values and diverse contexts of healthcare practice (Carpenter, 2010; Carpenter & Richardson, 2008; Edwards, Wickford, et al., 2011; Fryer et al., 2021; Greenfield, 2006; Hudon et al., 2019; Oyeyemi, 2011; Souri et al., 2020; Sturm et al., 2022). Consequently, there is increasing pedagogical interest in using case studies drawn from everyday practice to bring authenticity and contextual relevance to ethics education (Aguilar-Rodríguez et al., 2019; Fuscaldo et al., 2013).

In physiotherapy, empirical studies have shown that different work contexts, political and sociocultural systems have a direct influence on physiotherapists’ perception and experience of ethical issues (Delany et al., 2018; Fryer et al., 2021; Sturm et al., 2022). A recent example is the study of ethical issues physiotherapists experienced by Sturm et al. (2022). The authors reported physiotherapists working in specific countries described having to compromise their professional integrity due to overt threats and intimidation by suppressive professional organisations or leaders. They were directed to follow the societal or organisational hierarchies or risk jeopardizing their careers. Studies of ethical experiences of physiotherapists practicing in the African nations (Aderibigbe & Chima, 2019; Chigbo et al., 2015; Nyante et al., 2020; Oppong, 2019; Oyeyemi, 2011) and the Greater Middle East region (Edwards, Wickford, et al., 2011; Qamar et al., 2014; Souri et al., 2020) similarly discussed how cultural influences, such as religion or spirituality can directly affect ethical practice and decision-making. Despite increasing empirical evidence globally, the influence of cultural and societal contexts on ethical decision-making and therapists’ interpretation of health values  has not been examined in the East and Southeast Asian regions. There has also been little exploration of how societal and cultural context in which physiotherapy is practiced might be used to inform and shape curricula for ethics education.

In this paper, we present data about the ethical situations physiotherapists in Singapore encounter in their everyday clinical practice. Singapore is multi-racial, multi-religious and multi-cultural. The values of Singaporean people are very much rooted in their Asian heritage, with Confucianism as the prevailing social model (Ong, 2020; Tan, 1989; Yang et al., 2006). This research aims to contribute to knowledge about  the influence of context on clinical ethical issues and decision-making, as interpreted by physiotherapists in Singapore. The empirical data will then be used to inform subsequent ethics curricula. Identifying and analysing the factors influencing ethical issues, as they are experienced and interpreted by physiotherapists in the Singaporean context, is an important pedagogical strategy to inform the development of health ethics curricula.

II. METHODS

We used a qualitative methodology drawing on the research paradigm of phenomenology and conducted in-depth interviews. We recruited physiotherapy clinicians using purposive (Palys, 2008) and snowball sampling (Holloway & Galvin, 2016). Written and verbal consent was obtained. Semi-structured interviews were conducted by AL and ranged from 44 to 123 minutes. Audio recordings were professionally transcribed verbatim and were reviewed by AL. Content analysis was used to analyse the transcribed data, with interviewer’s written field notes as supporting reference. Data was systematically coded and categorised with the aim of identifying themes, their frequency, and relationships, through both description and interpretation (Elo & Kyngäs, 2008; Hsieh & Shannon, 2005; Lindgren et al., 2020; Vaismoradi et al., 2013). Assisted by QSR NVivo 12 Software, the data analysis was organised into five steps (Figure 1). Although illustrated as a linear, chronological sequence, analysis occurred in a non-linear, iterative manner till there was clarity and meaning in the themes that were identified.

Figure 1. Sequence of data analysis

III. RESULTS

A total of 42 physiotherapists from four settings: acute, community, specialised institutions, and private practice participated (Table 1). All participants have been practicing physiotherapists in Singapore for the past three years, with 33.3 % in geriatrics and neuro rehabilitation, 45.3% in musculoskeletal and sports, and 21.4% in cardiopulmonary and other niche specialties. Thirty-six participants are Singaporeans and six are from Malaysia, Philippines, Hong Kong, New Zealand, and the UK. Ages ranged from 27 to 54 years old.

Submitted: 22 January 2022
Accepted: 4 May 2022
Published online: 4 October, TAPS 2022, 7(4), 50-58
https://doi.org/10.29060/TAPS.2022-7-4/OA2748

Nguyen Tran Minh Duc, Khuu Hoang Viet & Vuong Thi Ngoc Lan

University of Medicine and Pharmacy at Ho Chi Minh City, Ho Chi Minh City, Vietnam

Abstract

Introduction: The Scholarly Project provides medical students with an opportunity to conduct research on a health and health care topic of interest with faculty mentors. Despite the proven benefits of the Scholarly Project there has only been a gradual change to undergraduate medical education in Vietnam. In the academic year of 2020-2021, the University of Medicine and Pharmacy (UMP) at Ho Chi Minh City launched the Scholarly Project as part of an innovative educational program. This study investigated the impact of the Scholarly Project on the research skills perception of participating undergraduate medical students.

Methods: A questionnaire evaluating the perception of fourteen research skills was given to participants in the first week, at midterm, and after finishing the Scholarly Project; students assessed their level on each skill using a 5-point Likert scale from 1 (lowest score) to 5 (highest score).

Results: There were statistically significant increases in scores for 11 skills after participation in the Scholarly Project. Of the remaining three skills, ‘Understanding the importance of “controls”’ and ‘Interpreting data’ skills showed a trend towards improvement while the ‘Statistically analyse data’ skill showed a downward trend.

Conclusion: The Scholarly Project had a positive impact on each student’s perception of most research skills and should be integrated into the revamped undergraduate medical education program at UMP, with detailed instruction on targeted skills for choosing the optimal study design and follow-up assessment.

Keywords:           Study Skills, Scholarly Project, Undergraduate, Medical Education, Self-Assessment

Practice Highlights

• The Scholarly Project is an essential component of the undergraduate medical education curriculum.
• Targeted researching skills is a valuable method to optimise competency-based criteria.
• The initial choice of study design is important to the overall research skill self-perceptive improvement.

Figure 1. Integration of the Scholarly Project into the new reformed undergraduate and postgraduate medical curriculum in Vietnam.

C. Study Setting and Participants

This one-group pretest-posttest study had a quasi-experimental design. Research skills assessed were chosen based on fourteen individual research skills (Kardash, 2000). The questionnaire has been used previously, with a Cronbach’s alpha calculated at 0.9 and item-total correlation varied between 0.49 to 0.76 (Kardash, 2000). The questionnaire was translated into Vietnamese, then the local language version was pre-tested and the final text was amended as necessary. The translation process was undertaken in accordance with Guidelines for the Cross-Cultural Adaptation Process (Beaton et al., 2000). Translations were evaluated and compared with the original questionnaire by the Education and Research Council of the UMP to ensure accuracy of the Vietnamese version prior to study initiation. Medical student surveys were administered during the first week of the Scholarly Project and students were asked to indicate their current level of performance for each skill and the extent to which they hoped that the project would develop each skill on a 5-point Likert scale from 1–5 (where higher scores indicate greater skill level). Surveys were repeated at midterm and during the last week of the Scholarly Project module; at these times the students used the same scale to rate the extent to which they felt capable of performing each skill and how they believed the internship had developed their skills in general. Medical students had to provide informed consent on the first page of the electronic form before accessing the rest of the questionnaire.  

D. Statistical Analysis

Raw data were extracted from the online survey link for each participating medical student and saved in Excel sheets. R (R Core Team, Vienna, Austria) was applied to analyse data. First, scores for each skill at baseline were compared with those obtained after project completion using a paired t-test (Student’s t-test). The same method was used to compare expected skill level evaluated at baseline and the actual skill level rating at the end of the Scholarly Project. A p-value of <0.05 was considered to be statistically significant.

III. RESULTS

A. Response Rate and Participant Data

Of 384 students participating in the Scholarly Project, 194 (50.5%) completed the survey. The majority of participants were male (60%) and had the role of project team member (75.3%) (See Table 1). The most common Scholarly Project design was a cross-sectional study (47.9%), followed by study protocol development (21.1%), case/case series report (11.9%), and literature review (10.3%) (Table 1). Twenty-one different departments with a wide range of specialties provided scientific mentors for the Scholarly Projects undertaken by 48 research groups (See Table 1).

Table 1. Demographic and project characteristics for survey respondents.

Values are mean ± standard deviation, or number of respondents (%).

B. Research Skills at Baseline, Midterm and Project Completion

At baseline, self-rated competency was highest for ‘Understand the importance of “controls”’, ‘Understand contemporary concepts’, ‘Identify a specific question’, and ‘Observe and collect data’ (Figure 2). All skills had self-evaluating levels above “moderate” (score of >3), except for ‘Write research for publication’ (mean score 2.696). Students expected that all skills would increase after participating in the Scholarly Project (p<0.001).

In the midterm survey, five skill groups showed significant improvement from baseline (Figure 2). These were ‘Make use of scientific literature’, ‘Identify a specific question’, ‘Observe and collect data’, ‘Relate results to the “bigger picture”’, and ‘Orally communicate research project skills’. Conversely, there was a significant decrease in self-rated skill for ‘Statistically analyse data’ and ‘Interpret data skills’, while other skill ratings were stable (Figure 2).

Figure 2. Change in self-rated medical research skills of 194 participants from baseline to the midterm of the Scholarly Project

M: mean; SD: standard deviation; CI: confidence interval

At the completion of the Scholarly Project, the five skills that showed improvement at the midterm assessment showed continued improvement, and another six skills had also improved significantly compared with baseline (Figure 3). However, scores for ‘Understand the importance of “controls”’, ‘Interpret data’ and ‘Statistically analyse data” did not change significantly from baseline, and the mean score for the latter parameter was actually slightly below baseline (Figure 3).

Figure 3. Change in self-rated medical research skills of 194 participants from baseline to completion of the Scholarly Project (SP)

 M: mean; SD: standard deviation; CI: confidence interval.

Looking more closely at analytical skills relating to six types of study design showed that self-rated skill for the ability to interpret data for a literature review decreased significantly, as did self-rated skill scores for statistically analyse data in relation to study protocol development and literature review (Table 2). In contrast, there was a significant improvement in self-rated skill for data interpretation for cross-sectional studies and for statistical analysis of data in cohort studies (Table 2).

Table 2. Self-evaluated skill level scores for ‘Interpret data’ and ‘Statistically analyse data’ from baseline to completion of the Scholarly Project

Values are mean ± standard deviation. *p<0.05 vs baseline.

IV. DISCUSSION

A. Impact of Scholarly Project on Students’ Perception of Research Skills

Our results show that ratings for most skills increased during and after the Scholarly Project. Increases in ratings for ‘Identifying a specific question’, ‘Orally communicate research projects’, and ‘Relate results to the “bigger picture”’ in our study were consistent with data from Schor et al. (2005), who reported that the Scholarly Project could be beneficial by fostering analytical thinking skills, improving oral communication skills, and enhancing skills for evaluating and applying new knowledge to their profession (Schor et al., 2005). A significant increase in ‘Make use of scientific literature’ in our study reflects the idea-forming process at the study design stage of the Scholarly Project, during which students could practice the ability to read and critically evaluate medical literature. These are essential components of undergraduate medical education, irrespective of whether students intend to pursue a career in academic medicine or in public or private clinical practice (Holloway et al., 2004).

B. Data-related Skills and the Concept of a Control Group

The two skills of ‘Statistically analyse data’ and ‘Interpret data’ are introduced mainly in the Advanced Statistics Module with a training period of 2 weeks before starting the Scholarly Project, and briefly presented in the ‘Basic statistics informatics’ module during the first year of training and in the ‘Basic epidemiology’ module during the third year of the undergraduate curriculum. Therefore, baseline assessments in our study took place after the Advanced Statistics Module, which could have influenced ratings on the above skills. Given that our midterm assessment was performed at a time when most students had not had the opportunity to practice these skills, there may have been a negative impact on self-evaluation. The change in scores for ‘Statistically analyse data’ and ‘Interpret data’ at the midterm assessment was therefore influenced by an external factor (the Advanced Statistics Module) and an internal factor (the Scholarly Project). Therefore, future assessments of the impact of the Scholarly Project on learning should not have the quasi-experimental design used here, but instead, use an interrupted time-series design. This will mean that several surveys would be conducted before starting the Advanced Statistics Module, with the aim of eliminating confounding factors.

The final assessment showed significant improvements in scores for ‘Statistically analyse data’ and ‘Interpret data skills’ compared with the midterm survey. When applied in students’ projects, the improvement of these two skills indirectly supported the aforementioned context. This highlights the value of active learning compared with passive learning. It has conclusively been shown that cramming statistical knowledge means that students do not understand basic concepts to apply appropriately (Leppink, 2017). As noted by Leppink, statistics should be integrated into medical subjects; familiarity with these subjects and the repeated use of these skills provides opportunities to develop statistical skills. The Scholar Project is a typical example of this trend. However, only the ‘Statistically analyse data skill’ showed a downward pattern, while the ‘Interpret data skill’ increased slightly, suggesting that the Scholarly Project should focus more on these skills. Additional studies that take these variables into account are needed.

The control group concept is taught in Basic Epidemiology during the third year of Basic Science and the first sessions of the Scholarly Project. The control group has a pivotal role in study design should have elements that match the experimental group’s characteristics, except for the intervention/variable applied to the latter (Kinser & Robins, 2013). This scientific control group enables the experimental study of one variable at a time, and it is an essential part of the scientific method. Two identical experiments are carried out in a controlled experiment: in one of them, the treatment or tested factor  is applied (experimental group), whereas in the other group (control), the tested factor is not applied (Pithon, 2013). However, due to the limitation that only four respondents had a project with a case-control study design, the ‘Understand the importance of “controls”’ skill only showed a modest improvement, despite having been taught previously, which is similar to a previous undergraduate research study (Kardash, 2000). Compared with cross-sectional study design, which was the most popular design for studies in this Scholarly Project, case-control studies often required a greater amount of human and facility resources. We suggest that a case-control study with a small sample size of 10–20 could be a suitable study design for medical students to understand how best to conduct research with a control group.

Of the 194 respondents in our study, 56.7% of the cohort should have been able to fully experience all fourteen of the skills assessed. In contrast, those who participated in study protocol development, literature review, and case/case series report projects had limited opportunities to practice analytical skills. Similar to our findings, a previous study demonstrated that only 13% of 475 projects conducted by medical students contained four main research skill areas, including research methods, information gathering, critical analysis and review, and data processing (Murdoch-Eaton et al., 2010). Furthermore, the COVID-19 outbreak during the academic year 2020-2021 significantly impacted the originally planned Scholarly Project data collection process. As a result, some research teams switched to more feasible design studies such as study development or literature review, which potentially influenced the two skills of statistical analysis and data interpretation skills. Therefore, it could be hypothesised that these conditions are less likely to occur if participants recognise the skills required for research before designing the study protocol. Thus, there is room for further progress in determining the optimal project descriptions provided to medical students participating in the Scholarly Project to allow them to benefit from the research opportunities and fully develop essential skills.

C. The Role of Scholarly Project in Medical Education in Vietnam

This Scholarly Project is an essential step in curriculum reform for Vietnam’s medical education system. In the last two decades, medical educators in Vietnam have collaborated to promote the social trend for undergraduate medical education, and identify the goals and outcomes of learning from medical graduates in expected knowledge, attitudes, and skills (Hoat et al., 2009). Furthermore, Vietnamese policymakers created an environment that enabled academic innovation by implementing the necessary changes to national university autonomy policies (Duong et al., 2021). These policies enable public universities to be financially independent, manage their operation and human resources, prioritise technology, and develop new curricula. The Scholarly Project helps to train physicians who are better prepared to meet patient requirements and health needs (Fan et al., 2012). Based on competency in medical education, the Scholarly Project focuses on outcomes, emphasises the application of knowledge and practice, and promotes greater learner-centeredness (Carraccio et al., 2002; Frank et al., 2010; Iobst et al., 2010). In addition, the Scholarly Project helps to reduce the time spent in passive lectures, which can negative affect medical students (Deslauriers et al., 2019; Schwartzstein et al., 2020; Schwartzstein & Roberts, 2017). Instead, students are encouraged to explore research topics based on their interests, human and institutional resources, and university mentors’ guidance and follow-up. Compared with the large class sizes from Vietnam’s traditional teaching method, the Scholarly Project (with an average of eight students and one mentor) provides low faculty-to-students ratios, creating desired small group learning. Starting for the first time in the 2020-2021 school year, Scholarly Project had to adapt to the impact of the COVID-19 pandemic, with two periods of online learning required in September 2020 and May 2021 due to local COVID-19 outbreaks. To help manage this, the university applied for technical assistance from Microsoft Office 365 with a full-access subscription to maintain the scheduled small group meetings between students and their mentors while optimising social distancing (Duong et al., 2021).

We recommend introducing the 14-skill questionnaire as a tool for medical students to self-monitor their improvement during participation in the Scholarly Project. From the mentors’ perspective, the questionnaire provides a reliable and convenient reference for providing feedback to students and suggestions about areas that need further improvement. These approaches could also be utilised in other institutions, either locally or internationally, who include a Scholarly Project for a number of reasons: (1) the Scholarly Project is a lengthy module that could be impacted by unexpected events (e.g. COVID-19); (2) the need for routine self-check and mentor feedback to facilitate the required research skills improvement; and (3) because the questionnaire is a validated, convenient and accessible method for both medical students and mentors.

D. Study Limitations

Although the survey was sent to all medical students participating in the Scholarly Project, only just over half of students responded. Therefore, the impact of the Scholarly Project on non-responding medical students may not reflect the trends reported here, limiting the generalizability of our findings. Nonresponse bias is another potential limitation, although this is not necessarily associated with a lower response rate (Davern, 2013; Halbesleben and Whitman, 2012). Participants might perceive that self-evaluation about how much their research skills had improved could indirectly reflect their level of participation in Scholarly Project, the contribution of their mentor, and the level of their academic performance, leading to social desirability bias in their responses. We attempted to reduce nonresponse and social desirability bias, and any perception that responses could impact on academic assessments, by making survey responses anonymous and keeping the study survey completely separate from any academic assessments (e.g. grade-point average). Another limitation is the lack of a control group of medical students, but this is difficult because participation in the Scholarly Project is mandatory for all students. Using a control group would have strengthened the study from a methodological perspective and allowed investigation of the impact of specific aspects of the Scholarly Project.

Respond shift bias is inevitable while conducting this research. To reduce this, instead of completing self-evaluation for all fourteen skills initially and then after the completion of the whole project, students should assess their skill level immediately after the completion of each Module. However, response shift bias happened because respondents perceived the purpose of the survey as assessing the program’s effectiveness. In the context of our research, even if assessments were completed after each Module, students would realise the aim of the survey meaning that respond-shift bias would not decrease considerably.

V. CONCLUSION

Scholarly Project is an excellent learning opportunity for medical students in the refreshed undergraduate medical curriculum. Participating in a Scholarly Project provides students with research experience, including the knowledge, structure, and support needed to engage in scholarly work. By providing the foundations for scholarly work, medical students can enter the health care workforce with solid clinical expertise and the basic skills required to conduct high-quality projects that improve the safety and quality of care delivered to patients. We suggest integrating the Scholarly Project curriculum throughout the undergraduate medical education curriculum in Vietnam. This is important in terms of early experience of medical research and fostering a good understanding of medical scientific research for all future doctors, regardless of their ultimate career destination.

Notes on Contributors

N.T.M.D. and K.H.V. drafted and revised the manuscript. V.T.N.L. helped in reviewing the manuscript. All authors (N.T.M.D., K.H.V., V.T.N.L.) have made substantial contributions to the conception and design of the work and the acquisition, analysis, and interpretation of data. All authors read and approved the final manuscript.

Ethical Approval

The authors declare that this study did not require human ethics approval and did not include experiments on animal or human subjects. This study was submitted to the Institutional Review Board (IRB) at University of Medicine and Pharmacy at Ho Chi Minh City, Ho Chi Minh City, Vietnam. This project was determined to be exempt from IRB review. All methods were carried out in accordance with relevant guidelines and regulations. Respondents were informed that their participation in the survey was completely voluntary and there were no risks associated with their participation.

Data Availability

The datasets generated and/or analysed during the current study are not publicly available for reasons of data protection but are available from the corresponding author on reasonable request.

Acknowledgement

The authors would very much like to acknowledge Ms. Le Minh Chau, Mr. Ung Nguyen Vu Hoang, Ms. Duong Kim Ngan, Mr. Nguyen Hai Dang, Ms. Tran Thi Hong Ngoc, Mr. Giang Luu Thanh Hoang, and Mr. Nguyen Hoang Nhan (University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam) for their support of this study.

Funding

No funding has been received for the study.

Declaration of Interest

The authors declare that they have no competing interests.

References

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186-3191. https://doi.org/10.1097/00007632-200012150-00014

Bickford, N., Peterson, E., Jensen, P., & Thomas, D. (2020). Undergraduates interested in STEM research are better students than their peers. Education Sciences, 10(6), 150. https://doi.org/10.3390/educsci10060150

Blockus, L., Kardash, C. M., Blair, M., & Wallace, M. (1997). Undergraduate internship program evaluation: A comprehensive approach at a research university. Council on Undergraduate Research, 18, 60–63.

Boninger, M., Troen, P., Green, E., Borkan, J., Lance-Jones, C., Humphrey, A., Gruppuso, P., Kant, P., McGee, J., Willochell, M., Schor, N., Kanter, S. L., & Levine, A. S. (2010). Implementation of a longitudinal mentored scholarly project: An approach at two medical schools. Academic Medicine, 85(3), 429–437. https://doi.org/10.1097/acm.0b013e3181ccc96f

Carraccio, C., Wolfsthal, S. D., Englander, R., Ferentz, K., & Martin, C. (2002). Shifting Paradigms. Academic Medicine, 77(5), 361–367. https://doi.org/10.1097/00001888-200205000-00003

Carson, S. (2007). A new paradigm for mentored undergraduate research in molecular microbiology. CBE—Life Sciences Education, 6(4), 343–349. https://doi.org/10.1187/cbe.07-05-0027

Dagher, M. M., Atieh, J. A., Soubra, M. K., Khoury, S. J., Tamim, H., & Kaafarani, B. R. (2016). Medical Research Volunteer Program (MRVP): Innovative program promoting undergraduate research in the medical field. BMC Medical Education, 16(1), Article 160. https://doi.org/10.1186/s12909-016-0670-9

Davern, M. (2013). Nonresponse rates are a problematic indicator of nonresponse bias in survey research. Health Services Research, 48(3), 905–912. https://doi.org/10.1111/1475-6773.12070

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116

Duong, D. B., Phan, T., Trung, N. Q., Le, B. N., Do, H. M., Nguyen, H. M., Tang, S. H., Pham, V. A., Le, B. K., Le, L. C., Siddiqui, Z., Cosimi, L. A., & Pollack, T. (2021). Innovations in medical education in Vietnam. BMJ Innovations, 7(Suppl 1), s23–s29. https://doi.org/10.1136/bmjinnov-2021-000708

Fan, A. P., Tran, D. T., Kosik, R. O., Mandell, G. A., Hsu, H. S., & Chen, Y. S. (2012). Medical education in Vietnam. Medical Teacher, 34(2), 103–107. https://doi.org/10.3109/0142159x.2011.613499

Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., Dath, D., Harden, R. M., Iobst, W., Long, D. M., Mungroo, R., Richardson, D. L., Sherbino, J., Silver, I., Taber, S., Talbot, M., & Harris, K. A. (2010). Competency-based medical education: Theory to practice. Medical Teacher, 32(8), 638–645. https://doi.org/10.3109/0142159x.2010.501190

Halbesleben, J. R. B., & Whitman, M. V. (2012). Evaluating survey quality in health services research: A decision framework for assessing nonresponse bias. Health Services Research, 48(3), 913–930. https://doi.org/10.1111/1475-6773.12002

Hoat, L. N., Lan Viet, N., van der Wilt, G., Broerse, J., Ruitenberg, E., & Wright, E. (2009). Motivation of university and non-university stakeholders to change medical education in Vietnam. BMC Medical Education, 9(1), Article 49. https://doi.org/10.1186/1472-6920-9-49

Holloway, R., Nesbit, K., Bordley, D., & Noyes, K. (2004). Teaching and evaluating first and second year medical students’ practice of evidence-based medicine. Medical Education, 38(8), 868–878. https://doi.org/10.1111/j.1365-2929.2004.01817.x

Iobst, W. F., Sherbino, J., Cate, O. T., Richardson, D. L., Dath, D., Swing, S. R., Harris, P., Mungroo, R., Holmboe, E. S., & Frank, J. R. (2010). Competency-based medical education in postgraduate medical education. Medical Teacher, 32(8), 651–656. https://doi.org/10.3109/0142159x.2010.500709

Kardash, C. M. (2000). Evaluation of undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. Journal of Educational Psychology, 92(1), 191–201. https://doi.org/10.1037/0022-0663.92.1.191

Kinser, P. A., & Robins, J. L. (2013). Control group design: enhancing rigor in research of mind-body therapies for depression. Evidence-Based Complementary and Alternative Medicine, 2013. https://doi.org/10.1155/2013/140467

Leppink, J. (2017). Helping medical students in their study of statistics: A flexible approach. Journal of Taibah University Medical Sciences, 12(1), 1–7. https://doi.org/10.1016/j.jtumed.2016.08.007

Manduca, C. (1997). Broadly defined goals for undergraduate research projects: A basis for program evaluation. Council on Undergraduate Research, 18(2), 64–69.

Murdoch-Eaton, D., Drewery, S., Elton, S., Emmerson, C., Marshall, M., Smith, J. A., Stark, P., & Whittle, S. (2010). What do medical students understand by research and research skills? Identifying research opportunities within undergraduate projects. Medical Teacher, 32(3), e152–e160. https://doi.org/10.3109/01421591003657493

Nguyen, T. V., Ho-Le, T. P., & Le, U. V. (2016). International collaboration in scientific research in Vietnam: An analysis of patterns and impact. Scientometrics, 110(2), 1035–1051. https://doi.org/10.1007/s11192-016-2201-1

Pithon, M. M. (2013). Importance of the control group in scientific research. Dental Press Journal of Orthodontics, 18(6), 13–14. https://doi.org/10.1590/s2176-94512013000600003

Schor, N. F., Troen, P., Kanter, S. L., & Levine, A. S. (2005). The scholarly project initiative: Introducing scholarship in medicine through a longitudinal, mentored curricular program. Academic Medicine, 80(9), 824–831. https://doi.org/10.1097/00001888-200509000-00009

Schwartzstein, R. M., Dienstag, J. L., King, R. W., Chang, B. S., Flanagan, J. G., Besche, H. C., Hoenig, M. P., Miloslavsky, E. M., Atkins, K. M., Puig, A., Cockrill, B. A., Wittels, K. A., Dalrymple, J. L., Gooding, H., Hirsh, D. A., Alexander, E. K., Fazio, S. B., & Hundert, E. M. (2020). The Harvard Medical School pathways curriculum: Reimagining developmentally appropriate medical education for contemporary learners. Academic Medicine, 95(11), 1687–1695. https://doi.org/10.1097/acm.0000000000003270

Schwartzstein, R. M., & Roberts, D. H. (2017). Saying goodbye to lectures in medical school — Paradigm shift or passing fad? New England Journal of Medicine, 377(7), 605–607. https://doi.org/10.1056/nejmp1706474

Zydney, A. L., Bennett, J. S., Shahid, A., & Bauer, K. W. (2002). Impact of undergraduate research experience in engineering. Journal of Engineering Education, 91(2), 151–157. https://doi.org/10.1002/j.2168-9830.2002.tb00687.x

*Nguyen Tran Minh Duc
217 Hong Bang Street, Ward 11,
District 5, Ho Chi Minh City, Vietnam
+84 988 127 948
Email: ntmduc160046@ump.edu.vn

Submitted: 13 January 2022
Accepted: 9 May 2022
Published online: 4 October, TAPS 2022, 7(4), 35-49
https://doi.org/10.29060/TAPS.2022-7-4/OA2699

Yuan Kit Christopher Chua^1*, Kay Wei Ping Ng^1*, Eng Soo Yap^2,3, Pei Shi Priscillia Lye⁴, Joy Vijayan¹, & Yee Cheun Chan¹

¹Department of Medicine, Division of Neurology, National University Hospital Singapore, Singapore; ²Department of Haematology-oncology, National University Cancer Institute Singapore, Singapore; ³Department of Laboratory Medicine, National University Hospital Singapore, Singapore; ⁴Department of Medicine, Division of Infectious Diseases, National University Hospital Singapore, Singapore

*Co-first authors

Abstract

Introduction: In-class engagement enhances learning and can be measured using observational tools. As the COVID-19 pandemic shifted teaching online, we modified a tool to measure the engagement of instructors and students, comparing in-person with online teaching and different class types.

Methods: Video recordings of in-person and online teachings of six identical topics each were evaluated using our ‘In-class Engagement Measure’ (IEM). There were three topics each of case-based learning (CBL) and lecture-based instruction (LLC). Student IEM scores were: (1) no response, (2) answers when directly questioned, (3) answers spontaneously, (4) questions spontaneously, (5) initiates group discussions. Instructor IEM scores were: (1) addressing passive listeners, (2) asking ≥1 students, (3) initiates discussions, (4) monitors small group discussion, (5) monitoring whole class discussions.

Results: Twelve video recorded sessions were analysed. For instructors, there were no significant differences in percentage time of no engagement or IEM scores when comparing in-person with online teaching. For students, there was a significantly higher percentage time of no engagement for the online teaching of two topics. For class type, there was overall less percentage time of no engagement and higher IEM scores for CBL than LLC.

Conclusion: Our modified IEM tool demonstrated that instructors’ engagement remained similar, but students’ engagement reduced with online teaching. Additionally, more in-class engagement was observed in CBL. “Presenteeism”, where learners were online but disengaged was common. More effort is needed to engage students during online teaching.

Keywords:           Engagement, Observational Tool, Online Learning, E-learning, COVID-19, Medical Education, Research

Practice Highlights

• Lectures to large class (LLC) and case-based learning (CBL) are associated with lower levels of student engagement when conducted on a virtual platform.
• Instructors’ engagement during online teachings remained similar to that of in-person teachings.
• LLC is associated with reduced student engagement than CBL.

I. INTRODUCTION

Educational theories suggest that learning should be an active process. According to social constructivist theory, learning can be better achieved by social interactions in the learning environment (Kaufman, 2003). Active learning strategies fostering the students to interact with each other and the instructor such as discussions, talks, questions, may yield desirable learning outcomes in terms of knowledge, skills, or attitudes (Rao & DiCarlo, 2001). Therefore, using in-class learner engagement as an important keystone of active learning strategies is known to stimulate and enhance the learner’s assimilation of content and concepts (Armstrong & Fukami, 2009; Watson et al., 1991).

There is good evidence for the importance of engagement in online learning and use of an engagement metric has been advocated to better understand student online interactions to improve the online learning environment (Berman & Artino, 2018). While medical literature suggests that virtual education games foster engagement (McCoy et al., 2016), the level of engagement and learning fostered by online methods for group discussion and teaching is unknown. Teleconferencing is among some of the methods suggested for maintaining education during the COVID-19 pandemic (Chick et al., 2020).

Possible methods of quantifying student engagement include direct observation and student self-report. O’Malley et al. (2003) has published a validated observation instrument called STROBE to assess in-class learner engagement in health professions without interfering with learner activities. This observation instrument is used to document observed dichotomized types of instructor and student behaviors in 5-minute cycles and quantify the number of questions asked by the instructor and students in different class subtypes. This instrument as well as revised forms of this instrument has since been used as “in-class engagement measures” to compare instructor and student behaviors in different class types (Alimoglu et al., 2014; Kelly et al., 2005).

In our institution, a hybrid curriculum of case-based learning as well as lecture-style courses is used to teach the post graduate year one (PGY-1) interns. We had video recordings of these courses performed in-person prior to the COVID-19 pandemic. With the advent of the pandemic, these courses were shifted onto Zoom teleconferencing platform, but delivered by the same instructors, in the same class format.

We therefore aimed to determine and compare in-class learning engagement levels via observing instructor and student behaviours in different platforms of learning (either observed online or in-person retrospectively via video recording) delivered by the same instructor before and during the COVID-19 pandemic. We also aimed to compare instructor and student behaviours in different class types (either case-based learning or lecture style instruction). To do this, we planned to modify a known in-person observational tool for student engagement – “STROBE” (O’Malley et al., 2003) for use in analysing and recording the behaviours of students in both online and in-person teaching.

II. METHODS

A. Observed Class Types

In this study, we observed two different class types, case-based learning (CBL), as well as lecture-based instruction to teach basic medical/surgical topics to a large classroom (LLC) of PGY-1 interns. Video recordings of these in-person teachings were made in 2017. Both these class types were replicated in the same format on an online Zoom teleconferencing platform and were delivered by nearly all of the same tutors using the same content and Powerpoint slides during the COVID-19 pandemic in 2020. We aimed to view the 2017 video-recordings of the in-person teachings and compare them with the 2020 online teaching of PGY-1 interns. Written consent was obtained from the tutors and implied consent from the students. Students were informed beforehand via email that the sessions were going to be observed and they were again reminded at the start of each session where they had the chance to opt out. Subsequently, all student feedback and observation scores were amalgamated and de-identified. This study was approved by the institution’s ethics board. 

Three topics each of case-based learning as well as lecture-style instruction were selected in chronological order as scheduled for students. Each topic of instruction was allotted up to a maximum of 90 minutes of time, but the instructor could choose to end the class earlier if the session was completed. Description of both class types are below.

1) Description of case-based learning in large classroom

The content of the learning was designed by the instructor, and consisted of clinical cases involving patient scenarios, where the main pedagogy was problem-solving and answering case-based questions relating to the patient scenario (e.g., diagnosis, reading clinical images or electrocardiograms, creating an investigation or treatment plan). Each case would typically take about 15 to 20 minutes to complete, and there would typically be five to six cases. Students were expected to answer the questions, and the instructor gave feedback on the answers and provided additional information, sometimes via additional Powerpoint slides. Class discussions were encouraged where students were encouraged to debate and discuss with each other over their classmates’ answers. The titles of the case-based learning were “ECG – tachydysrhythmias”, “Approach to a confused patient” and “Approach to chest pain”. 

2) Description of lecture in large classroom

This is a typical lecture-style instruction performed with participation of around 86 PGY1-interns and one instructor. The instructor delivers information via a Powerpoint slide presentation and rarely adds clinical case-based questions into the slides to invite student discussion. The titles of the lectures were “Cardiovascular health – hypertensive urgencies”, “Trauma – chest, abdomen and pelvis” and “Stroke”. 

B. Instructor and Student Characteristics

The instructors all had at least ten years of teaching experience in medical education, and all had been teaching the same topics to the PGY-1 interns for at least the last five years. Student feedback scores on their teaching activities have been satisfactorily high (mean 4.63 for 2019, the year prior to the shift to online learning for the pandemic). All the tutors (except for one instructor who taught “Stroke”) had taught the same topics using the same content and Powerpoint slides in 2017 via in-person teaching which was caught on camera.

The students were all PGY-1 interns, who have been asked by the institution to attend at least 70% of a mandatory one-year long teaching program where they are given weekly instruction on various medical or surgical topics. The teaching program commences from May of each year. There were 86 PGY-1 interns commencing their rotations in our institution and attending the teaching program from May 2020. There were 75 PGY-1 interns attending the teaching program in the video recordings caught in 2017.

C. Observation Tool

A revised form of STROBE (O’Malley et al., 2003) was used to analyze and record the behaviors of the instructor and students in classes, to provide a more objective third-person measure of student engagement. The original STROBE tool was an instrument that was developed to objectively measure student engagement across a variety medical education classroom settings. The STROBE instrument consists of 5-minute observational cycles repeated continuously throughout the learning session with relevant observations recorded on a data collection form. Within each cycle, observers record selected aspects of behavior from a list of specified categories that occur in each interval recorded. Observations include macrolevel elements such as structure of class, major activity during time, and a global judgment of the proportion of class members who appear on task, as well as microlevel elements such as instructor’s behavior and the behaviors of four randomly selected students. Observers also record who the behaviours of instructors and students were directed at. After which, observers tally the number of questions asked by the students and instructor in the remainder of the 5 minutes. The revision of this tool was made by the 3 Clinician-educators from the research team (CYC, YES, KN), having discussed what kind of instructor and student behaviors were considered as “active student engagement”, keeping the main statements and principles of the original STROBE tool. The scale was modified to make it suitable for use in an online learning setting, where the observers may not be able to observe the student’s body language cues when the student does not turn on his/her video function. We called this modified scale our ‘In-class engagement measure’. The modified scales were as follows:

A 5-item list of instructor and student behaviors was therefore created and rated from 1 to 5 each, with different scales for instructor and student. For the student behavior scale, each item was to show progressively increasing levels of interaction, and perceived engagement, both with the instructor and with each other. For the instructor behavior list, each item was also about progressively interactive behaviors by the instructor to get the students to engage. We called these scales our “In-class Engagement Measure (IEM)”. The scales were as follows:

Student: 

1. No response even when asked
2. Answers only when directly questioned
3. Answers questions spontaneously
4. Speaks to instructor spontaneouslyg.,Poses questions, discusses concepts
5. Speaks to instructor and 1 or more other student during a discussion

Instructor: 

1. Talking to entire class while all the students are passive receivers 
2. Telling/asking to one or a group of students, or teaching/showing an application on a student
3. Starting or conducting a discussion open to whole class, or assigning some students for some learning tasks
4. Listening/monitoring actively discussing one or a group of students
5. Listening/monitoring actively discussing entire class

For the student behaviour list, we also sub-categorized the student behaviour item “1”, where “1*” was defined as no response when a question was posed to a specific student and not just the whole class, where the student-in-question would have his/her name called by the tutor. 

D. Observation Process

Drawing from the described process for the STROBE observation tool (O’Malley et al., 2003), as well as other described modifications of the STROBE tool (Alimoglu et al., 2014), we used the same observation units and cycles. Modifications to the original described process for the STROBE observation tool was made to make it suitable for not being in-person when observing a large group of students and their instructor. Three observers from the research team (CYC, YES, KN) observed and recorded the instructor and student behaviors for the three case-based learning and three lecture-style learning conducted live online in 2020, and as a video recording of in-person teaching in 2017. A total of 12 lectures were therefore analyzed. One observation unit was a 5-minute cycle. The 5-minute cycle would proceed as such: The observer would write the starting time of the cycle and information about the class (number of students, title of session). The observer would select a student from the class and observe that student for 20 seconds and mark the type of engagement observed according to the IEM scale created. As the observers were not in-person for the teaching at either the 2017 video recording, and for the 2020 online learning, students who responded to the instructor or posed questions were marked at the same time by all the three observers. The 5-minute cycle would consist of four 20-second observations of individual learners, so marking of student engagement would be performed four times within that cycle with different students in succession. The observer would also observe the instructor for that 5-minute cycle and similarly mark the instructor’s behavior once for that 5-minute cycle. For the remainder of the modified STROBE cycle, the observer would tally the number of questions asked by all the students and the instructor. 

Observers independently and separately observed and marked the students’ and instructors’ behaviors. Due to the lack of in-person observation, students who responded or posed questions during the session were uniformly chosen for marking by the three observers. If a student had already been marked once during that cycle, the same student was not used for remaining three observations within the same cycle. At the end of the marking, two observers (KN and YES) compared their scores for both students and instructor. The marks given by the third observer (CYC) was used to validate the final score awarded and used as the tiebreaker when there was a discrepancy in the marks given by the first two observers. 

E. Collation of Post Teaching Survey Feedback

Apart from the data derived from our modified observational tool, we also reviewed data from surveys conducted by the educational committee after each of these teaching sessions (see Table 1). These were general surveys used to solicit student feedback on the teaching sessions. They were distributed in-person in 2017, with the same forms distributed to the students online in 2020.  Responses from the students were in response to five statements, with scoring 0 to 5 (1 for Strongly disagree, 2 for disagree, 3 for neither agree nor disagree, 4 for agree, and 5 for Strongly agree). These feedback forms had an overall feedback score marked by the student, as well as a score marked by the student in response to a question assessing for self-reported engagement – “The session was interactive and engaging”. The other questions were “The session has encouraged self-directed learning and critical thinking”, “The session was relevant to my stage of training”, “The session helped me advance my clinical decision-making skills”, and “The session has increased my confidence in day-to-day patient management”. Means of the feedback scores were taken as a qualitative guide, and we analyzed the overall feedback scores (“Overall feedback score” in Table 1), and the scores in response to the question assessing for self-reported engagement (“Self-reported engagement feedback score” in Table 1). 

F. Statistical Analyses

Descriptive statistics were used to determine frequencies and median number of questions asked, as well as mean student feedback scores and absolute duration of each teaching session. Fisher exact test was also performed to analyze the differences in scores between different lectures and case-based learning, and the scores in the 2017 in-person learning versus that of the 2020 online learning. For analysis of the scores, we dichotomized our scores using the cut-off of “1”, or our first item on the behavior list for both students and instructors, as we felt that the first item reflected an extreme non-participation for both student and instructor, which if left to continue, can result in negative learning and teaching behaviors.

III. RESULTS

A. Class Types, Characteristics, Feedback Scores

A total of 12 sessions were observed, consisting of in-person and online teaching sessions of six topics (Table 1). There were 3 topics of CBL and LLC each. Duration of the class sessions range from 30-55 minutes for the in-person sessions and 40-90 minutes for the online sessions. Total number of PGY-1 students eligible to attend the in-person teaching sessions in 2017 was 82, and 86 for the online teaching sessions in 2020. Student attendance for the in-person sessions ranged from 11 (13.4%) to 31 (37.8%) and that for the online session ranged from 28 (32.6%) to 77 (89.5%). Median (range) of feedback scores for in-person sessions were 4.57 (4.25 to 4.72) vs 4.32 (4.04 to 4.61) for online sessions. Median (range) of self-reported engagement scores for in-person sessions were 4.55 (4.25 to 4.79) vs 4.34 (4.00 to 4.67) for online sessions (Table 1).

 

Table 1. Class types and characteristics (*Different tutors, but using same content)

B. Instructors’Engagement Behaviour

1) Comparing in-person vs online teaching: Percentage time during which there is no engagement/interaction (or scoring “1” on the IEM score). This ranges from 0-80% for in-person teaching vs 0-100% for online teaching (Table 2A). For each topic, there is no significant difference between percentage time of no engagement. 

Most frequent IEM scores. Most frequent IEM scores for each 5-minute segment were 3 for in-person teaching (48.9%) and online teaching (52.9%) (Table 2B).

2) Comparing CBL vs LLC: Percentage time during which there is no engagement/interaction. This ranges from 0-23.1% for CBL vs 50-100% for LLC (Table 2A).

Most frequent IEM scores. Most frequent IEM score was 3 for CBL (77.3%) and 1 for LLC (71.4%). (Table 2B).

Table 2A. Comparison of instructors’ behaviour showing percentage time with no engagement (scoring “1” on the IEM score)

Table 2B. Numbers (percentages) of a particular IEM score received for a 5-minutes segment of teaching – for instructors

Table 3A. Comparison of students’ behaviour showing percentage time with no engagement (scoring “1” on the IEM score)

Table 3B. Numbers (percentages) of a particular IEM score received for a 5-minutes segment of teaching – for students

D. Number of Questions Asked Per 5-minute Cycle

Median number of questions asked by instructors ranged from 0-2 for in-person teaching and 1-3 for online teaching (See Appendix 1). These range from 1-3 for CBL vs 0-1 for LLC.

Median number of questions asked by students in all sessions were 0.

The results for this study can be derived from the dataset uploaded onto the online repository accessed via https://doi.org/10.6084/m9.figshare.18133379.v1 (Chua et al., 2022).

IV. DISCUSSION

We modified the known STROBE instrument (O’Malley et al., 2003) to create an observational tool “IEM” which could be used to quantify instructor and student engagement despite the observer not being present in-person. Our IEM scores were derived by taking scores that were in agreement when independently scored by two main observers (YES and KN). The third observer (CYC) was used as the validator of the scores by the two main observers. When there was a discrepancy in the scores awarded by the two observers, the score which was in agreement with the score awarded by CYC was used. To give an indication of the IEM tool’s effectiveness where the observer is not present in-person, we postulated that our modified IEM score should still demonstrate the well-documented difference in engagement between lecture-style learning and case-based learning sessions (Kelly et al., 2005). Our modified IEM score did indeed show more frequent higher scores as expected for case-based learning sessions (Tables 2B and 3B). We also compared our IEM scores with the students’ self-reported engagement scores (Table 1) that had been collected as part of student feedback. The general correlation in the trend of observed IEM scores with that of the students’ self-reported engagement scores also suggest the usefulness of our modified STROBE tool in situations where the observer is not present in-person, although this needs to be further validated in prospective studies.

Our initial study hypothesis was that students may find themselves more engaged in online teaching sessions and open to posing questions to the instructor and their peers, due to the presence of the “chat”, “likes” and “poll” functions available on the Zoom tele-conferencing platform, which may be more familiar to a younger generation accustomed to using social media. We had postulated that live online lectures would encourage further engagement from students who would not otherwise participate in-person, due to the less intimidating online environment where they can ask and answer questions more anonymously (Kay & Pasarica, 2019; Ni, 2013). In an Asian-pacific context, video conferencing had been found to be able to improve access for participation for more reticent participants who prefer written expression, through alternative communication channels like the ‘chat box’, although there was a potential trend to reduced engagement. (Ong et al., 2021).

Our data, shows, that Zoom teleconferencing during the COVID-19 pandemic can be associated with reduced student engagement. The percentage time where there was no engagement was significantly higher with online sessions (Table 3A) and the most frequent IEM score was lower (1 for online vs 3 for in-person), for CBL sessions (Table 3B). This phenomenon in medical education during the COVID-19 pandemic has previously been described. Using student and instructor feedback, students were more likely to have reduced engagement during virtual learning (Longhurst et al., 2020; Dost et al., 2020), and would have increased difficulties maintain focus, concentration and motivation during online learning (Wilcha, 2020).

Our data also suggests that for the instructor to even try to achieve close to the same levels of engagement as before, a longer duration of time was spent by each instructor per topic when executing CBL (Table 1). This may include time where the instructor needs multiple attempts at questioning and discussion before there is a student response. It is also possible that for in-person learning, the instructor relies greatly also on non-verbal cues (e.g., body language, nods of the head, collective feel of the room) to determine if a question has been satisfactorily answered, and therefore can move on quicker than when on a Zoom platform where one cannot see most, or even every student.

The higher number of attendees for online learning compared to in-person attendance (see Table 1) highlights one of the strengths of online learning, which is where online learning is more easily accessible for students who would save on time getting to a designated lecture room and provides flexibility for students to enter and exit (Dost et al., 2020). Unfortunately, this also likely encourages the phenomenon of “presenteeism”, where students are not focused on the learning session, but instead engage in other tasks simultaneously, e.g., reading or composing emails, or completing work tasks instead of having dedicated protected teaching time. Resident learners have been described to participate in nearly twice as many non-teaching session related activities per hour during an online session than when in-person (Weber & Ahn, 2020). This has likely contributed to the number of 1* scores we had, where the student has logged into the Zoom platform, but is not available to even respond in the negative when called upon to answer a question. This presenteeism, however, is not just a problem for online learning, but even for in-person learning, where pretending to engage has been found to be a significant unrecognized issue (Fuller et al., 2018).

The main implication that our study highlights that to improve student engagement when using online learning, a face-to-face platform cannot simply be transposed into a virtual platform. It had been suggested that engagement during live virtual learning could be enhanced with the use of interactive quizzes with audience polling functions (Morawo et al., 2020) and possibly other methods such as “gamification” (Nieto-Escamez & Roldan-Tapia, 2021). Our instructors for the CBL sessions had used both poll functions and live questioning for their sessions, but without increased success in engagement. Smaller groups are likely required to enhance student engagement, but this would lead to the need for increased time and teaching manpower. Increasing the opportunity for interaction via a virtual platform would also require the need to create additional online resources, which would take up more faculty time where creating new resources can take at least three times as much work compared to a traditional format (Gewin, 2020). Online resources would need to be modified in such a way that increases student autonomy to increase student engagement in medical education (Kay & Pasarica, 2019). Our study also shows that as a first step, in time and resource-limited settings, a case-based approach to teaching would be more ideal to enhance student engagement than lecture style teaching.

A culture of accountability also needs to be fostered within the online teaching sessions, where students need to be educated on how Zoom meetings can be more enriching when cameras are on (Sharp et al., 2021). PGY-1 interns, as recent graduates, also need to be educated on the aspect of professionalism when entering the medical work force, where they can be called upon to answer questions during meetings or conferences. When initial questions are not voluntarily answered, our tutors often practice “cold-calling”, which can help keep learners alert and ready (Lemov, 2015). Unfortunately, these evidence-based teaching methods that work well when the student is in-person, ultimately will fail if online students are not educated on their need to be accountable to the instructor or their peers.

This study has several limitations. Firstly, the level of student engagement may also be affected by external factors, such as a different physical learning environment, class size and avenues of communication. The stresses of the on-going pandemic may also have affected student engagement, as a decrease in quality of life and stress would negatively impact student motivation (Lyndon et al., 2017). Secondly, the topics for lecture to large class and case-based learning were not identical as these topics were picked in chronological order and there were no topics in the curriculum that had material for both the lecture and case-based learning class types. This difference in topics may have potentially contributed to confounding when we try to make direct comparisons between the two class types, although, we have attempted to mitigate this by including a variety of topics in each class type. Thirdly, the improved student engagement and feedback scores for in-person learning may also have had some bias given the smaller student size for in-person learning. It is also possible that only the more motivated, and hence more likely to be engaged students, would turn up for in-person learning. Fourthly, due to the online nature as well as the retrospective viewing of the video recordings, the observers were not present in-person to observe the non-verbal cues of the students or instructors. The tool, however, was modified to take into account only the verbal output that could be observed online or via video recording. Lastly, our IEM tool will benefit from more studies and research to further confirm its validity in observing students when the observer is not present in-person.

V. CONCLUSION

Lectures are associated with reduced student engagement than case-based learning, while both class types are associated with lower levels of student engagement when conducted on a virtual platform. Instructor levels of engagement, however, remain about the same. This highlights that a face-to-face platform cannot simply be transposed into a virtual platform, and it is important to address this gap in engagement as this can lower faculty satisfaction with teaching and ultimately result in burnout. Blended teaching or smaller group teaching as the world turns the corner in the COVID-19 pandemic may be one way to circumvent the situation but is also constrained by faculty time and manpower. Our study also shows that as a first step, in time and resource-limited settings, a case-based approach to teaching would be more ideal to enhance student engagement than lecture style teaching.

Notes on Contributors

Dr Ng Wei Ping Kay and Dr Chua Yuan Kit Christopher are co-first authors and contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. They contributed to drafting and revising the work and approved the final version to be published. They agree to be accountable for all aspects of the work.

Dr Lye Pei Shi Priscillia contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. She contributed to drafting and revising the work and approved the final version to be published. She agrees to be accountable for all aspects of the work.

Dr Joy Vijayan contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.

Dr Yap Eng Soo contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.

Dr Chan Yee Cheun contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.

Ethical Approval

I confirm that the study has been approved by Domain Specific Review Board (DSRB), National Healthcare Group, Singapore, an institutional ethics committee. DSRB reference number: 2020/00415.

Data Availability

The data that support the findings of this study are openly available in Figshare at https://doi.org/10.6084/m9.fig share.18133379.v1.

Acknowledgement

We would like to acknowledge Ms. Jacqueline Lam for her administrative support in observing the recordings and online-teaching. 

Funding

There was no funding for this research study.

Declaration of Interest

The authors report no conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Alimoglu, M. K., Sarac, D. B., Alparslan, D., Karakas, A. A., & Altintas. (2014). An observation tool for instructor and student behaviors to measure in-class learner engagement: A validation study. Medical Education Online, 19(1), 24037. https://doi.org/10.3402/meo.v19.24037

Armstrong, S. J., & Fukami, C. V. (2009). The SAGE Handbook of Management Learning, Education and Development. SAGE Publications Ltd. https://www.doi.org/10.4135/9780857021038

Berman, N. B., & Artino, A. R. J., (2018). Development and initial validation of an online engagement metric using virtual patients. BMC Medical Education, 18(1), 213. https://doi.org/10.1186/s12909-018-1322-z

Chick, R. C., Clifton, G. T., Peace, K. M., Propper, B. W., Hale, D. F., Alseidi, A. A., & Vreeland, T. J. (2020). Using technology to maintain the education of residents during the COVID-19 Pandemic. Journal of Surgical Education, 77(4), 729–732. https://doi.org/10.1016/j.jsurg.2020.03.018

Chua, Y. K. C., Ng, K. W. P., Yap, E. S., Lye, P. S. P., Vijayan, J., & Chan, Y. C. (2022). Evaluating online learning engagement (Version 1) [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.18133379.v1

Dost, S., Hossain, A., Shehab, M., Abdelwahed, A., & Al-Nusair, L. (2020). Perceptions of medical students towards online teaching during the COVID-19 pandemic: A national cross-sectional survey of 2721 UK medical students. BMJ Open, 10(11), e42378. https://doi.org/10.1136/bmjopen-2020-042378

Fuller, K. A., Karunaratne, N. S., Naidu, S., Exintaris, B., Short, J. L., Wolcott, M. D., Singleton, S., & White, P. J. (2018). Development of a self-report instrument for measuring in-class student engagement reveals that pretending to engage is a significant unrecognized problem. PLOS ONE, 13(10), e0205828. https://doi.org/10.1371/journal.pone.0205828

Gewin, V. (2020). Five tips for moving teaching online as COVID-19 takes hold. Nature, 580(7802), 295–296. https://doi.org/10.1038/d41586-020-00896-7

Kaufman, D. M. (2003). Applying educational theory in practice. BMJ, 326(7382), 213–216. https://doi.org/10.1136/bmj.326.7382.213

Kay, D., & Pasarica, M. (2019). Using technology to increase student (and faculty satisfaction with) engagement in medical education. Advances in Physiology Education, 43(3), 408–413. https://doi.org/10.1152/advan.00033.2019

Kelly, P. A., Haidet, P., Schneider, V., Searle, N., Seidel, C. L., & Richards, B. F. (2005). A comparison of in-class learner engagement across lecture, problem-based learning, and team learning using the STROBE classroom observation tool. Teaching and Learning in Medicine, 17(2), 112–118. https://doi.org/10.1207/s15328015tlm1702_4

Lemov, D. (2015). Teach like a champion 2.0: 62 techniques that put students on the path to college. (2nd ed.). Jossey-Bass.

Longhurst, G. J., Stone, D. M., Dulohery, K., Scully, D., Campbell, T., & Smith, C. F. (2020). Strength, weakness, opportunity, threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and Republic of Ireland in response to the Covid-19 pandemic. Anatomical Sciences Education, 13(3), 301–311. https://doi.org/10.1002/ase.1967

Lyndon, M. P., Henning, M. A., Alyami, H., Krishna, S., Zeng, I., Yu, T.-C., & Hill, A. G. (2017). Burnout, quality of life, motivation, and academic achievement among medical students: A person-oriented approach. Perspectives on Medical Education, 6(2), 108–114. https://doi.org/10.1007/s40037-017-0340-6

McCoy, L., Pettit, R. K., Lewis, J. H., Allgood, J. A., Bay, C., & Schwartz, F. N. (2016). Evaluating medical student engagement during virtual patient simulations: A sequential, mixed methods study. BMC Medical Education, 16, 20. https://doi.org/10.1186/s12909-016-0530-7

Morawo, A., Sun, C., & Lowden, M. (2020). Enhancing engagement during live virtual learning using interactive quizzes. Medical Education, 54(12), 1188. https://doi.org/10.1111/medu.14253

Ni, A. Y. (2013). Comparing the effectiveness of classroom and online learning: Teaching research methods. Journal of Public Affairs Education, 19(2), 199-215. https://doi.org/10.1080/15236803.2013.12001730

Nieto-Escamez, F. A., & Roldan-Tapia, M. D. (2021). Gamifica- tion as online teaching strategy during COVID-19: A mini-review. Frontiers in Psychology, 12, 648522. https://doi.org/10.3389/fpsyg.2021.648552

O’Malley, K. J., Moran, B. J., Haidet, P., Seidel, C. L., Schneider, V., Morgan, R. O., Kelly, P. A., & Richards, B. (2003). Validation of an observation instrument for measuring student engagement in health professions settings. Evaluation & the Health Professions, 26(1), 86–103. https://doi.org/10.1177/0163278702250093

Ong, C. C. P., Choo, C. S. C., Tan, N. C. K., & Ong, L. Y. (2021). Unanticipated learning effects in videoconference continuous professional development. The Asia Pacific Scholar, 6(4), 135-141. https://doi.org/10.29060/TAPS.2021-6-4/SC2484

Rao, S. P., & DiCarlo, S. E. (2001). Active learning of respiratory physiology improves performance on respiratory physiology examinations. Advances in Physiology Education, 25(2), 55–61. https://doi.org/10.1152/advances.2001.25.2.55

Sharp, E. A., Norman, M. K., Spagnoletti, C. L., & Miller, B. G. (2021). Optimizing synchronous online teaching sessions: A guide to the “new normal” in medical education. Academic Pediatrics, 21(1), 11–15. https://doi.org/10.1016/j.acap.2020.11.009

Watson, W. E., Michaelsen, L. K., & Sharp, W. (1991). Member competence, group interaction, and group decision making: A longitudinal study. Journal of Applied Psychology, 76(6), 803–809. https://doi.org/10.1037/0021-9010.76.6.803 

Weber, W., & Ahn, J. (2020). COVID-19 conferences: Resident perceptions of online synchronous learning environments. Western Journal of Emergency Medicine, 22(1), 115–118. https://doi.org/10.5811/westjem.2020.11.49125

Wilcha, R. J. (2020). Effectiveness of virtual medical teaching during the COVID-19 crisis: Systematic review. JMIR Medical Education, 6(2), e20963. https://doi.org/10.2196/20963

*Chua Yuan Kit Christopher
5 Lower Kent Ridge Road,
National University Hospital,
Singapore 119074
+65 7795555
Email: christopher_chua@nuhs.edu.sg

Submitted: 6 January 2022
Accepted: 4 May 2022
Published online: 4 October, TAPS 2022, 7(4), 22-34
https://doi.org/10.29060/TAPS.2022-7-4/OA2735

Amelah Abdul Qader^1,2, Hui Meng Er³ & Chew Fei Sow³

¹School of Postgraduate Studies, International Medical University, Kuala Lumpur, Malaysia; ²University of Cyberjaya, Faculty of Medicine, Cyberjaya, Malaysia; ³IMU Centre for Education, International Medical University, Kuala Lumpur, Malaysia

Abstract

Introduction: The direct ophthalmoscope is a standard tool for fundus examination but is underutilised in practice due to technical difficulties. Although the smartphone ophthalmoscope has been demonstrated to improve fundus abnormality detection, there are limited studies assessing its utility as a teaching tool for fundus examination in Southeast Asian medical schools. This study explored the perception of medical students’ toward using a smartphone ophthalmoscope for fundus examination and compared their abilities to diagnose common fundal abnormalities using smartphone ophthalmoscope against direct ophthalmoscope. 

Methods: Sixty-nine Year-4 undergraduate medical students participated in the study. Their competencies in using direct ophthalmoscope and smartphone ophthalmoscope for fundus examination on manikins with ocular abnormalities were formatively assessed. The scores were analysed using the SPSS statistical software. Their perceptions on the use of smartphone ophthalmoscopes for fundus examination were obtained using a questionnaire.

Results: The students’ competency assessment scores using the smartphone ophthalmoscope were significantly higher than those using the direct ophthalmoscope. A significantly higher percentage of them correctly diagnosed fundus abnormalities using the smartphone ophthalmoscope. They were confident in detecting fundus abnormalities using the smartphone ophthalmoscope and appreciated the comfortable working distance, ease of use and collaborative learning. More than 90% of them were of the view that smartphone ophthalmoscopes should be included in the undergraduate medical curriculum.

Conclusion: Undergraduate medical students performed better in fundus examination on manikins with ocular abnormalities using smartphone ophthalmoscope compared to direct ophthalmoscope. Their positive perceptions toward smartphone ophthalmoscope support its use as a supplementary teaching tool in undergraduate medical curriculum.

Keywords:           Medical Students, Smartphone, Ophthalmoscope, Teaching Tool

Practice Highlights

• The smartphone ophthalmoscope is a useful supplementary teaching tool for fundus examination in undergraduate medical education.
• Fundus examination is performed at a safe working distance from the patient using a smartphone ophthalmoscope.
• Students are able to detect fundus abnormalities with greater ease and accuracy using a smartphone ophthalmoscope compared to a direct ophthalmoscope.
• Students appreciate the collaborative learning through peer discussion of the fundus findings using the smartphone ophthalmoscope.

I. INTRODUCTION

Fundus examination is one of the essential procedures which provides information about ocular conditions that may compromise the quality of vision and lead to blindness (Leonardo, 2018). The direct ophthalmoscope (DO) is one of the robust ocular clinical examination tools to be grasped during clinical skill training in medical schools as well as clinical practice. However, students have difficulty mastering the technique of using it  (Kim & Chao, 2019), particularly when they have to coordinate their hand movements at a very near distance to the patient and close one eye when examining the patient’s fundus through the pupil  (MacKay et al., 2015). They also have to adjust the power of the direct ophthalmoscope lenses to get a clearer picture if there is a refractive error with the patient’s eye or their own eyes. Instead of concentrating on detecting fundus findings, the students are preoccupied with adjusting the direct ophthalmoscope.

Technical constraints may be the main reason for the underuse of direct ophthalmoscopes. Experienced physicians who use the direct ophthalmoscope may lack confidence and frequently miss significant abnormalities (Purbrick & Chong, 2015),  causing delayed diagnosis of preventable eye disorders and permanent vision impairment (Myung et al., 2014). This has led to the exploration of alternative tools to overcome some of these challenges (Giardini et al., 2014; Kim & Chao, 2019). Smartphone ophthalmoscope, for example, is a breakthrough digital portable retinal imaging system that allows medical practitioners to view the fundus with high-definition images or video of a routine ophthalmoscope examination.

The D-EYE smartphone ophthalmoscope was developed by Doctor Andrea Russo in 2015 (Russo et al., 2015). It is a small, portable, and inexpensive retinal imaging system that can capture retinal images using an attachment to a smartphone that uses a cross-polarisation technique to reduce corneal reflections. It is integrated with the smartphone’s autofocus feature to accommodate the patient’s refractive error.  

A. Problem and Rationale

Fundus examination requires extensive practice to develop adequate interpretation skills (Leonardo, 2018). Medical students are taught to use the direct ophthalmoscope in order to recognise retinal signs of life-threatening disorders (Benbassat et al., 2012). The International Council of Ophthalmology recognises direct ophthalmoscope examination as one of the seven core ocular medical education competencies. All graduating medical students are expected to recognise common abnormalities of the ocular fundus using a direct ophthalmoscope (Dunn et al., 2021). However, there is a lack of competencies among the medical graduates using this tool (MacKay et al., 2015). This needs to be addressed as at least 2.2 billion people globally have visual impairment or blindness, of which at least 1 billion have deterioration in vision that could have been prevented if they were screened or detected earlier, World Health Organization (2019). Tan et al. (2020) reported several favourable studies carried out in Italy, UK, and India on the advantages of smartphone ophthalmoscopes for fundus examination and visualisation of the retinal image. In a randomised cross-over study done by Curtis et al. (2021) on the ease of use of D-EYE smartphone ophthalmoscope versus direct ophthalmoscope, 44 Year-one medical students in Canada examined the patients’ fundus for optic disc assessment and compared their findings with the respective photographs provided. The ease of use and confidence was more significant with the D-EYE smartphone ophthalmoscope.

Although the smartphone ophthalmoscope is available in Southeast Asian countries such as Malaysia, it is not commonly used in public hospitals and general practitioner clinics. This is probably due to resource constraint issues in developing countries. Moreover, there are limited studies assessing its use, in particular, there is no literature report on such studies among undergraduate medical students in Southeast Asia. However, based on the positive findings from the literature, it has been proposed that smartphone ophthalmoscope be included in clinical skill training for fundus examination among undergraduate medical students at the university where this study was conducted. Therefore, this study was carried out to explore the students’ perceptions of using smartphone ophthalmoscopes for fundus examination and determine whether their competencies in fundus examination improved using this tool compared to using the direct ophthalmoscope. In this study, the D-EYE smartphone ophthalmoscope was chosen over the other types of smartphone ophthalmoscope due to reasons including the ease of data management using the available app facility, cost feasibility and convenience. The two research questions of the study were:

1) What were the perceptions of medical students on the use of smartphone ophthalmoscope for fundus examination?

2) Was there a difference between students’ competencies in fundus examination when using the smartphone ophthalmoscope compared to the direct ophthalmoscope?

The cognitive theory of multimedia learning can be applied in the context of fundus examination using a smartphone ophthalmoscope. Using a smartphone ophthalmoscope, the student can visualise the fundus on the smartphone screen. According to the cognitive theory of multimedia learning (Figure 1), the students engage in active cognitive processing in order to create a cohesive mental representation of their experiences based on their recall knowledge of fundus structures and ocular abnormalities. This will allow them to integrate the findings with other relevant information. They can then describe their findings and organise the selected images into a “mental model” of the items they are learning. Finally, their prior knowledge of ocular disorders is incorporated and reconciled with these verbal explanations and graphical representations.

Figure 1. Cognitive theory of multimedia learning

According to the social constructivism theory, learning is social, active and constructed through social interaction (Lötter & Jacobs, 2020). Technologies have been shown to enhance students’ problem solving by breaking down complex concepts into sub-problems (Kim & Hannafin, 2011). A smartphone ophthalmoscope is an appropriate tool for encouraging active interaction between the students and lecturer to work on real-world problems in the teaching and learning environment. When the students perform fundus examination using the smartphone ophthalmoscope, they can see the findings on the screen together with their peers and the lecturer. This will allow them to gain more knowledge and understanding as they can discuss and link the new ideas in the context of their prior knowledge.

II. METHODS

The study was approved by the International Medical University Joint-Committee on Research and Ethics (IMU-JC). Informed consent was obtained from all respondents. The nature and purpose of the study were explained to them. The respondents were assured of anonymity and confidentiality of the collected information.

A. Study Setting

The data were collected from Year-4 undergraduate medical students who undertook ophthalmology rotation for the academic year 2020/2021, at the University of Cyberjaya, Malaysia.

In the fourth year of the medical curriculum, the students undertake four major postings (Orthopedic, Family Medicine, Psychiatry and a speciality posting) over two semesters (Semesters 7 and 8). These are conducted in four rotations per year (rotations 1 & 2 in Semester 7, rotations 3 & 4 in Semester 8). The speciality posting includes Ophthalmology, Anaesthesia, ENT and Radiology. The duration of each of these speciality posting is two weeks. In the ophthalmology posting the students are taught the principles of history taking and ocular examination in the Clinical Skill Training Department and in the hospital, where they clerk patients with eye conditions. Additionally, they learn about basic common eye conditions during interactive sessions and case-based discussion sessions in small groups. However, during the COVID-19 pandemic, the posting was affected by lockdown measures. Therefore, the case-based discussion sessions were conducted online, and ocular examination was demonstrated through online interactive video sessions. Nevertheless, there was a window of opportunity where the students could return to the campus physically for a one-week revision. During this period, the students practice ophthalmoscopy examination on manikins in the Clinical Skill Training Department.

B. Study Design

The direct ophthalmoscope examination technique was introduced to the students virtually through video demonstrations and during online interactive discussion sessions. During the revision week, the students were trained for two hours to perform fundus examination using the direct ophthalmoscope. For the smartphone ophthalmoscope, they were briefed and trained on its use for 20-30 minutes. The training was conducted by a member of the teaching staff (who is the researcher in this study, AMAQ). Following that, the students were required to examine various slides of fundus images provided in the manikins (M1 and M2).

The selected slides on the manikins represented the common pathological fundus findings, i.e. optic disc swelling, branch retinal vein occlusion, optic atrophy/glaucoma and diabetic retinopathy/ maculopathy. Each student performed the fundus examination on M1 and M2 using the direct and smartphone ophthalmoscopes separately (approximately 2-3 minutes on each manikin) on the same day. The students were required to fill in their findings based on their observation (without discussing with their peers) on the formative assessment forms (shown in Appendix 1) and indicate the tools they utilised (direct or smartphone ophthalmoscopes). The formative assessment form was adapted from Mamtora et al. (2018) and had been validated by two ophthalmologists in the department.

To avoid bias, all the completed formative assessment forms were collected and submitted to another researcher (SCF) who was not involved in marking (to remove information on the tool used by the student on each form). These were then returned to the researcher in this study, (AMAQ) for marking.

After completing the formative assessment, the students were requested to fill in an online questionnaire regarding their perception on the use of smartphone ophthalmoscope for fundus examination. This questionnaire (Appendix 2) was adapted from Nagra & Huntjens (2020). In addition, the students were requested to provide the reasons for their suggestions to include smartphone ophthalmoscopes or replace direct ophthalmoscopes with smartphone ophthalmoscopes in the medical curriculum.

C. Data Analysis

All data were statistically analysed using SPSS version 23. The paired t-test was used to compare the performance of the students in the formative assessments using direct ophthalmoscopes and smartphone ophthalmoscopes. The number of students getting the correct diagnosis using both tools was statistically analysed using the McNemar (Chi Square) test (Liao & Lin, 2008). The statistical significance was determined based on the p-values (the difference is significant if p ≤ 0.05). The responses of the students in the perceptions questionnaire related to the ease of use, confidence, and preference were analysed.

III. RESULTS

Sixty-nine Year-4 medical students participated in this study. The demographic data are shown in Table 1.

Table 1: Demographic data

A. Comparison of Formative Assessment Scores Using Smartphone Ophthalmoscope and Direct Ophthalmoscope

The mean scores of the students were higher using the smartphone ophthalmoscope (59%) than the direct ophthalmoscope (39%). The same trend was observed for the students with and without refractive error. The results are shown in Table 2. A higher number of students were able to make the correct diagnosis for all fundus abnormalities using the smartphone ophthalmoscope compared to the direct ophthalmoscope. The difference is statistically significant (p-value < 0.05). The results are presented in Table 3. The data that support the findings are openly available in Figshare at https://figshare.com/s/d45da87ea42c596e714b

Table 2: Comparison of formative assessment scores using direct ophthalmoscope (DO) and smartphone ophthalmoscope (SPO).

*p-value (paired t-test)

Submitted: 22 September 2021
Accepted: 27 April 2022
Published online: 4 October, TAPS 2022, 7(4), 1-21
https://doi.org/10.29060/TAPS.2022-7-4/OA2785

Yao Chi Gloria Leung¹*, Kennedy Yao Yi Ng²*, Ka Shing Yow³*, Nerice Heng Wen Ngiam⁴, Dillon Guo Dong Yeo⁴, Angeline Jie-Yin Tey⁵, Melanie Si Rui Lim⁶, Aaron Kai Wen Tang⁷, Bi Hui Chew⁸, Celine Tham⁹, Jia Qi Yeo¹⁰, Tang Ching Lau^11,12, Sweet Fun Wong^13,14, Gerald Choon-Huat Koh^15,16** & Chek Hooi Wong^14,17**

¹Department of Anaesthesiology, Singapore General Hospital, Singapore; ²Department of Medical Oncology, National Cancer Centre Singapore, Singapore; ³Department of General Medicine, National University Hospital, Singapore; ⁴Department of General Medicine, Singapore General Hospital, Singapore; ⁵Department of General Medicine, Tan Tock Seng Hospital, Singapore; ⁶Department of General Paediatrics, Kandang Kerbau Hospital, Singapore, ⁷Department of Psychiatry, Singapore General Hospital, Singapore; ⁸Tan Tock Seng Hospital, Singapore; ⁹Ng Teng Fong General Hospital, Singapore, ¹⁰National Healthcare Group Pharmacy, Singapore, ¹¹Department of Medicine, NUS Yong Loo Lin School of Medicine, Singapore; ¹²Division of Rheumatology, University Medicine Cluster, National University Hospital, Singapore; ¹³Medical Board and Population Health & Community Transformation, Khoo Teck Puat Hospital, Singapore; ¹⁴Department of Geriatrics, Khoo Teck Puat Hospital, Singapore; ¹⁵Saw Swee Hock School of Public Health, National University of Singapore, Singapore; ¹⁶Future Primary Care, Ministry of Health Office of Healthcare Transformation, Singapore; ¹⁷Health Services and Systems Research, Duke-National University of Singapore Medical School, Singapore

*Co-first authors

**Co-last authors

Abstract

Introduction: Tri-Generational HomeCare (TriGen) is a student-initiated home visit programme for patients with a key focus on undergraduate interprofessional education (IPE). We sought to validate the Readiness for Interprofessional Learning Scale (RIPLS) and evaluate TriGen’s efficacy by investigating healthcare undergraduates’ attitude towards IPE.

Methods: Teams of healthcare undergraduates performed home visits for patients fortnightly over six months, trained by professionals from a regional hospital and a social service organisation. The RIPLS was validated using exploratory factor analysis. Evaluation of TriGen’s efficacy was performed via the administration of the RIPLS pre- and post-intervention, analysis of qualitative survey results and thematic analysis of written feedback.

Results: 79.6% of 226 undergraduate participants from 2015-2018 were enrolled. Exploratory factor analysis revealed four factors accounting for 64.9% of total variance. One item loaded poorly and was removed. There was no difference in pre- and post-intervention RIPLS total and subscale scores. 91.6% of respondents agreed they better appreciated the importance of interprofessional collaboration (IPC) in patient care, and 72.8% said MDMs were important for their learning. Thematic analysis revealed takeaways including learning from and teaching one another, understanding one’s own and other healthcare professionals’ role, teamwork, and meeting undergraduates from different faculties.

Conclusion: We validated the RIPLS in Singapore and demonstrated the feasibility of an interprofessional, student-initiated home visit programme. While there was no change in RIPLS scores, the qualitative feedback suggests that there are participant-perceived benefits for IPE after undergoing this programme, even with the perceived barriers to IPE. Future programmes can work on addressing these barriers to IPE.

Keywords:           Interprofessional Education, Student-Initiated Home Visit Programme, RIPLS, Validation

Practice Highlights

• We validated the Readiness for Interprofessional Learning Scale (RIPLS) in Singapore, a multi-ethnic Asian country.
• A student-initiated, interprofessional, longitudinal home visit program is feasible.
• While there was no significant change in RIPLS scores, participants reported qualitative benefits of the programme in their attitudes towards IPE.
• Qualitative feedback highlighted four main barriers to IPE: Time constraints, unmotivated teammates, administrative burden, and unsuitable patients.

I. INTRODUCTION

Interprofessional education (IPE) aims to prepare healthcare professionals for effective collaboration, and while becoming increasingly common, is challenging to initiate, implement, evaluate and sustain (Fahs et al., 2017). Key challenges include designing a curriculum that integrates IPE with traditional academic frameworks, active engagement of facilitators and students, and accommodating various professions (Sunguya et al., 2014). IPE is context-specific, evolving, and involves continuous interaction and interdependence, and many traditional top-down approaches such as forums and lectures do not effectively teach it (Briggs & McElhaney, 2015).

Experiential IPE programmes employ a ground-up approach and potentially tackle some of the aforementioned challenges. Students involved in on-the-ground interprofessional healthcare visits to older adults showed that such experiences improved student collaboration and students’ self-perception of interprofessional team care-related skills (Blythe & Spiring, 2020; Conti et al., 2016; McManus et al., 2017; Toth-Pal et al., 2020; Vaughn et al., 2014). Therefore, a group of undergraduates from the National University of Singapore (NUS) Yong Loo Lin School of Medicine initiated an experiential student-led IPE programme which is aimed at improving health outcomes in older people with frequent hospital readmissions. This longitudinal service-learning programme was anchored by several educational aims including enhancing students’ IPE outcomes and improving attitudes towards IPE.

Formal evaluation of such programmes and investigating student IPE attitudes after being involved in a longitudinal home visit programmes are lacking in the current IPE literature (Grice et al., 2018). This study aims to evaluate TriGen’s effectiveness by investigating student IPE attitudes through the use of the Readiness for Interprofessional Learning Scale (RIPLS). Since the RIPLS has not been validated in the Singapore context, this study also aims to validate this scale.

II. METHODS

A. Programme Design

TriGen is a collaboration between NUS Yong Loo Lin School of Medicine, Khoo Teck Puat Hospital, a Northern regional hospital in Singapore, and North West Community Development Council, a grassroots organisation (Ng et al., 2020a, 2020b). A non-profit ground-up social initiative by healthcare undergraduates, it has the dual aim of i) serving the medical and social needs of older patients by providing longitudinal home visits by interprofessional student teams; ii) educating and empowering undergraduate students through a service-learning approach, with a key focus on improving attitudes towards IPE. The programme was designed under the mentorship of university faculty members, and was earmarked as a co-curricular activity aimed at improving students’ attitudes towards IPE and IPC. Older patients with frequent hospital readmissions (three or more times over six months) were followed up by healthcare undergraduates enrolled in Medicine, Nursing, Pharmacy, Social Work, Physiotherapy or Occupational Therapy courses in Singapore.

The programme begins with healthcare undergraduates undergoing didactic, skill-based training and team-based simulation training covering possible scenarios encountered during home visits (Annex 1). Each team comprising 2-3 interdisciplinary undergraduates conduct fortnightly visits to 1-2 patients over 6 months. At the midpoint and endpoint of the programme, healthcare undergraduates assessed the patients’ needs and presented at multi-disciplinary meetings (MDMs) chaired by healthcare professionals and grassroots staff, who guided the undergraduates to execute a management plan.

This IPE programme was designed based on educational principles for adult learners outlined by Knowles (1984). First, it provided healthcare undergraduates with opportunities for experiential learning anchored in the service-learning approach. Second, it was largely problem-based group learning with most training sessions being team-based and scenario-based. MDMs were also problem-based and encouraged undergraduates to brainstorm ideas to address their patients’ issues. Third, the service they provided in this programme modelled the work they may engage in after graduation. What they learned in this programme was of immediate relevance to their current study and future practice. Lastly, the programme provided autonomy to healthcare undergraduates to direct their own learning. This programme was voluntary and allowed participants’ flexibility for further self-study of topics of interest. Key student outcomes include readiness for IPE (including teamwork and collaboration, professional identity, roles and responsibility), and a better appreciation for IPC.

B. Evaluation Approach

This study used the framework by Kirkpatrick (1959) expanded by Barr et al. (2005) to evaluate the effectiveness of TriGen in improving healthcare undergraduates’ attitudes towards IPE, particularly in evaluation levels 1, 2a and 2b, which centre on learner’s reactions, attitude perceptions, and acquisition of knowledge or skills (Table 1). The use of quantitative and qualitative data collection in a survey was thought to be most appropriate in capturing the data and making the evaluation richer, and was hence the approach utilised for this research (Figure 1).

Evaluation Level	Methods and Measures	Timeframe
Level 1: Learners’ reactions Participants’ views of their learning experience and opinions about the program	Participants’ self-reported feedback of IPE learning	Post-intervention
	Qualitative feedback	Post-intervention
Level 2a: Modification of attitudes perceptions	Participants’ self-reported feedback of IPE learning	Post-intervention
	Qualitative feedback	Post-intervention
	Readiness for Interprofessional Learning Scale	Pre- and post-intervention
Level 2b: Acquisition of knowledge/skills Concepts, procedures, principles, and skills	Qualitative feedback	Post-intervention

Table 1: Components of Kirkpatrick/Barr et al. evaluation framework as applied to TriGen

Figure 1: Flowchart of study components

C. Quantitative Measures

The RIPLS (Parsell & Bligh, 1999) was among the first scales developed for measurement of attitudes towards interprofessional learning. It assesses student readiness for IPE and IPC with other health care professionals and has been reported to be sensitive to differences in the students’ attitude towards IPE (Berger-Estilita et al., 2020). While there are a few studies validating it in Asian countries (China, Indonesia, Japan), none have been performed in Singapore (a multi-ethnic Asian country with English language as a predominant language of instruction (Ganotice & Chan, 2018; Lestari et al., 2016; Li et al., 2018; Tamura et al., 2012).

The RIPLS, a 19-item questionnaire comprising 4 subscales (“Teamwork and Collaboration”; “Positive Professional Identity”; “Negative Professional Identity” and “Roles and Responsibilities”), was administered pre- and post-intervention (McFadyen et al., 2005). Higher RIPLS scores imply greater readiness for interprofessional learning. This study validates the RIPLS in the Singapore context for the first time, then employs it for quantitative evaluation of TriGen. Additionally, separate from the RIPLS, three questions were added as a direct measure of participants’ reaction (Level 1), “I better appreciate the importance of IPC in the care of patients through the programme”, “The multidisciplinary meetings organised were important for my learning”, and “I would recommend the programme to my friends.”

1) Statistical Analysis: The Shapiro-Wilk test was used to assess if the data followed a normal distribution (Shapiro & Wilk, 1965). Factor analysis was conducted to explore the construct validity of the RIPLS, and Cronbach’s alpha was computed to determine internal consistency. The suitability of the correlation matrix was determined by the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity. The numbers of factors retained for the initial solutions and entered into the rotation were determined with the application of Kaiser’s criterion (eigenvalues >1). The initial factor extraction was performed using principal component analysis. Exploratory factor analysis was then conducted based on the RIPLS four-subscale structure. A paired t-test comparing baseline and post-intervention responses was computed for each survey item to determine significant differences (p ≤ 0.05). One-way ANOVA was performed to assess for demographic factors that correlated with pre-intervention and magnitude of change in RIPLS scores; if it demonstrated an overall difference between groups, post-hoc Tukey’s HSD was performed. For all statistical analyses, the Statistical Package for Social Sciences (SPSS, Version 23.0, Chicago, Illinois) was used.

D. Qualitative Measures

Post-intervention qualitative feedback regarding participants’ learning experiences was collected through online surveys. Questions include: What did you learn about interprofessional collaboration? What are your learning points after completing the project? Would you recommend this project to your peers, and what are your reasons? These questions were chosen to better understand participants’ reaction to the programme, their attitudes toward IPE and IPC, and other key learning points they may have.

1) Thematic Analysis: All survey participants were encouraged to participate in the qualitative research, with a total of 163 recruited to give written qualitative feedback on the programme. Given the relatively large sample of data, thematic analysis was chosen to explore and interpret the dataset, distilling it into recurring ideas (Braun & Clarke, 2006; Kiger & Varpio, 2020). Analysis was performed on participants’ qualitative descriptions of their learning experiences, with constant comparison analysis used to identify patterns in participants’ responses and develop a coding schema. Two coders independently identified major themes from the text within all transcripts, with reference to the research questions. They discussed and resolved any disagreements. No member checking was performed. A common coding schema was generated and applied to all the transcripts.

E. Ethical Approval

Ethical approval was obtained from the NUS institutional review board (B-15-272). Study participation was entirely voluntary and anonymous. Informed consent was taken from participants before data collection commenced, and they were allowed to withdraw from the research at any point in time. No incentives were provided to study participants.

III. RESULTS

226 healthcare undergraduates participated in TriGen from 2015-2018. Response rate for the RIPLS was 79.6%.

A. Demographics

Median age was 21 (range 18-41). 62.2% of participants were female, 37.8% were male. 31.7% were medical students, 12.8% nursing students, 42.2% pharmacy students, 10.0% social work students, and 3.3% therapy students. First- and second-year students comprised 62.2% of participants, while third- to fifth-years comprised 37.8%. 65.0% participated in previous IPE activities.

B. Construct Validity

The KMO index was 0.902, indicating sampling adequacy. The chi-square index for Barlett’s test of sphericity was 1919.445 (df171, p<0.001), indicating suitability for factor analysis.

Principal component analysis yielded four components largely consistent with the four-subscale model of the RIPLS (Barr et al., 2005) (Annex 2). However, one item, “I am not sure what my professional role will be”, had a low loading value of 0.285 under the original subscale of “Roles and Responsibility” and a borderline low loading value of 0.459 under the subscale of “Negative Professional Identity”. It was removed from subsequent analyses in view of its poor fit into the theoretical construct (Table 2).

No	Statements	Teamwork and Collaboration	Negative Professional Identity	Positive Professional Identity	Roles and Responsibilities
1	Shared learning will help me to think positively about other healthcare professionals.	0.794
2	Learning with other health and social care students before qualification would improve relationships after qualification.	0.781
3	Team-working skills are essential for all health and social care students to learn.	0.773
4	Shared learning will help me to understand my own limitations.	0.767
5	Communication skills should be learnt with other health and social care students.	0.746
6	Learning with other students/professionals will make me become a more effective member of a health and social care team.	0.742
7	For small group learning to work, students need to trust and respect each other.	0.739
8	Shared learning with other healthcare students will increase my ability to understand clinical problems.	0.723
9	Patients would ultimately benefit if health and social care students/professionals worked together to solve patient problems.	0.650
10	It is not necessary for undergraduate health and social care students to learn together.		0.882
11	I don’t want to waste time learning with other health and social care students.		0.854
12	Clinical problem-solving skills can only be learnt with students from my own department.		0.799
13	Shared learning will help to clarify the nature of patient problems.			0.658
14	Shared learning before qualification will help me become a better team worker.			0.642
15	I would welcome the opportunity to work on small group projects with other health and social care students.			0.614
16	Shared learning with other health and social care professionals will help me to communicate better with patients and other healthcare professionals.			0.567
17	The function of nurses and therapists is mainly to provide support for doctors.				0.836
18	I am not sure what my professional role will be.		0.459*		0.285
19	I have to acquire much more knowledge and skills than other health or social care students.				0.517

Table 2: Exploratory Factor Analysis of the RIPLS – Contribution of Items to Each Component

*The highest loading value of each item under the four subscales are shown (except for item 18). A loading value of >0.5 was taken to be satisfactory. Item 18, “I am not sure what my professional role will be.”, was deemed borderline satisfactory at a loading value of 0.459 in the subscale Negative Professional Identity. Its loading value was lower at 0.285 in its original subscale Roles and Responsibility.

D. Baseline RIPLS Score

Table 3: Total RIPLS scores.

Subscale scores can be found in Annexes 3 to 6.

Table 4: RIPLS (Individual items analysis)

G. Self-Reported Feedback on Interprofessional Learning

91.6% participants agreed they could “better appreciate the importance of interprofessional collaboration in the care of patients”. 72.8% said MDMs were important for their learning and 91.9% of respondents would recommend the programme to their friends.

H. Qualitative Feedback

163 of 180 survey respondents participated in the qualitative research (response rate 90.6%). (Fig 1) 34.4% of respondents were male and 65.6% female. 33.1% of respondents were studying Medicine, 12.3% Nursing, 40.5% Pharmacy, 11.0% Social Work and 3.1% Therapy. 54.6% of respondents were in early years of study (Year 1–2). 74.8% had previous exposure to IPE. Thematic analysis yielded the following themes:

1) Learning and teaching one another: Healthcare undergraduates found value in learning from one another. They shared knowledge and skills gained from their respective curriculum with one another.

 

“I feel more equipped and prepared to teach and learn from other healthcare professionals”

21-year-old female third-year medical student

 

“I learnt a lot from my social work team leader and how to consider the social aspects of issues the elderly face”

20-year-old male first-year medical student

 

2) Understanding the role of other healthcare professionals: Healthcare undergraduates learned the role of other healthcare professionals and gained new insights into how different healthcare professionals contributed to the care of the patient.

 

[I have] learn[ed] … how we can tap on each other[’s] strengths to come up with a care plan for the patients

21-year-old female third-year pharmacy student

 

Understanding what medicine, nursing [and] pharmacy does make quite a lot of difference to how we perceive and thus, work with them.

23-year-old female second-year social work student

 

3) Understanding one’s own role: Healthcare undergraduates reported developing a greater understanding of the roles and responsibilities they played as a part of a multi-disciplinary team.

 

I am now more aware of the role and responsibility I have as a healthcare professional.

21-year-old female first-year pharmacy student

 

Working in a multi-disciplinary team gave me a feel of how it may be like caring for a patient as a team in my future career.

20-year-old female first-year social work student

 

4) Teamwork: Healthcare undergraduates appreciated the need for collaboration and teamwork within a multi-disciplinary team. They learned about the importance of compromise.

Working with different people, in terms of personality, faculty, etcetera – I learnt to give and take and be more understanding towards the others.

21-year-old second-year social work student

 

It has allowed me to better understand … how the different professions can come together to better serve the needs of patients.

20-year-old female second-year pharmacy student

 

5) Opportunity to meet people from other faculties: Healthcare undergraduates valued meeting people from other faculties and developing collaborative relationships they would otherwise not have had the opportunity to.

 

I got to know seniors in medicine and peers from pharmacy.

20-year-old female first-year nursing student

 

It is a very unique experience, having the chance to interact with … other university students from different healthcare faculties.

20-year-old female second-year pharmacy student

 

6) Factors limiting learning: Factors limiting learning included time constraints, unmotivated teammates, administrative burden and lack of suitable patients. For the latter, some undergraduates felt that their care was restricted to companionship for patients who were already able to manage their own chronic conditions well and did not require further help from the healthcare undergraduates.

IV. DISCUSSION

A. Validation of the RIPLS in Singapore

This study validated the RIPLS in the Singapore context. The final model is the same as proposed by McFadyen et al. (2005). without item 18 “I am not sure what my professional role will be”, from “Roles and Responsibility” subscale. The poor fit of this item into this study’s theoretical construct could be because participants are mostly in their pre-clinical years and may not understand professional roles and responsibilities due to their limited on-job experience, a reason also proposed by McFadyen et al. (2005) and Tyastuti et al. (2014). Tyastuti et al. (2014) found this item, along with “I have to acquire much more knowledge and skills than other healthcare students” (item 19) from the same subscale had loading factors of <0.5 and removed the entire “Roles and Responsibility” subscale from the Indonesian version of the RIPLS. Other studies validating the RIPLS also experienced issues with this subscale (Lauffs et al., 2008; Lestari et al., 2016; McFadyen et al., 2005).

B. Baseline RIPLS score

The mean baseline RIPLS score is comparable with that by Chua et al. (2015), another study conducted in Singapore which measured change in the RIPLS after a one-day IPE conference. They also found higher baseline RIPLS scores for medical undergraduates versus other faculties, a finding also noted in this study and another done in a culturally similar country (Lestari et al., 2016). However, this finding seems inconsistent as other studies (Aziz et al., 2011; de Oliveira et al., 2018) have found the contrary.

Chua et al. (2015) also found that prior IPE experience resulted in higher baseline RIPLS scores, a finding not replicated in this study. We hypothesise that while 65.0% of this study’s participants had previous IPE exposure (versus 10.6% in Chua et al. (2015)), the heterogenous nature of IPE programmes they previously participated in may have had differing efficacy in improving IPE attitudes.

This study found undergraduates in their later years had a lower baseline “Teamwork and Collaboration” subscale score, versus those in their early years. We postulate that undergraduates with more clinical experience better understand the challenges of IPE in practice, a finding echoed by Judge et al. (2015).

That pharmacy students, but not medical students, were mandated by their curriculum to fulfil volunteering hours which could explain the former’s lower baseline scores for total RIPLS and subscales “Teamwork and Collaboration” and “Positive Professional Identity” since they are likely less motivated by IPE when choosing to participate.

Social work undergraduates’ low baseline “Roles and Responsibility” score likely reflects their minimal exposure to medical social work unless they elected for healthcare modules in their senior years of study.

C. Change in Pre- and Post-intervention RIPLS Scores

Our study did not show a significant difference between the pre- and post-intervention RIPLS total score and the “Teamwork and Collaboration” subscale. Additionally, there was a decrease seen in post-intervention scores under the “Positive Professional Identity” subscale for Year 1-2 students and the “Negative Professional Identity” subscale in medical students and social work students. This is in contrast with the literature, where previous studies involving conferences (Chua et al., 2015) or solitary learning modules (Wakely et al., 2013; Zaudke et al., 2016) demonstrated a significant difference in the total RIPLS score pre- and post- intervention. Possible reasons for this are further discussed in section E.

There was a significant increase in the post-intervention score amongst female students under the “Roles and Responsibility” subscale. Previous studies have suggested that there are gender specific differences in perception towards IPE with female students having a more positive attitude towards IPE (Hansson et al., 2010; Wilhelmsson et al., 2011). In addition, the individual item analysis showed that negatively coded statements relating to the subscale of “Roles and Responsibility” such as “the function of nurses and therapists is mainly to provide support for doctors” (Item 17) and “I am not sure what my professional role will be” (Item 18) had significant increases in scores post-intervention. This is encouraging and demonstrates the success of the programme in helping students understand the respective roles and responsibility of each profession which is a crucial part of IPE and eventually IPC.

Other significant findings in the individual item analysis include a decrease in scores for the statements “Shared learning with other health and social care professionals will help me to communicate better with patients and other healthcare professionals” (Item 13), and “Shared learning will help to clarify the nature of patient problems” (Item 15). These findings suggest that the programme can be improved by incorporating more modules on communication between healthcare professionals and shared problem-solving.

D. Qualitative Feedback

While the lack of a significant difference between the pre- and post-intervention RIPLS scores suggest no changes in attitudes, the qualitative data revealed that the majority of undergraduates better appreciated the importance of IPC for patient care and many felt that that MDMs were useful for their learning.

Qualitative analysis revealed five major themes in the undergraduates’ learning pertaining to IPE. Participants learned from and taught each other. Being able to freely learn from and teach one another requires mutual trust and respect which are key elements of collaborative practices (de Oliveira et al., 2018). Participants reported better understanding of their own and other healthcare professionals’ roles; these are recognised as crucial components of collaborative practice (Canadian Interprofessional Health Collaborative, 2010). Undergraduates also shared that they learned about teamwork, specifically, conflict resolution and compromise. Finally, undergraduates appreciated the opportunities to meet fellow undergraduates from different faculties. It has been observed in many successful IPE programmes that informal social interactions are potentially as important as the actual IPE activities (Lie et al., 2016). We observed that the relationships built between participants of the programme often persisted beyond the completion of the programme; these relationships could benefit the institution and healthcare system (Hoffman et al., 2008).

E. Possible Reasons Underlying Lack of Improvement in RIPLS Scores

First, as mentioned earlier, the RIPLS has been described to have psychometrics issues, with multiple researchers modifying the subscales (Mahler et al., 2015). Second, Schmitz and Brandt (2015) suggested that RIPLS is insensitive to course improvements and to pre- versus post-intervention change in attitudes. We chose the RIPLS at the start of 2014 as it had been widely used and validated and simple to administer, and we also sought to validate it in Singapore for the first time. Unfortunately, few studies on its potential issues had been published at the time to inform the design of this study. Third, the longitudinal nature of the programme may have permitted undergraduates greater insight to the challenges of IPE and realities of collaborating within interprofessional teams, tampering their idealism.

Lestari et al. (2016) described how nursing and midwifery undergraduates had lower RIPLS scores as compared to medical and dentistry undergraduates as they had prior clinical experience and likely observed less than exemplary interactions amongst members of healthcare teams. Similarly, Makino et al. (2013) found that graduates of an IPE programme had a lower mean score on the Modified Attitudes Toward Health Care Teams Scale (ATHCTS) as compared to current students. The authors suggested that the alumni’s negative attitude may be due to their real-world experience. Several structural issues in clinical practice have been identified that contribute to this trend, for example competition between professionals (Tremblay et al., 2010) and power struggles (Paradis & Whitehead, 2015).

F. Barriers to IPE

Undergraduates reported four main barriers: time constraints, unmotivated teammates, administrative burden, unsuitable patients. Other studies including Alexandraki et al. (2017) and West et al. (2016) have also faced time constraints. As this programme is voluntary, undergraduates had to take time off their already packed curriculum to participate, and the selection of volunteers was not a stringent process. Additionally, as participants were contributing to clinical care, documentation of visits is required. Multiple studies showed that physicians deemed documentation and administrative work burdensome and excessive time spent on these may be associated with physicians’ burnout (Patel et al., 2018; Wright & Katz, 2018).

In addressing these barriers, incorporating academic credits for participation, a more stringent selection of participants, streamlining administrative work and prudent choice of patients may be considered. These measures are already being implemented by the programme organisers to improve the programme.

G. Strengths and Limitations

The strength of this study lies in the use of both quantitative and qualitative data grounded on an established framework by Kirkpatrick (1959) for the evaluation of a novel experiential IPE programme. The limitations of our study include it being single-institution and that the participants are volunteers which thus form a self-selected group. Hence, the results may not be generalisable. There was also no control arm for the intervention. In addition, there was a large variation in baseline RIPLS score seen in the programme, which can be potentially improved with a more robust study design that controls for baseline differences. Lastly, the use of only a survey for data collection may limit the depth of qualitative data obtained. Further studies could include qualitative interviews.

V. CONCLUSION

We validated the RIPLS in Singapore and demonstrated the feasibility of an interprofessional student-initiated home visit programme. While there was no significant change in RIPLS scores, the qualitative feedback suggests that there are participant-perceived benefits for IPE after undergoing this programme, even with the perceived barriers to IPE. Future programmes can work on addressing these barriers to IPE.

Notes on Contributors

Gloria Yao Chi Leung contributed to the conception and design of the work, the acquisition, analysis, and interpretation of data for the work, drafting and revising the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Kennedy Yao Yi Ng contributed to the conception and design of the work, analysis and interpretation of data for the work, drafting and revising the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Yow Ka Shing contributed to the conception and design of the work, the acquisition and interpretation of data for the work, drafting and revising the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Nerice Heng Wen Ngiam contributed to the conception and design of the work, the acquisition of data for the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Dillon Guo Dong Yeo contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Angeline Jie-Yin Tey contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Melanie Si Rui Lim contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Aaron Kai Wen Tang contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Chew Bi Hui contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Celine Yi Xin Tham contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Yeo Jia Qi contributed to the conception and design of the work, drafting the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Lau Tang Ching contributed to the conception and design of the work, critical revision of the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Wong Sweet Fun contributed to the conception and design of the work, critical revision of the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Gerald Choon Huat Koh contributed to the conception and design of the work, interpretation of the data for the work, critical revision of the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Wong Chek Hooi contributed to the conception and design of the work, interpretation of the data for the work, critical revision of the manuscript, approves of the publishing of the manuscript, and agrees to be accountable for the accuracy of the work.

Ethical Approval

Ethical approval was obtained from the NUS institutional review board (B-15-272). Study participation was entirely voluntary and anonymous. Informed consent was taken from participants before data collection commenced, and they were allowed to withdraw from the research at any point in time. No incentives were provided to study participants.

Data Availability

According to institutional policy, research dataset is available on reasonable request to the corresponding author.

Acknowledgement

The authors would like to thank the Tri-Generational HomeCare Organising Committee from 2014 to 2018 for supporting the study. They would like to extend their thanks to the National University of Singapore, Yong Loo Lin School of Medicine, Dean’s Office; the North West Community Development Council; Khoo Teck Puat Hospital, Singapore; Geriatric Education and Research Institute, Singapore. Finally, they would like to thank the volunteers for their generosity and the patients for their hospitality.

Funding

National University of Singapore, Yong Loo Lin School of Medicine, Dean’s Office; the North West Community Development Council; Khoo Teck Puat Hospital, Singapore provided funding support for the purchase of medical consumables, refreshments and logistics for the program.

Declaration of Interest

There are no conflicts of interest.

References

Alexandraki, I., Hernandez, C. A., Torre, D. M., & Chretien, K. C. (2017). Interprofessional education in the internal medicine clerkship post-LCME standard issuance: Results of a national survey. Journal of General Internal Medicine, 32(8), 871–876.https://doi.org/10.1007/s11606-017-4004-3

Aziz, Z., Teck, L. C., & Yen, P. Y. (2011). The attitudes of medical, nursing and pharmacy students to inter-professional learning. Procedia – Social and Behavioural Sciences, 29, 639–645. https://doi.org/10.1016/j.sbspro.2011.11.287  

Barr, H., Koppel, I., Reeves, S., Hammick, M., & Freeth, D. (2005). Effective interprofessional education: argument, assumption, and evidence (1st edition). Wiley-Blackwell.

Berger-Estilita, J., Fuchs, A., Hahn, M., Chiang, H., & Greif, R. (2020). Attitudes towards interprofessional education in the medical curriculum: A systematic review of the literature. BMC Medical Education, 20(1), Article 254. https://doi.org/10.1186/s12909-020-02176-4

Blythe, J., & Spiring, R. (2020). The virtual home visit. Education for Primary Care, 31(4), 244–246. https://doi.org/10.1080/14739879.2020.1772119

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Briggs, M. C. E., & McElhaney, J. E. (2015). Frailty and interprofessional collaboration. Interdisciplinary Topics in Gerontology and Geriatrics, 41, 121–136. https://doi.org/10.1159/000381204

Canadian Interprofessional Health Collaborative. (2010). A national interprofessional competency framework. https://phabc.org/wp-content/uploads/2015/07/CIHC-National-Interprofessional-Competency-Framework.pdf

Chua, A. Z., Lo, D. Y., Ho, W. H., Koh, Y. Q., Lim, D. S., Tam, J. K., Liaw, S. Y., & Koh, G. C. (2015). The effectiveness of a shared conference experience in improving undergraduate medical and nursing students’ attitudes towards inter-professional education in an Asian country: A before and after study. BMC Medical Education, 15, Article 233. https://doi.org/10.1186/s12909-015-0509-9

Conti, G., Bowers, C., O’Connell, M. B., Bruer, S., Bugdalski-Stutrud, C., Smith, G., Bickes, J., & Mendez, J. (2016). Examining the effects of an experiential interprofessional education activity with older adults. Journal of Interprofessional Care, 30(2), 184–190. https://doi.org/10.3109/13561820.2015.1092428

de Oliveira, V. F., Bittencourt, M. F., Navarro Pinto, Í. F., Lucchetti, A. L. G., da Silva Ezequiel, O., & Lucchetti, G. (2018). Comparison of the readiness for interprofessional learning and the rate of contact among students from nine different healthcare courses. Nurse Education Today, 63, 64–68. https://doi.org/10.1016/j.nedt.2018.01.013

Fahs, D. B., Honan, L., Gonzalez-Colaso, R., & Colson, E. R. (2017). Interprofessional education development: Not for the faint of heart. Advances in Medical Education and Practice, 8, 329–336. https://doi.org/10.2147/AMEP.S133426

Ganotice, F. A., & Chan, L. K. (2018). Construct validation of the English version of Readiness for Interprofessional Learning Scale (RIPLS): Are Chinese undergraduate students ready for ‘shared learning’? Journal of Interprofessional Care, 32(1), 69–74. https://doi.org/10.1080/13561820.2017.1359508

Grice, G. R., Thomason, A. R., Meny, L. M., Pinelli, N. R., Martello, J. L., & Zorek, J. A. (2018). Intentional interprofessional experiential education. American Journal of Pharmaceutical Education, 82(3), Article 6502. https://doi.org/10.5688/ajpe6502

Hansson, A., Foldevi, M., & Mattsson, B. (2010). Medical students’ attitudes toward collaboration between doctors and nurses – A comparison between two Swedish universities. Journal of Interprofessional Care, 24(3), 242–250. https://doi.org/10.3109/13561820903163439

Hoffman, S. J., Rosenfield, D., Gilbert, J. H. V., & Oandasan, I. F. (2008). Student leadership in interprofessional education: Benefits, challenges and implications for educators, researchers and policymakers. Medical Education, 42(7), 654–661. https://doi.org/10.1111/j.1365-2923.2008.03042.x 

Judge, M. P., Polifroni, E. C., & Zhu, S. (2015). Influence of student attributes on readiness for interprofessional learning across multiple healthcare disciplines: Identifying factors to inform educational development. International Journal of Nursing Sciences, 2(3), 248–252. https://doi.org/10.1016/j.ijnss.2015.07.007

Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 42(8), 846–854. https://doi.org/10.1080/0142159X.2020.1755030

Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21–26.

Knowles, M. S. (1984). Andragogy in Action: Applying Modern Principles of Adult Learning (1st edition). Jossey-Bass.

Lauffs, M., Ponzer, S., Saboonchi, F., Lonka, K., Hylin, U., & Mattiasson, A.-C. (2008). Cross-cultural adaptation of the Swedish version of Readiness for Interprofessional Learning Scale (RIPLS). Medical Education, 42(4), 405–411. https://doi.org/10.1111/j.1365-2923.2008.03017.x

Lestari, E., Stalmeijer, R. E., Widyandana, D., & Scherpbier, A. (2016). Understanding students’ readiness for interprofessional learning in an Asian context: A mixed-methods study. BMC Medical Education, 16, Article 179. https://doi.org/10.1186/s12909-016-0704-3 

Li, Z., Sun, Y., & Zhang, Y. (2018). Adaptation and reliability of the Readiness for Inter Professional Learning Scale (RIPLS) in the Chinese health care students setting. BMC Medical Education, 18(1), Article 309. https://doi.org/10.1186/s12909-018-1423-8

Lie, D. A., Forest, C. P., Walsh, A., Banzali, Y., & Lohenry, K. (2016). What and how do students learn in an interprofessional student-run clinic? An educational framework for team-based care. Medical Education Online, 21(1), Article 31900. https://doi.org/10.3402/meo.v21.31900 

Mahler, C., Berger, S., & Reeves, S. (2015). The Readiness for Interprofessional Learning Scale (RIPLS): A problematic evaluative scale for the interprofessional field. Journal of Interprofessional Care, 29(4), 289–291. https://doi.org/10.3109/13561820.2015.1059652

Makino, T., Shinozaki, H., Hayashi, K., Lee, B., Matsui, H., Kururi, N., Kazama, H., Ogawara, H., Tozato, F., Iwasaki, K., Asakawa, Y., Abe, Y., Uchida, Y., Kanaizumi, S., Sakou, K., & Watanabe, H. (2013). Attitudes toward interprofessional healthcare teams: A comparison between undergraduate students and alumni. Journal of Interprofessional Care, 27(3), 261–268. https://doi.org/10.3109/13561820.2012.751901

McFadyen, A. K., Webster, V., Strachan, K., Figgins, E., Brown, H., & McKechnie, J. (2005). The Readiness for Interprofessional Learning Scale: A possible more stable sub-scale model for the original version of RIPLS. Journal of Interprofessional Care, 19(6), 595–603. https://doi.org/10.1080/13561820500430157

McManus, K., Shannon, K., Rhodes, D. L., Edgar, J. D., & Cox, C. (2017). An interprofessional education program’s impact on attitudes toward and desire to work with older adults. Education for Health, 30(2), 172–175. https://doi.org/10.4103/efh.EfH_2_15

Ng, K. Y. Y., Leung, G. Y. C., Tey, A. J.-Y., Chaung, J. Q., Lee, S. M., Soundararajan, A., Yow, K. S., Ngiam, N. H. W., Lau, T. C., Wong, S. F., Wong, C. H., & Koh, G. C.-H. (2020a). Bridging the intergenerational gap: The outcomes of a student-initiated, longitudinal, inter-professional, inter-generational home visit program. BMC Medical Education, 20(1), Article 148. https://doi.org/10.1186/s12909-020-02064-x

Ng, K. Y. Y., Leung, G. Y. C., Yow, K. S., Ngiam, N. H. W., Yeo, D. G. D., Tey, A. J.-Y., Lim, M. S. R., Tang, A. K. W., Chew, B. H., Tham, C. Y. X., Yeo, J. Q., Lau, T. C., Wong, S. F., Wong, C. H., & Koh, G. C.-H. (2020b). Impact of an interprofessional, longitudinal, undergraduate student-initiated home visit program towards interprofessional education. Research Square. https://doi.org/10.21203/rs.3.rs-23744/v1

Paradis, E., & Whitehead, C. R. (2015). Louder than words: Power and conflict in interprofessional education articles, 1954-2013. Medical Education, 49(4), 399–407. https://doi.org/10.1111/medu.12668

Parsell, G., & Bligh, J. (1999). The development of a questionnaire to assess the readiness of health care students for interprofessional learning (RIPLS). Medical Education, 33(2), 95-100. https://doi.org/10.1046/j.1365-2923.1999.00298.x

Patel, R. S., Bachu, R., Adikey, A., Malik, M., & Shah, M. (2018). Factors related to physician burnout and its consequences: A review. Behavioural Sciences, 8(11), Article 98. https://doi.org/10.3390/bs8110098  

Schmitz, C. C., & Brandt, B. F. (2015). The Readiness for Interprofessional Learning Scale: To RIPLS or not to RIPLS? That is only part of the question. Journal of Interprofessional Care, 29(6), 525–526. https://doi.org/10.3109/13561820.2015.1108719

Shapiro, S. S., & Wilk, M. B. (1965). an analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591–611. https://doi.org/10.2307/2333709

Sunguya, B. F., Hinthong, W., Jimba, M., & Yasuoka, J. (2014). Interprofessional education for whom? – Challenges and lessons learned from its implementation in developed countries and their application to developing countries: A systematic review. PloS One, 9(5), Article e96724. https://doi.org/10.1371/journal.pone.0096724 

Tamura, Y., Seki, K., Usami, M., Taku, S., Bontje, P., Ando, H., Taru, C., & Ishikawa, Y. (2012). Cultural adaptation and validating a Japanese version of the readiness for interprofessional learning scale (RIPLS). Journal of Interprofessional Care, 26(1), 56–63. https://doi.org/10.3109/13561820.2011.595848

Toth-Pal, E., Fridén, C., Asenjo, S. T., & Olsson, C. B. (2020). Home visits as an interprofessional learning activity for students in primary healthcare. Primary Health Care Research & Development, 21, Article e59. https://doi.org/10.1017/S1463423620000572

Tremblay, D., Drouin, D., Lang, A., Roberge, D., Ritchie, J., & Plante, A. (2010). Interprofessional collaborative practice within cancer teams: Translating evidence into action. A mixed methods study protocol. Implementation Science, 5, Article 53. https://doi.org/10.1186/1748-5908-5-53

Tyastuti, D., Onishi, H., Ekayanti, F., & Kitamura, K. (2014). Psychometric item analysis and validation of the Indonesian version of the Readiness for Interprofessional Learning Scale (RIPLS). Journal of Interprofessional Care, 28(5), 426–432. https://doi.org/10.3109/13561820.2014.907778

Vaughn, L. M., Cross, B., Bossaer, L., Flores, E. K., Moore, J., & Click, I. (2014). Analysis of an interprofessional home visit assignment: Student perceptions of team-based care, home visits, and medication-related problems. Family Medicine, 46(7), 522–526 

Wakely, L., Brown, L., & Burrows, J. (2013). Evaluating interprofessional learning modules: Health students’ attitudes to interprofessional practice. Journal of Interprofessional Care, 27(5), 424–425. https://doi.org/10.3109/13561820.2013.784730

West, C., Graham, L., Palmer, R. T., Miller, M. F., Thayer, E. K., Stuber, M. L., Awdishu, L., Umoren, R. A., Wamsley, M. A., Nelson, E. A., Joo, P. A., Tysinger, J. W., George, P., & Carney, P. A. (2016). Implementation of interprofessional education (IPE) in 16 U.S. medical schools: Common practices, barriers and facilitators. Journal of Interprofessional Education & Practice, 4, 41–49. https://doi.org/10.1016/j.xjep.2016.05.002

Wilhelmsson, M., Ponzer, S., Dahlgren, L. O., Timpka, T., & Faresjö, T. (2011). Are female students in general and nursing students more ready for teamwork and interprofessional collaboration in healthcare? BMC Medical Education, 11, Article 15. https://doi.org/10.1186/1472-6920-11-15

Wright, A. A., & Katz, I. T. (2018). Beyond burnout — Redesigning care to restore meaning and sanity for physicians. The New England Journal of Medicine, 378(4), 309–311. https://doi.org/10.1056/NEJMp1716845

Zaudke, J. K., Paolo, A., Kleoppel, J., Phillips, C., & Shrader, S. (2016). The impact of an interprofessional practice experience on readiness for interprofessional learning. Family Medicine, 48(5), 371–376.

*Chek Hooi WONG
90 Yishun Central,
Khoo Teck Puat Hospital,
Singapore 768828
9 Lower Kent Ridge Rd, Level 10,
+65 6807 8001
Email: wong.chek.hooi@ktph.com.sg

Submitted: 5 January 2022
Accepted: 24 February 2022
Published online: 5 July, TAPS 2022, 7(3), 33-41
https://doi.org/10.29060/TAPS.2022-7-3/OA2739

Javier Zheng Huan Thng¹, Fion Yun Yee Tan¹, Marion Margaret Hui Yong Aw^1,2 & Shijia Hu³

¹Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Paediatrics, Khoo Teck Puat-National University Children’s Medical Institute, National University Health System, Singapore; ³Faculty of Dentistry, National University of Singapore, Singapore

Abstract

Introduction: In paediatric practice, healthcare professionals are required to connect with the child and interact at his/her level. However, it can be very difficult for medical students to put themselves in the shoes of the young child, to empathize and understand how a child actually feels while being treated. The Teddy Bear Hospital (TBH) can serve as a platform for medical students to learn how to communicate and empathise with children. Additionally, virtual reality (VR) can be used to portray a child’s viewpoint. This study aims to assess how TBH and VR can improve learning outcomes for medical students.

Methods: A cohort study was conducted on 20 first-year medical students taking part in TBH sessions. The medical students did a Pre-, Post- and 1-year Post-intervention Jefferson Scale of Empathy to assess their empathy levels. They also completed a 1-year Post-intervention quantitative and qualitative survey on their experience.

Results: There was a significant increase in Jefferson score compared to Pre-intervention (116.95 ± 8.19) for both Post-intervention (121.65 ± 11.03) and 1-year Post-intervention (123.31 ± 8.86). More than 80% believed that participating in TBH improved their confidence and ability to interact with children, while 50% felt that VR scenarios helped prepare them for the TBH. Thematic analysis of qualitative responses described (1) Personal development, (2) Insights into interacting with children, and (3) Structure and curriculum.

Conclusion: TBH improved empathy and communication with children among pre-clinical medical students and the use of VR can be used to augment sessions.

Keywords:           Education, Medical Student, Simulation Training, Teddy Bear Hospital, Virtual Reality

• The Teddy Bear Hospital sessions, consisting of physical simulation of medical scenarios with children, increased empathy of medical students.
• The use of virtual reality scenarios to portray the viewpoint of a child can augment the teaching of the Teddy Bear Hospital.
• The Teddy Bear Hospital and virtual reality sessions improved the comfort, increased confidence, and self-perceived ability of interacting with children in medical students.

I. INTRODUCTION

In paediatric practice, healthcare professionals are required to connect with the child and interact at his/her level. This is especially important, when the child is encountering new and unfamiliar situations or when they are unwell and face potentially challenging and painful procedures (Mead & Bower, 2002). However, it can be very difficult for new practitioners to put themselves in the shoes of the young child, to empathise and understand how a child actually feels while being treated (Dwamena et al., 2012). This is particularly true for medical students, who not only have to grapple with the unfamiliar medical aspects of pediatric disease, but may also need to manage a frightened and uncooperative child at the same time (MacDonald-Wicks & Levett-Jones, 2012). Furthermore, having limited interaction with children means that most medical students have a difficult time empathising with them.

Empathy is an important element in a physician-patient relationship, as it has shown to improve communication and therapeutic goals (Mercer & Reynolds, 2002). However, several studies have shown significant decrease in empathy over the course of medical school (Neumann et al., 2011). Although there have been numerous methods and approaches developed to enhance empathy in medical students (Batt-Rawden et al., 2013), including interventions based around the patient narrative (e.g. creative writing, blogging, drama, poetry, fiction, and film), problem-based learning, interpersonal skills training, patient interviews, and experiential learning (simulation of patient experience). Among these interventions, experiential (immersive) learning experiences have shown promising results (Halton & Cartwright, 2018). These scenarios can be conducted either physically or virtually. It can involve the learner completing a simulated task or experience a scenario from the point of view of someone else so as to put themselves in someone else’s shoes.

The advent and advancement of virtual reality (VR) media provide the opportunity for the portrayal of different viewpoints (Lok et al., 2006). A recent study found that empathy in medical students was improved with VR portraying the viewpoint of an older patient with conditions such as macular degeneration and hearing loss (Dyer et al., 2018). However, there has not been any research done on the use of VR to simulate the point of view of a child in a medical setting. The power of immersive media can be harnessed to simulate interactions between a child and healthcare professionals, exhibiting both positive and negative examples. More importantly, it can be used to introduce an inexperienced practitioner to the viewpoint of a child patient. This will help foster empathy and drive home the effectiveness of behaviour management skills (Stewart et al., 2013).

The Teddy Bear Hospital (TBH) is an international initiative, carried out by students from Medical Faculties, aimed at reducing children’s anxiety around medical environments, procedures and professionals (Bloch & Toker, 2008; Santen & Feldman, 1994; Siegel et al., 2018). TBH simulates the different medical environments and clinical situations which children may encounter in a friendly manner using they own toys (such as teddy bears). This simulation takes the form of multi-station role-plays, which could include a check-up by the doctor, watching a doctor applying a cast for a limb fracture or performing a procedure (e.g., taking a blood sample), or receiving an injection (all performed on teddy bears, who are the patients). In addition to reducing anxiety in young children, the TBH has also been used to teach medical students communication with children in a medical setting (Nheu et al., 2018; Ong et al., 2018). Although the qualitative feedback from medical students have been generally positive (Nheu et al., 2018; Ong et al., 2018), there have not been objective measures of the effect of the TBH on the empathy of medical students.

Both VR and TBH are immersive interventions that can potentially increase the empathy of medical student, they can also improve the learning experience by allowing students to experience effective techniques to interact with children and practice those techniques. However, each intervention has inherent disadvantages. VR usually follows a scripted scenario that does not allow students to practice and interact in real-time. While TBH is time and resource intensive, and cannot be conducted is situations such as a pandemic. Therefore, evaluating the effects of a combination of the 2 techniques can inform on the individual effectiveness as well as in combination.

This study aims to determine the effect of virtual and physical simulation on the empathy and learning experience of medical students. This will be done through a combination of VR instruction and practical simulation in a TBH experience, in which medical students are exposed to and educated on the positive interactions with a child patient in various medical scenarios.

II. METHODS

This is a cohort study of first year medical students who took part in TBH sessions in January 2020 and were followed up in January 2021. Subjects were recruited from volunteers who signed up to participate in TBH sessions. Subjects were reassured that their responses are strictly confidential and will have no impact on their grades or teaching received.  Medical students who have participated in previous TBH sessions were excluded. Written informed consent was taken prior to participation. This study was approved by the Institutional Review Board (NUS-IRB Reference Number: S-19-151) and the study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments.

As this was a pilot study, no sample size calculation was conducted prior to study initiation.

A. Survey Content and Timing

The Jefferson Scale of Empathy (Sidney Kimmel Medical College, Thomas Jefferson University, PA, USA) was used to assess the level of empathy (Hojat et al., 2018). It is a 20-item scale that was developed specifically to measure empathy in health professions education, including medical students. The S-version of the survey, for medical students, was administered in this study. The survey forms were purchased from Thomas Jefferson University, Center for Research in Medical Education and Health Care. The Jefferson scores were calculated out of a maximum of 140. The survey was conducted in its original English version as the language medium for education in Singapore is English and all participants are proficient.

1) Pre-intervention: Prior to the TBH session, the subjects completed the Jefferson Scale of Empathy to assess baseline empathy scores. The subjects then underwent a small group teaching session of one hour including viewing of two VR scenarios, conducted by the same instructors (JT & FT). The small group teaching sessions covered topics pertaining to doctor-patient relationship, and the developmental and psycho-affective challenges of interacting with children. The two VR scenarios depicted a child’s point of view in the setting of getting an injection (mooc vid, Scenario 1, 2019) and visiting the dentist (mooc vid, Scenario 2, 2019). The scenarios are interactive and the point of view can be manipulated by the viewer. After watching the VR scenarios, discussions where subjects can critique the healthcare providers’ actions and share their learning points were carried out.  During this session, the subjects were also briefed about the different scenarios in the TBH session, including the content to be covered.

2) TBH session: The TBH session consisted of 5 scenarios namely: (1) Orthopedic Specialist: Meet Mr. Bones, (2) Respiratory Therapist: Help Teddy Breathe, (3) Family Medicine: Help Teddy Stay Healthy, (4) Personal Hygiene: Help Teddy Stay Clean, (5) Operating Theatre: Teddy has a Painful Tummy. The injection process and dental procedures were incorporated into scenarios 3 and 4 respectively. The medical students were divided into teams of 2 to 3, each interacting with a group of 5 to 8 children of the same age. Each team rotated through the different scenarios for 10 minutes. At each scenario, the medical students explained the task to the children and conducted hands-on simulation.

3) Post-intervention: After their first TBH session, the subjects completed a second Jefferson Scale of Empathy.

4) 1-year Post-intervention: One year after the TBH session, students completed a third Jefferson Scale of Empathy, along with a self-administered survey regarding the long-term impact of TBH sessions and the effectiveness of the VR scenarios.

To assess the long-term impact of TBH sessions and the effectiveness of the VR scenarios, a 1-year Post-intervention survey was administered in December 2020. The survey was adapted from a previous study (Ong et al., 2018) and piloted for understanding and readability.

The 1-year Post-intervention impact of TBH sessions was assessed using five questions on a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) regarding the effect of TBH sessions on improving the medical student’s ability, confidence and comfort level in communicating with children, and ability to empathize with children’s fear in the healthcare environment. Similarly, the effectiveness of the VR scenarios was assessed using three questions on a 5-point Likert scale, regarding the effect of the VR scenarios on preparation for TBH session, as well as improving the comfort level and confidence in engaging children. The remaining two questions obtained qualitative feedback regarding any difficulties faced as well as takeaways obtained during the TBH session.

B. Data Collection

The subjects completed the TBH sessions in small groups of up to eight volunteers, with up to 30 children, aged between 4 to 8 years old, in each session. It was explained to the children that they were participating in an activity to teach them about what happens during visits to a doctor. These sessions were conducted at the participating pre-schools with medical instruments and teddy bears as simulated patients.

Due to the COVID-19 pandemic situation and restrictions, the survey was collected via an online survey, instead of a planned Focused Group Discussion.

C. Data Analysis

Normality of data was checked using the Shapiro-Wilk test, data analysis of the Jefferson Scale of Empathy score was done via Paired samples correlation, Cohen’s d score and Paired Samples T test using Statistical Package for Social Sciences (SPSS) software (IBM SPSS Statistics 26, Armonk, NY, USA). Descriptive statistics were presented for the survey on learner experience.

Qualitative analysis was done on the medical students’ open-ended responses regarding the long-term impact of TBH sessions and the effectiveness of the VR scenarios. Using thematic content analysis and a qualitative descriptive framework (Creswell & Poth, 2019), emerging themes and illustrative quotes for each theme were extracted. The codes and themes were first done independently by two of the authors (JT & FT), any discrepancies were discussed and reconciled. After which, 2 other authors (SH & MA) reviewed and verified the finalised codes and themes, which were mapped for interpretation. Descriptive summaries and illustrative quotes were used to describe each theme.

III. RESULTS

A. Study Demographics

In total, 20 first time participants were recruited and consent was taken. All 20 were first year medical students; 9 males and 11 females. All participants (n=20) completed the Pre-intervention and Post-intervention survey while 16 completed the 1-year Post-intervention survey, with 4 participants declining to participate in the follow-up. The data that support the findings of this study are openly available in Figshare at https://doi.org/10.6084/m9.figshare.17973662 (Hu, 2022).

B. Jefferson Scale of Empathy

There was a significant (p=0.026) increase in the Jefferson score between the Pre-intervention (116.95 ± 8.19) and Post-intervention (121.65 ± 11.03). Similarly, there was a significant (p=0.002) increase from the Pre-intervention to 1-year Post-intervention (123.31 ± 8.86). (Figure 1) However, there was no difference between the Post-intervention and 1-year Post-intervention scores.

Cohen’s d score was used to determine the effect size of the intervention on the mean difference between the Pre- and Post-intervention score (0.48), and the Pre- and 1-year Post-intervention score (0.72). This corresponds to a medium effect of the intervention on the mean differences.

Figure 1. Jefferson Scale of Empathy Score before, immediately after and 1 year after TBH. Error bars represent the standard deviation. Asterisk (*) indicate significant differences between the groups, p<0.05.

C. Quantitative Survey

Quantitative responses from the survey on learner experience were categorised into positive responses (agree, strongly agree) and negative/neutral responses (neutral, disagree, strongly disagree). The total number of responses collected was 16 at 1-year Post-intervention (Table 1).

Table 1. Subject’s perception towards their TBH experience

The subjects were generally confident (75%) and enjoyed interacting with young children (94%). More than 80% believed that participating in TBH improved their confidence and ability to interact with children, and 75% felt that participating in TBH made them more comfortable communicating with children. 63% felt that TBH helped improve their ability to empathize with children’s fears of the healthcare setting. Overall, 75% of subjects did not face difficulties during the TBH sessions. In terms of the VR scenarios, around 50% felt that it was effective in preparing for the actual TBH session, with 56% feeling more comfortable and 44% feeling more confident interacting with children.

D. Qualitative Survey

Three major themes were generated from the analysis of the open-ended responses from 16 of the 20 subjects: (1) Personal development, (2) Insights into interacting with children and (3) Structure and curriculum of TBH. These themes were further categorised into sub-themes as shown in Table 2.

Table 2. Major themes, sub-themes and transcript highlights of qualitative survey

1) Personal development

TBH provided an opportunity for medical students in this study to interact with children in a safe and stress-free environment. These students would otherwise have limited experience communicating with children in that age-group, until actual clinical interactions with sick children.

“I was given the opportunity to interact with the children (by) myself without help…it was a good learning experience.” (P15)

The experience gave these medical students more confidence and reduced their anxiety about interacting with children, as they realized that it was not as difficult as they thought to communicate with children.

“Communicating with children is not as scary as I thought as long as I build a good rapport with the children.” (P01)

Medical students also learnt useful pointers on how to interact and communicate with children, which alleviated their worries and improved their confidence. These pointers include using simpler words, hand actions, speaking at eye level and being more expressive.

“(TBH) allowed me to practice how I interact with children (by) using easier words (and) hand actions” (P01)

“I was taught how to better interact with children in terms of speaking at eye level” (P08)

2) Insights into interacting with children

TBH was a unique platform to understand children’s learning needs and empathize with their views towards healthcare. This was achieved via interactions with the children.

“By bonding with the children and hearing their opinions on the healthcare setting, it gives (me) a sense of connection between the children and myself which allows me to empathize with them better.” (P12)

These interactions allowed medical students to learn how to adapt their teaching styles to suit the different needs of each individual child.

“I learnt to interact with different groups of children with different learning abilities and TBH taught me how to cater to the different needs and abilities of the kids instead of treating them as a homogenous group.” (P07)

These interactions also taught medical students that the children have different levels of fear towards healthcare. While some children were apprehensive, others were unafraid of the TBH sessions.

“The children shared their fears about healthcare providers and I think the bulk of this fear comes from not being able to see the benefit of receiving these ‘painful’ treatments.” (P09)

“The children seemed to be really excited during the TBH sessions, I don’t recall that the children expressed any fear about their experiences” (P16)

In addition to understanding the children’s fears towards healthcare, medical students learnt how to manage these fears by acknowledging them and putting themselves in the children’s shoes.

“We should never downscale a child’s fear towards healthcare. There is a need to try to understand where they are coming from, (and) to comfort and encourage them to the best of our ability.” (P13)

3) Structure and curriculum

To understand the impact of the newly added VR scenarios, feedback regarding its strengths and limitations were collated. One strength is that the students experienced various case studies through the VR scenarios. This allowed them to be better prepared for the TBH sessions and become more aware of the children’s emotions during their interaction.

“I felt that the VR video gave us a lot to think about… Through the discussion and critiquing the VR doctor’s actions, I was more conscious on how the children may feel in a hospital setting.” (P07)

Another strength is that during the training, medical students were also asked how they would respond to the hypothetical scenarios. By putting them in the hot seat, they were able to exchange ideas and learn from one another.

“The volunteer trainers asked us questions during the volunteer training, when I answered correctly it gave me more confidence and when I listen to other people’s answers, I feel more equipped to handle different situations too” (P15)

The VR scenarios were limited in terms of application to the actual TBH sessions. The scenarios only showed how to engage an individual child, which was different from actual TBH sessions, in which medical students had to teach a group of children.

“Although it did give me a sense of how it is like to interact with children, the actual TBH session was a bit different from the VR (scenarios)” (P05)

Feedback was also collated regarding the strengths and limitations of the TBH sessions.

One strength was the established curriculum so participants did not have to worry about the content but can instead focus on honing their communication skills. The fact that this was a student-run program, and that medical students had their peers as fellow participants, put them at ease and made it a more conducive environment.

“(TBH) placed me in a friendly and relaxed environment centred on the children thus allowing me to focus in developing on interactions with the children” (P02)

A limitation of the TBH sessions was that it was challenging for participants to balance teaching the children and empathising with them.

“I think it’s … difficult to see from the children’s perspective during the session… We are usually more focused on imparting rather than listening because of the inability to elicit responses from all of them.” (P13)

Another limitation of the TBH sessions was that participants are usually able to attend only one session. This may have held them back from being fully adept in their interactions with children.

“As I was only able to attend 1 session, I was unable to fully practice and become more confident in these skills.” (P07)

Finally, there were disagreements about whether the ratio of medical students to children was suitable. While some felt that it was appropriate, others had trouble managing the children.

“The ratio (of) facilitators to children were just nice (and) not overwhelming for either parties” (P15)

“It was just too chaotic and hard to manage so many of them especially since they’re so bubbly and curious” (P16)

IV. DISCUSSION

This was the first study to examine objectively and subjectively, the effect of TBH and VR training on the empathy levels of medical students. There was a statistically significant improvement in the medical student’s level of empathy immediately after the TBH session, which persisted 1 year after the TBH session. The outcomes of this study revealed that the medical students who participated in TBH augmented by VR training assisted them to better empathize with the children they worked with.

Compared with the previous work by Ong et al in the same institution, involving only traditional lecture preparation for TBH (Ong et al., 2018), it appears that the addition of VR training showed an increase in medical students’ perceived improvement in both interacting and teaching children. In that study conducted without the use of VR scenarios, 53% reported improvement in interacting with children and 39% reported improvement in teaching children, compared to the present study of 88% and 75% respectively. Moreover, VR training alone achieved improvements of 44% and 50% which is similar to TBH session alone. A recent study found that VR simulation alone improved the empathy of dental students when managing children (Hu & Lai, 2022), this is encouraging since the VR intervention was not as time and resource intensive as the TBH. Although the cohorts of medical students were different, each cohort is from the same year of training (i.e., first year medical students) and the TBH teaching was done with a standardised protocol and thus relatively similar across the cohorts. It appears that a combination of the 2 methods of training provided a greater magnitude to the student’s perceived improvement in interacting and teaching children. This suggests that although the VR training cannot replace the real TBH interaction with children, it can be used to augment the learning experience of students in communicating with children. Moreover, it can be deployed on a much larger scale to the entire cohort, instead of smaller groups like the labour intensive TBH teaching.

According to the qualitative analysis, the TBH is a good platform for medical students to practice and develop their communication skills with children. Through TBH, they pick up techniques on how to better communicate with children both from the trainers and their peers. Similar to previous studies, TBH has also been described to be a useful avenue to gain insights into children’s perspectives of the healthcare setting and therefore trains them to be better attuned to the needs and feelings of children (Nheu et al., 2018; Ong et al., 2018). Nonetheless, the medical students suggested some areas of improvements. For example, it was pointed out that the VR scenarios did not depict the actual TBH session, as such, it did not help in teaching children during TBH. However, the goal of the TBH session is to increase medical student’s empathy through teaching rather than training them to be educators. Therefore, the VR scenarios can still bring the desired benefits in terms of the medical student’s ability to empathize and communicate with children. Additionally, some felt that they are unable to fulfil both a teaching and learning role simultaneously and were overwhelmed. This may hinder them from achieving the desired outcome of increasing their empathy. A potential solution would be to expand the existing course material to further help those who struggle with engaging the children or having fewer children paired to each medical student teacher.

There were some limitations to this study. As there was no control group to compare against, it was difficult to determine if the improvements seen from Pre-intervention to Post-intervention were due to the VR training and participation in the TBH session alone. Future studies could consider including a control group in addition to the intervention group. The observed maintenance of the level of empathy from Post-intervention to 1-year Post-intervention could have been confounded by factors such as varying levels of clinical exposure. Additionally, the quantitative survey on learner experience was done 1 year after the TBH session and may be at risk for recall bias, future studies should consider collecting data at multiple time points to evaluate any difference between immediate and long-term effects of the interventions. However, all respondents provided feedback that were very detailed and informative, suggesting that they were able to recall the experience well. Moreover, due to the COVID-19 pandemic situation and restrictions, the survey was collected via an online survey, instead of the planned Focused Group Discussion, resulting in the inability to ask follow-up or clarifying questions. Lastly, the study was conducted on a small group of volunteers who may have biases due to interest in the discipline. However, a significant improvement was still noted in this small pilot study. Expansion of the program into the general curriculum to include a more diverse group of students will be needed to ascertain the effect of TBH on medical students in general.

V. CONCLUSION

In conclusion, the TBH experience for medical students was effective in increasing their levels of empathy and confidence in interacting and teaching children as it provides an opportunity for medical students to interact with children and understand their views of healthcare. The use of VR can augment the TBH experience or be used in situations where the student is unable to attend TBH sessions in person.

Notes on Contributors

JT and FT are considered co-first authors. JT participated in data collection, conducted the data analysis and interpretation, led the writing, and revised the manuscript for important intellectual content. FT participated in data collection, conducted the data analysis and interpretation, led the writing, and revised the manuscript for important intellectual content. MA conceived the idea and revised the manuscript for important intellectual content. SH conceived the idea, conducted the data analysis and interpretation, led the writing, and revised the manuscript for important intellectual content.

Ethical Approval

This study was approved by the Institutional Review Board (NUS-IRB Reference Number: S-19-151) and the study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments.

Data Availability

The data that support the findings of this study are openly available in Figshare repository, https://doi.org/10.6084/m9.figshare.17973662.

Acknowledgement

The authors would like to thank Dr Lee Shuh Shing and Ms Lim Yih Lin for their help with the statistical analysis and Manzalab for the help in creating the virtual reality scenarios.

Funding

This work was supported by the USPC-NUS Joint Innovative Projects in Higher Education grant (USPC-NUS 2018 SoM).

Declaration of Interest

The authors declare that they have no competing interests.

References

Batt-Rawden, S. A., Chisolm, M. S., Anton, B., & Flickinger, T. E. (2013). Teaching empathy to medical students: An updated, systematic review. Academic Medicine, 88(8), 1171-1177. https://doi.org/10.1097/ACM.0b013e318299f3e3

Bloch, Y. H., & Toker, A. (2008). Doctor, is my teddy bear okay? The “Teddy Bear Hospital” as a method to reduce children’s fear of hospitalization. Israel Medical Association Journal, 10(8-9), 597-599.

Creswell, J. W., & Poth, C. N. (2019). Qualitative inquiry & research design: Choosing among five approaches. SAGE.

Dwamena, F., Holmes‐Rovner, M., Gaulden, C. M., Jorgenson, S., Sadigh, G., Sikorskii, A., Lewin, S., Smith, R. C., Coffey, J., & Olomu, A. (2012). Interventions for providers to promote a patient‐centred approach in clinical consultations. Cochrane Database of Systematic Reviews, (12).

Dyer, E., Swartzlander, B. J., & Gugliucci, M. R. (2018). Using virtual reality in medical education to teach empathy. Journal of the Medical Library Association, 106(4), 498-500. https://doi.org/10.5195/jmla.2018.518

Halton, C., & Cartwright, T. (2018). Walking in a patient’s shoes: An evaluation study of immersive learning using a digital training intervention. Frontiers in Psychology, 9, 2124. https://doi.org/10.3389/fpsyg.2018.02124

Hojat, M., DeSantis, J., Shannon, S. C., Mortensen, L. H., Speicher, M. R., Bragan, L., LaNoue, M., & Calabrese, L. H. (2018). The Jefferson Scale of Empathy: a nationwide study of measurement properties, underlying components, latent variable structure, and national norms in medical students. Advances in Health Sciences Education, 23(5), 899-920.

Hu, S. (2022). Raw Jefferson Scores. [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.17973662

Hu, S., & Lai, B. W. P. (2022). Increasing empathy for children in dental students using virtual reality. International Journal of  Paediatric Dentistry. https://doi.org/10.1111/ipd.12957

Lok, B., Ferdig, R. E., Raij, A., Johnsen, K., Dickerson, R., Coutts, J., Stevens, A., & Lind, D. S. (2006). Applying virtual reality in medical communication education: Current findings and potential teaching and learning benefits of immersive virtual patients. Virtual Reality, 10(3-4), 185-195.

MacDonald-Wicks, L., & Levett-Jones, T. (2012). Effective teaching of communication to health professional undergraduate and postgraduate students: A systematic review. JBI Evidence Synthesis, 10(28), 1-12. https://doi.org/10.11124/jbisrir-2012-327

Mead, N., & Bower, P. (2002). Patient-centred consultations and outcomes in primary care: A review of the literature. Patient education and counseling, 48(1), 51-61.

Mercer, S. W., & Reynolds, W. J. (2002). Empathy and quality of care. British Journal of General Practice, 52(Suppl), S9-S12.

mooc vid. (2019, September 13). Scenario 1 [Video]. YouTube. https://www.youtube.com/watch?v=4f6bZ_Kc2xw

mooc vid. (2019, September 13). Scenario 2 [Video]. YouTube. https://www.youtube.com/watch?v=oz4dm45Imkw

Neumann, M., Edelhäuser, F., Tauschel, D., Fischer, M. R., Wirtz, M., Woopen, C., Haramati, A., & Scheffer, C. (2011). Empathy decline and its reasons: A systematic review of studies with medical students and residents. Academic Medicine, 86(8), 996-1009. https://doi.org/10.1097/ACM.0b013e318221e615

Nheu, L., Uahwatanasakul, W., & Gray, A. (2018). Medical students’ experience of a Teddy Bear Hospital as part of a paediatric curriculum. Focus on Health Professional Education: A Multi-disciplinary Journal, 19(3), 40.

Ong, L., Chua, K. H., Soh, J. Y., & Aw, M. M. H. Y. (2018). Healthcare encounters in young children: Impact of Teddy bear Hospital, Singapore. The Asia Pacific Scholar, 3(3), 24-30. https://doi.org/10.29060/TAPS.2018-3-3/OA1055

Santen, L., & Feldman, T. (1994). Teddy bear clinics: a huge community project. MCN: The American Journal of Maternal/Child Nursing, 19(2), 102-106.

Siegel, B., Lewis, H., Bryan, L., & Batisky, D. (2018). The Teddy Bear Hospital as an outreach tool for reducing medical anxiety: A randomized trial. (2018). Pediatrics, 142(1_MeetingAbstract), 783. https://doi.org/10.1542/peds.142.1MA8.783

Stewart, M., Brown, J. B., Weston, W., McWhinney, I. R., McWilliam, C. L., & Freeman, T. (2013). Patient-centered medicine: Transforming the clinical method. CRC press.

*Shijia Hu
Faculty of Dentistry,
National University of Singapore
9 Lower Kent Ridge Road #10-01
National University Centre for Oral Health,
Singapore 119085
Email: denhus@nus.edu.sg