Submitted:4 August 2020
Accepted: 29 October 2020
Published online: 4 May, TAPS 2021, 6(2), 97
https://doi.org/10.29060/TAPS.2021-6-2/MA1601

Piyanjali de Zoysa¹, G. A. Chathra Erandi², D. B. Umaya Wijayaratne², K. P. Navodya Shavindi Jinani², Piumi Kavindya Kandanaarachchi²

¹Department of Psychiatry, University of Colombo, Sri Lanka; ²University of Colombo, Sri Lanka

https://youtu.be/pfWo4cOlPzw

Video Player

Submitted:6 November 2020
Accepted: 10 March 2021
Published online: 4 May, TAPS 2021, 6(2), 98
https://doi.org/10.29060/TAPS.2021-6-2/MA1602

Dujeepa D Samarasekera¹, Bettina Lieske², Derrick Aw³, Shuh Shing Lee¹, Yih Lin Lim¹, Chee Yen Ang¹, Su Ping Yeo¹, Dow Rhoon Koh⁴

¹Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Division of Colorectal Surgery, Department of Surgery, National University Hospital, Singapore; ³Department of General Medicine, Sengkang General Hospital, Singapore; ⁴Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

https://youtu.be/8Sb7kPOb1Rc

Video Player

Submitted: 30 September 2020
Accepted: 18 November 2020
Published online: 4 May, TAPS 2021, 6(2), 88-90
https://doi.org/10.29060/TAPS.2021-6-2/PV2407

Teng Chun Koh¹, Eugene Zhi Jie Lee¹, Charlene Jie Lin Yak¹, Jack Botao Sun¹, Joshua Ren Wei Tay¹, Ann Chong Hui Fong² & Clara Yuen Pun Mok³

¹Faculty of Dentistry, National University of Singapore, Singapore; ²Office of Undergraduate Education and Student Affairs, Faculty of Dentistry, National University of Singapore, Singapore; ³Discipline of Endodontics, Operative Dentistry and Prosthodontics, Faculty of Dentistry, National University of Singapore, Singapore

I. INTRODUCTION

Freshmen orientation programmes are important milestones of university life. In Dentistry, orientation helps build camaraderie and friendship among the small undergraduate student enrolment. Before the COVID-19 pandemic, freshmen orientation featured overnight camps, mass games and group activities which involved face-to-face interactions. With the pandemic, precautionary measures were put in place to curb the spread of the virus. A key measure was the radical move away from face-to-face activities. This had a profound impact on the conduct of the orientation programme in Dentistry. The authors share their experiences as student organisers of a first-ever online freshmen orientation programme (involving 72 freshmen and 60 seniors), and highlight key lessons gleaned from the experience.

II. E-ORIENTATION – CHALLENGING THE STATUS QUO

Planning and implementing an e-Orientation programme is vastly different from that of a conventional face-to-face one. While the latter leverages physical space, infrastructure, resources, and interactions to create a conducive environment to achieve the goals of Orientation, e-Orientation operates in the online space and relies heavily on technology. However, the ongoing COVID-19 pandemic necessitated a turn towards online platforms and tools for the execution of university orientation programmes worldwide (Ullman, 2020), and it was no different for students in the Faculty of Dentistry. As the organisers and planners of the programme, the lack of prior experience with such an online approach proved daunting for us and the challenges we faced included:

A. Managing Unfamiliar Online Tools

While technology was widely available to support the e-Orientation, familiarity with it was lacking. It was therefore necessary to overcome a steep learning curve and get acquainted with the use of new tools such as Zoom, the video-conferencing tool. 

B. Security, Privacy and Safety Concerns

A key concern with the e-Orientation revolved around security and privacy of the freshmen and seniors facilitating the online activities. Zoom was the platform of choice for our e-Orientation, due to its numerous security features, such as the usage of end-to-end encryption to secure meetings and the utilisation of meeting passcodes to ensure that only designated participants could access and enter meetings for e-Orientation (Zoom Communications Inc., n.d.).  Additionally, with many of the activities taking place remotely online, the physical safety of the participants was also a concern if any of them injured themselves during the activities. 

C. Sustaining Engagement of Freshmen

The long hours spent in front of the camera, coupled with the difficulty in engaging their peers over a virtual setting, resulted in participants quickly losing interest within a short time. The remote nature of the engagement also made it more difficult for the participants to deepen their interactions. The participants’ focus could be easily lost with them disengaging from the activities going on online. There was a need for the seniors to constantly exude high levels of energy and enthusiasm needed to keep the freshmen engaged. The lack of reciprocation from the freshmen did little to encourage the seniors.

D. Grappling with Technological Difficulties

An online orientation opened the possibility of technological difficulties presenting a stumbling block to the smooth flow of the entire programme. While the impromptu nature of these occurrences meant these problems could not be entirely headed off beforehand, proper planning for contingencies allowed us to deal with these issues swiftly and effectively when they arose.

III. RISING ABOVE THE CHALLENGES

The e-Orientation was organised primarily using Zoom with each Orientation group comprising 10 freshmen and 2 Orientation Group Leaders (OGLs). This helped to optimise the group size and maximise the online interactions. Over a two-week period, the activities were rolled out for two to three hours on alternate days. The longer duration and shorter engagement each day was deliberate to avoid online fatigue and provide ample opportunity to informally catch up online in smaller groups after the official programme ended each day. The following strategies contributed to the success of organising the e-Orientation.

A. Drawing from the Familiar and Conventional

Traditional games were creatively adapted for adoption online. An example was Human Cluedo where the freshmen messaged their group mates an assigned word to initiate conversations on their own. In the classic game of Cluedo, players attempt to deduce words that are hidden in an envelope, with the player who guesses right winning. In our version of Human Cluedo, every freshman was assigned a word, and had to complete a “murder” by making 2 other freshmen, their “victims”, say this word in their day-to-day conversations over text. If they did so successfully without arousing suspicion, they got a point for completing the “murder”! The “victims” on the other hand had to be wary and deduce the word that the murderer was trying to get them to say – the component of the game that was inspired by the original game. Inspiration was also sought from online games which the participants would be familiar with.

B. Experimenting with Newer Methods of Information Dissemination

While instructions could be given out efficiently through briefings in a face-to-face setting, this was not possible online. To reach out to the participants more effectively and expediently, an Instagram page was used to disseminate information quickly among the participants.

 C. Empowering and equipping Orientation Group Leaders

An Orientation Group Leader (OGL) handbook was developed to guide the student leaders in running the programme independently. It accorded them the flexibility to make adaptations to some of the activities if it was necessary. Zoom meetings were organised to familiarise the OGLs with the use of Zoom and dry runs proved quintessential in foreseeing and eradicating potential technical difficulties. 

IV. GOING VIRTUAL – A REALITY CHECK

Looking back on the e-Orientation, its success was the result of several values. First, the need to be open to new ideas and not discount any suggestions. The e-Orientation experience required new ideas to be tested to make sure these would still achieve the objectives of the Freshmen Orientation programme. Second, the need to see change as fresh opportunities. This also meant that any unexpected twists and turns to the original plans had to be embraced positively and recognised as opportunities to do something differently but with the potential to be better. Thirdly, perseverance proved an important ingredient in the recipe for success. With a major change to the original plans and being thrust into new unknowns, an easy way out would have been to cancel the orientation programme. This would have been an easier option, but not necessarily the correct one. The seniors persisted and rallied together to overcome the difficulties and eventually they succeeded in their foray into an online orientation programme. These values will go a long way in our training to become oral healthcare professionals of the future.

 V. CONCLUSION

There was a palpable sense of apprehension and fear in the beginning. There were many unknowns – would the camp proceed smoothly? Would the juniors enjoy themselves? Would it be overly awkward? Many thought that an online orientation programme would not be as fun and would not achieve the intended objectives.

The willingness to be open to new ideas; a positive attitude towards changes and uncertainty, and a spirit of perseverance helped to overcome the initial fear and scepticism. While this e-Orientation was definitely not on the agenda, its successful conduct has opened our eyes to how it could offer a viable alternative to the tried and tested conventional face-to-face programme. The convenience, accessibility and flexibility of the online platform, together with suitably designed online activities differentiated this Orientation into a unique experience, and possibly resulting in high participation rates among the freshmen.

With an eye on the future, the knowledge learnt through this experience can be passed on to the next batch of junior Orientation leaders, and may also come in helpful should we be involved in the organisation of online versions of regional events, such as the Asia-Pacific Dental Students Association (APDSA) conference, in the future. On a more personal note, this experience has also made each one of us more cognisant of pertinent issues inextricably linked to the use of social media, such as security and privacy, and will shape our behaviour on online platforms in the future as well.

If asked whether we would do this all over again, we would gladly step forth with a resounding Yess!!!

Notes on Contributors

Teng Chun Koh directed the project, reviewed literature and took the lead in writing and editing the manuscript.

Eugene Zhi Jie Lee contributed to the design of the project and ideas for this manuscript.

Charlene Jie Lin Yak contributed to the design of the project and ideas for this manuscript.

Jack Botao Sun contributed to the ideas for the project and this manuscript.

Joshua Ren Wei Tay was invited to make some brief corrections to the manuscript.

Ann Chong Hui Fong advised and provided feedback on the manuscript, aiding development of the manuscript.

Clara Yuen Pun Mok advised and provided feedback on the manuscript, aiding development of the manuscript.

All authors discussed and contributed to the final manuscript.

Acknowledgement

This e-Orientation could not have been successfully carried out without the support and encouragement from fellow classmates and friends, who hosted the e-Orientation games and activities with enthusiasm and passion. Their hard work indubitably contributed to the success of the event.

Funding

The authors have no funding to report. 

Declaration of Interest

The authors have no conflict of interest to declare.

References

Ullman, E. (2020, October 27). How to take college orientation online. https://www.techlearninguniversity.com/how-to/how-to-take-college-orientation-online

Zoom Communications Inc. (n.d.). Security: Zoom trust center. Retrieved January 26, 2021, from https://explore.zoom.us/en-us/trust/security.html

*Teng Chun Koh
9 Lower Kent Ridge Road, #10-01,
National University Centre for Oral Health,
Singapore 119085
Email: e0405935@u.nus.edu

Submitted: 1 October 2020
Accepted: 3 December 2020
Published online: 4 May, TAPS 2021, 6(2), 78-87
https://doi.org/10.29060/TAPS.2021-6-2/OA2406

Yoshitaka Maeda¹, Yoshikazu Asada², Yoshihiko Suzuki¹, Akihiro Watanabe³, Satoshi Suzuki³ & Hiroshi Kawahira¹

¹Medical Simulation Center, Jichi Medical University, Japan; ²Center for Information, Jichi Medical University, Japan; ³Faculty of Health and Medical Sciences, Kanagawa Institute of Technology, Japan

Abstract

Introduction: Students in the early years of medical school should learn clinical site risk assessment skills. However, the effect of this training on clinically inexperienced students is not clear, and it is difficult for students to predict risks from a wide range of perspectives. This study aims to develop and implement three patterns of safety walk rounds (SWR) in a class of students with no clinical experience.

Methods: Three types of SWR were conducted: (A) 37 students observed a familiar classroom and predicted safety risks; (B) 39 students created a profile of a fictitious student in advance and then used Type A parameters; (C) 100 students participated. First, Type A was conducted as a practice. Next, students observed a hospital and predicted risks. All participants in Types A to C had no clinical experience. We classified all risks into perception, comprehension, and action.

Results: For each safety walk-round, there were two types of risk prediction. In Type A, risks such as perception and comprehension were more than 80%. In Types B and C, action risks were 60%. Students had little experience in observing facilities and none at finding safety risks.

Conclusion: Each method had a different risk prediction tendency. Combining the methods could enable students to acquire comprehensive skills in assessing hidden environmental patient safety risks.

Keywords:            Patient Safety Education, Undergraduate Education, Risk Assessment Skill, Safety Walk-Rounds

Practice Highlights

• Proposes a patient safety education method incorporating safety walk rounds (SWR).
• Clarifies the risk prediction tendency of clinically inexperienced students in each SWR pattern.
• Students conducting SWR in familiar classrooms tend to predict certain risks.
• Creating fictitious user profiles before SWR enables prediction of action risks.
• Combining different SWR types could enable comprehensive risk assessment skills.

I. INTRODUCTION

In Japan, first-year medical students are recent high school graduates. 60% of universities that train medical professionals provide patient safety education to fourth year medical students at the start of clinical training (Ishikawa et al., 2008). Further, lower grade educational methods do not include specific guidelines for patient safety education, and students in lower grades do not have sufficient medical knowledge to immediately apply their patient safety knowledge in clinical practice. This problem has been pointed out not only in Japan but also in the US and Canada (Alper et al., 2009). Conversely, the Telluride Interdisciplinary Roundtable (Mayer et al., 2009) and Lucian Leape Institute (2010) showed that patient safety education should be included in the curriculum of all grades. This would enable students to learn the necessity and importance of patient safety knowledge and consider patient safety as an implementation science while continuously practising patient safety skills (Nakajima, 2012).

However, many medical schools teach basic patient safety knowledge through lectures on accident analysis tools, legal responsibility knowledge, ethics, and infection (Mayer et al., 2009); however, students lack education on non-technical skills (Mayer et al., 2009; Nakajima, 2012; Walton et al., 2010). Students should be trained in awareness of safety weaknesses, threats (risks) in the workplace or operations, and how to avoid these risks (Doi et al., 2012). Topic 6 of the World Health Organization’s (WHO’s) Patient Safety Curriculum Guide indicates that students need to take appropriate corrective action when they see an unsafe situation or environment (Walton et al., 2010). However, the WHO guidelines do not explain how these risk assessment skills can be taught to students. Literature that examines the effectiveness of risk assessment skills training for early-year medical students is deficient.

To address these issues, we focused on Safety walk rounds (SWR), in which a safety manager goes to a workplace, listens to staff opinions on safety, and observes the workplace to identify safety issues before an accident (Hafey, 2017; Womack, 2013). Singer and Tucker (2014) pointed out that SWR enhances safety culture. The effects of SWR in the radiology department have reduced the number of unsafe events by half (Donnelly et al., 2008). Additionally, other studies reported that safety managers grow more sensitive to safety issues using SWR and that motivation regarding safety is increased (Frankel et al., 2003; Singer & Tucker, 2014). However, its educational effect and applicability to educating clinically inexperienced students are not clear since SWR has not been used for education.

This study aims to develop and implement three patterns of SWR in a class of students with no clinical experience. We clarify the risk prediction tendency of students in each SWR pattern and discuss the effects.

 II. METHODS

A. Development Process of Three Types of SWR

We developed three patterns of SWRs to help clinically inexperienced students predict risk and considered what motivates students to learn. We used the ARCS model proposed by J. M. Keller in 1983, which is a framework using four elements: Attention (stimulating the learner’s interest, intellectual curiosity, and inquisitiveness); Relevance (making the content familiar and meaningful); Confidence (encouraging learners to learn); and Satisfaction (giving the learner a sense of satisfaction and motivation to learn more) (Keller, 1987). ARCS is an acronym for these elements.

· SWR in daily situations (SWR-D): Experts are better at predicting risks than novices as the latter has limited knowledge of important aspects of each situation (Murata et al., 2009). Hence, clinically inexperienced students might find it difficult to predict risks in clinical situations. Using the ARCS model, students need to be given Attention, Confidence and Satisfaction. Therefore, we developed SWR in daily situations (SWR-D). Students observe daily situations in classrooms and school buildings for instances of safety risk and take pictures. We use daily situations as classrooms and school buildings are familiar environments for students, and there are many safety risks for educating students.

· SWR in daily situations using the Persona method (SWR-DP): It may be difficult for students who have never performed SWR to observe safety risks in daily situations, and students’ Confidence should be high. Therefore, we combined the persona method with SWR to create a virtual profile of a virtual user, including name, gender, age, and information about the system (e.g., technological literacy). The Persona method has been used to examine the safety of driving support systems (Lindgren et al., 2007). We hypothesize that the Persona method would help students to predict risk from the perspective of a specific user. The student considers the problems the virtual user will face and their behaviour (Cooper, 2004; Mulder & Yaar, 2006). Students created fictitious student profiles (personas) and conducted SWR assuming that the persona students would spend one day in school buildings and classrooms.

· SWR in clinical situations (SWR-C): It may be difficult for students to associate SWR with patient safety in clinical practice, as SWR in daily situations were not related to clinical practice conducted. Further, students might not be motivated to learn—using the ARCS model, students need to see Relevance and Satisfaction. Therefore, we developed the SWR in clinical situations (SWR-C). First, to practice SWR, students performed SWR-D. After SWR-D, they observed clinical situations in hospitals to predict risks (SWR-C).

B. Description of Participants and SWR Implementation Process

This study involved first-year medical students and third-year students in the medical engineer training courses who had no clinical experience. After participating in each SWR pattern, students were asked for their opinions.

1) SWR-D: The participants included 37 students in the third-year medical engineer training course and 100 students who had been in medical school for one month. SWR-D was administered to the third-year (2018) medical engineer training students in one session. SWR-D was implemented as one of the required general education courses for first-year medical students in 2019. An exercise using SWR-D was given to all participants who worked in groups of four to five. Students photographed incidences of safety risk (30 minutes) and collaborated to identify the risks in each photo (20 minutes).

2) SWR-DP: The participants included 39 students (different from SWR-D) in the third-year medical engineer training course. SWR-DP was administered to the third-year (2019) medical engineering students in one session and conducted in groups of four to five. Each group considered one persona (virtual student profile) for the first 20 minutes, and SWR-D was conducted as before.

3) SWR-C: The participants included 100 students (same as SWR-D) who had been in medical school for one month. Each student was assigned one clinical department in advance. Two weeks after the SWR-D, early exposure training was conducted. During training, students found safety risks in clinical situations and outlined the identified risks in reports as photography was not allowed for confidentiality.

C. Statistical Analysis

The risks predicted by students in each SWR pattern were counted and classified into the following: perception—difficulty perceiving something that exists in the outside world (e.g., signs that are difficult to read); comprehension—difficulty understanding the meaning of something that exists in the outside world and in planning what action to take (e.g., signs that are difficult to understand); and action—difficulty performing the intended action (e.g., places where it is difficult to walk).

These classifications are based on Norman’s seven stages of action (Norman & Draper, 1986) where human actions are classified into seven stages: forming the goal, forming the intention, specifying an action, executing the action, perceiving the state of the world, interpreting the perception, and evaluating the outcome (Norman, 1988). This is a representative model widely used for the design evaluation of man-machine systems such as computers (Fleming & Koman, 1998) to understand human cognitive behaviour that leads to human error in medical treatment (Zhang et al., 2002; Zhang et al., 2004).

D. Ethical Considerations in This Research

The ethical requirements in this study are in accord with the Declaration of Helsinki. We emphasized and explained to students that participation was voluntary and that declining to cooperate would have no influence on their grades. We also explained that consent to participate could be withdrawn at any time, that the results of this study may be published after processing, and that the students’ personal information would not be revealed. The students entered their consent in the e-learning system Moodle. This study was considered exempt by the Jichi Medical University Review Board (Number 18-014).

III. RESULTS

The total number of perception, comprehension, and action risks in each SWR is shown in Figure 1. Table 1 shows typical predicted risks and some of the images taken by students. The data that support the findings of this study are openly available in Figshare at <http://doi.org/10.6084/m9.figshare.13012664 (Maeda et al., 2021)>.

Figure 1. Classification results of risks predicted by students in each SWR pattern

Table 1. Typical risks and captured images for each SWR pattern

IV. DISCUSSION

A. Student Risk Predicting Tendency in Each SWR Pattern

The importance of institutional design in patient safety has been pointed out in many publications. For example, environmental design is being considered to avoid various risks, such as falls and patient suicides, mixing up patients, and improper handling of tubes and connectors (Joseph & Rashid, 2007; Michalska & Szewieczek, 2007; Reiling, 2006; Reiling et al., 2003; Reiling et al., 2008). In particular, it has been pointed out that the indoor environment (e.g., noise and lighting) and interior design (e.g., furniture and materials) are important (Joseph & Rashid, 2007). When considering the design, it is necessary to predict both the direct impact risks and the indirect impact risks of accidents as points of view when predicting onsite risks. Direct impact risks are the aspects of hospital design that can directly impact safety outcomes, such as patient falls and medical errors (Joseph & Rashid, 2007). This is considered to correspond to “action” in this study. For example, a tall counter design is directly linked to the undesirable consequence of a wheelchair user being unable to receive a meal. Indirect impact risks are the aspects of hospital design that can cause users to make incorrect decisions that lead to accidents and errors (Joseph & Rashid, 2007). This is considered to correspond to “perception” and “comprehension” in this study. For example, a paediatric refrigerator’s “be careful of allergies” poster does not directly lead to an unsafe outcome; however, if a child is unable to understand, then an incorrect decision to eat food to which the child is allergic may lead to an unsafe outcome.

In SWR-D, many indirect impact risks (perception and comprehension) were predicted. Conversely, few action risks (direct impact risks) were predicted. SWR-DP and SWR-C results were the opposite to SWR-D; many direct impact risks and few indirect impact risks were identified.

In SWR, it is necessary to identify risks through brainstorming and simulation using the operators’ experience and reasoning (Okubo et al., 2014). However, different techniques identify different risk types. Indirect impact risks such as perception and comprehension are related to human internal thinking, such as incorrect decisions. To identify these risks, students need to observe from the perspective of the person concerned and brainstorm the risks. Conversely, the action is observable—a risk that students can predict by observing the actions of the concerned person or simulating behaviour as the concerned person. Subsequently, what caused each SWR to favour predicting one type of risk over another?

In SWR-D, most students observed a familiar daily situation from their (able-bodied) perspective. In SWR-DP, students observed a familiar daily situation from a persona’s point of view that differed from that of an able-bodied person. In SWR-C, they observed unfamiliar clinical situations from the patients’ perspective (it is unknown whether this perspective was different from their own). This suggests that when students observe a familiar environment from their own perspective, they concentrate on brainstorming about risks but do not conduct much action simulation (acting on a simulated basis and identifying risks). Consequently, the risks related to action were few.

Conversely, if students observe risk from others’ perspective, they may not be able to brainstorm well, and they may tend to predict risks by performing action simulations. In SWR-DP, many images of simulations, such as trying to use a cafeteria as a persona (for example, a wheelchair user), were recorded. Notably, novice nurses tend to observe bedsides without being able to imagine the patient’s condition or behaviour (Daikoku & Saito, 2017); it seems difficult to predict risks associated with unfamiliar subjects (people/environment) only by brainstorming.

However, during SWR-C, we asked students to predict risks only by observation to avoid interference with patient care, and students were unable to perform action simulations. Nevertheless, the risks associated with actions were the most predicted. According to the risks predicted by the students (Table 1), students likely found patients who were confused and observed their behaviour. In the clinical situation, there were several observable patients that students could predict many risks based on observable actions.

In summary, for clinically inexperienced students to predict many indirect impact risks (perception, comprehension), it is better for them to make observations from their own perspectives in a daily situation where brainstorming can be easily conducted. It is better to ask students to observe an unfamiliar person and environment to identify more direct impact risks (actions). It would be better to create a fictitious user profile (persona) and conduct SWR (SWR-DP) or conduct SWR in a clinical situation unfamiliar to students (SWR-C). In any case, if each SWR is implemented independently, the predicted risks are biased. Therefore, by combining each SWR, it may be possible to develop skills that enable students to find direct impact and indirect impact risks in a well-balanced manner.

B. Limitations

This study evaluates the educational benefits of three SWR patterns and discusses their effectiveness. We could not compare the three patterns of SWR for the following aspects. First, we could not examine the relationship between participants’ background and SWR. This study targeted first-year medical students (SWR-C) and third-year students in the medical engineer training course (SWR-D, SWR-DP). Each student’s age and expertise were different. Therefore, the background of each participant could have affected the participant’s risk prediction tendency. However, all students shared the common background of having no clinical experience and had basic education on patient safety and conducted SWR with a basic knowledge of human factors, such as medical accident analysis methods. Second, we could not conduct a comparative study of each SWR by statistical analysis as there were differences in the way each SWR was conducted. SWR-D and SWR-DP were administered as a group, while SWR-C was administered individually because of restrictions in clinical practice. We could not calculate the average number of hazard predictions per student.

Further, each SWR was conducted in a compulsory class; therefore, there were a large number of students per faculty member. Consequently, we could not observe all students. Therefore, we have little record of how students predicted risks, especially in SWR-C. This is because the simulation actions in clinical situations and photography were restricted.

Additionally, when examining the safety of facility design, it is important to predict risks to healthcare professionals as well as patients (Reiling et al., 2003). However, in SWR-C, 90% of the risks were related to patients. Future studies should examine training methods that enable clinically inexperienced students to predict risks to healthcare professionals.

V. CONCLUSION

In this study, we proposed a patient safety education method incorporating SWR. We conducted SWR-D, SWR-DP, and SWR-C sessions and clarified the risk assessment tendency of students in each SWR pattern. For students to predict many indirect impact risks (perception comprehension), it is better to have students observe a daily situation in which it is easy to identify risks from their own points of view (SWR-D). To find many direct impact risks (action), it is better for students to create a persona and observe a daily situation (SWR-DP) or clinical situation (SWR-C). This suggests that a combination of these SWRs would provide students with the skills to comprehensively predict the patient safety risks in facilities and the environment. By continuously conducting all SWR session types starting at lower grade levels, it is expected that skills related to risk assessment will be effectively acquired. It is expected that SWR education from pre-graduates will increase the number of medical professionals who can conduct an appropriate risk assessment in the field, resulting in improved quality and safety of healthcare.

Notes on Contributors

Yoshitaka Maeda, PhD, a Research Associate at the Medical Simulation Center at Jichi Medical University, Japan, contributed to the conceptualization, methodology, validation, formal analysis, investigation, resources, data curation, writing (original draft, review, and editing), and visualization.

Yoshikazu Asada, PhD, an Assistant Professor at the Center for Information at Jichi Medical University, Japan, contributed to the methodology, validation, formal analysis, and visualization.

Yoshihiko Suzuki, MD, an Assistant Professor at the Medical Simulation Center at Jichi Medical University, Japan, contributed to the conceptualization and methodology.

Akihiro Watanabe, MS, a Research Associate at the Faculty of Health and Medical Sciences at Kanagawa Institute of Technology, Japan, contributed to the validation and investigation.

Satoshi Suzuki, PhD, a Professor at the Faculty of Health and Medical Sciences at Kanagawa Institute of Technology, Japan, contributed to the data curation, writing (review and editing), and visualization.

Hiroshi Kawahira, MD, PhD, FACS, a Professor at the Medical Simulation Center at Jichi Medical University, Japan, contributed to the writing (review and editing), supervision, and project administration.

All the authors have read and approved the final manuscript.

Ethical Approval

This study was approved by the Jichi Medical University Institutional Review Board (protocol number 18-014).

Data Availability

The data that support the findings of this study are openly available in Figshare repository, http://doi.org/10.6084/m9.figshare.13012664

Funding

There is no funder for this study.

Declaration of Interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

 References

Alper, E., Rosenberg, E. I., O’Brien, K. E., Fischer, M., & Durning, S. J. (2009). Patient safety education at US and Canadian medical schools: Results from the 2006 clerkship directors in internal medicine survey. Academic Medicine, 84(12), 1672-1676. https://doi.org/10.1097/acm.0b013e3181bf98a4

Cooper, A. (2004). The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity (Vol. 2). Sams Publishing.

Daikoku, R., & Saito, Y. (2017). Characteristics of gaze in expert nurses during observation at the bedside. Japanese Journal of Nursing Art and Science, 15(3), 218-226. https://doi.org/10.18892/jsnas.15.3_218

Doi, T., Kawamoto, K., & Yamaguchi, K. (2012). Difference by level of awareness and the years of experience to patient treatment safety. Japanese Journal of Radiological Technology, 68(5), 608-616. https://doi.org/10.6009/jjrt.2012_JSRT_68.5.608

Donnelly, L. F., Dickerson, J. M., Lehkamp, T. W., Gessner, K. E., Moskovitz, J., & Hutchinson, S. (2008). IRQN award paper: Operational rounds: A practical administrative process to improve safety and clinical services in radiology. Journal of the American College of Radiology, 5(11), 1142-1149. https://doi.org/10.1016/j.jacr.2008.05.017

Fleming, J., & Koman, R. (1998). Web navigation: Designing the user experience. O’Reilly Media.

Frankel, A., Gandhi, T. K., & Bates, D. W. (2003). Improving patient safety across a large integrated health care delivery system. International Journal for Quality in Health Care, 15(suppl_1), i31-i40. https://doi.org/10.1093/intqhc/mzg075

Hafey, R. B. (2017). Lean safety gemba walks: A methodology for workforce engagement and culture change. Productivity Press.

Ishikawa, M., Hirao, T., & Maezawa, M. (2008). Study of patient safety education for undergraduates. Medical Education, 39(2), 115-119. https://doi.org/10.11307/mededjapan1970.39.115

Joseph, A., & Rashid, M. (2007). The architecture of safety: Hospital design. Current Opinion in Critical Care, 13(6), 714-719. https://doi.org/10.1097/mcc.0b013e3282f1be6e

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2-10. https://doi.org/10.1007/BF02905780

Lindgren, A., Chen, F., Amdahl, P., & Chaikiat, P. (2007). Using personas and scenarios as an interface design tool for advanced driver assistance systems. In: Stephanidis C. (eds) Universal Access in Human-Computer Interaction. Ambient Interaction. UAHCI 2007. Lecture Notes in Computer Science, vol 4555. Springer. https://doi.org/10.1007/978-3-540-73281-5_49　

Lucian Leape Institute. (Eds.). (2010). Unmet needs: Teaching physicians to provide safe patient care. National Patient Safety Foundation.

Maeda, Y., Asada, Y., Suzuki, Y., Watanabe, A., Suzuki, S., & Kawahira, H. (2021). Proposal for safety walk-round education to develop risk prediction skills of novice health professional students [Data set]. Figshare. http://doi.org/10.6084/m9.figshare.13012664

Mayer, D., Klamen, D. L., Gunderson, A., & Barach, P. (2009). Designing a patient safety undergraduate medical curriculum: The Telluride interdisciplinary roundtable experience. Teaching and Learning in Medicine, 21(1), 52-58. https://doi.org/10.1080/10401330802574090

Michalska, J., & Szewieczek, D. (2007). The 5S methodology as a tool for improving the organization. Journal of Achievements in Materials and Manufacturing Engineering, 24(2), 211-214.

Mulder, S., & Yaar, Z. (2006). The user is always right: A practical guide to creating and using personas for the web. New Riders.

Murata, A., Hayami, T., & Moriwaka, M. (2009). Visual information processing characteristics of drivers in prediction of dangerous situation-Comparison among novice, expert and non-licensed person. In IEEE SMC Hiroshima Chapter (Eds.), Proceedings of Fifth International Workshop on Computational Intelligence & Applications, (pp. 254-257). IEEE SMC Hiroshima Chapter.

Nakajima, K. (2012). Patient safety and quality of medical care. Topics: III. Management of patient safety and quality of medical care: Theory and practice; 4. Patient safety and quality improvement education for undergraduate medical students. Nihon Naika Gakkai zasshi. The Journal of the Japanese Society of Internal Medicine, 101(12), 3477. https://doi.org/10.2169/naika.101.3477

Norman, D. A. (1988). The psychology of everyday things. Basic Books.

Norman, D. A., & Draper, S. W. (1986). User centered system design: New perspectives on human-computer interaction. L. Erlbaum Associates.

Okubo, Y., Kuroda, R., & Yamamoto, K. (2014). Health and safety patrol in universities. Journal of Environment and Safety, 5(3), 169-175. https://doi.org/10.11162/daikankyo.14C0702

Reiling, J. (2006). Safe design of healthcare facilities. BMJ Quality & Safety, 15(suppl 1), i34-i40. https://dx.doi.org/10.1136%2Fqshc.2006.019422

Reiling, J., Breckbill, C., Murphy, M., McCullough, S., & Chernos, S. (2003). Facility designing around patient safety and its effect on nursing. Nursing Economics, 21(3), 143.

Reiling, J., Hughes, R. G., & Murphy, M. R. (2008). The impact of facility design on patient safety. In Hughes, R. G (Ed.), Patient safety and quality: An evidence-based handbook for nurses. Agency for Healthcare Research and Quality.

Singer, S. J., & Tucker, A. L. (2014). The evolving literature on Safety walkrounds: Emerging themes and practical messages. BMJ Quality & Safety, 23(10), 789–800. http://dx.doi.org/10.1136/bmjqs-2014-003416

Walton, M., Woodward, H., Van Staalduinen, S., Lemer, C., Greaves, F., Noble, D., Ellis, B., Donaldson, L., & Barraclough, B. (2010). The WHO patient safety curriculum guide for medical schools. BMJ Quality & Safety, 19(6), 542-546. http://dx.doi.org/10.1136/qshc.2009.036970

Womack, J. (2013). Gemba walks: Expanded 2nd edition. Lean Enterprise Institute.

Zhang, J., Patel, V. L., & Johnson, T. R. (2002). Medical error: Is the solution medical or cognitive? Journal of American Medical Informatics Association, 9(6 Suppl 1), s75-77. https://dx.doi.org/10.1197%2Fjamia.M1232

Zhang, J., Patel, V. L., Johnson, T. R., & Shortliffe, E. H. (2004). A cognitive taxonomy of medical errors. Journal of Biomedical Informatics, 37(3), 193-204. https://doi.org/10.1016/j.jbi.2004.04.004

*Yoshitaka Maeda, PhD
3311-1, Yakushiji, Shimotsuke-shi,
(Kinen-tou 7th floor),
Tochigi,Japan, 329-0498
Tel: +81-285-58-7455
Email: y-maeda@jichi.ac.jp

Submitted: 30 April 2020
Accepted: 8 September 2020
Published online: 4 May, TAPS 2021, 6(2), 66-77
https://doi.org/10.29060/TAPS.2021-6-2/OA2391

Pauline Luk & Julie Chen

The University of Hong Kong, Hong Kong

Abstract

Introduction: A novel initiative allowed third year medical students to pursue experiential learning during a year-long Enrichment Year programme as part of the core curriculum. ‘connect*ed’, an online virtual community of learning was developed to provide learning and social support to students and to help them link their diverse experiences with the common goal of being a doctor. This study examined the nature, pattern, and content of online interactions among medical students within this community of learning to identify features that support learning and personal growth.

Methods: This was a quantitative-qualitative study using platform data analytics, social network analysis, thematic content analysis to analyse the nature and pattern of online interactions. Focus group interviews with the faculty mentors and medical students were used to triangulate the results.

Results: Students favoured online interactions focused on sharing and learning from each other rather than structured tasks. Multimedia content, especially images, attracted more attention and stimulated more constructive discussion. We identified five patterns of interaction. The degree centrality and reciprocity did not affect the team interactivity but mutual encouragement by team members and mentors can promote a positive team dynamic.

Conclusion: Online interactions that are less structured, relate to personal interests, and use of multimedia appear to generate the most meaningful content and teams do not necessarily need to have a leader to be effective. A structured online network that adopts these features can better support learners who are geographically separated and engaged in different learning experiences.

Keywords:            Online Learning, Undergraduate, Interaction, Experiential Learning

Practice Highlights

• Image-based messages and less structured online activities focused on experience-sharing engage students and stimulate a more constructive discussion.
• The proactivity of students and mentors can foster a positive team dynamic and learning experience.
• A team or group leader is not always necessary to promote group interaction.

In general, all mentors were more active than students as teachers initiated new posts and were often keen to share information with students (Appendix 3). Even when students were encouraged to create new posts, they tended to focus on completing the tasks in the Inquiry Pods.

Diagram 1: Patterns of online interactions by teams

3) Pattern 3: Student degree centrality: Team 3 is such an example, showing that the thick arrows are pointing towards students, meaning that the interaction is initiated by students. The mentor took a less important role in the conversations. The degree centrality is towards students and the mentor was outside of the interaction centre. For instance, in team 3, the mentor posted 12 posts, 19 comments, and 20 reactions, while the seven students posted 1 to 12 post, 17 to 68 comments, and 0 to 64 reactions respectively. In this pattern, mentor was situated outside the conversation circle. The degree of centrality shifted to students.

4) Pattern 4: Student degree centrality: The dynamics of interaction leaned towards active students, which were represented by the thick edges towards certain students. In this pattern, there were usually multiple centre points that did not include the mentor. For instance, in team 4, the mentor posted 11 posts, 27 comments, and 90 reactions, while the seven students posted 0 to 3 post, 28 to 78 comments, and 0 to 39 reactions respectively. The degree of centrality shifted to multiple students.

5) Pattern 5: Diversified degree centrality: Mentors were active in posting, having similar frequency of comments as the students and the number of comments among all members are the same, and having low reactions. In this pattern, there are multiple conversation nodes and most are interactions between students. Those interactions are more student-driven and indicate multi-centred conversation. For instance, in team 5, the mentor posted 18 posts, 36 comments, and 3 reactions, while the six students posted 0 to 3 post, 31 to 62 comments, and 4 to 29 reactions respectively. The degree of centrality shifted to more than one student. In this pattern, there is more interaction between students as shown by the bi-directional arrows. The degree centrality is low with diversified centres.

These patterns show that teams could have single-centred interaction (Pattern 1 & 2) or multi-centred interaction (Pattern 3, 4, & 5) with each representing different team interactions. Team activity, and not the centeredness of the interaction, was associated with the effectiveness of collaboration and the completion of tasks. In addition, most teams demonstrated one-way communication when interacting. That means the reciprocity of a network is low. In Teams 1 and 2, the interaction dynamics favoured the mentors, while in Teams 3, 4, and 5, the dynamics leaned towards active students. In contrast, team 5 demonstrated strong reciprocity.

However, after comparing the patterns of all 33 teams, there is no indication that a certain pattern was better than the others. There was no significant difference in the on-time completion rate for the IP assignments for the five most active teams (86.9%) compared with all 33 teams (85.5%).

C. Content of Posts

In connect*ed, students shared their Enrichment Year experience using text, photo, video, or related links of other websites. The nature of interactions was predominantly text-based, as it is easier to post and interact using the text. However, image-based messages attracted more attention and stimulated a more constructive discussion.

There were three particular areas that generated greater levels of interactivity. Firstly, students were very willing to share and reflect on their personal experiences. Taking the ‘Communicator’ Inquiry Pod as an example, students shared their observations on communication in their respective settings by posting on the team wall:

“There is a huge contrast here, where students actively ask questions even if the setting involves 80+ students. I suppose the background behind the two nationalities have a huge role in it, as Asians tend to be a bit more shy compared to the extrovertiveness commonly shared by Westerners. While we should embrace who we were and are, I think it is also beneficial to observe others and learn from such observations.”

Student A (studied abroad)

This text-based conversation thread compared and contrasted effective classroom communication in different countries. It enabled students to reflect and to draw on their own experiences to benefit all team members.

The second area of interest for students was social support. One of the most popular activities was the posting of photos and videos about their Enrichment Year activities including when they are performing social service missions, cooking a gourmet meal or joining group gatherings during festive occasions. Those posts generated numerous responses and reactions indicating a keen interest in reaching out and maintaining social connectedness.

Thirdly, students were more active online when there was information being shared related to the medical practice and they are more willing to discuss their views as shown in this sequence from Team 3:

“Being a MBBS student, people around may ask us for medical advice. They think we are knowledgeable to make a diagnosis based on their description and believe we are able to help. However, as we are not yet qualified, it is inappropriate for us to give any professional opinion. Sometimes, I would like to share what I have learnt and suggest some possible solutions. Nevertheless, at the same time, I am afraid my opinion would affect their health seeking behaviour, for instance, they might just follow what I share instead of seeing a doctor.”

Student B 

“It’s true that we are not knowledgeable enough to give medical advice and it will be misleading to our relatives and friends if they take our opinions as professional advice and decide not to seek proper medical opinions. Thus, we should always remind ourselves of the role as medical students and think about the impacts of our words.”

Student C 

“I understand your feeling as my relatives and friends also ask me for medical advice. It will be safer to advise them to seek help from medical professionals for diagnosis or other serious health issues. However, as a medical student, I think it is possible for us to give them some lifestyle advice without causing harmful consequences, for instance, smoking cessation, diet with lower cholesterol content and moderate exercise. Although we are not qualified to make a diagnosis at the moment, we can still use our medical knowledge as a way to promote public health and arise their health conscious.”

Student D

Students more actively express their opinions when the topics under discussion are related to the profession they are aiming to join.

D. Feedback and Focus Group Interviews

Although there were only 9 student interviewees, we found repeated themes and content suggesting data saturation.  This may be because connect*ed comprised only 10% of the overall Enrichment Year and students did not pay particular attention to this component resulting in little variation in responses during the FGD.

The main theme that arose from the FGD with students and mentors was about the most rewarding aspect of the online interaction in connect*ed. Both groups indicated that this was the social connectivity attained through student sharing of day-to-day life during the Enrichment Year.

“The photo and video did not need much effort to share with others, but they are more interesting and can let me know more about how others were doing during their Enrichment Year.”

Student E

 “I am very interested in knowing the life of others in other universities. I hope they can share more and we could see others’ videos.”

Student F

 “Sharing things we learned with the team could help us to be more socialized.”

Student G

Mentors also enjoyed knowing more about students’ Enrichment Year life and believed that students should enjoy themselves while learning.

“The connect*ed is a good example helping students to bridge their knowledge and core value. The sharing of experience (related to the Enrichment Year) is important, it engages students in the discussion”

Mentor X

“The platform support each other very well. This is a platform for socializing and communicating. I know what students are doing if they posted on the team wall.”

Mentor Y

The value of social connectivity for support was further emphasized by students suggesting that the platform was more useful for social networking than learning.

 “connect*ed provided a platform for us sharing the struggles and support each other when I was having my Enrichment Year.”

Student I

This view was echoed by mentors who believed that connect*ed provided necessary support for students especially those who were overseas. Mentors used the chat function on Workplace to have personalized communication with their members and to offer advice.

 “I used the chat function on the Workplace, which is more personal and can support each other very well…. I can have immediate interaction with students.”

Mentor Z

Students found mentors were motivating and encouraged them to interact in teams which led to some mentor-centric team interactions.

 “Our mentor is very motivating and encourages all members to participate in the discussion. She guided us through to complete the inquiry pods.”

Student J

 “In my team, there are some inactive members who have demotivated me to interact. If there is an active member, I think it would help.”

Student K

The findings also indicated that proactivity by student members, participation by the mentor, responsiveness, and social/non-academic discussions fostered a positive team dynamic and a positive online learning experience, regardless of whether the team interaction was primarily single-centred or multi-centred.

IV. DISCUSSION

This study examined the nature, pattern, and content of online interactions among medical students within a virtual community of learning among the inaugural cohort of the Enrichment Year to identify features that support learning and personal growth.

Our results found that students favoured online interactions that were less structured, image-rich and focused on sharing of experience to learn from each other and to support one another. Multimedia content, especially images, attracted more attention and stimulated discussion that was more constructive. This is consistent with findings in the literature that show that images have a positive influence on learning and engagement (Chan & Unsworth, 2011; Stuijfzand et al., 2016). Sharing of personal experience helped students to reflect on their own experience and explore how others experienced their Enrichment Year. The results support previous studies that suggest self-reflection and community building enhances experiential learning (Arnold & Paulus, 2010; Pai, 2016). This builds a virtual community that allows students to share their struggles which students found to be a crucial aspect of giving and receiving social support. The use of the different modes of communication available on Workplace, including the text messaging and voice calls as well as social media posting provided flexible avenues of support to students. The finding is very similar to the outcomes of a project involving a mobile application for experiential learning activities (Schnepp & Rogers, 2017). We also observed that the number of positive reactions (like, love, haha or wow) far exceeded the number of negative reactions (sad and angry). This is a visual form of encouragement from mentors and peers that reflected their interest in engaging with each other.

From the patterns derived from our social network analysis, we found that the interaction could be uni-directional or bi-directional, but there was no correlation of the interaction with team effectiveness in completing tasks on time. As seen in the social network patterns identified, active mentors can drive team interaction. However, in contrast with other findings in the literature, the degree centrality and reciprocity do not affect the team interactional dynamic (Jan & Vlachopoulos, 2019). Regardless of the directionality of the predominant interaction, if there are active members in the team or the mentors are motivating, these individuals are the key to generating more interaction and enhancing online learning experience of students.

Ideally, both mentors and all students should be active, but having at least one or two more active students, can raise the team dynamic. Once some students are willing to share their experience and give timely responses, it can stimulate others to join. Continued encouragement of active members and mentors can promote a positive team dynamic. In terms of degree centrality, the observation that no pattern of interaction was superior to the others suggests that a single leader is not always necessary to for the team to be effective.

This study suggests that interactions will occur most naturally when students are doing what they feel is useful such as maintaining social support with each other and their mentor. In order to be accepted, learning initiatives such as linking learning with experiential activities will need to be less formal and integrate more smoothly with students’ demonstrated desire for social support and interest to share experience. In addition, attention to team formation and ensuring opportunity develop team cohesion would be essential as students in the FGD, commented that when there were members they do not know well, it will be a hindrance for interaction. As the connect*ed is one of the graded components of the Enrichment Year, we observed that the assessment could serve as an external motivator encouraging students to contribute to the work and support their team. However, it could also have a negative impact as it is perceived as an additional burden and may pressure some students to participate for the sake of participating and doing so in an inauthentic way.

 V. CONCLUSION

The online virtual community, connect*ed, to support experiential learning for medical students is still at an early stage. Features of connect*ed that facilitated learning and personal growth included a focus on student support and sharing especially with multimedia, less structured interactions, and teams with active members and/or mentor. It is important to note that interaction does not equate to learning (Jan & Vlachopoulos, 2019), and so the use of an online network that adopts these features may better support learners but the effectiveness of achieving formal learning outcome should be further studied. We will continue to modify and evaluate the functionality of the connect*ed community to ensure it is fit-for-purpose to support students’ needs and learning.

Notes on Contributors

Pauline Luk and Julie Chen contributed to the design and implementation of the research, analysis of the results and writing of the manuscript. PL drafted the manuscript, and JL edited and contributed to the intellectual content of the manuscript and provided overall supervision of the project. Both authors and approved the final manuscript.

Ethical Approval

This research received approval from the HKU Institutional Review Board (UW18-121). Consent was obtained from participants for the research study.

Acknowledgements

The authors sincerely appreciate the support from the mentors and students who participated in this study, the collaboration with the Education University of Hong Kong, and the administrative and technical support rendered by Mr. Francis Tsoi and Miss Joyce Tsang throughout the project.

Funding

This project was funded by the Hong Kong University Grants Committee (UGC) Funding Scheme for Teaching and Learning Related Proposals (2016-19 Triennium).

Declaration of Interest

The authors report no conflicts of interest.

References

Alamodi, A. A., Abu-Zaid, A., Eshaq, A. M., & Al-Kattan, K. (2018). The summer enrichment program: A multidimensional experiential enriching experience for junior medical students. The American Journal of the Medical Sciences, 356(2), 185-186. https://doi.org/10.1016/j.amjms.2018.05.005

Arnold, N., & Paulus, T. (2010). Using a social networking site for experiential learning: Appropriating, lurking, modeling and community building. The Internet and Higher Education, 13(4), 188-196.  https://doi.org/10.1016/j.iheduc.2010.04.002

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: an open source software for exploring and manipulating networks. In the International AAAI Conference on Weblogs and Social Media. 361-362. Retrieved April 1, 2020, from https://vbn.aau.dk/ws/files/328840013/154_3225_1_PB.pdf

Chan, E., & Unsworth, L. (2011). Image–language interaction in online reading environments: Challenges for students’ reading comprehension. Australian Educational Researcher, 38(2), 181-202.  https://doi.org/10.1007/s13384-011-0023-y

Greenberg, G. (2006). Can we talk?  Electronic portfolios as collaborative learning spaces. In A. Jafari & C. Kaufman (Eds.), Handbook of Research on ePortfolios. Idea Group Inc.

HKU. (2017). Educational aims and institutional learning outcomes. Retrieved April 1, 2020, from http://www.handbook.hku.hk/ug/full-time-2017-18/important-policies/educational-aims-and-institutional-learning-outcomes

Jan, S., & Vlachopoulos, P. (2019). Social network analysis: A framework for identifying communities in higher education online learning. Technology, Knowledge and Learning, 24(4), 621-639. https://doi.org/10.1007/s10758-018-9375-y   

Kallail, K. J., Shaw, P., Hughes, T., & Berardo, B. (2020). Enriching medical student learning experiences. Journal of Medical Education and Curricular Development, 7, 4. https://doi.org/10.1177/2382120520902160

Lave, J., & Wenger, E. (1991). Situated learning: legitimate peripheral participation. Cambridge University Press

Lyons, Z. (2017). Establishment and implementation of a psychiatry enrichment programme for medical students. Australasian Psychiatry, 25(1), 69-72. https://doi.org/10.1177/1039856216671663

Ma, L. (2012). Some philosophical considerations in using mixed methods in library and information science research. Journal of the American Society for Information Science and Technology, 63(9), 1859-1867.  https://doi.org/10.1002/asi.22711

Pai, H. C. (2016). An integrated model for the effects of self-reflection and clinical experiential learning on clinical nursing performance in nursing students: A longitudinal study. Nurse Education Today, 45, 156. https://doi.org/10.1016/j.nedt.2016.07.011

Saqr, M., Fors, U., Tedre, M., & Nouri, J. (2018). How social network analysis can be used to monitor online collaborative learning and guide an informed intervention. PLoS One, 13(3). http://dx.doi.org/10.1371/journal.pone.0194777

Schnepp, J., & Rogers, C. (2017). Evaluating the acceptability and usability of EASEL: A mobile application that supports guided reflection for experiential learning activities. Journal of Information Technology Education: Innovations in Practice, 16, 195.

Stuijfzand, B. G., van Der Schaaf, M. F., Kirschner, F. C., Ravesloot, C. J., van Der Gijp, A., & Vincken, K. L. (2016). Medical students’ cognitive load in volumetric image interpretation: Insights from human-computer interaction and eye movements. Computers in Human Behavior, 62, 394-5632.

Vygotsky, L. S. (1978). Mind in Society. Harvard University Press. https://doi.org/10.2307/j.ctvjf9vz4

*Pauline Luk
5/F William MW Mong Block
21 Sassoon Road,
Pokfulam, Hong Kong
Email: pluk@hku.hk

Submitted: 21 August 2020
Accepted: 12 November 2020
Published online: 4 May, TAPS 2021, 6(2), 57-65
https://doi.org/10.29060/TAPS.2021-6-2/OA2378

Nicholas Beng Hui Ng^1,2, Mae Yue Tan^1,2, Shuh Shing Lee³, Nasyitah binti Abdul Aziz³, Marion M Aw^1,2 & Jeremy Bingyuan Lin^1,2

¹Khoo Teck Puat-National University Children’s Medical Institute, National University Health System Singapore; ²Department of Paediatrics, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Centre for Medical Education (CenMED), Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: The coronavirus disease 2019 (COVID-19) pandemic has brought about additional challenges beyond the usual transitional stresses faced by a newly qualified doctor. We aimed to evaluate the impact of COVID-19 on interns’ stress, burnout, emotions, and implications on their training, while exploring their coping mechanisms and resilience levels.

Methods: Newly graduated doctors interning in a Paediatric department in Singapore, who experienced escalation of the pandemic from January to April 2020, were invited to participate. Participants completed the Perceived Stress Scale (PSS), Maslach’s Burnout Inventory (MBI), and Connor Davidson Resilience Scale 25-item (CD-RISC 25) pre-pandemic and 4 months into COVID-19. Group interviews were conducted to supplement the quantitative responses to achieve study aims.

Results: Response rate was 100% (n=10) for post-exposure questionnaires and group interviews. Despite working through the pandemic, interns’ stress levels were not increased, burnout remained low, while resilience remained high. Four themes emerged from the group interviews – the impacts of the pandemic on their psychology, duties, training, as well as protective mechanisms. Their responses, particularly the institutional mechanisms and individual coping strategies, enabled us to understand their unexpected low burnout and high resilience despite the pandemic.

Conclusion: This study demonstrated that it is possible to mitigate stress, burnout and preserve resilience of vulnerable healthcare workers such as interns amidst a pandemic. The study also validated a multifaceted approach that targets institutional, faculty as well as individual levels, can ensure the continued wellbeing of healthcare workers even in challenging times. 

Keywords:            COVID-19, Stress, Burnout, Resilience, Junior Doctor, Intern

Practice Highlights

• Intern doctors face additional and unique challenges in a pandemic, besides the usual stresses of their school-to-work transition.
• Our study shows that a multi-faceted approach that target institution, faculty and individual can lead to reduced burnout and preserved resilience in these doctors.

1) Theme 1 – Psychological Impact (Feelings): Most interns perceived that the pandemic had caused drastic changes in their personal and work lives, with various psychological impacts. They expressed increased emotional exhaustion such as stress and burnout, that is mainly related to changes in their clinical duties (Theme 2). The interns also shared about risks of COVID-19 infection to self and especially to family and loved ones, increasing their worries and stress. Interns followed physical distancing measures and team segregation at work, but several interns avoided their loved ones at home, especially the elderly and immunocompromised. For these interns, they further shared feelings of isolation and loneliness. Positive emotions such as feeling secure, valued and protected existed simultaneously and were mainly associated with the protective measures and support systems (Theme 4) in the workplace. Some also reported that the posting was still enjoyable and felt proud to be working in the pandemic.

2) Theme 2 – Impact on Duties: The interns highlighted there were many changes in institutional work processes and their duties due to the pandemic. Due to manpower changes, there were pervasive reports of physical fatigue. There were however those who felt the workload was still manageable. Interns also raised the issue of non-timely information and unclear protocols which often led to confusion and uncertainty in their work.

3) Theme 3 – Impact on Teaching and Learning: There were mixed comments on this. As a result of strict physical distancing and team segregation, initial planned teaching sessions on general paediatrics were cancelled and the interns felt they “missed out” on their clinical training.  Sessions were subsequently conducted using web-based platforms, which many found helpful. All interns felt that learning was restricted in the pandemic. Although it was beneficial to learn about pandemic response and management of suspected or affected COVID-19 patients, they felt their exposure to general paediatrics was reduced due to the limited variety of ward cases. However, there were some who felt there was better quality of teaching on the ward rounds as consultants had more time to teach with fewer elective and non-urgent cases in the rotations of non-COVID care.

4) Theme 4 – Preventive Measures and Support Systems: Despite the impacts on the interns’ psychology, duties and learning, they also shared on the various protective measures and support systems they perceived helped them cope. This was also the main reason for reported positive feelings of protection and support. Departmental and institutional work processes were implemented to take care of the interns’ physical and psychological welfare such as a rotational system of team segregation, which they reported provided a strict work-rest cycle as well as respite from COVID-care. In addition, seniors and faculty also ensured interns were competent and comfortable dealing with COVID-19 patients prior to taking on high risk duties such as swabbing patients. Support from multiple levels (seniors, department, institution) helped them through. In particular, the seniors and faculty provided support to the interns through regular “check-in” meetings where they could share concerns and provide feedback. The interns also shared that as a result of the strong support received, they were able to develop adaptability, perseverance and resilience, and they were even grateful to be in healthcare at this time.

IV. DISCUSSION

According to the demand-control-support model (Thomas, 2004), occupational stress causes burnout when job demands are high, individual autonomy is low and when job stress interferes with home life (Campbell et al., 2001; Linzer et al., 2001). On that note, we hypothesised that with the COVID-19 pandemic, interns would have increased stress and burnout, in addition to their routine difficulties in the transition from student to doctor. The pandemic-related concerns our interns had were similar to many healthcare workers globally – including the fear of contracting COVID-19 and more so transmitting it to vulnerable loved ones (Chen et al., 2020). Physical fatigue was also seen in our interns given the more intensive work schedule (Sasangohar et al., 2020). Although the total amount of admissions during the period was reduced to 40% of the usual load, the need for team segregation had led to a smaller pool of interns covering each clinical area. In addition, each intern had to do more in-house night calls while on active service. Segregation also meant that there would be less cross-coverage of duties where interns would receive less support from peers who would otherwise have been able to help with the workload on the ground. Another important aspect that had led to reported stress among many was the frequent changes in clinical workflows coupled with the lack of timely and reliable information (Wu et al., 2020). Many interns also highlighted concerns with regards to compromise and interference with their paediatric internship training (Liang et al., 2020). Despite all these, objectively the interns’ perceived stress was maintained without increase in burnout.

Burnout is known to be inversely related to resilience – this pattern is also reflected in our results. Resilience is the process of adapting well in the face of adversity, trauma, tragedy, threats or even significant sources of stress (Southwick et al., 2014). Our interns had high resilience scores, above what has previously been published among physicians (McKinley et al., 2020). One reason for this may be the development of resilience through a time of crisis, a phenomenon well encapsulated by the Crisis Theory: during a crisis or disequilibrium such as the current pandemic, people make attempts to adapt and seek solutions to restore stability. (Brooks et al., 2017; Caplan, 1964). The development of resilience is increasingly emphasised as an integral strategy to combat burnout. Potentially, the mitigating factors, coping mechanisms and support shared by our interns in the interviews, could explain their low burnout and high resilience.

Our interns perceived many systemic measures helped them cope with the pandemic – giving testament to the importance of institutional leadership in implementing safeguards for psychological health (Dewey et al., 2020; Wu et al., 2020). Protocols relating to staff protection, availability of personal protective equipment (Rasmussen et al., 2020) were some of the measures common to institutions worldwide. Furthermore, interns being the most junior member of the team, were spared from doing aerosolising procedures such as intubation, nebulisation administration and airway suctioning that were deferred to clinicians with prior experience and training. This allowed interns time to learn and improve in their competency and confidence prior to assuming these responsibilities. The interns were also thankful for the protected work-rest cycles (Wu et al., 2020), and that they were allowed to take paid leave – which is essential, more so in the pandemic to reduce fatigue and allowed time for rejuvenation.

Other than institutional support, direct support from seniors and faculty were significant in our interns’ responses in helping them, supporting the importance of mentorship (Ramanan et al., 2006). Despite feeling that they might not have reliable and timely access to important updates, they felt supported under the direct guidance of seniors who took the lead on the ground. Regular fortnightly ‘check-in’ sessions were conducted to elicit concerns, obtain feedback, and ensure continual wellbeing. This channel of communication was well received by interns: they appreciated the faculty’s concerns, had the autonomy of being able to input and contribute to the care of patients, the opportunity to air grievances confidentially and importantly, had closure on concerns they have raised regarding their rotations and training (Fischer et al., 2019). The enhanced collegiality between interns, support from seniors and improved cooperation among healthcare workers during this time of crisis naturally also contributed to reduced burnout levels, a finding well established in literature.(Li et al., 2013)

In terms of the impact of training, teaching sessions were initially discontinued to maintain physical distancing. Moreover, the interns had a higher proportion of time spent in the provision of COVID-19 care, which meant traditional general paediatric exposure was compromised. However, within 4 weeks of the pandemic, departmental teaching activities were restored via web-based sessions which interns found useful. The role of faculty in persisting with academic continuity, is again important in mitigating the impact of the pandemic on learning – some interns felt they had more teaching on the wards as consultants had more time to teach for each patient.

We believe that the perceived continual institutional and senior support for our interns allowed them to maintain high personal resilience, that could have mitigated their stress and burnout. In this pandemic, interns demonstrated adaptability and perseverance to the many changes, ability to persevere as well as finding gratitude amidst the challenges and focusing on their goal to help patients and fight the pandemic, which are all known features of resilience (Bird & Pincavage, 2016; Zwack & Schweitzer, 2013).

To our knowledge, this is the first research study in the pandemic that objectively evaluated the impact of the COVID-19 on interns’ psychological state, resilience and training. However, we recognise our study limitations. The small population would mean that it would be difficult to derive statistical comparisons in the pre- and post-exposure results. However, we believe the temporal exposure of the pandemic for this group of interns during their posting, made the pre- and post-pandemic results valid. The results were further supported by qualitative findings from a good group interview participation (100%) and in-depth discussion, that provided substantial explanations to the trend of results. We recognise that 2-4 months might be a short duration for negative psychological effects such as stress, and burnout to set in. Nonetheless, the amount of unprecedented changes and intensity of work for the interns involved within this period, were undoubtedly high. Another study limitation is the inclusion of Paediatric interns only and the possible lower exposure to COVID-19 as compared to their adult counterparts due to decreased disease morbidity and mortality in children. Although this factor could potentially result in less impact on the psychological factors studied, we believe other interns are likely to face similar concerns and challenges in the pandemic, due to their similar backgrounds and job scopes across most departments and disciplines.

This study elucidated the impact of the pandemic on interns in terms of their stress, burnout, as well as clinical duties and training. Despite increasing concerns on the psychological well-being of healthcare workers in the pandemic, our study has demonstrated that it is possible to mitigate their stress, burnout and preserve resilience, even in vulnerable new medical graduates. Our findings objectively validated the importance and effectiveness of the multi-faceted approach that target institution, faculty as well as the individual level, to build resilience and combat burnout in healthcare providers in this pandemic and beyond.

Notes on Contributors

Nicholas BH Ng contributed to conception and design of study, interpretation of data, drafting and critical revising of the article. Mae Yue Tan contributed to analysis and interpretation of data, drafting and critical revising of the article. Shuh Shing Lee contributed to analysis and interpretation of data, drafting and critical revising of the article. Nasyitah bte Abdul Aziz contributed to analysis and interpretation of data, drafting of the article. Marion M Aw contributed to interpretation of data, drafting and critical revising of the article. Jeremy BY Lin contributed to conception and design, interpretation of data, drafting and critical revising of the article. All authors gave final approval of the version to be published.

Data Availability

The data for this study can be found at https://doi.org/10.6084/m9.figshare.12924029.v1. The access to these datasets are available for use subject to approval of the authors of this article.

Ethical Approval

Ethics approval was obtained from the NHG Domain Specific Review Board (DSRB), with NHG DSRB reference number of 2020/00392.

Acknowledgement

The authors would like to thank the interns who participated in this study. 

Funding

Funding for this study was obtained from NUHS Fund Limited – Medical Affairs (Education) Fund.

Declaration of Interest

All authors have no conflicts of interest to declare.

References

Bird, A., & Pincavage, A. (2016). A curriculum to foster resident resilience. MedEdPORTAL, 12, 10439. https://doi.org/10.15766/mep_2374-8265.10439

Brooks, S. K., Dunn, R., Amlôt, R., Rubin, G. J., & Greenberg, N. (2017). Social and occupational factors associated with psychological wellbeing among occupational groups affected by disaster: A systematic review. Journal of Mental Health, 26(4), 373-384. https://doi.org/10.1080/09638237.2017.1294732

Campbell, D. A., Jr., Sonnad, S. S., Eckhauser, F. E., Campbell, K. K., & Greenfield, L. J. (2001). Burnout among American surgeons. Surgery, 130(4), 696-702; discussion 702-695. https://doi.org/10.1067/msy.2001.116676

Caplan, G. (1964). Principles of preventive psychiatry. Basic Books.

Chen, Q., Liang, M., Li, Y., Guo, J., Fei, D., Wang, L., He, L., Sheng, C., Cai, Y., Li, X., Wang, J., & Zhang, Z. (2020). Mental health care for medical staff in China during the COVID-19 outbreak. Lancet Psychiatry, 7(4), e15-e16. https://doi.org/10.1016/s2215-0366(20)30078-x

Chia, R., & Moynihan, Q. (2020, February 20). This alarming map shows where the coronavirus has spread in Singapore, one of the worst-hit areas outside of China Business Insider Singapore. Business Insider. https://www.businessinsider.com/coronavirus-singapore-map-shows-spread-worst-hit-outside-china-2020-2?IR=T.

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behaviour, 24(4), 385-396.

Connor, K. M., & Davidson, J. R. (2003). Development of a new resilience scale: The Connor-Davidson resilience scale (CD-RISC). Depression and Anxiety, 18(2), 76-82. https://doi.org/10.1002/da.10113

Dewey, C., Hingle, S., Goelz, E., & Linzer, M. (2020). Supporting clinicians during the COVID-19 pandemic. Annals of Internal Medicine, 172(11), 752-753. https://doi.org/10.7326/M20-1033

Fischer, J., Alpert, A., & Rao, P. (2019). Promoting intern resilience: Individual chief wellness check-ins. MedEdPORTAL, 15, 10848. https://doi.org/10.15766/mep_2374-8265.10848

Li, B., Bruyneel, L., Sermeus, W., Van den Heede, K., Matawie, K., Aiken, L., & Lesaffre, E. (2013). Group-level impact of work environment dimensions on burnout experiences among nurses: A multivariate multilevel probit model. International Journal of Nursing Studies, 50(2), 281–291. https://doi.org/10.1016/j.ijnurstu.2012.07.001

Liang, Z. C., Ooi, S. B. S., & Wang, W. (2020). Pandemics and their impact on medical training: Lessons From Singapore. Academic Medicine. https://doi.org/10.1097/acm.0000000000003441

Linzer, M., Visser, M. R., Oort, F. J., Smets, E. M., McMurray, J. E., & de Haes, H. C. (2001). Predicting and preventing physician burnout: results from the United States and the Netherlands. The American Journal of Medicine, 111(2), 170-175. https://doi.org/10.1016/s0002-9343(01)00814-2

Low, Z. X., Yeo, K. A., Sharma, V. K., Leung, G. K., McIntyre, R. S., Guerrero, A., Lu, B., Lam, C. C. S. F., Tran, B. X., Nguyen, L. H., Ho, C. S., Tam, W. W., & Ho, R. C. (2019). Prevalence of burnout in medical and surgical residents: A meta-analysis. International Journal of Environmental Research and Public Health, 16(9). https://doi.org/10.3390/ijerph16091479

Maslach, C. J. S., & Leiter, M. P. (2016). Maslach burnout inventory manual. Mind Garden Inc.

McKinley, N., McCain, R. S., Convie, L., Clarke, M., Dempster, M., Campbell, W. J., & Kirk, S. J. (2020). Resilience, burnout and coping mechanisms in UK doctors: A cross-sectional study. British Medical Journal Open, 10(1), e031765. https://doi.org/10.1136/bmjopen-2019-031765

Ministry of Health (MOH), Singapore. (2020). Circuit breaker to minimise further spread of COVID-19. https://www.moh.gov.sg/news-highlights/details/circuit-breaker-to-minimise-further-spread-of-covid-19. (Retrieved April 3, 2020)

Ng, N. B. H (2020). The COVID-19 Pandemic: Impact on Paediatric Postgraduate Year One Doctors [Data set]. Figshare. https://figshare.com/s/74c81ca193638a553ea2

Ramanan, R. A., Taylor, W. C., Davis, R. B., & Phillips, R. S. (2006). Mentoring matters. Mentoring and career preparation in internal medicine residency training. Journal of General Internal Medicine, 21(4), 340-345. https://doi.org/10.1111/j.1525-1497.2006.00346.x

Rasmussen, S., Sperling, P., Poulsen, M. S., Emmersen, J., & Andersen, S. (2020). Medical students for health-care staff shortages during the COVID-19 pandemic. The Lancet, 395(10234), e79-e80. https://doi.org/10.1016/s0140-6736(20)30923-5

Rotenstein, L. S., Torre, M., Ramos, M. A., Rosales, R. C., Guille, C., Sen, S., & Mata, D. A. (2018). Prevalence of burnout among physicians: A systematic review. The Journal of the American Medical Association, 320(11), 1131-1150. https://doi.org/10.1001/jama.2018.12777

Sasangohar, F., Jones, S. L., Masud, F. N., Vahidy, F. S., & Kash, B. A. (2020). Provider burnout and fatigue during the COVID-19 pandemic: Lessons learned from a high-volume intensive care unit. Anesthesia and Analgesia, 131(1), 106–111. https://doi.org/10.1213/ane.0000000000004866

Southwick, S. M., Bonanno, G. A., Masten, A. S., Panter-Brick, C., & Yehuda, R. (2014). Resilience definitions, theory, and challenges: Interdisciplinary perspectives. European Journal of Psychotraumatology, 5,(1), 25338. https://doi.org/10.3402/ejpt.v5.25338

Sturman, N., Tan, Z., & Turner, J. (2017). “A steep learning curve”: Junior doctor perspectives on the transition from medical student to the health-care workplace. BMC Medical Education, 17(1), 92. https://doi.org/10.1186/s12909-017-0931-2

Thomas, N. K. (2004). Resident burnout. The Journal of the American Medical Association, 292(23), 2880-2889. https://doi.org/10.1001/jama.292.23.2880

Wu, P. E., Styra, R., & Gold, W. L. (2020). Mitigating the psychological effects of COVID-19 on health care workers. Canadian Medical Association Journal, 192(17), E459-e460. https://doi.org/10.1503/cmaj.200519

Zwack, J., & Schweitzer, J. (2013). If every fifth physician is affected by burnout, what about the other four? Resilience strategies of experienced physicians. Academic Medicine, 88(3), 382-389. https://doi.org/10.1097/ACM.0b013e318281696b

*Jeremy Bingyuan Lin
1E Kent Ridge Road,
NUHS Tower Block Level 12,
Singapore 119228
Tel: (65) 6772 4847
Email: jeremy_lin@nuhs.edu.sg

Submitted: 28 July 2020
Accepted: 18 November 2020
Published online: 4 May, TAPS 2021, 6(2), 48-56
https://doi.org/10.29060/TAPS.2021-6-2/OA2367

Oscar Gilang Purnajati¹, Rachmadya Nur Hidayah² & Gandes Retno Rahayu²

¹Faculty of Medicine, Universitas Kristen Duta Wacana, Yogyakarta, Indonesia; ²Department of Medical Education, Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia

Abstract

Introduction: Objective Structured Clinical Examination (OSCE) examiners come from various backgrounds. This background variability may affect the way they score examinees. This study aimed to understand the effect of background variability influencing the examiners’ score agreement in OSCE’s procedural skill.

Methods: A mixed-methods study was conducted with explanatory sequential design. OSCE examiners (n=64) in the Faculty of Medicine Universitas Kristen Duta Wacana (FoM-UKDW) took part to assess two videos of Cardio-Pulmonary Resuscitation (CPR) competence to get their level of agreement by using Fleiss Kappa. One video portrayed CPR according to performance guideline, and the other portrayed CPR not according to performance guidelines. Primary survey, CPR procedure, and professional behaviour were assessed. To confirm the assessment results qualitatively, in-depth interviews were also conducted.

Results: Fifty-one examiners (79.7%) completed the assessment forms. From 18 background categories, there was a good agreement (>60%) in: Primary survey (4 groups), CPR procedure (15 groups), and professional behaviour (7 groups). In-depth interviews revealed several personal factors involved in scoring decisions: 1) Examiners use different references in assessing the skills; 2) Examiners use different ways in weighting competence; 3) The first impression might affect the examiners’ decision; and 4) Clinical practice experience drives examiners to establish a personal standard.

Conclusion: This study identifies several factors of examiner background that allow better agreement of procedural section (CPR procedure) with specific assessment guidelines. We should address personal factors affecting scoring decisions found in this study in preparing faculty members as OSCE examiners.

Keywords:            OSCE Score, Background Variability, Agreement, Personal Factor

Practice Highlights

• The examiners’ background variability influences the OSCE scoring agreement results.
• The reason for assessment inaccuracy remains unclear regarding the score agreement.
• The absence of assessment instruments that could provide a loophole for examiners to improvise.
• Personal factors affecting scoring decisions found in this study should be addressed in preparing OSCE examiners.

I. INTRODUCTION

To assess medical students’ competencies in a variety of skills, most medical schools in Indonesia implement the Objective Structured Clinical Examination (OSCE) both as a clinical skills examination at the undergraduate stage and as a national exit exam (Rahayu et al., 2016; Suhoyo et al., 2016). Most OSCE stations test both communication domains and specific clinical skills that will be assessed based on rubrics and scoring checklists which relies on examiners’ observations (Setyonugroho et al., 2015).  The OSCE has a challenge in its complexity to standardise  the scores, which are very depend on OSCE examiners’ perceptions (Pell et al., 2010). In a well-designed OSCE the examinees performance should only influence the examinees’ score, with minimal effects from other sources of variance (Khan et al., 2013). Research showed that there are influences of examiner’s background variability on OSCE results although they have been asked to standardise their behaviour (Pell et al., 2010) The decision and behaviour of OSCE examiners will affect the quality of assessment, including making a pass or fail decision, considering the complexity of knowledge, skill, and attitude in medical education (Colbert-Getz et al., 2017; Fuller et al., 2017).  

Examiners’ observations also rely on their clinical practice experience, OSCE examining experience, and gender conformity (Mortsiefer et al., 2017). Even in OSCE that is held in the most standard conditions, the examiner factor has the biggest role in scoring inaccurately (Mortsiefer et al., 2017). However, the reason for this inaccuracy remains unclear since there are concerns regarding the scoring agreement of examiners in OSCE and how the result might be affected by this issue. There is a need to consider the influence of examiners’ background variability (gender, educational level, clinical practice experiences, length of clinical practice experiences, OSCE experience, and OSCE training experience) when preparing teachers as OSCE examiners. This study aimed to understand background variability as a factor influencing examiners’ scoring agreement in assessing students’ performance in procedural skill, as the first step of faculty development program to ensure the standard quality for examiners.

II. METHODS

A. Study Design

This mixed-method study used a sequential explanatory design. This mixed-method approach is expected to provide more comprehensive results and better understanding than using a separated method (Creswell & Clark, 2018).

This study comprised of 2 sequential phases of data collection and analysis (QUANTITATIVE: qualitative) using sequential design. First, quantitative data were collected as a cross-sectional study of the examiners’ strength of agreement using Fleiss Kappa while assessing the clinical skill performance recorded in the 2 videos: one video portrayed CPR according to performance guideline and the other portrayed CPR not according to performance guideline. We used these 2 videos in order to portray more comprehensively how the consistency of OSCE examiner agreement both on good and poor clinical skill performance.

In the second phase, in-depth interviews were used to complement the quantitative results to gain more information and a detailed confirmation about how the scores were decided (Stalmeijer et al., 2014). In this stage of study, researchers explored and explained the examiners’ OSCE experiences and behaviour when they give a score on a clinical skill examination and the influences on their scoring regarding their backgrounds.

B. Materials and/or Subjects

The strength of agreement of the videos’ score came from 64 OSCE examiners FoM UKDW. Mortsiefer et al., (2017), explained that more subjects are better when investigate examiner characteristics associated with inter-examiner reliability (Mortsiefer et al., 2017).  In the second phase, in-depth interviews were conducted with 6 examiners of FoM UKDW, selected by purposive sampling regarding their scores and how they represented their own unique background (Table 1).

Researcher (OGP) provided all the participants with written information about this research and addressed ethical issues in an informed consent form. Researcher ensured participants understand the research protocol and clarified any questions regarding this study. Participants who agreed to take part, sign the informed consent form prior to the data collection. 

We held interviews in FoM UKDW with maximum 30 minutes of duration each interview. The inclusion criteria for examiners who were selected for this study were involved as full-time faculty members, had over 4 times OSCE examination experience, and had done OSCE examiner training, expecting that they had enough interaction with other faculty members and had influences from medical doctor education (Park et al., 2015). The exclusion criteria were participant did not answer the research invitation and did not fill the assessment form completely. Main researcher (OGP) conducted the interview. Main researcher was a male, student of Master of Health Profession Education Universitas Gadjah Mada, and the staff of FoM UKDW.

C. Statistics

1)Quantitative data analysis: We grouped examiners into 18 groups based on their background which were gender, educational level, clinical practice experiences, length of clinical practice experiences, OSCE experience, and OSCE training experience as shown in Table 1. We analysed all gathered data using IBM SPSS Statistics 25 and Microsoft Office Excel 365 (IBM Corp., Chicago). We presented quantitative data as a strength of agreement in percentage. The strength of agreement was calculated using Fleiss Kappa to determine the agreement between each group of each examiner background on whether CPR performances (primary survey, CPR Procedure, and professional behaviour), that portrayed in those 2 videos, were exhibiting score either “0”, “1”, “2”, or “3” based on the assessment guideline and rubric’s criteria (Purnajati, 2020). Based on recent research, agreement above 60% was considered as a substantial and adequate agreement (Stoyan et al., 2017; Vanbelle, 2019).

2) Qualitative data analysis: In-depth interviews were analysed using thematic analysis. We prepared a structured list of questions. It consisted of one key question: What was your experience in scoring the OSCE? The other additional questions evaluated the experiences of examiners in OSCE scoring including: the use of other references, differences in assessment weighting, use of own decision, clinical practice experience affecting the decision, and gender related decision making. Next, the collected data resulting from in-depth interviews were recorded using audio file recorder, read, and categorised into themes whenever they were related. The transcripts and identified themes were then given to an external coder in this study. This step was followed by our agreement for each theme. There was no repeated interview.

III. RESULTS

A. Quantitative Data Result

We deposited both quantitative and qualitative data in an online repository  (Purnajati, 2020).  The study participants in this quantitative phase were 64 OSCE examiners who are full-time faculty members. Twelve participants were excluded because did not fulfil the inclusion criteria. Fifty-one (79.7%) examiners who returned the completed assessment form are described below in Table 1.

The assessment rubric was divided into three main competencies: (1) primary survey, (2) CPR procedure, and (3) professional behaviour. The results showed overall agreement on each main competency based on each examiners’ background variability by using Fleiss Kappa. The percentage of agreement is shown in Figure 2, 3, and 4.

Figure 2. Primary Survey percentage of overall agreement (n = 51). Agreement above 60% (*) is considered as a substantial and adequate agreement

Figure 3. CPR Procedure percentage of overall agreement (n=51). Agreement above 60% (*) is considered as a substantial and adequate agreement

After completing the CPR competency assessment, all examiners’ background characteristics met a cutoff of approval above 60% in assessing CPR procedure except for examiners with clinical practice experience <3 years, OSCE testing experience <2 years, and OSCE examiner training> 5 years (Figure 3). This finding showed a good strength of agreement in assessing CPR procedure regardless of examiners’ background. However, there were many instances where the cut-off point of 60% was not achieved in the aspects of primary surveys and professional behaviour (Figure 2 and 4), which showed fair strength of agreement between examiners when they examined these competencies.

B. Qualitative Data Results

Two theme categories were determined: (1) OSCE experience and (2) specific behaviour in OSCE. The first theme contains of 3 sub-themes: (1) student performance, (2) examiner background effect, and (3) using assessment instrument. The second theme consists of 5 sub-themes: (1) use of assessment references, (2) score weighting, (3) personal inferences, (4) clinical experience, and (5) gender conformity.

Theme 1: Examiners argued that they understand the difference in student performance in performing clinical skills and can distinguish from the coherent skills performed by students according to checklist.

“Very easy in giving an assessment, because everything is in accordance with the assessment rubric”

                                                                                      (ID 35)

“The plot is clear, well organised”                      

      (ID 26)

“You can compare the inadequacies; it is enough to be compared”

                                                                                      (ID 11)

“The 2 different students are quite striking, so in my opinion it is not too difficult”

      (ID 28)

Nevertheless, some examiners had difficulty to distinguish student performance when only used a checklist. Examiner background did not affect their way in scoring clinical skills performance, but some background may have the potential to affect their scoring, such as clinical practice experience.

“I am trying to avoid personal interpretations, as much as possible, but of course that cannot be 100 percent. In my opinion, the assessment rubric still gives room for subjectivity”

                                                                                     (ID 28)

In this research, it seemed easy for examiners to understand the assessment instrument when giving score to those 2 videos and their understanding were good.

Theme 2: Interviews revealed that: 1) Examiners use other references such as their clinical experience in assessing the skills;

“If the assessment guideline is unclear, the students are also unclear, yes I will improvise. Or when the assessment guideline is clear and the students are unclear which criteria are included, yes I will improvise”

     (ID 35)

“Maybe yes, because once again the template at the beginning is not very clear”

 (ID 23)

2) Examiners use different ways in giving weight of competence, for example, procedural steps are considered more important than primary survey;

“For those that I feel have a small weight because the instructions are also short, so I don’t have to look carefully”

 (ID 24)

“When I feel that competence is not important, it does not get my emphasis, the more emergency that will get more attention.”

 (ID 28)

3) The first impression of examinees might affect their decision in scoring their performance;

“That first impression will affect me in giving value. I will be more critical. I see more, pay more attention to the small things they do”

(ID 24)

4) Clinical practice experience drives examiners to establish a personal standard on how a doctor should be;

“Clinical experience when practice is one of the judgments”

(ID 24)

“The reference is just my instinct because it has been running as a doctor after all these years. Yes, I use my previous knowledge”

(ID 26)

And 5) Gender of examinees does not affect their decision, while their professionalism (e.g. showing respect to patients) will surely affect their decision.

“I pay more attention especially to politeness and professional behaviour”

(ID 24)

“Students of any gender still have the same standard of evaluation, a score of professionalism which is more influential”

(ID 23)

 IV. DISCUSSION

Examiners’ agreement in this study was high in assessing the CPR procedure, which has a fixed and specific procedure in almost all groups of examiners. These results are consistent and can be explained by results from previous studies, which show that assessment with specific cases will provide high inter-examiner agreement (Erdogan et al., 2016). The differences in the examiner’s background will not have much influence on their agreement in giving an assessment in a specific case. This was supported by the opinions of examiners in the in-depth interviews who stated that in the CPR assessment procedure, assessment instruments are clear, easy to understand, with clear procedure flow, and performance that is easily distinguished, which made it easier for examiners to be able to distinguish student performance. A specific assessment instrument that could not provide a loophole for examiners to improvise assessment, made the opportunity for examiners to portray their subjectivity was minimised. This simplicity could lead to high agreement among examiners in specific competencies as shown in this study and based on clear evidence can increase the reliability of the assessment (Daniels et al., 2014) .

In this study, it was found in the primary survey assessment and professional behaviour which has an assessment guide that is not as specific as the CPR procedure, the percentage of agreement between examiner groups was lower, with only a few of them reaching 60% of agreement. This difference happened for reasons confirmed in the in-depth interviews which raised the issue that although the examiners tried to minimise their subjectivity in assessing, but it was said that there were still gaps in the assessment guide that still gives room for subjectivity. There are also examiners who were dissatisfied with the checklist, so they used their personal decisions in evaluating students.

According to a recent study, this could be due to the lack of specific instructions in the general assessment guidelines which will result in lower inter-examiner reliability compared to the use of more specific assessment guidelines (Mortsiefer et al., 2017). In the primary survey section and professional behaviour, there were also aspects of communication that were judged to be more susceptible to bias than physical examination skills because physical examination is more well-documented, clear instructions, and more widely accepted by examiners (Chong et al., 2018) The validity and reliability of a clinical skills assessment depend on factors including how the student’s performance on the exam, the character of the population, the environment, and even the assessment instrument itself can affect how examiners carry out the assessment (Brink & Louw, 2012). These phenomena were seen in the in-depth interviews which revealed that there were certain moments namely when the student being tested does not match the expectations written in the assessment guide and when the assessment guide is not clear so that it still gives room for subjectivity examiner. In addition, in the in-depth interviews the results also revealed that the examiners differentiated their attention on certain competencies with certain criteria such as the length of information in the assessment rubric, so that competencies that were considered not important did not get as much attention.

This finding may be in line with previous research which stated that constructs and conceptual definitions in this category that still provide a gap in the subjectivity of examiners cause shifting attention focus and weighting of their judgments to be different so that there are differences in important aspects between examiners (Schierenbeck & Murphy, 2018; Yeates et al., 2013). The difference in these important aspects can bring examiners to reorganise competency weights so that simpler and easier competencies (in this case those that have clearer and more detailed assessment guidelines) will be done first, and more complex ones (in this case, guides that have lower rigidity ratings) will be assessed later with the possibility of using more narratives (Chahine et al., 2015). This reorganisation can reflect how the examiners’ decision, allowing them to direct their attention to the more important aspects as the testers revealed in in-depth interviews with this research.

The personal factor, such as assessment references is a potential variability of the assessment conducted by the examiner. Examiners are trained and understand the use of assessment instruments, but produce varying assessments because they do not apply assessment criteria appropriately, but use personal best practice, use other test participants better as benchmarks, use patient outcomes (e.g. correct diagnosis, do patients understand, etc.), and use themselves as a comparison (Gingerich et al., 2014; Kogan et al., 2011; Yeates et al., 2013).

Another personal factors, including first impressions, can occur spontaneously unconsciously and can be a source of difference in judgment between examiners (Gingerich et al., 2011). First impressions based on observers’ observations have the same decisions and influences as social interactions, so it makes sense that first impressions are able to influence judgments, can be accurate and have a relationship with the final assessment results, but do not occur in examiners in general (Wood, 2014; Wood et al., 2017).

In providing assessments, there are gaps for examiners to give different competency weights to other examiners. Providing assessments based on targets that differ from competency standards and comparisons with the performance of other examinees will make the examiners recalibrate their own weighting and this is an explanation why there are variations in assessment and differences in the important points of the examinees’ performance among examiners (Gingerich et al., 2018; Yeates et al., 2015; Yeates et al., 2013).

The variability of personal factors between examiners can be conceptualise more as a different emphasis on building doctor-patient relationships and / or certain medical expertise rather than variations in the examiner’s background itself. The examiners’ own understanding can be conceptualized as a combination of whether what the examinees do is good enough and whether what they do is enough to build a doctor-patient relationship.

This research had some limitations such as it only used specific cases (i.e., CPR) to minimise the bias of the assessment instrument so that it would reveal more bias in the examiners themselves. In more complicated cases such as communication skills and clinical reasoning it is also necessary to provide a more complete picture of how the examiners’ scores agree in other cases. Generalization also became a limitation in this study because it only involved examiners from one medical education institution, however the study participants sufficiently described the variability of the examiner’s background.

V. CONCLUSION

This study identifies several factors of examiner background variability that influence examiners’ judgment in terms of inter-examiner agreement. Female examiners, bachelor education, less OSCE experience, and non-clinician examiners allow better agreement of procedural section (CPR procedure) with specific assessment guidelines. Cases that have unspecified assessment guidelines in this research, primary survey and professional behaviour, have lower agreement among examiners and must be examined deeper. We should note that personal factors of OSCE examiners can influence assessment discrepancies. However, the reasons for using these personal factors in scoring OSCE performance might be affected by unknown biases that require further research. Therefore, to improve clinical skills assessment such as OSCE for undergraduate medical programme, we must address personal factors affecting scoring decisions found in this study in preparing faculty members as OSCE examiners.

Notes on Contributors

Oscar Gilang Purnajati, MD was student of Master of Health Professions Education Study Program, Faculty of Medicine, Universitas Gadjah Mada, Indonesia. He concepted the research, reviewed the literature, designed the study, acquisited funding, conducted interviews, analysed quantitative data and transcripts, and wrote the manuscript.

Rachmadya Nur Hidayah, MD., M.Sc., Ph.D is lecturer of Department of Medical Education, Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia. She supervised author Oscar Gilang Purnajati, developed the concepted framework for the study, critically analysed the data, cured the data, and reviewed the final manuscript.

Prof. Gandes Retno Rahayu, MD., M.Med.Ed, Ph.D is professor at the Department of Medical Education, Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia. She supervised author Oscar Gilang Purnajati, advised the design of the study, critically analysed the data, gave critical feedback to the conducted interviews, reviewed the final manuscript.

All the authors have read and approved the final manuscript.

Ethical Approval

This study was approved by Health Research Ethics Committee Faculty of Medicine Universitas Kristen Duta Wacana (Reference No.1068/C.16/FK/2019).

Data Availability

All data were deposited in an online repository. The data is available at Open Science Framework with DOI: https://doi.org/10.17605/OSF.IO/RDP65

Acknowledgements

The author would like to thank Hikmawati Nurrokhmanti, MD, M.Sc for helping with the process of coding the in-depth interview transcripts. The author also would like to thank the staffs of Faculty of Medicine, Universitas Kristen Duta Wacana for supporting the research.

Funding Statement

This work was supported by the Universitas Kristen Duta Wacana (No. 075/B.03/UKDW/2018) as a part of study scholarship.

Declaration of Interest

No potential conflict of interest relevant to this article was reported.

Abbreviations and specific symbols

OSCE: Objective Structured Clinical Examination.

References

Brink, Y., & Louw, Q. A. (2012). Clinical instruments: Reliability and validity critical appraisal. Journal of Evaluation in Clinical Practice,18(6), 1126-1132. https://doi.org/10.1111/j.1365-2753.2011.01707.x

Chahine, S., Holmes, B., & Kowalewski, Z. (2015). In the minds of OSCE examiners: Uncovering hidden assumptions. Advances in Health Sciences Education : Theory and Practice, 21(3), 609–625. https://doi.org/10.1007/s10459-015-9655-4

Chong, L., Taylor, S., Haywood, M., Adelstein, B.-A., & Shulruf, B. (2018). Examiner seniority and experience are associated with bias when scoring communication, but not examination, skills in objective structured clinical examinations in Australia. Journal of Educational Evaluation for Health Professions, 15(17). https://doi.org/10.3352/jeehp.2018.15.17

Colbert-Getz, J. M., Ryan, M., Hennessey, E., Lindeman, B., Pitts, B., Rutherford, K. A., Schwengel, D., Sozio, S. M., George, J., & Jung, J. (2017). Measuring assessment quality with an assessment utility rubric for medical education. MedEdPORTAL : The Journal of Teaching and Learning Resources, 13, 1-5. https://doi.org/10.15766/mep_2374-8265.10588

Creswell, J. W., & Clark, V. L. P. (2018). Designing and conducting mixed method research. SAGE Publications, Inc.

Daniels, V. J., Bordage, G., Gierl, M. J., & Yudkowsky, R. (2014). Effect of clinically discriminating, evidence-based checklist items on the reliability of scores from an internal medicine residency OSCE. Advances in Health Sciences Education : Theory and Practice, 19(4), 497-506. https://doi.org/10.1007/s10459-013-9482-4

Erdogan, A., Dong, Y., Chen, X., Schmickl, C., Berrios, R. A. S., Arguello, L. Y. G., Kashyap, R., Kilickaya, O., Pickering, B., Gajic, O., & O’Horo, J. C. (2016). Development and validation of clinical performance assessment in simulated medical emergencies: An observational study. BMC Emergency Medicine, 16, 4. https://doi.org/10.1186/s12873-015-0066-x

Fuller, R., Homer, M., Pell, G., & Hallam, J. (2017). Managing extremes of assessor judgment within the OSCE. Medical Teacher, 39(1), 58-66. https://doi.org/ 10.1080/0142159X.2016.1230189

Gingerich, A., Kogan, J., Yeates, P., Govaerts, M., & Holmboe, E. (2014). Seeing the ‘black box’ differently: Assessor cognition from three research perspectives. Medical Education, 48(11), 1055–1068. https://doi.org /10.1111/medu.12546

Gingerich, A., Regehr, G., & Eva, K. W. (2011). Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine : Journal of the Association of American Medical Colleges, 86(10), S1-S7. https://doi.org/10.1097/ACM.0b013e31822a6cf8

Gingerich, A., Schokking, E., & Yeates, P. (2018). Comparatively salient: Examining the influence of preceding performances on assessors’ focus and interpretations in written assessment comments. Advances in Health Sciences Education: Theory and Practice, 23(5), 937-959. https://doi.org /10.1007/s10459-018-9841-2

Khan, K. Z., Ramachandran, S., Gaunt, K., & Pushkar, P. (2013). The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Medical Teacher, 35(9), 1437-1446. https://doi.org/ 10.3109/0142159X.2013.818634

Kogan, J. R., Conforti, L., Bernabeo, E., Iobst, W., & Holmboe, E. (2011). Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education, 45(10), 1048-1060. https://doi.org/10.1111/j.1365-2923.2011.04025.x

Mortsiefer, A., Karger, A., Rotthoff, T., Raski, B., & Pentzek, M. (2017). Examiner characteristics and interrater reliability in a communication OSCE. Patient Education and Counseling, 100(6), 1230-1234. https://doi.org/ 10.1016/j.pec.2017.01.013

Park, S. E., Kim, A., Kristiansen, J., & Karimbux, N. Y. (2015). The Influence of Examiner Type on Dental Students’ OSCE Scores. Journal of Dental Education, 79(1), 89-94.

Pell, G., Fuller, R., Homer, M., & Roberts, T. (2010). How to measure the quality of the OSCE: A review of metrics – AMEE guide no. 49. Medical Teacher, 32(10), 802-811. https://doi.org/ 10.3109/0142159X.2010.507716

Purnajati, O. G. (2020). Does objective structured clinical examination examiners’ backgrounds influence the score agreement? [Data set]. Open Science Framework. https://doi.org/ 10.17605/OSF.IO/RDP65

Rahayu, G. R., Suhoyo, Y., Nurhidayah, R., Hasdianda, M. A., Dewi, S. P., Chaniago, Y., Wikaningrum, R., Hariyanto, T., Wonodirekso, S., & Achmad, T. (2016). Large-scale multi-site OSCEs for national competency examination of medical doctors in Indonesia. Medical Teacher, 38(8), 801-807. https://doi.org /10.3109/0142159X.2015.1078890

Schierenbeck, M. W., & Murphy, J. A. (2018). Interrater reliability and usability of a nurse anesthesia clinical evaluation instrument. Journal of Nursing Education, 57(7), 446-449. https://doi.org/10.3928/01484834-20180618-12

Setyonugroho, W., Kennedy, K. M., & Kropmans, T. J. B. (2015). Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review. Patient Education and Counseling, 98(12), 1482-1491. https://doi.org/ 10.1016/j.pec.2015.06.004

Stalmeijer, R. E., McNaughton, N., & Van Mook, W. N. (2014). Using focus groups in medical education research: AMEE Guide No. 91. Medical Teacher, 36(11), 923-939. https://doi.org/ 10.3109/0142159X.2014.917165

Stoyan, D., Pommerening, A., Hummel, M., & Kopp-Schneider, A. (2017). Multiple-rater kappas for binary data: Models and interpretation. Biometrical Journal, 60(5), 381-394. https://doi.org/ 10.1002/bimj.201600267

Suhoyo, Y., Rahayu, G. R., & Cahyani, N. (2016). A national collaboration to improve OSCE delivery. Medical Education, 50(11), 1150–1151. https://doi.org/ 10.1111/medu.13189

Vanbelle, S. (2019). Asymptotic variability of (multilevel) multirater kappa coefficients. Statistical Methods in Medical Research, 28(10-11), 3012-3026. https://doi.org /10.1177/0962280218794733

Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education : Theory and Practice, 19(3), 409-427. https://doi.org/ 10.1007/s10459-013-9453-9

Wood, T. J., Chan, J., Humphrey-Murto, S., Pugh, D., & Touchie, C. (2017). The influence of first impressions on subsequent ratings within an OSCE station. Advances in Health Sciences Education : Theory and Practice, 22(4), 969-983. https://doi.org/10.1007/s10459-016-9736-z  

Yeates, P., Moreau, M., & Eva, K. (2015). Are examiners’ judgments in osce-style assessments influenced by contrast effects? Academic Medicine : Journal of the Association of American Medical Colleges, 90(7), 975-980. https://doi.org /10.1097/ACM.0000000000000650

Yeates, P., O’Neill, P., Mann, K., & Eva, K. (2013). Seeing the same thing differently: Mechanisms that contribute to assessor differences in directly-observed performance assessments. Advances in Health Sciences Education : Theory and Practice, 18(3), 325-341. https://doi.org/10.1007/s10459-012-9372-1

*Oscar Gilang Purnajati
Faculty of Medicine, 
Universitas Kristen Duta Wacana,
Jl. Dr. Wahidin Sudirohusodo No. 5-25.
Yogyakarta City,
Special Region of Yogyakarta
55224, Indonesia.55224, Indonesia.
Tel: +62-274-563929
Email: oscargilang@staff.ukdw.ac.id

Submitted: 8 July 2020
Accepted: 23 October 2020
Published online: 4 May, TAPS 2021, 6(2), 38-47
https://doi.org/10.29060/TAPS.2021-6-2/OA2338

Enjy Abouzeid¹, Rebecca O’Rourke², Yasser El-Wazir¹, Nahla Hassan¹, Rabab Abdel Ra’oof¹ & Trudie Roberts²

¹Faculty of Medicine, Ismailia, Egypt; ²LIME, University of Leeds, United Kingdom

Abstract

Introduction: Although, several factors have been identified as significant determinants in online learning, the human interactions with those factors and their effect on academic achievement are not fully elucidated. This study aims to determine the effect of self-regulated learning (SRL) on achievement in online learning through exploring the relations and interaction of the conception of learning, online discussion, and the e-learning experience.

Methods: A non-probability convenience sample of 128 learners in the Health Professions Education program through online learning filled-out three self-reported questionnaires to assess SRL strategies, the conception of learning, the quality of e-Learning experience and online discussion. A scoring rubric was used to assess the online discussion contributions. A path analysis model was developed to examine the effect of self-regulated learning on achievement in online learning through exploring the relations and interaction among the other factors.

Results: Path analysis showed that SRL has a statistically significant relationship with the quality of e-learning experience, and the conception of learning. On the other hand, there was no correlation with academic achievement and online discussion. However, academic achievement did show a correlation with online discussion.

Conclusion: The study showed a dynamic interaction between the students’ beliefs and the surrounding environment that can significantly and directly affect their behaviour in online learning. Moreover, online discussion is an essential activity in online learning.

Keywords:            Online Learning, Conception of Learning, E-learning Experience, Human-Computer Interface, Self-regulated Learning, Path Analysis

Practice Highlights

• The learner who views learning as a constructive process will show better use of self-regulated learning strategies.
• Learners’ beliefs and perceptions can shape the learning experience.
• Online discussion can directly and significantly affect academic achievement in online learning.
• Self-regulated learning is responsible for a small portion of the change in academic achievement.
• Online discussion may affect self-regulated learning negatively.

In just a few years, online e-learning has become part of the mainstream in medical education for postgraduates in both developed and developing countries. The use of online e-learning may provide solutions for many educational problems, especially for health professions graduates. It can help them achieving their developmental and educational goals despite the lack of time and overburdened schedules. This raised the need for better understanding of learning in online learning context.

The training that most schools offer to students and instructors on online leaning is mainly limited to using technologies that allow learners to interact with instructors and other learners effectively and flexibly. However, learners in online learning are facing several and complex challenges due to the nature of this context. Online learning is a form of distance learning that represent not only the access to learning experience via the use of technology and internet but also it relies on connectivity, flexibility and ability to promote varied interactions (Hiltz & Turoff, 2005). It characterised by autonomy and relative isolation due to the lack of face-to-face support. One of these important challenges is the need for self-regulated skills. It has been reported that these skills are more important in online learning as compared to traditional one (Azevedo et al., 2008).

Self-regulation is defined as the degree to which students are metacognitively, motivationally, and behaviourally active participants in their learning process (Zimmerman, 1986). This definition focused on students’ proactive use of specific behaviours to improve their academic achievement. In short, the ability to regulate one’s learning process is a critical skill to achieve personal learning objectives in online courses due to the absence of the support and guidance that is typically available in face-to-face learning environments (e.g., an instructor setting deadlines and structuring the learning process). Therefore, online learners need to determine when and how to engage with course content without any other support than the course content and structure, which can pose a challenge for many learners (Lajoie & Azevedo, 2006).

Hence, it seems reasonable to assume that SRL may be a reliable predictor of academic performance. It has been shown that self-regulated learners are more effective learners (Toering et al., 2012), who attain higher grades in medical education (Lucieer et al., 2016). However, the effect of SRL on academic achievement in online learning is still unclear.

Several factors may interact and affect learning in online learning. However, some had received only limited discussion in the medical education literature while others had relatively little empirical testing. Although several research studies have investigated the effect of conception of learning on learners’ approaches, efforts, and motivation, however the effect of conception of learning on self-regulation is still insufficiently explored. Moreover, it can be assumed that students in online learning context may show different conceptions of learning as studies have shown that conception of learning is a context-depended construct that may differ according to the domain of the study or the surrounding context (Chiu et al., 2016; Tsai & Tsai, 2014). Additionally, SRL processes depend on both the learner and the surrounding environment (Bembenutty, 2006). As a result, we assumed that the learners’ perception of the quality of the surrounding learning environment might directly affect their behaviour and outcomes.  In other words, the quality and interactivity of the learning environment may shape the learners’ attitude towards the learning experiences and influence the behavioural control of the learner (Zhao, 2016).

Figure 1: The study conceptual framework

Therefore, a model was hypothesized to explore the interaction between self-regulated learning, the conception of learning, online discussion, and the e-learning experience in an online environment, and how this interaction may affect academic achievement. This cross-sectional study provides an exciting opportunity to advance our knowledge about the learning process in online learning by raising the following questions: 

1. What is the relationship between SRL and academic achievement in online learning?
2. What are the interactions between personal characteristics, beliefs, behaviours, and environment in online learning?
3. Does these interactions affect academic achievement in online learning?

II. METHODS

A. Type of the Study and Setting

An observation cross-sectional study was performed at the Faculty of Medicine, Suez Canal University, Egypt. The Medical Education Department offers postgraduate online learning programs in Medical Education to the graduates of Health Professions Education specialties. The program is one of the first online programs in health professions education in the Arab region. It is a two-year program in which students submitted weekly assignments through WordPress / Eleum and receive online feedback on the same Learning Management system (LMS). Also, participate in an online discussion forum through the web-based application Listserv on Google group.

B. Participants and Sampling

‘Out of 231 learners in the online program, a non-probability convenience sample of 128 learners was recruited in the current study; of which, 88 participants had an input in the online discussion’. The subjects were selected from all the program fellows based on their approval to be included in the study sample. The participants were asked to participate in the study through a mass email composed of a detailed description of the nature of the study, the purpose of the study and its relevance to the field of medical education. In all cases, fellows were informed that any information they included in the questionnaires would be treated with confidentiality.

C. Data Collection Tools

Instruments were selected in the current study because it was constructed and used in relevant contexts and the design of the final version of the questionnaires were validated using factor; reliability and test- retest analysis.

1) Measuring learners’ self-regulated learning: The Online Self-Regulated Learning Questionnaire (OSLQ) was used to measure the self-regulated learning behaviours of the fellows (Barnard et al., 2008). The OSLQ consists of six subscale constructs including: environment structuring; goal setting; time management; help seeking; task strategies; and self-evaluation.

2) Measuring learners’ conception of learning: The mental model section of the Inventory of Learning Style (ILS) was used to explore the learners’ conception of learning. The questionnaire was kindly provided by J.D. Vermunt, who originally developed this inventory (Vermunt, 1998). The conception of learning section is composed of 25 items categorised under five scales: construction of knowledge, intake of knowledge, use of knowledge, stimulating education & cooperation of learning.

3) Measuring of the quality of e-learning experience: The e-Learning Experience Questionnaire was used to explore the role of the learning environment (Ginns & Ellis, 2007). The questionnaire consisted of subscales which would reflect students’ perceptions of Good Teaching, Good Resources Clear Goals and Standards, Appropriate Assessment, Generic skills, Appropriate Workload and student interaction.

4)Online discussion: The assessment of the fellows’ input in the online discussion was done by using a scoring rubric that was included in a framework proposed by Nandi et al. (2009). This framework defines several themes on which qualitative online interaction can be designed and assessed. The scoring rubric composed of three broad categories: content, interaction quality and participation.

5) Academic achievement: The fellows’ final grade is the sum of the educational units’ mean which, in turn, is the sum of the unit assignments’ mean was used as an indicator of academic achievement. The academic achievement was categorized into four categories according to the final mean of the units: excellent: means 9-10, very good: means 8, good: means 7 and pass: means 6 and fail means > 6.

III. RESULTS

Data analysis was conducted using Statistical Package for the Social Sciences (SPSS®) version 20 software and International Business Machines SPSS Amos™ version 20. Out of the 231 learners in the Health Professions Education program through distance learning, 128 postgraduate learners were included in the study. The sample composed of 40 males and 88 female learners. Furthermore, they were divided according to their previous academic rank into 2 groups (Dr: 69 & Prof: 59 students). Student t-test revealed that there is no significant difference between male and female in SRL, t (126) = 1.43, conception of learning, t (126) = 0.13, quality of E-learning experience, t (126) = 0.78, online discussion, t (126) = -1.46 and academic achievement, t (126) = -0.79, p<0.05.

*Enjy Abouzeid
6A Hassan El Bassry Street, 
Ismailia, Egypt
Email: Enjyabouzeid@yahoo.com

Submitted: 24 June 2020
Accepted: 8 September 2020
Published online: 4 May, TAPS 2021, 6(2), 31-37
https://doi.org/10.29060/TAPS.2021-6-2/OA2328

Julie Yun Chen^1,2, Weng-Yee Chin¹, Agnes Tiwari³, Janet Wong³, Ian C K Wong⁴, Alan Worsley⁴, Yibin Feng⁵, Mai Har Sham⁶, Joyce Pui Yan Tsang^1,2 & Chak Sing Lau⁷

¹Department of Family Medicine and Primary Care, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ²Bau Institute of Medical and Health Sciences Education, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ³School of Nursing, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ⁴Department of Pharmacology and Pharmacy, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ⁵School of Chinese Medicine, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ⁶School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong; ⁷Department of Medicine, Li Ka Shing Faculty of Medicine, The University of Hong, Hong Kong

Abstract

Introduction: The demanding nature of medical and health sciences studies can cause stress among students in these disciplines affecting their wellbeing and academic performance. The Perceived Stress Scale (PSS-10) is a widely used measure of perceived stress among medical students and healthcare professionals that has not yet been validated among medical and health sciences students in Hong Kong. The aim of this study is to establish the construct validity and reliability of the PSS-10 in this context.

Methods: 267 final year medical and health sciences students were surveyed using the PSS-10. The data were analysed using exploratory factor analysis for construct validity and Cronbach’s alpha coefficient and corrected item-total correlations for reliability.

Results: Exploratory factor analysis revealed a two-factor structure for PSS-10, with Cronbach’s alpha of 0.865 and 0.796, indicating good internal consistency. Corrected item-total correlations showed satisfactory correlation ranged from 0.539 to 0.748 for all items and their respective subscale. Both tests supported PSS-10 as a two-factor scale.

Conclusion: The PSS-10 is a valid measure for assessing perceived stress in Hong Kong medical and health sciences students.

Keywords:            Undergraduate Students, Medicine, Nursing, Pharmacy, Health Sciences, Validation, Perceived Stress

Practice Highlights

• It is important to have a valid instrument for early detection of stress in health science students.
• Perceived Stress Scale (PSS-10) has a two-factor structure, a finding that is consistent with most other studies.
• PSS-10 has satisfactory internal consistency and reliability.
• PSS-10 can be used to assess the level of stress in medical and health sciences students.

I. INTRODUCTION

Undertaking studies in healthcare disciplines can be stressful as the programmes are demanding and students are often competing with higher achieving peers from admission to graduation. Significant stress can lead to psychological distress that has negative implications on current and future performance. Medical students have a higher prevalence of distress and poorer mental quality of life than their non-medical peers (Dyrbye et al., 2006; Shin et al., 2016), and also experience sleep deprivation, anxiety, and feelings of social isolation as revealed in focus group interviews conducted by Henning et al. (2010). There may also be a negative impact in quality of patient care (Firth-Cozens, 2001) and higher rate of medical errors (West et al., 2009). High perceived stress level correlated to impaired clinical performances in nursing students, including application of knowledge, clinical skills and communication (Ye et al., 2018). High level of stress and impaired quality of life were also found in third year pharmacy students in the United States (Marshall et al., 2008). In a study on pre-medical and health sciences students, higher perceived stress was a predictor of poor academic achievement (Henning et al., 2018).

As in many Asian cultures, Hong Kong students in general are under pressure to perform well in school as education is viewed as a crucial stepping-stone to success (S. Chan, 1999; Tan & Yates, 2011). This pressure may be particularly pronounced in medical students who manifest a greater degree of psychological distress, including perceived stress, depressive symptoms and anxiety, than other university students (Wong et al., 2005). A survey on medical students from the University of Hong Kong also revealed that majority of medical students were screened positive for minor psychiatric disorders and up to 95% of them were burned out (Chau et al., 2019). Many students may be “pushed” into a career path by extrinsic factors such as parental expectation (Sreeramareddy et al., 2007) or as a part of family tradition. Asian medical students may also tend to focus on academic achievement and seek to outperform their peers (Henning et al., 2011). Given the risk for developing high level of stress for these students, and the particularly intense environment in Hong Kong, it is important to have a valid instrument for early detection of stress so that appropriate strategies may be instituted at an early stage.

The Perceived Stress Scale (PSS-10) (Cohen, 1988) is widely used to measure perceived stress among healthcare students and doctors in different countries (Jones et al., 2015; Wongpakaran & Wongpakaran, 2010), and healthcare workers in Hong Kong (Chua et al., 2004). Healthcare students and healthcare workers may respond differently to a stressful event, as shown in the studies by Chua et al. (2004) and Wong et al. (2004), where the psychological effects of the SARS outbreak were different for healthcare students and workers. PSS-10 has been translated and validated in various languages, including Spanish, Turkish, Portuguese, Chinese, Thai and Japanese, among different populations such as patients, students, pregnant women, and adults in the general population (Lee, 2012). These validation studies are fundamentally robust, yet validating the PSS-10 is important in the specific undergraduate medical and health professions educational context in Hong Kong. Our study population is subject to different cultural, societal and educational influences that affect the perception of stress and the understanding of the items in the instrument so validation studies done elsewhere may not be applicable to our local context. The aim of this study is therefore, to establish the construct validity and reliability of the PSS-10 for use in this population.

 II. METHODS

A. Participants and Data Collection

All final year students undertaking studies in Li Ka Shing Faculty of Medicine in the University of Hong Kong (HKUMed) in the academic year of 2014-2015 were the target population of this study. A research assistant, who was not involved in teaching and assessment of the students, invited the students to participate in the study during a designated compulsory face-to-face teaching session for each programme. Those who provided written consent completed a written questionnaire in January – February 2015 or June 2015. The specific time for each cohort was chosen to avoid known stressful periods such as exams. The questionnaire included the PSS-10 and demographic information.

 B. Measure

The Perceived Stress Scale (PSS-10) (Cohen, 1988) was chosen as the instrument for measuring perceived stress. We considered other often-used instruments including the Depression Anxiety Stress Scale (DASS) (Lovibond & Lovibond, 1995) that measures depression and anxiety, in addition to stress and the General Health Questionnaire (GHQ) (Goldberg & Hillier, 1979) that measures medical complaints as a reflection of emotional stress, but these looked at broader conceptualisations of psychological distress beyond the scope of our study. PSS-10 was the most fit-for-purpose in measuring stress in terms of respondents’ views about their lives. In addition, we wished to be able to compare the stress in medical and health professions students to other key local comparator populations (e.g. university students, doctors, general population etc) and using the same instrument would facilitate this comparison.

PSS-10 is a 10-item instrument that assesses the extent of stress of respondents. PSS-10 is the abbreviated version of the original instrument with 14 items (PSS-14). A brief version with four items (PSS-4) is also available. Among the three versions of PSS, PSS-10 was found to be superior in psychometric properties, in terms of validity and reliability, than the other two versions (Lee, 2012). In the PSS-10, respondents rate statements about how unpredictable, uncontrollable, and overloaded they find their lives on a 5-point Likert scale from “never” to “very often”. Each response is converted to a score of 0 to 4 with the overall PSS score computed as the total score of the 10 items, with four reverse-coded items. The higher the score, the worse the perceived stress, with a maximum score of 40. There is no specific cut-off score that corresponds to high or low stress. We used the original English version of PSS-10 because as an English-medium university, students at HKUMed are taught in English (except during bedside teaching and clinical practicums) and students are proficient in English.

 C. Data Analysis

To establish the construct validity of the PSS-10, exploratory factor analysis (EFA) was performed on the responses to PSS-10 items by final year medical students, using principal component extraction with varimax rotation and the criterion of eigenvalue greater than 1.00. The Kaiser-Meyer-Olkin (KMO) measure equal to or greater than 0.5 was used to indicate sampling adequacy, while the Barlett’s Test of Sphericity with p<0.001 was used to ensure the appropriateness of the data set for EFA. Cumulative variance explained in the factor structure identified by EFA model was reported.

Cronbach’s alpha coefficient and corrected item-total correlations were used to examine reliability. Cronbach’s alpha coefficient was calculated to assess the internal consistency of each scale, which was considered acceptable if greater than 0.7 (Nunnally, 1994). Corrected item-total correlations were evaluated by Pearson’s correlation coefficient. A correlation of more than 0.4 was considered satisfactory (Wolfinbarger & Gilly, 2003).

 III. RESULTS

A total of 267 students completed the survey, with an overall response rate of 86.5%. 104 (39%) of the respondents were male (Table 1). Female students had significantly higher perceived stress than male students (20.84 vs 18.59; p<0.001). Table 2 shows the descriptive statistics for PSS-10 items and total score by programme of study.

*Julie Chen
Department of Family Medicine &
Bau Institute of Medical and Health Sciences Education,
Li Ka Shing Faculty of Medicine,
University of Hong Kong
21 Sassoon Rd, Pok Fu Lam
Hong Kong
Email: juliechen@hku.hk

Submitted: 19 June 2020
Accepted: 21 October 2020
Published online: 4 May, TAPS 2021, 6(2), 25-30
https://doi.org/10.29060/TAPS.2021-6-2/OA2327

Nicola Ngiam^1,2 & Chuen-Yee Hor¹

¹Centre for Healthcare Simulation, National University of Singapore, Singapore; ²Khoo Teck Puat-National University Children’s Medical Institute, National University Hospital, Singapore

Abstract

Introduction: Standardised patients (SPs) have been involved in medical education for the past 50 years. Their role has evolved from assisting in history-taking and communication skills to portraying abnormal physical signs and hybrid simulations. This increases exposure of their physical and psychological domains to the learner. Asian SPs who come from more conservative cultures may be inhibited in some respect. This study aims to explore the attitudes and perspectives of Asian SPs with respect to their role and case portrayal.

Methods: This was a cohort questionnaire study of SPs involved in a high-stakes assessment activity at a university medical school in Singapore.

Results: 66 out of 71 SPs responded. Racial distribution was similar to population norms in Singapore (67% Chinese, 21% Malay, 8% Indian). SPs were very keen to provide feedback to students. A significant number were uncomfortable with portraying mental disorders (26%) or terminal illness (16%) and discussing Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS, 14%) or Sexually Transmitted Diseases (STDs, 14%). SPs were uncomfortable with intimate examinations involving the front of the chest (46%, excluding breast), and even abdominal examination (35%). SPs perceive that they improve quality of teaching and are cost effective.

Conclusion: The Asian SPs in our institution see themselves as a valuable tool in medical education. Sensitivity to the cultural background of SPs in case writing and the training process is necessary to ensure that SPs are comfortable with their role. Additional training and graded exposure may be necessary for challenging scenarios and physical examination.

Keywords:            Standardised Patients, Perspective, Asian, Medical Education, Survey

Practice Highlights

• The Asian SPs in our institution see themselves as a valuable tool in medical education.
• Sensitivity to the cultural background of SPs in case writing and the training process is necessary to ensure that SPs are comfortable with the roles that they portray.
• Additional training and graded exposure for SPs may be necessary for challenging scenarios and physical examination in the Asian context.

Standardised patients (SPs) have been involved in medical education since the 1960s (Barrows & Abrahamson, 1964). SP methodology has been widely used in North America and Europe. By the 1990s, majority of American medical schools were using the SP methodology in teaching clinical skills, assessments and for providing feedback to learners (Anderson et al., 1994). The prevalence of employing SP methodology in medical education in Asia is presumed to be less ubiquitous. It is therefore imperative to understand the views of Asian SPs so that the SP methodology can be fostered.

SPs started out simulating medical symptoms and patient concerns as well as evaluating medical interviewing skills in 1976 (Barrows & Abrahamson, 1964; Stillman et al., 1976). Their role has evolved to demonstrating abnormal physical signs, providing feedback on medical interviewing skills and being involved in hybrid simulations. This increases their exposure to the medical environment and to different medical experiences that they may not have experienced before. Certain experiences may potentially cause psychological distress. The SP could be in a vulnerable position and personal attitudes and beliefs towards illness should be taken into consideration when engaging SPs to portray these roles.

This is particularly true in Asian SPs. For example, patients may not be willing to discuss mental health issues for fear of social stigma and shame (Kramer et al., 2002). Asian SPs are also likely to be more conservative and modest with regards to physical examination. This can be extrapolated from findings that cultural attitudes toward breast cancer screening tests and modesty are some reasons why Asian women are reluctant to seek out breast cancer screening (Parsa et al., 2006).

In the past, SPs were routinely employed in objective structured clinical examinations at our medical school. They were not required to provide any form of feedback to the learners. We endeavoured to develop a more structured SP training program at our institution. In the initial phase, this study was conducted to survey the attitudes of the SPs who work at our institution towards case portrayal and the value of SP methodology.

II. METHODS

This was an anonymous cohort questionnaire study. An online questionnaire was administered to standardized patients who were recruited to work at a high-stakes objective structured clinical examination at a university medical school. Participants were sent a link to an electronic survey by email after the event. Participation was voluntary. Questions about race, age, gender and years of experience as an SP were asked. The importance of the contribution of an SP to medical education and their comfort with discussing medical conditions, portraying abnormal signs and undergoing different physical examinations were evaluated. A Likert scale of 1-5 was used where appropriate. This questionnaire study is covered by the institutional review board approval (Study Reference Number: 09-288) of the standardised patient program in our institution. Being an anonymous, voluntary survey, the consent was implied when the participants filled and returned the completed survey.

Descriptive statistics and the electronic survey were generated using Vovici software version 6 (Vovici Corp, Dulles, Virginia, United States).

III. RESULTS

66 out of 71 SPs (93%) responded. 40% of the SPs were aged 31-40 years (Figure 1) and 72% were female. Racial distribution was similar to population norms in Singapore (67% Chinese, 21% Malay, 8% Indian, 4% others).

Figure 1: SP comfort with portrayal

With regards to their role, 95% of SPs felt it was important for them to be involved in teaching students and providing feedback. A significant number were uncomfortable with portraying mental disorders (26%) or terminal illness (16%) (Figure 1) and discussing HIV/AIDS (14%) or sexually transmitted diseases (14%) (Figure 2). With regards to death and dying, 6% of SPs were uncomfortable discussing this while another 6% were unsure about it. As expected, SPs were uncomfortable with examinations involving the front of the chest (46%, excluding breast examination) and even abdominal examination (35%). The 60% of the female SPs surveyed were uncomfortable with breast examination (Figure 3). SPs perceive themselves to improve the quality of teaching (98%) and to be cost effective (98%). The majority of this group of SPs (83%) felt that this was a viable option for sustainable employment.

Figure 2: SP comfort with discussing topic

Figure 3: SP comfort with physical examination

IV. DISCUSSION

The benefits of SP methodology in providing a safe environment for practice and experiential learning are well established. In an effort to expand the use of SP methodology at our institution, information regarding the acceptability and feasibility were required. In the past, SPs were mainly employed in summative assessment activities and did not provide learners with feedback. Before pushing the boundaries of the SP job description, it was important to understand the perspectives of our SPs and which areas of SP work they would feel comfortable or uncomfortable with.

The areas of interest were comfort with portraying roles that involved taboo topics such as mental health issues, sexually transmitted disease, death and dying. In many Asian cultures, mental illness is stigmatizing; it reflects poorly on family lineage and can influence others’ beliefs about the suitability of an individual for marriage. (Kramer et al., 2002). Many people of Asian descent view people with mental illnesses as dangerous and aggressive (Lauber & Rössler, 2007) and believe that mental illness is a punishment from God (Fogel & Ford, 2005). In China, mental health problems are believed to be a result of weak character, having evil spirits, or punishment for not respecting ancestors (Lam et al., 2006). Asian American women avoid seeking treatment for depression and suicide ideation because of Asian family and community stigma associated with mental health issues (Augsberger et al., 2015). With regards to sexual practices and sexually transmitted disease, literature shows that Chinese men regard homosexual-related stigma and discrimination as major barriers to HIV testing. Most men were reluctant to obtain an HIV test in fear that their homosexual identity would be exposed, and they sometimes encountered discrimination even from medical personnel (Wei et al., 2014). Living with HIV in an Asian society is fraught with difficulty in the context of fear and disapproval (Ho & Goh, 2017). Death and dying are generally considered taboo in Asian cultures. Open discussions about death are regarded as a bad omen (Hall & Hall, 1976). Even for those who are dying, discussion about death is avoided because it is believed that such talk may hasten the dying process or even cause death prematurely (Xu, 2007). The avoidance if discussion about death and dying in traditional Chinese culture has been found to impede the ability to discuss advanced care planning (Cheng, 2018). These cultural beliefs were reflected in the discomfort expressed by some study participants with portrayal of roles involving mental health issues, HIV or sexually transmitted diseases, terminal illness and death and dying. This is evidence that some of our SPs do have traditional Asian perspectives regarding these sensitive issues but it is encouraging that a larger proportion are comfortable with these issues. This informs us that SPs should be given advanced notice regarding the content of the case that they are expected to portray so that they can make an informed choice when accepting roles. This is especially important when taboo or sensitive content is involved. SPs should also be given an option to withdraw from the assignment if they feel uncomfortable with the content of the case after they have been trained for the case.

In view of the more conservative nature of Asians, the hypothesis was that there could be areas of the body that SPs would not be willing to have examined by students. Asian women seem to be more conservative as only 53% of respondents in a study did breast self-examinations (Sim et al., 2009; Tan et al., 2005) reported that, between 2000 and 2003, 21.5% of women in Singapore presented with stage III or IV breast cancer which may potentially be due to cultural attitudes toward breast cancer screening tests and modesty, which inhibit Asian women from participating in breast cancer screening (Parsa et al., 2006). Spiritual and religious beliefs were found to act as a barrier to breast cancer screening in Singaporean Malay women (Shaw et al., 2018). As expected, more than half of our female SPs were uncomfortable with breast examination. When both genders were considered, examination of the front of chest (excluding breast) and abdominal examination were also flagged as concerns. This made us aware of the hesitance of some SPs in this area and the need to explore this further while trying to expand the role of the SP. In developing our SP program, consent for physical examination needs to be explained in detail and comfort of the individual SP with any physical examination must be taken into consideration.

The SPs in our study perceived themselves to be of value in medical education. Standardized patients in a study in Switzerland felt motivated, engaged, and willing to invest effort in their task and did not mind the increasing demands of their work as long as the social environment in SP programs was supportive (Schlegel et al., 2016). This is encouraging for a developing SP program to know as we feel confident to expand the job scope of our SPs as long as adequate explanation and training is provided to support the SPs. With more structured coaching and exposure, we expect that SPs will become more comfortable with more challenging roles and would be willing to push the boundaries of their comfort zone.

One limitation of this study is the large majority of female participants. This was a convenience sample to optimize response rate. Further studies should aim to include a more balanced gender representation. Another limitation would be that only quantitative data was collected. In exploring perspectives, focused interviews with qualitative analysis would have provided a more in-depth understanding of the beliefs and values of the SPs.

V. CONCLUSION

This study provides initial insights into the perspectives of Asian SPs at a university medical school in an Asian country. They see themselves as a valuable tool in medical education and are willing to expand their role in the curriculum. Faculty and trainers need to be sensitive to the cultural background of our SPs in case writing and the training process to ensure that SPs are comfortable with the roles that they portray. This is of particular relevance to SP programs that employ predominantly Asian SPs. There is evidence from this study of discomfort with portraying patients with mental health issues, terminal illness and sexually transmitted diseases. The areas of exposure required in physical examination also need to be carefully considered. Additional training and graded exposure may be necessary for SPs willing to be involved in these scenarios and certain types of physical examination. Concerns about the scenario from the SPs may not be immediately apparent. The results presented here will make SP trainers more aware of the possibility of SP discomfort. Future research will be required on what type of training and what other factors will promote comfort with these scenarios as well as the impact of taking on such roles on the SPs.

Notes on Contributors

Nicola Ngiam conceptualized and designed the study, analyzed the data and interpreted the results, wrote the manuscript draft, revised it, read it and gave final approval of the manuscript.

Hor Chuen-Yee developed the methodological framework for the study, performed data collection and data analysis, revised the manuscript, read it and gave final approval of the manuscript.

Ethical Approval

This study is covered by the institutional review board approval (Study Reference Number: 09-288).

Acknowledgement

We thank Dr Dimple Rajgor for her assistance in editing, formatting, reviewing, and in submitting the manuscript for publication.

Funding

No funding source required.

Declaration of Interest

The authors have no conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Anderson, M. B., Stillman, P. L., & Wang, Y. (1994). Growing use of standardized patients in teaching and evaluation in medical education. Teaching and Learning in Medicine: An International Journal, 6(1), 15-22.

Augsberger, A., Yeung, A., Dougher, M., & Hahm, H. C. (2015). Factors influencing the underutilization of mental health services among Asian American women with a history of depression and suicide. BMC Health Services Research, 15, 542-542. https://doi.org/10.1186/s12913-015-1191-7.

Barrows, H. S., & Abrahamson, S. (1964). The programmed patient: A technique for appraising student performance in clinical neurology. Academic Medicine, 39(8), 802-805.

Cheng, H. W. B. (2018). Advance care planning in Chinese seniors: Cultural perspectives. Journal of Palliative Care, 33(4), 242-246.

Fogel, J., & Ford, D. (2005). Stigma beliefs of Asian Americans with depression in an internet sample. Canadian Journal of Psychiatry, 50(8), 470-478.

Hall, E. T., & Hall, E. (1976). How cultures collide. Psychology Today, 10(2), 66-74.

Ho, L. P., & Goh, E. C. L. (2017). How HIV patients construct liveable identities in a shame based culture: The case of Singapore. International Journal of Qualitative Studies on Health and Well-Being, 12(1), 1333899. https://doi.org/10.1080/17482631.2017.1333899

Kramer, E., Kwong, K., Lee, E., & Chung, H. (2002). Cultural factors influencing the mental health of Asian Americans. The Western Journal of Medicine, 176(4), 227-231.

Lam, C. S., Tsang, H., Chan, F., & Corrigan, P. W. (2006). Chinese and American perspectives on stigma. Rehabilitation Education, 20(4), 269-279. https://doi.org/10.1891/088970106805065368

Lauber, C., & Rössler, W. (2007). Stigma towards people with mental illness in developing countries in Asia. International Review of Psychiatry, 19(2), 157-178.

Parsa, P., Kandiah, M., Abdul, H. R., & Zulkefli, N. (2006). Barriers for breast cancer screening among Asian women: A mini literature review. Asian Pacific Journal of Cancer Prevention, 7(4), 509-514.

Schlegel, C., Bonvin, R., Rethans, J., & der Vleuten Van, C. (2016). Standardized patients’ perspectives on workplace satisfaction and work-related relationships: A multicenter study. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 11(4), 278-285.

Shaw, T., Ishak, D., Lie, D., Menon, S., Courtney, E., Li, S. T., & Ngeow, J. (2018). The influence of Malay cultural beliefs on breast cancer screening and genetic testing: A focus group study. Psycho‐Oncology, 27(12), 2855-2861.

Sim, H., Seah, M., & Tan, S. (2009). Breast cancer knowledge and screening practices: A survey of 1,000 Asian women. Singapore Medical Journal, 50(2), 132-138.

Stillman, P. L., Sabers, D. L., & Redfield, D. L. (1976). The use of paraprofessionals to teach interviewing skills. Pediatrics, 57(5), 769-774.

Tan, E., Wong, H., Ang, B., & Chan, M. (2005). Locally advanced and metastatic breast cancer in a tertiary hospital. Annals of the Academy of Medicine, Singapore, 34(10), 595-601.

Wei, C., Yan, H., Yang, C., Raymond, H., Li, J., Yang, H., Zhao, J., Huan, X., & Stall, R. (2014). Accessing HIV testing and treatment among men who have sex with men in China: A qualitative study. AIDS Care, 26(3), 372-378.

Xu, Y. (2007). Death and dying in the Chinese culture: Implications for health care practice. Home Health Care Management & Practice, 19(5), 412-414.

*Nicola Ngiam
Department of Medicine 
National University Health System
1E Kent Ridge Rd,
Singapore 119228
Email address: nicola_ngiam@nuhs.edu.sg