Published online: 2 January, TAPS 2019, 4(1), 13-23
DOI: https://doi.org/10.29060/TAPS.2019-4-1/OA1058

Sonali Prashant Chonkar^1,2, Hester Lau Chang Qi², Tam Cam Ha³, Melissa Lim², Mor Jack Ng² & Kok Hian Tan^1,2,4,5,6

¹Duke-NUS Medical School, Singapore; ²Division of Obstetrics & Gynaecology (O&G), Kandang Kerbau Women’s and Children’s Hospital (KKH), Singapore; ³The University of Wollongong, Australia, ⁴SingHealth Duke-NUS Joint Office of Academic Medicine, Singapore; ⁵Yong Loo Lin School of Medicine, NUS, Singapore; ⁶Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore

Abstract

Background: Students’ learning approaches have revealed that deep learning approach has a positive impact on academic performance. There are suggestions of a waning interest in deep learning to surface learning.

Aim: To assess if digital media can reduce the incidence of surface learning approach among medical students

Method: A digital video introducing three predominant learning approaches (deep, strategic, surface) was shown to medical students between March 2015 and January 2017. The Approaches and Study Skills Inventory for Students (ASSIST), was administered at the beginning and end of their clinical attachment, to determine if there were any changes to the predominant learning approaches. A survey was conducted using a 5-point Likert scale to assess if video resulted in change.

Results: Of 351 students, 191 (54.4%) adopted deep, 118 (33.6%) adopted strategic and 42 (12.0%) adopted surface as their predominant learning approach at the beginning of their clinical attachment. At the end of their clinical attachment, 171 (49.6%) adopted deep, 143 (41.4%) adopted strategic and 31 (9.0%) adopted surface learning as their predominant learning approach. The incidence of students predominantly using surface approach decreased from 42 (12.0%) to 31 (9.0%), although not statistically significant. Qualitative feedback from students stated that they were more likely to adopt non-surface learning approaches after viewing the video.

Conclusion: This evaluation highlighted the potential of digital media as an educational tool to help medical students reflect on their individual learning approaches and reduce the incidence of surface learning approach.

Keywords:            Learning Approaches, ASSIST, Digital Media, Video, Deep Learning, Surface Learning

Practice Highlights

• Digital media can help educate the students about three learning approaches: deep learning, strategic learning and surface learning approach.
• Digital media can encourage students to reflect on their own predominant learning approach.
• Digital media has potential to reduce the medical students’ reliance on surface learning approach.
• Approaches to learning may be influenced by curriculum structure.

I. INTRODUCTION

Medical students are exposed to staggering amounts of information during their career and have to assimilate information, apply clinical reasoning and undertake high stakes assessments. Most schools however, do not concentrate so much on the way students comprehend this knowledge. The way a student learns is affected by how the student was taught, the policies of the department and school, and the student’s own learning style (Newble & Entwistle, 1986). It is an acquired trait dependent on the learning context (Entwistle, 1997). There is little effort employed by teachers to enhance the possibility that individual students will achieve their full potential. Studying students’ approach to learning thus becomes an important factor that can determine both the quality and quantity of students’ learning (Amini, Tajamul, Lotfi, & Karimian, 2012).

In the recent years, there is a greater shift in focus to explore students’ approach to learning, particularly on these three distinct learning approaches – deep, strategic and surface approaches (Subasinghe & Wanniachchi, 2009; Wickramasinghe & Samarasekera, 2011; Samarakoon, Fernando, & Rodrigo, 2013; Shankar, Balasubramanium, & Dwivedi, 2014; Reid, Evans, & Duvall, 2012; Shah et al., 2016; Shankar, Dubey, Binu, Subish, & Deshpande, 2005; Aaron & Skakun, 1999; Cebeci, Dane, Kaya, & Yigitoglu, 2013; Amini et al., 2012). Deep learners focus developing interest in ideas, on reflecting and making connections between related concepts, thereby gaining a more thorough understanding, as well as better retention of content. Strategic learners adopt a systematic manner of studying with overt emphasis on learning certain concepts to excel in assessments. However, there may not be adequate integration across the topics as compared to deep approach and learners sometimes lack conceptual understanding (Leite, Svinicki, & Shi, 2010). Surface learners memorise all the information with little or no conceptual understanding and as a result, often find little interest in the concepts learned and tend not to read beyond what is stated in the syllabus. They may also be poorly motivated and result in being ineffective learners with a low level of understanding. Trigwell and Prosser (1991) suggested a negative correlation between surface learning approach and quality of learning, and the opposite for deep learners. Subasinghe et al. (2011) concluded that the adoption of deep and strategic learning approach will be beneficial to medical students, since their learning involves critical analysis and application of concepts for complex clinical situations. They also found that students with deep approach tend to achieve higher performance and vice versa.

While it is comforting that majority of the medical students preferred deep and strategic learning approaches (Subasinghe & Wanniachchi, 2009; Shankar et al., 2014; Reid et al., 2012; Shankar et al., 2005), Amini et al., (2012) and Aaron and Skakun (1999) found their medical students were more inclined towards the surface learning approach. Cecebi et al. (2013) found that their third year medical students preferred surface learning compared to the first and second year, suggesting the possibility of a waning interest in the deep learning approach as students progress through medical school. Being aware of our medical students’ predominant learning approaches in order to shift them to move towards more effective learning approaches was an important endeavour.

In this age of advanced information technology, teaching methods have evolved to meet the opportunities and challenges in undergraduate medical education (S.O. Ekenze, Okafor, O. S. Ekenze, Nwosu, & Ezepue, 2017; Shelton, Corral, & Kyle 2017). Multimedia e-learning enhances both teaching and learning (Ruiz, Mintzer, & Leipzig, 2006). Videos are being utilised in higher education to deliver useful content that can engage students (Mitra, Lewin-Jones, Barrett, & Williamson, 2010). We used digital video as an intervention to engage our students to reflect on the three predominant learning approaches, so that they become aware of their own predominant learning approach and reassess their approach. The advantages of using a video is that it can be easily used in multiple locations, updates added as required, short time frame to deliver information and can be re-watched at any time convenient to the audience.

This study aimed (i) to evaluate the learning approaches of medical students in Singapore, (ii) to assess if a video digital media intervention is effective in introducing the idea of ‘surface’, ‘strategic’ and ‘deep’ learning approaches and whether any change in that approach resulted.

II. METHODS

A. Participants

The study population comprised 351 Singaporean medical students from SingHealth who attended the Obstetrics and Gynaecology (O&G) clinical rotation a SingHealth from March 2015 to January 2017.

B. Study Design

We used the Approaches and Study Skills Inventory for Students (ASSIST) questionnaire (see Appendix) to assess the students’ learning approaches at the beginning and the end of the students’ O&G posting in SingHealth (Figure 1).

The ASSIST questionnaire is a revised version of the Approaches to Studying Inventory (ASI), developed by Tait, Entwistle and McCune (1998). The questionnaire has been validated in various cultures globally, including amongst eastern cultural population such as Chinese university students and also across the western student population in Britain and Scotland (Gadelrab, 2011; Albedin, Jaarfar, Husain, & Abdullah, 2013). The ASSIST questionnaire comprises of 52 questions divided into 13 subscales of 4 questions. Each questions is scored on a five-point Likert scale (1 for “low” through 3 for “average” to 5 for “high”), with 16 questions pertaining to surface and deep learning each and 20 questions relating to strategic learning. An example of a question assessing deep approach is “I try to relate ideas I come across to those in other topics or other courses whenever possible.”; strategic approach is “I think I’m quite systematic and organised when it comes to revising for exams.”; and surface approach is “I find I have to concentrate on just memorising a good deal of what I have to learn.” The scores for sets of 4 were combined into 13 subscales and further grouped to give each respondent a score for deep, strategic and surface approach. The predominant learning approach is defined as the approach which has the highest mean score amongst the three approaches. The predominant learning approach is calculated based on the mean of the respective questions for each of the three learning approaches.

Figure 1. An illustration of the study design – Beginning with the administration of ASSIST questionnaire, to the video intervention and the follow-up ASSIST questionnaire again to determine any change in learning approach

After completing the first questionnaire the same group of students were shown a video (Chonkar et al., 2018) entitled, “Be a better learner in medicine”, featuring three students. The video provided an introduction to three learning approaches and a description of these approaches. It then illustrated how learners adopting the three learning approaches would react and respond in three different scenarios in the context of learning the management of an high risk obstetrics condition: a normal tutorial, when studying in their own time and when being assessed during a test. As outlined by Rondon-Berrios and Johnston (2016), there are multiple techniques of successful teaching in a clinical environment including, role modelling and pattern recognition. The video capitalises on these two techniques to show how students who adopted these three different learning approaches would usually behave and illustrated the outcomes at the end of the video.

In the first scenario of the video, the student who adopted the surface learning approach decided to memorise information about the high-risk obstetric conditions from sources such as handbooks and lecture notes. There was no attempt to understand the basic pathophysiology of the condition or the underlying principle of management. The second student who adopted the strategic approach, used resources like important literature highlighted by the tutor and past year questions for targeted preparation to perform well in the final exam. There were attempts in clinical reasoning while managing the obstetric high risk condition. The student did not spend enough time to thoroughly understand underlying principles of management of medical disorder in pregnancy but rather selectively spent time studying selected topics which were deemed to be important. The deep learner went in depth to understand the basic pathophysiology and systematically approached the management with clinical reasoning with respect to stages of pregnancy that the patient could be in for that obstetric condition. The student further understood the critical aspect of the underlying principle of management of obstetric high risk conditions i.e. trying to balance the risks of prematurity due to early delivery versus the risks of continuing pregnancy endangering the mother and foetus, which can be applied to almost all other obstetric conditions. The student in the third scenario equipped herself by applying this underlying principle to other obstetric high-risk conditions when studying.

The final scene depicted the three students attempting their final posting examination. The surface learner experienced ‘brain freeze’ with clinical reasoning and was unable to remember the facts. The student adopting the strategic approach was able to apply some clinical reasoning to the question which she had previously identified to be important from examination point of view but did not do so well when another condition that was tested was not in her ‘radar screen’ of topics. The student who adopted the deep learning approach was able to apply correct clinical reasoning and underlying obstetrics principles to both obstetric conditions tested, even though one of the conditions was not familiar to her yet she was able to answer well. The video ended with a summary of the three learning approaches and the importance of adopting a non-surface learning approach to enhance learning and retention of knowledge.

A survey containing five relevant questions was conducted with answers on a 5-point Likert scale to assess if video was effective in helping students understand the three learning approaches and reflect on their own approach. The survey explored if students were willing to change their learning approach to a more favourable one. Positive response was then defined as those responses that were rated a 4 or 5 (Agree/Strongly Agree), Neutral response was those rated as 3 and Negative response was those rated as 1 or 2 (Disagree/Strongly Disagree).

C. Statistical Analyses

The data were analysed using Microsoft Excel program and Statistical Package for Social Sciences (SPSS) version 22. Scores were aggregated pre- and post-intervention. The differences between the mean scores from ASSIST questionnaire administered pre- and post-intervention were analysed using Mann-Whitney test for standard deviation and p-value significance. Significant p-value was taken as p<0.05 in this study.

III. RESULTS

A total of 351 students completed the pre-video ASSIST questionnaire. Six students were unavailable to complete the ASSIST questionnaire at the end of the posting. The demographic characteristics of the 345 medical students are listed in Table 1.

A. Type of Predominant Learning Approach Pre and post-video intervention

As seen in Table 2, more students adopted the deep learning approach prior to the beginning of the rotation (Deep 54.4% vs Surface 12.0% vs Strategic 33.6%). After 6-weeks of rotation in O&G, the strategic learners increased from 33.6% to 41.4%. The surface learners decreased from 12.0-9.0% and the deep learners decreased from 54.4% to 49.6%.

Further analysis of the pre- and post-intervention scores of the ASSIST questionnaire actually showed that only the increased in strategic learners was statistically significant. The decrease in the numbers of deep and surface learners was both statistically insignificant (Table 3).

Table 1. Demographic characteristics of medical students

Table 2. Predominant learning approach adopted by students at the beginning and the end of 6-weeks rotation

Table 3. The change in of the mean aggregate scores of ASSIST questionnaire for the various predominant learning approaches at the beginning and the end of 6-weeks rotation, after the video intervention

Table 4. Participants’ response to survey on video “Be a Better Learner in Medicine”

B. Participants’ Response to Survey on Video “Be a Better Learner in Medicine”

The video survey was completed by a total of 269 students. As seen in Table 4, 249 (92.0%) students thought the learning approaches explained in the video were clear, 234 (86.9%) students agreed the video helped them reflect on their predominant learning approach and 201 (74.7%) admitted that they are more likely to adopt non-surface learning approach after viewing the video. The video was found to be engaging by 193 (71.0%) students and 201 (74.7%) students thought that the duration of the video was just right.

C. Qualitative Feedback on the Video “Be a Better Learner in Medicine”

There were many qualitative comments from the students who completed the survey. There were 95 comments expressing positive views, 21 comments expressing negative views and 4 neutral comments with respect to the video depicting the learning approaches.

Some of the positive comments were “It is a more useful tool and approach to learning”, “I feel inspired by the video”, “Video illustration is helpful to elaborate different types of learner”, “The video made me reflect a little on how I learn”, “Likes the way the video brings the idea of deep learning” and “Good video to stimulate thinking on learning method”.

Addition examples of positive comments were “I feel that my own approach has been ineffective and I want my learning to improve”, “Recognise how important it is to adopt deep learning”, “It shows that my current learning method is inadequate and it shows how I should work on it”, “It is clear that deep learning allows students to learn from principle”, “I think it is important and useful to understand what we’re learning and apply it in new scenarios or situations”, “Deep is better approach with longer retention & understanding eventually beneficial to future patients.”

Some of the negative comments were “Ideally, deep approach is desirable but due to a lack of time, sometimes surface approach can be more time effective as temporary stop-gap measure”, “The amount needed to study and time needed to be a deep learner is a deterrent”, “May not have the luxury of time. (Deep learning requires a lot of time)”, “Time constraint”, “Sometimes deep learning takes time, and the short time we are given to study may push students more towards strategic thinking.”

IV. DISCUSSION

Medical students have to retain large amounts of information and at the same time have to keep themselves abreast with the latest research. The students also need to cope with heavy workloads and tight course schedules in medical school while struggling to understand and retain information with good time management skills being essential. Our study found that the majority of our students adopted the deep approach as their predominant learning approach (54.4%) followed by the strategic approach (33.6%) prior to the intervention. It is heartening to note that surface learning approach was the least preferred approach adopted by our medical students. While encouraging, there was still a substantial proportion (12%) of medical students who adopted surface learning approach as their predominant learning approach. With the rapidly advancing medical sciences, students may not have enough time to see, read and assimilate all necessary information before their assessments. This may force some of the slow learners to adopt the surface learning approach. Some students commented that surface learning approach could be adopted easily as a temporary measure due to time constraints. This might explain why 12% of medical students adopted predominantly surface learning approach in our study, and those students who were more inclined towards surface learning approach in previous studies (Aaron & Skakun, 1999; Cebeci et al., 2013; Amini et al., 2012).

After the video intervention, we noted a decrease in the surface learning approach and an increase in the strategic learning approach. As reflected in Table 3, the mean score of surface learning decreased from 62.6 to 61.4 (p=0.06.) and that of strategic and deep learning approaches increased (strategic – 72.7 to 74.6, p=0.014; deep 74.8-75.9, p = 0.120). Though this decrease in surface learning approach and increase in deep learning approaches were not statistically significant, these results reflected the potential for digital media (in the form of a video) to effect a change in the students’ learning approaches. As depicted in the qualitative comments in the video survey, some students stated that time constraint was an important factor that prevented students from adopting deep approach. The increase in strategic learning approach may be due to the need for students to pass certain assessments at the end of posting examination, as required by their schools and clerkship. A number of students had commented that adoption of strategic learning approach helped them achieve good outcomes during assessments. The students might have felt that adopting the strategic approach would be more expedient by selecting and studying the important topics amongst the intensive curricular topics and that might help them do well in a short period of attachment. The packed shortened curriculum may influence approaches to learning as evidenced from the comments in this study.

The tool used to assess students’ learning approach – ASSIST questionnaire also posed some limitations for this study. Despite the ASSIST questionnaire being validated in many cultures and, it may not fully be reflective of the true approach to learning of students, especially if they answered the questions in a way that they thought would have been the approved answers (Reid et al., 2012). In addition, ASSIST questionnaire is a self-rated tool, so the results are subjective. There may be greater awareness and a change in the students’ perception towards their learning approaches after watching the video illustrating how adopters of the various learning approaches would behave. This would have affected the results of the ASSIST questionnaire.

The video helped raise awareness of various learning approaches and engage students to reflect and reassess their predominant learning approach. The number of students adopting non-surface learning approach increased after watching the video. There is a potential to discourage them from adopting surface learning approach as reflected in the qualitative feedback which showed that many became aware of the disadvantages of surface learning. A number of students were also convinced after the video that the surface learning approach was not favourable for them to retain a lot of information that they will need to use in clinical setting. While this was a good outcome, the difference was not statistically significant. Moreover, the video was just a short intervention of 12 minutes and not followed by further education or reinforcement of the appropriate learning approach throughout the posting. It is also difficult to change entrenched learning approach behaviour. In addition, most of the medical students are in their final few years of medical school, and have most likely been using their current learning approach successfully over the previous few years in medical school, and see no point in changing their learning approach. Other possible limitations include the small sample size of medical students who were involved in this study that have affected the results.

Multiple previous studies have found that many factors are involved in encouraging deep learning approach among medical students such as appropriate workload, clear goals with informative feedback and targeted assessments (Reid, Duvall, & Evans, 2005; Rushton, 2005). To facilitate deep learning, the curriculum can be adjusted to address these factors. Instead of factual overload, faculty can allocate more teaching time towards case-based scenarios and tutorials that focus on applying the understanding of pathophysiology, concepts and care management principles learned. This will aid students in tackling complex patient problems, a reality they will face when they become active practitioners. Smaller tutorial groups to focus on individualised targeted feedback after formative assessment may help in encouraging the usage of deep learning approach. The format of assessment can also be more encompassing, to reward understanding instead of pure memorising (Rushton, 2005).

The predominant deep and strategic learning approach is a reflection of self-motivation among medical students, which is inherent (Amini et al., 2012). But many students may not be aware of their own predominant learning approach and may not realize that they may adopt different approaches according to different circumstances. With the increase in the number of medical students and limited number of core faculty and time, finding a solution to encourage independent deep learning and discourage surface learning approach amongst our students would be beneficial in the long run, ensuring that students develop the most favourable learning approach from the start and keep honing this learning approach skill as they progress in their medical career.

Medical schools will also need to look into their curriculum and time-lines of individual clinical attachments periodically to ensure that they are able to deliver core knowledge without compromising on understanding of concepts. Perhaps, targeting students in their pre-clinical years and introducing the concept of various learning approaches would set a good foundation and aid in deeper learning during clinical years. Discouraging medical students from adopting surface learning approach would be beneficial in achieving expected long term goals and this would ultimate translate into higher quality education and patient care.

V. CONCLUSION

This study highlights the potential of digital media as an educational tool to enable medical students to become better learners (Gadelrab, 2011). With the use of a video, students were able to reflect on their predominant learning approaches, which is relevant with the evolving teaching andragogy that emphasises self-directed learning. The students were exposed to the potential pitfalls of adopting surface learning in the video and this likely encouraged some of the surface learners not to use surface learning approach, in order to better help them cope with the increasing need to apply strategic thinking, deep thinking and critical analysis during complex situations. This project can help spearhead efforts to optimize students’ learning, to move the medical education landscape forward.

Notes on Contributors

Sonali Chonkar, Melissa Lim and Kok Hian Tan conceived, designed the video script and the study. Melissa Lim acquired the data; Mor Jack Ng analysed and interpreted the data. Kok Hian Tan critically revised the article and gave invaluable inputs at every stage of writing and data acquisition/analysis. Tam Cam Ha and Hester Lau assisted in the revision and editing of the article.

Ethical Approval

The study was approved and given the exempt status by our institution’s Centralised Institutional Review Board of SingHealth (CIRB) committee. All students participated voluntarily and informed consent was obtained before participating in the study. The CIRB reference number for our study is 2013/232/D.

Acknowledgements

The authors would like to thank all medical students of NUS Yong Loo Lin School of Medicine, Duke-NUS Medical School and NTU Lee Kong Chian School of Medicine for participating in this joint study. We would like to thank Ms Mabel Yap from Duke-NUS Secretariat Office for her assistance in the research. We would like to thank Ms Hester Lau, Ms Tang Wan Chu, Ms Goh Jia Ying, Ms Amy Tan, Ms Alicia Lim and the staff of Division of Obstetrics & Gynaecology at KKH for the assistance in the production of the video.

Funding

This study was supported by a Teaching Enhancement Grant (TEG – AY 2014/2015) from NUS Centre for Development of Teaching and Learning.

Declaration of Interest

The authors declare that they have no competing interests.

References

Aaron, S., & Skakun, E. (1999). Correlation of students’ characteristics with their learning styles as they begin medical school. Academic Medicine, 74(3), 260-262.

Albedin, N. F. Z., Jaarfar, Z., Husain, S., & Abdullah, R. (2013) The Validity of ASSIST as a measurement of learning approach among MDAB students. Procedia – Social and Behavioural Sciences, 90, 549-557.

Amini, M., Tajamul, S., Lotfi, F., & Karimian, Z. (2012). A survey of study habits of medical students in Shiraz Medical School. Future of Medical Education Journal, 2(3), 28-34. https://doi.org/10.22038/fmej.2012.354.

Cebeci, S., Dane, S., Kaya, M., & Yigitoglu, R. (2013). Medical students’ approaches to learning and study skills. Procedia – Social and Behavioural Sciences, 93, 732-736. https://doi.org/10.1016/j.sbspro.2013.09.271.

Chonkar, S. P., Lau, H. C. Q., Tang, W. C., Goh, J. Y., Mohammad N. B, A., Lim M., & Tan H. K. (2018, September 18). Be a predominantly deep learner in medicine [Video file]. Retrieved from https://youtu.be/ttnSCoun_sA.

Ekenze, S. O., Okafor, C. I., Ekenze, O. S., Nwosu, J. N., & Ezepue, U. F. (2017). The value of internet tools in undergraduate surgical education: Perspective of medical students in a developing country. World Journal of Surgery, 41(3), 672-680. http://doi.org/10.1007/s00268-016-3781-x.

Entwistle, N. (1997). Contrasting perspectives on learning. In F. Marton, D. Hounsell & N. Entwistle (Eds.), The Experience of Learning: Implications for teaching and studying in higher education (3^rd ed) (pp. 3-22). Retrieved from: http://www.docs.hss.ed.ac.uk/iad/Learning_teaching/Academic_teaching/Resources/Experience_of_learning/EoLChapter1.pdf.

Gadelrab, H. F. (2011). Factorial structure and predictive validity of approaches and study skills inventory for students (ASSIST) in Egypt: A confirmatory factor analysis approach. Electronic Journal of Research in Educational Psychology, 9(3), 1197-1218.

Leite, W. L., Svinicki, M., & Shi, Y. (2010). Attempted validation of the scores of the VARK: Learning styles inventory with multitrait–multimethod confirmatory factor analysis models. Educational and Psychological Measurement, 70(2), 323-339. https://doi.org/10.1177/0013164409344507.

Mitra, B., Lewin‐Jones, J., Barrett, H., & Williamson, S. (2010). The use of video to enable deep learning. Research in Post-Compulsory Education, 15(4), 405-414. https://doi.org/10.1080/13596748.2010.526802.

Newble, D. I., & Entwistle, N. J. (1986). Learning styles and approaches: Implications for medical education. Medical Education, 20(3), 162-175. https://doi.org/10.1111/j.1365-2923.1986.tb01163.x.

Reid, W. A., Evans, P., & Duvall, E. (2012). Medical students’ approaches to learning over a full degree programme. Medical Education Online, 17(1), 17205-17211. http://dx.doi.org/10.3402/meo.v17i0.17205.

Reid, W. A., Duvall, E., & Evans, P. (2005). Can we influence medical students’ approaches to learning? Medical Teacher, 27(5), 401-407. https://doi.org/10.1080/01421590500136410.

Rondon-Berrios, H., & Johnston, J. R. (2016). Applying effective teaching and learning techniques to nephrology education. Clinical Kidney Journal, 9(5), 755-762. http://doi.org/10.1093/ckj/sfw083.

Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81(3), 207-212.

Rushton, A. (2005). Formative assessment: A key to deep learning? Medical Teacher, 27(6), 509-513. https://doi.org/10.1080/01421590500129159.

Samarakoon, L., Fernando, T., & Rodrigo, C. (2013). Learning styles and approaches to learning among medical undergraduates and postgraduates. BioMed Central Medical Education, 13(1), 42-47. https://doi.org/10.1186/1472-6920-13-42.

Shah, D. K., Yadav, R. L., Sharma, D., Yadav, P. K., Sapkota, N. K., … Islam, M. N. (2016). Learning approach among health sciences students in a medical college in Nepal: A cross-sectional study. Advances in Medical Education and Practice. 7, 137-143. https://doi.org/10.2147/AMEP.S100968.

Shankar, P. R., Balasubramanium, R., & Dwivedi, N. R. (2014). Approach to learning of medical students in a Caribbean medical school. Education in Medicine Journal, 6(2), e33-40. https://doi.org/10.5959/eimj.v6i2.235.

Shankar, P. R., Dubey, A. K., Binu, V. S., Subish, P., & Deshpande, V. Y. (2005). Learning styles of preclinical students in a medical college in western Nepal. Kathmandu University Medical Journal, 4(3), 390-395.

Shelton, P. G., Corral, I., & Kyle, B. (2017). Advancements in undergraduate medical education: Meeting the challenges of an evolving world of education, healthcare, and technology. Psychiatric Quarterly, 88(2), 225-234. https://doi.org/10.1007/s11126-016-9471-x.

Subasinghe, S. D. L. P., & Wanniachchi, D. N. (2009). Approach to learning and the academic performance of a group of medical students – Any correlation. Student Medical Journal, 3(1), 5-10.

Tait, H., Entwistle, N. J., & McCune, V. (1998). ASSIST: A reconceptualisation of the approaches to studying inventory. In C. Rust (Ed.), Improving student learning: Improving students as learners. Oxford: Oxford Brookes University, The Oxford Centre for Staff and Learning Development.

Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22(3), 251-266. https://doi.org/10.1007/BF00132290.

Wickramasinghe, D. P., & Samarasekera, D. N. (2011). Factors influencing the approaches to studying of preclinical and clinical students and postgraduate trainees. BioMed Central Medical Education, 11(1), 22-28. https://doi.org/10.1186/1472-6920-11-22.

*Sonali Prashant Chonkar
Address: KK Women’s and Children’s Hospital,
100 Bukit Timah Road
Singapore 229899
E-mail: sonali.chonkar@kkh.com.sg

Published online: 7 May, TAPS 2019, 4(2), 39-47
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2034

Annie L. Kilpatrick^1,2, Ketsomsouk Bouphavanh³, Sourideth Sengchanh³, Vannyda Namvongsa³ & Amy Z. Gray^1,2

¹Centre for International Child Health, Department of Paediatrics, University of Melbourne, Australia; ²The Royal Children’s Hospital, Australia; ³Education Development Centre, Faculty of Medicine, University of Health Sciences, Lao People’s Democratic Republic

Abstract

Practice Highlights

• At a time when information has never been more available, students in many non-English speaking countries and low resource settings face a daily challenge of information availability and inequity created by both access to technology and educational materials, which are not limited to Laos.
• For learning resources to be accessible for Lao students they need to be cost effective, language appropriate and device appropriate.
• Equipping teachers, who remain an integral tool in students learning, to face the challenges of learning resource access is vital given they face the same access barriers and must keep both their technical knowledge and teaching skills current.
• Rather than large scale development of new educational resources, there is a role of non-traditional education resource development through blogging, social media and curating of materials, alongside engaging students to empower them to access the materials they need.

I. INTRODUCTION

Lao People’s Democratic Republic (Lao PDR or Laos) is a landlocked country bordering Myanmar, Cambodia, China, Thailand and Vietnam and remains classified as a least developed country (LDC) (United Nations [UN], 2015). The history of the education and healthcare development plays an important role in the current education context for Laos with many different countries contributing to development of these sectors (Dodd, Hill, Shuey, & Fernandes Antunes, 2009). Many of these stakeholders have ties with the University of Health Sciences (UHS), the sole university in the country responsible for training medical doctors through both bachelor level and post-graduate programs (Akkhavong et al., 2014).

Teachers at the UHS have largely completed medical training in-country. Limited English language capacity presents major difficulties in the ability to access up-to-date teaching resources (Milosavljevic, Vuletic, & Jovkovic, 2015; Pavel, 2014). From a geographical and linguistic perspective, the closest country involved in the Lao medical education system is Thailand, with Thai language having similarities to Lao and students and medical staff exchange occurring with Thai hospitals. Historically, the French colonial history of the country means French language is spoken among older generations and is still being taught in the medical school. More recently the importance of English as an international language has been emphasised (Pavel, 2014). The multiple languages spoken by different partner countries, combined with different approaches to medical issues creates challenges for students and staff at UHS above and beyond developing their medical competencies.

Compounding problems of language is the relative lack of learning resources in Lao language, with most text books available to students written in Thai or English. This reflects a wider paucity of written Lao language books in the general community and poorly developed reading culture which is slowly changing (Duerden, 2017). Even fewer relevant online resources exist in Lao language, meaning staff and students must navigate the vast array of online material without accompanying language capacity to distil or search information effectively. This situation impacts both the student’s capacity to learn and the staff’s capacity to teach.

Whilst there may be a range of available strategies for addressing these challenges for teaching and learning, with limited resources it is critical to understand what the best and most effective investments in educational resources might be. Yet there is little information available regarding the learning preferences and access to resources of university students in Laos, or more generally in other low resource settings. Previous studies in health science learning have mainly focused on social media use (Pimmer, Linxen, & Gröhbiel, 2012), electronic learning (e-learning; Bediang et al., 2013), computer literacy (Bediang et al., 2013; Ranasinghe, Wickramasinghe, Pieris, Karunathilake, & Constantine, 2012) and clinical skills learning (Papanna et al., 2013; Widyandana, Majoor, & Scherpbier, 2010). However, these studies do not cover a broad range of learning preferences, consider resources used for different content areas, nor consider issues of access more broadly. There were recent requests for e-learning tools in Myanmar (Bjertness et al., 2016). They highlight similar challenges in making appropriate learning and teaching material available in a country which may not have the capacity to meet the large, immediate demand for resources – but without the additional language challenges faced in Laos.

This study is the first to gather information regarding student learning preferences in Laos. We aimed to understand the current access staff and students have to educational resources, in particular online learning material and their learning preferences and needs. This information is paramount in aiding the university in their understanding of student and staff needs, in order to develop a systematic and appropriate strategy for educational resource development at UHS.

II. METHODS

A mixed-methods cross-sectional study of learning preferences and access to educational resources of staff and students at UHS was undertaken between May and November 2015. Research was guided by a phenomenological approach, which aims to examine ‘the lived experience’ of a person or several people in relation to a concept or phenomenon of interest (Liamputtong, 2009). A cross-sectional survey of staff and students was followed by focus group interviews. These two methods of data collection were designed to be complementary. The use of the surveys, representing views of a larger cross-section of the staff and student population, and focus group interviews, representing more in-depth understanding of the experience, allowed for concurrent triangulation (Castro, Kellison, Boyd, & Kopak, 2010).

The survey, consisting of fifteen questions, was developed by the researchers. The questions were designed with the intent to capture a range of information addressing the aims of the study. Questions related to current learning resources, access to educational aids and online learning and perceived educational resource needs. The survey was initially written in English, and then translated into Lao language for completion by participants. Participants included 183 fourth and sixth-year medical students, 105 nursing, 72 dentistry, 64 pharmacy, 5 physiotherapy and 4 medical technology students, along with 53 medical residents and 19 faculty staff. Fifth-year medical students were not available due to external clinical rotation. Staff were included since any strategies to address educational resource availability would need to take into account the access and capacity of staff responsible for teaching.

Surveys were distributed to students in lectures and to faculty staff and medical residents in education meetings in the university and hospital respectively. Researchers explained the study, its anonymity and the voluntary nature of its completion to participants, in Lao language, and consent was implied by completion and return of the survey. Survey results were entered into an EpiData database then analysed using Statistical Package for the Social Sciences (Version 24). Categorical data were described according to the number and percentage of participants in each group. Comparison between groups was performed using chi-square.

Stratified sampling was used for the focus groups, with students volunteering from specific sub-groups after they had completed the survey. One focus group was conducted with each of the main student cohorts – medical, pharmacy, nursing and post-graduate medical. Students were provided with a written plain language statement, a verbal explanation, and opportunity to withdraw, then signed a written consent form.

Focus groups were conducted using an interview guide structured around three main questions – current learning materials and how these are accessed; main barriers to accessing learning material and; the learning material needs. Interviews were conducted in English with translation into Lao. Data were audio recorded and transcribed into written documents. The written Lao content was translated to the English language before analysis. Inductive qualitative content analysis was performed by the primary researcher (AK) to elaborate on the survey findings.

III. RESULTS

507 (61.3% female, 36.7% male) students and staff completed the survey from a total of 800 potential participants (response rate 63.4%). Almost half of the participants were older than 25 years of age (Table 1) and the vast majority (81.5%, 413/507) had been in the workforce before medical school. The majority (52.7%, 267/507) of participants were from provinces outside of the capital (Table 1). The largest student cohort was medical students (36.1%) followed by nursing students (20.7%) (Table 1). Faculty staff made up less than 4% of participants (Table 1) and there were no significant differences in their responses compared to students.

The majority (98.0%) of participants own smart phones, for example 18/19 faculty staff, 178/183 medical students and 97/105 nursing students. Smart phones are participant’s main access to the internet, with less access to computers or tablets (Table 2). Over 90% of survey participants (478/507) reported barriers to internet access, with 56.8% (288/507) not having access to internet in their home and very few having access in their educational institutions (Table 2). One hundred percent of faculty staff reported barriers to internet access compared to 95% of nursing and 93% of medical students. There are many barriers to internet access including cost (62.9%), speed (65.1%), language (41.0%) and a lack of understanding (8.1%), and these were also verified in focus group discussions. Sixteen of the 19 staff members said cost was a barrier compared to 114 of the 183 medical students and 67 of the 105 nursing students. Of the 19 staff members, 3 reported language as a barrier, compared with 69/183 medical students and 34/105 nursing students.

Table 1. Demographics of 507 study participants

A. Educational Resources

More than half of participants (294/507) access the internet daily for study purposes whereas 20.9% (106/507) access the internet monthly or less. Medical students and residents have a higher percentage of daily internet use (71%) compared with non-medical students (48%, p < 0.001) although there was no difference in smartphone ownership or internet at home. Around one third of participants used internet search tools, social media and text books for education daily (Figure 1). The social media applications used included Facebook (85.8%), WhatsApp (69.6%) and Line (42.6%).

Table 2. Technology ownership, internet access and barriers among the study cohort of staff and students at The University of Health Sciences Lao PDR

Figure 1. Percentage of students and staff at The University of Health Sciences Lao PDR using specific educational resources on a daily basis

Focus group participants most commonly described accessing learning materials via personal smart phones for internet searches. Internet site searches involved mainly the use of the google search engine and sites such as You-Tube or Wikipedia rather than formal searching for journal articles or other medical resources. Subscriptions for journal article access or peer reviewed medical information sites were scarce with cost again listed as a barrier and also a source of inequity.

“On YouTube we looking for procedures or listening to heart-sound. Sometimes looking for lecture.”

(Medical student)

Social media sites are used to share difficult or interesting cases, pictures or videos. Lecture notes were used frequently, either as handouts of digital presentations or photocopied projector slides, but were generally not available online.

With regard to learning specific skills including procedures, clinical examination and pathology traditional educational resources such as ward teachers and textbooks still dominate other modalities, including online learning (Figure 2). This is despite concerns raised by students in focus groups about the currency of knowledge available through these avenues.

Figure 2. Use of educational resources for learning in specific content areas by students at The University of Health Sciences, Lao PDR

Focus groups described access to only a small number of appropriate books in the university library which were perceived to be “too old”, “not in Lao” and insufficient in number.

“…there is only one copy of the text we need.”

(Nursing student)

Personal ownership of books is limited to those who can afford it. These books are commonly shared between friends and are stated to be predominantly written in Thai language.

Focus groups described their ideal properties of learning materials as easy to access, in the Lao language and up-to-date. Access included availability to all students prior to the relevant lesson, availability on their smart phone despite often limited internet bandwidth, and being available offline when the internet was not available. Lao language books, guidelines and other resources are needed.

B. Language

More than 80% (413/507) of participants stated that all, or the majority, of their learning resources were in Lao language. The staff cohort had an overall lower percentage to this, with 53% (10/19) stating that all or the majority of their resources were in Lao. Very few to no Lao language resources were used by 7.7% (39/507) of participants. Lao was the most popular preference of learning resource language (38%), yet strong preferences were also described for having the same resource in various combinations of Lao, Thai and English.

Focus groups described the difficulty of finding any Lao language learning materials. Students reported using mostly Thai language resources or English which was then translated through sometimes multiple steps.

“It is so hard to find Lao language. We put English into Google translate to make into Thai and then make this to Lao in our head.”

(Medical resident)

IV. DISCUSSION

Access to appropriate and adequate learning resources is a daily challenge for students and staff at UHS in Lao PDR with an overwhelming need for greater access to learning resources, which are up-to-date, in their local language and available both on and offline. The study highlights the rise of social media as a learning tool even in an environment where there are multiple barriers to internet access. At the same time there is still high use of traditional learning resources, including ward teachers and text books, for specific skills such as physical examination and procedures – even when other media, such as videos, may have potential advantages. Finally, the research illustrates the disadvantage caused by language capacity, in particular the inequity in access to resources due to second language skills with the same problems of access impacting on what staff teach.

Our study population is comparable to both the general Lao population and the UHS student population, with a response rate of over 60%. Most of the survey participants were from provinces outside of the capital city of Vientiane, which is consistent with the general Lao population (UN, 2015). Given the Lao population is relatively young compared to other countries (UN, 2015), it is notable that almost half of the participants were older than 25 years of age and many had worked between completing secondary school and commencing their university degree, demonstrating the financial strains that students must also contend with in order to complete their degrees.

Consistent with previous literature from low-income countries internet speed (Bjertness et al., 2016) and cost (Aboshady et al., 2015; Bjertness et al., 2016) are major challenges. The average internet download speed in Lao PDR is more than five times slower than the speed achieved in Australia (Thompson, Sun, Möller, Sintorn, & Huston, 2016). This is the average, with most students only being able to afford 2g or 3g internet for smartphones and therefore having much slower download speed making smartphone internet access impractical for learning. The cost of the internet for students was reported to be on average one US dollar per day which is a large relative expense compared to the Gross National Income (GNI) in Lao PDR of $1740 USD/year (The World Bank Group, 2017a). With advances in technology and internet access globally it may be anticipated that there will be improvement in internet access, speed and affordability. Sustainable Development Goal 9.c states an aim to provide universal and affordable access to Internet in the least developed countries by 2020 (United Nations Department of Economic and Social Affairs, 2014). However, countries such as Lao PDR who would benefit significantly from these changes, also often lag behind other countries in how quickly they can access them, or how quickly these solutions become affordable to those who need them.

Not dissimilar to well-resourced countries (Arnbjornsson, 2014; Avcı, Çelikden, Eren, & Aydenizöz, 2015; Cartledge, Miller, & Phillips, 2013; Hollinderbäumer, Hartz, & Uckert, 2013) a large percentage of Lao students use social media applications for study. In a study by Guarino et al. (2014), in a high resource setting, the main reason for use of social media was logistical purposes. Our study population commonly use social media for sharing case-based information raising a clear concern regarding confidentiality in what is an unmonitored social media environment. In addition to this there is difficulty moderating the content or quality of shared information. In a country where access to evidence-based information is challenging, there is a risk that false information and misconceptions are shared without correction. There is literature supporting the safe use of social networking in medical education (Cartledge et al., 2013; Cheston, Flickinger, & Chisolm, 2013). Whilst there is no existing policy within the Lao University on social media this study highlights the potential need for policy development. A university website or specific faculty or clinical websites could be better forums for sharing case-based information, but due to resource constraints these are presently not viable options. A final alternative is for learning resources to be made available through the social media platforms which are used and through which information is already shared.

Consistent with a previous study (Guarino et al., 2014) participants have stated a higher use of Google and Wikipedia over journal articles or other evidence based medical sites. The reason hypothesised in the study by Guarino et al. (2014) was possible diffidence of students, rather than access, which was commonly cited in our study. Along with ease of free access to Wikipedia, although not explicitly stated, Lao students may find the language accessibility of Thai and some Lao articles another motivating factor to use this site.

The preferred language for any learning resource in this context is understandably Lao. English is now recognised by the UHS as the most important second language for their graduates, and participants in our study recognise the need to access English language learning resources. However, the language competence of most students and staff means Lao resources remain vital, yet few exist. Our study highlights the time students are spending searching for and translating resources from other languages. In a country in which human resource capacity is still being built and the standard of high school and university education remains behind its regional neighbours (The World Bank Group, 2017b) – this is time that cannot afford to be lost. This is a potential source of inequity in medical education globally – whereby countries with the lowest English language literacy often have the greatest need for better educational resources, the least capacity to generate resources in their own language, and the largest gaps in education delivery. Addressing this deficit requires a strategic approach which optimises the impact and reach of any intervention, as well as the use of limited financial and human resources available to develop and translate materials and takes into account technological limitations.

There are multiple different strategies which could be considered to solve the problem of access to learning and teaching resources. One approach option is to translate desired textbooks into Lao language and have these readily available to students and staff. However, this consumes substantial amount of resources and time and by completion the texts themselves may outdate quickly. Support for faculty to review the quality and currency of lecture material, which could then be made available online, would build capacity of teachers and allow what is taught to be visible. Work is already being done to improve free access to online journal databases and evidence-based medical websites but these efforts are still hampered by difficulties with language or internet access. Alternatively, non-traditional methods such as blogging to create key Lao language resources, lecture materials and provide clear summaries may provide a solution that is more efficient and more easily adaptable over time. Furthermore, inviting the students to collaborate on these materials may empower them to learn, as they are currently disempowered through language barriers and lack of access to educational materials. This would facilitate students with higher English language and computer skills to support their peers. The significant difference in daily internet use between the medical cohort and other groups would need to be taken into account for any resources. The greater daily internet use by medical students compared to other student cohorts did not reflect greater access to Information Technology (IT) or the internet. It may reflect greater IT skills among the medical cohort, or alternatively better understanding of the internet resources they can access, or usefulness of those materials.

Finally, the importance of teachers as facilitators of learning cannot be ignored. Students stated for all practical learning situations that ward teachers are the most used resource over text books and internet. Any e-learning resource created in response to this study must be seen as an adjunct to support teachers; to standardise and optimise the quality of teaching and learning content, facilitate information sharing but not replace teachers. Equipping teachers, who are vital learning resources for students, to face this challenge is paramount as they face the same challenges that students do in accessing information, acquiring second language skills and keeping both their technical knowledge and teaching skills current.

Potential limitations of the study include students feeling obliged to participate or unable to speak openly in focus groups due to involvement of UHS staff in the research team. We attempted to minimise this impact by using both quantitative and qualitative data methods to explore the same issues, ensuring students were given multiple opportunities to opt out of focus groups and keeping the focus group environment informal. Strengths of this study include the high response rate of participants with a combination of quantitative and qualitative data collected. Focus groups were important for students as they created discussions that enabled students to discuss and compare challenges. They were also a learning opportunity as students heard from each other how they studied and the different resources available.

Despite these limitations our study clearly demonstrates an acute issue of access to local language resources for teaching and learning among staff and students at the sole institution responsible for training doctors in Lao PDR. Whilst Laos may be somewhat unique in its lack of available local language resources, both for the general as well as the medical population -the problem of availability of medical texts or resources in a local tongue exists among other non-English speaking countries (Sabbour, Dewedar, & Kandil, 2010). Many of these countries may also be low resource countries who are reliant on donor funds, and donor priorities to determine what is funded. Solutions such as investment in English language capability of students take time to generate change, rely on changes to the basic education system, detract from time spent on medical content, disadvantage students from lower socio-economic backgrounds and may not lead to better outcomes overall (Dearden, 2014).  Furthermore, they create an imbalance when the English language capacity of the teachers may not receive the same investment. Solutions such as the increasing capacity of artificial intelligence to accurately translate online materials may be a more disruptive solution; but will also potentially require new approaches to appropriately integrating materials from diverse sources into curricula.

V. CONCLUSION

Our research highlights the daily challenges and the inequity faced by many non-English speaking countries in this time of exponential availability of information online. English language capacity, minimal local language resources and difficult access to information technology underpinned by the historical context of the Lao education system create enormous barriers for teaching and learning. For learning resources to be accessed sufficiently they need to be cost effective, language appropriate and device accessible. This will not only improve the learning for university students in Lao PDR, but the capacity of their teachers and also their future working selves in continuing professional development.

Notes on Contributors

Dr Annie L. Kilpatrick is a general paediatric trainee who worked as a paediatric clinical education fellow in Lao PDR in 2015 to 2016. She designed and conducted the surveys and focus groups, retrieved data, performed calculations, interpreted results, conceived and wrote the manuscript.

Dr Ketsomsouk Bouphavanh is the Director of The Education Development Centre and the Vice Dean of the Faculty of Medicine at the University of Health Sciences in Vientiane, Lao PDR. He was involved in conceiving and coordinating the research project and contributed to the manuscript.

Dr Sourideth Sengchanh is a paediatrician who works in The Education Development Centre at the University of Health Sciences in Vientiane, Lao PDR. He was involved in coordinating the project, facilitating and translating survey completion and focus groups and contributed to the manuscript.

Dr Vannyda Namvongsa is a paediatrician in Vientiane, Lao PDR who is heavily involved in teaching medical students and paediatric trainees. She was involved in coordinating the research project, facilitating and translating survey completion and focus groups and contributed to the manuscript.

Dr Amy Z. Gray is a consultant paediatrician and senior lecturer. She designed and conducted surveys and focus groups, contributed to writing and editing the manuscript.

Ethical Approval

Ethics approval for the project was obtained from the Lao National Health Ethics Committee (2015.76.NIOPH.72.VIE) and The University of Melbourne Ethics Committee (1544310).

Acknowledgements

This work was completed in full collaboration with the staff of The Education Development Centre, University of Health Sciences Lao PDR who facilitated the data collection. The authors especially acknowledge the time that the Lao students and staff spent completing the surveys and participating in the focus groups.

Funding

No funding source was required for this paper or research study.

Declaration of Interest

The authors report no declarations of interest.

References

Aboshady, O. A., Radwan, A. E., Eltaweel, A. R., Azzam, A., Aboelnaga, A. A., Hashem, H. A., … Hassouna, A. (2015). Perception and use of massive open online courses among medical students in a developing country: Multicentre cross-sectional study. BioMed Central Open, 5(1), e006804.
https:doi.org/10.1136/bmjopen-2014-006804 

Akkhavong, K., Paphassarang, C., Phoxay, C., Vonglokham, M., Phommavong, C., Pholsena, S. (2014). The Lao People’s Democratic Republic health system review (Series Ed.). Asia Pacific Observatory on Health Systems and Policies, 4(1). Manila: World Health Organization Regional Office for the Western Pacific. Retrieved from
https://iris.wpro.who.int/handle/10665.1/10448

Arnbjornsson E. (2014). The use of social media in medical education: A literature review. Creative Education, 5(24), 5.

Avcı, K., Çelikden, S. G., Eren, S., & Aydenizöz, D. (2015). Assessment of medical students’ attitudes on social media use in medicine: A cross-sectional study. BioMed Central Medical Education, 15(1), 18.

Bediang, G., Stoll, B., Geissbuhler, A., Klohn, A. M., Stuckelberger, A., Nko’o, S., Chastonay P. (2013). Computer literacy and e-learning perception in Cameroon: The case of Yaounde faculty of medicine and biomedical sciences. BioMed Central Medical Education, 13(1), 57.

Bjertness, E., Htay, T.T., Maung, N.S., Soe, Z.W., Aye, S.S., Ottersen, O.P., … Amiry-Moghaddam, M. (2016). E-learning resources in Myanmar. The Lancet, 388(10063), 2990-2991.

Cartledge, P., Miller, M., & Phillips, B. (2013). The use of social-networking sites in medical education. Medical Teacher, 35(10), 847-857.

Castro, F., Kellison, J., Boyd, S., & Kopak, A. (2010). A methodology for conducting integrative mixed methods research and data analyses. Journal of Mixed Methods Research, 4(4), 342–360.

Cheston, C. C., Flickinger, T. E., & Chisolm, M. S. (2013). Social media use in medical education: A systematic review. Academic Medicine, 88(6),893-901.

Dearden, J. (2014). The Conversation: Lessons taught in English are reshaping the global classroom. Retrieved May 2018, from https://theconversation.com/lessons-taught-in-english-are-reshaping-the-global-classroom-25944

Dodd, R., Hill, P.S, Shuey, D., & Fernandes Antunes, A. (2009). Paris on the Mekong: Using the aid effectiveness agenda to support human resources for health in the Lao People’s Democratic Republic. Human Resources for Health, 7(1), 16.

Duerden, J. (2017, January 25). Nascent book culture spreads in rural Laos. Nikkei Asian Review. Retrieved from
https://asia.nikkei.com/Location/Southeast-Asia/Myanmar-Cambodia-Laos/Nascent-book-culture-spreads-in-rural-Laos

Guarino, S., Leopardi E., Sorrenti, S., De Antoni, E., Catania, A., Alagaratnam, S. (2014). Internet-based versus traditional teaching and learning methods. The Clinical Teacher, 11(96), 449-453.

Hollinderbäumer, A., Hartz, T., & Uckert, F. (2013). Education 2.0—How has social media and web 2.0 been integrated into medical education? A systematical literature review. German Medical Science Journal for Medical Education, 30, 7-12. https://doi.org/10.3205/zma000857

Liamputtong, P. (2009). Qualitative Research Methods. Melbourne, Australia: Oxford University Press.

Milosavljević, N., Vuletić, A., Jovković, L. (2015). Learning medical English: A prerequisite for successful academic and professional education. Srpski Arhiv za Celokupno Lekarstvo [Serbian Archives of Medicine], 143(3-4), 237-240.

Papanna, K. M., Kulkarni, V., Tanvi, D., Lakshmi, V., Kriti, L., Unnikrishnan, B., … Sumit Kumar, S. (2013). Perceptions and preferences of medical students regarding teaching methods in a Medical College, Mangalore India. African Health Sciences, 13(3), 808-813.

Pavel, E. (2014). Teaching English for medical purposes. Bulletin of the Transilvania University of Braşov. Series VII: Social Sciences • Law, 7(56)(2), 39-46. Retrived from
http://webbut.unitbv.ro/BU2014/Series%20VII/BULETIN%20VII/06%20Pavel%202-2014.pdf

Pimmer, C., Linxen, S., Gröhbiel, U. (2012). Facebook as a learning tool? A case study on the appropriation of social network sites from mobile phones in developing countries. British Journal of Educational Technology, 43(5), 726-738.
https://doi.org/10.1111/j.1467-8535.2012.01351.x

Ranasinghe, P., Wickramasinghe, S.A., Pieris WR, Karunathilake I., Constantine, G.R. (2012). Computer literacy among first year medical students in a developing country: A cross sectional study. BioMed Central Research Notes, 5(1), 504.

Sabbour, S. M., Dewedar, S. A., & Kandil, S. K. (2010). Language barriers in medical education and attitudes towards Arabization of medicine: Student and staff perspectives. Eastern Mediterranean Health Journal, 16(12), 1263–1334.

Thompson, J., Sun, J., Möller, R., Sintorn, M., & Huston, G. (2016). Akamai’s [state of the internet], Q1 2016 report. Cambridge, MA: Akamai Technologies, Inc. Retrieved from
https://www.akamai.com/kr/ko/multimedia/documents/state-of-the-internet/akamai-state-of-the-internet-report-q1-2016.pdf

United Nations Department of Economic and Social Affairs. (2014). Open Working Group proposal for Sustainable Development Goals. Retrieved May 2018, from
https://sustainabledevelopment.un.org/sdgsproposal.html.

United Nations. (2015). Country Analysis Report: Lao PDR: Analysis to inform the Lao People’s Democratic Republic – United Nations Partnership Framework 2017-2021. Retrieved May 2018, from http://www.la.one.un.org/images/Country_Analysis_Report_Lao_PDR.pdf

Widyandana, D., Majoor, G., & Scherpbier, A. (2010). Transfer of medical students’ clinical skills learned in a clinical laboratory to the care of real patients in the clinical setting: The challenges and suggestions of students in a developing country. Education for Health (Abingdon, England), 23(3), 339.

The World Bank Group. (2017a). Data for Lao PDR, Lower middle income [Data]. Retrieved from
http://data.worldbank.org/?locations=LA-XN.

The World Bank Group. (2017b). Education [Data]. Retrieved from https://data.worldbank.org/topic/education?locations=LA-TH-VN-KH-MM

*Amy Z. Gray
50 Flemington Road,
Parkville Victoria 3052, Australia
Phone: +61 3 9345 4647
Facsimile: (03) 93456667
Email: amy.gray@rch.org.au

Published online: 7 May, TAPS 2019, 4(2), 32-38
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2054

Stefan Kutzsche¹ & Erwin Jiayuan Khoo²

¹Centre for Education, International Medical University, Malaysia; ²Clinical School, International Medical University, Malaysia

Abstract

We are reporting the results of implementing Learning from Observation and Discussions at Clinical Campus International Medical University, Kuala Lumpur, Neonatal Intensive Care Unit (NICU). This initiative was conceived and successfully implemented with the aim to identify medical students’ learning perception from self-reported learning experiences. A total of 80 semester eight medical students were invited to participate in the study. A structured, validated and reliable instrument developed from a work skill development framework was used to assess students’ perception of learning through discussions and observations (Total D&O), input from their experience providing future ideas (Total Ideas) and guided ward rounds as a new learning format (Total Visit). Informed consent was obtained from 42 students who participated over the ten-month period of the study. Data was analysed with ANOVA and structural regression equation modelling. The study showed that both Total Visit and Total Discussion & Observation can predict Total Perception of Learning. According to student evaluations, the Total Visits rating was the best single predictor summarising positive perception of rounds at the neonatal intensive care unit based on the significance values, partial eta squared and power. Students ranked the process of guided rounds at the neonatal intensive care unit as valuable in providing educational experiences and integral to their learning perception.

Keywords:            Perception of Learning, Bedside Teaching, Clinical Neonatology, Observational Learning

Practice Highlights

• The study seeks to test whether observational learning by medical students can predict a student’s perception of learning at NICUs.
• Students benefit and learn from active and purposeful NICU rounds supervised by a neonatologist.
• Total Visits rating was the best single predictor of a positive perception of NICU rounds based on the significance values, partial Eta squared and power.
• The study supports the adoption of observation-based learning exercises to augment the traditional case presentations in medical student training

I. INTRODUCTION

Despite a decline in practice, bedside teaching (BST) remains an important component of education for students of medicine and other health professions in helping to develop knowledge, skills and attitudes (Peters & Ten Cate, 2014; Stickrath et al., 2013). BST can be adapted to various clinical departments including the neonatal intensive care unit (NICU). Approved learning outcomes for fourth-year medical students rotating at the Special Care Nursery (SCN) involve demonstration of medical knowledge, comprehension of pathophysiology and formulating management plans.

Although it is widely accepted that medical students benefit and learn from active and purposeful NICU rounds, there is little prospect of learning proficiently without guidance and a purposeful curriculum (Biggs & Tang, 2011). Some teaching hospitals even restrict access to NICUs for medical students due to the risk of infections or possible conflicts of interest with other health professionals and parents. Hence, students are more likely to refer to ward rounds at the SCN where clinically stable infants may need feeding training, rooming-in, phototherapy or antibiotics, and are usually expected to be discharged within a few days or weeks. In contrast, NICUs are designed for newborns in need of specialised, high-tech medical and nursing care, including respiratory support. The unit provides care for the most complex conditions in the neonatal period, and may also include surgical care and transport of critically ill newborn infants. The NICU experience of medical students includes observational learning, sharing of knowledge, good practice, and identifying ethical issues. However, patients in the SCN and NICU are difficult to enlist for autonomous and self-directed learning. It is evident that clinical teaching in the NICU where hands-on practice is limited, and multiple learning models co-exist, includes understanding, application, critical, thinking, creativity and communication in addition to formal learning (Bannister, Hilliard, Regehr & Lingard, 2003). The clinical teacher, therefore, performs a key and highly demanding role in ensuring that learning outcomes are met.

A patient near learning perception of bedside teaching and demonstration in the NICU is rarely examined. We, therefore, looked at policy interventions that could improve education quality and student learning in an upper-middle-income country. We asked the question: ‘How does a guided NICU round impact medical students’ perception of learning neonatology?’ The hypothesis (H¹) was, there is a significant relationship between Total Perception of Learning with a) discussions based on immediate observations (Total D&O), b) a student´s future ideas based on the bedside experience (Total Ideas) and c) the NICU round as a new format of observational learning (Total Visit). The H⁰ was there is no significant relationship between Total Perception of Learning with Total D&O, Total Ideas and Total Visit.

II. METHODS

A. Study Design and Setting

The study was conducted as part of a programme evaluation in neonatal postings at the International Medical University (IMU), Clinical Campus, Seremban, Malaysia. Fourth-year medical students engage in an eight-week rotation in paediatric medicine. The programme features a one week posting in neonatal medicine which includes a facilitator guided NICU round. Groups of 5 or 6 students assess patients through observation and discussions with the facilitator. The students do not interact with visiting parents or get too close to procedures performed by ward staff. Patient records are available at the bedside to provide specific information for the discussion. After a briefing, students are introduced to a selection of NICU patients e.g. newborn infants with problems associated with extreme prematurity (GA 24-26 weeks), extremely low birth weight (< 1000g), respiratory distress syndrome, sepsis, persistent pulmonary hypertension of the newborn, asphyxia or congenital malformations. The rounds end with a reflection and debrief session. The total time spent for the NICU bedside teaching was 70 minutes. The single facilitator endeavour (an experienced neonatologist with competence in health professions education) throughout the NICU rounds for all groups ensured internal standardisation and accountability of teaching and instruction for all participating students.

B. Survey of Student Learning Perception

Respondents completed a consent statement outlining data transfer practices, privacy practices, and other relevant policies at the university before being allowed to take part in the survey. This study received institutional review board (IRB) exemption. Institutional agreement to conduct the study was obtained as part of the ongoing course evaluation and improvement programme.

On conclusion of the NICU teaching round students were invited to respond to an online questionnaire (Survey Monkey) covering a) how they perceived learning during NICU rounds, b) how discussions and observations supported their learning in neonatology, and c) what future perspectives they were reflecting on d) the NICU visit as a supplementary learning activity. The survey was based on a Work Skill Development Framework (Bandaranaike & Willison, 2010). Key themes of the framework are clinical education, clinical decision-making, role modelling, student presentations, establishment of trainee autonomy, and providing a safe learning environment. Items were related to the following domains: facilitate an understanding of a NICU round, resource utilisation, planning and management, life-long learning, problem-solving, critical thinking and applied communication skills in the NICU. The themes of perceived learning during NICU rounds was made up of 3 items that encompass comments related to students’ exposure, facilitation and students’ anxiety. The 3 items of student learning include knowledge and understanding, student-staff relationship and thinking positively. Learning from Discussions and Observation address rules and responsibilities, care of very low birth weight infants, ethics and teamwork. Improving staff relationship, communication with parents and staff and family involvement makes up the themes of possible impact on future ideas.

Students’ autonomy level reached from ‘highly guided’ where expectations are explained to ‘higher levels’ of student self-determination. To predict possible future use of understanding and learning there was a cognitive focus to the learning domain (social, emotional and cultural intelligence).

The format of a typical five-level Likert item was applied. Items were validated within the domains, and a Cronbach alpha score of 0.89 confirmed the internal consistency of the questionnaire. Mean values and standard deviations were calculated from the response data.

C. Data Analysis

Correlation between Total Perception of learning with a) Total D&O, b) Total Ideas) and c) Total Visit was performed by applying the Pearson test (Table 1). A score > 0.8 indicated that the dimensions are measuring similar concepts. Conversely, low scores were associated with concepts that were distinct from one another.

Analysis of Variance (ANOVA) and regression analysis were used to analyse the impact of the Total D&O, Total Ideas and Total Visits on Total Perception of Learning. A significant impact was indicated by an F-ratio higher than the critical value and a significance value < 0.05. The relationship between Total Perception of Learning and the three predictors was analysed using coefficients. Beta values achieved via regression analysis tested whether the predictors have a positive impact on Total Perception, and how the model significantly predicts the outcome using the regression equation. In order to ensure that the variables are sufficiently independent of each other, collinearity measures were tested to estimate the degree of redundancy using the Tolerance and Variance Inflation Factor (VIF) values for determining multicollinearity (Table 3). The observed power indicated the reliability and accuracy of results.

The medical students taking part in the survey also provided open written comments to evaluate the learning environment and the guidance received from the clinical teachers. These comments were used to support the findings obtained from the quantitative analyses.

III. RESULTS

A total of 42 students out of 80 responded over a period of ten months.

Table 1 shows the correlation between the variables studied in this research. Scores above 0.8 denote a strong relationship between the variables. Similarly, low scores indicate that the variables have smaller effects on one another.

Table 2 shows the results of the ANOVA test. The F-ratio value indicates the extent to which the predictors (Total D&O, Total Ideas and Total Visit) impact on the dependent variable (Total Perception of Learning). The significance value < .001 shown in Table 2, is presented below. The significance value should be < 0.05 for the model of regression which is shown in Table 3. ANOVA is used to measure significant differences between the means of two groups of data. Ideally, a large F-ratio value and a significance value < 0.05 is sought for the model of regression to be valid. The results in Table 2 indicate a significant impact of the Total D&O, Total Ideas and Total Visits on a student’s Total Perception of Learning with F-ratio = 58.3 and Sig < .001 confirming the above.

Note: a. Predictors: (Constant)
Total D and O (discussions based on immediate observations)
Total Ideas (a student´s future ideas based on the bedside experience)
Total Visit (the NICU round as a new format of observational learning)
b. Dependent Variable: Total Perception of Learning
Table 1. Correlational statistics between Total Perception, Total Visit, Total D and O and Total Ideas

Note: Model 1 measures the overall difference between the data and the values predicted by the estimation model.
The residual is the measure of the distance from the data point to the regression line.
a. Predictors: (Constant)
Total D and O (discussions based on immediate observations)
Total Ideas (a student´s future ideas based on the bedside experience)
Total Visit (the NICU round as a new format of observational learning)
b. Dependent Variable: Total Perception of Learning
Table 2. ANOVA table

 Note: a. Dependent Variable: Total Perception of Learning
Table 3. Table of coefficients of the regression model

Table 3 shows the results obtained after applying the regression analysis, representing the respective coefficients. The beta values provide a measure of the contribution of each dimension to the model. The B values indicate the relationship between Total Perception of Learning and the three predictors (Total D&O, Total Ideas and Total Visit). They are used to generate the unstandardised equation of the line of best fit (Equation 1) which represents the degree to which each independent variable contributes to predicting the Total Perception of Learning.

Y (Total Perception) = 7.485 + 1.083 (Total Visit) + 0.493 (Total D and O) – 0.086 (Total Ideas)

Equation 1. Unstandardised equation of the line of best fit

The equation indicates that Total Visit (the NICU round as a new format of observational learning) makes the greatest contribution to the Total Perception of Learning. There was no significant evidence to favour the contribution of Total Ideas on Total Perception of Learning.

Inspection of the values for ‘t’ in Table 3, the highest t-value is related to Total Visit which conforms the results shown by the line of best-fit equation. Collinearity statistics refer to the degree of overlap between the dimensions being measured (redundancy). The Tolerance value and VIF values show no multi-collinearity which tends to confirm the absence of a relationship between the independent variables selected in this study.

Note: Dependent Variable – Total Perception of Learning
a. R Squared = 0.975 (Adjusted R Squared = 0.896)
b. Computed using alpha = .05
Table 4. ANOVA tests between-subjects effect

The results presented in Table 4 show the observed power for the Total Visit variable is 0.963. The values are reliable and the significance value is 0.005. Hence, the impact of Total Visit on the Total Perception of Learning is highly significant at the 95% confidence level. The observed power for Total Discussion and Observation and Total Ideas are 0.868 (reliable) and 0.441 (moderately reliable) respectively. The significance value, for Total Discussion and Observation, was 0.014, however, the value for Total Ideas (0.188) indicates that the latter factor is not a significant predictor of Total Perception of Learning among the students.

The partial Eta squared values, (Total Visit 0.818, Total Discussion & Observation 0.720) indicate that Total Visit has a higher effect and is a more statistically significant predictor of Total Perception of Learning compared to Total Discussion and Observation.

IV. DISCUSSION

 Learning is a change in behaviour, skills and knowledge due to experience. Observational learning should include visual, auditory, verbal and kinaesthetic experiences to enable students to experience learning across different modalities. The present study analysed the process of learning by investigating student perception of learning (Total Perception) gained through discussion and observation (Total D&O), the student´s thoughts on future application of knowledge (Total Ideas) and their NICU round experience (Total Visit). The study showed that the NICU Visit and Discussion & Observation guided by the clinical teacher impact on Total Perception of Learning. According to student evaluations, the Total Visits rating was the best single predictor of a positive perception of learning arising from NICU rounds.

Our findings suggest that interventions are likely to be more effective at improving student learning when observational learning, clinical guiding, discussions and feedback are factored in the design of education strategies. Students placed significant value on NICU exposure which linked knowledge to a real clinical environment. They appreciated the contextual authenticity in supporting a positive learning experience, supporting Hunter’s (2008) recommendation that time should be allocated to give students the experience of NICU providing a safe practice environment that is built on a foundation of knowledge, learning and reflection (Hunter, Spence, McKenna, & Iedema, 2008).

It is accepted that students often experience difficulties in making the transition from theory to practice, which may reflect a failure of the training program to integrate these two components in ways that are relevant and meaningful to the students (Wrenn, J. & Wrenn, B., 2009). Students prefer experiential learning and need to be independent and self-directed. Adult learners may engage in learning because they like it and simply enjoy finding out about things (Merriam, 2001). The fact that students are exposed to clinical practice may strongly influence the way they develop their own learning (Pollard, 2008). The study showed that NICU-integrated learning drives students to adapt to an authentic hospital environment and social norms, thus improving insight into patient-centred care, effective communication, trust and teamwork.

If the teacher individualises and personalises learning as much as possible, letting the adult learner accomplish assignments that pertain to the real-life observation will provide a natural inclination to learn from NICU rounds (Holyoke & Larson, 2009).

Students appreciated the NICU face-to-face guidance and the briefing/debriefing sessions. The teaching focus emphasises the teachers’ clinical knowledge, as a subject expert, and how it can be transmitted efficiently to the students. Students found that immediate feedback and the discussion sessions enhanced the quality of the learning experience, and motivated them to learn. More comprehensive indicators of student learning may go beyond fundamental learning objectives and include student perceptions of their increase in interest in the subject, critical thinking, interpersonal outcomes and self-esteem (Koon & Murray, 1995).

There are few opportunities in medical student training programs to integrate academic learning with practice in the NICU, which could motivate learners to strive for academic and clinical excellence. The study highlights the value of BST and observational learning as an adjunct to traditional case presentations and assessment. The study also demonstrates that the quality of the BST experience in the NICU is related to student preparation for the encounter, governance, soft skills competency, expert teacher-guidance and debrief sessions. However, it must be emphasised that the NICU is a place for skilled health professionals to care for severely ill newborn infants, and the best interests of the patient outweigh those of the educational endeavour.

We adopted the single facilitator endeavour, being cognizant of the fact that effectiveness and reliability of instruction and student learning outcome depend on standardisation and accountability of teaching. Internal standardisation among a group of clinical facilitators depends on collaboration and development of a common and agreed understanding and application of standards of teaching, instruction and feedback.

A. Limitations
The overall response rate to the survey was 52.5%, which may be explained by other commitments (e.g. preparing for exams). Investigating how students learn in the NICU environment may be influenced significantly by the clinical environment itself and by the formal curriculum. A student’s first encounter with a neonatal posting may be overwhelming because they have much to learn as a new subject, and the duration of the posting may not be long enough. In the context of learning, students selected from semester 10 rather than semester 8 cohorts, may have adapted more quickly to the real-life situation in the NICU. In this study, the strategy focused on the students’ ability to understand clinical practice in accordance with presented information led by teacher-guided experience. Thus, the teacher plays a key role in determining a student’s learning and positive perception of learning during NICU rounds and in ensuring that learning outcomes are met. The cognitive focus of the present study rested on a student’s perception of his/her learning instead of the assessment of learning outcome of knowledge gain. According to a study by Hulland et al. (2017), the purposes of rounds identified by medical students were a) communication, b) medical education, c) patient care and d) assessment. The absence of assessment of knowledge gain in informal learning such as NICU rounds may present a problem area in student education.

Supervised clinical observation which is not hands-on in the NICU setting in this study enhances medical students’ learning perception of neonatology practice. Looking into the incorporation of the undergraduate neonatal curriculum in high-income countries such as Australia, Scandinavia, UK and US, the results of this study are not surprising. Weighing evidence of these results can be difficult because students have different understandings of the cultural, religious and social context that may lead to response bias. A response bias can be controlled by using interval questions, baseline data or data from a control group. However, baseline measurements were not available, because IMU students did not see critically ill newborn infants at the NICU prior to this study. Comparison with control group data was not an option because incorporating unequal educational strategies in the same cohort is a significant ethical concern and may offend the cognitive, intellectual and moral progress of student learning. Although this is a small study with the limitations mentioned, the pattern of findings raises intriguing issues for planning teaching strategies needed to guide the development of an undergraduate neonatal curriculum for medical students.

V. CONCLUSION

The study showed that NICU rounds as a format for observational learning by medical students and the clinical expert-guided discussions based on immediate observations can predict a student’s learning perception. According to student evaluations, the rating of the NICU round was the best single predictor of a student’s positive perception of learning. The survey demonstrated a fortification of students’ understanding of perinatal and neonatal health issues through contextual authenticity in the NICU learning environment. The findings support the adoption of observation-based learning exercises to augment traditional case presentations in medical student training. Non-formal learning strategies can help create an innovative and supportive clinical learning environment for medical students that facilitates and promotes the achievement of learning outcomes. Moreover, in the context of an upper-middle-income country, participation or incentives that shift educational preferences and learning behaviours in neonatology is a promising strategy to improve learning perception of students.

Notes on Contributors

Stefan Kutzsche (S. K.) and Erwin Khoo Jiayuan (E. K. J.) were involved in the design of the work. S. K. was the single NICU facilitator during the study period and administered data collection through Survey Monkey.

S. K. and E. K. J. conducted data analysis and interpretation. Both drafted the article, revised it critically and approved the final version for possible publication in TAPS.

Ethical Approval

This study was exempted by Institutional Review Board (IRB).

Acknowledgements

The authors wish to thank fourth-year medical students who had participated in this study. We would also like to acknowledge Dr Niusha Shafiabady and Shamir Das for their statistical assistance.

Funding

Authors had no funding to declare.

Declaration of Interest

The authors declare that they have no conflict of interest.

References

Bandaranaike, S., & Willison, J. (2010). Work skill development framework: An innovative assessment for work integrated learning. In Campbell, M. (Ed.), Work integrated learning: Responding to challenges: Proceedings of the 2010 Australian Collaborative Education Network (ACEN) National Conference (pp. 1-19). Perth, Australia: ACEN. Retrieved from http://cdn1.acen.edu.au/wp-content/uploads/2012/07/ACEN-2010-Proceedings.pdf

Bannister S. L., Hilliard R. I., Regehr G., & Lingard L. (2003). Technical skills in paediatrics: A qualitative study of acquisition, attitudes and assumptions in the neonatal intensive care unit. Medical Education, 37(12), 1082-90.
https://doi.org/10.1111/j.1365-2923.2003.01711.x

Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: What the student does (4th ed.). Berkshire, England: McGraw-Hill Education.

Holyoke, L., & Larson, E. (2009). Engaging the adult learner generational mix. Journal of Adult Education, 38(1), 12-21.

Hulland, O., Farnan, J., Rabinowitz, R., Kearns, L., Long, M., Monash, B., … Fromme, H. B. (2017). What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and students. Journal of Hospital Medicine, 12(11), 892-897. http://doi.org/10.12788/jhm.2835

Hunter, C. L., Spence, K., McKenna, K., & Iedema, R. (2008). Learning how we learn: An ethnographic study in a neonatal intensive care unit. Journal of Advanced Nursing, 62(6), 657-664. http://doi.org/10.1111/j.1365-2648.2008.04632.x

Koon, J., & Murray, H. G. (1995). Using multiple outcomes to validate student ratings of overall teacher effectiveness. The Journal of Higher Education, 66(1), 61-81.
http://doi.org/10.1080/00221546.1995.11774757

Merriam, S. B. (2001). Andragogy and self‐directed learning: Pillars of adult learning theory. New Directions for Adult and Continuing Education, 2001(89), 3-14.
http://doi.org/10.1002/ace.3

Peters, M., & Ten Cate, O. (2014). Bedside teaching in medical education: A literature review. Perspectives on Medical Education 3(2), 76-88. http://doi.org/10.1007/s40037-013-0083-y

Pollard, K. C. (2008). Non‐formal learning and interprofessional collaboration in health and social care: The influence of the quality of staff interaction on student learning about collaborative behaviour in practice placements. Learning in Health and Social Care, 7(1), 12-26.
http://doi.org/10.1111/j.1473-6861.2008.00169.x

Stickrath, C., Noble, M., Prochazka, A., Anderson, M., Griffiths, M., Manheim, J., … Aagaard, E. (2013). Attending rounds in the current era: What is and is not happening. Journal of the American Medical Association Internal Medicine, 173(12), 1084-1089. http://doi.org/10.1001/jamainternmed.2013.6041

Wrenn, J., & Wrenn, B. (2009). Enhancing learning by integrating theory and practice. International Journal of Teaching and Learning in Higher Education, 21(2), 258-265.

*Stefan Kutzsche
International Medical University
Address: 126 Jln Jalil Perkasa 19 Bukit Jalil
57000 Kuala Lumpur, Malaysia
E-mail: drignana@gmail.com

Published online: 7 May, TAPS 2019, 4(2), 25-31
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2072

Yeleswarapu S. Padmini, Pratibha K. Agarwal & Lourdes M. Daniel

Department of Child Development, KK Women’s and Children’s Hospital (KKH), Singapore

Abstract

Introduction: The aim of the study was to evaluate the efficacy of the developmental paediatrics teaching programme for undergraduate medical students posted to the Department of Child Development (DCD), in a tertiary children’s hospital in Singapore.

Methods: Efficacy of the teaching program was evaluated objectively using multiple-choice questions (MCQs) and subjectively using a self-assessment evaluation form developed in-house. Feedback was obtained from the students and tutors at the end of the posting.

Results: 36 students participated in this evaluation. There was a significant positive difference in the pre and end of posting MCQ scores (p < 0.001). The difference in the students’ self-assessment of their knowledge and skills of child development before and at the end of their DCD posting was similarly positively significant (p < 0.001). Feedback from the students using a series of open-ended questions indicated that they had a very positive learning experience and this included learning beyond their stated learning objectives. Feedback from the tutors was reflective of some of the challenges they faced while teaching.

Conclusion: The integrated approach to teaching developmental paediatrics that includes didactic, reflective, interactive and problem-based learning results in a highly positive learning experience and enhances both the knowledge and skills in child development. This enables delivery of a holistic curriculum that goes beyond teaching mainly routine developmental assessment.

Keywords:            Developmental Assessment, Developmental Paediatrics, Problem-based learning, Didactic Teaching, Medical Students

Practice Highlights

• Teaching medical students Developmental Assessment should not be limited to teaching them to conduct developmental assessment. They should be able to analyse child development in the context of impact on the functional ability in the child.
• Students should have the opportunity of engaging with not only families of neurotypical children, but those with disabilities, for them to understand the impact on the family.
• For the tutors, there is greater value in using various modalities of teaching approaches when teaching developmental paediatrics.
• Teaching can be incorporated within busy clinics through reflective learning.

I. INTRODUCTION

Increasing numbers of children are being identified with developmental and/or behaviour problems (Centers for Disease Control and Prevention, 2018). This is partly due to increased awareness of both the parents and clinicians, as well as increased survival of children born prematurely with complex medical conditions (Boyle et al., 2012). Primary care physicians are tasked with early identification of children with developmental delays/disorders during routine or opportunistic health surveillance consults. Decades of rigorous research indicates that early identification of developmental delays and timely early intervention can positively alter the child’s long term developmental trajectory (Oberklaid, Wake, Harris, Hesketh, & Wright, 2002; Scherzer, Chhagan, Kauchali, & Susser, 2012; Shevell, Majnemer, Platt, Webster, & Birnbaum, 2005). There is, however, evidence to suggest gaps in the knowledge and confidence of primary care providers in identifying and managing children presenting with developmental delays (Hastings, Lumeng, & Clark, 2014; Nicol, 2006). This could possibly be attributed to a lack of adequate training opportunities in child development during and after medical school (Nicol, 2006).

Medical education is constantly evolving to meet the demands of increasing numbers of students and changing emphasis on teaching and learning, with the eventual goal to improve patient care and satisfaction. The role of the educator, therefore, is to teach and guide a trainee/medical student to translate basic clinical knowledge into effective patient care. In the context of developmental paediatrics, medical students need to develop the knowledge and skills to take a comprehensive developmental history, identify developmental delays/disorders and develop clinical reasoning skills to evaluate further in terms of investigations and further management (Bellman, Byrne, & Sege, 2013). In addition, they should be able to understand the impact of the child’s disability on the child’s functioning as well as the impact on the family (Feyereislova & Nathan, 2014; Tracy & Graves, 1996). This holistic approach to patient management will enable young doctors of tomorrow to appreciate the critical role that families play in enhancing the developmental monitoring and support for their children. Most of the studies so far have evaluated teaching students to identify normal developmental milestones and conduct developmental screening using modalities such as didactic lectures and videos (Feyereislova & Nathan, 2014; Nicol, 2006). There is a paucity of studies addressing a more holistic developmental paediatric teaching programme for medical students.

With increasing emphasis on the clinicians to take on the role of educators, some of the challenges that the tutors face as clinical teachers include increasing student numbers, lack of adequate teaching opportunities in a busy clinic, tutor fatigue, difficulties recruiting patients for teaching purposes and lack of adequate training opportunities to prepare the tutors to be teachers (Frazer et al., 1999; Taylor, Tisdell, & Gusic, 2007). We developed a developmental paediatric teaching programme for the medical students, to provide a more holistic curriculum, delivered against the challenges faced including limited time and resources. The curriculum was based on the Paediatric Core Curriculum (PCC) which was designed by the expert PCC committee of the Lee Kong Chian (LKC) medical school. The modalities of delivery of the curriculum were decided after consultations with the experts within the Department of Child Development (DCD). The aim of the study was to evaluate the efficacy of the integrated teaching programme in enhancing the skills and knowledge of undergraduate medical students posted to the DCD. Skill areas included the ability to perform a developmental assessment (DA), to identify developmental delays and develop a management plan. Knowledge areas included knowledge of normal and abnormal patterns of development including aetiology and risk factors, management of a child presenting with developmental delay, awareness of the role of therapists and the range of community services available to the child and family.

II. METHODS

A. Site and Participants

The teaching programme was conducted at the DCD for fourth-year undergraduate medical students of LKC school of Medicine attached to the department from September 2016 to January 2017. The students were posted in three streams of four groups per stream, with four to five students per group for a total duration of eight half-day sessions. 36 medical students from streams one and two participated in the evaluation. As there was no funding for this project and due to constraints of resources in terms of time and manpower, the evaluation was limited to the two consecutive streams only.

B. Pre-posting Process

Information on learning objectives as prescribed in the LKC curriculum related to child development was made available to the medical students prior to their posting. Other reading materials included articles as outlined in the LKC curriculum such as assessment and management of developmental delays, learning difficulties, autism and ADHD. The students were also given a DA milestones chart that was developed in-house. All the tutors were sent the learning objectives and the DA milestones chart to ensure consistency of teaching practices.

C. Teaching Programme

The goal of our teaching programme was to produce competent students who had the required skills and knowledge for conducting appropriate DA, identify developmental delays and be aware of the management of the same as outlined in the LKC learning objectives. The students’ schedule was planned based on the learning objectives and resources available in terms of time, space and manpower. The posting started with an introductory session on child development. The students were allotted to clinics which included new cases and follow up cases. Each group had one to two ward-based sessions, one problem-based learning (PBL) session and they attended one observation of standardised developmental screening session. At the end of the posting, there was a debrief session with the LKC lead tutor for the DCD. On average the time spent on the various teaching components was approximately:

1. Didactic lecture – 12%
2. PBL – 12%
3. Practical DA – 25%
4. Observation of standardised developmental screening – 12%
5. Reflective learning in clinic sessions – 34%
6. Debrief session – 5%

D. Teaching Methodology

An integrated approach to teaching was adopted which included didactic (Luscombe & Montgomery, 2016), interactive learning (Feyereislova & Nathan, 2014), reflective learning (Sandars, 2009) and PBL (Preeti, Ashish, & Gosavi, 2013). All these strategies have been proven to be efficacious on their own in enhancing the learning of the trainees. Given the extent of the learning outcomes to be achieved, it was felt that the teaching methodology should incorporate all the above modalities of teaching. The introductory didactic lecture reviewed the developmental history, normal patterns of child development (0-5 years), factors affecting development and identification and management of developmental delays. It included videos to demonstrate how to conduct a DA. Some of the videos were developed in-house while others were obtained from various websites (Pathways, 2013).

During the clinic sessions, the students had opportunities to conduct DA on the index patient, on their typically developing siblings if accompanied and time permitted, learn various aspects of child development including identifying and management of developmental delays and disorders, and understand disability and its impact on the child and family. This was partly facilitated by observing clinician-parent interactions and reflective learning after each clinic session. The students were tasked with participating in patient evaluation by using various in-house developed screeners such as language and literacy screeners. There was a session with the assistant psychologists who routinely conduct developmental screening using Ages and Stages Questionnaires (ASQ-3) or Brigance III screener (Moodie et al., 2014) for all new cases referred to the DCD. This session provided the students with an opportunity to learn the use of developmental screeners. The ward-based sessions provided the medical students more hands-on opportunities to conduct DA on both neurotypical children, as well as children with developmental delays.

For the PBL session, the students were divided into two groups of two to three students and each group was given a clinical case vignette at the beginning of their DCD posting. Each case vignette was accompanied by learning objectives for the students to explore medical, physiological, anatomical and basic science concepts that relate to the clinical scenario. The students were expected to work in the groups for self-study and the final discussion was facilitated by one of the tutors. This provided with the opportunity for collaborative learning. The discussion included a holistic evaluation of the given case, starting from history taking to identification and management including approach towards the child and family.

A debrief session was conducted by the LKC lead tutor at the end of each group posting to obtain verbal feedback from the students using a series of open-ended questions regarding their experience of the DCD posting. The main aim of this session was to identify any potential gaps in the teaching that would enable us to modify the teaching programme for further groups. It was also another opportunity to answer any queries that the students may have in relation to any aspect of child development.

The students were asked the following questions:

1. Did they feel that their learning outcomes were met, and if not why?
2. Which sessions did they find useful and why?
3. Which sessions did they not find useful and why?
4. What aspect of the posting did they appreciate the most?
5. Any further suggestions to enhance the learning opportunities for subsequent groups?

The responses were recorded on paper by the tutor. All the 36 students participated in the debrief sessions.

All the DCD LKC tutors were emailed to obtain feedback at the end of each stream posting. They were asked to state their experience of participating in teaching the medical students and identify the challenges they may have had if any. Only three out of the eight tutors responded back.

III. OUTCOME MEASURES

• Pre- and end of posting MCQs (n=10) included questions in relation to developmental milestones, screening tools for DA and red flags for delayed development. There were also case vignettes that covered the approach towards a child presenting with a developmental delay/ disorder.
• Self-assessment evaluation form with responses on a 5-point Likert scale ranging from 1 (poor) to 5 (excellent) to indicate knowledge and skills before and after the DCD posting. The knowledge and skills domain included 8 and 3 responses respectively (Table 1).
• Verbal feedback as obtained from the groups and written feedback from the tutors separately. No qualitative analysis of the same was conducted.

Table 1. Students’ self-assessment of knowledge between pre and end of the DCD posting (post)

IV. RESULTS

A. MCQs

There was a significant improvement in the overall score between the pre and end of posting MCQs with the scores improving from 7.0 ± s1.6 (pre-posting score) to 8.9 ± 0.9 (end of posting score), which was highly significant (p < 0.001) (Figure 1).

 

Figure 1. Pre and end of posting MCQs

Students reported a significant improvement in their self-assessment of knowledge and skills in relation to child development after the DCD posting and this was found to be statistically significant (Table 1). In the context of knowledge, the domain where the maximum positive change was noted was awareness of the range of services for pre-schoolers with developmental delays (paired t-test 20.2; p < 0.001) followed by management of a child presenting with developmental delays (paired t-test 17.7; p < 0.001). The domain of least change, albeit statistically significant was knowledge of routine developmental assessment and new-born screening (paired t-test 11.2; p < 0.001).

In the context of skills, all the three domains indicated a similar positive shift between the pre- and end of posting evaluation (p < 0.001).

B. Student Feedback

The feedback from the students was positive overall. In relation to the specific questions that were asked, the summary of the responses is as below.

1) Did they feel that their learning outcomes were met, and if not why? All the thirty-six students commented that their learning outcomes were met. In fact, they learnt well and above their stated learning objectives. Some comments included:

“I understand the liaison that’s needed with the various agencies working with the child and family, be it in the school, hospital or community.”

“I now understand the social and financial implications of having a child with developmental delay.”

2) What sessions they found useful and why? All the thirty-six students reported that they found new case clinics and ward-based sessions most useful as they had more opportunities to practise DA. In addition, some students reported that they liked the PBL session as this session helped them understand a case more holistically. Some comments included:

“Ward-based sessions helped understand development in typically developing children, while clinic sessions helped identify children with developmental delays and disorders.”

“PBL session was very useful. It forced us to ‘do our homework’. Brought us back to the basics. It helped to consolidate what we have learnt during the posting so far.”

3) What sessions that they did not find useful? Some students felt that they did not have adequate learning opportunities during busy follow-up clinic sessions.

4) What aspect of the posting do they appreciate the most? While some students commented that they appreciated the approachability and teaching provided by the tutors in the DCD the most, others commented that it was the social aspects of the various conditions and learning how to support and counsel the family they appreciated the most. Some comments included.

“The tutors were very approachable and willing to teach in spite of a busy clinic.”

“Exploration of social concerns and counselling were very insightful.”

5) Any further suggestions to enhance the learning opportunities for subsequent groups? Some of the suggestions that came from them included scheduling the DA lecture at the beginning of the stream posting rather than at the start of the DCD posting. This would give the students the knowledge and skills to enable them to practise DA even when posted in the other paediatric departments. They requested for more ward-based tutorials and a longer DCD posting.

C. Tutor Feedback

Although all the tutors were willing to teach the students, some of the challenges that they reported included lack of adequate teaching opportunities during clinics, especially when the clinics were busy. Some of the tutors felt that the clinical sessions such as autism specialist clinics were too specialised and hence are not suitable for the students. The tutors also did not have the time to commit to ward-based sessions on a regular basis. During the sessions when two students were attached to a clinic, some tutors felt that this could overwhelm the child and family and could affect the child’s DA.

V. DISCUSSION

Our study confirms the significant benefits of and enhancement of positive learning outcomes using an integrated approach of teaching developmental paediatrics to undergraduate medical students. The improvement covered a broad range of areas including knowledge of normal and abnormal developmental patterns, management of a child presenting with developmental delays, awareness of the range of services for such children and skills to conduct a DA and formulate a management plan for a child with developmental delay. These are the essential skills required of any practitioner working with children. The students had exposure to neurotypical children as well as to children with disabilities and their families during the posting. This helped them appreciate normal from abnormal patterns of development, as well as the social impact of the disability on the child and family. The end of posting feedback from the students reflected the learning of the students beyond the prescribed learning outcomes during the posting. Despite the challenges mentioned in the tutors’ feedback, their intent in teaching DA effectively and being able to provide a conducive teaching environment was evident from the positive feedback reported by the students and from the outcome measures.

It is being increasingly recognised that teaching developmental paediatrics should be holistic, and incorporate all the aspects of child development (McDonald, 2018; Tracy & Graves, 1996). While it is imperative that the students are familiar with the developmental milestones and how to elicit them, it is equally important for them to understand the functional aspects of the child’s developmental milestones. This will enable them to give appropriate anticipatory guidance to the parents/carers. It is also essential that the students are able to appreciate the functional impact of the child’s delays/disability on both the child and the family. This knowledge will enable to help them learn to counsel the parents appropriately. Teaching through exposure to children with special needs and their families will significantly improve knowledge and skills of the medical students (McDonald, 2018; Ryan & Scior, 2014; Whitehall, McCulloch, Edwards, & McDonald, 2015).

VI. LIMITATIONS OF THE STUDY

While some of the teaching sessions were structured, others that involved teaching within the clinics were unstructured and hence the students were exposed to various teaching modalities within these clinical sessions. At times during busy clinics, teaching the students could have been compromised, although they would still have had the opportunity to learn from clinician-parent/patient interactions. Although feedback was obtained from the students and tutors after the posting, qualitative evaluation of the same was not conducted. Less than 50% of the tutors responded back, hence the tutor feedback as stated above is not completely reflective of their experiences including challenges in teaching the students. The teaching curriculum was delivered through various modalities, which could not be individually evaluated in terms of the efficacy of each one of them separately. Functional aspects of child developmental milestones, which is a very important component of child development were discussed by the more experienced tutors during various sessions, but this was not evaluated. The student’s knowledge of developmental paediatrics was assessed through MCQs and no formal evaluation of skills was conducted in view of the short duration of the DCD posting. While we are encouraged by the positive results of our educational model, we would also acknowledge that we did not aim to demonstrate its superiority over other educational models as there was no such direct comparison in our study. Further research into comparing various educational models need to be explored.

VII. CONCLUSION

The teaching programme developed by the department was deemed to very effective by the undergraduate medical students, in enhancing their knowledge and skill in developmental paediatrics. Increasing student numbers would require teaching methodologies to constantly evolve, without having an impact on the quality of teaching or clinical care. While the above methodology of curriculum has worked well for smaller group setting, it may have to be modified as the numbers increase.

Notes on Contributors

Dr Padmini Yeleswarapu is a Senior Clinical Lecturer, Yong Loo Lin School of Medicine (YLLSoM), Adjunct Assistant Professor, Duke-NUS Graduate Medical School and is working as a Consultant Paediatrician, Department of Child Development, KK Women’s and Children’s Hospital. She is also a Clinical Teacher in Lee Kong Chian School of Medicine (LKCSoM).

Dr Pratibha Agarwal is a Senior Consultant in the Department of Child Development, KKH. She was a Senior Clinical Lecturer in YLL-NUS School of Medicine from 2009 to 2014, an Adjunct Assistant Professor in Duke-NUS from 2010 to 2014 and currently a Clinical Teacher in Lee Kong Chian School of Medicine (LKCSoM).

A/Prof Mary Daniel is the Senior Consultant and Head of Department of Child Development, KKH. She is an Adjunct Associate Professor of Duke-NUS, YLLSoM and LKCSoM; is a reviewer for the Annals of the Academy of Medicine, Singapore and on the Editorial Board of Journal of Clinical Pathology and Diagnosis.

Ethical Approval

Ethical approval was sought from the Institutional Review Board (IRB) and was deemed not necessary.

Acknowledgements

We would like to thank the medical students who participated in the study and Ang Cheng Hiang and Lek Evelyn for their support and help in data management.

Funding

The research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of Interest

The authors warrant that the article is original, does not infringe upon any copyright or other proprietary right of any third party, is not under consideration by another journal and has not been previously published. Authors have no conflicts of interest, including financial, institutional and other relationships that might lead to bias.

References

Bellman, M., Byrne, O., & Sege, R. (2013). Developmental assessment of children. British Medical Journal, 346, e8687. https://doi.org/10.1136/bmj.e8687

Boyle, E. M., Poulsen, G., Field, D. J., Kurinczuk, J. J., Wolke, D., Alfirevic, Z., & Quigley, M. A. (2012). Effects of gestational age at birth on health outcomes at 3 and 5 years of age: Population based cohort study. British Medical Journal, 344, e896. https://doi.org/10.1136/bmj.e896

Centers for Disease Control and Prevention, National Center for Health Statistics. (2018, April 9). Key findings: Trends in the prevalence of developmental disabilities in U. S. children, 1997–2008 [Webpage]. Retrieved from
https://www.cdc.gov/nchs/ppt/nchs2012/ss-22_blumberg.pdf

Feyereislova, S., & Nathan, D. (2014). How best to teach developmental assessment? A single-blinded randomised study. Archives of Disease in Childhood, 99(12), 1083-1086. https://doi.org/10.1136/archdischild-2013-305536

Frazer, C., Emans, S. J., Goodman, E., Luoni, M., Bravender, T., & Knight, J. (1999). Teaching residents about development and behavior: Meeting the new challenge. Archives of Pediatrics and Adolescent Medicine, 153(11), 1190-1194.
https://doi.org/10.1001/archpedi.153.11.1190

Hastings, E. A., Lumeng, J. C., & Clark, S. J. (2014). Primary care physicians’ knowledge of and confidence in their referrals for special education services in 3- to 5-year-old children. Clinical Pediatrics, 53(2), 166-172.
https://doi.org/10.1177/0009922813503036

Luscombe, C., & Montgomery, J. (2016). Exploring medical student learning in the large group teaching environment: Examining current practice to inform curricular development. BioMed Central Medical Education, 16(1), 184.
https://doi.org/10.1186/s12909-016-0698-x

McDonald, J. (2018). Twelve tips for teaching child development and disability to medical students. Medical Teacher, 40(2), 135-139. https://doi.org/10.1080/0142159X.2017.1365118

Moodie, S., Daneri, P., Goldhagen, S., Halle, T., Green, K., & LaMonte, L. (2014). Early childhood developmental screening: A compendium of measures for children ages birth to five. Retrieved from Administration for Children and Families, U.S. Department of Health and Human Services Website:
https://www.acf.hhs.gov/sites/default/files/opre/compendium_2013_508_compliant_final_2_5_2014.pdf

Nicol, P. (2006). Using the ages and stages questionnaire to teach medical students developmental assessment: A descriptive analysis. BioMed Central Medical Education, 6(1), 29. https://doi.org/10.1186/1472-6920-6-29

Oberklaid, F., Wake, M., Harris, C., Hesketh, K., & Wright, M. (2002). Child health screening and surveillance: A critical review of the evidence. Retrieved from
http://www.esdy.edu.gr/files/011_Ygeias_Paidiou/Koinonikh_Paidiatrikh_tei_mathimata/A6_Paidia_Screening_Australia_2002.pdf

Pathways [Screen name]. (2013, February 6). 6 month old baby typical and atypical development [Video file]. Retrieved from https://www.youtube.com/watch?v=sUO6baJ0FcA

Preeti, B., Ashish, A., & Gosavi, S. (2013). Problem based learning (PBL) – An effective approach to improve learning outcomes in medical teaching. Journal of Clinical and Diagnostic Research, 7(12), 2896-2897. https://doi.org/10.7860/JCDR/2013/7339.3787

Ryan, T. A., & Scior, K. (2014). Medical students’ attitudes towards people with intellectual disabilities: Literature review. Research Developmental Disabilities, 35(10), 2316-2328. https://doi.org/10.1016/j.ridd.2014.05.019

Sandars, J. (2009). The use of reflection in medical education: AMEE guide no. 44. Medical Teacher, 31(8), 685-695. https://doi.org/10.1080/01421590903050374

Scherzer, A. L., Chhagan, M., Kauchali, S., & Susser, E. (2012). Global perspective on early diagnosis and intervention for children with developmental delays and disabilities. Developmental Medicine and Child Neurology, 54(12), 1079-1084. https://doi.org/10.1111/j.1469-8749.2012.04348.x

Shevell, M., Majnemer, A., Platt, R. W., Webster, R., & Birnbaum, R. (2005). Developmental and functional outcomes at school age of preschool children with global developmental delay. Journal of Child Neurology, 20(8), 648-653.
https://doi.org/10.1177/08830738050200080301

Taylor, E. W., Tisdell, E. J., & Gusic, M. E. (2007). Teaching beliefs of medical educators: Perspectives on clinical teaching in paediatrics. Medical Teacher, 29(4), 371-376.
https://doi.org/10.1080/01421590701510553

Tracy, J., & Graves, P. (1996). Medical students and people with disabilities: A teaching unit for medical students exploring the impact of disability on the individual and the family. Medical Teacher, 18(2), 119-124.
https://doi.org/10.3109/01421599609034145

Whitehall, J., McCulloch, R., Edwards, M., & McDonald, J. (2015). Engagement of undergraduate medical students of paediatrics in special schools for children with disabilities. Journal of Paediatrics and Child Health, 51(8), 798-801.
https://doi.org/10.1111/jpc.12831

*Yeleswarapu Sita Padmini
Department of Child Development,
KK Women’s and Children’s Hospital
100 Bukit Timah Rd,
Women’s Tower, Level 5,
Singapore 229899
Email: yeleswarapu.padmini@singhealth.com.sg

Published online: 7 May, TAPS 2019, 4(2), 14-24
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2062

Chiara Marie Miranda Dimla, Maria Paz S. Garcia, Maria Petrina S. Zotomayor, Alfaretta Luisa T. Reyes, Ma. Angeles G. Marbella & Carolynn Pia Jerez-Bagain

Department of Pharmacology, College of Medicine, University of the East-Ramon Magsaysay Memorial Medical Center, Inc., Philippines

Abstract

Practice Highlights

• The Objective Structured Examination in Pharmacology (OSEP) is an objective assessment tool to further evaluate student performance in pharmacology.
• Congruent with OBE (outcome-based education), OSEP aligns assessment with the desired outcomes.
• A one-station oral exam patterned after Objective Structured Clinical Examination (OSCE) to test skill of choosing a p-drug for a paper case.
• Rubrics are developed and used to score, ensuring more reliability and validity of assessment.
• Almost all Pharmacology students expressed positive perception of the OSEP as an assessment tool.

I. INTRODUCTION

At the University of the East-Ramon Magsaysay Memorial Medical Center, Inc. (UERMMMCI) College of Medicine, Pharmacology is a yearlong subject, encompassing basic and clinical pharmacology, taught in the second year of the basic medical education program. It is designed to prepare medical students to prescribe drugs to patients on a rational basis. The methods of teaching used in the course are lectures, laboratory experiments, video presentations, role-playing, mini-cases and small group discussions (SGDs) using hypothetical cases. Student assessment is through written examinations, short quizzes, laboratory grades, and preceptorial grades using a rubric for small group discussions.

The SGDs are geared to develop active learning so students will better comprehend and apply the concepts that are discussed in the lectures. These give the students a chance to articulate themselves and compare their understanding of the different concepts. In addition, they hone their listening and interpersonal skills by collaborating with their peers and learn to search for, critically appraise and apply current literature. Hypothetical cases of commonly encountered clinical conditions are discussed for two hours per preceptorial session. Each paper case is developed and given prior to each SGD during which students come up with a diagnosis, enumerate therapeutic goals, and discuss non-pharmacologic, as well as pharmacologic interventions, which include prescription writing and giving patients advice. To arrive at these, the students apply the process of rational use of medicines as recommended by the World Health Organization (WHO) Guide to Good Prescribing, a process that involves decision-making based on the efficacy, safety, suitability and cost of several drug options (de Vries, Henning, Hogerzeil, & Fresle, 1994).

In view of this, assessing students’ individual competence with objectivity and fairness becomes a challenge for the faculty, more so since student performance in the SGDs is 8% of the entire grade in pharmacology.

There are several ways of assessing student competence. These may be formative, summative or both. Competence is a concept that includes not only knowledge, skills and attitude but also, problem-solving skills such as critical thinking and reasoning, that of being a team player as well as being able to communicate effectively, in all formats. Any of these may be the focus of assessment of competence (Ilic, 2009).

The OSEP is one tool that assesses the competence of the second year student on knowledge gained during the entire year in pharmacology, skills in applying this knowledge in choosing an appropriate drug, and in using communication skills during the one-on-one oral examination.

The Objective Structured Examination in Pharmacology (OSEP) was conceptualised as an additional assessment tool patterned after the Objective Structured Clinical Examination (OSCE) although its main focus is on the application of the knowledge and skill of students in rational drug prescribing. It is conducted near the end of each school year, in which the medical student of pharmacology randomly chooses a case similar to those previously discussed in the preceptorial case sessions and answers questions intended to find out whether he/she demonstrates knowledge of pertinent drugs and is able to apply these in rational drug prescribing. This form of assessment has been used by the UERMMMCI College of Medicine, Department of Pharmacology for the past 3 years. It was patterned after the widely-used and validated OSCE to assess the clinical performance of medical students (Harden, Stevenson, Downie, & Wilson, 1975; Zayyan, 2011).

In addition, the OSEP complies with the directive of the Philippines’ Commission on Higher Education via CMO No. 18 Series of 2016 for all Philippine medical schools to shift to outcomes-oriented approach (Commission on Higher Education, 2016). The OSEP enables a more structured and objective method of assessing if the student has achieved the outcome expected in pharmacology, which is to prescribe drugs rationally after consideration of the totality of the patient, the state of illness, the current management. It conforms more with the outcome-based curriculum since it allows the instructor to observe the student’s response to a hypothetical patient and assign a corresponding grade based on a prepared rubric. Rubrics are scoring guidelines used in outcome-based education (Reddy, 2007) which are also utilised in the OSCE. These intend to improve assessment of students, to make it more accurate, valid, reliable and fair thus, minimising bias in terms of grading the application of their knowledge from their small group discussions of cases.

Patterned also after the OSCE, an Objective Structured Practical Examination (OSPE) in pharmacology has been described and utilised in other countries, particularly in India. The OSPE consisted of several stations during which a student was appraised on cognitive and psychomotor skills acquired during the course. Through direct observation over a specified period of time, knowledge on prescription components, dose calculation, dosage forms, drug information sources, routes of drug administration and applied pharmacology, among others, were graded (Malhotra, Shah, & Patel, 2013; Vishwakarma, Sharma, Matreja, & Giri, 2016). A number of studies have shown the OSPE to be a valid and reliable assessment tool, not only in pharmacology, but in other basic medical science courses such as anatomy and physiology (Deshpande et al., 2013; Hasan, Malik, Hamad, Khan, & Bilal, 2009).

Student feedback, therefore, is vital in determining whether the OSEP is an effective tool to further assess their performance in pharmacology. This will also provide a way to improve the teaching and grading of the second year medical students in the course.

Furthermore, it is inherent that several factors will affect the taking of the OSEP like time allotment, student preparation and environment. Hence, this study not only intends to find out the perception of the second year medical students of the OSEP as an assessment tool but also their perception of the content and conduct of the OSEP.

The general objective of the study was to determine the students’ perception of the Objective Structured Examination in Pharmacology (OSEP) as a tool to assess what they have learned in pharmacology on the rational use of medicines and to determine the students’ perception of the content and conduct of the OSEP.

A survey tool containing sixteen statements was fielded to the students. It utilized a Likert Scale to measure the level of agreement to each statement (see Appendix).

The specific objectives were to determine: 1) the response rate; 2) the percentage distribution of students’ responses categorised into four levels of agreement to the survey statements: 1-“strongly disagree”, 2-“disagree”, 3-“agree”, and 4-“strongly agree”; 3) the median scores per survey statement; 4) the proportion of students with positive perception of the effectiveness, content and conduct of the OSEP; wherein positive perception was operationally defined as a median score for each survey statement.

For positive statements, median scores of 3 and 4 were analysed as positive perception while for negative statements, median scores of 1 and 2 were also analysed as positive perception. Conversely, the negative perception was a median score of 1 or 2 for positive statements and a median score of 3 or 4 for negative statements.

II. METHODS

The study proposal was developed and the survey tool was refined after pilot testing. The research protocol was then submitted and the approval of the Institutional Ethics Review Board was obtained before retrospectively collecting the data.

All second-year pharmacology students were eligible to be included in the study. There were no exclusion criteria. They were enjoined to participate in the survey as part of the post-activity evaluation that is integral in the college’s continuous initiative of curriculum development. Adequate orientation to the OSEP and that of the research study were done prior. Informed consent to utilise their survey responses in the study was requested. Only the responses of those students with informed consent were included in this paper.

Sample size estimation was done, using the available software tool in EpiTools epidemiological calculators from the internet site epitools.ausvet.com.au which followed the formula n = [(Z²)*P(1*P)]/e², wherein Z = value from standard normal distribution corresponding to desired confidence level (CI); (Z = 1.96 for 95% CI); P = expected true proportion which was set at 50%; and e = desired precision which was set at 5%.

The required sample size of 385 was exceeded with the total number of 414 participants using convenience sampling technique. This implies that the conclusions and recommendations derived from this study are valid and applicable to future batches.

The survey was conducted at the end of the School Year 2016-17, wherein the medical sophomores of the pharmacology course were asked to fill up and submit the survey tool. In the context of the need to conduct a post-activity evaluation, a survey of the students’ perception of the OSEP was conducted. During the orientation about the OSEP, the objectives, rationale and conduct of the said activity were relayed to the students ahead of time.

A 4-point Likert scale was utilised to generate responses on either side of the scale, eliminating neutral options, which would be difficult to analyse. While not the traditional 5-point scale as originally expounded by Rensis Likert, the 4-point scale is also widely used and accepted (Lee & Paek, 2014; Lozano, García-Cueto, & Muñiz, 2008). It was considered appropriate as the questionnaire measured attitudes toward an assessment method, not complex, variegated issues. Moreover, in the context of a post-activity evaluation, the 4-point Likert scale was a strategy to encourage responses to either agree or disagree on the items raised in each survey statement, especially since an option to write textual comments and recommendations was available at the time; although, not within the scope of this research paper.

Ethics approval of the study proposal was obtained July 12, 2017, prior to retrospective data collection. The approval was given by the Institutional Ethics Review Committee (ERC) of the Research Institute for Health Sciences (RIHS) under RIHS ERC Code: 0415/C/M/17/74. The students’ participation was voluntary and informed consent was obtained.

In anticipation of securing the ethics approval for the research undertaking, the following steps were observed: 1) the required data elements to meet the research objective were integrated into the survey form, which was pre-tested to a small group of third-year medical students who had taken the OSEP in the previous year; 2) a portion to obtain informed consent was included in the survey form; 3) the objectives and conduct of the anticipated research study were explained to the students; 4) the ethical rights of the students were explained; 5) measures to uphold ethical considerations were anticipated by the researchers. Once the ERB approval was obtained, the data was collected retrospectively to describe the student’s perception of the OSEP as an additional assessment tool and to generate hypotheses of factor relationships for future research undertakings. The responses of only those students who signed the Informed Consent form were included in this paper.

Descriptive statistics were generated from the collated data.

The decision to analyse the data into positive perception and negative perception by combining the affirmative and opposing responses to each survey statement stems from the context of a post-activity evaluation, wherein the direction of developing the OSEP in the pharmacology curriculum will be guided by the proportion of students to either side of the scale, irrespective of the intensity of their affirmation or opposition.

III. RESULTS

A. Profile of Participants

The second-year medical student participants were predominantly in the 20 to 25-year-old age group (90%) and single Filipinos. The male to female ratio was 1:2.

B. Effectiveness of the OSEP

For the six survey statements intended to elicit perceptions on the effectiveness of the OSEP as an assessment tool (Table 1), the median score was 3 and 4, which indicated a positive perception for the positive statements (Survey Statement No. 1 to 6, except 4).

Note: Percentage positive perception, X = a+b, except in 4* where X*= c+d
Table 1. Students’ perception of the OSEP as an effective assessment tool

Survey Statement (SS) no. 4 is a negative statement and initially, a positive perception was defined as a median score of 1 and 2. However, Table 1 shows that the median score for SS no. 4 is 3 and that there is a small difference between those with positive and negative perception. The distribution of responses reveals that about 60% of the students do “agree” and “strongly agree” to SS no. 4: “The OSEP made me realise that I lack the ability to prescribe drugs on my own.” This predominant perception should be taken positively because it made the students realise that they have to identify and improve on the other factors that will increase their ability to rationally prescribe independently, the right drug for the right patient, at the right dose and at the right time. To the summary statement SS no. 6: “I can say that the OSEP is an accurate assessment tool of my ability to choose the p-drug through the process of Rational Drug Therapy.”, ninety percent gave an affirmative response.

C. Content of the OSEP

For the four survey statements intended to elicit perceptions on the content of the OSEP as an assessment tool (Table 2), the results show a positive perception with a median score of 4 for the three positive statements and 2 for SS no. 8 which is a negative statement.

Note: Percentage positive perception, X = a+b, except in 8* where X* = c+d
Table 2. Students’ perception of the content of the OSEP

D. Conduct of the OSEP

For the six survey statements intended to elicit perceptions on the conduct of the OSEP as an assessment tool (Table 3), the results show a positive perception with a median score of 4 for the five positive statements and 2 for SS no. 13 which is a negative statement. The general perception was that there was ample preparation for the OSEP and that it was conducted in an objective and orderly manner with realistic expectations within the given time.

Percentage positive perception, X = a+b, except in 13* where X* = c+d
Table 3. Students’ perception of the conduct of the OSEP

The survey had an almost 100% response rate for each category (Figure 1). The interpretation of the median scores revealed that 88% have a positive perception of the OSEP as an assessment tool (Table 1). As to the content and conduct of the exam activity, 93% (Table 2) and 94% (Table 3), also have a positive perception of the OSEP.

 Figure 1. UERMMMCI’s pharmacology students’ perception of the OSEP

IV. DISCUSSION

In both teaching and learning, evaluation is important especially in health professions education wherein the emphasis on performance-based assessments is on testing complex learning which involves higher-order “knowledge and skills” (Peeters, Sahloff, & Stone, 2010). It behoves medical educators, therefore, to evaluate the effectiveness of any teaching or learning strategy.

Assessment, too, is an important part of the curriculum. It encompasses domains that need assessment, the tools to be used, the purpose of the assessment process, the timing of assessment, and the people who will do the assessment. It is a vital component of the learning process. It shows the progress of students, their strengths and weaknesses and guides them on which topics are important. Furthermore, it serves as a method for promotion and a measure of teaching effectiveness (Etheridge & Boursicot, 2017) and is an important factor that drives learning.

Assessment tools are characterised by validity, reliability, feasibility, cost, as well as educational impact. Not all domains of competency can be assessed, however, by a single assessment method. There should be a variety of assessment tools that must be used so that the advantage of one method can overcome the disadvantage of the other (Al-Wardy, 2010). Indeed, student assessment is a major concern among those teaching in the UERMMMCI College of Medicine, thus, a lot of ways of improving assessment methods to make them more valid and reliable are being done.

The OSEP is one assessment method that is used to further evaluate the performance of the second year medical student in pharmacology towards the end of the course. This is an added tool that may allow for students to be graded more objectively and fairly. This is also more congruent with outcome-based education (OBE), an educational design that advocates for assessment methods aligned with the desired outcomes.

One good feature of the OSEP is its use of rubrics in scoring a student’s performance because this ensures reliability and validity of the assessment. Also utilised in OSCE, rubrics are criterion-based assessment tools that make the evaluation process easier. They are a “set of scoring guidelines” which are in accordance with outcome-based education (Reddy, 2007). The use of rubrics in the assessment of performance has several benefits like consistency in scores, formation of learning and improved instruction. Through rubrics, feedback, as well as self-assessment, is made easy because they make criteria and expectations explicit. Making use of rubrics ensures a more reliable scoring of performance (Jonsson & Svingby, 2007). In the conduct of the OSEP, a great majority of the students perceived that the case scenario was clearly stated, that the questions were not confusing or ambiguous and that they were arranged logically (SS no.’s 7, 8, and 9). The systematic and organised delivery of the case and guide questions was vital in the development of the rubric for scoring. For example, for the student requirement: “give your P-drug (personal drug), dose and frequency”, the faculty scoring guide (rubric) contained a list of possible answers with the most rational drug for the case bearing full points if all three components for the item are enumerated by the student. The active engagement of the department in the preparation of the OSEP case materials and the accompanying rubric adequately contributed to the positive perception of the great majority of students that the faculty preceptor facilitated the oral examination objectively (SS no. 15).

The OSCE is a valid model for the OSEP because of its objective approach in assessing components of clinical competence in a well-planned or structured way (Harden, 1988). In one study conducted, the examinees believed that in medicine, the OSCE is the most comprehensive method of clinical assessment. They consider it as easier, fair, valid and reliable. It is practical and less biased, and its outcome is not influenced by factors like personality, gender and ethnicity. These are some of the considerations that guided the development of the OSEP. Furthermore, the objective and structured approach through the use of defined rubrics will enable the learner to identify specific areas of strengths and weaknesses in his/her competence and in the pharmacology curriculum. In line with this, another study found that if students are able to identify the “What” and “How” to better prepare themselves for the course or exam, this will enable them to get higher scores in the future (Khan, Ayub, & Shah, 2016). The ability of the OSEP to bring the students to a realisation of the gaps in knowledge and the level of confidence in prescribing drugs on their own was also revealed in this study (SS no.’s 3 and 5). These perceptions were shared by a great majority of the participants.

In a study assessing the validity of OSCE questionnaires, it was found that OSCE reduced bias. The overall perception, acceptance and satisfaction of both examiners and examinees on the validity of the OSCE were encouraging (Idris, Hamza, Hafiz, & Eltayeb A, 2014) and these study findings may be useful in providing direction to both instruction and learning.

The results of a study conducted among nursing students (Mahmoud & Mostafa, 2011) showed that they accepted the OSCE as a tool for assessing their clinical performance. For most, they considered it as a useful practical experience that reduced the possibility of failure.

But like other assessment tools, the OSCE also has its disadvantages. Some students look at OSCE as very stressful (Kim, 2016). Nevertheless, the importance of such feedback to develop and improve further the OSCE is being recognised (El-Nemer & Kandeel, 2009). In its implementation, there is a need for students to be trained extensively on time management and emotional stress relief (Mahmoud & Mostafa, 2011). The pharmacology students at the time of the OSEP were also observed by the faculty examiners to be anxious and stressed. Despite this, the general perception of the students was that the examination was conducted smoothly, in a conducive environment, and that the task expected of them was realistic (SS no.’s 16, 14 and 13).

Several studies on the OSPE, as applied in pharmacology, have likewise supported its utility as a teaching and examination tool (Deshpande et al., 2013). Medical students in universities in India, Nepal and Bangladesh have positively accepted OSPE as a form of examination. It is being recommended as an additional form of assessment to complement more traditional methods.

In contrast to the OSPE, the OSEP performed in this institution is a one-station exam that specifically aims to assess an individual student on the knowledge of core drugs used in common illnesses and the skill in applying the WHO-prescribed process of choosing a p-drug for a hypothetical case. The importance of performing this is that it allows the examiner to determine how much the student “practices” the desired outcome of choosing and prescribing drugs rationally.

The findings of students’ positive perception of the OSEP as an assessment tool, including its content and conduct is a valuable indicator of its utility, acceptability and validity as an additional means of measuring individual performance. This is supported by the findings in one study that mentioned the importance of student participation in developing newer assessment tools in the curriculum of medicine because it can influence its direction and development, as well as faculty teaching. More so, assessment formats that are considered valid, authentic and transparent become more acceptable to the students (Pierre, Wierenga, Barton, Branday, & Christie, 2004).

In another study, the perceptions and experiences of students show that the OSCE is an important and accepted tool for assessment. However, in a process of curriculum review, an assessment method like that of the OSCE should be assessed regularly. It is, therefore, necessary that the OSCE should be planned and organised carefully (Small, Pretorius, Walters, Ackerman, & Tshifugula, 2013). These concepts are also applicable to the implementation and development of the OSEP.

The general positive perception that the OSEP bears the basic qualities of an effective assessment tool was revealed in this study. The OSEP was regarded as an accurate evaluation tool by 90% of the students which can properly assess their ability to choose the p-drug through the process of Rational Drug Therapy (SS no. 6). Among the factors that probably contributed to the positive perception and acceptance of the OSEP is the adequate preparations done by both the faculty preceptors and the students in their roles as examiners and examinees respectively, coupled with the timely and practical orientation module that was held to communicate the rationale, objectives, content and conduct of the OSEP.

V. CONCLUSION

In conclusion, the medical students perceived the OSEP as an effective assessment tool in pharmacology with a positive perception of its content and conduct. This can pave the way for the development of additional, objective examination tools within the course and in other biomedical science subjects. Continuous efforts to improve the validity and positive perception of assessments tools must be institutionalised. Providing feedback to each student after their OSEP may be considered to further enhance learning while balancing this with limitations in time, and the probability of exam leakage. Furthermore, related studies like exploring the relationship between final grade and the OSEP are recommended because both are indicative of individual performance in the course.

Notes on Contributors

Chiara Marie Miranda Dimla, M.D., M.S.P.H., DPPS is an Associate Professor in the Department of Pharmacology, College of Medicine, UERMMMCI, Philippines with a Master of Science in Public Health, UERMMMCI Graduate School, Philippines.

Maria Paz S. Garcia, M.D. is an Associate Professor and Present Head of the Department of Pharmacology, College of Medicine, UERMMMCI, Philippines.

Maria Petrina S. Zotomayor, M.D. is a Professor and Past Chair of the Department of Pharmacology, College of Medicine, UERMMMCI, Philippines.

Alfaretta Luisa T. Reyes, M.D. is a Professor Emeritus and Past Dean of the College of Medicine, UERMMMCI, Philippines.

Ma. Angeles G. Marbella, M.D. is an Associate Professor and Past Head of Department of Pharmacology, College of Medicine, UERMMMCI, Philippines.

Carolynn Pia Jerez-Bagain, M.D. is an Assistant Professor of the Department of Pharmacology, College of Medicine, UERMMMCI, Philippines and has finished coursework of Master in Health Science Education in the UERMMMCI Graduate School, Philippines.

The primary authors are C. M. M. Dimla, M. P. S. Garcia and M. P. S. Zotomayor. The co-authors are A. L. T. Reyes, M. A. G. Marbella and C. P. Jerez-Bagain.

All were involved in conceptualising and writing the research proposal. All participated in its implementation, including data collection and analysis of results. While the primary authors did the final analysis and discussion of the results, all participated in writing and approving the final manuscript.

Ethical Approval

This study was approved by the Institutional Ethics Review Committee (ERC) of the Research Institute for Health Sciences (RIHS) under RIHS ERC Code: 0415/C/M/17/74.

Acknowledgements

The authors are grateful for the encouragement and support of the UERMMMCI community especially to the following:

• Administration;
• Research Institute for Health Sciences;
• Office of the Dean, College of Medicine;
• Fellow faculty members and support staff of the Department of Pharmacology, for their contributions in the conceptualisation and implementation of the OSEP, as well as in the data collection phase of this study; and
• Medical sophomore students of Pharmacology who are our inspiration.

Likewise, heartfelt thanks to our families, for their unwavering faith and unconditional love.

Funding

No funding is involved in this paper.

Declaration of Interest

All authors declared no conflict of interest in this paper.

References

Al-Wardy, N. (2010). Assessment methods in undergraduate medical education. Sultan Qaboos University Medical Journal, 10(2), 203-209.

Commission on Higher Education. (2016). Policies, standards and guidelines for Doctor of Medicine (M.D.) Program. Retrieved from Office of the President of the Phillippines https://ched.gov.ph/cmo-18-s-2016-2

Deshpande, R. P., Motghare, V. M., Padwal, S. L., Bhamare, C. G., Rathod, S. S., & Pore, R. R. (2013). A review of objective structured practical examination (OSPE) in pharmacology at a rural medical college. International Journal of Basic & Clinical Pharmacology, 2(5), 629-633. https://doi.org/10.5455/2319-2003.ijbcp20131021

de Vries, T. P. G. M., Henning, R. H., Hogerzeil, H. V., & Fresle, D. A. (1994). Guide to good prescribing: A practical manual. Retrieved from World Health Organization Website: https://apps.who.int/iris/handle/10665/59001

El-Nemer, A., & Kandeel, N. (2009). Using OSCE as an assessment tool for clinical skills: Nursing students’ feedback. Australian Journal of Basic and Applied Sciences, 3(3), 2465-2472. Retrieved from http://ajbasweb.com/old/ajbas/2009/2465-2472.pdf

Etheridge, L., & Boursicot, K. (2017). Performance and workplace assessment. In J. A. Dent, R. M. Harden, & D. Hunt (5th ed.), A practical guide for medical teachers (pp. 267-273). Edinburgh, Scotland: Elsevier.

Harden, R. M. (1988). What is an OSCE? Medical Teacher, 10(1), 19-22. https://doi.org/10.3109/01421598809019321

Harden, R. M., Stevenson, M., Downie, W. W., & Wilson, G. M. (1975). Assessment of clinical competence using objective structured examination. British Medical Journal, 1(5955), 447-451. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1672423

Hasan, S., Malik, S., Hamad, A., Khan, H., & Bilal, M. (2009). Conventional/traditional practical examination (CPE/TDPE) versus objective structured practical evaluation (OSPE)/semi objective structured practical evaluation (SOSPE). Pakistan Journal of Physiology, 5(1), 58-64.

Idris, S. A., Hamza, A. A., Hafiz, M. M., & Eltayeb A., M. (2014). Teachers’ and students’ perceptions in surgical OSCE exam: A pilot study. Open Science Journal of Education, 2(1), 15-19. Retrieved from
http://www.openscienceonline.com/journal/archive2?journalId=733&paperId=273

Ilic, D. (2009). Assessing competency in evidence based practice: Strengths and limitations of current tools in practice. BioMed Central Medical Education, 9(53). https://doi.org/10.1186/1472-6920-9-53

Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144. https://doi.org/10.1016/j.edurev.2007.05.002

Khan, A., Ayub, M., & Shah, Z. (2016). An audit of the medical students’ perceptions regarding objective structured clinical examination. Education Research International, 2016. https://doi.org/10.1155/2016/4806398

Kim, K. (2016). Factors associated with medical student test anxiety in objective structured clinical examinations: A preliminary study. International Journal of Medical Education, 7, 424-427. https://doi.org/10.5116/ijme.5845.caec

Lee, J., & Paek, I. (2014). In search of the optimal number of response categories in a rating scale. Journal of Psychoeducational Assessment, 32(7), 663-673. https://doi.org/10.1177/0734282914522200

Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73-79. https://doi.org/10.1027/1614-2241.4.2.73

Mahmoud, G. A., & Mostafa, M. F. (2011). The Egyptian nursing student’s perceptive view about an objective structured clinical examination (OSCE). Journal of American Science, 7(4), 730-738. Retrieved from http://www.jofamericanscience.org/journals/am-sci/am0704/102_5258am0704_730_738.pdf

Malhotra, S. D., Shah, K. N., & Patel, V. J. (2013). Objective structured practical examination as a tool for the formative assessment of practical skills of undergraduate students in pharmacology. Journal of Education and Health Promotion, 2, 53. https://doi.org/10.4103/2277-9531.119040

Peeters, M. J., Sahloff, E. G., & Stone, G. E. (2010). A standardized rubric to evaluate student presentations. American Journal of Pharmaceutical Education, 74(9), 171. Retrieved from
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2996761

Pierre, R. B., Wierenga, A. R., Barton, M. A., Branday, J. M., & Christie, C. D. C. (2004). Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BioMed Central Medical Education, 4, 22. https://doi.org/10.1186/1472-6920-4-22

Reddy, M. Y. (2007). Effect of rubrics on enhancement of student learning. The Journal of Doctoral research in Education (Educate~), 7(1), 3-17. Retrieved from
http://educatejournal.org/index.php/educate/article/viewFile/117/148

Small, L. F., Pretorius, L., Walters, A., Ackerman, M., & Tshifugula, P. (2013). Students’ perceptions regarding the objective, structured, clinical evaluation as an assessment approach. Health SA Gesondheid – Journal of Interdisciplinary Health Sciences, 18(1), 127-134. https://doi.org/10.4102/hsag.v18i1.629

Vishwakarma, K., Sharma, M., Matreja, P. S., & Giri, V. P. (2016). Introducing objective structured practical examination as a method of learning and evaluation for undergraduate pharmacology. Indian Journal of Pharmacology, 48(Suppl 1), S47-S51. https://doi.org/10.4103/0253-7613.193317

Zayyan, M. (2011). Objective structured clinical examination: The assessment of choice. Oman Medical Journal, 26(4), 219-222. https://dx.doi.org/10.5001%2Fomj.2011.55

*Chiara Marie Miranda Dimla
64 Aurora Blvd, Doña Imelda,
Quezon City, Philippines 1113,
University of the East-Ramon Magsaysay Memorial Medical Center
Telephone: +63 2715 8163 / +63 2713 3302
Email: cmdimla@uerm.edu.ph / chiaramdimla@gmail.com

Published online: 7 May, TAPS 2019, 4(2), 7-13
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2079

Yuki Kataoka^1,2, Hiraku Tsujimoto², Yasushi Tsujimoto³, Yuka Urushibara-Miyachi⁴, Yuda Miyamoto⁵, Hideto Yasuda⁶, Hiromitsu Yamashita⁵, Ryohei Yamamoto⁶ & Hiroshi Nishigori⁷

¹Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Japan; ²Hospital Care Research Unit, Hyogo Prefectural Amagasaki General Medical Center, Japan; ³Department of Nephrology and Dialysis, Kyoritsu Hospital, Japan; ⁴Science and Religion, School of Divinity, University of Edinburgh, Scotland; ⁵Outpatient Facility Kameda Clinic, Japan; ⁶Department of Intensive Care Unit, Kameda General Hospital, Japan; ⁷Center for Medical Education, Kyoto University, Japan

Abstract

Practice Highlights

• We described the current status of medical education research in Japan.
• We found a low subsequent publication proportion for abstracts and low representativeness in journals.
• The characteristics of conference abstracts were not related to subsequent publication.

I. INTRODUCTION

The need for the best evidence has been indicated in medical education (Harden, Grant, Buckley, & Hart, 1999). Every medical educator is expected to engage in scholarship by contributing new, peer-reviewed resources that advance the field (Norman, 2002; Simpson et al., 2007). The impact factors of medical educational journals, which reflect some aspects of scholarship activity, have been increasing in recent years (Azer, Holen, Wilson, & Skokauskas, 2016). Most medical education studies are conducted in the US or European countries. Studies from Asia are relatively less (Doja, Horsley, & Sampson, 2014).

The Japan Society for Medical Education (JSME) was founded in 1969. JSME is the largest medical education organisation in Japan. More than 2000 members are engaged in educational and research activities (Suzuki, Gibbs, & Fujisaki, 2008). However, no data is currently available on the current status of medical education research in Japan.

II. METHODS

A. Aims and Objectives

Our research objectives are as follows:

• To describe the characteristics of the conference abstracts of JSME, the Association for Medical Education in Europe (AMEE) Conference, and Asia Pacific Medical Education Conference (APMEC) from Japan.
• To describe the characteristics of articles published in medical education journals from Japan.
• To evaluate the relationship between the characteristics of conference abstracts and subsequent publication in peer-review journals.

B. Design

We conducted a meta-epidemiological investigation, which is a variant method of systematic review and usually used to describe the distribution of research evidence in a specific setting (Murad & Wang, 2017). We have developed a protocol a priori.

C. Types of Study

We included research abstracts presented in JSME from April 2013 to March 2017. We also included research abstracts presented by researchers affiliated with Japanese institutions at the AMEE and APMEC conferences from April 2012 to January 2017. We excluded abstracts of invited presentations, symposium, panel discussions, or educational workshops. We included full articles published by researchers affiliated with Japanese institutions in Medical Education, Academic Medicine, Medical Teacher, BMC Medical Education, Advances in Health Science and Education, Perspectives on Medical Education, or Teaching and Learning in Medicine from April 2012 to March 2017. We searched PubMed using the following search formula: “Name of Journal” [journal] AND “Japan” [affiliation] AND “2012/04/01” [PDAT]: “2017/03/31” [PDAT] NOT “Letter” [pt] in September 2017.

D. Definition of Study Characteristics

We evaluated abstracts without presentation data. We classified abstracts as quantitative, qualitative, or mixed method study. We defined text mining, which is an automatic assessment technique for qualitative text data as a qualitative study (Zhang et al., 2012). We defined quantitative evaluation without data, which were abstracts with the interpretation of numerical results without data as a quantitative study. We defined abstracts without evaluation as “without data”. We classified abstracts as “undergraduate”, “postgraduate”, “continuous medical education (CME)”, “interprofessional education”, and “other” according to the study participants.

We classified topics as follows: knowledge/attitude, clinical diagnosis, clinical interventions, teaching and leadership, assessment, other following the previous study (Cook, Bordage, & Schmidt, 2008). We classified purposes of research as follows: description, justification, or clarification following the previous study (Cook et al., 2008). Description means the simple observation undertaken to answer the question: What was done? Justification means comparing one educational intervention with another to answer the question: Did it work? Clarification means updating models or theories to answer the questions: How does it work? Why does it work? (Cook et al., 2008).

E. Data Extraction

We judged the affiliations of the first author and presentation styles using the Microsoft Excel® 2016 search function. The other characteristics were extracted by some of the authors (YK, HT, YT, YU-M, YM, HY, and HY) and confirmed by others (YK and YU-M). We resolved any disagreements through discussions.

F. Outcome Measures

The primary outcome was the publication proportion. The denominator was the number of abstracts included. The numerator was the number of abstracts published in peer-review journals. One author (YK) searched Google Scholar and PubMed using the first authors’ names in both Japanese and English. The search was conducted in September 2017. Published manuscripts were matched with other authors’ names and titles. We included JSME, AMEE Conference, and APMEC in 2014 because one previous study reported that the median publication duration from the conference was 20 months (Walsh, Fung, & Ginsburg, 2013).

G. Statistical Analysis

We summarised data using descriptive statistics. We calculated risk ratios and 95% confidence interval using general linear models. We used STATA® ver. 14.2 (Stata Corp., College Station, TX, USA). P < 0.05 was considered statistically significant.

H. Ethical Considerations

Because all the data were retrieved from public databases, this study did not require institutional review board approval.

III. RESULTS

A total of 1399 JSME conference abstracts, 193 AMEE conference and APMEC abstracts, and 31 original articles were included (Figure 1). The characteristics of the studies are presented in Table 1. During the study period, 6727 original articles were published in seven journals. Original articles from Japan were found to constitute 0.46% (31/6727).

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference; CME – Continuing Medical Education; IPE – Interprofessional education.
*AMEE Conference from 2012 to 2016 and APMEC from 2012 to 2015.
**NA, not available.
***The number of authors in JSME 2013 is not included.
Table 1. Characteristics of the included abstracts and full articles

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference.
Figure 1. Flow diagram

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference; CME – Continuing Medical Education; IPE – Interprofessional education.
Table 2. The relationship between publication and characteristics in JSME 2014 abstracts

From a total of 382 abstracts, 37 abstracts (10%) presented at the JSME 2014 conference were published. From a total of 39 abstracts, four (10%) abstracts presented at AMEE Conference 2014 and APMEC 2014 from Japan were published, respectively. A total of 11 (30%) articles were published in English journal from JSME. Other 26 abstracts (70%) published in 18 Japanese journals with peer review. All four articles were published in English journal from the AMEE and APMEC conferences. Exploratory analysis showed that the number of authors, affiliation of the first author, presentation style, method, topics, and purposes of the research were not statistically significant associated with subsequent publication rate (Table 2).

IV. DISCUSSION

We evaluated the characteristics of abstracts pertaining to medical education from Japan. Walsh et al. (2013) reported that the proportion of quantitative/mixed/qualitative study in medical education conference in America and Canada were 55%, 8% and 15%, respectively. Cook et al. (2008) reported the proportion of published medical education articles categorised as clarification to be 6 to 26%. Our study revealed almost the same proportion (17% to 35%). The characteristics of our study were not significantly differed from previous reports.

The representativeness of Japan in medical education journals was 0.46%. Fukui, Takahashi, and Rahman (2013) reported that the Japanese contribution was 3.6% in basic science journals and 0.74% in general medicine journals during 2001-2010. The current status of medical education research in Japan is almost equivalent to that of general medicine.

The proportion of subsequent publication (10%) was very low. Scherer, Langenberg and von Elm (2007) reported that the full publication rate was 45% in biomedical science conference abstracts. Other previous studies have reported publication rate for medical education conference abstracts to be were 35% to 44% (Sawatsky et al., 2015; Smith et al., 2014; Walsh et al., 2013). Characteristics of abstracts we have investigated were not related to publication (Table 2).

A possible explanation for the lower subsequent publication proportion may be the worse average quality of each abstract (Sawatsky et al., 2015), or inadequate time of researchers (Smith et al., 2014). The acceptance rates for the conference abstracts in JSME is nearly 100% (personal communication). The number is higher than other medical education conferences. For example, the acceptance rate of AMEE 2018 is nearly 45%, which is estimated from the 3766 submissions (Harden, 2018) and the 1658 occurrences of “background” in the abstract book (The Association for Medical Education in Europe, 2018). This difference would be evidence of poor abstract quality. Love et al. (2016) revealed that education for clinicians to gain skills including quantitative and qualitative methods improved the medical education research output. A lack of quantitative research education programs for post-graduates in Japan was identified (Arimura et al., 2010). To provide education programs for medical teachers in Japan is one of the options to promote educational researches. Considering the high burnout rate of doctors including residents (Tateno et al., 2018), the Japan Ministry of Health, Labour and Welfare (2018) is now attempting to reform the work lives of doctors to reduce their burden. It is important to use any gained surplus time for research productivity improvement.

By offering educational programs on medical education research for several physicians who lack knowledge of research (Kurita et al., 2016; Suzuki & Fukushima, 2014), consequently, the number of studies on medical education would increase. The total quality and publication rate may improve (Huang, 2016). In April 2018, the board certification system changed to standardise the certification of fellows in Japan (Hirokuni, 2017). To improve the system, the post-graduate medicine curriculum should be evaluated (Iwata, Mosby, & Sakane, 2017). There are several opportunities to conduct medical educational research.

Our study has several limitations. First, seven individual authors judged the characteristics of abstracts to reduce burden, which may cause misclassifications. To reduce the information bias, another author confirmed the decision, and we followed the pre-defined protocol. Second, we did not search for medical education abstracts in other conferences, such as the Association of American of Medical Colleges (AAMC) or Ottawa Conference. Furthermore, some medical education abstracts have been presented in clinical conferences, such as general internal medicine or paediatrics (Sawatsky et al., 2015; Smith et al., 2014). In general, medical education research is not often pre-registered, so we believe that to focus on medical education conferences was the best way to know the current status but further evaluation is warranted.

V. CONCLUSION

We described the current status of medical education research in Japan. We identified a low subsequent publication rate for conference abstracts and low representativeness in medical education journals. Further investigation is required to improve the number of publications.

Notes on Contributors

Yuki Kataoka, MD, MPH, is a Medical Head at the Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Japan. He contributed to the design of the research, retrieving data, analysing, prepared the draft of the article, and approved the final version of the article.

Hiraku Tsujimoto, MD, is a research fellow at the Hospital Care Research Unit, Hyogo Prefectural Amagasaki General Medical Center, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Yasushi Tsujimoto, MD, MPH, is an attending staff member at the Department of Nephrology and Dialysis, Kyoritsu Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Yuka Urushibara-Miyachi, MD, MHPE, is an MSc Science and Religion student at the School of Divinity, the University of Edinburgh. She contributed to the design of the research, retrieving data, and approved the final version of the article.

Yuda Miyamoto, MD, is a resident at the Outpatient Facility Kameda Clinic, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hideto Yasuda, MD, is an attending staff member at the Department of Intensive Care Unit, Kameda General Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hiromitsu Yamashita, MD, is a fellow at the Outpatient Facility Kameda Clinic, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Ryohei Yamamoto, MD, is a fellow at the Department of Intensive Care Unit, Kameda General Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hiroshi Nishigori, MD, MMEd, PhD, is an Associate Professor at the Center for Medical Education, Kyoto University, Japan. He contributed to the design of the research and approved the final version of the article.

Ethical Approval

As all the data were retrieved from public databases, this study did not require institutional review board approval.

Acknowledgements

This study was conducted as a part of a project by the Japan Society for Medical Education. The Society supported data retrieval but has no role in the conceptualisation, analysis, or preparation of this manuscript.

Funding

This work was supported in part by a grant from the Hyogo Prefectural Amagasaki General Medical Center fiduciary funds (for English editing).

Declaration of Interest

The authors report no conflicts of interest. All the authors are responsible for the content and writing of the article.

References

Arimura, Y., Nishida, T., Minami, M., Yokoyama, Y., Mishina, H., Yamazaki, S., … Fukuhara, S. (2010). Challenges in the conduct of clinical research: View of the graduates of a clinical research training program in Japan. Igaku Kyoiku [Medical Education (Japan)], 41(4), 259–265.
https://doi.org/10.11307/mededjapan.41.259

Association for Medical Education in Europe. (2018, August 25). ABSTRACT BOOK AMEE 2018 [PDF file]. Retrieved from https://amee.org/getattachment/Conferences/AMEE-2018/Abstracts/AMEE-2018-Abstract-Book.pdf

Azer, S. A., Holen, A., Wilson, I., & Skokauskas, N. (2016). Impact factor of medical education journals and recently developed indices: Can any of them support academic promotion criteria? Journal of Postgraduate Medicine, 62(1), 32-39. https://doi.org/10.4103/0022-3859.173202

Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128–133.
https://doi.org/10.1111/j.1365-2923.2007.02974.x

Doja, A., Horsley, T., & Sampson, M. (2014). Productivity in medical education research: An examination of countries of origin. BioMed Central Medical Education, 14, 243.
https://doi.org/10.1186/s12909-014-0243-8

Fukui, T., Takahashi, O., & Rahman, M. (2013). Japanese representation in leading general medicine and basic science journals: A comparison of two decades. The Tohoku Journal of Experimental Medicine, 231(3), 187–191.
https://doi.org/10.1620/tjem.231.187

Harden, R. M. (2018, March 19). Harden’s Blog: APMEC 2018 Singapore, measuring what matters, Bawa-Garba and fake news, Ottawa conference in Abu Dhabi, and AMEE 2018 in Basel [Blog post]. Retrieved from https://www.mededworld.org/hardens-blog/reflection-items/March-2018/HARDEN-S-BLOG-APMEC-2018-Singapore–Measuring-what.aspx

Harden, R. M., Grant, J., Buckley, G., & Hart, I. R. (1999). BEME guide no. 1: Best evidence medical education. Medical Teacher, 21(6), 553–562. https://doi.org/10.1080/01421599978960

Hirokuni, Y. (2017). 「新たな専門医制度の開始に向けた声明」を公表いたしました [“Statement for the start of a new  specialist system” was announced] [PDF file]. Retrieved from http://www.japan-senmon-i.jp/news/doc/statement_170804_.pdf

Huang, D. (2016). Positive correlation between quality and quantity in academic journals. Journal of Informetrics, 10(2), 329–335. https://doi.org/10.1016/j.joi.2016.02.002

Iwata, K., Mosby, D. J., & Sakane, M. (2017). Board certification in Japan: Corruption and near-collapse of reform. Postgraduate Medical Journal, 93(1101), 436.
https://doi.org/10.1136/postgradmedj-2017-134994

Kurita, N., Murakami, M., Shimizu, S., Kumasawa, J., Azuma, T., Kataoka, Y., … Fukuhara, S. (2016). Preferences of young physicians at community hospitals regarding academic research training through graduate school: A cross-sectional research. BioMed Central Research Notes, 9(1), 227. https://doi.org/10.1186/s13104-016-2036-0

Love, J. N., Yarris, L. M., Santen, S. A., Kuhn, G. J., Gruppen, L. D., Coates, W. C., … Farrell, S. E. (2016). A novel specialty-specific, collaborative faculty development opportunity in education research: Program evaluation at five years. Academic Medicine, 91(4), 548–555.
https://doi.org/10.1097/ACM.0000000000001070

Ministry of Health, Labor and Welfare. (2018). 「医師の働き方改革に関する検討会」が医師の労働時間短縮に向けた取組を緊急アピール [The meeting group to discuss work-style reform for doctors urges medical institutions to step up emergency measures to reduce physicians’ working hours] [PDF file]. Retrieved from
https://www.mhlw.go.jp/stf/newpage_02920.html

Murad, M. H., & Wang, Z. (2017). Guidelines for reporting meta-epidemiological methodology research. Evidence Based Medicine, 22(4), 139–142. https://doi.org/10.1136/ebmed-2017-110713

Norman, G. (2002). Research in medical education: Three decades of progress. British Medical Journal, 324, 1560–1562. https://doi.org/10.1136/bmj.324.7353.1560

Sawatsky, A. P., Beckman, T. J., Edakkanambeth Varayil, J., Mandrekar, J. N., Reed, D. A., & Wang, A. T. (2015). Association between study quality and publication rates of medical education abstracts presented at the society of general internal medicine annual meeting. Journal of General Internal Medicine, 30(8), 1172–1177. https://doi.org/10.1007/s11606-015-3269-7

Scherer, R. W., Langenberg, P., & von Elm, E. (2007). Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews, 2007(2), MR000005. https://doi.org/10.1002/14651858.MR000005.pub3

*Yuki Kataoka
Department of Respiratory Medicine
Hyogo Prefectural Amagasaki General Medical Center
Higashi-Naniwa-Cho 2-17-77,
Amagasaki, Hyogo 660-8550, Japan
Email: youkiti@gmail.com

Published online: 3 September, TAPS 2019, 4(3), 57-66
DOI: https://doi.org/10.29060/TAPS.2019-4-3/OA2085

Astrid Pratidina Susilo¹, Brahmaputra Marjadi^2,3, Jan van Dalen⁴& Albert Scherpbier⁴

¹Faculty of Medicine, University of Surabaya, Indonesia; ²Faculty of Medicine, Universitas Wijaya Kusuma Surabaya, Indonesia; ³School of Medicine, Western Sydney University, Australia; ⁴Faculty of Health, Medicine, and Life Sciences, Maastricht University, The Netherlands

---

Abstract

Practice Highlights

• Patients consent to a proposed procedure if information from the physicians match their own.
• Patients’ perception of the information from their physician is influenced by the doctor-patient relation.
• The hierarchical and communal culture hinder patients to discuss their concern with physicians.
• Skills of exploration and facilitation should be strengthened for all health professionals.
• Involvement of other health professionals as advocates may help patients’ make voluntary and informed decisions.

I. INTRODUCTION

Underlying the informed consent process is the patient’s decision-making which is a reflection of patient’s autonomy. Patients have the right to make a voluntary decision about a medical procedure after obtaining adequate information from the physician (King & Moulton, 2006).

In Western settings, decision-making is reported to be both an analytical and intuitive process (Broadstock & Michie, 2000; Power, Swartzman, & Robinson, 2011). Studies which assumed that decision-making was analytical and rational saw decision-making as weighing benefits and risks. Proper information provision, e.g., on the adequacy of information received in an informed consent process was measured by the patients’ recall of information or by their level of satisfaction (Burns, Keogh, & Timon, 2005; Howlader et al., 2004; McGaughey, 2004; McKeague & Windsor, 2003). Patients could not always recall the information, which suggested inadequacy of information (Burns et al., 2005; McGaughey, 2004). Recommendations focused on improvement ofinformation-delivery, for example by using decision aids (Brehaut, Saginur, & Elwyn, 2009; Burns et al., 2005; Schenker, Fernandez, Sudore, & Schillinger, 2011). The more naturalistic paradigm argued that decision-making was also intuitive, with interplays between cognition and emotion (Broadstock & Michie, 2000). Patients dynamically form a representation of their illnesses as the obtained information changes. A decision is made based on this representation adjusted to patients’ belief and experience. Options are accepted if they are satisfactory although not necessarily optimal (Broadstock & Michie, 2000; Power et al., 2011).

As decision-making is context-based (Blank, Graves, Sepucha, & Llewellyn-Thomas, 2006; Broadstock & Michie, 2000), the process in Asia may differ from that in the West. Asian cultures tend to be strongly hierarchical, where large power distance among people is pervasive (Hofstede, Hofstede, & Minkov, 2010). The physician-patient relationship in Asia also tends to be hierarchical (Claramita & Susilo, 2014). Many Asian cultures are also strongly communal, where personal decision-making is heavily influenced by the community (Hofstede et al., 2010) including family members (Pun, Chan, Wang, & Slade, 2018; Susilo, Nurmala, van Dalen, & Scherpbier, 2012; Susilo et al., 2013). Patients’ involvement in health decision-making is often low regardless of their level of education (Claramita, van Dalen, & van der Vleuten, 2011). Patients let physicians make decisions for them to avoid confrontation and/or because of the trust patients have in physicians on medical issues (Lin, Huang, Chiang, & Chen, 2012). Unfortunately, studies on patient decision-making in Asia have been sparse and provide limited insight due to narrow selection of patients (Lin et al., 2012; Sekimoto et al., 2004) or clinicians (Slingsby, 2004).

We conducted this study to expand the evidence in an Asian context. Our research question is: How is the process of patients’ decision-making for informed consent in Indonesia related to the hierarchical and communal Indonesian culture? We investigated the role of information given to patients during the informed consent process in their medical decision-making, and the factors influencing this process unique for the Indonesian context. Insights from this study could assist in preparing health professionals to better support patients’ decision-making and maintain respect for patients’ autonomy.

II. METHODS

As we aimed to understand less known phenomena, this study was conducted in line with the Grounded Theory approach (Corbin & Strauss, 2008; Kennedy & Lingard, 2006). A theoretical framework was developed based on interviews with patients and their families.

A. Study Participants

This study was conducted at an Indonesian hospital with around 300 beds and multi-specialist services. The study population was all former adult in-patients who had been hospitalised up to one year prior to the study. Eligible patients had been offered a diagnostic or therapeutic procedure that required written informed consent. Participants were selected purposively from a list of eligible patients compiled by the Medical Record Department. The sample selection process aimed for diversity (Fraenkel, Wallen, & Hyun, 2012) with regard to medical procedure, age, sex, and educational attainment. Participants had to be fluent in the Indonesian language and be accessible for interviews. Eligible patients were contacted by phone and were provided with verbal information about the study. Those who were interested to participate were invited to give a face-to-face interview. Written information about the study was given to participants prior to obtaining their written informed consent for the study. Family members who were identified as important stakeholders, involved in the decision-making process, were also interviewed with the patients’ approval.

A two-stage sampling process (Kennedy & Lingard, 2006) was employed. The initial sample consisted of elective and urgent patients only. After analysis of seven interviews, a theoretical sampling was derived from the preliminary data and used to expand the sample by including patients with repeated procedures and life-threatening conditions.

B. Data Collection

Semi-structured interviews were conducted to explore the patients’ decision-making process, factors that contributed to the decision, and sources of information used by patients. The interview guide is provided in the Appendix. Interviews were conducted by APS at a location of participants’ choice, mostly in their residence. Each interview took approximately one hour. BM joined one interview session to confirm interview techniques. APS and BM are native Indonesian physicians with no involvement in the care for these patients. Both were trained in qualitative research. Interviews were audio-recorded and notes were handwritten as a backup. When data were considered sufficiently saturated, i.e. when no new information emerged during the interviews, no further interviews were conducted.

C. Data Analysis

The audio recordings were transcribed verbatim and combined with handwritten notes. APS and BM separately conducted open coding on the transcripts of five interviews. Coding categories were then compared and differences were discussed until consensus was reached. APS coded the remainder of the transcripts. The coding categories were sharpened during this iterative process and emerging themes identified and discussed among the authors. Axial coding was performed by comparing similar codes in different interviews in line with the constant comparative method as a necessary component of Grounded Theory (Corbin & Strauss, 2008; Kennedy & Lingard, 2006). The authors discussed the final integration of the themes to the theoretical framework to achieve consensus.

D. Rigour

In order to ensure quality and rigour, we triangulated different data sources (patients and their family) and different researchers. Researchers’ reflections throughout the project were recorded as memos to support data interpretation. Member-checking was conducted by summarising important issues at several points during the interviews (Fraenkel et al., 2012).

E. Ethical Consideration

No physical risk was identified. To avoid influencing the patients’ medical decision-making, only discharged patients were selected for this study. All participants were given oral and written information before being asked for consent. Participants were assured that their decision for participation was voluntary and had no repercussions on their current and future medical opportunities and quality of their care. Identities of participants, physicians, and hospital were kept confidential.

III. RESULTS

Twenty-seven participants (fifteen patients and twelve family members) consented to the study and were interviewed (Table 1). Eleven patients had consented to the proposed medical procedures and four refused to give consent. Ten of the 15 patients had tertiary education. Patients had various health problems and were treated by different physicians.

Table 1. Basic information of participants

A. Emerging Themes

The emerging themes are reported under five major themes, with ‘P(number, diagnosis/procedure)’ denoting a patient, ‘F(number)’ a family member and ‘A’ APS, the interviewer.

1) Decision-making as an evolving process:All interviewed patients reported attempting to develop an ‘explanation’ of what was happening to them and what should be done to rectify their health problem. Each explanation was highly influenced by the patient’s beliefs, values, emotion, and own and others’ past experiences. These explanations helped them to understand and cope with their illness. Some explanations were close to the biomedical concept; for example, P9 (spontaneous abortion) described the objective of curettage as “to clean the remaining blood that was still in the tummy, so… in the next few months it is possible to get [pregnant] straightaway”. Other explanations were not as clear, such as from P8 (gastritis): “I was ill. I didn’t know what the illness was. Someone ‘made’ it, [it’s] my neighbour. I couldn’t walk. I couldn’t do anything”.

Patients’ explanations evolved with the changes in their health conditions. The explanations also evolved as patients obtained further information, either actively sought or passively received. Along this process, patients identified gaps in their explanations and attempted to fill those to develop a satisfactory explanation for their conditions. For instance, P6 (uterine inertia) was informed that the condition of the baby in the womb was worsening as shown as a weakening activity in a Non-Stress Test. The physician suggested a caesarean section. However, P6 did not want surgery and developed her own explanation that the weakening activity was due to her baby being asleep during the test.

The information patients used to develop their explanations came from their physician and various other sources: media, family and friends. Other health professionals (like other physicians approached for a second opinion) were also sources of information. Nurses were sometimes consulted, although not frequently.

Some patients – notably those with chronic conditions – had been contemplating their health problem and the necessary medical procedure(s) prior to being offered the procedure in question. Such patients had had ample time to incorporate alternative treatment options into their explanations. Some patients reported having tried several alternative methods, such as traditional medicine. If outcomes were unsatisfactory, the patient would try another treatment while revising their explanations to incorporate this negative experience. For example, P4 (inguinal hernia) contemplated for four years and tried different solutions before consenting to surgery.

2) Matching own explanation with the information from the physicians:Patients compared formally obtained information from their physician in the informed consent process, particularly about the proposed treatment with their existing explanations. Regardless of the comprehensiveness of the physician’s information (which assessment was beyond the scope of this study), as long as this information matched the patient’s explanation, the patient tended to readily consent to the proposed medical procedure.

[I would prefer] to avoid surgery, [because] it would be much more comfortable to have simple drug treatments. That’s what I thought, and as it turned out my idea matched with Dr X… I felt it click in my heart, because the doctor said, “Let’s treat this with drugs first, okay?”

(P3, chronic tonsillitis)

A match between physician’s information and patient’s explanations led patients to perceive that the information was adequate at the time of decision. If the physician’s information did not match the patient’s explanations, the patients tended to continue to develop explanations and fill the gaps. This process included seeking more information from their physician or others. For example, P12 (myoma) consulted three other physicians despite having had much information from her attending physician.

In searching for more information, patients tended to focus mostly on the gaps or discrepancies in their explanations based on prior knowledge, experience, belief, and value, instead of seeking an overall completeness of information. The husband of P6 (caesarean section) explained why he requested a printout but did not feel the need to ask about the risks of surgery:

Since I am used to dealing with data, naturally I would like to see the data… so I had the graphs printed. My questioning stopped after I got the data… I wasn’t told about the risks of surgery, but I believe… as long as the quality is good, the drugs are good, because the condition of [the patient] was without hypertension, without [other] risks for pregnancy, so I thought there would be no problem… I have studied about high risks in pregnancy, for example if she had had prior surgery… and in several mother and child health books, the risks include for example high blood pressure… but I saw there was none of those risks in [the patient].

On the other hand, when patients had other concerns, they paid less attention to physicians’ information. The mother of P10 (internal bleeding) hardly remembered what the physician had discussed as she was worried about the cost of the surgery during the encounter.

3) Changing severity of the disease disrupts the development of the explanation:Increasing severity of the disease tended to push patients toward consenting to the proposed procedure regardless of their satisfaction with the information they had obtained at that stage. The severity changed influenced patients’ emotion and the patients’ ability to comprehend the information they received. The feelings of confusion, fear, and/or anxiety seemed to be stronger than the patients’ desire to search for more information.

P5: I was also afraid, I was down. I didn’t ask whether there were other alternatives. I trusted the physician.

A: If you look back now, do you think it was a correct decision to have the fluid removed?

P5: Well, quite correct. However, we also don’t know whether there were other solutions or not. Because we didn’t ask. We just trusted the physician.

(P5, pleural effusion)

In contrast to those with severe or emergency conditions, patients who considered their conditions not severe or diminished tended to postpone or refuse a proposed treatment.

P11: I thought my condition was not too severe, so [ultrasonography] was not necessary… because in those two to three days I was actually feeling better.

(P11, liver dysfunction)

4) Influence of physician-patient relation in the decision-making process:When patients trusted their physician and/or the hospital, they tended to consent more readily to the proposed medical procedure regardless of the perceived adequacy of information. The trust also led the participants to give a positive evaluation of the decision despite their lack of knowledge about the condition and/or procedure.

F15: [The decision for the medical procedure] can be considered correct because I had no medical knowledge. I only think that the physician has given the best decision, that’s all… because I know that [the hospital] sincerely gives help to patients who need it.

 (Husband of P15, pneumonia and hypoglycaemia)

In contrast, distrust with the physician seemed to influence patients to refuse consent. P14 (coronary heart disease) refused an intervention and asked for early discharge, a decision which he linked to his perception of the physician. He suspected a commercial motive by the physician, as the procedure was expensive. He did not discuss his concern further as he considered the physician not approachable.

The physician-patient relations also influenced the way patients sought information from their physicians. Some patients admitted to having had concerns that they did not express to their physicians. These patients were hesitant to bother their physicians further. Even when their physicians invited patients to ask questions, these patients felt it was inappropriate to ask, or they did not want to offend the physician by expressing their doubts; they chose to maintain a harmonious relation with their physicians. F13 (polycythaemia) did not ask for more information for fear that the physician would see the questions as an attempt to outsmart him and this would be considered as inappropriate behaviour. Furthermore, patients did not always share their emotions openly with their physicians although during the interviews many reported the emotional impact of their illness. P3 (chronic tonsillitis) reported that his physician asked if he was afraid of the surgery and he denied, although he was actually rather afraid.

5) Closing the event:Once the medical procedure had been undertaken, many patients decided not to seek further information regardless of whether there were still gaps in their explanation. This decision seemed to reflect their acceptance of the explanation. For example, when asked whether she still had any unresolved questions about her procedure, P12 (myoma) said: “No, and I also don’t want to know… It will make me think of bad things”. The wife of P13 (polycythaemia), who happened to be a nurse in the same hospital, admitted:“I am actually [still] curious about the [causal] factor… Because Dr Z said it could be due to food, it could be due to excessive intake…”. However, she decided not to pursue her questions any further to avoid making her husband anxious about his condition. For some participants, the event was not as easily closed. For example, P8 (gastritis) went to a traditional healer after discharge and only closed the event when she believed that the healer extracted needles, stones, and cloths (alleged signs of black magic) from her body.

B. Theoretical Framework Describing Patients’ Decision-Making Process

The dynamic process of patients’ decision-making, including the interplaying components within, is depicted in Figure 1. The grey box represents the decision-making of the patient as an evolving process. This consists of several small processes; each is represented by a blue box. First, the process is initiated when the patients feels their first symptoms. Patients build an explanation of their illness and how to treat it, and this explanation evolves over time. Second, during the informed consent process patients assess the match between the information from their physician and their explanation at the time; the degree of match determines the patient’s response to the proposed medical procedure. Third, the culmination of the process is the decision. In the informed consent process, this can be the moment the patient signs the consent form. Fourth, finally the patient will close the event after a medical procedure or non-alternatives by accepting whatever stage their explanation is at the moment, including the fact that some questions may remain unanswered.

This evolution is influenced by external factors outside the grey box; pictured with black arrows. Along the process, patient receives information from different sources including the attending physician. The disease severity modulates the development of the explanation as well as the final decision-making process. Patients’ perception of the information from their physician is influenced by the physician-patient (and/or hospital-patient) relationship.

Finally, the outermost box showed that the whole process is inseparable from the societal culture, which in this context tended to be strongly hierarchical and communal.

Figure 1. The theoretical framework of patient’s decision-making process

IV. DISCUSSION

The decision-making process of patients in a hierarchical and communal culture is a development of an explanation about their illness and the treatment which evolves as the severity of the disease changes. This explanation is shaped by the information they obtain from different sources. This general pathway of decision is in line with the naturalistic paradigm from Western studies (Broadstock & Michie, 2000; Little, 2009). Similarly to Western findings, we found that many patients still had concerns although they have consented (McKeague & Windsor, 2003; Schenker et al., 2011). These similarities indicate that some phenomena in patient’s decision-making are universal.

Other findings were specific to our Indonesian context. Our participants reported feeling uncomfortable to discuss their concerns or share their emotion with their physicians. This feeling is likely to be stronger in our culture of hierarchy and communality (Hofstede et al., 2010). As physicians occupy a much higher hierarchical position (Claramita & Susilo, 2014), patients do not want to be seen as outsmarting them. Patients also do not want to upset their relation with the physician, a typical feature of a communal culture that upholds harmony, (Hofstede et al., 2010) also reported from a similar culture in Taiwan (Lin et al., 2012). Patients seemed to be willing to assume a passive role and accepted that some concerns were left unanswered.

Our finding of patients’ submissive role confirmed previous studies from this cultural context which showed that patients generally have low involvement in health consultations (Claramita et al., 2011; Kim, Kols, Bonnin, Richardson, & Roter, 2001). However, patients in the hierarchical context also desire a more partnership-style relation with their physicians in which they could have more involvement (Claramita, Nugraheni, van Dalen, & van der Vleuten, 2013). Patients’ effort to gather information from different sources may have indicated that they do have a desire to actively participate in the health decision. Yet, the effort to gather information can also be interpreted as an attempt to make meaning of their illness experience, which is a common adjustment process (Park, 2010). The making of both meanings and decisions seem to be interlinked. In a communal culture information from family and friends is prominent (Hofstede et al., 2010). Patients incorporate information from non-physician sources to develop an explanation about their illness and the treatment although such information is not necessarily in line with that from a physician and could enhance patients’ concern.

When a hierarchical relation hinders patients to discuss their unspoken concern with their physicians, patients may accept underdeveloped or unmatched explanations of their illness at the time of consent. Such patients may experience a higher level of regret than those who were actively involved in the decision (Sawka et al., 2012). If undesired outcomes ensue, patients may blame themselves or the health care system for making the ‘wrong’ decision (Brehaut et al., 2003; Little, 2009).

To face these cultural challenges, we propose some measures to improve patients’ decision-making. Provision of additional information with printed media or decision aids (Brehaut et al., 2009; Burns et al., 2005; Schenker et al., 2011) could support patients to obtain basic information necessary in the informed consent process. Strengthening communication skills of physicians is strongly suggested (Silverman, Kurtz, & Draper, 2013). A guideline tailored to the Southeast Asian culture is available. It may be used to train health professionals to strengthen their skills in exploration, tailoring information and, especially, interpreting subtle non-verbal gestures (Claramita, Susilo, Rosenbaum, & van Dalen, 2016).

As direct confrontation is undesirable in a strong communal culture (Hofstede et al., 2010), patients may not raise their concerns despite exploration by the physician. Therefore, merely training physicians might be insufficient to ensure that patients make voluntary informed decisions. Other studies from similar contexts emphasised the involvement of non-physician health professionals (e.g. nurses) to relay patients’ concerns to the physicians (Lee, Lee, Kong, Kim, & Kim, 2009; Lin et al., 2012; Susilo et al., 2013).

Strengthening the communication skills of health professionals should be started at their education. The informed consent process can serve as a valuable topic to learn communication skills together with the legal, ethical, and cultural aspects in an integrated fashion (Susilo et al., 2013). The insight from this study can inform the development of communication skills training in health professions education. This education should emphasise the importance of understanding the cultural context and tailoring the communication competence of health professionals according to these cultural characteristics (Claramita et al., 2016).

Our study had a few limitations. First, our choice to look at decision-making contexts may have predisposed our data analysis toward decision-making. This subjectivity is an inherent nature of qualitative studies that we need to be aware of (Fraenkel et al., 2012).

Second, from the point of view of qualitative study, there is a limitation in the variation of our sampling. This study was conducted at a single hospital, whose particular setting may have influenced the results of the study. For example, the ethnicities of the patients are Javanese and Chinese and the levels of education of the patients are high school or tertiary education. These categories of patients are the most prevalent ethnic and level of education in this hospital. What we meant with Chinese in this study is Indonesian Chinese who have lived in this country for generations and assimilated with local culture (Urban, 2013). Although Indonesia has diverse ethnicities, similar to other ethnics in Indonesia, both Javanese and Indonesian Chinese hold strong hierarchical and communal culture (Hofstede et al., 2010). There are also some variations of educational attainment in our sampling although this is not the representation of the whole spectrum of education.

Nevertheless, we managed to vary the sampling according to sex, age, and medical procedures. Within the medical procedures, there were also variations based on the type of the health problem or specialisation involved (pulmonology, internal medicine, surgery, obstetrics) and the decision (consent and refusal). We also have representation of emergency and elective procedures, as well as one-time procedure and repeated ones (phlebotomy).

Despite these sampling constraints, the theoretical framework arising from this study was adequate to explain the stories of all participants, thus supporting the comprehensiveness of the framework (Corbin & Strauss, 2008).

Third, since the interviews were conducted a few months after the decision was made, participants may have had inaccurate and/or selective recollection of their decision-making experience. To reduce this phenomenon, which is analogous with ‘recall bias’ in retrospective quantitative studies (Althubaiti, 2016), family members who were involved in the decision-making process were also interviewed (either together or separately) and their accounts were triangulated.

Finally, considering that some phenomena in the decision-making process are universal, we notice the potential of our theoretical model to be applied in other cultural settings. Our data were not adequate to detail this issue because our participants came from one cultural context. It would be interesting to test the applicability of our theoretical model by comparing this result to others from different cultural settings.

V. CONCLUSION

The decision-making process in a hierarchical and communal context resembled that of the Western world. The highlighted differences were in the physician-patient relation and the prominent non-physician source of information. The hierarchical and communal social cultures added to the complexity of this relation and hindered patients to discuss their concerns with the physician in the informed consent process. Our findings could help inform education for health professional. It is important to include the understanding of patients’ decision-making processes in a hierarchical and cultural context to the training of communication skills of health professionals and develop the cultural competence. The awareness of these characteristics can help health professionals to ensure the voluntary nature of patients’ informed decisions.

Notes on Contributors

Astrid Pratidina Susilo is a researcher in health professions education and an anesthesiologist.

Brahmaputra Marjadi is a senior lecturer and researcher in public health and medical education.

Jan van Dalen is a researcher in health professions education.

Albert Scherpbier is a professor in quality improvement in medical education.

Ethical Approval

This study was approved by the hospital’s board of directors and ethics committee (Letter No. 277/SDM/IV-2007).

Acknowledgements

The authors wish to thank all participants in this study.

Funding

There is no external funding involved in this study.

Declaration of Interest

Authors have no conflict of interest, including financial, consultant, institutional and other relationships that might lead to bias.

References

Althubaiti, A. (2016). Information bias in health research: Definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare, 9, 211-217.

Blank, T., Graves, K., Sepucha, K., & Llewellyn-Thomas, H. (2006). Understanding treatment decision making: Contexts, commonalities, complexities, and challenges. Annals of Behavioral Medicine, 32(3), 211-217.

Brehaut, J. C., O’Connor, A. M., Wood, T. J., Hack, T. F., Siminoff, L., Gordon, E., & Feldman-Stewart, D. (2003). Validation of a decision regret scale. Medical Decision Making, 23(4), 281-292.

Brehaut, J. C., Saginur, R., & Elwyn, G. (2009). Informed consent documentation necessary but not sufficient. Contemporary Clinical Trials, 30(5), 388-389.

Broadstock, M., & Michie, S. (2000). Processes of patient decision making: Theoretical and methodological issues. Psychology & Health, 15(2), 191-204.

Burns, P., Keogh, I., & Timon, C. (2005). Informed consent: A patients’ perspective. The Journal of Laryngology & Otology,119(1), 19-22.

Claramita, M., Nugraheni, M. D., van Dalen, J., & van der Vleuten, C. P. M. (2013). Doctor-patient communication in Southeast Asia: A different culture? Advances in Health Sciences Education Theory and Practice, 18(1), 15-31.

Claramita, M., & Susilo, A. P. (2014). Improving communication skills in the Southeast Asian health care context. Perspectives on Medical Education, 3(6), 474-479.

Claramita, M., Susilo, A. P., Rosenbaum, M., & van Dalen, J. (2016). Core communication skills for health professionals in Southeast Asian health care settings. In M. Claramita, A. P. Susilo, M. Rosenbaum, & J. van Dalen (Eds.), Communication of health care professionals and patients in the context of Southeast Asian culture(pp. 1-17). Jakarta, Indonesia: EGC.

Claramita, M., van Dalen, J., & van der Vleuten, C. P. M. (2011). Doctors in a Southeast Asian country communicate sub-optimally regardless of patients’ educational background. Patient Education and Counseling, 85(3), 169-174.

Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory(3rd ed.). Thousand Oaks, CA: SAGE Publications, Inc.

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education(8th ed.). New York, NY: Mc-Graw Hill.

Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations: Software of the mind: Intercultural cooperation and its importance for survival (3rd ed.). New York, NY: McGraw Hill.

Howlader, M. H., Dhanji, A. R., Uppal, R., Magee, P., Wood, A. J., & Anyanwu, A. C. (2004). Patients’ views of the consent process for adult cardiac surgery: Questionnaire survey. Scandinavian Cardiovascular Journal, 38(6), 363-368.

Kennedy, T. J., & Lingard, L. A. (2006). Making sense of grounded theory in medical education. Medical Education, 40(2), 101-108.

Kim, Y. M., Kols, A., Bonnin, C., Richardson, P., & Roter, D. (2001). Client communication behaviors with health care providers in Indonesia. Patient Education and Counseling, 45(1), 59-68.

King, J. S., & Moulton, B. W. (2006). Rethinking informed consent: The case for shared medical decision-making. Americal Journal of Law & Medicine, 32(4), 429-501.

Lee, S., Lee, W. H., Kong, B. H., Kim, I. S., & Kim, S. (2009). Nurses’ perceptions of informed consent and their related roles in Korea: An exploratory study. International Journal of Nursing Studies, 46(12), 1580-1584.

Lin, M. L., Huang, C. T., Chiang, H. H., & Chen, C. H. (2012). Exploring ethical aspect of elective surgery patients’ decision-making experiences. Nursing Ethics, 20(6), 672-683.

Little, M. (2009). The role of regret in informed consent. Journal of Bioethical Inquiry,6(1), 49-59.

McGaughey, I. (2004). Informed consent and knee arthroscopies: An evaluation of patient understanding and satisfaction. Knee, 11(3), 237-242.

McKeague, M., & Windsor, J. (2003). Patients’ perception of the adequacy of informed consent: A pilot study of elective general surgical patients in Auckland. The New Zealand Medical Journal,116(1170).

Park, C. L. (2010). Making sense of the meaning literature: An integrative review of meaning making and its effects on adjustment to stressful life events. Psychological Bulletin, 136(2), 257-301.

Power, T. E., Swartzman, L. C., & Robinson, J. W. (2011). Cognitive-emotional decision making (CEDM): A framework of patient medical decision making. Patient Education and Counseling, 83(2), 163-169.

Pun, J. K. H., Chan, E. A., Wang, S., & Slade, D. (2018). Health professional-patient communication practices in East Asia: An integrative review of an emerging field of research and practice in Hong Kong, South Korea, Japan, Taiwan, and Mainland China. Patient Education and Counseling, 101(7), 1193-1206.

Sawka, A. M., Straus, S., Gafni, A., Meiyappan, S., David, D., Rodin, G., … Goldstein, D. P. (2012).Thyroid cancer patients’ involvement in adjuvant radioactive iodine treatment decision-making and decision regret: An exploratory study. Supportive Care in Cancer, 20(3), 641-645.

Schenker, Y., Fernandez, A., Sudore, R., & Schillinger, D. (2011). Interventions to improve patient comprehension in informed consent for medical and surgical procedures: A systematic review. Medical Decision Making, 31(1), 151-173.

Sekimoto, M., Asai, A., Ohnishi, M., Nishigaki, E., Fukui, T., Shimbo, T., & Imanaka, Y. (2004). Patients’ preferences for involvement in treatment decision making in Japan. BMC Family Practice, 5, 1.

Silverman, J., Kurtz, S., & Draper, J. (2013). Skills for communicating with patients(3rd ed.). London, England: Radcliffe Publishing Ltd.

Slingsby, B. T. (2004). Decision-making models in Japanese psychiatry: Transitions from passive to active patterns. Social Science & Medicine, 59(1), 83-91.

Susilo, A. P., Nurmala, I., van Dalen, J., & Scherpbier, A. (2012). Patient or physician safety? Physicians’ views of informed consent and nurses’ roles in an Indonesian setting. Journal of Interprofessional Care, 26(3), 212-218.

Susilo, A. P., van Dalen, J., Scherpbier, A., Tanto, S., Yuhanti, P., & Ekawati, N. (2013). Nurses’ roles in informed consent in a hierarchical and communal context. Nursing Ethics, 20(4), 413-425.

Urban, G. S. (2013).The eternal newcomer: Chinese Indonesian identity from Indonesia to the United States. LUX: A Journal of Transdisciplinary Writing and Research from Claremont Graduate University,3(1), 19.

*Astrid Pratidina Susilo
University of Surabaya,
Jalan Raya Kali Rungkut,
Surabaya 60293, Indonesia
Tel: +62-31-298-1000
Email: astrid.pratidina@gmail.com

Published online: 3 September, TAPS 2019, 4(3), 13-23
DOI: https://doi.org/10.29060/TAPS.2019-4-3/OA2052

Cristelle Chow¹, Raveen Shahdadpuri¹ & Fred Stevens²

¹KK Women’s and Children’s Hospital (KKH), Singapore; ²School of Health Professions Education, Maastricht University, The Netherlands

Abstract

Practice Highlights

• Longitudinal incorporation of cultural awareness into residency training is preferred over rigid curricula.
• Hospital policies, faculty development, reorientating training and assessment methods are essential to integrating cultural awareness into residency training.

I. INTRODUCTION

Cultural competence in health care refers to the ability of an organisation to provide care to patients with diverse values, beliefs and behaviours, in order to meet different social, cultural and linguistic needs (Betancourt, Green, & Carrillo, 2002). There is increasing awareness that cultural competence is required to reduce racial and ethnic disparities in health care (Betancourt, Green, Carrillo, & Ananeh-Firempong II, 2003). These disparities commonly result from care expectations, and the ability to understand the medical condition and its management, which are driven by cultural factors (Gornick, 2000).

In paediatrics, cultural competent health care improves health services satisfaction and treatment compliance (Britton & American Academy of Pediatrics Committee on Pediatric Workforce, 2004). Similarly, failure to consider a patient’s language and culture can result in significant health care barriers (Brotanek, Seeley, & Flores, 2008). Cultural training in paediatrics has been designed to enhance the knowledge base of local cultures, allow the development of negotiation skills during cultural encounters, and to provide patient-centred care while taking into consideration family beliefs and practices (Macdonald, Carnevale, & Razack, 2007).

However, many traditional cultural competence programs have been designed to reduce health care barriers between underserved sociocultural groups and more privileged individuals (Gornick, 2000), which could inadvertently reinforce racial and ethnic biases (Gregg & Saha, 2006). Residents could also make assumptions that culture is dictated by ethnicity, although both are not interchangeable (Kumas-Tan, Beagan, Loppie, MacLeod, & Frank, 2007). Additionally, although cultural competence programs can increase healthcare professionals’ knowledge in managing culturally diverse patients, these are often standalone, discrete programs focusing on specific competencies which may have limited long-term impact on patient care (Horvat, Horey, Romios, & Kis-Rigo, 2014; Renzaho, Romios, Crock, & Sønderlund, 2013). Cultural competence programs can also be difficult to implement into already overbooked resident schedules and longitudinal integration into the overall curriculum can be challenging (Seeleman, Suurmond, & Stronks, 2009; Stevens & Goulbourne, 2014). Hence, the importance of this study is to add the direct practical experiences of health professionals and patients, moving beyond course information to actual application of learning.

In today’s diverse world, cultural competency training in healthcare cannot merely be focused on superficially understanding minor ethnic groups, but rather, on truly appreciating the patient perspective and how it is impacted by culture. For example, a “culturally competent” resident may be able to describe traditions or practices of an ethnic minority, but be unable to understand why a patient refused a recommended treatment. Additionally, at present, little is known about what programmatic interventions should look like, or should include in the clinical context which could be effectively implemented into existing residency programs to support on-the-job learning of cultural competency.

Furthermore, as the majority of published studies involve homogeneous groups of Western-based health professionals interacting with non-Western patients (Horvat et al., 2014; Renzaho et al., 2013), this study’s inclusion of heterogenous non-Western health professionals and patients provides a broader perspective of cultural awareness. A paediatric hospital setting also has the unique caregiver viewpoint which can be more complete and perceptive compared to an adult’s patient viewpoint, as the patient’s care is viewed from a well person’s perspective, often with limited input from the patient herself.

Seeleman et al. (2009) described a broad conceptual framework for teaching and learning cultural competence. This included knowledge, awareness of how culture and social context shapes individual behaviour, prejudice and thinking, and the ability to provide understandable information to patients in an adaptable and creative way. Hence, building upon this conceptual framework, this study aims to utilise actual on-the-job experiences to formulate practical, evidence-based recommendations for cultural awareness training to be integrated into existing residency programs, with the aim of providing residents with an ability to view culture in healthcare holistically, in the absence of a traditional cultural competency curriculum. Cultural awareness, an appreciation of the dimensions of culture and values, is more applicable in this study instead of cultural competence, which refers to the application of cross-cultural skills through awareness, knowledge and sensitivity, as no actual cultural competency assessments were performed. Cultural competency assessments are also limited in their quantitative nature, and are not sufficiently explorative. Furthermore, cultural awareness is part of the process of an organisation achieving cultural competence and needs to be thoroughly explored prior to discussing cultural competence. As the theme of culture in this study’s context has not been adequately studied, this study focuses on exploring cultural awareness to determine the cultural attitudes of healthcare staff and patients, as a primer to future studies on cultural competence.

Culture is understood primarily through interpersonal interactions, and cultural differences drive the premise of cultural competency curricula. In order to create a practical dimension of this study, rather than to blindly adopt Western-style cultural competency curricula, this study also aims to explore interpersonal interactions as well as whether on-the-job experiences provide informal opportunities for learning about culture.

The research questions for this study are:

1. How do residents currently demonstrate their awareness of cultural differences between themselves and their colleagues, patients and caregivers?
2. What is the perceived impact of cultural differences on interpersonal and group interactions among healthcare staff, patients and caregivers?
3. What are the learning experiences of residents through interacting with their culturally different colleagues, patients and caregivers?

A. Conceptual Theoretical Framework

This study focused on learning experiences because medical professionals in-training most likely acquire cultural awareness through the cross-cultural encounters that they personally experience (Javier, Hendriksz, Chamberlain, & Stuart, 2013), which is based on the experiential learning theory (Kolb, Boyatzis, & Mainemelis, 2001). This theory has also formed the basis of early transcultural health professions education (Carpio & Majumdar, 1993; Lockhart & Resick, 1997).

As reducing cultural competency to a set of knowledge, skills and attitudes can result in rigid stereotyping, it is more important for health care professionals to value and work towards culturally competent practices instead. Therefore, it is hoped that through residency training, residents can demonstrate being affectively neutral while being relational and sensitive (Stevens & Goulbourne, 2014). This study focuses on residents’ behaviour because demonstration of their cultural awareness illustrates their stage in the conscious and progressive process of cultural competence, that requires a readiness to learn, reassess, adapt, accept and to experience personal acculturation (Kumagai & Lypson, 2009). The residents’ demonstration is a reflection of the perceived impact of cultural differences, both in interpersonal and group interactions, which is another focus of this study.

While the health professional is experiencing this process, the patient is also experiencing the cultural encounter: he/she determines whether the physician is culturally sensitive, multidimensional, and acknowledges his or her needs (Conrad & Poole, 2012). Hence, the experiences of patients and caregivers are also an integral part of the research process through their observation of residents’ demonstration of cultural awareness and their perceived impact of cultural differences on residents’ behaviour. Based on the developmental phases of cultural competency as described by Culhane-Pera, Reif, Egli, Baker and Kassekert (1997), it is desired that patients and caregivers experience the residents’ acceptance of different cultural beliefs, values and behaviours on health, disease and treatments (Level 3).

II. METHODS

A. Design

A qualitative phenomenological approach is taken to this study which used in-depth exploration of experiences of a small number of individuals. In this study, phenomenology focused on the relationships of people, things and places just as they have been experienced and created, to understand how culture is “in use” (Frykman & Gilje, 2003). These relationships and experiences shape how cultural awareness is formed and embedded in the lives of the individuals.

This study was approved by the SingHealth Centralised Institutional Review Board.

B. Setting

The study context was a tertiary paediatric hospital, where patient and staff populations are ethnically and culturally diverse. The ethnic composition of Singapore consists of predominantly Chinese, followed by Malay, Indian and other ethnicities, including Caucasians and mixed-races. Singaporeans are an immigrant population, originating mainly from Asia but also from Western countries as well.

C. Subjects

Paediatric residents, faculty, nurses and caregivers of children who had experienced medical care in the institution were purposefully sampled to ensure diversity in cultural backgrounds. These cultural differences included country of birth, first language, religion, location where undergraduate and/or postgraduate qualification was obtained, age and gender. The staff sample included residents, faculty and nurses with different educational and work experiences. Caregivers of different countries of origin, occupations and socio-economic status were selected. Recruitment was performed by asking specific individuals of specific backgrounds for consent, and conducting the interview, before asking other individuals. Recruitment continued until data saturation was reached and the themes expressed were repetitive. In total, six residents, four faculties, four nurses and five caregivers participated in this study, with a total sample size of 19 individuals. Of the six residents, half were junior and half were senior residents, and half of them had received their education locally. Of the four faculties, two were consultants, one was a senior consultant and an associate consultant. Two faculty members were born overseas and one was educated locally. Of the four nurses, two were nurse managers, one was a senior staff nurse and the other a staff nurse. Two nurses received initial training overseas before starting work in Singapore. The educational level of the caregivers ranged from primary school to Masters’ degree and their occupations were in areas of education, administration, finance and customer service. The three main races in Singapore were represented in all participant groups.

D. Instruments

In order to elicit experiential descriptions which may involve personal and potentially sensitive information from participants, in-depth semi-structured individual interviews were utilised. The interviewer was a faculty member involved in residents’ and/or nurses’ education but not in assessments and was not directly involved in the medical care of the participants and their children.

Information about demographic and cultural background was obtained from all participants. Interview questions were structured to encourage conversational responses and clinical vignettes (Table 1) were used to help participants contextualise their discussions. Clinical vignettes were derived from various collective experiences of healthcare staff in culturally sensitive situations experienced in the workplace setting. Structured interview questions to staff included questions such as “Sometimes we may connect better with some colleagues or some patients. Do you think so? What do you think makes you feel more comfortable with them?”. For caregivers, clinical vignettes were not relevant hence the interview started with asking about a healthcare professional they trusted and why they felt this way. The interview then progressed to asked specifically about their experiences having their child being managed by a paediatric resident, and their cultural preferences in their healthcare team.

Table 1. Clinical vignettes

E. Data Analysis

Thematic analysis was performed on by hand through interview transcriptions by two independent reviewers (CC and RS) to identify emerging themes through descriptive phrases utilised by participants during interviews. Both reviewers analysed all transcripts individually prior to further discussion of the themes. Themes between sources were compared to determine the presence of convergence or divergence. The final themes emerged were synthesised to more clearly elucidate the phenomenon of cultural awareness among paediatric residents. Any discrepancy in opinions by the two independent reviewers proceeded with a further discussion with a third reviewer (FS).

III. RESULTS

In this section, the results are arranged according to the main themes derived from the research, based on issues and concerns that arose from the interviews.

A. Residents’ Awareness of Cultural Differences

In general, residents’ awareness of cultural differences among patients is present but limited to specific aspects.

1) Linguistic needs of patients:In the first clinical vignette, most residents made an initial assumption that this non-local patient would be able to speak English. A resident also shared about the patient’s responsibility in obtaining a translator, instead of this being the initiative of healthcare providers.

Time was also mentioned as a major factor in not being able to meet the patients’ linguistic needs, as waiting for a translator was time-consuming.

Faculty expressed their concerns that residents are often not aware of the importance of the source of history from a non-native language-speaking family and may avoid interacting with the family altogether:

“…what I have seen is that (residents) would have just gone in and examined the patient as per usual and… not really speak to the family.”

All residents were able to identify that despite usage of translators, information can get lost in translation. They were also aware of their limitations in speaking to patients who preferred a language the residents themselves were not fluent with, and hence tried to overcome this through finding appropriate translators wherever possible.

2) Sociocultural needs of patients:Resident seniority in training and own sociocultural background had an impact on their awareness of cultural differences. Senior residents were more likely to discuss self-prejudices and reflect upon how cultural background can result in patients having gender and racial bias. A resident of a minority race was concerned that increasing ethnic segregation has resulted in poorer inter-cultural awareness.

The majority of residents did express empathy when responding to the second clinical vignette, “Try to understand where the parents are coming from”, and attempted to accommodate parental requests. However, all residents, and most nurses, did not identify cultural factors as having a role in the parent’s refusal of the lumbar puncture. The exceptions were a resident and nurse of Malay ethnicity, who were able to mention cultural influences in the caregiver’s decision. A non-ethnic Malay resident said:

“I often wondered to myself too… so actually more often than not I do realise it’s with Muslim patients, but when you ask them why, they cannot really say why actually.”

This reflects her genuine curiosity in whether a cultural influence was present, which is a key feature in developing cultural competence.

Most faculty consistently felt that residents tended to put more emphasis on the medical aspects rather than the sociocultural aspects of patient care. Similarly, nurses felt that doctors may not actively seek out cultural backgrounds of patients, and perceived that nurses are more aware of cultural concerns compared to doctors.

Caregivers did not explicitly mention that residents were insensitive to their cultural needs, but were more affected by residents’ lack of ability to identify parental struggles. A mother shared:

“They will keep on asking me, ‘Oh the medications… did he really take?’ Ah, so… we were like, OK, we are like bad parents like that we never give him medication,”

This reflects her perception of residents’ insensitivity in their history-taking approach. She also felt that although residents took the time to speak to her about her son, they tended to be unnecessarily technical and overly apologetic.

3) Sociocultural background of colleagues:In discussing the final clinical vignette, faculty and residents did not feel a significant difference in professional behaviour between locally and non-locally trained doctors.

Consistently, faculty, residents and nurses felt that locally-trained junior doctors tend to be superior with regards to their understanding of systems-based practice but not professional behaviour. To residents and nurses, seniority, experience in the local system and personality were more important than the sociocultural background of junior doctors.

B. Impact of Cultural Differences on Interpersonal and Group Interactions

There are two group interactions emerged from this data. First is the interactions between paediatricians and caregivers and second is the cultural differences on interpersonal and group interactions. The first four sub-themes are more prevalent to the interactions between paediatricians and caregivers while the last sub-theme is related to interprofessional relationships.

1) Cultural and ethnic bias of caregivers towards doctors: While working in a multi-cultural hospital environment, some faculty and residents have experienced racist encounters and blatant complaints.

A non-local paediatric resident from another culturally diverse country observed:

“I feel that there’s still bias among different ethnicities… for example, ‘I want to see a Chinese doctor’, ‘I want to see an Indian doctor’, ‘I don’t want to…’”

A paediatric resident recounted an uncomfortable experience:

“The parent actually said, ‘Oh why is there a Malay doctor looking after my child.’”

This has also resulted in her questioning the impact of her own ethnicity on others:

“Sometimes I do worry whether people accept me… because especially the… people’s, the public’s perception of my religion and all that and you know… things that have nothing to do with me but you sometimes feel like, OK, do they judge me…”

Both faculty and residents experienced patients making requests to see doctors of a particular ethnicity, usually, but not always, to facilitate communication in a familiar language. A non-local resident questioned:

“So does that mean that, a Chinese doctor will provide better care for a Chinese patient, because they can relate to each other better?”

To deal with patient requests for doctors from specific ethnicities, faculty were firm that they will not accede to them. However, if the problem was communication due to language barriers, an appropriate translator would be sought.

2) Conceptualising “difficult parents” and attitudes towards them: Throughout the interviews with faculty, residents and nurses, a common emerging theme was the concept of “difficult parents”. A resident shared:

“The making of a difficult parent, I don’t feel it’s like, any race or culture, I feel like it’s different things… it’s like how the parents were raised, and then, what their views are on life… because I mean you have difficult parents from across all races… I don’t even think it’s related to education.”

Although this was echoed by most healthcare staff, there were certain subgroups of parents who were perceived to be more “difficult”. For example, residents identified some of these parents to be parents of newborn babies, parents of chronically ill patients who were familiar with navigating the healthcare system, and parents who were resistant to any healthcare intervention. Some residents also highlighted experiences with parents from certain countries and of certain ethnicities who tended to be more “difficult” and would mentally prepare themselves for the encounter.

A resident, while not explicitly mentioning his own bias, tended to explain certain negative behaviours through the caregivers’ backgrounds by using certain descriptive terms such as “one was a professor in (a university)”and “one is from… (specific country)”. However, a senior resident self-reflected about being cautious in over-labelling “difficult parents”:

“Personally, I think we have to ask ourselves, whenever someone is perceived to be difficult… whether at that time… in that situation, whether we would have done the same…”

In response to these “difficult parents”, faculty expressed concerns that residents may sometimes shy away from communicating with them. They may also not be keen to find out the underlying reason why the parents are behaving in a particular way, and use the term “difficult” to simply explain the behaviour. Residents may not be aware that parents actually do notice this avoidant behaviour.

Caregivers collectively did not mention that they preferred doctors of a specific ethnicity, unless it was to facilitate communication. There was also no specific concern about being served by healthcare staff of a different ethnicity.

Caregivers were aware that foreign doctors (i.e. international medical graduates not originally from Singapore) may do things differently, and some preferred local doctors as they “could communicate more effectively”. A parent perceived that local doctors were more passionate about their work compared to foreign doctors, especially with regards to addressing parent or patient needs rather than just completing the consultation. Despite this, parents would accept foreign doctors who are knowledgeable, caring and sincere.

3) Communication: Faculty and nurses unanimously felt that residents, especially those with less experience, were not able to pitch conversations with parents at the right level. According to faculty:

“Typically they’ll follow a cookie cutter checklist as opposed to tailor the advice to the individual patient”.

A junior resident also acknowledged her limitations in doing this effectively, as she mentioned:

“I would… approach the parents as I would with any parents… I would pretty much give the same package of information to everybody…”

In a similar manner, a parent raised a concern about residents not being spontaneous in providing detailed information, such as investigation results, even though both parents were well educated.

Additionally, caregivers felt that with residents there tended to be a lack of “human interaction”. A caregiver also compared nursing behaviour to doctor behaviour,

“The doctors will only call (my child’s) name when they need to do the examination, or some investigation, but for nurses, usually when we walk by… they will call “Hey, (name of child), Hi! How are you today?’… it’s different.”

4) Impact on patient care: Faculty and residents were aware that healthcare staff can form their own judgments and bias about patients. As mentioned by a faculty:

“There’s some element of judgment, that’s very clear, a lot of times… I can see my staff judging people for the way they speak for where they might come from… clearly everybody judges here.”

However, all faculty and residents agreed that patient care was the same regardless of the sociocultural background of the patient.

Interestingly, a non-local parent felt that there may be a possible disparity of care between local and foreign patients,

“Maybe if the patients are local… (the healthcare staff) look like want to take care more… maybe the way they… talk to us or behave to us maybe a bit different.”

5) Inter-professional relationships:Being foreign or from a minority race had an impact with regards to developing inter-professional relationships with colleagues. Language was identified as a barrier, if colleagues spoke a language not understood by non-locals.

A Muslim resident found non-Halal out-of-hospital dinners a personal struggle to attend, which compromised team bonding. These were not concerns of those from majority ethnic groups.

C. Learning Experiences

Most residents’ learning about cultural differences was from personal experiences, rather than from formal training. This was reflected in the following phrases from residents, “I think a lot of my learning is best done informally like at clinically, on the ground”, “it’s more a sum of your experiences”and “clinical experience comes in, dealing with people from different backgrounds”.

Residents were able to reflect upon patient complaints made against them, experiences of miscommunication, having their opinion rejected by caregivers, and with “difficult” parents. They also learnt that when dealing with a challenging conversation, “bring(ing) the focus back to the child”was critical.

Residents also mentioned the importance of role-modelling by seniors,

“… what I really like is watching the registrar or the consultant… then I learn it for the next time.”

1) Role of a formal cultural competency curriculum:Residents trained in Western medical schools shared their experiences with cultural competency modules. Residents trained in Asian medical schools, however, claimed they had not received such prior training. However, all residents, regardless of training institution, shared their concerns about the limitation of a structured cultural competency curriculum. A resident shared,

“I don’t think there’s anything really teachable… even if you had to sit in a course, I don’t know how much everyone would actually learn…”

Faculty had mixed views about the implementation of such a curriculum – those who had experienced one were keen for structured teaching, while those who did not, remained sceptical. The main reason for this was that experiential learning appeared to be more important than a formal cultural competency curriculum.

IV. DISCUSSION

This study provided insight into aspects of cultural awareness among healthcare staff and was able to address the study’s research questions to a large extent. Some proposed recommendations based on the themes derived from issues and concerns raised in the interviews are presented in Table 2. These suggestions on programmatic interventions in the clinical context could supplement on-the-job experiential learning of cultural awareness, addressing the current literature gap in this area.

Note: Components are characterised as follows: FD – Faculty Development; L – Leadership; P – Policy; IC – Involve Caregivers

Table 2. Recommendations for integrating cultural awareness into existing residency programs

In addressing the study’s research questions, firstly, although paediatric residents were aware of their culturally diverse work environment, they were lacking in addressing some fundamental patient needs. With a few exceptions, they tended to have an instrumental rather than expressive view on patient care – getting their job done rather than exploring the patient’s perspective, as described in a study on paediatrician communication (Wassmer et al., 2004). In similar studies in Western contexts, residents acknowledged the importance of cross-cultural factors in medical care (Park et al., 2005; Park et al., 2006), but no studies have been done specifically exploring faculty or nursing observations of actual resident behaviour for comparison. Park et al.’s (2006) study discusses the same barriers that were raised by residents to providing culturally competent care, including time, language and availability of interpreters. To manage this tension between instrumental and expressive behaviour, in this study residents often deferred to nurses to address issues such as getting a translator, or finding out more about the patient’s background, while focusing their limited time on primarily medical aspects – vital parameters, prescriptions and therapy. Hence, nurses, being more culturally aware, should be empowered to highlight issues to doctors where appropriate. This can be done via faculty encouraging nurses to speak up during ward rounds, or having interprofessional ward rounds. Furthermore, observed clinical encounters of residents with patients on ward rounds often help faculty provide feedback to residents on their sensitivity in addressing parental concerns. Observed clinical encounters have been supported, but with a specific mention of interactions with patients from diverse backgrounds, which is more clearly understood in Western rather than in this study’s context (Park et al., 2006).

Secondly, cultural differences in interpersonal relationships can result in cultural and ethnic bias of caregivers towards doctors, and bring about the concept of “difficult parents”.

It is unfortunate, but not surprising, that there exists an ethnic bias in the hospital setting. Much of this bias is unconscious and a reflection of harmful ignorance, rather than an intention to offend. While the association of doctors’ bias and healthcare disparities have been discussed in the literature, with a recent survey showing 40% of doctors practising in the United States reporting bias towards specific patient groups (Peckham, 2016), patients’ bias against doctors has not been studied. On the flip side, in this study, doctors’ bias was not deemed to result in poorer patient care. Similarly, studies have shown that implicit bias in healthcare has not shown to negatively influence clinical judgment, though it has been associated with poorer physician-patient relationships (Blair, Steiner, & Havranek, 2011). An Israeli study also found that ethnicity significantly affected the patient’s perception of the cultural competence of the physicians and their satisfaction with their medical care, possibly reflecting the cultural bias associated with historical conflict and recent migrant influx (Ohana & Mash, 2015). For residents who may be victims of cultural or ethnic bias, faculty should address these experiences in a sensitive, professional manner, and firm policies against discrimination should be reinforced. Engaging and involving residents in formulating these policies may also emphasise the importance of the issues of cultural and ethnic bias to healthcare leadership.

The concept of “difficult parents” emerged from many interviews. Healthcare staff believed that the “making” of a “difficult parent” stemmed from sociocultural upbringing, and some residents were also forthcoming in identifying certain patient groups whom they felt were more consistently “difficult”. While the avoidant behaviour of residents is understandable due to fear and inexperience, faculty are indispensable in facilitating residents’ uncovering the hidden meaning behind a child’s illness, understanding the reciprocal nature of partnerships, and defining the physician’s and parent’s goals, roles and expectations (Sunde, Mabe, & Josephson, 1993), through provision of culturally competent communication, which involve (using the mnemonic ‘LISTEN’): Listen, Interest, Sincerity, Trust, Empathy, and Neutrality (F. Stevens, personal communication, March 2, 2017). Through the intentional process of facilitated feedback, residents can also be more aware of their own unconscious biases and cultural assumptions. Although communication courses are rampant in medical education, these resource-consuming courses may often teach “desirable” behaviour rather than focus on relationship-building. Hence, integrating the ‘LISTEN’ mnemonic into facilitated on-the-job learning by faculty is likely to be more beneficial for residents longitudinally.

Additionally, there appears to be a role for parents as teachers, to educate residents about the importance of family-centred care for the young patients and their families (Blasco, Kohen, & Shapland, 1999). However, this requires caregivers to be comfortable with speaking about their experiences, as some may be shy or worried about opening up to their doctors. This can be integrated into on-the-job training through asking caregivers and patients about what concerns them the most and giving them time to provide feedback about the team’s care after the clinical encounter.

There is also a need to increase cultural sensitivity within the workplace, to improve inter-professional working relationships and enhance team interactions. Focusing on inter-professional teams, through implementation of the team management theory, can help counter exclusion, distrust and fear by creating bonds through working and interacting together (Schreiber, 1996). Ensuring that teams are diverse can encourage team members to learn from each other and breakdown sociocultural barriers. Although Pololi, Cooper and Carr’s (2010) study mentioned about under-represented minority faculty members experiencing difficulties with cross-cultural interprofessional communication, this did not appear to be a concern in this study’s findings. Buddy and mentorship systems, leadership training in cultural sensitivity and faculty development in providing and assessing cultural competent communication are possible interventions. These aspects were also emphasised in Pololi et al.’s (2010) study, with an additional mention of the danger of devaluing research in health disparities, which is a problem that conservative multicultural societies like Singapore may also face.

The last research question explores learning experiences, which often arise from residents’ personal experiences and observations, but this learning is incomplete without adequate reflection and discussion as described in the experiential learning theory (Kolb et al., 2001). Physician self-reflection, self-knowledge, and self-critique have been identified as critical components of cultural competence (Tervalon & Murray-Garcia, 1998). Therefore, through challenging patient encounters and patient complaints, faculty have to encourage these processes for residents to develop deeper cultural awareness and sensitivity towards others. Some recommendations include complaint debriefing sessions and provision of culturally competent healthcare as a hospital key performance indicator. Leadership and role-modelling also serves as an important tool to facilitate residents’ learning of culturally sensitive behaviours, based on the social learning theory, reinforcing the importance of faculty development in this area (Dauvrin & Lorant, 2015). Interestingly, in Park et al.’s (2005) study exploring residents’ experiences in learning cross-cultural care, residents perceived that attendings were not consistently supportive or knowledgeable in cross-cultural care, hence role-modelling by faculty was not deemed to be particularly helpful in their learning. In this context, however, culture was interpreted largely as race and ethnicity, rather than other aspects of culture that become more evident through interpersonal interactions which this study also explored. Mixed faculty and resident perceptions on the role of a structured cultural awareness and competency curricula are also present in studies done in Western contexts, with a unanimous agreement that cultural awareness training goes beyond courses, as on-the-job experiences are essential for impactful learning (Beagan, 2003; Park et al., 2006; Shepherd, 2018). Similarly, while the assessment of cultural awareness should be attempted, it has to be multifaceted and not dependent on conventional assessment methods such as Objective Structured Clinical Examinations (Dogra & Wass, 2006).

As the majority of studies involve homogeneous groups of Western-based health professionals interacting with non-Western patients (Horvat et al., 2014; Renzaho et al., 2013), this study’s inclusion of heterogeneous, non-Western health professionals provides a broader perspective on cultural awareness. Furthermore, a paediatric study provides the additional caregiver viewpoint, which can be more holistic and sensitive compared to an adult patient’s viewpoint as the focus is on the quality of another person’s care and not oneself. The limitations of this study are a small sample size, given some homogeneity in the themes that emerged, and that only a single institution was studied, as institutional culture also has an impact on resident training and behaviour.

V. CONCLUSION

Resource-intensive cultural competency curricula may not always be feasible to be integrated into busy residency programs. Based on the broad issues developed in this study, there are small but significant evidence-based practical methods to integrate cultural awareness into the clinical context of postgraduate training, to ensure longitudinal reinforcement of the importance of cultural sensitivity.

Notes on Contributors

Cristelle Chow and Raveen Shahdadpuri are general paediatricians with an interest in clinical education. Fred Stevens is an Associate Professor with Maastricht University and his research includes medical education and culture.

Ethical Approval

This study was approved by the SingHealth Centralised Institutional Review Board with CIRB reference number 2016/2648.

Acknowledgements

The authors would like to acknowledge KK Women’s and Children’s Hospital staff, trainees and patients who were interviewed for this study.

Funding

This study did not receive any external funding.

Declaration of Interest

The authors do not have any conflict of interest to declare.

References

Beagan, B. L. (2003). Teaching social and cultural awareness to medical students: “It’s all very nice to talk about it in theory, but ultimately it makes no difference”. Academic Medicine, 78(6), 605-614.

Betancourt, J. R., Green, A. R., & Carrillo, J. E. (2002). Cultural competence in health care: Emerging frameworks and practical approaches. Retrieved from https://www.commonwealthfund.org/
publications/fund-reports/2002/oct/cultural-competence-health-ca
re-emerging-frameworks-and

Betancourt, J. R., Green, A. R., Carrillo, J. E., & Ananeh-Firempong II, O. (2003). Defining cultural competence: A practical framework for addressing racial/ethnic disparities in health and health care. Public Health Reports, 118(4), 293-302.

Blair, I. V., Steiner, J. F., & Havranek, E. P. (2011). Unconscious (implicit) bias and health disparities: Where do we go from here? The Permanente Journal, 15(2), 71-78.

Blasco, P. A., Kohen, H., & Shapland, C. (1999). Parents-as-teachers: Design and establishment of a training programme for paediatric residents. Medical Education, 33(9), 695-701.

Britton, C. V., & American Academy of Pediatrics Committee on Pediatric Workforce. (2004). Ensuring culturally effective pediatric care: Implications for education and health policy. Pediatrics,114(6), 1677-1685.

Brotanek, J. M., Seeley, C. E., & Flores, G. (2008). The importance of cultural competency in general paediatrics. Current Opinion in Pediatrics, 20(6), 711-718.

Carpio, B. A., & Majumdar, B. (1993). Experiential learning: An approach to transcultural education for nursing. Journal of Transcultural Nursing, 4(2), 4-11.

Conrad, C., & Poole, M. S. (2012). Strategic organizational communication: In a global economy(7th ed.). Chichester, England: John Wiley and Sons Ltd.

Culhane-Pera, K. A., Reif, C., Egli, E., Baker, N. J., & Kassekert, R. (1997). A curriculum for multicultural education in family medicine. Family Medicine, 29(10), 719-723.

Dauvrin, M., & Lorant, V. (2015). Leadership and cultural competence of healthcare professionals. Nursing Research, 64(3), 200-210.

Dogra, N., & Wass, V. (2006). Can we assess students’ awareness of cultural diversity? A qualitative study of stakeholder’s views. Medical Education, 40(7), 682-690.

Frykman, J., & Gilje, N. (2003). Being there: An introduction. In J. Frykman & N. Gilje (Eds.), Being there: New perspectives on phenomenology and the analysis of culture(pp. 7-52). Lund, Sweden: Nordic Academic Press.

Gornick, M. E. (2000). Disparities in Medicare services: Potential causes, plausible explanations, and recommendations. Health Care Financing Review, 21(4), 23-43.

Gregg, J., & Saha, S. (2006). Losing culture on the way to competence: The use and misuse of culture in medical education. Academic Medicine, 81(6), 542-547.

Horvat, L., Horey, D., Romios, P., & Kis-Rigo, J. (2014). Cultural competence education for health professionals. Cochrane Database of Systematic Reviews, 2014(5), CD009405. https://doi.org/10.1002/14651858.CD009405.pub2

Javier, J. R., Hendriksz, T., Chamberlain, L. J., & Stuart, E. (2013). Cross-cultural training in pediatric residency: Every encounter is a cross-cultural encounter. Academic Pediatrics, 13(6), 495-498.

Kolb, D. A., Boyatzis, R. E., & Mainemelis, C. (2001). Experiential learning theory: Previous research and new directions. In R. J. Sternberg & L. F. Zhang (Eds.), Perspectives on thinking, learning, and cognitive styles(pp. 227-247). New York, NY: Routledge.

Kumagai, A. K., & Lypson, M. L. (2009). Beyond cultural competence: Critical consciousness, social justice and multicultural education. Academic Medicine, 84(6), 782-787.

Kumas-Tan, Z., Beagan, B., Loppie, C., MacLeod, A., & Frank, B. (2007). Measures of cultural competence: Examining hidden assumptions. Academic Medicine, 82(6), 548-557.

Lockhart, J. S., & Resick, L. K. (1997). Teaching cultural competence: The value of experiential learning and community resources. Nurse Educator, 22(3), 27-31.

Macdonald, M. E., Carnevale, F. A., & Razack, S. (2007). Understanding what residents want and what residents need: The challenge of cultural training in pediatrics.Medical Teacher, 29(5), 444-451.

Ohana, S., & Mash, R. (2015). Physician and patient perceptions of cultural competency and medical compliance. Health Education Research, 30(6), 923-934.

Park, E. R., Betancourt, J. R., Kim, M. K., Maina, A. W., Blumenthal, D., & Weissman, J. S. (2005). Mixed messages: Residents’ experiences learning cross-cultural care. Academic Medicine,80(9), 874-880.

Park, E. R., Betancourt, J. R., Miller, E., Nathan, M., MacDonald, E., Ananeh-Firempong II, O., & Stone, V. E. (2006). Internal medicine residents’ perceptions of cross-cultural training: Barriers, needs, and educational recommendations. Journal of General Internal Medicine, 21(5), 476-480.

Peckham, C. (2016, January 13). Medscape lifestyle report 2016: Bias and burnout. Retrieved from https://www.medscape.com/
slideshow/lifestyle-2016-overview-6007335 

Pololi, L., Cooper, L. A., & Carr, P. (2010). Race, disadvantage and faculty experiences in academic medicine. Journal of General Internal Medicine, 25(12), 1363-1369.  

Renzaho, A. M., Romios, P., Crock, C., & Sønderlund, A. L. (2013). The effectiveness of cultural competence programs in ethnic minority patient-centered health care – A systematic review of the literature. International Journal for Quality in Health Care, 25(3), 261-269.

Schreiber, E. J. (1996). Muddles and huddles: Facilitating a multicultural workforce through team management theory. The Journal of Business Communication (1973), 33(4), 459-473.

Seeleman, C., Suurmond, J., & Stronks, K. (2009). Cultural competence: A conceptual framework for teaching and learning. Medical Education, 43(3), 229-237.

Shepherd, S. M. (2018, January 22). Cultural awareness training for health professionals can have unintended consequences [BMJ Opinion Blog]. Retrieved from https://blogs.bmj.com/bmj/
2018/01/22/stephane-m-shepherd-cultural-awareness-training-for-health-professionals-can-have-unintended-consequences

Stevens, F., & Goulbourne, J. S. (2014). Medical education, cultural competence in. In W. C. Cockerham, R. Dingwall, & S. R. Quah (Eds.), The Wiley Blackwell encyclopedia of health, illness, behavior and society(1st ed., pp. 1355-1339). Chichester, England: Wiley and Sons.

Sunde, E. R., Mabe, P. A., & Josephson, A. (1993). Difficult parents: From adversaries to partners. Clinical Pediatrics,32(4), 213-219.

Tervalon, M., & Murray-Garcia, J. (1998). Cultural humility versus cultural competence: A critical distinction in defining physician training outcomes in multicultural education. Journal of Health Care for the Poor and Underserved, 9(2), 117-125.

Wassmer, E., Minnaar, G., Abdel Aal, N., Atkinson, M., Gupta, E., Yuen, S., & Rylance, G. (2004). How do paediatricians communicate with children and parents? Acta Paediatrica, 93(11), 1501-1506.

*Dr Cristelle Chow
KK Women’s and Children’s Hospital,
100 Bukit Timah Road, Singapore 229899
Tel: +65 6394 8735
E-mail: cristelle.chow.ct@singhealth.com.sg

Published online: 7 January, TAPS 2020, 5(1), 61-69
DOI: https://doi.org/10.29060/TAPS.2020-5-1/OA2152

Yoshitaka Maeda¹, Yoshikazu Asada², Yoshihiko Suzuki¹ & Hiroshi Kawahira¹

¹Medical Simulation Centre, Jichi Medical University, Japan; ²Center for Information, Jichi Medical University, Japan

Abstract

Practice Highlights

• This method helps inexperienced students brainstorm various scenarios needed for risk prediction.
• It was possible for students to practice risk assessment using What-If Kiken-Yochi Training (W-KY).
• Students could predict a wide variety of risks regarding patients and medical personnel using W-KY.
• It was difficult to predict management, clinical rules, and stakeholders’ risks even with W-KY.

I. INTRODUCTION

A. Issues on Patient Safety Education for Medical Students in Japan

In Japan, students enrol at medical schools immediately after graduating from high school. Safety education is often provided only at higher levels (4^th or 5^th year) in over 60% of medical schools in Japan, when clinical practice begins (Ishikawa, Hirao, & Maezawa, 2008). The following are the reasons: (1) educational practical methods for clinically inexperienced students are not specifically referred to in any guidelines or papers for students in the early years; (2) the educational effect on such students is unclear; (3) higher grade students have sufficient medical knowledge and can immediately apply their patient safety knowledge in clinical practice; and (4) there are few teachers specialising in medical safety, thus making it easier to determine the theme of education for higher grade students. On the contrary, 70% of medical schools only teach the minimum medical safety knowledge through lectures to students who have not yet practiced safety in clinical practice. The main contents of the lectures are analysis tools to prevent the recurrence of incidents, such as Root Cause Analysis and Failure Mode and Effects Analysis, legal responsibility knowledge, ethics, and infection (Mayer, Klamen, Gunderson, & Barach, 2009). However, as there are certain things that can be included immediately in clinical practice, it is difficult to keep students motivated.

The Telluride Interdisciplinary Roundtable (Mayer et al., 2009) and Lucian Leape Institute (2010) indicated that patient safety education should be conducted through a longitudinal curricular approach (including patient safety education in the curriculum of all grades), and it is important to educate lower grade students who have no clinical experience. This would enable students to learn the necessity and importance of patient safety knowledge, and to consider patient safety as implementation science while continuously practicing patient safety skills (Nakajima, 2012). Ishikawa et al. (2008) also emphasised the importance of continuous patient safety education, starting with beginner students so they can acquire good safety habits.

The Telluride Interdisciplinary Roundtable (Mayer et al., 2009) and Lucian Leape Institute (2010) outlined the patient safety competencies that students should acquire. In particular, students lack education on “non-technical skills” (Nakajima, 2012). Some of the necessary non-technical skills students need when they start working after graduation are risk assessment (situational awareness) skills to prevent accidents. This involves advance awareness of safety weaknesses and threats (risks) in the workplace or operations and the ability to avoid these risks (Doi, Kawamoto, & Yamaguchi, 2012; Takahashi, 2010). The World Health Organization’s (WHO’s) Patient Safety Curriculum Guide shows in Topic 6 (Understanding & Managing Clinical Risk) that students have to take correct action when they see an unsafe situation or environment (Walton et al., 2010). For example, when they see wet steps, they should predict the possibility of patients falling. WHO’s guide outlines the four-step process to manage clinical risks: (1) identify the risk, (2) assess the frequency and severity of the risk, (3) reduce or eliminate the risk, and (4) assess the costs saved by reducing the risk versus the costs of not managing the risk. However, this guide does not explain the teaching methods to enable students to identify or predict risks.

B. Educational Method Issues for Risk Assessment in Japan

Kiken-Yochi Training (KYT), a type of risk prediction training, was created by Sumitomo Metal Industries Co., LTD in 1974 (Chen & Mao, 2011), and helps workers understand risks in many kinds of industries, including the medical field (Doi et al., 2012; Hirokane, Shiraki, & Ohdo, 2010). KYT originated in Japan (Ito, Taguchi, & Fujinami, 2014) and has become a common safety management method (Ji, 2014). It increases workers’ awareness of risks and motivation to practice in a team and improves their problem-solving skills (Chen & Jin, 2012). KYT also enables workers to easily conduct on-site risk assessment. In a metal auto parts factory, the accident rate fell by six percent one year after the implementation of KYT (Poosanthanasarn, Sriboorapa, Fungladda, & Lohachit, 2005). In recent years, KYT use has also begun to spread in the medical field in countries other than Japan (Noor, Irniza, Emilia, Anita, & Suriani, 2016). In standard KYT (S-KY), the facilitator presents the learner with illustrations or photographs depicting the work site and guides them through four steps: (1) extracting items and risks considered to be dangerous in the illustration, (2) rating each item’s risk, (3) planning solutions, and (4) selecting urgent solutions (Japan Industrial Safety & Health Association, 2008). In particular, an important skill for medical staff at clinical sites is the ability to predict a myriad of risks from a broad perspective using the illustration in Step 1. More effective KYT has recently been designed, such as KYT using video instead of illustrations and KYT applications for tablets that enable students to easily take risk assessment training alone, such as during breaks, and KYT combining medical simulations of real clinic situations (Kadoyanagi, 2016; Nagamatsu, Miyazaki, & Harada, 2011; Takahashi et al., 2017; Yoneda et al., 2017).

However, in S-KY, the ability of experts to predict risks is higher than that of novices because novices do not have enough knowledge of important areas in the illustrations (Murata, Hayami, & Moriwaka, 2009). Hirokane et al. (2010) pointed out that experts are able to predict risks specifically in order to consider effective solutions to prevent accidents. Therefore, KYT is usually conducted for clinically experienced students, and is rarely implemented for inexperienced students. The reasons are as follows: (1) in S-KY, risk prediction in the illustrations is limited to “specific circumstances”; (2) if medical students practice risk prediction using only these illustrations, they cannot accurately identify risks at clinical sites when they encounter a situation different from the “specific circumstances” in the illustration in the future; and (3) if teachers use illustrations of scenes from non-medical sites to avoid the inability to predict risks in clinically inexperienced students who do not have enough medical knowledge, it is difficult for students to link the risk prediction content with actual clinical sites, to become motivated to learn, and to obtain educational effects from the training.

C. Hypothesis: What-If KYT (W-KY) Applicability

What-If analysis is used as a brainstorming method. It is widely used in the field of service design and brainstorms what kind of things will occur if a particular situation changes (Stickdorn & Schneider, 2012). Based on this method, Mochizuki and Komatsubara (2016) propose a step 0 prior to the existing risk assessment in KYT that is aimed at traffic safety, letting the participants identify various alternate situations: “If the situation of the illustration were different”. Next, the participants perform a risk assessment for each situation developed in Step 0. For example, the facilitator shows participants illustrations depicting sunny daytime “driving scenes on a straight road”. Participants conduct what-if brainstorming and consider various situations, such as “if the weather is different from the illustration”, “if it was night-time”, or “if it is a mountain pass”. They then conduct a risk assessment for each situation. This modification is called the W-KY. The subsequent steps are the same as the S-KY (Steps 1 to 4). By adding Step 0, participants are then better able to predict various risks from the illustrations, and their risk sensitivity increases.

Based on these articles related to S-KY and W-KY, S-KY might be more feasible for clinically experienced students than inexperienced students. Therefore, W-KY might be feasible for clinically inexperienced students, because the What-If analysis (Step 0) might help students’ brainstorming. It also might increase motivation to practice and improve students’ problem-solving capabilities. The central question of this paper is whether the W-KY is effective and feasible for clinically inexperienced students. Existing studies have not adopted the W-KY to medical treatment, and the effect on clinically inexperienced students is unclear.

D. Purpose of This Study

In this study, based on the W-KY, we propose and implement a training method to help clinically inexperienced students predict risks. We also clarify differences in the risk assessment tendencies of students in W-KY versus the S-KY.

II. METHODS

A. Methods in W-KY

We conducted the S-KY and W-KY for 120 medical students half a year after admission in 70-minute compulsory classes. The purpose of these classes was to teach diversity of thinking and thinking of others through discussions in small groups of seven to eight people. The S-KY and W-KY were conducted over two classes (140 minutes). Specifically, the students conducted group work in the following order using three kinds of illustrations (Figure 1) depicting clinical sites. In our class, after individual students brainstormed ideas, they shared ideas with others and learned about the diversity of ideas.

Figure 1. Illustrations used in S-KY and W-KY (Courtesy of Japan Industrial Safety & Health Association [n.d.])

Step 1 (S-KY Step): Individual students predicted what risks exist in the three illustrations, then they used posters to share their prediction results with the group.

Step 2 (W-KY Step): Individual students brainstormed “situations that differed from the illustration” for each of the three illustrations. For example, in Illustration 1, one student thought of “a situation in which only one person was guiding the gurney”.

Step 3 (W-KY Step): Individual students predicted what risks exist in the situations they brainstormed in Step 2, then shared the results in their groups using posters.

Step 4 (S-KY and W-KY Steps): Each student considered solutions to mitigate the risks that they had predicted and submitted a report on them at the end of the class. Specifically, each student selected one risk considered to be the biggest risk in each illustration, and considered three solutions to prevent the risk.

B. Clarification of Differences in Risk Assessment Tendency of Students Between W-KY and S-KY

We compared the prediction results of S-KY in Step 1 and W-KY in Step 3 using the P-mSHELL model (Kawano, 2002). This model shows that factors of medical incidents are patient, management, software, hardware, environment, and liveware (individuals/ teams). P-mSHELL represents the initials of these factors. To evaluate the effectiveness of safety education, it is necessary to ensure that risks related to Human Factors can be predicted from a broad viewpoint. There are several models that explain the cause of Human Factors issues: Lewin’s equation model (human behaviour is determined by factors related to the person and their environment; Lewin, 1936), the 4M model (factors related to Man, Machine, Media, and Management), and Reason’s categories (factors related to patient and provider; task, technology, and tool; and team, environment, and organisation; Reason, 1997). All models consider both human and environmental factors as the background of Human Factors issues. Among these models, P-mSHELL is a highly detailed model that explains human and environmental factors in medicine, thus making it easy to evaluate the effectiveness of safety education. The P-mSHELL model is based on the SHEL model that has been used in analysis of Human Factors issues in aviation. Molloy and O’Boyle (2005) pointed out this model is useful in examining errors in clinical site, and may have some potential in training medical staff about Human Factors. Therefore, the P-mSHELL model is frequently referenced to understand human error in medical care. It is expected that the target students of this research will be able to predict the safety weaknesses (risks) in the clinical site from the point of view of P-mSHELL and to take preventative solutions. For this reason, this study verifies the educational effect by comparing the danger predicted by S-KY and W-KY using this model. We also summarise student brainstorming results in Step 2 “regarding situations that differ from the illustration” and examine its effect on student risk assessment. Then, we classify the solutions described in the report by students from the point of view of P-mSHELL.

C. Ethical Considerations in This Research

Regarding the ethical use of the results of S-KY and W-KY conducted in the class for research, we emphasised and explained to students that cooperation in this study was voluntary and that declining to cooperate would have no influence on their grades. We explained that students’ grades are scored based on the rubric described in the syllabus and that consent to cooperate in the research could be withdrawn at any time. It was explained that the results of this study may be published after processing, but the student’s personal information would not be revealed in the publication. The students entered their consent to use the results of S-KY and W-KY for this research in the e-learning system, Moodle, and 120 out of 123 students agreed to participate. In this research, we analysed data from students who agreed to participate. This study was considered exempt by the Jichi Medical University Review Board (protocol number 18-014).

III. RESULTS

A. Results of Predicted Risk by S-KY and W-KY

Figure 2 (left) shows the total number of risks predicted by students for each illustration. In addition, Figure 2 (right) shows the average number of predicted risks for each student group. We clarified the difference between S-KY and W-KY using the students’ t-test.

First, students were able to predict a variety of risks regardless of whether S-KY or W-KY was used. With the exception of liveware (team) in Illustration 3, the risks for all elements of P-mSHELL were predicted. In addition, the risks varied widely in the ease of prediction by the illustration (Figure 2, left). In Illustration 1, patient, environment, and liveware (individual) risks were predicted in S-KY and W-KY. In Illustration 2 predicted risks were liveware (person), and in Illustration 3, patient. Risks related to management, software, and liveware (team) were very few in both S-KY and W-KY, and for some illustrations, student groups predicted no risks in some categories.

Despite W-KY being implemented after S-KY, patient and environment risks had approximately the same number in S-KY and the W-KY. The number of liveware (individual) and hardware risks predicted in W-KY was less than in S-KY—liveware (individual): t(16) = 3.47, p < .05; Hardware: t(16) = 3.44, p < .05. In W-KY, many were predicted in Illustrations 1 and 2.

For example, in scenario 1, risks such as, “staff transferred patient to the wrong room” and “the patient would be injured if the stretcher breaks” were predicted in S-KY. In W-KY, risks such as, “If the corridor is dark, staff are not able to notice changes in the patient’s condition”, “If the corridor gets wet, the stretcher may slip and fall”, “If the patient is elderly, the patient will fracture a bone due to impact”, “If the patient has dementia, he forgets about treatment when he wakes up and removes the infusion tube himself”, and “If the staff is busy, he forgets to change the drip, emptying the drip and harming the patient” were predicted.

 Figure 2. Result of predicted risk by S-KY and W-KY (Based on the classification of P-mSHELL)

B. Results of Brainstorming Situations That Differed from the Illustration in Step 2

Table 1 shows how the students brainstormed situations that differed from each illustration in Step 2 as mentioned in the Methods section (refer to A. Methods in W-KY). In Table 1, the situations brainstormed by the students are organised based on each element of P-mSHELL.

First, the situations brainstormed by the students were remarkably diverse regarding patient, liveware (individual), hardware, and environmental factors. Although the number of liveware (individual) and hardware risks predicted in W-KY were less than in S-KY, the liveware (individual) risks predicted in W-KY were diverse.

On the other hand, the situation about management, software and liveware (team) were small in number (Table 1). This is also reflected in the low number of risks predicted for these factors (Figure 2).

Table 1. Results of brainstorming in Step 2 about “situation different from illustration”

C. Solutions Proposed by Students to Mitigate the Risks

In Figure 3, the solutions proposed by the students to mitigate the risks were classified by P-mSHELL. In addition, Table 2 shows a concrete example of solutions for each element. In this paper, it has not been possible to analyse what kind of solutions were considered for each risk in S-KY and W-KY because students considered solutions in a post-class report.

In both S-KY and W-KY, students were able to consider a wide variety of solutions for almost all P-mSHELL factors. Although predicted risks related to management, software, and liveware (team) were very few, students were able to propose a lot of solutions related to them. There were many solutions, especially for software and hardware. For example, for software, double-checking, pointing and calling (occupational safety method), and creating a checklist were suggestion; in hardware, changing the shapes and names of the medicine in order to make it difficult to make mistakes was proposed.

Table 2. Examples of solutions considered by students

 Figure 3. Result of solutions of predicted risks (based on the classification of P-mSHELL)

IV. DISCUSSION

A. Similarity and Difference Between S-KY and W-KY

Students should be able to predict risks in all elements of P-mSHELL as they can minimally experience the risks of each element in a clinical site. The results show that the elements of risk in P-mSHELL in both S-KY and W-KY are similar. Students could predict many patient and environment risks. These elements were drawn in advance in each illustration as shown in Figure 1. In other words, it was possible for students to brainstorm risks regarding stakeholders and medical devices drawn in illustrations in S-KY and W-KY. One of the possible reasons students could predict several patient and environment risks is that even a clinically inexperienced medical student has experience as a patient or has experienced the same situation (e.g., fatigue, immorality, lack of knowledge, etc.) as liveware (individual). Kazaoka, and Otsuka (2003) indicate that nursing students tend to recognise risks that they can imagine as their own and consider important, such as liveware (individual). On the other hand, management, software, and liveware (team) risks were very few in both S-KY and W-KY. The number of patient, environment, management, software, and liveware (team) risks had approximately the same number in S-KY and W-KY. The number of liveware (individual) and hardware risks in W-KY was less than in S-KY. In other words, it is considered that W-KY may cover the risks that can be predicted with S-KY.

Next, we discuss the strengths of W-KY. The risk description by students in W-KY included the risks and information related to various situations. Particularly in W-KY, students were able to brainstorm a wide variety of situations regarding patient, liveware (individual), hardware, and environmental risks (Table 1). This tendency is the same as the risk prediction tendency of S-KY and W-KY, and the reason for this tendency is considered to be the same. In W-KY, for example, regarding the transporting of stretchers, inexperienced students could consider various dynamic situations that can cause accidents, such as wet corridors, crowded corridors, violent or acutely ill patients, and lack of human resources. These situations that students brainstormed, listed in Table 1, are paraphrased as medical accident risks. In other words, in S-KY, students predict only medical accidents related to illustrations (results of accidents), whereas in W-KY, students were able to predict many high-risk situations that can cause medical accidents, and medical accidents (possible cause and results of accidents).

Through W-KY, students may have learned what a high-risk situation is and that clinical tasks can change into various dynamic situations that differ from illustrations. As a result, they may have learned the necessity of risk prediction in clinical practice and the significance of learning patient safety. These points will be explored in future research. In addition, Hirokane et al. (2010) pointed out that it is very important to predict risks specifically to prevent accidents. Therefore, it is possible that even inexperienced students can do this by using W-KY.

B. Limitations of W-KY

Only few students could predict risks management, software, and liveware (team) in S-KY and W-KY. However, students were able to consider solutions for almost all P-mSHELL factors (Figure 3). Contrary to the results of predicting risks in W-KY and S-KY, students were able to propose many solutions related to the aforementioned elements. This means that students can brainstorm these elements. In particular, regarding the software, students can mention the establishment and thoroughness of rules and manuals when planning solutions, and can think sufficiently from that viewpoint. Therefore, improving the format of discussion in W-KY may allow students to predict risks from a wider range of perspectives.

V. CONCLUSION

In this research, we proposed and implemented a training method to help clinically inexperienced students predict various risks. W-KY (brainstorming situations in illustrations and predicting risks based on them), as implemented in this research, allowed clinically inexperienced students to predict risks. We found that the elements of risk in P-mSHELL in both S-KY and W-KY are similar. Students could predict many patient and environment risks. However, with regard to management, software, and liveware (team) factors, S-KY and W-KY appeared to be difficult.

In addition, W-KY enables the prediction of high-risk situations that can cause medical accidents. This is important for predicting risks—including possible causes of accidents—specifically to prevent accidents.

In summary, W-KY can cover the risks that can be predicted by S-KY, and also allows students to consider various dynamic high-risk situations that can cause accidents. This suggests that W-KY can be used instead of S-KY.

In this study, it was not possible to analyse what kind of solutions to prevent risks were considered for each risk in S-KY and W-KY, as students considered solutions in a post-class report. Because this study was conducted as part of the class, we could not obtain data of student perspectives on learning outcomes or transferring learning to practice. The first-year medical school students in this research have the opportunity to receive medical safety education again when they are in the fourth year. At that time, we have plans to provide opportunities for practicing skills by applying W-KY in clinical practice, which will be explored in a future research. In addition, comparisons between students with clinical experience and inexperienced students and between W-KY/S-KY and other educational programs on accident prevention are also future topics of this research.

Notes on Contributors

Yoshitaka Maeda, PhD, is a research associate at the Medical Simulation Center at Jichi Medical University, Japan. He conducted the research supervision, class implementation, and data analysis.

Yoshikazu Asada, PhD, is an assistant professor at the Center for Information at Jichi Medical University, Japan. He contributed to the data compilation regarding the effect of this education.

Yoshihiko Suzuki, MD, is an assistant professor at the Medical Simulation Center at Jichi Medical University, Japan. He contributed to the design and planning of this class.

Hiroshi Kawahira, MD, is a professor at the Medical Simulation Center at Jichi Medical University, Japan. He gave advice on writing this paper and on data aggregation.

Ethical Approval

This study was approved by the Jichi Medical University Institutional Review Board (Protocol number 18-014).

Funding

There is no funding involved for this paper.

Declaration of Interest

The authors report no conflict of interest. The authors alone are responsible for the content and writing of this article.

References

Chen, Q., & Jin, R. (2012). Safety4Site commitment to enhance jobsite safety management and performance. Journal of Construction Engineering and Management, 138(4), 509-519. http://doi.org/10.1061/(ASCE)CO.1943-7862.0000453

Chen, N., & Mao, Y. (2011). Specific statistics and control method study on unsafe behavior in Chinese coal mines. Procedia Engineering, 26, 2222-2229. https://doi.org/10.1016/j.proeng.2011.11.2428

Doi, T., Kawamoto, K., & Yamaguchi, K. (2012). Difference in the level of patient treatment safety analysed by the years of experience of the radiological technologies. Nihon Hoshasen Gijutsu Gakkai zasshi [Japanese Journal of Radiological Technology], 68(5), 608-616. http://doi.org/10.6009/jjrt.2012_jsrt_68.5.608

Hirokane, M., Shiraki, W., & Ohdo, K. (2010). Hazard prediction activities for safety education. Doboku Gakkai Ronbunshuu F/JSCE Journal of Construction Engineering and Management, 66(1), 55-69. https://doi.org/10.2208/jscejf.66.55

Ishikawa, M., Hirao, T., & Maezawa, M. (2008). Study of patient safety education for undergraduates. Igaku Kyoiku / Medical Education (Japan), 39(2), 115-119. https://doi.org/10.11307/mededjapan1970.39.115

Ito, K., Taguchi, H., & Fujinami, K. (2014). Posing questions during experimental operations for safety training in university chemistry experiments. International Journal of Multimedia and Ubiquitous Engineering, 9(3), 51-62. http://dx.doi.org/10.14257/ijmue.2014.9.3.06

Japan Industrial Safety and Health Association. (2008). Sugu ni jissen shirīzu Anzen wo Sakidori Kiken Yochi [Practice series: Ahead of safety, risk prediction]. Tokyo, Japan: JISHA.

Japan Industrial Safety and Health Association. (n.d.). Iryō kango | kiken yobō kunren (KYT) muryō irasutoshīto-shū [Medical/Nursing | Risk prediction training (KYT) free illustration sheet collection]. Retrieved from https://www.aemk.or.jp/kyt/iryo/page/5

Ji, H. J. (2014). A study on safety culture construction for coal mine. Applied Mechanics and Materials, 644-650, 5949-5952. https://doi.org/10.4028/www.scientific.net/AMM.644-650.5949

Kadoyanagi, T. (2016). Shūdan sanka taiken-gata kōshū-yō kizai (dōga KYT)-tō no un’yō jōkyō ni tsuite (tokushū jukō-sha no tokusei ni ōjita kōtsū anzen kyōiku) (Operation of group training equipment [Video KYT]. [Traffic safety education according to the characteristics of the students]). Gekkan Kōtsū, 47(7), 44-55.

Kawano, R. (2002). Iryō risuku manējimento seminā tekisuto [Medical risk management seminar text]. Tokyo, Japan: Tepco Systems.

Kazaoka, T., & Otsuka, K. (2003). Study of nursing students’ recognition of the risks of medical treatment accidents: Using the scenario of role playing drug misadministrations. Journal of Japan Society of Nursing Research, 26(5), 133-143. https://doi.org/10.15065/jjsnr.20030912009

Lewin, K. (1936). Principles of topological psychology. New York, NY: McGraw-Hill. http://dx.doi.org/10.1037/10019-000

Lucian Leape Institute. (2010). Unmet needs: Teaching physicians to provide safe patient care. Boston, MA: National Patient Safety Foundation.

Mayer, D., Klamen, D. L., Gunderson, A., & Barach, P. (2009). Designing a patient safety undergraduate medical curriculum: The telluride interdisciplinary roundtable experience. Teaching and Learning in Medicine, 21(1), 52-58. https://doi.org/10.1080/10401330802574090

Mochizuki, R., & Komatsubara, A. (2016). Proposal of traffic KYT with using What-if analysis. The Human Factors in Japan, 20(2), 79-82.
https://doi.org/10.11443/jphf.20.2_79

Molloy, G. J., & O’Boyle, C. A. (2005). The SHEL model: A useful tool for analyzing and teaching the contribution of Human Factors to medical error. Academic Medicine, 80(2), 152-155.

Murata, A., Hayami, T., & Moriwaka, M. (2009). Visual information processing characteristics of drivers in prediction of dangerous situation-comparison among novice, expert and non-licensed person. In Proceedings: Fifth international workshop on computational intelligence & applications, 2009(1), 254-257. Hiroshima, Japan: IEEE SMC Hiroshima Chapter.

Nagamatsu, I., Miyazaki, I., & Harada, C. (2011). Kango kiso kyōiku ni okeru kiken yochi torēningu (KYT) wo toriireta iryō anzen kyōiku ni kansuru kōsatsu: Dōga jirei o mochiita puroguramu no kōka [A study on medical safety education incorporating risk prediction training (KYT) in nursing education: Effects of programs using movie]. Nihon kango gakkai ronbun-shū. Kango kyōiku, 42, 158-161. Retrieved from
https://iss.ndl.go.jp/books/R000000004-I023685394-00?ar=4e1f&locale=e

Nakajima, K. (2012). 4. Patient safety and quality improvement education for undergraduate medical students. Nihon Naika Gakkai Zasshi [Journal of the Japan Society of Internal Medicine], 101(12), 3477-3483. https://doi.org/10.2169/naika.101.3477

Noor, A. Y., Irniza, R., Emilia, Z. A., Anita, A. R., & Suriani, I. (2016). Kiken yochi training (KYT) in reducing accidents at workplaces: A systematic review. International Journal of Public Health and Clinical Sciences, 3(4), 123-132.

Poosanthanasarn, N., Sriboorapa, S, Fungladda, W., & Lohachit, C. (2005). Reduction of low back muscular discomfort through an applied ergonomics intervention program. The Southeast Asian Journal of Tropical Medicine and Public Health, 36(Suppl. 4), 262-270.

Reason, J. T. (1997). Managing the risks of organizational accidents. Aldershot, England: Ashgate Publishing Limited.

Stickdorn, M., & Schneider, J. (2012). This is service design thinking: Basics-tools-cases. Amsterdam, The Netherlands: BIS Publishers.

Takahashi, A., Mishina, M., Takagi, M., Shimazaki, K., Ishida, T., & Umezaki, S. (2017). Taburetto tanmatsu wo mochiita anzen kyōzai no kunren kōka to anzen kanri e no ōyō: Teisō jūtaku kenchiku genba wo taishō to shite [Effect of safety teaching materials using tablet and its application to safety management: For low-rise residential building]. Specific research reports of National Institute of Occupational Safety and Health, 47, 39-44.

Takahashi, R. (2010). Nontekunikkusukiru torēningu e no chosen [Challenge to non-technical skills training]. In K. Nakajima, & Y. Kodama (Eds.), Iryō anzen koto hajime [Beginning of medical safety] (pp. 36-51). Tokyo, Japan: Igaku Syoin.

Walton, M., Woodward, H., Van Staalduinen, S., Lemer, C., Greaves, F., Noble, D., … Barraclough, B. (2010). The WHO patient safety curriculum guide for medical schools. BMJ Quality & Safety, 19(6), 542-546.     http://dx.doi.org/10.1136/qshc.2009.036970

Yoneda, T., Itami, K., Seki, K., Kawabata, A., Kubota, Y., Yasuhara, O., & Maesako, T. (2017). Changes in nursing students’ risk awareness and understanding of medical incidents after exposure to a simulated incident. Japan Journal of Educational Technology, 41(Suppl.), 17-20.
https://doi.org/10.15077/jjet.S41011

*Yoshitaka Maeda
Medical Simulation Center,
Jichi Medical University,
3311-1, Yakushiji,
Shimotsuke-shi, Tochigi, Japan
Tel: +81 285 58 7455
Email: y-maeda@jichi.ac.jp

Published online: 7 January, TAPS 2020, 5(1), 54-60
DOI: https://doi.org/10.29060/TAPS.2020-5-1/OA2095

Min Jia Chua¹, Gen Lin Foo² & Ernest Beng Kee Kwek²

¹National Healthcare Group, Ministry of Health Holdings, Singapore; ²Department of Orthopaedic Surgery, Woodlands Health Campus, Singapore

Abstract

Practice Highlights

• Study to evaluate value of mentoring by faculty and near-peer mentoring to orthopaedic surgery resident.
• Residents viewed mentoring by faculty and near-peer mentoring programmes as being beneficial and crucial.
• Ideal mentoring programme should be tiered and allow choice of mentors.
• Near-peer mentoring should be a requirement but not necessarily monitored.

I. INTRODUCTION

Mentoring has long been a crucial element of effective resident medical education (Sambunjak, Straus, & Marusic, 2006), with many programmes adopting mentoring by faculty as an integral component of their residency programme. Various models of mentoring and types of mentoring activities have been described including didactic sessions, regular mentor-mentee meetings and group projects (Kashiwagi, Varkey, & Cook, 2013). The benefits of mentoring have also been shown in various studies, by aiding personal and professional development during residency, helping with

career preparation (Ramanan, Taylor, Davis, & Phillips, 2006), improving professional and social skills (increased self-confidence, improved communication skills; Buddeberg-Fischer, & Herta, 2006).

There is however, little data looking at how Orthopaedic residents view mentorship programmes (Flint, Jahangir, Browner, & Mehta, 2009). Furthermore, most studies look at mentoring by faculty with little emphasis on near-peer mentoring. In fact, a literature review has shown that no studies have looked at and compared the two entities in orthopaedic residency programmes.

In our residency programme, a tiered mentorship framework, where both mentoring by faculty and near-peer mentoring are practised, has been in place since 2014. In the former, a mentor of associate consultant grade and above who is in post-fellowship training will mentor a resident while in the latter, a senior resident mentor will mentor a junior resident two residency years below him or her.

The objective of having a near-peer mentorship framework in our programme was to bridge some of the gaps in traditional mentorship. In mentorship by faculty models, there will inevitably be hierarchical distance between mentors and mentees and residents may not feel as at ease approaching their mentors for certain issues. Furthermore, there may be a shortage of faculty members who are also strapped for time and may not be able to devote enough time for holistic mentorship of residents. These mentors are also further away from their residency training years and may not be able to understand some of the issues their mentees face in the current residency climate. It was envisaged that senior residents who are near-peers on the ground will be able to address some of the abovementioned shortcomings in the traditional mentorship model.

Currently, mentorship by faculty is formally monitored by the programme and mandates at least bi-annual meetings with a mentoring form to be filled in while near-peer mentorship is a self-directed initiative by the residents which is more informal with no stipulated frequency of meetings and no compulsory documentation under the residency programme requirements.

Residents were either assigned mentors or chose mentors at the start of the second year of their residency but due to various factors including efflux of faculty mentors or other administrative reasons, some residents do not have either senior resident or faculty mentors or both.

The objectives of this study were to 1) evaluate the orthopaedic surgery residents’ perception of mentoring by faculty and near-peer mentoring and 2) establish factors perceived as being important in mentors and a successful mentoring environment.

II. METHODS

An anonymised online survey with voluntary participation was administered to all orthopaedic surgery residents, from residency years 2 to 5 (R2 to R5), in our orthopaedic surgery residency programme. No identifiers were collected to ensure protection of the privacy of survey respondents. The choice of survey as the tool was to maximise response rates without compromising on data collection through comprehensive survey questions. There were two sets of questions evaluating the residents’ perception of mentoring by faculty and near-peer mentoring. The near-peer mentoring questions differed depending on whether the resident was a senior or junior resident and whether the resident possessed a mentee or mentor respectively. The mentoring by faculty questions varied depending on the presence of a mentor. The survey questions presented to the respondent were modified real-time based on their initial answers to the previous questions, hence eliminating questions which were not relevant.

These survey questions were adapted from a census survey conducted by the American Academy of Orthopaedic Surgeons (Flint et al., 2009) on residents with regard to their experience in, and opinion of mentorship programmes and the prevalence of such programmes.

The questions administered were largely multiple-choice questions and scaled-response questions with a few open-ended questions. The multiple-choice and scaled-response questions covered the characteristics and perception of the mentoring environment (including how beneficial and crucial they found the mentoring programme, their satisfaction with the programme, their ideal mentorship framework, etc.), the perception of the value of mentoring (for instance to what extent they felt it supported their educational experience, aided with networking and making career decisions) and the characteristics of an ideal mentor. For the scaled response questions, respondents were asked to rate the importance of and their satisfaction with the different facets of their mentoring experience or environment on a scale ranging from 1 to 5.

The open-ended section of the survey allowed residents to air what they had achieved or hoped to achieve through the mentorship programme as well as general comments about the programme and suggestions for improvement.

Standard institutional review board procedures were followed and ethics board approval was obtained. Data analysis was performed using SPSS.

III. RESULTS

The survey was administered to a total of 33 residents across the residency batches from R2 to R5 in the residency year, with a response rate of 100%. Of the respondents, 19 were junior residents (8 R2 and 11 R3) and 14 were senior residents (6 R4 and 8 R5).

Of the junior residents, 68.4% (13/19) of them had senior residents while 78.6% (11/14) of the surveyed senior residents had junior resident mentees, with two of the senior residents having two junior resident mentees. 84.8% (28/33) of surveyed residents had faculty mentors (further details in Table 1).

Table 1. Breakdown of residents with faculty and senior resident (SR) mentors/mentees

Of the junior residents with a senior resident mentor, 53.8% (7/13) of them chose their own mentors while the rest had their mentors assigned. Of the residents with a faculty mentor, 60.7% (17/28) of them chose their own mentors. 69.2% (9/13) of junior residents met up with their senior resident mentors at least twice a year while 82.1% (23/28) of residents met up with their faculty mentors at least half-yearly or more frequently.

It was found that junior residents viewed senior resident mentors as being moderately beneficial and crucial to their training, with average scores of 3.92 on a scale of 1 to 5 (Figure 1). Of note, it was found that residents with senior resident mentors viewed near-peer mentorship as being more crucial and beneficial compared to their counterparts without senior resident mentors. Similar results were also echoed by the residents regarding their faculty mentors, with average scores of 3.89 and 4.14 for how crucial and beneficial faculty mentors were to residents with mentors and average scores of 3.6 and 4.0 for residents without mentors.

Figure 1. Chart showing how crucial and beneficial residents viewed senior resident and faculty mentors

In terms of satisfaction levels with the mentoring programme, residents with mentors (senior resident and faculty mentors) were also more satisfied with the mentoring programme than their peers without mentors, with average scores of 4.31 and 4.33 for residents with senior and faculty mentors respectively compared to average scores of 3.75 and 4.00 for residents without senior and faculty mentors.

A further subgroup analysis yielded that respondents with a self-selected mentor from both the senior resident and faculty mentor groups had higher scores for satisfaction levels (4.28 and 4.41 respectively) compared to those who had assigned mentors (4.16 and 4.20). Those with self-selected mentors in the faculty mentor group also felt that their mentor aided them more in supporting their educational experience and in making career decisions.

84.6% (11/13) of junior residents who had a senior resident mentor felt that their senior resident mentor was able to provide them with advice about career, employment, or difficult cases in the future while 89.3% (25/28) of residents with faculty mentors felt the same about their faculty mentors.

In terms of desired characteristics in a senior resident mentor, approachability, willingness to share and experience were the top three most desired characteristics (Figure 2A). Similar results were echoed in the results for faculty mentors, with ability to give feedback also highly valued (Figure 2B).

66.7% (22/33) of all residents felt that near-peer senior resident mentoring should be required in the resident programme but only 30.3% (10/33) thought that it should be formalised. Some of those who felt that near-peer mentoring should not be required expressed that they would like it to be up to the individual resident and that residents who are in need would approach senior residents directly of their own accord. 78.7% (26/33) of residents surveyed were of the opinion that faculty mentorship by faculty was required. Those who felt that faculty mentorship should not be required offered reasons including the hectic schedule of consultants and the unpredictable flow of faculty members into the private practice which would make it difficult for residents to maintain the same mentor throughout residency training.

Figure 2A. Chart depicting the desired characteristics scores in senior resident faculty mentors, ranked from most to least desired

Figure 2B. Chart depicting the desired characteristics scores in faculty mentors, ranked from most to least desired

IV. DISCUSSION

Traditionally, the art of medicine and especially surgery has been passed down through an apprenticeship model (Gofton & Regehr, 2006). Multiple studies have shown that various aspects such as career paths, research output as well as personal development can be influenced greatly by mentorship (Calligaro, Dougherty, Sidawy, & Cronenwett, 2004; Gedde, Budenz, Haft, Lee, & Quigley, 2007). In fact, close to 80% of paediatric residents polled in a survey gave feedback that having a mentor was either very useful or crucial to survival during their residency years (Curtis, Adam, & Shelov, 1995). Over 20 years later, it appears that little has changed, with similar results being echoed in our study, with high scores reported for how crucial and beneficial faculty and senior resident mentorship was viewed as being by residents.

There are myriad challenges which one will face when mentoring orthopaedic residents. In comparison to teaching medical students, teaching an orthopaedic resident is often longer, more intense and calls for greater responsibility for teaching while compared to fellowship training (post-residency for subspecialty training); the mentor is teaching a much less knowledgeable and skilled protégé (Levine, Braman, Gelberman, & Black, 2013). There is hence an enhanced requirement for timely provision of feedback and assessment, and for nurturing the resident through his or her years of training. As Hill and Boone (2002) mentioned in their paper, mentoring aims “to provide a young aspiring professional with a tangible and immediate role model”. (p. 73)

For successful mentoring to occur, it requires reciprocity and has to be a “collaborative learning relationship” (Bland, Taylor, Shollen, Weber-Main, & Mulcahy, 2009). Levine et al. (2013) proposed certain obligations for a mentee to fulfil to maintain the mentor-mentee relationship and enhance the likelihood of success, including demanding and accepting feedback, seeking opportunities for self-improvement, following through, being prepared, requiring accountability, and more.

For the mentors, some traits which have been viewed as being important by surgical trainees include enthusiasm for the practice of surgery, commitment to excellence, good surgical technique, good leadership qualities, integrity and professionalism (Healy, Glynn, Malone, Cantillon, & Kerin, 2012). Certain more desirable traits of mentors which we have noted in our study include approachability, willingness to share and impart knowledge, experience as well as ability to give feedback. A mentor who has these traits will be better equipped to pass on his knowledge and skills to his mentee and hence provide a more well-rounded learning environment for the resident.

Nevertheless, we understand that mentoring is no mean feat and hence, it is recommended that both faculty members and senior residents take it upon themselves to train specific skill sets and develop desirable traits for mentoring which will enable them to successfully take on this mantle of responsibility. Mentoring workshops have been shown to improve mentoring competency and thereby improve outcomes for optimally mentored mentees (Lau et al., 2016; Gandhi & Johnson, 2016).

Our study also found that residents with self-selected mentors had higher satisfaction with the mentoring programme and felt that these mentors were able to support them more in their educational experience and making career decisions, a finding echoed in other studies (Yamada, Slanetz, & Boiselle, 2014). We postulate that this is because residents who selected their own mentors would be able to choose mentors who had desirable traits and whom they saw as ideal role models or were like-minded in their approach. Mentees have been shown to select role models—partners with whom they enjoy working and often report a mutual attraction or chemistry that sparks the development of the relationship (Kram, 1983).

As previously alluded to, mentoring by faculty is practised in many institutions and residency programmes but near-peer mentoring is often overlooked. The original intent of having a tiered mentorship framework in our programme was to ensure a holistic approach to mentoring of residents. It was envisioned that the senior resident mentor would be able to help ease the junior resident into the training programme and deal with immediate junior-level issues as well as assist in intermediate examination preparation while the senior faculty mentors would be able to provide a more overarching view as well as guidance and career advice. In the literature, it was found that mentees were generally less intimidated by near-peer mentors, felt they could relate better to them, and the near-peers understood them better as they had recently gone through situations they (the mentees) were now facing (Akinla, Hagan, & Atiomo, 2018). It was also shown that near-peer mentoring was a way of promoting professional and personal development, aid transition and maintain well-being of mentees (Akinla et al., 2018).

Residents in our programme were generally of the opinion that tiered mentorship was necessary but that near-peer mentoring should be informal, without any stipulated frequency of meetings, documentation of meetings and forms to fill up unlike mentoring by faculty. This is likely because senior residents are usually just a few years senior to the junior resident and both the mentors and mentees prefer a more informal environment and style of mentoring. This also enables junior residents to be more at ease to freely discuss any issues with the senior resident mentor. Furthermore, both the senior and junior residents have far more opportunities for interaction compared to the faculty mentor and the resident.

In the survey administered, aside from the previously mentioned results, we have also found that near-peer mentoring has positive effects on both the mentor as well as the mentee, with mentees commenting that senior resident mentors could provide pertinent directions as they started junior residency. Similarly, senior resident mentors felt that senior resident mentorship gave them an avenue to provide guidance and encouragement to their juniors, to allow them to benefit from past mistakes and to learn more efficiently. Furthermore, a tiered mentoring system contributes to a positive learning environment, creating a culture of camaraderie amongst seniors and juniors, resulting in a more closely-knit professional community.

Our residency programme encompasses both senior resident and faculty mentoring with the details on its implementation in our programme covered above. Moreover, it is the first to look at tiered mentoring and its impact on residents in orthopaedic surgery, with a 100% response rate among residents in our programme.

However, we acknowledge that this study does have some limitations. The first is that of the small sample size, which contributed to many of the variables being studied not having a statistically significant difference. Though we are limited by the small sample size due to the number of residents in the programme, we were able to achieve a high response rate as well as sub-divide the residents based on other variables including presence of a mentor and method of selection of mentor to allow for comparison between different groups. Other limitations include the fact that this study was only carried out in one institution which practised said model of mentorship. In addition, the mentorship programme had only been in place for two years when the survey was administered and its effects may not have been that far-reaching and apparent given the relatively short period of implementation, though this could form the basis for a longitudinal study in the future at further time points.

Future studies should be undertaken in the future to explore this field further. Studies involving various institutions which practice this model could be considered. Another area of study could be to look at objective measures like the Orthopaedic In-Training Examination and clinical rotation scores, exit examination success percentages and more. This would allow us to find out the tangible measurable effects that mentoring has on residents.

V. CONCLUSION

In conclusion, our study has found that mentoring by faculty and near-peer mentoring programmes are generally viewed as being beneficial and crucial to residency training and we believe that programmes should consider instituting tiered mentorship to reap the benefits.

We propose that an ideal mentoring environment should be 1) one that is tiered (encompassing both mentoring by faculty and near-peer mentoring), 2) a system that allows residents to choose their mentors, and 3) have near-peer mentoring as a required part of residency but on a more informal basis.

Notes on Contributors

Dr Chua Min Jia is a senior resident in orthopaedic surgery at the National Healthcare Group. He was involved in devising of the survey questionnaire, data collection and analysis and was the primary author of this manuscript.

Dr Foo Gen Lin is an associate consultant in the Department of Orthopaedic Surgery at the Woodlands Health Campus in Singapore. He was involved in devising of the survey questionnaire and editing of the manuscript.

Dr Ernest Kwek Beng Kee is a senior consultant and Chief of the Department of Orthopaedic Surgery at the Woodlands Health Campus in Singapore. He was involved in devising of the survey questionnaire, editing and approval of the final manuscript as well as overall supervision of the project.

Ethical Approval

This study was exempted by the National Healthcare Group Domain Specific Review Boards (DSRB).

Acknowledgement

The authors wish to thank all the orthopaedic surgery residents in our programme who participated in this study.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Interest

There is no conflict of interest to declare.

References

Akinla, O., Hagan, P., & Atiomo, W. (2018). A systematic review of the literature describing the outcomes of near-peer mentoring programs for first year medical students. BMC Medical Education, 18(1), 98.

Bland, C. J., Taylor, A. L., Shollen, S. L., Weber-Main, A. M., & Mulcahy, P. A. (2009). Faculty success through mentoring: A guide for mentors, mentees, and leaders. New York, NY: Rowman & Littlefield Education.

Buddeberg-Fischer, B., & Herta, K. D. (2006). Formal mentoring programmes for medical students and doctors – A review of the Medline literature. Medical Teacher, 28(3), 248-257.

Calligaro, K. D., Dougherty, M. J., Sidawy, A. N., & Cronenwett, J. L. (2004). Choice of vascular surgery as a specialty: Survey of vascular surgery residents, general surgery chief residents, and medical students at hospitals with vascular surgery training programs. Journal of Vascular Surgery, 40(5), 978-984.

Curtis, J. A., Adam, H., & Shelov, S. P. (1995). A formal mentoring program in a pediatric residency. Academic Medicine, 70(5), 453-454.

Flint, J. H., Jahangir, A. A., Browner, B. D., & Mehta, S. (2009). The value of mentorship in orthopaedic surgery resident education: The residents’ perspective. Journal of Bone and Joint Surgery America, 91(4), 1017-1022.

Gandhi, M., & Johnson, M. (2016). Creating more effective mentors: Mentoring the mentor. AIDS and Behavior, 20(Suppl 2), 294-303.

Gedde, S. J., Budenz, D. L., Haft, P., Lee, Y., & Quigley, H. A. (2007). Factors affecting the decision to pursue glaucoma fellowship training. Journal of Glaucoma, 16(1), 81-87.

Gofton, W., & Regehr, G. (2006). Factors in optimizing the learning environment for surgical training. Clinical Orthopaedics and Related Research, 449, 100-107.

Healy, N. A., Glynn, R. W., Malone, C., Cantillon, P., & Kerin, M. J. (2012). Surgical mentors and role models: Prevalence, importance and associated traits. Journal of Surgical Education, 69(5), 633-637.

Hill, J. A., & Boone, S. (2002). Personal perception on mentoring. Clinical Orthopaedics and Related Research, 396, 73-75.

Kashiwagi, D. T., Varkey, P., & Cook, D. A. (2013). Mentoring programs for physicians in academic medicine: A systematic review. Academic Medicine, 88(7), 1029-1037.

Kram, K. E. (1983). Phases of the mentor relationship. Academy of Management Journal, 26(4), 608-625.

Lau, C., Ford, J., Van Lieshout, R. J., Saperson, K., McConnell, M., & McCabe, R. (2016). Developing mentoring competency: Does a one session training workshop have impact? Academic Psychiatry, 40(3), 429-433.

Levine, W. N., Braman, J. P., Gelberman, R. H., & Black, K. P. (2013). Mentorship in orthopaedic surgery-road map to success for the mentor and the mentee: AOA critical issues. Journal of Bone and Joint Surgery America, 95(9), e59.

Ramanan, R. A., Taylor, W. C., Davis, R. B., & Phillips, R. S. (2006). Mentoring matters. Mentoring and career preparation in internal medicine residency training. Journal of General Internal Medicine, 21(4), 340-345.

Sambunjak, D., Straus, S. E., & Marusic, A. (2006). Mentoring in academic medicine: A systematic review. Journal of the American Medical Association, 296(9), 1103-1115.

Yamada, K., Slanetz, P. J., & Boiselle, P. M. (2014). Perceived benefits of a radiology resident mentoring program: Comparison of residents with self-selected vs assigned mentors. Canadian Association of Radiologists Journal, 65(2), 186-191.

*Chua Min Jia
Ministry of Health Holdings,
1 Maritime Square, Singapore 099253
Tel: +65 6357 7713
E-mail: minjia.chua@mohh.com.sg