Submitted: 16 October 2019
Accepted: 11 February 2020
Published online: 1 September, TAPS 2020, 5(3), 62-70
https://doi.org/10.29060/TAPS.2020-5-3/OA2204

Benny Kai Guo Loo¹, Koh Cheng Thoon², Jessica Hui Yin Tan¹, Karen Donceras Nadua² & Cristelle Chu-Tian Chow¹

¹General Paediatric Service, KK Women’s and Children’s Hospital, Singapore; ²Infectious Diseases Service, KK Women’s and Children’s Hospital, Singapore

Abstract

Practice Highlights

• RAT programme with structured, dedicated teaching platforms can improve residents’ teaching competencies.
• Assigning tutor responsibilities to residents can increase their level of enthusiasm and perception of self-importance as a tutor.
• Busy work commitment, congested lesson schedules and duty-hour regulations are barriers in RAT programmes.
• Logistic support is effective in overcoming common barriers in RAT programmes.

I. INTRODUCTION

Residents play a vital role in the education of medical students. They spend up to 25% of their time teaching and research has shown that they can improve medical students’ knowledge and examination scores (McKean, & Palmer, 2015). With the shift of postgraduate medical training in Singapore to the residency system under Accreditation Council on Graduate Medical Education-International (ACGME-I) in 2011, teaching is recognised as an important competency for residents (Accreditation Council for Graduate Medical Education, 2017).

Studies have demonstrated the benefits of residents-as-teachers (RAT) programmes for both residents and medical students (Hill, Yu, Barrow, & Hattie, 2009). A controlled trial of RAT programme involving 24 obstetrics and gynaecology residents showed that their teaching skills have improved via objective structured teaching examination (Gaba, Blatt, Macri, & Greenberg, 2007). Subjectively, residents feel that teaching improves their clinical knowledge (Post, Quattlebaum, & Benich III, 2009). A successful RAT programme also enhances the students’ perception of the resident as a physician (Wamsley, Julian, & Wipf, 2004), as well as higher overall satisfaction with the clinical posting (Huynh, Savitski, Kirven, Godwin, & Gil, 2011). Furthermore, effective resident teachers can influence the students’ future career choices (Musunuru, Lewis, Rikkers, & Chen, 2007).

Although residents acknowledge that teaching is part of their duty and have the desire to teach, there are also many barriers encountered. A common problem is the heavy burden of clinical duties and lack of uninterrupted time (Wamsley et al., 2004). The limitation of time was further worsened by the introduction of duty-hour regulations (Brasher, Chowdhry, Hauge, & Prinz, 2005). The residents understandably prioritise clinical work and are exhausted after completion of clinical duties, leaving little time or energy to focus on education. Other challenges include a lack of confidence and insufficient training as an educator (Yedidia, Schwartz, Hirschkorn, & Lipkin, 1995). A survey of paediatric residents indicated that prior training in teaching can benefit the residents in teaching students (Busari, Scherpbier, Van Der Vleuten, & Essed, 2000).

A. Conceptual Framework

As learning involves knowledge organisation through the continuous addition and modification of concepts and relations over time, it is understandable that experts would have a more complex knowledge structure compared to novices (Meller, M. Chen, R. Chen, & Haeseler, 2013). The difference in knowledge structure between the experts and novices would be significant. Intermediate learners, such as paediatric residents, in this case paediatric residents, with a knowledge structure more similar to novices, would, therefore, be able to better appreciate and address the cognitive problems encountered by the medical students. Paediatric residents would be in an optimal position to minimise the distance between what the novice already knows and what needs to be learned, also referred to as the “zone of proximal development” (Ten Cate, Snell, Mann, & Vermunt, 2004).

Engaging residents as teachers utilise the principles of near-peer teaching. Social and cognitive congruence, demonstrated by Schmidt and Moust (1995), as well as Lockspeiser, O’Sullivan, Teherani, and Muller (2008), supports the near-peer teaching relationship. Social congruence enables residents to communicate with students in an informal, empathic way which in turn encourages student engagement and drives learning (Schmidt & Moust, 1995). Cognitive congruence, whereby residents have a better appreciation of students’ deficits in knowledge, enables residents to clarify problems at a level appropriate and relevant to students (Lockspeiser et al., 2008).

Utilising these principles, we have developed the RAT programme, Paediatric Residents As Teaching Advocates (PRATA), for our residents to engage in formal, structured teaching duties. The authors believe that residents can teach more effectively with a structured teaching platform, and medical students will experience fewer barriers with logistic support from RAT programme.

II. METHODS

A controlled, prospective, pre-post study was carried out at KK Women’s and Children’s Hospital (KKH), Department of Paediatric Medicine, from June 2014 to April 2016. As this study was categorised as an education quality improvement, Singhealth Centralised Institutional Review Board (CIRB) indicated that formal IRB review was not required. Implied informed consent was obtained from participants during a briefing prior to the data collection.

A. Study Setting and Participants

KKH is the largest academic paediatric medical centre in Singapore, with a capacity of more than 800 inpatient beds. The paediatric residency programme also trains more than half of the paediatric residents nationwide.

Third-year medical students from the NUS Yong Loo Lin School of Medicine are attached to KKH for 1-month clinical posting as part of their paediatric training. Every batch of students consists of 6 tutorial groups and each group has 5 to 6 students. Each tutorial group is assigned to a ward team for the entire posting. The students are also required to evaluate and provide feedback on their tutors at the end of the posting.

Four to five paediatric consultants are assigned to each tutorial group as their dedicated tutors. These consultants are paediatric specialists who have completed a recognised training programme and are accredited by the Specialist Accreditation board. Every consultant has more than 3 years’ experience of teaching medical students.

All paediatric residents in their first 3 years of training (n=49) had attended workshops on “Effective Bedside Teaching using the Five-Minute Preceptor” (half-day programme with lectures and practice sessions on microteaching skills for effective clinical teaching) and “Giving Effective Feedback” (half-day programme with lectures and role-play sessions on a 4-step model for constructive feedback). These residents were invited to take part in the PRATA programme through email and participation was voluntary. Thirty-three paediatric residents (67%) participated in the PRATA programme.

B. Study Implementation

The study coordinators randomly assigned 3 medical student tutorial groups to be intervention groups and the remaining 3 groups as controls at the beginning of each posting. A total of 51 intervention and 51 control groups were assigned over 17 batches of students. The intervention included assigning 1 resident to the tutor group with a structured teaching schedule to assist with logistics coordination (Figure 1). The resident was to conduct 2 bedside teaching sessions, focusing on history taking and physical examination techniques. Two administrators from the education office were involved in the scheduling of the teaching roster according to the residents’ ward roster and work commitments, and in line with ACGME-I duty-hour restrictions. This ensured that the arrangements were specifically catered for the respective residents as they were often in different areas of work and thus have varying work commitments.

Note: A total of 17 batches of students with 51 intervention and 51 control groups participated in the programme.

Figure 1. PRATA programme protocol diagram (per batch)

For comparison, the existing teaching interactions between residents and medical students in the wards were evaluated in the control groups. As the students were attached to the same ward team for the posting, the residents in the ward team could teach the medical students. Often, these teaching sessions were unplanned, brief and limited by the burden of clinical work.

Apart from the additional resident tutor in the intervention groups, both groups of students received the same standard of medical education, comprising of the same series of planned lectures, the number of bedside tutorials by the consultants, and the duration of ward and clinic attachments.

C. Survey Forms

Paper surveys were given to all students at the beginning (9-question survey) and the end of the posting (18-question survey). These surveys were anonymised and collected by the study coordinators. The survey design was based on 2 studies by Copeland and Hewson (2000), and Hill et al. (2012). These rating instruments were chosen as they were widely used in assessing RAT programmes, applicable in a variety of clinical setting and frequently cited to date. To suit the local context, relevant questions were extracted by the study coordinators based on residents’ feedback and modified accordingly. A 5-point Likert scale was used for the questions, ranging from strongly disagree to strongly agree. A free-text box was included for any comments on both surveys.

Effectiveness of Resident Teaching	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
My medical knowledge and/or clinical skills have improved significantly during the clerkship after being taught by residents
We received adequate teaching from residents during the clerkship
The resident established a good learning environment
The resident stimulated me to learn independently
The resident offered regular and constructive feedback (both positive and negative) in a timely manner and in an appropriate setting
The resident clearly specified what we were expected to know and do for the tutorial
The resident was able to adjust the teaching according to my needs (experience, competence, interest)
The resident asked questions that promoted learning (e.g. clarification, reflective questions)
The resident gave clear explanations or reasons for his/her opinions, advice and actions etc.
The resident was able to effectively coach me on my clinical skills
The resident taught or demonstrated effective patient and/or family communication skills
The residents are enthusiastic about teaching
The resident made teaching relevant to patient care
Overall, the resident was an effective teacher

Barriers to Resident Teaching	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Ward work was too busy and residents had no time for teaching
There were too many lectures and tutorials outside of the ward to allow for resident teaching

Personal Impact of Resident Teaching	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
The resident tutor was a positive role model for me
After being taught and interacting with my resident teacher, it would make me more likely consider/select Paediatrics as my choice for future residency training

Any other comments:

The pre-programme survey (Appendix) asked the students about their expectations of residents as teachers, as well as the perceived barriers to receiving resident teachings. The post-programme survey (Table 1) focused on the students’ evaluation of residents’ teaching competencies, the impact of resident teachings and the barriers encountered. For the post-programme survey, the intervention groups evaluated the resident from their tutor group, whereas the control groups evaluated the resident from the ward team.

D. Statistical Analysis

Data were expressed as mean scores with standard deviations. For questions about residents’ teaching competencies, post-programme differences were evaluated between groups using the Mann-Whitney U test. For questions on barriers to receiving resident teachings, pre and post-programme differences were evaluated within and between groups using the Wilcoxon signed-rank test and the Mann-Whitney U test respectively. A p-value of less than 0.05 was taken to be significant for all tests. Analyses were conducted using SPSS version 19.

III. RESULTS

A. Participant Demographics

A total of 502 medical students from 17 batches participated in the PRATA programme over 23 months. Overall, there was 100% pre-programme and 81% (410 students) post-programme participation. The post-programme participation was 74% in the control group and 88% in the intervention group. The baseline demographics were comparable between the groups.

B. Resident Teaching Competencies

A higher score indicated better teaching competency. The intervention group students gave significantly better scores across all aspects for the residents’ teaching competencies as compared to the control groups (Table 2). The biggest difference was noted in the question of the enthusiasm shown by the residents. The intervention groups scored 4.34 and the control groups scored 3.92. The next greatest difference was from the question of residents giving regular and constructive feedback. The scores for the intervention and control groups were 4.23 and 3.83 respectively. The overall assessment of residents as effective teachers showed a similar trend as the intervention groups scored 4.27 and the control groups scored lower at 3.89. The differences in these competencies were all statistically significant.

Note: ^*Lower mean score indicates less severe barrier.
^#Wilcoxon signed-rank test used for within-group comparisons.
^{^}Mann-Whitney U test used for between-group comparisons.
Table 3. Results of survey questions relating to barriers to receiving resident teachings

IV. DISCUSSION

RAT programmes have demonstrated improvement in residents’ teaching competencies and contribute significantly to the education of medical students (Hill et al., 2009). Although barriers to resident teachings such as excessive workload and duty hour regulations have been identified (Wamsley et al., 2004), there is limited literature on overcoming these obstacles in RAT programmes.

The main aims of the PRATA programme are to provide paediatric residents with a structured teaching platform to improve their teaching competencies and logistic support to overcome common barriers in RAT programmes. The residents in the PRATA programme had the added elements of a dedicated environment for teaching and assuming the role and responsibility as a teaching faculty. The education office also provided these residents with logistic support to reduce scheduling conflicts and duty-hour violations.

This study revealed that the PRATA programme improved the students’ perceptions of the residents’ teaching competencies. This is consistent that RAT programmes enhance resident teaching skills (Zackoff, Jerardi, Unaka, Sucharew, & Klein, 2015). The residents in the intervention groups achieved higher scores in all teaching competency domains, and this included innate characteristics (displaying enthusiasm), techniques (giving constructive feedback) and bedside skills (demonstrating effective communication). A higher score indicated better proficiency in that domain. Although both groups of residents were equipped with the same set of teaching skills, those in the PRATA programme were provided with a structured teaching platform as compared to the unpredictable and potentially chaotic ward setting in the control groups. Furthermore, assigning the residents responsibility as the group tutors possibly increased their level of enthusiasm and perception of self-importance as a tutor. Our study demonstrated that residents in RAT programmes that provide structure to apply their teaching skills can be perceived as better educators.

The results also showed that students from both student groups were very worried about the barriers to receiving resident teachings before the programme. They were more concerned about the residents’ busy work commitments as compared to their congested lesson schedules. Our programme was able to reduce the impact of these barriers as both scores improved significantly in the intervention groups. In contrast, the control groups continued to perceive after the posting that their lesson schedules did not permit residents to teach. Overall, the students from the intervention groups seemed to have a more positive learning experience as there were 20% more students who considered working in Paediatric Medicine in the future. The PRATA programme highlighted that logistic support was an important factor to decrease the students’ concerns of these barriers, which could impede them seeking learning opportunities from the residents.

We have identified some limitations of our study. Firstly, the residents participated in the programme voluntarily, therefore we could have recruited residents who were more passionate or more experienced in teaching. Therefore, we aim to recruit all residents and stratify them by their level of training for the subsequent studies. Secondly, there was no standardisation of the teaching topics. The residents in the intervention groups could have taught in the areas they were more confident in, whereas the residents in the control group could only teach about the patients in the ward. For future studies, we can assign the same teaching topics when evaluating the residents’ teaching competencies. Lastly, the improvement of scores in the intervention groups, although statistically significant, can be complex to interpret in the practical setting. A qualitative study on these aspects can give more insight, such as the specific traits or techniques that residents in the programme displayed, or the practical burden of the barrier on the students’ learning.

V. CONCLUSION

This study demonstrated that providing a structured teaching platform could enhance residents’ teaching competencies. This was an important factor as existing resident teachings commonly occur while performing clinical work and in the chaotic ward setting. The study also showed that logistic support could help overcome common barriers in RAT programmes, such as busy work commitments. We believe this combination is important to include in future RAT programmes.

Notes on Contributors

Dr Benny Kai Guo Loo is an Associate Consultant in General Paediatric Service at KK Women’s and Children’s hospital. He has a keen interest in medical education and is currently a Co-ordinator for Yong Loo Lin School of Medicine, NUS and Physician Faculty for Singhealth Paediatric Residency Programme.

A/Prof Koh Cheng Thoon is the Head and Senior Consultant of Infectious Diseases Service at KK Women’s and Children’s hospital. He is also the Academic Vice-Chair for Education in Singhealth Duke-NUS Paediatric Academic Clinical Programme and he was awarded the Wong Hock Boon Society-SMA Charity Fund Outstanding Mentor Award in 2016.

Dr Jessica Hui Yin Tan is a Senior Resident in General Paediatrics Service at KK Women’s and Children’s hospital. She is also a clinical lecturer at the Yong Loo Lin School of Medicine, NUS.

Dr Karen Donceras Nadua is an Associate Consultant in Infectious Diseases Service at KK Women’s and Children’s hospital. She is also a clinical teacher at the Lee Kong Chian School of Medicine.

Dr Cristelle Chu-Tian Chow is a Consultant in General Paediatrics Service at KK Women’s and Children’s hospital. She obtained her Master of Health Professions Education from Maastricht University and is currently the Associate Programme Director for Singhealth Paediatric Residency Programme.

Ethical Approval

The study was categorised as an education quality improvement hence formal Singhealth Centralised Institutional Board review is not required.

Acknowledgements

The authors would like to thank Paediatric Graduate Medical Education and Academic Clinical Programme secretariat, faculty and residents who have helped to make PRATA programme a success. We would also like to thank Prof Sandy Cook, Senior Associate Dean in Duke-NUS, for her guidance in the writing of this manuscript.

Funding

The authors report no funding source for this study.

Declaration of Interest

The authors declare no conflict of interest.

References

Accreditation Council for Graduate Medical Education (2017). Common program requirements. Retrieved from https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf.

Brasher, A. E., Chowdhry, S., Hauge, L. S., & Prinz, R. A. (2005). Medical students’ perceptions of resident teaching: Have duty hours regulations had an impact? Annals of Surgery, 242(4), 548.

Busari, J. O., Scherpbier, A. J. J. A., Van Der Vleuten, C. P. M., & Essed, G. E. (2000). Residents’ perception of their role in teaching undergraduate students in the clinical setting. Medical Teacher, 22(4), 348-353.

Copeland, H. L., & Hewson, M. G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2), 161-166.

Gaba, N. D., Blatt, B., Macri, C. J., & Greenberg, L. (2007). Improving teaching skills in obstetrics and gynecology residents: Evaluation of a residents-as-teachers program. American Journal of Obstetrics and Gynecology, 196(1), 87-e1.

Hill, A. G., Srinivasa, S., Hawken, S. J., Barrow, M., Farrell, S. E., Hattie, J., & Yu, T. C. (2012). Impact of a resident-as-teacher workshop on teaching behavior of interns and learning outcomes of medical students. Journal of Graduate Medical Education, 4(1), 34-41.

Hill, A. G., Yu, T. C., Barrow, M., & Hattie, J. (2009). A systematic review of resident‐as‐teacher programmes. Medical Education, 43(12), 1129-1140.

Huynh, A., Savitski, J., Kirven, M., Godwin, J., & Gil, K. M. (2011). Effect of medical students’ experiences with residents as teachers on clerkship assessment. Journal of Graduate Medical Education, 3(3), 345-349.

Lockspeiser, T. M., O’Sullivan, P., Teherani, A., & Muller, J. (2008). Understanding the experience of being taught by peers: The value of social and cognitive congruence. Advances in Health Sciences Education, 13(3), 361-372.

McKean, A. J. S., & Palmer, B. A. (2015). Psychiatry resident-led tutorials increase medical student knowledge and improve national board of medical examiners shelf exam scores. Academic Psychiatry, 39(3), 309-311.

Meller, S. M., Chen, M., Chen, R., & Haeseler, F. D. (2013). Near-peer teaching in a required third-year clerkship. The Yale Journal of Biology and Medicine, 86(4), 583.

Musunuru, S., Lewis, B., Rikkers, L. F., & Chen, H. (2007). Effective surgical residents strongly influence medical students to pursue surgical careers. Journal of the American College of Surgeons, 204(1), 164-167.

Post, R. E., Quattlebaum, R. G., & Benich III, J. J. (2009). Residents-as-teachers curricula: A critical review. Academic Medicine, 84(3), 374-380.

Schmidt, H. G., & Moust, J. H. (1995). What Makes a Tutor Effective? A Structural Equations Modelling Approach to Learning in Problem-Based Curricula. Academic Medicine, 70(8), 708-714

Ten Cate, O., Snell, L., Mann, K., & Vermunt, J. (2004). Orienting teaching toward the learning process. Academic Medicine, 79(3), 219-228.

Wamsley, M. A., Julian, K. A., & Wipf, J. E. (2004). A literature review of “resident-as-teacher” curricula. Journal of General Internal Medicine, 19(5), 574-581.

Yedidia, M. J., Schwartz, M. D., Hirschkorn, C., & Lipkin, M. (1995). Learners as teachers. Journal of General Internal Medicine, 10(11), 615-623.

Zackoff, M., Jerardi, K., Unaka, N., Sucharew, H., & Klein, M. (2015). An observed structured teaching evaluation demonstrates the impact of a resident-as-teacher curriculum on teaching competency. Hospital Paediatrics, 5(6), 342-347.

*Benny K. G. Loo
Division of Paediatric Medicine
KK Women’s and Children’s Hospital
100 Bukit Timah Road, Singapore 229899
Tel: +65 62255554
Email: benny.loo.k.g@singhealth.com.sg

Submitted: 7 July 2019
Accepted: 30 January 2020
Published online: 1 September, TAPS 2020, 5(3), 54-61
https://doi.org/10.29060/TAPS.2020-5-3/OA2170

Kieng Wee Loh¹, Jerome Ingmar Rotgans², Kevin Tan³, Nigel Choon Kiat Tan³

¹National Healthcare Group, Ministry of Health Holdings, Singapore; ²Office of Medical Education, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore; ³Office of Neurological Education, Department of Neurology, National Neuroscience Institute, Singapore

Abstract

Practice Highlights

• Clinical reasoning is a combination of two concurrent processes–pattern recognition in familiar circumstances (illness scripts); and deliberate analysis in unfamiliar scenarios (hypothetico-deductive approach).
• Validated tools exist to assess aspects of clinical reasoning–Script Concordance Tests (SCTs) for illness scripts; and Extended Matching Questions (EMQs) for hypothetico-deductive reasoning.
• Doctors who (a) were educated locally; and (b) worked locally for a longer period, tend to reason more similarly to local expert clinicians in the area of neurological localization.
• Development of clinical reasoning in neurology appears to be specific to a given clinical context
• To optimize the development of clinical reasoning in neurology, internal medicine residency programmes could consider maximizing trainees’ exposure to the local medical context before rotating them to a neurology posting.

I. INTRODUCTION

Clinical reasoning is the cognitive process of integrating and weighing clinical information together with past experiences to evaluate diagnostic and management dilemmas (Monteiro & Norman, 2013). Together with an appropriate knowledge base, this is central to clinical competence (Elstein, Shulman, & Sprafka, 1990; Groen & Patel, 1985), Clinical reasoning is especially important for the skill of neurological localisation (Gelb, Gunderson, Henry, Kirshner, & Jozefowicz, 2002; Nicholl & Appleton, 2015), which involves interpreting clinical signs and symptoms to identify the site of neuroanatomical abnormalities–a crucial first step in making a neurologic diagnosis. Accurate clinical reasoning is an important core competency (Connor, Durning & Rencic, 2019), and is essential in minimising diagnostic errors (Durning, Trowbridge, & Schuwirth, 2019).

The ‘dual process’ paradigm of clinical reasoning proposes that a combination of rapid intuition and deliberate analysis is employed in clinical decision making (Elstein, 2009; Eva, 2005; Monteiro & Norman, 2013). In familiar circumstances, relevant clinical information is compared with past experiences to arrive at a diagnosis (Elstein, 2009), akin to pattern recognition. This content-specific knowledge is organised into mental networks (‘illness scripts’) for easy retrieval (Boushehri, Arabshahi, & Monajemi, 2015; Norman, Young, & Brooks, 2007). In unfamiliar situations, however, a ‘hypothetico-deductive’ approach is utilised instead, where hypotheses are formulated through conscious deliberations and later tested (Boushehri et al., 2015; Elstein, 2009; Monteiro & Norman, 2013). These reasoning processes work in parallel, but experts are more adept at switching between both approaches whilst maintaining a higher performance level in each (Boushehri et al., 2015; Eva, 2005; Monteiro & Norman, 2013).

Several studies have examined predictors of academic performance in medical undergraduates (Ferguson, James, & Madeley, 2002; Hamdy et al., 2006; Kanna, Gu, Akhuetie, & Dimitrov, 2009). Previous studies have examined sociodemographic characteristics and educational background as potential predictors, as these have practical relevance in reviewing admission criteria and teaching methods for undergraduate programmes. Female gender (Adams et al., 2008; Ferguson et al., 2002; Guerrasio, Garrity, & Aagaard, 2014; Stegers-Jager, Themmen, Cohen-Schotanus, & Steyerberg, 2015; Woolf, Haq, McManus, Higham, & Dacre, 2008), ethnic majority status (Stegers-Jager et al., 2015; Vaughan, Sanders, Crossley, O’Neill, & Wass, 2015; Woolf, Cave, Greenhalgh, & Dacre, 2008; Woolf & Haq et al., 2008; Woolf, Potts, & McManus, 2011) and older age (Kusurkar, Kruitwagen, Ten Cate, & Croiset, 2010) were found to be significant predictors; educational background (Kusurkar et al., 2010) and past academic performance (Ferguson et al., 2002; Hamdy et al., 2006; Kanna et al., 2009; Stegers-Jager et al., 2015; Woloschuk, McLaughlin, & Wright, 2010) also showed positive associations. However, academic performance does not solely reflect reasoning skill, especially in postgraduates (Woloschuk et al., 2010; Woloschuk, McLaughlin, & Wright, 2013).

The ‘dual process’ theory also identifies clinical experience as important for clinical reasoning, especially in the formation of illness scripts (Elstein, 2009; Eva, 2005; Monteiro & Norman, 2013). Yet this is seldom explored, with few studies on postgraduates, a group where clinical experience might be more relevant.

The current Singapore postgraduate training system is based on the United States’ residency system (Huggan et al., 2012). Medical graduates, whether local or overseas-trained, must first register with the Singapore Medical Council (SMC) to start practising medicine locally. They can then apply for graduate medical education programmes (‘residency’) in various sponsoring institutions to train in a speciality; residency entry can occur immediately after or several years after graduation.

In Singapore, internal medicine residents rotate between subspecialty departments (such as cardiology or neurology) in no fixed order, hence two residents rotated to neurology may differ in the amount of working experience as a resident, as a clinician practising locally, and as a doctor in general. Moreover, experiences may differ between the various sponsoring institutions, and also between disparate undergraduate medical programmes. These differences might influence clinical reasoning.

Additionally, most Singaporean male graduates defer their medical careers to complete a two-year stint with the Singapore Armed Forces (SAF), as part of their compulsory National Service. Within the SAF, medicine is rarely practised in conventional clinical settings, and the quality of clinical experience may be affected. Clinical experience may hence differ between genders.

Several instruments have been designed to assess clinical reasoning (Amini et al., 2011; Boushehri et al., 2015), but these were infrequently used in studies (Groves, O’rourke, & Alexander, 2003; Postma & White, 2015). Some studies utilised unvalidated questionnaires (Groves et al., 2003); others did not specifically assess clinical reasoning (Postma & White, 2015). Moreover, the focus of each instrument varies–Extended Matching Questions (EMQ; Beullens, Struyf, & Van Damme, 2005) on ‘hypothetico-deductive’ reasoning; Script Concordance Test (SCT; Lubarsky, Charlin, Cook, Chalk, & van der Vleuten, 2011) on illness scripts (Amini et al., 2011; Boushehri et al., 2015). As both approaches are complementary, it may thus be prudent to employ multiple instruments to better evaluate clinical reasoning as an outcome measure, especially for the important skill of neurological localisation (Gelb et al., 2002; Nicholl & Appleton, 2015).

Given the gaps in the extant literature, we thus aimed to determine predictors of postgraduate performance in clinical reasoning tests, within the context of neurological localisation.

II. METHODS

A. Subjects

Subjects comprised 162 internal medicine residents from two sponsoring institutions (National Healthcare Group and Singapore Health Services). Each resident completed a three-month neurology rotation at the National Neuroscience Institute (NNI), Singapore, from January 2014 to June 2016. Waiver of further ethical deliberation was granted by the SingHealth Centralised Institutional Review Board (CIRB) for this education program improvement project; subjects were anonymised and implied informed consent was obtained from all participants. We excluded 17 subjects who failed to complete the required assessments, leaving 145 (90%) subjects for eventual analysis.

B. Predictor Variables

We investigated three sociodemographic characteristics–gender, undergraduate medical education (UME) and graduate medical education (GME). UME denotes the location of undergraduate training institution, and was classified into local (Singapore) and overseas. GME refers to the residency programmes of the two sponsoring institutions, anonymised as ‘A’ and ‘B’.

Clinical experience was judged by three metrics–overall experience (OE), local experience (LE) and residency experience (RE; Figure 1). OE and LE were calculated as the number of months from graduation and SMC registration respectively, to the month of test attempt. RE, defined as the residency training year, was categorised as ‘Year 1’, ‘Year 2’ and ‘Year 3’.

Figure 1. Measures of clinical experience

We obtained data on gender, GME and RE from our institution records; UME and month of SMC registration were obtained from the SMC Registry of Doctors. Graduation month was derived from our institution records for local graduates and estimated for overseas graduates from the dates of their alma mater’s most recent graduation ceremony, available online.

C. Outcome Measures

We used two validated methods of assessment, Script Concordance Test (SCT; Lubarsky et al., 2011) and Extended Matching Questions (EMQ; Beullens et al., 2005), to evaluate clinical reasoning in neurological localisation. We specifically selected the SCT and EMQ tests that had previously demonstrated construct validity and reliability in our Singapore context (Tan, Tan, Kandiah, Samarasekera, & Ponnamperuma, 2014; Tan et al., 2017).

An SCT contains case scenarios with 3-5 part questions (Fournier, Demeester, & Charlin, 2008; Figure 2)–a relevant diagnostic or management option; a new clinical finding; and a five-point Likert Scale indicating the new finding’s effect on the initial option. A scoring key is derived from scores by an expert panel; subsequent test-takers are then scored for degree of concordance to the experts (Fournier et al., 2008; Wan, 2015).

Our locally-validated SCT (Tan et al., 2014) contained 14 scenarios, each with 3–5 question items, totalling 53 items; reliability and generalisability were acceptable (Cronbach α 0.75, G-coefficient 0.74). Questions and scoring keys were derived from local experts. We analysed only the neurological localisation component (7 scenarios).

Figure 2. Script Concordance Test (SCT)–Sample questions (Tan et al., 2017)

EMQs are multiple-choice questions consisting of case scenarios, each with a single answer drawn from a shared list of at least 7 options (Case & Swanson, 1993; Fenderson, Damjanov, Robeson, Veloski, & Rubin, 1997). Our locally-validated EMQ (Tan et al., 2017) contained 45 scenarios with a shared answer list of 25 options (Figure 3); reliability and generalisability were excellent (Cronbach α 0.85, G-coefficient 0.85).

Figure 3. Extended Matching Questions (EMQ)–Sample questions (Tan et al., 2014)

Subjects completed both timed closed-book tests via an online portal during the second month of their three-month neurology rotation, done as a formative assessment. Scores were expressed in percentages. Subjects had no prior exposure to the SCT or EMQ during their neurology rotation or as practising doctors, and were introduced to the test format on the day of assessment. One worked example of the SCT and EMQ was provided to the subjects before the test.

D. Statistical Analysis

Descriptive statistics were calculated to test assumptions of normality before proceeding with multivariate analysis. We used SPSS Statistics version 20, and considered p-values <0.05 as statistically significant; all tests were two-tailed.

Multivariate stepwise linear regression models were used to assess the relationship between predictor variables (gender, UME, GME, OE, LE and RE) and outcome measures (SCT and EMQ scores). Tolerance values were computed to assess multicollinearity, with values below 0.60 considered problematic (Chan, 2004). Overall model performance was assessed using Nagelkerke’s R², and effect sizes measured with Cohen’s f². Effect sizes of 0.02, 0.15 and 0.35 were considered low, medium and large respectively (Cohen, 1988).

III. RESULTS

The majority of the 145 subjects were female, local graduates and belonged to residency ‘B’ (Table 1).Mean and standard deviation of SCT and EMQ scores were 68.03 ± 8.24% and 81.84 ± 12.17% respectively. Population statistics did not reveal a need for non-parametric tests.

IV. DISCUSSION

As there is a paucity of literature about postgraduate performance in clinical reasoning, this study provides a unique opportunity to evaluate its predictors, especially clinical experience. We used validated instruments to measure clinical reasoning skills in neurological localisation, and elucidated multivariate associations between clinical reasoning, clinical experience, and sociodemographic characteristics of our subjects.

Our results suggest that local graduates tend to score better in both clinical reasoning tests. Consistent with the existing literature (Postma & White, 2015), this indicates that educational background plays an important role in the development of clinical reasoning skills. Since SCT performance reflects the degree of concordance with verdicts made by local experts (Tan et al., 2014; Tan et al., 2017), this suggests that being educated locally may promote a similar outcome of reasoning. It is also possible that local undergraduate programmes provide better training in neurological localisation, as local graduates performed better in the EMQ, an instrument where scoring is independent of local experts’ views.

Interestingly, we found no significant associations between clinical experience and SCT performance. The accumulation of context-specific experiential knowledge is crucial for developing effective illness scripts (Elstein, 2009), hence SCT scores were expected to rise with increasing clinical experience (Boushehri et al., 2015; Kazour, Richa, Zoghbi, El-Hage, & Haddad, 2017; Lubarsky, Chalk, Kazitani, Gagnon, & Charlin, 2009; Norman et al., 2007). However, heuristics also play an important role (Boushehri et al., 2015; Elstein, 2009; Norman et al., 2007), suggesting that efficiency of knowledge organisation may be independent of clinical exposure. Alternatively, the study period may be insufficient for subjects to fully develop their illness scripts.

In contrast, more experienced doctors performed better at the EMQ, validating the premise that expertise is at least partially linked to experience and acquiring a strong knowledge base (Elstein et al., 1990; Monteiro & Norman, 2013; Neufeld, Norman, Feightner, & Barrows, 1981). Interestingly, only local experience was a significant predictor, but not overall experience. This suggests that overseas experience may not significantly improve clinical reasoning skills in neurological localisation and that the acquisition of such skills is a context-specific process (Durning, Artino, Pangaro, van der Vleuten, & Schuwirth, 2011; Durning et al., 2012; McBee et al., 2015).

Our findings have two potential implications for graduate medical education in Singapore. Firstly, the design of internal medicine residency programmes. To optimise the development of clinical reasoning in neurology, programmes could maximise local experience by assigning residents with less local experience to neurology only in the final year of the three-year residency. However, as local experience may also influence clinical reasoning in other subspecialties, further research is necessary to ascertain the optimal posting configuration that maximises clinical reasoning development across all disciplines.

Secondly, context appears to influence clinical reasoning in neurological localisation. Our results suggest that training location plays a role at both undergraduate and postgraduate levels. This might be due to context specificity, which attributes performance variations to situational factors (Durning et al., 2011; Durning et al., 2012; McBee et al., 2015). Since local exposure appears to be beneficial, it implies that overseas graduates and clinicians may require more time to acclimatise or familiarise themselves with the Singapore clinical context.

Our study has several strengths. We used validated, reliable tests to specifically assess clinical reasoning skills for neurological localisation. Homogeneity of the subject cohort, along with the consistent time-frame of test attempts, also allowed us to minimise confounders such as intrinsic motivation (Ferguson et al., 2002; Kusurkar et al., 2010; Vaughan et al., 2015), instructional design (Postma & White, 2015), and duration of neurology exposure.

There were also limitations. This is a single-centre, single subspecialty study with moderate sample size, and our results may not be applicable to other aspects of neurology besides neurological localisation. Further studies are needed to validate whether the results are generalisable beyond the neurology SCT and EMQ, and to other postgraduate populations. The study design was also limited due to the nature of secondary data analysis, thus information on other potentially important variables such as ethnicity (Adams et al., 2008; Guerrasio et al., 2014; Stegers-Jager et al., 2015; Woolf & Cave et al., 2008; Woolf & Haq et al., 2008), age (Woolf et al., 2011) and previous academic performance (Ferguson et al., 2002; Hamdy et al., 2006; Kanna et al., 2009; Stegers-Jager et al., 2015; Woloschuk et al., 2010) could not be fully obtained for evaluation. There is thus a possibility that other confounding variables may influence the findings in this study. Test questions may also intrinsically favour local graduates as they were formulated by local experts, but this is less likely as globally relevant clinical scenarios were tested.

V. CONCLUSION

In conclusion, our study suggests that local clinical experience and site of undergraduate education predict postgraduate clinical reasoning skill in validated tests of neurological localization. We believe context specificity likely underpins a significant part of clinical reasoning. Our findings have practical implications on residency programme design and highlight the need to provide overseas graduates and clinicians time to adapt to the local clinical context.

Notes on Contributors

Dr Loh Kieng Wee is a Medical Officer under the Ministry of Health Holdings, Singapore.

Jerome Ingmar Rotgans is an Assistant Professor at the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.

Dr Kevin Tan is Education Director, Vice-Chair (Education) and a Senior Consultant at the Department of Neurology, National Neuroscience Institute, Singapore.

Dr Nigel Choon Kiat Tan is Deputy Group Director Education (Undergraduate), Singapore Health Services, and a Senior Consultant at the Department of Neurology, National Neuroscience Institute, Singapore.

Ethical Approval

Ethical exemption has been granted from the SingHealth Centralised Institutional Review Board A, CIRB Ref: 2020/2228.

Acknowledgements

The authors would like to acknowledge the technical assistance & support provided by the Office of Neurological Education, National Neuroscience Institute, Singapore.

Funding

The research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of Interest

Authors have no conflict of interest, including financial, institutional and other relationships that might lead to bias.

References

Adams, A., Buckingham, C. D., Lindenmeyer, A., McKinlay, J. B., Link, C., Marceau, L., & Arber, S. (2008). The influence of patient and doctor gender on diagnosing coronary heart disease. Sociology of Health and Illness, 30(1), 1–18.

Amini, M., Moghadami, M., Kojuri, J., Abbasi, H., Abadi, A. A., Molaee, N., & Charlin, B. (2011). An innovative method to assess clinical reasoning skills: Clinical reasoning tests in the second national medical science Olympiad in Iran. BMC Research Notes, 4(1), 418.

Beullens, J., Struyf, E., & Van Damme, B. (2005). Do extended matching multiple-choice questions measure clinical reasoning? Medical Education, 39(4), 410–417.

Boushehri, E., Arabshahi, K. S., & Monajemi, A. (2015). Clinical reasoning assessment through medical expertise theories: Past, present and future directions. Medical Journal of The Islamic Republic of Iran, 29(1), 222.

Case, S. M., & Swanson, D. B. (1993). Extended‐matching items: A practical alternative to free‐response questions. Teaching and Learning in Medicine, 5(2), 107–115.

Chan, Y. H. (2004). Biostatistics 201: Linear regression analysis. Singapore Medical Journal, 45(2), 55–61.

Cohen, J. (1988). Statistical Power Analysis for the Behavioural Sciences. Hillsdale, NJ: Lawrence Earlbaum Associates.

Connor, D. M., Durning, S. J., & Rencic, J. J. (2019). Clinical reasoning as a core competency. Academic Medicine. http://dx.doi.org/10.1097/ACM.0000000000003027

Durning, S., Artino, A. R., Pangaro, L., van der Vleuten, C. P., & Schuwirth, L. (2011). Context and clinical reasoning: Understanding the perspective of the expert’s voice. Medical Education, 45(9), 927–938.

Durning, S. J., Artino, A. R., Boulet, J. R., Dorrance, K., van der Vleuten, C., & Schuwirth, L. (2012). The impact of selected contextual factors on experts’ clinical reasoning performance (Does context impact clinical reasoning performance in experts?). Advances in Health Sciences Education, 17(1), 65–79.

Durning, S. J., Trowbridge, R. L., & Schuwirth, L. (2019). Clinical reasoning and diagnostic error: A call to merge two worlds to improve patient care. Academic Medicine. http://dx.doi.org/10.1097/ACM.0000000000003041

Elstein, A. S. (2009). Thinking about diagnostic thinking: A 30-year perspective. Advances in Health Science Education, 14(sup1), 7–18.

Elstein, A., Shulman, L., & Sprafka, S. (1990). Medical problem solving: A ten-year retrospective. Evaluation and the Health Professions, 13, 5–36.

Eva, K. W. (2005). What every teacher needs to know about clinical reasoning. Medical Education, 39(1), 98–106.

Fenderson, B. A., Damjanov, I., Robeson, M. R., Veloski, J. J., & Rubin, E. (1997). The virtues of extended matching and uncued tests as alternatives to multiple choice questions. Human Pathology, 28(5), 526–532.

Ferguson, E., James, D., & Madeley, L. (2002). Factors associated with success in medical school: Systematic review of the literature. British Medical Journal, 324(7343), 952–957.

Fournier, J. P., Demeester, A., & Charlin, B. (2008). Script concordance tests: Guidelines for construction. BMC Medical Informatics and Decision Making, 8, 18.

Gelb, D. J., Gunderson, C. H., Henry, K. A., Kirshner, H. S., & Jozefowicz, R. F. (2002). The neurology clerkship core curriculum. Neurology, 58(6), 849–852.

Groen, G. J., & Patel, V. L. (1985). Medical problem solving: some questionable assumptions. Medical Education, 19(2), 95–100.

Groves, M., O’rourke, P., & Alexander, H. (2003). The association between student characteristics and the development of clinical reasoning in a graduate-entry, PBL medical programme. Medical Teacher, 25(6), 626–631.

Guerrasio, J., Garrity, M. J., & Aagaard, E. M. (2014). Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006-2012. Academic Medicine, 89(2), 352–358.

Hamdy, H., Prasad, K., Anderson, M. B., Scherpbier, A., Williams, R., Zwierstra, R., & Cuddihy, H. (2006). BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice. Medical Teacher. 28(2), 103–116.

Huggan, P. J., Samarasekara, D. D., Archuleta, S., Khoo, S. M., Sim, J. H., Sin, C. S., & Ooi, S. B. (2012). The successful, rapid transition to a new model of graduate medical education in Singapore. Academic Medicine, 87(9), 1268–1273.

Kanna, B., Gu, Y., Akhuetie, J., & Dimitrov, V. (2009). Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated Internal Medicine residency training program. BMC Medical Education, 9, 42.

Kazour, F., Richa, S., Zoghbi, M., El-Hage, W., & Haddad, F. G. (2017). Using the script concordance test to evaluate clinical reasoning skills in psychiatry. Academic Psychiatry, 41(1), 86-90.

Kusurkar, R., Kruitwagen, C., Ten Cate, O., & Croiset, G. (2010). Effects of age, gender and educational background on strength of motivation for medical school. Advances in Health Science Education, 15(3), 303–313.

Lubarsky, S., Chalk, C., Kazitani, D., Gagnon, R., & Charlin, B. (2009). The script concordance test: A new tool assessing clinical judgement in neurology. Canadian Journal of Neurological Sciences, 36, 326-331.

Lubarsky, S., Charlin, B., Cook, D. A., Chalk, C., & van der Vleuten, C. P. M. (2011). Script concordance testing: A review of published validity evidence. Medical Education, 45(4), 329–338.

McBee, E., Ratcliffe, T., Picho, K., Artino, A. R., Schuwirth, L., Kelly, W., … Durning, S. J. (2015). Consequences of contextual factors on clinical reasoning in resident physicians. Advances in Health Sciences Education, 20(5), 1225–1236.

Monteiro, S. M., & Norman, G. (2013), Diagnostic reasoning: Where we’ve been, where we’re going. Teaching and Learning in Medicine, 25(sup1), S26–S32.

Neufeld, V. R., Norman, G. R., Feightner, J. W., & Barrows, H. S. (1981). Clinical problem-solving by medical students: A cross-sectional and longitudinal analysis. Medical Education, 15, 315–322.

Nicholl, D. J., & Appleton, J. P. (2015). Clinical neurology: Why this still matters in the 21st century. Journal of Neurology, Neurosurgery and Psychiatry, 86(2), 229–233.

Norman, G., Young, M., & Brooks, L. (2007). Non-analytical models of clinical reasoning: The role of experience. Medical Education, 41(12), 1140–1145.

Postma, T. C., & White, J. G. (2015). Socio-demographic and academic correlates of clinical reasoning in a dental school in South Africa. European Journal of Dental Education, 10, 1–8.

Stegers-Jager, K. M., Themmen, A. P. N., Cohen-Schotanus, J., & Steyerberg, E. W. (2015). Predicting performance: Relative importance of students’ background and past performance. Medical Education, 49(9), 933–945.

Tan, K., Chin, H. X., Yau, C. W. L., Lim, E. C. H., Samarasekera, D., Ponnamperuma, G., & Tan, N. C. K. (2017). Evaluating a bedside tool for neuroanatomical localization with extended-matching questions. Anatomical Sciences Education, 11(3), 262-269.

Tan, K., Tan, N. C. K., Kandiah, N., Samarasekera, D., & Ponnamperuma, G. (2014). Validating a script concordance test for assessing neurological localization and emergencies. European Journal of Neurology, 21(11), 1419–1422.

Vaughan, S., Sanders, T., Crossley, N., O’Neill, P., & Wass, V. (2015). Bridging the gap: The roles of social capital and ethnicity in medical student achievement. Medical Education, 49(1), 114–123.

Wan, S. H. (2015). Using the script concordance test to assess clinical reasoning skills in undergraduate and postgraduate medicine. Hong Kong Medical Journal, 21(5), 455–461.

Woloschuk, W., McLaughlin, K., & Wright, B. (2010). Is undergraduate performance predictive of postgraduate performance? Teaching and Learning in Medicine, 22(3), 202–204.

Woloschuk, W., McLaughlin, K., & Wright, B. (2013). Predicting performance on the Medical Council of Canada Qualifying Exam Part II. Teaching and Learning in Medicine, 25(3), 237–241.

Woolf, K., Cave, J., Greenhalgh, T., & Dacre, J. (2008). Ethnic stereotypes and the underachievement of UK medical students from ethnic minorities: Qualitative study. British Medical Journal [Internet], 377, a1220. https://doi.org/10.1136/bmj.a1220

Woolf, K., Haq, I., McManus, I. C., Higham, J., & Dacre, J. (2008). Exploring the underperformance of male and minority ethnic medical students in first year clinical examinations. Advances in Health Science Education, 13(5), 607–616.

Woolf, K., Potts, H. W. W., & McManus, I. C. (2011). Ethnicity and academic performance in UK trained doctors and medical students: Systematic review and meta-analysis. British Medical Journal [Internet], 342, d901.  https://doi.org/10.1136/bmj.d901

*Nigel Choon Kiat Tan
Office of Neurological Education,
Department of Neurology,
National Neuroscience Institute
11 Jalan Tan Tock Seng,
Singapore 308433
Email: nigel.tan@alumni.nus.edu.sg

Submitted: 1 July 2019
Accepted: 17 December 2019
Published online: 1 September, TAPS 2020, 5(3), 42-53
https://doi.org/10.29060/TAPS.2020-5-3/OA2166

Eng Lai Tan¹, Sook Yee Gan¹, Wei Meng Lim¹, Peter C. K. Pook¹ & Vishna D. Nadarajah²

¹Institute for Research, Development & Innnovation, School of Pharmacy, International Medical University, Kuala Lumpur, Malaysia; ²IMU Centre for Education, School of Medicine, International Medical University, Kuala Lumpur, Malaysia

Abstract

Practice Highlights

• Competencies from research projects need to be transparent to both students and supervisors.
  
• Diversity of research projects should reflect the different career pathways of pharmacists.
  
• Ethical and professional dilemmas from research projects is opportunity for reflective learning.

I. INTRODUCTION

The Pharmacy profession has undergone tremendous changes over the years, and its scope has expanded. The roles of Pharmacists have extended beyond the traditional boundaries of drug preparation and distribution to ensuring that optimal therapeutic outcomes are achieved through patient-centred cognitive services (Bond, 2006; van Mil & Fernandez-Llimos, 2013). Pharmacists play increasing roles in patient education and counselling, health promotion and disease prevention, disease state management as well as being engaged in inter-professional consultation with other healthcare professionals in specialised patient settings (Holland & Nimmo, 1999; Tsuyuki & Schindel, 2008). The new roles for pharmacists evolve in parallel with evidence-based medicine (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). Therefore, research skills are essential for both the practice and advancement of the pharmacy profession. In pharmacy, as in all undergraduate science programmes, research is a critical and essential component of the curriculum, although for pharmacy, being a professional degree, competency for practice takes precedence in the priority of the curriculum (Nykamp, Murphy, Marshall, & Bell, 2010). Nevertheless, there is a need to build a strong research program and culture within a pharmacy degree curriculum through sustainable educational initiatives that complement rather than compromise competencies needed for practice. Research being a critical part of scholarship is necessary for inculcating the attributes related to professional competency such as creative and critical thinking as well as problem-solving. Moreover, research also improves student learning skills and encourages the pursuit of research-related careers (Banks, Haynes, & Sprague, 2009; Nykamp et al., 2010).

The implementation of research in pharmacy curricula varies between institutions. Many colleges and universities require students to undertake coursework in research methodology, biostatistics, drug information, and literature evaluation, but only a small fraction of them chose to complete an extensive project with data collection, analysis, and reporting research findings (Murphy, Peralta, & Kirking, 1999). However, increasing emphasis and proportion of time allocated for actual data collection and analyses over the years attest to the recognition of research experience in pharmacy training (Murphy, Slack, Boesen, & Kirking, 2007). Undergraduate or first-degree research training requires a supportive environment and intellectual partnership amongst students and their faculty mentors. Through research, these students are able to apply knowledge gained in the classroom as they define new problems and formulate new research questions (Ash Merkel, 2003). Incorporating research into the curriculum is important as a means of inculcating scholarship in the community of learning, to motivate undergraduates to become independent thinkers and to prepare students for graduate programs (Adamsen, Larsen, Bjerregaard, & Madsen, 2003). A study by Tan revealed that undergraduate students who were guided by suitable research mentors experienced improved thinking, communication, and interpersonal skills. They also manifested heightened levels of self-confidence, resourcefulness, goal-consciousness, creativity and responsibility towards others. These were in contrast to the general feeling of insecurity and uncertainty at the beginning of the research endeavour (Tan, 2007).

Investing in research is often regarded as a costly endeavour which involves dedicated time from faculty members (Nykamp et al., 2010). Furthermore, providing research opportunities for undergraduate students inevitably involves internal funding as well as the involvement of considerable time and proportion of the faculty member. Several major barriers to implementing undergraduate research have been reported. Among these include a lack of faculty members with appropriate expertise and sufficient time for research supervision; other major impediments include the lack of dedicated time for data collection, opportunities, funding, training and support (Nykamp et al., 2010; Paalman, 2002). The logistics of managing research projects for a large number of students have been reported in some studies to be difficult or impossible. Universities have the option of eliminating laboratory experience from their undergraduate research project because of costs associated with maintaining laboratory personnel and the acquisition of expendable laboratory supplies and major equipment (Brandenberger, 1990).

Literature and contextual delivery of pharmacy programs across the world suggest the need to determine the impact of incorporating a research program for an undergraduate pharmacy curriculum (Awaisu & Alsalimy, 2015; Bunnett, 1984; Chopin, 2002; Doerschuk, 2004; Osborn & Karukstis, 2009; Warner, 1998). Will a semester dedicated to entirely research help students achieve the graduate competencies for the pharmacy profession through experiential learning? Hence, this study was conducted to evaluate the impact of a dedicated semester-long research program in an undergraduate pharmacy curriculum.

Before the commencement of the research semester, students select their research projects from a range of areas relevant to Pharmacy that include pharmacy practice, pharmaceutical technology, pharmacy chemistry and life sciences. These projects could be further categorised as laboratory-based, community-based or education research. Through the student mobility program and unique research partnership with other local and overseas institutions, students have opportunitiesto conduct their research projects in these external institutions. Students then defend their project proposals in the Research and Ethics Management Committee which ensures the quality, suitability and ethical aspects of a research project. In the Research Methodology module, students are given theoretical instructions in conducting a literature review, scientific writing and research ethics as well as training in statistics. The research semester spans a period of 16 weeks and should, therefore, be designed to provide an immersive experience in the rigours of research. This study measures the impact of the implementation of dedicated research semester on students’ perceived competencies related to research writing, presentation and critical thinking skills, ethical knowledge and preparedness to undertake research.

II. METHODS

This study is part of the regular programme audit conducted by the School of Pharmacy, the findings of which had been reported to the school’s curriculum and examination committee for quality assurance purpose. Student feedback was sought from final year undergraduate pharmacy students at the International Medical University in Kuala Lumpur, Malaysia in the second half of 2016. Students’ participation was on a voluntary basis after informed consent was obtained and was also a part of the periodic curriculum assessment conducted by the School of Pharmacy. Feedback was obtained from students using an online questionnaire before and after the research project. The questions were developed to measure a student acquiescence of specific perceived competencies relevant to pharmacists as informed by the literature. The face and content validity of the questionnaire were conducted with a small cohort of undergraduate pharmacy students prior to the survey. The pre-research online questionnaire consisted of questions related to the nature, types and placement sites of the projects as well as factors affecting students’ project preference. Ranking questions were also included for students’ self-assessment of their writing, presentation, critical thinking and research skills. In addition, there were also open questions that solicit information pertaining to students’ perceptions about their preparedness to undertake a research project, their knowledge about ethics in research and their anticipated career options. Where Likert-scale was used, the scores from each respondent were added up to achieve the final total. Wilcoxon sign-ranked test was used to compare all ranked data pertaining to students’ perception of their competencies.  The post-research survey consisted of similar questions found in the pre-research questionnaire but with an additional section on evaluating the impact of research projects that the students have undertaken. It contained questions to solicit students’ perceptions on the enhancement of their knowledge in specific subject matter, the achievement of research objectives, the challenges they faced during the implementation of research project, skills developed and gained through the research project as well as the aspects of research ethics which are applicable to their future profession as pharmacists and the impact of research experience on their employment potential. All statistical tests were performed using SPSS Statistics and Microsoft Excel.

III. RESULTS

A. Study Population

Out of 180 students, a total of 113 responses was received in surveys conducted prior to and after the commencement of their research projects in the year 2016. These students were supervised by academic staff from all the four departments of the School of Pharmacy, namely life sciences (31%), pharmaceutical chemistry (21%), pharmaceutical technology (10%), pharmacy practice (22%) as well as the School of Dentistry (2%) and the School of Medicine (14%). Eighteen students were involved in community-based or education research while 95 students conducted lab-based research including those who were attached to other local institutions (eight students) or international partner universities (seven students) through collaborative research.

Most students of the cohort were successfully allocated to the project which was their first (61%), second (20%) or third choice (6%) although some (13%) were not allocated to the project of their choice. Interestingly, the main factor affecting the pharmacy students’ preference was research interest (61%). Some students selected the project based on their choice of supervisors (19%), their peer’s choice (10%) while other did not have a specific preference (10%).

B. Students’ Ratings on Preparedness, Ethics Understanding, Writing and Presentation Skills

In quantitative measures, we evaluate the percentage distribution of student ratings before and after the implementation of research projects pertaining to preparedness to undertake a research project, understanding of research ethics, writing skill and presentation skill. For preparedness to undertake a research project, students are more prepared to undertake a research project in the future, the percentage rating for good had increased from 30.1% to 40.7% (Appendix). However, the frequency distribution of the outcome measures of students’ preparedness after the project implementation was quite similar without any significant difference to before project implementation (p=0.472) (Figure 1A).

Before the research project, approximately 32.8% of students rated themselves for having good and excellence understanding in research ethics. However, after the research project, the percentage has increased to 45.1%. Statistical analysis showed significant improvement for the understanding of research ethics (p=0.002; Figure 1B).

There was 23.9% and 35.4% of students rated themselves for having good and excellent writing skill and presentation skill respectively. After the research project, the improvement is promising as the percentages had increased to 35.4% (p=0.001; Figure 1C) and 51.3% (p=0.001; Figure 1D) respectively.

Note: The ratings before and after project implementation were compared and statistically analysed using the Wilcoxon signed-rank test.

Figure 1. Percentage distribution of student ratings before and after the implementation of research projects pertaining to their
(A) Preparedness to undertake research project (Mean value before=2.34, after=2.39, p=0.472);
(B) Understanding of research ethics (Mean value before=2.19, after=2.46, p=0.002);
(C) Writing skill (Mean value before=2.12, after=2.30, p=0.001); and
(D) Presentation skill (Mean value before=2.33, after=2.54, p=0.001).

C. Students’ Preparedness to Undertake a Research Project

Before the implementation of the research project, most students considered themselves well-prepared as they have improved their knowledge on the subject matter through personal reading and consultation with the respective supervisor (Table 1; Appendix A). They were well-informed about the research approach and techniques. However, these attributes were found lacking in those who have identified themselves as poor in preparation because they had less confidence and did not know what to expect. Students stated that they were now prepared to undertake research in the future because they have gained knowledge and exposure in research as well as trained with various laboratory and research skills (Table 1).

D. Students’ Self-Assessment of Their Writing and Presentation Skills

The surveys also considered students’ self-assessment on their writing and presentation skills. Overall, there was a significant improvement in both skills after the implementation of the research projects. They have contributed their improvement to the reading of journal articles, writing and presenting the research proposal and dissertation. Moreover, students perceived that writing a scientific report was different from writing other reports (Table 1). They found that it was challenging to discuss gathered information or results critically in a constructive manner as well as to ensure a good flow of content and clarity of the paper. As for presentation, some stated that stage fright affected their performance while others found that the question and answer section was quite challenging (Table 1).

E. Students’ Perceptions of Research Ethics

Students’ perceptions of their understanding of research ethics are shown in Table 1. They have related unethical practices as manipulation of data, dishonesty in data reporting as well as lack of objectivity and confidentiality (Table 1). They described good ethical practices as fairness, accountability, peer respect, minimising harm as well as respect for human dignity and rights. Students were able to learn about ethics through their research projects (Table 1). However, some of them indicated that they might have sound ethical knowledge but found themselves inadequate to make decisions and were unsure about how to apply it. Students were, however, able to identify several aspects of research ethics which are applicable to their future profession as pharmacists. These professional values include honesty, informed consent, evidence-based practice, patient safety, confidentiality, punctuality, integrity, responsibility and respect for others.

	Students’ Comments
	Before Undertaking Research Project	After Undertaking Research Project
Preparedness to undertake research	Done literature review and in the progress of writing proposal. Have a good insight into the knowledge and method of the research project. Under the guidance of a responsible and approachable supervisor, I am quite well prepared so far. I always feel that much information regarding my research project has not been fully understood. I do not think that I am fully prepared to handle everything on my own right now.	Through this research project, I have learnt to be independent as well as gained basic research skills (such as cell culture, aseptic technique, learning how to design a protocol for an experiment through reading articles). I am more confident in doing the experiment by myself and in analysing data. There are still a lot of lab instruments that have yet to be learned. I am not very confident in handling a whole project on my own yet.
Research ethics	I have learnt that I should report my results based on real and actual data. I understand the importance of being honest about our results and be responsible for everything that we do. I only know that the written consent form is important and that I cannot disclose the information of the participants to anyone.	I have learnt a lot about professionalism. I am clear that one of its implications is never to change any result or take any shortcut in order to obtain results. I have learnt to prioritise research ethics more than having perfect data.
Writing skill	Unable to describe an idea clearly. I am poor in explaining and discussing results. Grammar issue and poor in vocabulary. To write a simple essay may be easy but writing a scientific article requires much scientific writing skills.	I have a good flow of word processing with least or no grammatical errors. My sentence structures are straight forward and convey ultimate points to the readers. Reading journal articles had enhanced my writing skill. I am fully aware of the content I want to write. My level of writing skills heightened during past 6 semesters
Presentation skill	Lack of confidence. Find difficulty in conveying messages during presentation especially when answering questions. I cannot speak English fluently. Having stage frights.	I am able to convey my message clearly, influence and attract the attention of the audience. I have learnt how to speak more confidently in front of the public. I can present well because I do not have stage fright. I can present my ideas fluently, but I would need improvement on the question and answer session.

IV. DISCUSSION

This study confirmed that preparation for the research semester enhanced knowledge in topics of interest, research and collaborative skills for students. Through the research program, they are able to appreciate, explain, apply and relate the knowledge gained from research to lessons learnt in lectures. This exposure has enhanced their confidence in pursuing the research topics of interest after graduation. It would also improve their capability to work independently. This agrees with studies that confirmed that hands-on research experiences have been recognised to provide the best learning tools that enhance students’ learning, especially when done in collaboration with peers (Ash Merkel, 2003). The research program experience also had a positive impact on ethics and professionalism. Different instructional methods have been shown to be important for the comprehension of ethical knowledge, particularly those that involve intensive students’ participation and role-play (Guérin & Bussières, 2017). The research program described in this study provides the opportunity for application and comprehension as it requires full student participation for one semester and with preparation in the prior semester. The importance of engaging students in ethics and professionalism issues has been shown in a few studies related to pharmacy. In a study that compared the understanding of pharmaceutical ethical statements between pharmacy students and hospital pharmacists, a significant difference in understanding was reported in areas that were assessed, particularly in clinical research, advertising and marketing, dispensing medication, pharmaceutical care and code of ethics. The probable reason for this is knowledge in professional ethics obtained during undergraduate training might not be readily translated to actual professional practice (Guérin & Bussières, 2017). The importance of the right training in instilling ethical knowledge was again emphasised in another study in which the effectiveness of an early professional development series in a pharmaceutical care laboratory (PCL) course for first-year pharmacy students that highlighted the application of the Oath of a Pharmacist and the Pharmacists’ Code of Ethics. The results showed that while students entered their training with a strong appreciation for professionalism, they felt more confident in applying the Oath of a Pharmacist and the Pharmacists Code of Ethics to dilemmas in practice following each new learning activity that required active participation and application (Smith & Dinkins, 2015).

It was also noted that the research program could enhance skills related to scientific communication. Good writing and presenting skills are not naturally acquired skills but must be developed through multiple practice experience. Prior to the research program, these students have undergone several tasks that provided training to improve writing or presentation skills, namely assignments, practical reports, case reports, seminars and examinations in form of essays which were associated with the respective course. In addition, self-confidence was reported to be correlated with the writing ability or performance (Pajares & Johnson, 1994; Ranelli & Nelson, 1998). Another contributing factor to scientific communication is proficiency in the language of communication. Writing might pose a challenge to students who have poor language proficiency especially when it is a second language to students (Chokwe, 2013). In the context of this study, it is English language proficiency as the undergraduate pharmacy program is delivered in English while all students in this program are not native English speakers, but use English as a second language. 

Participation in oral presentations was shown to enhance students’ critical-thinking and problem-solving skills especially when followed with a question and answer session. In addition to enhancing student learning experience and reflection, it provides an opportunity for students to evaluate their own as well as their fellow classmates’ presentations and develop their skills in providing constructive feedback (Hunter, Laursen, & Seymour, 2007). Moreover, the need for troubleshooting, critical analysis of the results and problem-solving during the course of the project contributed to the development of these skills. The benefits of faculty-mentored research projects have been well-recognised and shown to facilitate active learning, cultivate self-confidence as well as enhance students’ critical-thinking and communication skills (Bauer & Bennett, 2003; Lopatto & Tobias, 2010). Translating classroom teaching and reading into hands-on learning activities has been reported to promote personal skills development and enhance the learning experience (Chopin, 2002). The research programme has provided students with an insight into research and development, hence opening a new career option. It was noted that students with research experience were more prepared to pursue careers in science (Lopatto, 2007). Importantly, our findings suggest the skills and competencies gained from the research projects are aligned to the expanding roles of pharmacists that require evidence-based practices as reviewed by Bond (2006) and van Mil and Fernandez-Llimos (2013).

Several important practical points pertaining to the implementation of undergraduate pharmacy research projects are worthy of considerations. Perhaps the most important being the importance of a 16-week immersive experience in the rigours of research. In the current study, the number of laboratory projects exceeded the projects related to pharmacy practice and pharmacy education. A possible solution to this would be to encourage staff who are practising pharmacists to develop research ideas with their counterparts from other areas of expertise such that there are diversity and relevance of these research projects for future practice. Moreover, only perceived students’ competencies are addressed in this study and follow-up studies with objective measurement of competencies will be carried out.

V. CONCLUSION

Overall, the Pharmacy students surveyed in this study indicated that the impact of a research program has offered them an immersive experience to develop various skills and increased their perceived competencies, knowledge as well as confidence in carrying out research work in diverse disciplines. For academics implementing research programs for undergraduate pharmacists, this study also shows some of the challenges students faced.

Notes on Contributors

Eng Lai Tan (E.L.T.), PhD., is an Associate Professor in the Department of Life Sciences, School of Pharmacy, International Medical University (IMU). He is also the associated Dean of School of Postgraduate Studies, IMU.

Sook Yee Gan (S.Y.G), PhD., is an Associate Professor in the Department of Life Sciences, School of Pharmacy, IMU.

Wei Meng Lim (W.M.L.), PhD., is a lecturer in the Department of Pharmaceutical Technology, School of Pharmacy, IMU.

Vishna D. Nadarajah (V.D.N.), PhD., is the Pro-vice Chancellor of Education.

Peter C.K. Pook (P.C.K.P.), PhD., is the Deputy Vice Chancellor of IMU.

E.L.T., S.Y.G., V.D.N. and P.C.K.P. were involved in the design of the work. E.L.T. and S.Y.G. have administered data collection through surveys. E.L.T, S.Y.G. and W.M.L. conducted data analysis and interpretation. E.L.T and S.Y.G. drafted the article. V.D.N. and P.C.K.P. initiated the study and revised it critically. All authors approved the final version for publication in TAPS.

Ethical Approval

This study was exempted by Institutional Review Board (IRB). This study is part of the regular programme audit conducted by the School of Pharmacy, the findings of which had been reported to the school’s curriculum and examination committee for quality assurance purpose.

Acknowledgements

The authors wish to thank Semester 7 Pharmacy students who had participated in the surveys.

Funding

No funding source was required.

Declaration of Interest

The authors report no declaration of interest.

References

Adamsen, L., Larsen, K., Bjerregaard, L., & Madsen, J. K. (2003). Moving forward in a role as a researcher: the effect of a research method course on nurses’ research activity. Journal of Clinical Nursing, 12(3). https://doi.org/10.1046/j.1365-2702.2003.00708.x

Ash Merkel, C. (2003). Undergraduate Research at the Research Universities. New Directions for Teaching and Learning, 2003(93), 39-54. https://doi.org/10.1002/tl.87

Awaisu, A., & Alsalimy, N. (2015). Pharmacists’ involvement in and attitudes toward pharmacy practice research: A systematic review of the literature. Research in Social and Administrative Pharmacy, 11(6), 725-748. https://doi.org/10.1016/j.sapharm.2014.12.008

Banks, M. L., Haynes, K., & Sprague, J. E. (2009). A model for motivating PharmD students to pursue a PhD degree. Currents in Pharmacy Teaching and Learning, 1(2), 93-97. https://doi.org/10.1016/j.cptl.2009.10.008

Bauer, K. W., & Bennett, J. S. (2003). Alumni perceptions used to assess undergraduate research experience. The Journal of Higher Education, 74(2), 210-230. https://doi.org/10.1080/00221546.2003.11777197

Bond, C. (2006). The need for pharmacy practice research. International Journal of Pharmacy Practice, 14(1), 1-2.   https://doi.org/10.1211/ijpp.14.1.0001

Bunnett, J. F. (1984). The education of butchers and bakers and public policy makers. Journal of Chemical Education, 61(6), 509. https://doi.org/10.1021/ed061p509

Chmielewski, J. G., & Stapleton, M. G. (2009). The biologists’ forum: The undergraduate research experience: it’s really not for everyone, students and faculty alike. BIOS, 80(2), 53-58.  https://doi.org/10.1893/011.080.0201

Chokwe, J. M. (2013). Factors impacting academic writing skills of english second language students. Mediterranean Journal of Social Sciences, 4(14), 377. http://doi.org/10.5901/mjss.2013.v4n14p377

Chopin, S. F. (2002). Undergraduate research experiences: the translation of science education from reading to doing. The Anatomical Record, 269(1), 3-10. https://doi.org/10.1002/ar.10058

Doerschuk, P. (2004). A research and mentoring program for undergraduate women in computer science. Paper presented at the Frontiers in Education Conference, 34^th Annual FIE 2004, S2H-7. http://doi.org/10.1109/FIE.2004.1408747

Guérin, A., & Bussières, J. (2017). Étude pilote du niveau d’accord à des énoncés sur l’éthique pharmaceutique d’étudiants en pharmacie et pharmaciens hospitaliers Québécois [A pilot study of the professional ethical thinking of Quebec hospital pharmacists and pharmacy students]. Annales Pharmaceutiques Francaises, 75(1), 67-76. https://doi.org/10.1016/j.pharma.2016.07.004

Holland, R., & Nimmo, C. (1999). Transitions, part 1: Beyond pharmaceutical care. The American Journal of Health-System Pharmacy, 56(17), 1758-1764. https://doi.org/10.1093/ajhp/56.17.1758

Hunter, A. B., Laursen, S. L., & Seymour, E. (2007). Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education, 91(1), 36-74. https://doi.org/10.1002/sce.20173

Lopatto, D. (2007). Undergraduate research experiences support science career decisions and active learning. CBE-Life Sciences Education, 6(4), 297-306.  https://doi.org/10.1187/cbe.07-06-0039

Lopatto, D., & Tobias, S. (2010). Science in solution: The impact of undergraduate research on student learning. Washington, D.C.: The Council on Undergraduate Research. Retrieved from https://www.worldcat.org/title/science-in-solution-the-impact-of-undergraduate-research-on-student-learning/oclc/680773869

Murphy, J. E., Peralta, L. S., & Kirking, D. M. (1999). Research experiences and research-related coursework in the education of doctors of pharmacy. Pharmacotherapy, 19(2), 213-220. Retrieved from  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2690916/

Murphy, J. E., Slack, M. K., Boesen, K. P., & Kirking, D. M. (2007). Research-related Coursework and Research Experiences in Doctor of Pharmacy Programs. The American Journal of Pharmaceutical Education, 71(6), 113. https://doi.org/10.1592/phco.19.3.213.30931

Nykamp, D. , Murphy, J. E. , Marshall, L. L., & Bell, A. (2010). Pharmacy students’ participation in a research experience culminating in journal publication. The American Journal of Pharmaceutical Education, 74(3), 47. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2865413

Osborn, J. M., & Karukstis, K. K. (2009). The benefits of undergraduate research, scholarship, and creative activity. Broadening participation in undergraduate research: Fostering excellence and enhancing the impact, 41-53. Retrieved from  https://osborn.pages.tcnj.edu/files/2013/12/2-Osborn-and-Karukstis-2009.pdf

Paalman, M. H. (2002). Undergraduate research, education and the future of science. The Anatomical Record, 269(1), 1-2. https://doi.org/10.1002/ar.10059

Pajares, F., & Johnson, M. J. (1994). Confidence and competence in writing: The role of self-efficacy, outcome expectancy, and apprehension. Research in the Teaching of English, 313-331. Retrieved from https://www.jstor.org/stable/40171341

Ranelli, P. L., & Nelson, J. V. (1998). Assessing writing perceptions and practices of pharmacy students. The American Journal of Pharmaceutical Education, 62(4), 426. Retrieved from https://experts.umn.edu/en/publications/assessing-writing-perceptions-and-practices-of-pharmacy-students

Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: what it is and what it isn’t. The British Medical Journal, 312(7023), 71-72. https://doi.org/10.1136/bmj.312.7023.71

Smith, M. G., & Dinkins, M. M. (2015). Early introduction to professional and ethical dilemmas in a pharmaceutical care laboratory course. The American Journal of Pharmaceutical Education, 79(10), 156. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4749904

Tan, E. B. (2007). Research Experiences of Undergraduate Students at a Comprehensive University. International Journal of Teaching and Learning in Higher Education, 19(3), 205-215. http://www.isetl.org/ijtlhe/past2.cfm?v=19&i=3

Tsuyuki, R. T., & Schindel, T. J. (2008). Changing Pharmacy Practice: The Leadership Challenge. Revue des Pharmaciens du Canada [Canadian Pharmacists Journal], 141(3), 174-180. https://journals.sagepub.com/doi/10.3821/1913701X2008141174CPPTLC20CO2

van Mil, J. W., & Fernandez-Llimos, F. (2013). What is ‘pharmaceutical care’ in 2013? Pharmacy Practice, 11(1), 1-2. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3780505/

Warner, J. W. (1998). A program for facilitating undergraduate research in biology. Journal of the Kentucky Academy of Sciences, 59, 2-5. Retrieved from https://www.biodiversitylibrary.org/item/104104#page/12/mode/1up

*Eng Lai Tan
School of Pharmacy,
International Medical University
126, Jalan Jalil Perkasa 19,
Bukit Jalil, 57000
Kuala Lumpur, Malaysia
Tel: +603-2731 7466
Email: englai_tan@imu.edu.my

Submitted: 15 May 2019
Accepted: 27 November 2019
Published online: 1 September, TAPS 2020, 5(3), 20-27
https://doi.org/10.29060/TAPS.2020-5-3/OA2157

Kosala N. Marambe¹, Deepthi H. Edussuriya² & Dharshana Jayawickrama¹

¹Medical Education Unit, Faculty of Medicine, University of Peradeniya, Sri Lanka; Department of Forensic Medicine, Faculty of Medicine, University of Peradeniya, Sri Lanka

Abstract

Practice Highlights

• Students call for a balanced approach in providing feedback.
• Sense of respect for the learner is solicited.
• Offering learner support is highly valued.
• Negative comments better be non-judgmental and descriptive.
• Accurate conceptualization of feedback process among trainers is a dire need.

I. INTRODUCTION

Feedback to a learner is an important aspect of teaching and learning and is essential in the acquisition of skills as well as development of competencies. In this era of outcome-based education, teacher feedback to learners plays a crucial role in the development of competence in medical training (Ende, 1983; Harden & Laidlaw, 2017). It becomes especially important in medical education in a hospital environment where teaching/learning occurs in an opportunistic manner in the presence of numerous other onlookers.

It is reiterated that in clinical practice, feedback reinforces good practice, identify deficiencies and directs the learner to narrow the gap between actual and desired performance (Bing-You, Paterson, & Levine, 1997; Bing-You & Trowbridge, 2009; Cantillon & Sargeant, 2008). Furthermore, Ponnamperuma (2015) argues desired performance should not be static but dynamic depending on the progress of the learner, emphasising the role of properly directed feedback in assisting enthusiastic learners to realise their potential. Similarly, Hesketh and Laidlaw (2002) argued feedback is essential element of the educational process that can help students reach their maximum potential. However, one should be mindful that feedback if not carefully handled can result in de-motivation and deterioration in performance (Cantillon & Sargeant, 2008).

Much highlighted educational alliance approach is based on a mutual understanding of purpose and of responsibility for learning. This alliance is complex with equal responsibility by the learner to learn and the teacher to teach (van de Ridder, Peters, Stokking, de Ru, & ten Cate, 2015). Barriers to effective feedback include, the purpose of feedback being unclear to the teacher (provider), and learner (recipient), unavailability of appropriate place or inadequate time to complete feedback and minimal or no training provided to the trainer in providing feedback (Hesketh & Laidlaw, 2002; Ponnamperuma, 2015).

van de Ridderet al. (2015) indicates that the effect of feedback may be small, if not given appropriately. However, it is well-known that for feedback to be of maximum benefit, learners’ too need to use and engage with the feedback processes appropriately. This concept which is known as “proactive recipience” includes, characteristics and behaviours of the receiver, sender; the message and context (Interpersonal communication variables); as well as recipient’s processes such as self-appraisal, assessment literacy, goal-setting, self-regulation, engagement and motivation (Winstone, Nash, Parker, & Rowntree, 2017).

A study conducted by Bose and Gijselaers (2013) supported the assumption that supervisors who promote residents to seek feedback with motives of self-improvement should also ensure that their students’ concerns of ego-protection and impression-defence are addressed during the process. van de Ridder et al. (2015) further states that the manner in which the message is framed, whether it is positive or negative affects students’ satisfaction and self-efficacy directly after the intervention even though it is known to fade-out over time.

Further it is believed that national culture plays an important role in the way behaviours are valued and tolerated in educational settings; Hofstede (2001), reported relatively higher power distance indices for South Asia. Sri Lanka being a South Asian country, the hierarchical nature of organisational arrangement is observed. Thus with the existence of high level of power distance between undergraduate students and academic staff, it is not customary for the students to argue with teachers or challenge teachers, even if the teachers deviate from accepted practices. In Asian cultures, the child-parent role pair is replaced by the teacher-student role pair (Hofstede, 2001) leading to observed paternalistic culture in Universities. Hence feedback tends be more of fault-finding with use of punitive measures as a means to change behaviour and it is likely that the trainees accept some of the negative behaviours of teachers (which are unlikely to be accepted by those in the West), considering it to have been done in good faith.

Bowen, Marshall and Murdoch-Eaton (2017) have identified three learner feedback behaviours; recognising, using, and seeking feedback. He proposes that core themes such as learner beliefs, attitudes, and perceptions; relationships; teacher attributes; mode of feedback; and learning culture influence these behaviours. Further, Perera, Lee, Win, Perera and Wijesuriya (2008) have reported perceived mismatch between the expectations of the provider and the receiver in an Asian context.

A survey on students’ perceptions of teacher feedback conducted in the Sri Lankan faculty by Mudiyanse, Pallegama and Marambe (2015) highlighted the existence of several deficiencies. Paucity of recent literature on perceptions of Sri Lankan undergraduate medical students on the feedback practices, prompted us to explore the issue of teacher feedback and its effect on students. As educationists, we believe an exploration of the actual situation would help understand the factors, which facilitate learning among Sri Lankan trainees, thus paving the way for informed decision-making. Hence the objective of the proposed study was to explore perceptions of Sri Lankan medical graduands to feedback received from teachers during clinical training.

II. METHODS

A qualitative inquiry informed by semi-structured interview was performed. The study sample was selected based on the final year results of a single intake of students in one medical Faculty within 3 months of release of results (this group of passed-out medical students awaiting award of degrees at the convocation is defined as graduands). Maximum variation within the sample was ensured through stratified random sampling technique. Fifteen students (8 males) were selected randomly by the research team from each achievement group namely, high achievers (those who secured classes–Group A), average students (those who secured ordinary passes–Group B) and poor performers (those who referred in any one final year subject–Group C).

Prospective participants were provided relevant information about the study via email and invited to be present for a short interview. They were informed that the interviews will be tape-recorded and transcribed for better comprehension and that confidentiality would be ensured. Those willing to participate in the study were instructed to contact the principal investigator by email. The volunteers were given appointments for the interview.

The nature of the study was explained to the volunteer and consent was obtained for audio recording by the Research Assistant (RA) prior to the scheduled interview. All interviews were conducted by the same interviewer in a sound-proof room, in the faculty-based on an interview schedule, which consisted of two open-ended questions. The probes used to obtain genuine responses were to recall a situation based on personal experience, during clinical training, where they were given, 1) constructive feedback and its impact on learning and 2) de-motivating feedback; and its impact on learning. Measures were taken to ensure anonymity.

Subsequently, the interviews were transcribed and read independently by three researchers with experience in qualitative research. Quotes were selected by the three researchers independently and grouped to identify broad themes. The broad themes identified were compared and consensus was achieved on common recurring themes, representative of the perceptions of undergraduate medical trainees on positive and negative experiences in receiving feedback.

III. RESULTS

Eight volunteers from the high achiever group (A), six from average group (B) and seven from referred group (C) amounting to a total of 21 students (63% males) were interviewed. Majority of the experiences with feedback (teacher to trainee) during clinical training in the third to fifth undergraduate year were rated as good. It was revealed feedback has been practiced more often, informally and systematically in the final year. According to interviewees, the majority of clinical teachers had listened to case histories presented by undergraduates and given feedback on the relevant and useful points for diagnosis and planning of management to the student concerned and others in the group during regular ward teaching sessions. Some of the assigned trainers have observed the physical examination techniques of students, clinical procedures and provided feedback to the student concerned, while others in the group also benefited from it.

Analysis of trainee perceptions of positive and negative feedback episodes during undergraduate medical training resulted in the identification of the four themes given below as those which motivated and enhance learning. Selected excerpts representative of all achievement groups, are included as evidence under each theme.

The positive experiences are centred on four themes presented below.

A. Balanced Approach to Feedback

Most of the interviewees valued when the feedback provider set the scene by acknowledging or appreciating the good aspects demonstrated by the trainee, while focusing on deficiencies whether it be history taking or examination.

“The teacher praised us for the good things we did in ward teaching sessions. Good practices were acknowledged during ward rounds … mistakes pointed out. Since the consultant capitalised on good things I did, I did not feel ‘belittled’.”

(A16)

“Teachers observed us, gave immediate feedback. First appreciated the thing done correctly, then advice was given to correct mistakes”

(B16)

“When good practices are highlighted we feel good. What we have missed were also pointed out … confidence increased”

(C7)

B. Demonstrating Respect for Learner

Respect for trainee and engaging in a dialogue, providing a non-threatening learning environment was mentioned as a factor which enhanced learning. These approaches include unbiased and unprejudiced behaviour, less hierarchical context, focus on specific performance and feedback phrased in descriptive, non-evaluative language. The other approaches include, use of neutral language, non-threatening tone of voice during feedback episodes, being sensitive to learners’ emotional status, taking measures to maintain privacy. The trainers’ behaviours underlying the episodes perceived as positive experiences by many interviewees consisted of features described above. Trainees sensed a lot of educational value in the caring, approachable, unbiased, respectful conduct of the trainers.

“The one-to-one feedback in clinical training is a good feature. We felt we could ask questions from the consultant and get some of the doubts clarified.”

(A15)

“We were treated like doctors. We could ask questions without fear, no big distance between student and teachers.”

(A5)

“So, I was encouraged to clarify my doubts in this manner. Never shout at us, so we were free to ask questions and behave in the ward. Adorable teacher, we could ask questions without fear.”

(A13)

“In a clinic setting, I was not thinking of privacy. Teacher called me to a room, … blamed for what was done. Got me to think if she was my mother … So, taught me about consent, privacy.”

(B15)

“The nature of the consultant, non-judgmental when I made mistakes, but tried to correct me by making a point clear. Not exceeding limits.”

(C11)

C. Clear Articulation and Offering Learner Support

Interviewees valued instances where the trainer provided focused advice, explained the mistakes made by trainees and offered adequate direction for improvement to the trainee. Providing clear descriptions of observed behaviour was felt to be more palatable and useful from the point of correcting one’s mistakes.

“I was okay with facts, I was observed during the process by a senior registrar and …, told me that I take more time–I was too slow. I was instructed to examine more abdomens of patients while in the ward, I did it and it became very useful.”

(A4)

“I was advised on how to organise histories, how to rearrange my history to suit final year requirements.”

(A8)

“We were supported, shown how to examine the patient correctly when we made mistakes. Directed us to read around …. Also, checked whether we have improved later”

(B1)

“What we have missed were also pointed out … We were shown correct techniques.”

(C7)

“end appointment viva, OSCE were held, one–to-one feedback. Wrong things we did were pointed out in a nice way. Focused, so we knew how to go about to improve knowledge.”

(C4)

D. Boosting Moral and Self- Esteem of Trainees

The interactions with trainees had been such that the trainer having identified the potential of students made them aware of it and motivated trainees to realise their full potential.

 “I was identified as a good student by the clinician …, I was made aware of my potential, this I found encouraging and I tried to work better from then on in fact, it resulted me getting classes in later years.”

(A6)

“I was asked a simple question about ECG, when I did it right, I was praised… ‘you will become a consultant’, and later I came across an unusual heart sound and I told the consultant about it, he examined and also got the Registrars to listen to that so, I was encouraged to clarify my doubts.”

(A13)

“Third year teachers motivated me. They supervised us and corrected. All of us got a chance to do hands-on skills training.…supported learning”

(C3)

The negative experiences are centred on four themes presented below.

A. Fault-Finding and Judgmental Nature

Some of the expressions on negative experiences highlighted instances where feedback lacked acknowledgement of what was done right and encouragement, and being more of fault-finding exercise.

“When there are gaps in knowledge, the way teachers reacted at the start was not encouraging. We worked really hard, spent 3 whole nights in the ward. We get punished because 2-3 students in the group made mistakes. We are not rewarded for work we do.”

(B1)

“One clinician wanted to discuss a particular problem. … My patient had this problem found through a scan. …Unfortunately when the teacher examined, positive signs present because of full bladder. I was told that I will definitely repeat. I cried. I avoided the teacher. Still it haunts in me. Comments such as ‘you do not have a passing face’ was made, I was worried. Sad, when first impressions get carried through the appointment.”

(B15)

“I volunteered to present a case, I could not make a correct diagnosis and I was told I have cheated.”

(C11)

B. Biased Nature

Above refers to teacher referring to previous performance of a student in giving feedback to a new situation. Students feel if a mistake is made once that student gets noted and will be targeted and treated differently for the same mistake.

“Some students are treated differently. Even if we give correct answers, told not up to the mark.”

(B16)

“Some students were selected and they got scolded in front of everybody, I felt some of them worked very hard, when they make even a slight mistake, without giving credit to what was done right they got scolded.”

(A16)

“I felt expressions made following observations of performance were highly biased. Sometimes tend to generalise and make comments about students which are not true.”

(C3)

C. Indiscreet Behaviour of Trainers

Trainees perceived some environments as hostile, disrespectful, revealing things that should be private as illustrated by following phrases that were used to describe negative experiences.

“Embarrassing when … shouts at us, I think people have reasons for making mistakes. During ward rounds, scolding in front of patients in Sinhala, discourage students. Thereafter, students avoid these patients and lose chances of learning. I felt we were treated as school children.”

(B16)

“Certain appointments too scared of the consultant did not expect good feedback. Very harsh comments.”

(A18)

“Told you will fail, I did not see a fair reason to do so. Specially at a time when one is about to complete training before final exam. Last opportunity to learn in the course is lost. All your expectations are lost. There are better methods to motivate. What was done was unpleasant. Comments made are: you are bad- cannot do, I do not think you can pass.”

(B17)

D. Punishments in Addition to Feedback

Graduates also mentioned about instances where punishments were made in addition to negative feedback.

“Comparing students and scolding these students in front of others made them depressed. I have heard of students being punished when a wrong answer is given.”

(B14)

“Maybe one wrong answer, repeat for one month.”

(C5)

“Sometimes members of a group have to undergo punishment due to the fault of others in the group, like not presenting histories, being absent.”

(C14)

IV. DISCUSSION

It can be concluded that in most feedback episodes, interviewees have benefitted from trainer feedback, which has been balanced and supportive. Acceptance of more frequent, informal, timely, balanced and focused feedback was obvious in the expressions of trainees of all groups. However, the existence of variations among clinicians in the style of giving feedback was also evident. The Sri Lankan graduands valued acknowledgement, appreciation, praise of demonstrated good practices or desirable behaviours, by trainers during feedback sessions. In the Sri Lankan culture praise or words such as ‘excellent’, ‘very good’ and ‘good’ are used by teachers to acknowledge achievement of expected standard by students. Thus, interviewees endorsed the potential of simple measures in reinforcing good behaviour reported by Thorndike (1931) and Krackov (2009). Interestingly, Sri Lankan graduands, a group of South Asians, highlighted most of the characteristics of effective feedback found in literature under positive experiences, though hardly mentioned practices such as providing feedback as desired by the recipient and checking for understanding of the plan of action/ remedial measures (Matua, Seshan, Akintola, & Thanka, 2014; Ramani & Krackov, 2012; Rose, Best, & Higgs, 2005). Also, emphasis on reinforcing desirable behaviour during feedback episodes was not prominent in the expressions. In this regard, Ponnamperuma (2015) was of the view that ‘reinforcing desirable behaviour is rather conservative in South East Asian region’ compared to the West.

Similar to the findings of Hewson and Little (1998), most of the Sri Lankan interviewees endorsed feedback episodes where clinicians created a respectful environment were effective in enhancing learning and self-esteem. In such environments interviewees reported they are not in fear and hence tend to ask questions for clarification, and request support from different levels of clinical teachers, work with an inquiring mind and thereby enhance their knowledge and skills.

The trainees have perceived many advantages of balanced feedback that is trainer initiating the feedback dialogue by acknowledging and praising (which is more than acknowledgement) the observed good practices, later commenting on observed gaps in competencies, discussing remedial measures and finally incorporating monitoring strategies to ensure the attainment of competencies. Boud and Molloy (2013), have argued these as desirable aspects, which enhance learner engagement in a feedback model. Sri Lankans also endorsed the effect of such feedback on boosting confidence (Bing-You et al., 1997; Cantillon & Sargeant, 2008; Krackov, 2009, 2011). Ready acceptance of such balanced or even- handed feedback reported by Hewson and Little (1998) and Ramani and Krackov (2012) was apparent among Sri Lankan trainees.

Ponnamperuma (2015) indicated one of the expected roles of feedback is to challenge learners to realize their potential which demands extending the desirable level of competency for learners progressing well. It was encouraging to note that some of the interview excerpts referred to fulfilment of this role by the local trainers.

However, analysis of narratives on negative feedback episodes revealed desirable practices are not entirely prevalent in the Sri Lankan context. In describing features of negative feedback episodes, words such as ‘hostile’ and ‘tensed’ were frequently mentioned and majority doubted its effect on motivation and promoting learning. According to interviewees this situation has led to the emergence of ‘feedback phobia’, resulting in the avoidance of such teachers. Reflecting on the negative experiences of many interviewees, it is clear that the trainees request feedback to be made on direct observations of trainee performance in all encounters, using neutral language and a respectful tone which are the prescribed good practices in literature (Ramani & Krackov, 2012; van de Ridder et al., 2015). As reiterated in literature, it is important for trainers to maintain privacy and confidentiality particularly, when providing negative feedback (Ponnamperuma, 2015).

Further, being conscious of the tone and language used in feedback encounters is an absolute necessity to avoid de-motivation (Hewson & Little, 1998; van de Ridder et al., 2015). In this regard Ponnamperuma (2015) states negative feedback, if handled in a wrong way, can damage self -esteem, create a ‘feedback phobia’ leading to avoidance of such trainers and training opportunities.

Evidence suggests feedback phobia is not uncommon in the Sri Lankan context. Hence it is essential to frequently remind the trainers to be objective, unbiased and unprejudiced during informal feedback encounters and thereby ensure a favourable emotional climate, being sensitive to recipient’s emotions. A creative trainer would thus initiate a dialogue with the trainee, provide a space for the recipient/trainee to clarify the feedback and later offer guidance and support to enable accomplishment of competencies (Harden & Laidlaw, 2017). However, it is known that creativity requires time and effort on the part of a busy clinical trainer. In order to change the situation for the better, it is recommended regular faculty development programs include theory and practice of constructive feedback models (Boud & Molloy, 2013).

Quoting some of the observed undesirable practices in the South East Asian region, Ponnamperuma (2015) expressed doubt on accurate conceptualisation of feedback by both the provider and the recipient. Analysis of the negative experiences described by the graduands, raises concerns on the level of conceptualisation of feedback among some of the trainers in the Sri Lankan context. For example, interviewees revealed, though identifying undesirable behaviour was commonly practiced by the trainers, in most instances, they were not backed up with options and support for improvement. Hierarchical and paternalistic culture seems prominent in the Sri Lankan context leading to arrogant behaviours of some teachers during feedback episodes (Hofstede, 2001). Furthermore, as exemplified by interviewee quotes, the reaction of some of the clinical trainers had not been proportionate to the mistake made by the trainee concerned and trainees had been left with a sense of helplessness leading to negative repercussions such as emotional distress. Imposing punishments like non-signing off of clinical appointments, in addition to providing negative feedback has resulted in developing feedback phobia and avoidance of training opportunities (Ponnamperuma, 2015).

The graduands have experienced at least a few feedback encounters which were not effective, de-motivating and stressful; this need to be addressed constructively. Ende (1983) was optimistic that after some practice and planning, provision of feedback in clinical medicine is not as difficult as one might think. In this background, there is a need for the administrators to initiate a dialogue on institutional-level student feedback policy. Developing a contextualised feedback framework through active engagement of clinical teachers encompassing Pendleton’s rules (Pendleton, Schofield, Tate, & Havelock, 2003) or Agenda-Led Outcome Based Assessment (Chowdhury & Kalu, 2004), which are learner mediated, will be a step in the right direction.

Recently, Findyartini, Bilszta, Lysk and Soemantri (2019) had reported the effectiveness of employing strategies ensuring active engagement of participants during Faculty development initiatives, leading to cross-cultural adaptation of best practices in clinical teaching. Hence organising workshops for clinical teachers where they are made to reflect on student narratives of positive and negative feedback episodes in the light of good practices could be a useful strategy to promote adoption of desirable behaviours.

V. CONCLUSION

Positive and negative feedback experiences were revealed by Sri Lankan graduands. Features of effective and ineffective feedback episodes in the minds of Sri Lankan trainees are mostly the same as those reported in literature based on Western studies. However, graduands have perceived the existence of conservative behaviours among some of the teachers such as being more interested in fault-finding with less emphasis on discussing corrective measures. Hence it is speculated some of the trainers could be having a ‘culturally coloured’ concept of ‘feedback’, which needs urgent attention in order to realise the educational benefits of feedback. Since Feedback is a powerful learning tool in medical education, all efforts should be made to enhance the skills of trainers on giving feedback to learners.

Notes on Contributors

Kosala N. Marambe, MBBS, PhD (Maastricht), is Head of Medical Education Unit and professor in Medical Education. Has more than a decade of experience in organising staff development programs for Health professions educators, conducting and publishing qualitative research.

Deepthi H. Edussuriya, MBBS, MPhil, PhD (Sri Lanka), is a senior lecturer in the Department of Forensic Medicine and has experience in conducting and publishing qualitative research.

Dharshana Jayawickrama, MBBS, was formerly a temporary lecturer in the Medical Education Unit and served as a Research Assistant.

Ethical Approval

Ethics approval for the study proposal was obtained from the institutional ethics review committee of the University, Sri Lanka (Ethics approval number 2017/EC/69).

Acknowledgements

Authors wish to thank graduands who volunteered to share their experiences and Dr. T.M.S.H. Dharmaratne of the Medical Education Unit (MEU) for his assistance during identifying/selecting codes and identifying themes.

Funding

The research received no specific grant from any funding agency.

Declaration of Interest

The authors have no conflict of interest.

References

Bing-You, R. G., Paterson, J., & Levine, M. A. (1997). Feedback falling on deaf ears: Residents’ receptivity to feedback tempered by sender credibility. Medical Teacher, 19(1), 40–44. https://doi.org/10.3109/01421599709019346

Bing-You, R. G., & Trowbridge, R. L. (2009). Why medical educators may be failing at feedback. Journal of the American Medical Association, 302(12), 1330–1331. https://doi.org/10.1001/jama.2009.1393

Bose, M. M., & Gijselaers, W. H. (2013). Why supervisors should promote feedback-seeking behaviour in medical residency. Medical Teacher, 35(11), e1573-1583. https://doi.org/10.3109/0142159X.2013.803059

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-712. https://doi.org/10.1080/02602938.2012.691462

Bowen, L., Marshall, M., & Murdoch-Eaton, D. (2017). Medical student perceptions of feedback and feedback behaviors within the context of the “educational alliance”. Academic Medicine, 92(9), 1303–1312. https://doi.org/10.1097/ACM.0000000000001632

Cantillon, P., & Sargeant, J. (2008). Giving feedback in clinical settings. British Medical Journal, 337, 1292-1294. https://doi.org/10.1136/bmj.a1961

Chowdhury, R. R., & Kalu, G. (2004). Learning to give feedback in medical education. The Obstetrician & Gynaecologist, 6(4), 243-247. https://doi.org/10.1576/toag.6.4.243.27023

Ende, J. (1983). Feedback in clinical medical education. Journal of the American Medical Association, 250(6), 777-781. https://doi.org/10.1001/jama.1983.03340060055026

Findyartini, A., Bilszta, J., Lysk, J., & Soemantri, D. (2019). Transnational clinical teacher training: Lessons learned and cross cultural implications. The Asia Pacific Scholar, 4(3), 79-90. https://doi.org/10.29060/TAPS.2019-4-3/OA2088

Harden, R. M., & Laidlaw, J. M. (Ed.). (2017). Essential skills for a medical teacher-An introduction to teaching and learning in medicine (2^nd Ed.). London, England: Elsevier.

Hesketh, E. A., & Laidlaw, J. M. (2002). Developing the teaching instinct: Feedback. Medical Teacher, 24(3), 245–248. https://doi.org/10.1080/014215902201409911

Hewson, M. G., & Little, M. L. (1998). Giving feedback in medical education: Verification of recommended techniques. Journal of General Internal Medicine, 13(2), 111–116. https://doi.org/10.1046/j.1525-1497.1998.00027.x

Hofstede, G. H. (2001). Culture’s consequences: Comparing values, behaviours, institutions, and organizations across nations (2^nd Ed., pp.1-100). California, CF: Sage publications.

Krackov, S. K. (2009). Giving feedback. In J. A. Dent & R. M. Harden (Eds.), A practical guide for medical teachers (3^rd Ed.). London, England: Elsevier.

Krackov, S. K. (2011). Expanding the horizon for feedback. Medical Teacher, 33(11), 873–874. https://doi.org/ 10.3109/0142159X.2011.617797

Matua, G. A., Seshan, V., Akintola, A. A., & Thanka, A. N. (2014). Strategies for providing effective feedback during preceptorship: Perspectives from an Omani hospital. Journal of Nursing education and practice, 4(10), 24-31. https://doi.org/10.5430/jnep.v4n10p24

Mudiyanse, R. M., Pallegama, R., & Marambe, K. (2015). Students’ perceptions of teacher feedback and the educational environment as measured by DREEM in a medical faculty in Sri Lanka. Journal of Medical science and clinical research, 3(6), 6101-6106.

Pendleton, D., Schofield, T., Tate, P., & Havelock, P. (2003). The consultation: An approach to learning and teaching. Oxford, England: Oxford University Press. https://doi.org/10.1093/med/9780192632883.001.0001 

Perera, J., Lee, N., Win, K., Perera, J., & Wijesuriya, L. (2008). Formative feedback to students: The mismatch between faculty perceptions and student expectations. Medical Teacher, 30(4), 395–399. https://doi.org/10.1080/01421590801949966

Ponnamperuma, G. (2015). Feedback as conceptualized and practiced in South East Asia. In Chenicheri Sid Nair and Patricie Mertova (Eds.), Enhancing learning and teaching through student feedback in medical and health sciences (1^st Ed., pp.43-59). Oxford, England: Chandos Elsevier. https://doi.org/10.1533/9781780634333.43

Ramani, S., & Krackov, S. K. (2012). Twelve tips for giving feedback effectively in the clinical environment. Medical Teacher, 34(10), 787-791. https://doi.org/10.3109/0142159X.2012.684916

Rose, M., Best, D., & Higgs, J. (2005). Transforming practice through clinical education, professional supervision and mentoring (1^st Ed., pp. 143-159). Edinburgh, United Kingdom: Elsevier. https://doi.org/10.1016/B978-0-443-07454-7.50024-1

Thorndike, E. L. (1931). Human Learning (pp. 162-181). New York, NY: The Century Co. https://doi.org/10.1037/11243-000

van de Ridder, J. M. M., Peters, C. M. M., Stokking, K. M., de Ru, J. A., & ten Cate, O. T. J. (2015). Framing of feedback impacts student’s satisfaction, self-efficacy and performance. Advances in Health Sciences Education, 20(3), 803–816. https://doi.org/10.1007/s10459-014-9567-8

Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17-37. https://doi.org/10.1080/00461520.2016.1207538

*Kosala N. Marambe
Faculty of Medicine,
University of Peradeniya,
Sri Lanka
Email: kosalnm@yahoo.com

Submitted: 25 May 2019
Accepted: 18 February 2020
Published online: 1 September, TAPS 2020, 5(3), 28-41
https://doi.org/10.29060/TAPS.2020-5-3/OA2160

Su Ann Khoo, Warier Aswin, Germac Qiao Yue Shen, Hashim Mubinul Haq, Badron Junaidah, Jinmian Luther Yiew, Mahendran Abiramy & Ganapathy Sashikumar

Children’s Emergency Department, KK Women’s and Children’s Hospital, Singapore

Abstract

Practice Highlights

• Effective communication is paramount in good physicians, nurses and allied health practices.
• A comprehensive blended learning communication workshop improves the communication skills and confidence among all levels of staffs in Children’s Emergency.
• “I Hear You” contains essential elements of effective communication and helped learners to remember while handling difficult communication-related scenarios.
• Patient-centric communication workshop reduced communication-related complaints in the Children’s Emergency.

I. INTRODUCTION

Cultivating the skill of effective communication is a vital component in the training of all healthcare personnel. Good communication skills are an essential component of healthcare and allied health. Effective communication between the doctor and the patient leads to better compliance, better health outcomes, decreased litigation, and higher satisfaction for both doctors and patients (Deveugele et al., 2005; Rider, Hinrichs, & Lown, 2006). In the emergency setting, this would reduce the number of reattendances, which in turn leads to better use of resources and reduce the burden of the Emergency Department (ED; Shendurnikar & Thakkar, 2013). Some of the barriers to good healthcare personnel to patient communication include the usage of medical jargons, inability to communicate in simple language, inappropriate use of body language, lack of time dedicated to communicating during the staff-patient encounter and frequent interruptions (Rowland-Morin & Carroll, 1990).

A large proportion of negative feedback given by patients towards healthcare providers–between 60% to 75%–is related to communication lapses (Krishel & Baraff, 1993; Lau, 2000; Rhee & Bird, 1996; Thompson & Yarnold, 1996). While reviewing 122 complaints received in the ED over 7 years, Hunt and Glucksman (1991) noted that the commonest cause of complaint was on attitude (37.7%) and poor communication accounted for 30% of it. In the Children’s ED, working with the Office of Patient Experience (OPE), we found a pattern of increasing communication-related negative complaints which prompted the initiation of the workshop and this study.

The goal of this communication workshop was to improve the communication skills, increase the level of confidence amongst emergency medicine personnel in dealing with communication issues and to reduce communications related patient feedback in the Children’s ED. This communication training programme was designed to address the issue of an increasing number of complaints received due to communication lapses among doctors and patients between July 2016 and June 2017.  The objective of this workshop was to design and implement a curriculum to effectively teach, deliver and reinforce effective communication skills among doctors and nurses in a busy ED. The advantages of blended learning formats are: They are valued by self-directed adult learners; help overcome limitations of adequate time and space; able to reach a larger number of students; save training costs; produce high student ratings; increase student perceptions of achieving course objectives; and have achieved academic results equivalent to strict face-to-face teaching (Ausburn, 2004; Gray & Tobin, 2010).

The aim of this study is to create an interprofessional communication workshop for the ED to reduce communication-related complaints. Secondly, the study also aims to introduce blended learning in the communication workshop, evaluating and understanding its impact as a teaching tool in the ED.

II. METHODS

The study of the workshop was conducted in five stages: (A) Needs assessment, (B) Curriculum development, (C) Pilot testing, (D) Educational intervention, and (E) Evaluation of the intervention (illustrated in Figure 1).

The research team reviewed complaints and compliments received in the Children’s ED over the 12 months preceding to implementation of this workshop. We then derived a list of the commonest complaint themes that guided the curricula development of this communication workshop. In a previous study by Mehta (2008), reviewing patients’ emails and feedback forms helped to identify training needs (Mehta, 2008; Rowland-Morin & Carroll, 1990; Shendurnikar & Thakkar, 2013). A needs assessment was also conducted among the doctors and nurses working in the Children’s ED.

Based on the literature, surveys and review of complaints, we chose four main themes for the development of the curriculum content. They were (A) Perception of waiting time and handling of dissatisfied patients, (B) Information delivery and expressive quality, (C) Physician’s attitude and lack of empathy/ inappropriate use of body language, and (D) Physician’s explanation of illness and treatment–these are all in keeping with the numerous studies that have been done on factors affecting patient satisfaction in ED (Krishel & Baraff, 1993; Lau, 2000; Rhee & Bird, 1996; Thompson & Yarnold, 1996). These studies focus on the perceived technical quality of care, perception regarding waiting time, information delivery and expressive quality, ED information received, health professional’s attitude, health professional’s explanation of illness and treatment and ease and convenience of care. These themes were applied in the creation of our video-based scenarios, real simulation scenarios during the workshop and delivery of lectures, as well as the development of our very own concept of ‘I Hear You’ (illustrated in Figure 2). Communication scenarios are ED-specific, and this has been given serious consideration and adapted to our multilingual and multicultural community. The needs will be addressed not only based on these themes, but the multisource and focus group survey received from doctors and nurses as previously mentioned.

We used a mixed-method design to develop the curriculum and evaluate the impact of this communication workshop; a similar method used by De Feijter, De Grave, Dornan, Koopmans, and Scherpbier (2011), utilising results from an evaluation questionnaire, data of communication-related complaints obtained from the OPE and focus groups to gauge the impact and learning experiences of the participants from the workshop.

A. Figures

Figure 1. Five stages in the study

Figure 2. “I Hear You” concept; representing the 6 essential elements of effective communication

Figure 3. Timings allocated during the face-to-face workshop

The delivery of the curriculum and contents were based on blended learning.

There were two main parts in the educational intervention; A) a pre-workshop web-based, self-directed, learning module with videos on five different scenarios, followed by B) a three-hour tutor-guided workshop. The workshop consisted of sessions going through scenarios in the videos, real face-to-face session with simulated patients, and small group feedback session with content specialists. The themes of the five main scenarios were: (a) long waiting time, (b) lost full blood count sample, (c) patient education, (d) medication error, and (e) patient management and delivery of medications. During simulation practices, three participants were involved. Each will be given a sheet of paper with different roles to play; one as the doctor or nurse, one as the patient and one as the observer. Each participant received different sheets of paper with instructions to the role player and scenario involved. (Refer to Appendix). Each workshop was conducted by 2 facilitators: 1 from Medical (Senior doctors) and 1 from Nursing (Nurse clinicians and senior staff nurses). The workshops were conducted on a weekly basis, on every Tuesday, for three hours (refer to Figure 3 for the details of 3-hour workshop utilisation). A total of 185 doctors (Resident Physicians, Residents and Medical officers), and 110 nurses were trained over the 16 months period and each of the personnel attended one of the 68 iterations of the workshop. The schedule was coordinated and planned into the roster for both nurses and doctors who were working on shifts. A facilitator guidebook was put together as a reference for all facilitators and to ensure standardisation of the delivery of teaching. The guidebook contained the specific objectives, scenarios and feedback questionnaires.

We monitored feedback from both patients in the ED regarding the quality of communication among doctors and nurses, working in the department after this workshop had been implemented, with the help of OPE. The effect of this workshop on patients’ satisfaction and learners’ improvement were assessed retrospectively in two ways: (1) Number of complaints received based on communication skills and attitude of medical staffs before and after the series of workshop; (2) Learners’ perception and confidence in handling difficult scenarios in the ED before and after the series of workshops. There are regular patients’ satisfactory surveys in the ED, and these questionnaires are distributed to patients after their encounter in the ED. Patients were encouraged to return the forms before formal discharge from the ED, via a box, or to email the Office of Patient Experience directly. There were also service staffs on the ground who provided help and received direct feedback from patients and caregivers. The number of complaints received pre-workshop and during pilot testing were compared with post-workshop. The confidence interval of a proportion was calculated using the Wilson procedure without a correction for continuity.

An effective or positive communication-related encounter consisted of four important elements: (A) Approach, (B) Manner, (C) Techniques in Interaction and (D) Verbal and non-verbal communication cues including eye contact, touch, as well as management of space (O’Hagan et al., 2014). Feedback was categorised as communication-related negative feedback when any of the important elements mentioned above were reported as inadequate or missing in the complaints by patients or caregivers. The evaluation of the workshop consisted of focus group sessions and feedback forms. All the participants filled a feedback form at the end of every workshop session.

Six focus group sessions were conducted. The grouping for the focus group sessions is mixed between doctors and nurses. These focus groups involved a total of 25 doctors and 15 nurses. These doctors were drawn from three different residencies (Family Medicine, Emergency Medicine, Paediatric Medicine), medical officers and resident physicians in the ED. The nurses were all from the Children’s ED. The criteria of selection were based on a purposive sampling of participants across age groups, seniority and experience levels. Informed consent were obtained from all the participants. The focus group discussion scripts were analysed using thematic analysis to identify themes in the participants’ feedback on how the workshop had helped them. The coding of the data was done independently by two reviewers and this was compared. Any differences in opinion were discussed with a third reviewer to achieve an agreeable and suited conclusion.

The trustworthiness of the data analysis and collection was ensured using data and investigator triangulation. Multiple focus groups were held with different groups of people. In terms of investigator triangulation, the coding was performed by two independent people as mentioned earlier. When a certain code or theme was unclear, or the investigator had clarifications with regards to the interviews in focus groups, the investigator went back to that particular individual to clarify their thoughts and views.

III. RESULTS

The needs assessment amongst nurses and junior doctors in the department showed that 70% are not confident in dealing with difficult situations and 90% have not received formal training in communication skills. They felt that there was a compelling need for a formal communications course to teach them skills and techniques in dealing with difficult situations and breaking bad news, a correct way to deliver information to parents and patients after consultation as well as addressing a dissatisfied parent on the long waiting time. Feedback gathered among patients attending the Children ED also indicated that the communication style and skills can be improved to improve the delivery of patient-centric care.

A review of the complaints received in the Children’s ED over the 12 months preceding to implementation of this workshop revealed that 73% of the complaints were communication-related. These complaints were collated directly by the OPE. The top 5 communication-related complaints revolved around long waiting time, lack of synchronisation in the explanation given between different doctors and nurses, clotting of blood samples, medication errors and explanation given to patients by doctors or nurses regarding their conditions.

The pilot testing was carried in the period of 1^st August 2017 to 31^st December 2017.

Blended learning was received well by the staffs in the department; many described as a “breath of fresh air”, compared to the other communication workshops carried out within the institution. In the setting of Children’s ED where doctors and nurse work shift hours, blended learning provided better flexibility and better use of resources.

The participants felt that the workshop was very relevant as it is situation- and department-specific. The new staffs found the workshop helped prepared them mentally of the patients’ and caregivers’ expectations in the Children’s ED. In the educational intervention, the participants found the videos used for the scenarios were useful, and easily accessible, although there were hiccups with internet connections and equipment occasionally. The participants also provided feedback on scenarios to be added on. A small percentage of 1% commented that some of the videos were too long (the videos ranged between 5 and 7 minutes). They liked the discussion sessions after each video as it allowed them to share their own experiences and difficulties. The facilitators would provide options for handing different difficult scenarios. They also liked the simulation scenarios as it helped them to reinforce learning, and learned from others through observation and direct feedback after the sessions.

The modified concept of “I Hear You” by the team was designed to enable staffs to remember the important steps of: i) open the discussion, ii) gathering information, iii) understand patient’s perspective, iv) share information, v) reach an agreement on problems and plans and vi) provide closure. A short and easy to remember phrase like “I Hear You” was found to be useful by staffs to remind, incorporate and practise all the 6 steps of effective communication.

Immediately after the course, 95% of the participants felt that they were able to better frame their communications. Thematic analysis of the focus group revealed 4 themes: “Empowerment of staff”, “Focused communication with parents”, “Confidence in dealing with difficult parents” and “Empathy towards patients and caregivers”.

A. Empowerment of staff

A key thing in the communication among staffs (nurses and doctors) with patients and their caregivers is empowerment. Working in an intensive and highly stressful environment in the ED often leads to a high-burnout rate, and when complaints are received, staff feel that their efforts are often not good enough. The face-to-face sessions have allowed staff to share their experiences with others, and to realise that each and every individual staff member is important in contributing to the care of the patients. Attending the workshop created and reinforced increased empowerment among the participants in dealing with difficult communication situations.

“I felt more empowered when I spoke to parents.”

(Focus group No: 1/ Participant No: 3)

“The course made me feel part of the team and that I was solving issue when speaking to parents.”

(Focus group No: 2/ Participant No: 2)

“I feel like we have the responsibility and trust to speak to families and help them understand the issues faced by the child and also the team.”

(Focus group No: 3/ Participant No:2)

B. Focused communication with parents

The workshop has helped participants to realise the importance of communication to increase the efficiency within the department, focusing back to the patients rather than emotions of anxious or angry caregivers, as well as when to escalate and ask for help on difficult situations.

“I always tried to focus back on the patient rather than the unimportant issues and that helped.”

(Focus group No: 2/ Participant No: 4)

“I kept thinking back to ‘I Hear You’, and the importance of focusing on the caregiver and the message they are trying to get across.”

(Focus group No: 1/ Participant No: 1)

C. Confidence in dealing with difficult parents

The workshop helped participants realise the importance of minding body language, phrases used and the tonality of their speech while trying to communicate effectively with both patients and caregivers.

“I felt confident immediately after the course and used keywords when speaking to parents, rather than going blind.”

(Focus group No: 4/ Participant No: 1)

“At least now, I feel more equipped to handle difficult communication encounters, like I have been trained and have a mental model.”

(Focus group No: 5/ Participant No: 3)

D. Empathy towards patients and caregivers

Understanding the circumstance to the behaviour, caregivers’ beliefs, concerns and expectations of illness and treatment are important points. This allows appropriate response to patients’ and caregivers’ statements about ideas, feeling and values.

“Parents usually have valid point; we just need to figure it out and respect that.”

(Focus group No: 2/ Participant No: 1)

“I try to focus on the matter and the patient, and not take the comments personally.”

(Focus group No: 5/ Participant No: 2)

“It felt like we could truly understand what the parents wanted and see beyond the initial unhappiness.”

(Focus group No: 4/ Participant No: 4)

E. Patient Feedback

There was 81.8% reduction in communication-related negative feedback monthly in the data collected by the OPE in the period of 1^st July 2017 to 31^st December 2017 as compared to 1^st January 2018 to 31^st December 2018 (95% confidence interval; CI 0.523, 0.8182). Over this period of 17 months, there were 99 communication-related negative feedback received: 68 of these were received in the first 6 months (1^st July 2017 to 31^st December 2017) when the pilot workshops were carried out, and only 31 communication-related negative complaints were received in the subsequent 12 months (1^st January 2018 to 31^st December 2018). The number of patients seen yearly in the Children’s ED averaged about 150,000 for both 2017 and 2018 (refer to Table 1). The number of reductions of communication-related negative feedback received monthly during the pre- and post-workshop period was statistically significant. It reflected that with every 10,000 patients seen in the ED monthly, there was a reduction of nine communication-related negative feedback per month, between the pre- and post-workshop implementation period.

Table 1. Comparison of complaints received pre- and post-workshop (total period of 17 months)

IV. DISCUSSION

The findings of this study offer new insights into doctor- or nurse-patient communication because the creation of the curriculum content and delivery of the teaching are built in the values of professionals working in a busy ED, rather than extrapolated from other fields of healthcare. The scenarios and videos were created based on commonest communication-related complaints and feedback from providers on the scenarios they found most challenging.

This study revealed a further emphasis on teaching and reinforcing effective communication skills; something we took for granted that all graduates from medical school have been equipped with. Even staff who have years of experience working in the ED can become complacent and needed reminders on the importance of patient-centred communication to improve the quality of care delivered to patients. The curriculum development based on evidence, review of complaints and feedback from staffs made it relevant and relatable to participants. This is different from existing communication workshop that is more exam-oriented, or touching on general aspects of communication which emphasised mainly on steps of communication without relating to a specific scenario.

Furthermore, the approach to the delivery and running of the workshop in the form of blended learning is well-received by participants. The process of watching pre-workshop videos allows participants time to reflect on their own thoughts and encounters in similar scenarios in the ED. The facilitator-guided workshop, in small groups of six to seven participants, allowed time for reinforcement via discussion of scenarios in the videos and simulation practices. The participants are free to share their views and feedback in a safe space, within the small group. Learners also find the concept of “I Hear You” easy to remember and serves as a reminder of the six steps of a good doctor- or nurse-patient communication.

The results indicate four themes that reflect on how the workshop has helped learners personally and in developing effective patient-centric communication; empowerment, focus of communication, confidence and empathy towards patients.  With advances in medical care and modern management concepts, health care institutions are moving towards patient-centred care, and aim to increase patients’ satisfaction and overall experience of clinical encounters.

One year into running the communication workshop, there was a striking 81.8% decrease in communication-related negative complaints received in the Children’s ED. More importantly, we also found that this workshop had helped to boost confidence and morale, especially among the doctors and nurses, in dealing with difficult situations.

Teachers and participants had learned that teaching a “soft skill” like communication is essential and unfortunately often overlooked because we assumed our doctors and nurses had already been well-equipped upon graduation of respective medical or nursing schools. The workshop provided a safe space for staffs to share their reflections and feedback on the video scenarios and during simulation practice of difficult communication situations in the ED. The staffs were free from distractions of involvement in interaction and were therefore in a position to provide comprehensive and reflective feedback. Learners identified important aspects of effective communication that often co-occur with one another. The feedback given by the facilitators demonstrated and helped learners realised how a lack of patient-centredness in approach underpins an absence rapport building and other behaviour associated with a positive manner towards patients.

It will be meaningful to continue tracking the progress of feedback from patients with regards to communication-related issues, as well as to follow-up with staffs who have been trained to ensure that the good practice and application of “I Hear You” continues. This workshop has continued to run to include the new doctors and nurses rotating, or working as permanent staff in the department. In time to come, we hope to extend the training in this communication workshop to allied health professionals who work directly or indirectly with the department to improve standardisation and patients’ overall experience in the Children’s Emergency. The knowledge and experience have also been shared with other EDs, with modifications suited to patient population and types of feedback received. These departments have sent observers to join our workshop sessions. We are hopeful that the approach and usefulness of this workshop continue to benefit all the healthcare providers and lead to improved care for patients.

V. LIMITATIONS

This study has several limitations. The video consisted of scenarios specific to the ED, hence may not be directly applicable to other healthcare settings. A patient encounter and experience in the ED consist not only encounter with the doctors and nurses, but also with the allied healthcare professionals in the ED. There is also a lack of local studies to compare the effectiveness of similar interventions, which have been proven useful in our institution.

VI. CONCLUSION

A focused, patient-centric and blended communication workshop was found to improve the communication skills and confidence among doctors and nurses in the ED, with a corresponding increase in patients’ satisfaction and a reduction in complaints related to communication lapses. This study serves as a starting point in the local context, to bring the emphasis and importance in teaching, informing and reinforcing the important aspects of communication that clinicians and educators consider relevant for effective doctor- and nurse-patient interactions in clinical practice. This workshop also helped to orient junior doctors to what is valued by patients, experienced peers and encourage greater awareness of the impact of particular approaches and techniques to effective communication with patients and caregivers.

Notes on Contributors

Dr SA Khoo is a Staff Registrar with the Children’s Emergency Department (ED), KKH. She did the literature reviews, participated in poster and oral presentation for this project, as well as the write-up of this manuscript.

Dr Warier Aswin, Senior Staff registrar with the Children ED, KKH; Nurse Clinician Germac Shen Qiao Yue; Dr Badron Junaidah Staff Physician with the Children’s ED, KKH; and Senior staff nurses Luther Yiew Jin Mian and Mubinul Haq Hashim, helped designed the program, faculty guide and videos for the workshop.

Dr Sashikumar Ganapathy, Deputy Head and Consultant with the Children’s ED, KKH, is the overall supervisor who conceptualised the workshop, handled focus group interviews, creation of videos and faculty guide.

Dr Mahendran Abiramy and all the previous contributors mentioned above are also faculties who trained the doctors and nurses in the workshop.

Ethical Approval

This study has been reviewed and approved by our institution’s Centralised Institutional Review Board of Singhealth (CIRB) committee. The CIRB reference number is 2017/2784.

Acknowledgements

Special acknowledgement to Mr Luther Yiew Jinmian and Singhealth Academy team who helped created the videos used for the teaching and discussions in communication workshop.

We would also like to thank Dr Lee Khai Pin, Senior Consultant and Head of Children’s Emergency Department, and Dr Arif Tyebally, Senior Consultant and Deputy Head of Children’s Emergency Department, for their support in the running this program.

The team is also grateful to the team from the Office of Patient Experience for providing us with the data of negative communication-related complaints and continued to monitor that for the department.

Funding

This program initiative was funded by AMEI (Academic Medicine Education Institute) Education Grant 2017.

Declaration of Interest

The authors report no conflict of interest.

References

Ausburn, L. J. (2004). Course design elements most valued by adult learners in blended online education environments: An American perspective. Education Media International, 41(4), 327-337.

De Feijter, J. M., De Grave, W. S., Dornan, T., Koopmans, R. P., & Scherpbier, A. J. (2011). Students’ perceptions of patient safety during the transition from undergraduate to postgraduate training: an activity theory analysis. Advances in Health Sciences Education, 16(3), 347-358.

Deveugele, M., Derese, A., De Maesschaick, S., William, S., Dariel, V. M., & Maeseneer, D. J. (2005). Teaching communication skills to medical students, a challenge in curriculum. Patient Education and Counselling, 58(3), 265-270.

Gray, K., & Tobin, J. (2010). Introducing an online community into a clinical education setting: A pilot study of student and staff engagement and outcomes using blended learning. BMC Medical Education, 10(6), 1-9.

Hunt, M. T., & Glucksman, M. E. (1991). A review of 7 years of complaints in an inner-city accident and emergency department. Emergency Medicine Journal, 8(1), 17-23.

Krishel, S., & Baraff, L. J. (1993). Effect of emergency department information on patient satisfaction. Annals of Emergency Medicine, 22(3), 568-572.

Lau, F. L. (2000). Can communication skills workshops for emergency department doctors improve patient satisfaction? Journal of Accident & Emergency Medicine, 17(4), 251–253.

Mehta, P. N. (2008). Communication skills-talking to parents. Indian Paediatrics, 45(4), 300-304.

O’Hagan, S., Manias, E., Elder, C., Pill, J., Woodward-Kon, R., McNamara, T., … McColl, G. (2014). What counts as effective communication in nursing? Evidence from nurse educators’ and clinicians’ feedback on nurse interactions with simulated patients. Journal of Advanced Nursing, 70(6), 1344-1356. 

Rhee, K. J., & Bird, J. (1996). Perceptions and satisfaction with emergency department care. The Journal of Emergency Medicine, 14(6), 679–683.

Rider, A., Hinrichs, M. M., & Lown, B. A. (2006). A model for communication skills assessment across the undergraduate curriculum. Medical Teacher, 28(5), 127-134.

Rowland-Morin, P. A., & Carroll, J. G. (1990). Verbal communication skills and patient satisfaction. Evaluation and the Health Professions, 13(2), 168-185.

Shendurnikar, N., & Thakkar, P. A. (2013). Communication skills to ensure patient satisfaction. The Indian Journal of Paediatrics, 80(11), 938-943.

Thompson, D. A., & Yarnold, P. R. (1996). Effects of actual waiting time, perceived waiting time, information delivery, and expressive quality on patient satisfaction in the emergency department. Annals of Emergency Medicine, 28(6), 657–665.

*Khoo Su Ann
Children’s Emergency Department,
KK Women’s and Children’s Hospital (KKH),
100 Bukit Timah Road, 229899
Email: khoo.su.ann@kkh.com.sg

Submitted: 1 November 2018
Accepted: 24 April 2019
Published online: 1 September, TAPS 2020, 5(3), 10-19
https://doi.org/10.29060/TAPS.2020-5-3/OA2113

Zhi Xiong Chen^1-5, Shuh Shing Lee², Dujeepa D. Samarasekera² & Renee Stalmeijer⁶

¹Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore; ²Centre for Medical Education, Yong Loo Lin School of Medicine, National University of Singapore; ³Office of Student Affairs, National University of Singapore; ⁴VIVA-KKH Paediatric Brain and Solid Tumour Programme, KK Women’s and Children’s Hospital, Singapore; ⁵National University Cancer Institute, Singapore; ⁶School of Health Professions Education, Faculty of Health Medicine and Life Sciences, Maastricht University, The Netherlands

Abstract

The expansion of biomedical sciences has seen a recent boom in the number of graduate students and early career scientists. However, the lack of motivation and increasing departure of graduates from research careers were not well explained in current literature. Elements such as intrinsic motivation and external factors may play moderating or independent roles in altering these outcomes. Using semi-structured interviews, we sought to investigate the role of intrinsic motivation and external factors in shaping biomedical sciences graduate students’ and early career biomedical scientists’ research experiences, and the impact on their perceived research performance and expected career longevity. Using thematic analysis, our data from 10 graduate students or early biomedical scientists participants from uncover several aspects of intrinsic motivation such as its intensity, foundation and nature, and specific external factors such as the presence or absence of social support, the need for career progression driven by societal pressure, and the nature of the research environment that could all influence the students’ and trainees’ psychological state of mind. In turn, this could impact their perceived research performance ability, and desire to stay or leave the biomedical research field. Our study provided an in-depth perspective into the underlying reasons for leaving biomedical sciences or the lack of ‘feel-good’ in research besides competition, funding and publications. This could lead to further discussions on policy changes and interventions that could improve biomedical sciences graduate education and training in future.

Practice Highlights

• Train supervisors in purposeful and meaningful mentoring skills.
• Enhance mental health support for biomedical sciences graduate students and trainees.
• Enhance support in areas that may affect motivation, e.g. family and financial.
• Develop career-proof programs that outline clear learning goals, approaches and outcomes.

I. INTRODUCTION

Biomedical scientists are important members of the healthcare community and developing them is a long-term commitment (Moses III & Martin, 2011). Their education and training enables them to advance healthcare through discoveries and innovations. In addition, they perform imperative roles in health professions education through the integrated teaching and learning of biomedical sciences that can transform the practice of medicine (Bandiera, Boucher, Neville, Kuper, & Hodges, 2013; Pawlina, 2009).

Many years of education and training are required to nurture each biomedical scientist to be competent in the field. On top of the general education that biomedical scientists received, they have to embark on building specific domain knowledge through graduate education. Thereafter, graduates often have to devote some years of postdoctoral work prior to becoming independent biomedical researchers, which is the eventual desired educational outcome. Indeed, continuing workplace education is required to enable them to discover, innovate and solve problems within and beyond their domain knowledge. Given such long-term and extensive investments are required to nurture biomedical scientists, it is therefore, disappointing that many promising graduates often leave biomedical sciences to pursue other non-related careers, while those in training often face declining scholarship and performance during the course of their education (Callaway, 2014; Fuhrmann, Halme, O’Sullivan, & Lindstaedt, 2011; Gould, 2015; Powell, 2015; “There is life after academia,” 2014).

The problems that plague biomedical sciences graduate students’ and early career biomedical scientists’ education and training have long been acknowledged and deemed far too complex to understand or solve (Bourne, 2013). Declining motivation, high pressure and stress, thinking in silos, negativity and narrow perspectives are but a few of the problems that we observe in Singapore. Research has shown that biomedical sciences students who are inclined to remain in biomedical sciences education and research have stronger ‘taste for science’ (Roach & Sauermann, 2010), focus on communal utility value (other-oriented) and not just agentic utility value (self-oriented; Brown, Smith, Thoman, Allen, & Muragishi, 2015), and supported by faculty mentoring (Lopatto & Williams, 2007; Rockey, 2014). However, how these factors contribute to motivation in biomedical sciences remains unclear. They also do not sufficiently explain the problems we observed in the biomedical sciences graduate education system of Singapore and possibly elsewhere. While the shortage of biomedical sciences jobs in academia and industry may contribute to the decline in motivation, it does not sufficiently explain our observation of declining motivation during graduate education or early post-PhD training that is independent of career opportunities.

Based on these observations and literature review, we developed a conceptual framework to investigate constructs that may affect motivation with perceived research performance, and career choice and sustainability as end-educational outcomes (Figure 1). These constructs are need-for-cognition, need-for-closure and intrinsic motivation. Briefly, need-for-cognition refers to an individual’s desire to engage in effortful cognitive activities for enjoyment (Cacioppo & Petty, 1982). Need-for-closure refers to an individual’s need to seek closure to questions and extent of discomfort in dealing with uncertainty (Roets, Kruglanski, Kossowska, Pierro, & Hong, 2015). Finally, intrinsic motivation refers to an individual’s likelihood to invest effort towards objectives based on innate reasons such as personal interest and attitudes (Koestner & Losier, 2002). This is highly dependent on the individual’s sense of autonomy, competence and purpose (Ryan & Deci, 2000).

Our preliminary work suggested that motivation inclined towards need-for-cognition, are more likely to lead to better performance in training and continuation in biomedical sciences careers, given permissive external conditions such as funding and availability (manuscript accepted). These are individuals who enjoy figuring out solutions for difficult puzzles, demonstrate knowledge-seeking behaviour and are inclined to devote effort towards goals based on their desire for improvement and self-interest instead of external pressures (Cacioppo & Petty, 1982; Koestner & Losier, 2002). However, this does not explain 1) why some individuals are more motivated than others, 2) the mechanism of constant cognitive exertion on performance in biomedical sciences education and training, and 3) the external factors that influence this.

Figure 1. Conceptual framework shows the interaction of various constructs affecting motivation and extrinsic factors on perceived research performance, and anticipated career choice and sustainability

Through seeking to explain this framework further, this study aims to obtain an in-depth understanding of motivation for biomedical sciences graduate education and training. In doing so, the study seeks to deliver novel insights that will guide the development of policies and curricula for future training of biomedical sciences graduate students and early career biomedical scientists, not just in Singapore but especially for institutions in Asia whereby they share greater similarity with Singapore in terms of context, background and culture.

The research questions are: In the perception of biomedical sciences graduate students and early career biomedical scientists,

1. How does intrinsic motivation affect them during graduate education and training?
2. What are the external factors that may affect them during graduate education and training, and how?

II. METHODS

A. Methodology

We addressed our research questions through a qualitative study using semi-structured interviews. A qualitative study will allow us to obtain and understand the underlying reasons and rich meaning of participants’ choices, decisions, attitudes, behaviours and viewpoints. This is because qualitative research is flexible to elicit more answers, describes variation and explains relationships either individually or as group norms (Sullivan & Sargeant, 2011). This qualitative research seeks to drill deeper into our preliminary questionnaire, which revealed that participants who were more intrinsically motivated tend to do better in training as well as choose to stay on in biomedical-related careers (manuscript accepted). Enjoying cognitive activities and external factors also help to better training performance (manuscript accepted). Therefore, semi-structured interviews were chosen to obtain deeper meaning from underlying reasons rich in context and experience, behind these questionnaire findings.

The interviews were conducted at Centre for Medical Education (CenMED), National University of Singapore (NUS) and involved Yong Loo Lin School of Medicine, Faculty of Science and NUS Graduate School for Integrative Sciences and Engineering, which offer discipline-specific or inter-discipline graduate programs in biomedical sciences as well as running laboratories that offer diverse training opportunities for early career biomedical scientists. CenMED acted as the third party to carry out the semi-structured interviews, independent of the investigators, to avoid bias due to vested interest in the study. CenMED contacted and recruited participants for the semi-structured interviews. The Principal Investigator (PI) provided documentation for the interviews such as interview questions and interview guide as well as briefed interviewers on the objectives of the research, context and background of the participants and study. For personal data protection purpose, participants were asked to authorise their consent accordingly and informed about the safeguard and confidentiality of their responses through anonymity or composite stories. Audio recordings of the interviews were transcribed verbatim by a transcriber and verified manually by the PI.

In this qualitative study, our approach of using semi-structured interview was informed by the literature on attrition and previous quantitative data collection. Subsequently, the ‘hows’ and ‘whys’ are inductively generated from the interview data (Al-Busaidi, 2008). As far as possible, quality standards were applied to this qualitative study in accordance with the Standards for Reporting Qualitative Research (O’Brien, Harris, Beckman, Reed, & Cook, 2014). Semi-structured interviews are also flexible and allow rich data to be obtained such as the views of and descriptions by participants that may reveal unexpected issues or concerns (Pope, van Royen, & Baker, 2002).

B. Context

At NUS, we observed an anecdotal decline in biomedical sciences graduate students and early career biomedical scientists, with those remaining in the field often facing uncertain, poor or mismatched employability. We also observed an anecdotal concomitant decline in motivation and direction amongst graduate students and early career scientists with respect to biomedical sciences research. Students tend to express the wish or desire to ‘just focus on their research and graduate’ and are observed to be increasingly less participative in research-or academic-related activities. This is comparable to the situation in other countries (Callaway, 2014; Fuhrmann et al., 2011; Gould, 2015; Powell, 2015; “There is life after academia,” 2014). Typical graduate programs take 3 to 4 years to complete with some graduate students extending for 6 months to a year. Graduate programs offer varying but competitive stipends, and can lead to a direct PhD or exit with Masters. Graduate curricula vary according to programs, schools and faculties. Typically, candidates admitted to the graduate programs will undertake research under the supervision of staff members from the school or faculty for their period of candidature, at the end of which, they submit a thesis that is examined by a Board of Examiners. In addition, they are required to fulfill a coursework component. Other perks such as conference allowance vary according to programs and student profiles. Training conditions for biomedical sciences graduate students and early career biomedical scientists vary according to labs and supervisors. There is no fixed training period for early career biomedical scientists. Majority of graduate students and early career scientists hail from Singapore and regional countries such as China, India and Malaysia.

C. Participants

Participants were purposively sampled (Tavakol & Sandars, 2014). The criteria of selection include biomedical sciences graduate students or early career biomedical scientists engaged in graduate programs or postdoctoral training in the above-mentioned schools and faculties in NUS who volunteered and consented to participate in the interview The composition of participants consisted of almost equal representation from both genders, with a mixture of Singaporeans and regional nationalities. For this study, data saturation was achieved after interviewing 10 participants. In other words, sufficient quality data has been collected to crystallise themes and sub-themes in our study, and that no new theme or sub-theme emerge from the data collected through further sampling of the tenth participant. At this point, no further sampling or data collection is required as the themes and sub-themes are rich and distinct enough for a framework or theory to be developed.

D. Analysis

The transcribed data was analysed using template analysis, along with the interviewers’ notes and memo writing where applicable (Brooks, McCluskey, Turley, & King, 2015). Different parts of the text were highlighted and indexed according to themes that were considered relevant and important. Themes were constructed through interpretation of the data. These themes helped to enrich or further shape the framework. During template analysis, codes were developed and collectively interpreted from the data as themes. Upon immersing and dwelling on the data, sub-themes were also interpreted from within these themes. Codes that did not contain useful data were discarded and codes with minimal data were combined (Brooks et al., 2015). As new themes were constructed, participants were recruited for further data collection until the 10th participant whereby no new major theme was identified after the interview. Trends and relationships were also be generated from the themes during analysis. Reflexivity was applied throughout the analysis to ensure a more effective and impartial analysis by considering the biases, assumptions and preconceptions that the interviewers and investigators might have during the course of the research. The analysed findings and interpretations were shared with 5 to 6 participants and about 20 to 30 peers in the form of informal meeting and presentation to seek their confirmation and further opinions.

III. RESULTS

Three major themes are important in influencing graduate students’ and trainees’ experience during their education and training. These are ‘Intrinsic Motivation’, ‘External Factors’ and ‘Psychological State of Mind’. However, the sub-themes that emerged from the main themes and the relationship among the main themes and sub-themes helped to expand the existing framework, provided greater clarity to understanding their motivation for doing biomedical sciences, and what enhances, sustains or kills their motivation (Figure 2).

A. Theme 1–Intrinsic Motivation

There are several reasons why some participants are intrinsically more motivated than others to commit to biomedical sciences training. These reasons are the intensity of their motivation, the foundations of their motivation, and the nature of their motivation.

Sub-theme 1–Intensity of Motivation: As mentioned below, an intrinsically strong motivation in coming to terms with the purpose of knowing the ‘why’ of what they are doing will pull students and trainees through difficult times. This high intensity intrinsic motivation sustained over a long period of time is required to overcome frequent challenges in biomedical sciences education and training, as it is often easier to give up than sustain in the field.

“If you don’t really have the right motivation in the first place, it is easy for you to just ‘let’s not do this anymore’… so I think you need to have strong motivation to do this in the first place. With whatever kind of work you are going to do, you need to have a reason that will pull you through the tough times, ya… if not it’s not going to help you last through the tough times.”–Participant 1552 (3^rd Year graduate student)

Sub-theme 2–Noble Purpose: In order for high intensity intrinsic motivation to be sustained over a long period of time, it needs to be fueled by a greater sense of purpose that is dependent on the character of the students and trainees, which is in turn, shaped by their previous experiences, intrinsic convictions, cultural and educational backgrounds. This helps students and trainees to be able to cope better, stay on course, keep the ‘vision’ and recover from setbacks during their rigorous education and training. As a result, these students and trainees are also more likely to feel positive, motivate themselves to perform better, and continue on the research path. An example of this is the desire to do good science with integrity and be a role model to the scientific community.

“I think… would be… integrity? Because I don’t think my project has actually much clinical value in the sense that there’s a lot of grey areas so we can’t really make any clear-cut conclusions, so what pushes me to… overcome these research challenges is my want to report honestly, because I feel when I read papers, sometimes… the findings can be exaggerated? Or not entirely true, in the sense that people normally report good things, they don’t really report bad things. So… if you’re asking me what motivates me to perform better as a researcher, it would be to… put the truth across, to not be afraid to tell people is this what I found, this is not great but this is how it is…”–Participant 1152 (2^nd Year Graduate Student)

Sub-theme 3–Types of Intrinsic Motivation: Participants mentioned several types of intrinsic motivation in biomedical sciences that they find rewarding and enjoyable. These deep-rooted desires motivate participants when pursuing their biomedical training. Firstly, the desire to be cognitively challenged promotes better training performance by contributing to the participants’ motivation to persevere despite repeated failures. Secondly, growing and sustaining the innate curiosity and passion to seek answers is an important element of intrinsic motivation for students and trainees to pursue biomedical research, which is largely dependent on educational and cultural upbringing.

“I don’t know if you heard of it, it’s like videogame theory, that you always have to make the next level slightly more stimulating but not that hard that the person will quit, but like hard enough that the person is willing to continue and it’s challenging enough that they won’t get bored.”–Participant 1440 (3^rd Year graduate student)

“I think you survive on one part is your passion, your interest generally for studying this field, another thing is you try to cultivate this kind of interest to keep up, to keep going…”–Participant 0855 (Graduate student)

Lastly, the ability to deal with uncertainty during biomedical sciences training brings an important balance to intrinsic motivation. A poor ability to cope with uncertainty is likely to chip away at intrinsic motivation. As research does not always produce successful outcomes, students and trainees have to be mentally prepared to accept a certain degree of unpredictability. A clear understanding and acceptance of this philosophy will help them to last the extra mile. Taken together, it shows that the nature and character of each individual’s intrinsic motivation is likely to determine perceived future performance and sustainability in biomedical sciences research.

“Maybe for some reasons, the experiment didn’t go very well so they don’t have much achievement, so they don’t feel these achievement…. They are probably feeling that it is more important to get something for them … like… very high chance to be recognised or get something done, however, it’s something unpredictable in the future, for people don’t like this uncertainty or don’t like these unpredictable stuff, they probably think ok… I guess it’s probably better to go to the company or something.”–Participant 0855 (Graduate student)

B. Theme 2–External Factors

In terms of external factors that may enhance or hinder motivation in biomedical training, which in turn, affects perceived performance and decision to remain in research career path, they can be classified broadly into social support, career advancement and research environment. These external factors together with intrinsic motivation can moderate each other and ultimately, impact the psychological state of students and trainees during their education and training.

Sub-theme 1–Social Support: For social support, the three main stakeholders are supervisors, peers and families. It is important for these stakeholders to create a strong support network that will enhance the likelihood of students and trainees staying on in biomedical sciences and doing well. Supervisors’ support plays an important aspect in motivating students and trainees to strive for excellence while providing that much-needed emotional and mental support. Supervisors require discipline knowledge in order to support them during times of discipline-associated difficulties as well as possess competencies in supervisory and mentorship skills. Conventionally, most would assume that supervisors are experts in the biomedical science field that their trainees are involved in as well as being skillful in supervision and mentorship. However, our study revealed that this might not be the case all the time.

“Right now I have no… minimal guidance. I’m not sure if it’s because… the area of neuroimaging we’re doing is new in NUS, so we don’t have many experts to guide, so I feel perhaps the university could provide a bit more guidance to us students. We have our Principle Investigators (PIs), but a lot of times he himself is not an expert in the field that he’s into.”–Participant 1152 (2^nd Year Graduate Student)

Interestingly, family is one of the most important stakeholders that influence students’ and trainees’ decisions to pursue, leave or adjust their respective studies or careers in biomedical sciences. This is not unsurprising as the Hofstede model shows that Asians tend to be collectivistic and inter-dependent on one another (Hofstede, 2011). This implies that Asian students and trainees are more likely to consult their family members or factor family considerations into their decisions. The impact or consequences that these decisions may have on family members is something that Asian students and trainees care very much about. This may be attributed to the economic systems and cultural philosophies that have been entrenched in Asian countries for generations, at least in part.

The final group of stakeholders who are crucial in this social support network are peers. Participants reiterated the importance of having peers who listen to and share their problems, which serve as a vital outlet for relieving stress and seeking solace in one another. Having supportive peers help students and trainees realise that they are not alone in the problems that they face during their education and training. As a result, it helps them to find strength and give them comfort to soldier on.

“It is good to have meetings with fellow grad students so that we can share experiences and then find some common… I mean it’s good to talk about things.”–Participant 1437 (2^nd Year graduate student)

Sub-theme 2–Career Advancement: A second external factor that influences students’ and trainees’ decisions to pursue and stay on in higher education or careers in biomedical sciences, is whether they perceived the field to be of value for their career advancement and alignment to their personal goals. More importantly, their striving for career advancement appears to be driven by what they perceived to be expected or required of them from society. However, this sub-theme is dependent on the family considerations of the students and trainees.

“If you want to continue like lecturer or being education, like academic in the tertiary level, so that’s another reason why I kind of committed to finishing my PhD, besides the research aspect. So I think that’s what drives me more sometimes, even when research doesn’t work out, or our experiments fail.”–Participant 1440 (3^rd Year graduate student)

“You have to go overseas do a post-doc before coming back to get a faculty position… I mean it’s being said among the graduate community. So there’s this unspoken rule and when it comes to that right, you have to be concern about family, whether you can suddenly just pluck your family from Singapore and move them over, because it’s not an individual decision. If you are single and your parents don’t mind then it’s fine but for me is my father, my parent is a single parent, my father is a single parent, so it’s not as easy to make that decision when it comes to pursuing academia.”–Participant 1002 (Graduate student)

Sub-theme 3–Research Environment: Participants also discussed extensively on the impact of their education and training environment to their motivation and desire to remain in the field and perceived likelihood of doing well. Some of the factors for a favourable environment include fostering a non-toxic workplace culture that is not unnecessarily hierarchical and having peripheral programmes aimed at developing complementary non-research skills and interests. This helps to protect the mental wellbeing of the students and trainees, which in turn, helps to grow and sustain their interest in the field.

“Their culture was to be more follow seniority, follow positions so… even though the researcher didn’t know what he was doing… That professor will only listen to his research fellow… the team is pushed apart because of this idea of hierarchy.”–Participant 1152 (2^nd Year Graduate Student)

“We spend a lot of time doing research and outside of research we don’t really want to do anything that is related to research. So I think in terms of… that could be more support programmes.”–Participant 1440 (3^rd Year graduate student)

C. Theme 3–Psychological State of Mind

Both intrinsic motivation and external factors have huge roles to play in moderating the influence from each other, which in turn, determine the psychological impact on students and trainees. This may influence their ability to do well in research and their decision to stay or leave biomedical sciences. Our study seems to suggest that the external factors, research environment and culture specifically, are responsible for creating a negative psychological impact on students and trainees by eliciting fear and stress.

Ideally, strong intrinsic motivation should be reinforced by positive external factors. However, strong intrinsic motivation can be weathered down by negative external factors. When the latter becomes overwhelming, it leads to a negative psychological state that not only affects the mental health of our students and trainees, causing burnout, undesirable behaviours and attitudes, and other dire consequences, but also leads them to seriously consider leaving biomedical sciences research for good.

“Failing is seen as a bad thing… the thing is biomedical research there is failure and you need time to fail to eventually succeed, but then how the academic structure is set up is that there’s no time for failure.”–Participant 1440 (3^rd Year graduate student)

“Because your fellow lab mates are not keen to talk these topics to you in a natural fashion, then it’s like talking to a wall right, you try to bounce off ideas, like you are trying to talk about it, then there’s no response. It’s a two-way thing.”–Participant 1552 (3^rd Year graduate student)

IV. DISCUSSION

In order to understand the motivation for performing high quality research as part of biomedical sciences graduate education and training, almost all participants mentioned that they enjoy being cognitively challenged and the intensity of this intrinsic motivation determines their sustainability in the field. This desire helps them to broaden their creative thinking and sustain their determination to succeed in the presence of failure.

Our study also showed that participants are unclear about the training they are supposed to receive and uncertain about what constitutes towards their education. This worsens the current training environment that is pressurising, disorganised, narrow in learning objectives, and centred on a fear of failure. Fear from the repercussions of failure due to a ‘pressure cooker’ research environment that celebrates positive results with little tolerance for negative ones can lead to unnecessary stress, lowering of self-esteem, diminishing of self-worth, and reinforcing the notion that one is not good enough. Unresponsive peers and superficial relationships with peers due to a competitive environment can exacerbate the situation, leading to a feeling of helplessness. Therefore, it is important for us to rethink and redefine the yardsticks that measure research success and be more accepting of failure. Supervisors should also be supported with programs that develop mentorship skills. Graduate programs and early career training should be more structured and comprehensive with clear learning outcomes.

For such attitudes and restructuring to happen, a paradigm shift in mindset is needed, which must percolate beyond supervisors to senior academic and research administrators as the latter determine the reward and recognition policies that shape the behaviours and attitudes of the former. Only then, will different stakeholders come together and form a partnership in order to reform and restructure the curriculum (Dasgupta, Symes, & Hyman, 2015).

When it comes to why some trainees are more intrinsically motivated to commit to research than others, our study showed that trainees who are exposed or influenced early in biomedical sciences; have a certain topic in mind; and are driven by curiosity, tend to be more committed in research than those without these attributes. More encouragingly, our study also highlighted several ‘right’ reasons as the intrinsic motivation to pursue research. These include the desire to help people through research such as finding cures to cancers and neurodegenerative diseases as well as uphold research integrity in biomedical research. Thus, it is important for students and trainees to ‘see the big picture’ of their research and that they do not lose sight of this ‘big picture’ even as it morphs and shape-shifts throughout their journey. This needs to be built into the graduate education and early career training system so as to provide them with a sustained sense of purpose and the inspiration to succeed in order to stem the flow of graduates leaving biomedical sciences research.

Not surprisingly, supervisors and peers are important people who can influence students and trainees to do better and remain in research, either directly or indirectly by altering the research environment (Kemp, Lazarus, Perron, Hanage, & Chapman, 2014). In fact, supervisors appeared to create a greater impact than peers. Thus, it is important for supervisors to be competent not only in their field of expertise but also in mentoring, communication, pedagogy and human management. As supervisors are also role models, they often determine the mindsets, attitudes and behaviours of their students and trainees, which can either be further enhanced or damaged by peers. Perhaps less expected was the important role that family plays in determining whether students and trainees enter, remain, press on, slow down or leave biomedical sciences research. In one instance, pursuing biomedical sciences research is perceived as a ticket to move the family up the social mobility ladder through education. In other instances, the desire to spend quality time with the family, set up a young family or take up family responsibilities in terms of caregiving or financial situations, can lead one to leave biomedical sciences research. Family encouragement or discouragement can also greatly influence their decisions to remain or leave biomedical sciences research. Moreover, females tend to experience a greater burden to choose between staying on or leaving research careers for family reasons such as getting married or caring for young children (Lodish, 2015). In order for students and trainees to focus and perform better in research and subsequently, remain in biomedical sciences, it is important that they feel supported in this area and not have to resolve the tension between research and family by themselves. Establishing clear expectations, flexible timelines and defined educational goals, coupled with a supportive environment and social network, may help to overcome this issue.

Other than overcoming a culture of fear-of-failure and transforming expectation pressure into high performance, future training programs need to incorporate mental health support to ensure the well-being of biomedical sciences graduate students and early career biomedical scientists (Evans, Bira, Gastelum, Weiss, & Vanderford, 2018). In line with literature, our study suggested an increasing likelihood of students and trainees’ suffering from burnout and mental health issues as well as becoming ‘unhinged’ from society-at-large in terms of social behaviour (Evans et al., 2018). In one case, a participant who declared to suffer from depression was determined to remain in research so as to help people through science. Referencing the framework, this may suggest that intrinsic motivation to stay in research and extrinsic ‘push’ factors to leave research are not mutually exclusive.

The issues confronting biomedical sciences research are not unique to Singapore. These include hyper competition in biomedical sciences that discourages risk-taking behaviour, removes time for creative thinking and turns away even the most promising students from the field; and an increasingly protracted and demanding path to career progression (Alberts, Kirschner, Tilghman, & Varmus, 2014, 2015). This is further exacerbated by the lack of funding and available positions generally experienced around the world (Kimble et al., 2015). However, most of these studies do not take into account how intrinsic motivation and external factors may moderate the experience imposed by these conditions. Furthermore, students and trainees may even leave the field or perceive themselves to be lacking in research ability independent of actual achievements, and competition for funding and positions, suggesting a role for intrinsic motivation and external factors (Roach & Sauermann, 2017).

Taken together, our study revealed the mechanisms on how enjoying cognitive activities may prime trainees for better research performance, resulting in more sustainable research careers (Figure 2). The study also unraveled deeper insights on the basis for students and trainees’ intrinsic motivation, the external factors and their inter-relatedness, which may influence their motivation to press on or stay on in biomedical sciences research (Figure 2).

Figure 2. Diagram shows the relationships among the main themes and sub-themes, and their impact on psychological state of mind, research performance and sustainability

Our study emphasised the need for significant changes in graduate education and early career training in order to future-proof and career-proof trainees in an ever-changing employment landscape whereby disruptive technology is creating new jobs as fast as they are destroying existing ones. These changes cannot be token adjustments. They require authentic and transformational leadership to tune in to the needs and aspirations of the students and trainees, match them to Singapore’s needs of tomorrow, reduce redundancies and take care of human sensitivities. Some of the key changes include having more structured graduate and early training programs so that students and trainees are aware of how they are going to be taught, what they will learn, and how they can actively contribute towards their learning with clear objectives and end goals in mind (Bosch, 2018; Dasgupta et al., 2015). Secondly, it is time to put the ‘philosophy’ back into the ‘Doctor of Philosophy’ (Bosch, 2018). Currently, many lament that their intense training goes too deep, leaving them with little time and opportunity for other core skills such as communication and management (Bosch & Casadevall, 2017).

Hence, we need to pause, reflect, soul-search, articulate and redefine what graduate education and early career training in biomedical sciences is really all about (Bosch, 2018). There are many worldviews concerning higher education and training but we need to get back to basics, which are for graduate education and early training to be broad, holistic and interdisciplinary to create thinkers, not just doers (Dasgupta et al., 2015; Lorsch & Nichols, 2011). Not only will this create biomedical scientists who are able to think more critically and thoughtfully, it will also enhance the retention rate of graduates in biomedical sciences research.

V. CONCLUSION

Our research will enable policy and decision makers to customise biomedical sciences graduate education and early career training programs that are fit for purpose. This may include providing closer support and proper guidance; developing clear learning objectives, processes and outcomes and following them; introducing education and training components to meet diverse interests; creating opportunities for and empowering students and trainees in ways that will help them to meet their education expectations and career aspirations.

Therefore, our study has provided a deeper and greater dimension as to why biomedical sciences graduate students and early career scientists are really lacking in motivation and leaving the field beyond funding and job availability issues. Given its global nature, we hope our findings will be useful for consideration by institutions around the world, which are offering or setting up graduate education and early career training programs in biomedical sciences especially for Asian institutions. This impetus to extend the call-for-action to other Asian institutions is strengthened by the fact that some of our participants hailed from major Asian countries such as China and India. Their contribution to our overall findings will be as highly relevant for other Asian institutions as it is for Singapore.

Notes on Contributors

Dr Zhi Xiong Chen, PhD, MHPE, is the Deputy Education Director from the Department of Physiology and the Assistant Dean of Students at NUS. He is also the Integration Lead Educator in undergraduate medical curriculum and a Joint Scientist at KKH. His research interests include health professions education.

Dr Shuh Shing Lee, PhD, is a medical educationalist in CenMED, National University of Singapore. She is actively involved in education research. Her research interests include teaching and learning approaches and technology, student learning and qualitative research.

Dr Dujeepa D. Samarasekera, MBBS, MHPE, FAMS, FAcadMEd, FAMEE, is the Director of CenMED, National University of Singapore and the President of the Western Pacific Regional Association of Medical Education. He also consults for the Singapore Ministry of Health on medical education matters.

Dr Renee Stalmeijer, PhD, is an assistant professor at the School of Health Professions Education, Maastricht University. She is trained in educational sciences and medical education. Her research interests include workplace learning and supervision, qualitative research methodology and quality assurance.

Ethical Approval

This research received approval from the NUS-Institutional Review Board (NUS-IRB; Ref No. B-15-256). Consent was obtained from participants for the research study.

Acknowledgement

We would like to thank CenMED for the support in helping to recruit participants, conduct the interviews and collating the audio recordings for the study.

Funding

No funding is involved in this paper.

Declaration of Interest

The authors declare that they have no competing interests.

References

Al-Busaidi, Z. Q. (2008). Qualitative research and its uses in health care. Sultan Qaboos University Medical Journal, 8(1), 11-19.

Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2014). Rescuing US biomedical research from its systemic flaws. Proceedings of the National Academy of Sciences, 111(16), 5773-5777. https://doi.org/10.1073/pnas.1404402111

Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2015). Opinion: Addressing systemic problems in the biomedical research enterprise. Proceedings of the National Academy of Sciences, 112(7), 1912-1913. https://doi.org/10.1073/pnas.1500969112

Bandiera, G., Boucher, A., Neville, A., Kuper, A., & Hodges, B. (2013). Integration and timing of basic and clinical sciences education. Medical Teacher, 35(5), 381-387. https://doi.org/10.3109/0142159X.2013.769674

Bosch, G. (2018). Train PhD students to be thinkers not just specialists. Nature, 554(7692), 277. https://doi.org/10.1038/d41586-018-01853-1

Bosch, G., & Casadevall, A. (2017). Graduate biomedical science education needs a new philosophy. mBio, 8(6). https://doi.org/10.1128/mBio.01539-17

Bourne, H. R. (2013). A fair deal for PhD students and postdocs. Elife, 2, e01139. https://doi.org/10.7554/eLife.01139

Brooks, J., McCluskey, S., Turley, E., & King, N. (2015). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology, 12(2), 202-222. https://doi.org/10.1080/14780887.2014.955224

Brown, E. R., Smith, J. L., Thoman, D. B., Allen, J. M., & Muragishi, G. (2015). From bench to bedside: A communal utility value intervention to enhance students’ biomedical science motivation. Journal of Educational Psychology, 107(4), 1116-1135. https://doi.org/10.1037/edu0000033

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42(1), 116-131. https://doi.org/10.1037/0022-3514.42.1.116

Callaway, E. (2014). Life outside the lab: The ones who got away. Nature, 513(7516), 20-22. https://doi.org/10.1038/513020a

Dasgupta, S., Symes, K., & Hyman, L. (2015). Leading change: Curriculum reform in graduate education in the biomedical sciences. Biochemistry and Molecular Biology Education, 43(2), 126-132. https://doi.org/10.1002/bmb.20862

Evans, T. M., Bira, L., Gastelum, J. B., Weiss, L. T., & Vanderford, N. L. (2018). Evidence for a mental health crisis in graduate education. Nature Biotechnology, 36(3), 282-284. https://doi.org/10.1038/nbt.4089

Fuhrmann, C. N., Halme, D. G., O’Sullivan, P. S., & Lindstaedt, B. (2011). Improving graduate education to support a branching career pipeline: Recommendations based on a survey of doctoral students in the basic biomedical sciences. CBE—Life Sciences Education, 10(3), 239-249. https://doi.org/10.1187/cbe.11-02-0013

Gould, J. (2015). How to build a better PhD. Nature, 528(7580), 22-25. https://doi.org/10.1038/528022a

Hofstede, G. (2011). Dimensionalizing cultures: The Hofstede model in context. Online Readings in Psychology and Culture, 2(1) https://doi.org/10.9707/2307-0919.1014

Kemp, M. W., Lazarus, B. M., Perron, G. G., Hanage, W. P., & Chapman, E. (2014). Biomedical Ph.D. students enrolled in two elite universities in the United Kingdom and the United States report adopting multiple learning relationships. PLoS One, 9(7), e103075. https://doi.org/10.1371/journal.pone.0103075

Kimble, J., Bement, W. M., Chang, Q., Cox, B. L., Drinkwater, N. R., Gourse, R. L., … Seidel, H. S. (2015). Strategies from UW-Madison for rescuing biomedical research in the US. Elife, 4, e09305. https://doi.org/10.7554/eLife.09305

Koestner, R., & Losier, G. F. (2002). Distinguishing three ways of being highly motivated: A closer look at introjection, identification, and intrinsic motivation. In E. L. Deci & R. M. Ryan (Eds.), Handbook of self-determination research (pp. 101-121). Rochester, NY: University of Rochester Press.

Lodish, H. F. (2015). Accommodating family life: Mentoring future female faculty members. Trends in Cell Biology, 25(3), 109-111. https://doi.org/10.1016/j.tcb.2014.12.007

Lopatto, D., & Williams, P. (2007). Undergraduate research experiences support science career decisions and active learning. CBE—Life Sciences Education, 6(4), 297-306. https://doi.org/10.1187/cbe.07-06-0039

Lorsch, J. R., & Nichols, D. G. (2011). Organizing graduate life sciences education around nodes and connections. Cell, 146(4), 506-509. https://doi.org/10.1016/j.cell.2011.08.004

Moses III, H., & Martin, J. B. (2011). Biomedical research and health advances. The New England Journal of Medicine, 364(6), 567-571. https://doi.org/10.1056/NEJMsb1007634

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89(9), 1245-1251. https://doi.org/10.1097/ACM.0000000000000388

Pawlina, W. (2009). Basic sciences in medical education: Why? How? When? Where? Medical Teacher, 31(9), 787-789.

Pope, C., van Royen, P., & Baker, R. (2002). Qualitative methods in research on healthcare quality. BMJ Quality & Safety, 11(2), 148-152.

Powell, K. (2015). The future of the postdoc. Nature, 520(7546), 144-147. https://doi.org/10.1038/520144a

Roach, M., & Sauermann, H. (2010). A taste for science? PhD scientists’ academic orientation and self-selection into research careers in industry. Research Policy, 39(3), 422-434. https://doi.org/10.1016/j.respol.2010.01.004

Roach, M., & Sauermann, H. (2017). The declining interest in an academic career. PLoS One, 12(9), e0184130. https://doi.org/10.1371/journal.pone.0184130

Rockey, S. J. (2014). Mentorship matters for the biomedical workforce. Nature Medicine, 20(6), 575. https://doi.org/10.1038/nm0614-575

Roets, A., Kruglanski, A. W., Kossowska, M., Pierro, A., & Hong, Y. Y. (2015). The motivated gatekeeper of our minds: New directions in need for closure theory and research. Advances in Experimental Social Psychology, 52, 221-283. https://doi.org/10.1016/bs.aesp.2015.01.001

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78. https://doi.org/10.1037110003-066X.55.1.68

Sullivan, G. M., & Sargeant, J. (2011). Qualities of qualitative research: Part I. Journal of Graduate Medical Education, 3(4), 449-452. https://doi.org/10.4300/JGME-D-11-00221.1

Tavakol, M., & Sandars, J. (2014). Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part II. Medical Teacher, 36(10), 838-848. https://doi.org/10.3109/0142159X.2014.915297

There is life after academia. (2014). Nature, 513(7516), 5. https://doi.org/10.1038/513005a

*Chen Zhi Xiong
Department of Physiology,
Yong Loo Lin School of Medicine,
National University of Singapore,
MD9, 2 Medical Drive,
Singapore 117593
Tel: +65 6516 3231
Email: zhixiong_chen@nus.edu.sg

Submitted: 31 July 2019
Accepted: 6 October 2019
Published online: 1 September, TAPS 2020, 5(3), 5-9
https://doi.org/10.29060/TAPS.2020-5-3/GP2178

Rachmadya Nur Hidayah¹, Richard Fuller² & Trudie E. Roberts³

¹Department of Medical Education and Bioethics, Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada, Indonesia; ²School of Medicine, Faculty of Health and Life Sciences, University of Liverpool, United Kingdom; ³Leeds Institute of Medical Education, Faculty of Medicine and Health, University of Leeds, United Kingdom

Abstract

Southeast Asian region has been experiencing significant challenges to their health care systems, driving some countries establishing policies to ensure the quality of the medical workforce through the introduction of national licensing examinations. This paper examines the experiences of Southeast Asian countries in implementing licensing examinations, highlighting the changes and development of the education policy and its impact on medical education. The lesson learnt from the high-stakes assessment policy offers a new perspective on how medical licensing examinations could represent a potential regional approach for a better health care.

Practice Highlights

• ASEAN countries have been implementing national licensing examinations to ensure the quality of health care education and practice, while at the same time, anticipate the increasing migration of health care professionals in the region.
• The consequences of medical licensing examinations on education varied between countries and regions implementing the policy. The development of National Licensing Examinations in Southeast Asia region has been driving changes in education as its consequences. Looking at the impact, there is a possibility that this policy will lead to greater impact on health care, which may be different than the consequences of similar policy in the Northern America and European context.
• The experience of ASEAN countries such as Indonesia, Thailand, and Lao, offers an insight on how the licensing examination policy drives improvement in medical education through collaboration between stakeholders. This effort should work alongside the strengthening of accreditation system for medical education. The improvement is necessary to support health care policy and produce better quality of medical and health care professionals.

I. BACKGROUND

Healthcare in Southeast Asia faces a major challenge–that of delivering high quality, safe care to one of the most populated areas in the world, alongside rapid regional development. The trend in disease burden is shifting to non-communicable diseases, along with the challenge of globalisation which contributes to the migration of patients and health care professionals. It is known that the migration from low- and middle- income countries, or from developing to developed countries, is more common than the opposite (Tangcharoensathien et al., 2018). The migration of health care workforce has been followed by the international movement in medical education (e.g. offshore medical schools), which have created global-local tensions in medical education. In the interest of ensuring the quality of care and patient safety, the WHO through its Global Code of Practice on the International Recruitment of Health Personnel, recommended the implementation of mandatory national licensing examinations (NLEs) by regulatory bodies have been viewed as supporting this high-stakes education policy, which was in line with the increasing practice of licensing examinations in Southeast Asia countries (Tangcharoensathien et al., 2018).

II. NATIONAL LICENSING EXAMINATIONS: LESSONS FROM NORTH AMERICA AND EUROPE

The policy of using licensing examinations for quality assurance of graduates (and arguably supporting better health care outcomes) has been used in other parts of the world. The United States Medical Licensing Examination (USMLE) and the Medical Council of Canada Qualifying Examination (MCCQE) are the most notable examples. Although firstly established as an effort to reduce variation among practitioners, the USMLE has been developed to an assessment system for international medical graduates as well as home graduates who wish to be licensed in the US or Canada. Both the USMLE and MCCQE went through changes in the administration; from essay and oral examination into a more systematic MCQ and clinical skills assessment in the last two decades. Licensure’s focus shifted into educational outcomes, projecting of continuing medical education and revalidation, the use of technology and psychometrics. NLEs in the Northern America are viewed as effort on ensuring the quality of both home medical graduates and migrating health care professionals.

The impact of NLEs in both countries have been a topic for research in this area. Changes in medical schools’ curricula, especially after the implementation of clinical skills assessment, were reported. Better performance in NLEs have been associated with better preventive and management in primary care; fewer patient complaints; better patient care for international graduates and better performance in postgraduate assessment. However, there were no evidence showing direct impact of NLEs to patients, which raises debate on whether the NLEs ensure patient safety. Despite the debate, the NLEs in both countries have become cornerstones of educational quality assurance system, which also relies on the accreditation system of medical schools and continuing professional development.

European countries have different approaches to the issue of health care professionals’ migration and quality assurance. As European Union (EU) member countries recognise the qualification of health care professionals graduating from other EU countries, there are equal opportunities for them to work within the EU, particularly for medical doctors. Although there are examinations for foreign graduates, the focus is more on language competency rather than medical knowledge and skills. Whilst there are concerns about the diversity of curricula among EU countries, many experts believed that the accreditation systems were sufficient to assure the quality of their graduates (Archer et. al, 2017). Despite this, countries such as Switzerland, Poland, and Germany, have established medical licensing examinations for their home graduates and non-European Economic Area (EEA) countries’ graduates. In 2015, the General Medical Council in the United Kingdom followed this step and plans to start their medical licensing assessment (MLA) (Archer et al., 2017) which the GMC will put into action in 2023. The MLA is designed to be embedded in medical schools’ assessment, which means that there will be consequences in education that could be anticipated in the future.

The two regions might have different approach in quality assuring the graduates and medical practice depending on how the education and healthcare system work in the regions. However, it is worth to notice that the policy, along with accreditation systems, could change the practice of education and possibly the health care system.

III. NATIONAL EXAMINATIONS IN ASEAN: THE POSSIBILITY OF REGIONAL APPROACH FOR BETTER CARE?

Although the approaches taken by the EU countries and ASEAN countries are similar, it is worth noting that the context of patient management policies in both settings are different, as well as their education systems. As they work towards the ASEAN Vision 2020 on Partnership in Dynamic Development, ASEAN countries agreed a number of frameworks to enable the free flow of services for professionals in health care. The mutual recognition arrangements (MRA) for nurses, medical and dental practitioners in ASEAN countries were created in 2006-2009 in response to the partnership. This arrangement required countries to establish regulations for migration of health care professionals. The regulations came in the form of a licensing examination, which have subsequently shaped health care professions education policies in a number of member states.

For these countries, introducing national licensing examinations was not solely about the need to regulate foreign health care professionals, but also to improve the quality of graduates. Most of ASEAN member states are developing countries sharing similar challenges: inadequate numbers of physicians, uneven distribution of health care professionals, and a developing education and health care system (Kittrakulrat, Jongjatuporn, Jurjai, Jarupanich, & Pongpirul, 2014; Sonoda et al., 2017; Tangcharoensathien et al., 2018). The challenges are often followed by the increasing number of medical schools to produce more doctors –as in Indonesia’s case, without a rigid quality assurance system, raising concern of lower quality of graduates. As these countries lacked regulation of health care professionals through existing licensing and re-licensing policies, they started to introduce the NLE as a way of using education and assessment to achieve better health care globally.

Four out of ten ASEAN member countries have implemented NLEs to date: Thailand, Philippines, Indonesia, and Malaysia. Each country has different reasons for introducing the policy. Thailand and Indonesia introduced the NLE for home and international graduates in 2006-2007, to ensure the quality of competent doctors and to improve the quality of their medical schools. In Indonesia, there has been an increasing number of medical schools in the last decade, leading to concerns of differential quality of education delivered which impact on the Government’s policy to improve the quality of health professions education. Philippines and Malaysia, both had NLEs for international graduates only, whose number has been increasing in the recent years. Even though the other countries in the South East Asia region currently do not have NLEs, the topic is being discussed amongst policy makers (Kittrakulrat et al., 2014; Sonoda et al., 2017). Vietnam and Lao are in the process of developing NLEs, and India is considering the NLE as part of their policy in ensuring international graduates are prepared to work in its healthcare system. The increasing trend of health care professions’ migration (Tangcharoensathien et al., 2018) means that the NLE is a relevant issue aligned with the idea of ASEAN Economic Community (AEC), where medical professions can be qualified to practice medicine in another country in this region (Kittrakulrat et al., 2014). For a comparison, other countries in Asia such as Taiwan and South Korea have been establishing medical/ health care professionals licensing examinations, which showed impact on medical education.

Although ASEAN countries have been implementing NLEs to address the influx of international graduates, the fact that the examinations are mostly delivered in local languange makes it complicated for a regional approach of licensing examinations in the era of AEC. Similarly, the difference in culture, economy, education and health care system, also poses another challenge for the free-flow of health care professionals in AEC. Variation of methods and types of licensing examinations exist in countries implementing NLEs; although the knowledge and clinical skills are the most common outcome to be assessed (Price et al., 2018). Even so, the current practice of NLEs might open opportunities for finding common and equal standard for patient care and medical education in ASEAN.

IV. LESSON LEARNT FROM MEDICAL LICENSING EXAMINATIONS IN ASEAN: INVESTING IN HEALTH CARE PROFESSIONALS’ EDUCATION

The development of the NLE in developing countries such as ASEAN members offers a pathway for improving medical and to further extent, health care professionals’ education systems. For these countries, even though the number of overseas qualified doctors may not be significant compared to domestic graduates, the policy was considered an important step towards producing competent health care professionals. It is about how a country moves forward by changing policies, improving education system, and aiming to produce better health care providers.

The policy to improve the quality of health professionals is in line with WHO strategy to invest in transforming the education and training of human resources in health. The transformational strategy emphasised by WHO relies on collaborative work between higher education and the health sectors through establishing national policies to secure the education standard and funding. Therefore, for ASEAN countries implementing the NLE, it represents an investment for health education improvement, and a means to support their health care systems in the future.

Lao’s experience in developing the NLE showed how the country changed its policy and strengthened their education system (Sonoda et al., 2017). In the process of establishing the NLE, Lao needed to strengthen their education system through policies and working with their stakeholders. In Lao’s case, these efforts were considered as significant moves toward better healthcare. The government took note that to produce competent health care professionals, they needed to improve the education system which made them revise educational curricula and assessment practices. The example from Lao’s experience showed that the introduction of NLE drives improvement in health care professions education which is necessary to produce competent professionals (Sonoda et al., 2017). In Thailand, the licensing examination drove changes in undergraduate education (e.g. recognising at-risk students) and postgraduate education. While in Singapore, the national examinations have been introduced in postgraduate studies for ensuring quality assurance (Ruth et al., 2018).

The extent of the NLEs’ roles in bringing health care improvement in ASEAN countries is potentially greater still. In Indonesia, which has the largest population in ASEAN, the NLE for medical graduates was part of government strategies to empower human resources for health and improve the quality of health care professionals. As reported by the Joint Committee on Medical Doctor Competence examination in 2013, the examination was designed to lever the quality of medical education, including medical schools, in the long run. Emergent work from Indonesia (Hidayah, 2018) highlights that the NLEs led to a number of positive consequences for medical education:  improvement in educational practices, improvement of learning resources and facilities, and collaboration between education institutions and health care providers. Since its implementation in 2007, other health care professionals such as dentistry, nursing, and midwifery have been developing, along with the strengthening of accreditation system. The collaboration between stake holders and the culture of Indonesia led to the collective work in scaling up the education. Hidayah’s work (2018) revealed that medical schools have been upgrading their education quality to produce competent graduates. The effort came in the form of increasing collaboration for curriculum and assessment development, expanding clinical placements, and collaboration for faculty development. For example, there have been regional collaboration for assessment practice between medical schools as reported by the National Committee in Indonesia. Especially for new and private schools, this collaboration helps them in leveraging their educational practice. Research highlights that in the context of developing medical education such as Indonesia, the NLE acts as a catalyst to drive improvement, linking the vital role of the regulator (i.e. the government) and major stakeholders (e.g. association of medical schools) (Hidayah, 2018). In the long term, this improvement in the quality of education is expected to bring improvement in health care by ensuring the quality of medical education and medical graduates.

In the broader field of health professions education, national licensing examinations remain a hotly debated policy (Archer et al., 2017). However, the experience of ASEAN countries in implementing NLEs reveals opportunities for a bigger role for this high-stakes assessment in the quality assurance system of health care professionals in order to enhance the quality of health care. In the future, lessons learnt from how an education policy can support developing countries in improving their health care quality, has the potential to enhance the approach to health care problems globally.

Notes on Contributors

Rachmadya Nur Hidayah conducted the research as her PhD project, and its results were used in this article. She conducted literature search, designed the concept of this review and write the manuscript.

Richard Fuller supervised the research project, contributed to the concept design of the manuscript and revised the draft of the manuscript.

Trudie E. Roberts supervised the research project, contributed to the concept design of the manuscript and revised the draft of the manuscript.

Ethical Approval

The authors sincerely thank Rebecca O’Rourke for her helpful feedback for the manuscript.

Acknowledgements

The authors sincerely thank Rebecca O’Rourke for her helpful feedback for the manuscript.

Funding

The work by Hidayah cited in this study was a doctoral study, fully funded by the Indonesia Endowment Fund for Education (LPDP – Lembaga Pengelola Dana Pendidikan), the Ministry of Finance, Republic of Indonesia.

Declaration of Interest

Although the study referred in this manuscript was a doctoral project funded by the government of Indonesia, the authors declare that there is no conflict interest related to the discussion presented in the manuscript.

References

Archer, J., Lynn, N., Coombes, L., Roberts, M., Gale, T., & Regan de Bere, S. (2017). The medical licensing examination debate. Regulation & Governance, 11(3), 315-322. https://doi.org/10.1111/rego.12118

Hidayah, R. N. (2018). Impact of the national medical licensing examination in Indonesia: Perspectives from students, teachers, and medical schools (Doctoral dissertation, University of Leeds). Retrieved from http://etheses.whiterose.ac.uk/20215/

Kittrakulrat, J., Jongjatuporn, W., Jurjai, R., Jarupanich, N., & Pongpirul, K. (2014). The ASEAN economic community and medical qualification. Global Health Action, 7, 24535. https://doi.org/10.3402/gha.v7.24535

Price, T., Lynn, N., Coombes, L., Roberts, M., Gale, T., de Bere, S. R., & Archer, J. (2018). The international landscape of medical licensing examinations: A typology derived from a systematic review. International journal of health policy and management, 7(9), 782. https://doi.org/10.15171/ijhpm.2018.32

Lim, M. C. R., Tsou, Y. K. K., Ong, C. P., Wong, K. W. S., Tan, C. S. G., Soon, S. W. W., … & Cruz, M. S. P. (2018). Family medicine post-graduate examination reforms in Singapore: A value-driven practical and contextual approach. The Asia Pacific Scholar, 3(3), 39-42. https://doi.org/10.29060/TAPS.2018-3-3/SC1057

Sonoda, M., Syhavong, B., Vongsamphanh, C., Phoutsavath, P., Inthapanith, P., Rotem, A., & Fujita, N. (2017). The evolution of the national licensing system of health care professionals: a qualitative descriptive case study in Lao People’s Democratic Republic. Human Resources for Health, 15(51), 1-13. https://doi.org/10.1186/s12960-017-0215-2

Tangcharoensathien, V., Travis, P., Tancarino, A. S., Sawaengdee, K., Chhoedon, Y., Hassan, S., & Pudpong, N. (2018). Managing in-and out-migration of health workforce in selected countries in South East Asia region. International journal of health policy and management, 7(2), 137. https://doi.org/10.15171/ijhpm.2017.49

*Rachmadya Nur Hidayah
Jl.Farmako Sekip Utara,Yogyakarta 55281
Faculty of Medicine, Public Health, and Nursing
Universitas Gadjah Mada, Indonesia
Email: r.nurhidayah@ugm.ac.id

Submitted: 30 May 2019
Accepted: 20 June 2019
Published online: 1 September, TAPS 2020, 5(3), 1-4
https://doi.org/10.29060/TAPS.2020-5-3/PV2164

Bonny L. Dickinson¹, Zhi Xiong Chen² & Aviad Haramati³

¹Department of Biomedical Sciences, Western Michigan University Homer Stryker M.D. School of Medicine, Kalamazoo, Michigan, USA; ²Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ³Centre for Innovation and Leadership in Education, Georgetown University School of Medicine, Washington, DC, USA

Abstract

Medical science educators are faculty members primarily responsible for teaching the medical sciences to healthcare profession students. These educators also have roles in other academic areas such as curriculum development, learner assessment, advising and mentoring, clinical duties, research, institutional service, leadership and administration. Academic institutions worldwide are increasingly focused on excelling in cutting-edge research, a major criterion for university rankings, which has led to significant resources and attention invested in those endeavours. At the same time, the primary goal of academic institutions is to educate and train healthcare professionals. As a result, medical science educators are often caught in the middle of these competing interests, leading to the ambiguity between personal career development and institutional priorities. It is in this context that we consider how medical science educators might navigate these issues and how academic institutions can support and strengthen this important cohort of faculty. First, with an attempt at humour, we begin by considering three stereotypical identities of medical science educators. We then discuss how the growing number of medical education associations and programs designed to support medical science educators are making a difference. Our goal is to provide a clear direction for the current career trajectory of medical science educators.

Keywords:        Recognition, Promotion, Innovation and Leadership in Education, Teaching Academy

Practice Highlights

• Medical science educators are critical to the mission of academic institutions.
• Institutions need to adopt guidelines to recognise, reward and promote medical science educators.
• The formation of a vibrant educator community to drive innovation and scholarship is important.
• Establishing a center for innovation and leadership can support and develop faculty in education.

I. THE MULTIPLE IDENTITIES OF THE MEDICAL SCIENCE EDUCATOR

Medical science educators have many roles, the most important of which is to educate students in the health professions. The many roles, responsibilities and expectations placed on these educators can shape distinct personalities or identities. Here, in a tongue-in-cheek fashion, we describe three stereotypical educator identities. The first is the ‘homemaker’. These educators engage in multiple routine tasks that few faculty enjoy (e.g., writing examination questions, directing courses, learner assessment, etc.). The ‘homemaker’ educators are usually in managerial roles, overworked, under-appreciated and often frustrated with the lack of time available to pursue scholarship or other rewarding endeavours. ‘Homemaker’ educators perceive them-selves as lacking the liberty or empowerment to actually embody their career aspirations.

The second stereotypical personality is the ‘punch clock employee’. These educators also often perform routine tasks, do what they are told, and rarely desire to excel beyond what they are expected to do. They perceive themselves as trapped in a cycle driven by their routines and, as a result, lack the motivation and aspiration to grow and develop as scholars.

The third identity is the ‘talk show host’ medical science educator. This identity is difficult to distinguish from those educators who are well versed in pedagogy and genuinely student-centric. The ‘talk show host’ educators enjoy teaching and may even be well-liked by students. This can, however, be borne out of a narcissistic need to be popular, in lieu of adopting a sincere concern for students’ learning. Their teaching approaches may also come across as self-absorbed and gimmicky, bordering on showboating. The ‘talk show host’ educator believes that what is important is what the teacher does, not what the students do (Biggs, 1999). Perhaps not surprising, ‘talk show host’ educators tend to accumulate teaching awards and steadily advance in their careers but lack a serious scholarship to accompany their teaching accolades.  This makes it difficult to distinguish them from educators with a genuine interest in students’ learning who insist on using evidence-based teaching techniques that may be unpopular with students, yet proven to support and optimise learning effectively.

What then is the ideal identity for medical science educators? We propose the ‘servant’ educator identity to describe educators who are genuinely interested in what the student does and how the student learns (Biggs, 1999).  How can this ‘servant’ educator identity be supported? We propose that two ‘I’s must play a role: individual and institution. At the individual level, the one question that all medical science educators should ask themselves is, “What is the reason that you have chosen to be an educator?” If the answer is to inspire and educate the next generation of health professionals, then the educator is on the right track. To support this goal, institutions must foster an environment that supports educator development to help them to discover, grow and achieve their career aspirations. This requires the institution to identify, encourage, recognise, reward, and promote educators who possess such attitudes (Bligh & Brice, 2009). Finally, institutions should aim to focus on the long-term vision instead of short-term gains, ‘heartware’ instead of hardware, and to support medical science educators and their work. Below, we discuss two ways that institutions can help nurture the ‘servant’ medical science educator.

II. SUPPORTING THE PROMOTION OF MEDICAL SCIENCE EDUCATORS

Medical schools have a long history of using well-established guidelines for granting promotion and tenure to faculty based largely on the number and quality of peer-reviewed publications and a track record of grant support for biomedical research. The recognition that these standards do not translate to the evaluation of faculty with predominant roles in teaching (both in the classroom and the clinic) and medical education scholarship has launched a now decades-long conversation about how to evaluate medical science educators for the purposes of appointment, promotion and tenure. In response to this challenge, the 2006 Consensus Conference on Educational Scholarship developed a portfolio-based format for documenting scholarly activity, using evidence of quantity, quality, and engagement with the education community (Simpson et al., 2007). In 2010, the Association of American Medical Colleges Task Force on Educator Evaluation created consensus guidelines to supplement the 2006 report for use by those responsible for evaluating the educational contributions of faculty. In that work, the task force also clearly articulated the need for institutional adoption of the criteria to recognise educators through academic promotion.

Today, faculty entering the field of medical education have access to a literature replete with guidelines for evaluating medical school faculty and recommendations for defining and strengthening scholarship in medical education. Faculty also have access to a rapidly expanding number of training programs to develop faculty as educational scholars. In addition, a growing number of medical education organisations, such as the International Association of Medical Science Educators (IAMSE) and the Association for Medical Education in Europe, and educational programs for health profession educators, have come into existence over the past 25 years. Whether this growth has translated into the formal adoption of clear guidelines for the recognition, reward, and promotion of medical science educators was recently examined by the Committee for the Advancement of Medical Science Educators (CAMSE), a subcommittee of the IAMSE Professional Development Committee. Following a comprehensive review of existing criteria for recognising excellence and professional advancement in medical education, CAMSE surveyed the IAMSE membership to assess whether the work of Boyer, Glassick, and others have influenced institutional adoption of guidelines for recognising, rewarding, and promoting medical science educators, and to evaluate biomedical science faculty awareness of these guidelines and their own institutional policies (Dickinson et al., 2018). The results of the survey identified several needs: 1) more time for faculty to pursue scholarship and innovation in medical education; 2) greater institutional recognition of medical science educators and their scholarly contributions; and 3) increased institutional support and resources for educator activities. An additional finding was that some medical science educators do feel recognised and rewarded for their work, as over half of respondents (57%) indicated that medical science educators can be promoted at their institutions based on their work as educators.

To further facilitate the reward, recognition, and promotion of medical science educators, CAMSE is creating toolkits for educators and those charged with evaluating educators. The toolkits will be applicable to a breadth of health science institutions with an international scope, and are framed using the quantity, quality, and engagement model advocated by Simpson, et al (Simpson et al., 2007). The educator toolkit provides a structured approach to developing a strong educator portfolio by helping faculty clearly translate their educational work and scholarship into a readily understood format for department chairs, promotion committee members, and other institutional leaders. Worksheets and detailed instructions provided with the toolkit are tailored to document quantity, quality, and engagement model in teaching, learner assessment, advising and mentoring, curriculum development, and leadership and administration. The evaluator toolkit is designed as a companion to this toolkit and is intended for institutional leaders tasked with evaluating educators. The focus of the toolkits on medical science educators addresses the gap in the recognition of their scholarly teaching and educational scholarship.  Further,  these toolkits may be readily adapted for use by clinician-educators. We anticipate that these user-friendly, open-access, IAMSE-recommended toolkits will be widely used by educators and adopted by institutions to enhance recognition, reward, and promotion of biomedical science educators.

III. SUPPORT FOR INNOVATION AND LEADERSHIP IN EDUCATION: BUILDING A COMMUNITY OF EDUCATORS

Medical education is undergoing rapid changes in many fundamental ways. Curricula are becoming more integrated, both horizontally across science disciplines and vertically with the incorporation of earlier clinical experiences. Teaching sessions are being shifted to more active learning formats, and assessment is now a strategy for learning, as much as it has been a measurement of learning. At the same time, the role of the teacher is moving dramatically from deliverer of information to facilitator of learning. These changes require core teaching faculty to possess specific skills gained through additional training. As institutions grapple with the challenges of helping content experts become educators, profound changes are happening to faculty members themselves. There are real shifts in terms of faculty members’ identity and how they view their role at the institution. As mentioned earlier, there are key challenges for academic institutions as to how to nurture, develop and reward these educators and ensure that they have a career path for promotion and advancement.

At Georgetown University Medical Center (GUMC), the leadership of the institution encouraged and supported the creation of a Center for Innovation and Leadership in Education (CENTILE). Specifically, this was a school-wide investment in faculty development in education, which enabled the formation of an inter-professional community of educators that drive innovation and scholarship. The goals of CENTILE are to promote excellence in teaching, to foster programmatic innovation in education, to facilitate scholarship in health professions education, and to develop future educational leaders.  CENTILE aims to help faculty gain the necessary skills through a variety of programs such as Education Grand Rounds and workshops from visiting leaders around the globe, and by convening an annual Colloquium for GUMC Educators in the Health Professions, wherein faculty share their creative innovations in education, and learn valuable insights and skills from one another. 

Another important development has been the establishment of the GUMC Teaching Academy in the Health Sciences, which provides opportunities for faculty to receive peer-recognition of accomplishments in education.  There are several levels of membership that aim to create a growth trajectory for faculty, beginning with the Protégé level, for residents and post-doctoral fellows with an interest in education; and the Associate Member, for new members of the faculty eager to explore their interest in education. The level of Principal Member, is one in which a faculty member is judged to have achieved excellence in one of five domains: Direct Teaching, Mentoring and Advising, Instructional Design/Curricular Development, Educational Scholarship or Educational Leadership. Each of those domains has specific criteria and requires submission of examples of quality, quantity, innovation and dissemination of accomplishments in that area (as described by Simpson et al., 2007).  The highest level of recognition is the Distinguished Member, in which a faculty member is deemed to have achieved excellence in three of the five domains listed above. Those applications are reviewed by both an internal group and two external reviewers. At present (May 2019), there are 110 members of the Teaching Academy at GUMC, of which 29 are at the Principal level and only 3 have attained the Distinguished Educator status. One of the important outcomes that has occurred in the 4 years since the establishment of the GUMC Teaching Academy is that the promotion and tenure committee has adopted the GUMC Teaching Academy criteria when reviewing applications for faculty appointment and promotion.

CENTILE was established to fill a need at GUMC and has a clear mandate to advance the educational mission of the academic medical centre by Educating the Educators (Haramati, 2015). Those members of the faculty who are determined to devote their careers in education now have a clear trajectory to obtain the necessary skills as well as to be evaluated and rewarded for their educational achievements. We believe that every academic health centre should identify resources, develop programs and standards, and ultimately, enable medical and health professions educators to pursue education in practice and in scholarship, and to be recognised appropriately for those accomplishments.

Notes on Contributors

Bonny L. Dickinson, PhD, MS-HPEd is a Professor and Vice-Chair of Biomedical Sciences at the Western Michigan University Homer Stryker M.D. School of Medicine, Kalamazoo, Michigan, USA.

Chen Zhi Xiong, BSc, MHPE, PhD is a senior lecturer and Integration Lead Educator (Medicine) at the Yong Loo Lin School of Medicine, and Assistant Dean of Students at the National University of Singapore.

Aviad Haramati, PhD is a Professor, Division of Integrative Physiology in the Department of Biochemistry, Molecular and Cellular Biology and Director of the Center for Innovation and Leadership in Education (CENTILE), Georgetown University School of Medicine, Washington, DC, USA.

Acknowledgement

The authors wish to thank the organisers of the Asia Pacific Medical Education Conference for selecting this work for presentation at the 2019 conference.

Funding

Funding for the manuscript was supported by each of the institutions involved.

Declaration of Interest

There is no conflict of interest from any of the authors with regards to this paper.

References

Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18(1), 57-75. https://doi.org/10.1080/0729436990180105

Bligh, J., & Brice, J. (2009). Further insights into the roles of the medical educator: The importance of scholarly management. Academic Medicine, 84(8), 1161-1165. https://doi.org/10.1097/ACM.0b013e3181ace633

Dickinson, B, L., Deming, N., Coplit, L., Huggett, K. N., Quesnelle, K., Sheakley, M., … Wragg, S. (2018). IAMSE member perspectives on the recognition, reward, and promotion of medical science educators: An IAMSE sponsored survey. Medical Science Educator, 28(2), 335-343. https://doi.org/10.1007/s40670-018-0548-z

Haramati, A. (2015). Educating the educators: A key to curricular integration. Academic Medicine, 90(2), 133-135. https://doi.org/10.1097/ACM.0000000000000444

Simpson, D., Fincher, R. E., Hafler, J. P., Irby, D. M., Richards, B. F., Rosenfeld, G. C., & Viggiano, T. R. (2007). Advancing educators and education by defining the components and evidence associated with educational scholarship. Medical Education, 41(10), 1002-1009. https://doi.org/10.1111/j.1365-2923.2007.02844.x

*Aviad Haramati
Center for Innovation and Leadership in Education,
Georgetown University School of Medicine,
Washington, DC 20057
Tel: 202-687-1021
Email: haramati@georgetown.edu

Submitted: 20 June 2019
Accepted: 12 September 2019
Published online: 5 May, TAPS 2020, 5(2), 5-13
https://doi.org/10.29060/TAPS.2020-5-2/OA2168

Simon Tso

Department of Dermatology, South Warwickshire NHS Foundation Trust, Warwick, United Kingdom

Abstract

Background: The transition experience of graduate-entry medicine degree programme students is less well understood as compared to those from undergraduate-entry medicine degree programmes.

Aim: This thematic analysis study aimed to explore the transition experience of graduate-entry medicine degree programme students at a United Kingdom medical school.

Methods: Twenty-one student volunteers from the University of Warwick 4-year graduate-entry medicine degree programme took part in this study with fourteen participants attended a further follow-up interview. Audio recordings of their semi-structured interviews were transcribed verbatim and analysed thematically.

Results: Results revealed three key transition periods within the University of Warwick Medical School’s graduate-entry medicine degree programme. Learning, professional identity development and managing coping strategies were the three key challenging issues dominating their transition experience. Medical students encountered a range of challenging issues throughout their medical school journey that could be categorised under three conceptual themes: challenges associated with the curriculum, challenges associated with their social role and generic life challenges.

Conclusions: The findings from this study could be useful to educators and medical schools in enhancing their student support services. It could also be useful to prospective and existing medical students in understanding the realities of undertaking a graduate-entry medicine degree programme.

Keywords:        Transition, Graduate-entry, Medical Student, Experience, Challenges, Identity

Practice Highlights

• There are three major transition periods within a graduate-entry medicine degree programme.
• Students could face challenges due to the curriculum, their social role and generic life events.
• Participation in patient care activities and observation of role models developed students’ identity.
  
  

I. INTRODUCTION

The term ‘transition’ originated from the Latin word ‘trānsitiō’ meaning ‘to go across’. This term had been defined by scholars in different ways. Parkes (1971) described psychological transition as the “abandonment of one set of assumptions and the development of a fresh set to enable the individual to cope with the new altered life space.” (p. 103). Levinson (1978) described transition as a “boundary zone between two states of greater stability. It involves a process of change, a shift from one structure to another” (p. 50). Schlossberg, Waters, & Goodman defined transition as “any event or non-event that results in changed relationships, assumptions and roles” (1995, p. 27). In the context of undergraduate students’ transition through their medical school journey, the literature described three key transition periods: the transition into an undergraduate medicine academic programme, the transition from the pre-clinical to clinical phase of an undergraduate medicine academic programme, and the transition into professional practice as newly qualified doctors (Radcliffe & Lester, 2003). Transitions have also been described within specific modules of the undergraduate medicine academic programme, for example, during dissection training (Lamdin, Weller, & Kerse, 2011). The literature reported medical students could encounter a broad range of challenging issues during these transition periods, which could be categorised under five broad themes: dealing with academic or work-related pressures, challenges of professional socialisation, managing uncertainties, concerns about harming patients (such as due to their lack of knowledge or skills), and general life stressors (such as financial pressures and personal life events; Dyrbye, Thomas, & Shanafelt, 2005; Prince, van de Wiel, Scherpbier, can der Vleuten, & Boshuizen, 2000; Rees & Monrouxe, 2011).

Two studies from the literature have transformed our understanding of medical students’ transition experience within their undergraduate medicine academic programmes. Firstly, the study by Radcliffe and Lester (2003) provided a broad overview of the challenges undergraduate-entry medical students could encounter during their medical school transition experience. The authors conducted an interview-based study with 21 fifth year (final year) medical students from the undergraduate-entry medicine degree programme at the University of Birmingham, United Kingdom. The authors analysed the data using a Grounded Theory approach and reported medical students experienced a series of transition periods within their degree programme. These transitions included role transition from secondary education to the first year of their degree programme, role and knowledge transition from preclinical science student status in the second-year to apprentice doctor in the third-year, and their role and knowledge transition on approaching qualification as doctors. They reported most medical students found the transition from secondary education to an undergraduate medicine academic programme stressful as they had to manage the changes to their lifestyle related to starting university, making new friends and competing with their peers. Many medical students found the less didactic teaching methods in their undergraduate-entry medicine degree programme as stressful (Radcliffe & Lester, 2003).

Next, the study by Rapport et al. (2009) offered insights into graduate-entry medical students’ transition experience through their medical school journey. The authors conducted an interview-based study with 44 medical students from the Swansea University Medical School, United Kingdom, to explore the impact of the students’ previous degree programme and life experience on their current degree programme experience. The data was analysed using thematic analysis and the study findings highlighted that prior work and life experiences had a significant and positive impact on their current graduate-entry medicine degree programme experience—findings that were consistent with my own observations (Tso, 2019). Students’ prior experiences had helped them understand complex concepts in professionalism, ethics and reflective learning, as well as how to manage patient encounters (Rapport et al., 2009). The impact of this study could have been greater if the scope of the authors’ investigation were to include graduate-entry medical students’ transition experience into professional practice as doctors.

II. METHODS

This study aimed to make a sociological contribution to understanding the transition experience of medical students from graduate-entry medicine degree programmes. The research question was ‘what are the challenges experienced by graduate-entry medicine degree programme students during their transition through medical school training?’

A two-staged interview-based study was conducted to explore the transition experience of graduate-entry medicine degree programme students at the University of Warwick in the United Kingdom. All registered students from the University of Warwick 4-year graduate-entry medicine degree MB ChB programme were invited to take part in the study via a student society’s electronic mailing list. One electronic mailing reminder was sent to all potential participants. Medical students who have suspended their studies and medical students from other medical schools undertaking placements at the University of Warwick were excluded. There were over 650 registered medical students. A mixed volunteer and quota (by student year group) sampling strategy was used to ensure representation of medical students from across all year groups.

Twenty-one students volunteered to take part in stage one semi-structured interviews each lasting between 30 to 45 minutes. Fourteen of 21 (67%) stage one participants volunteered to attend a follow-up stage two semi-structured interview each lasting between 15 to 25 minutes (took place approximately four to thirteen months following their previous interviews). The stage one and stage two interview questions were summarised in Table 1. All the interviews were audio-recorded using a digital recording device. Audio recordings of their semi-structured interviews were transcribed verbatim. Participants were invited to check and comment on the contents of their interview transcripts.

Published online: 5 May, TAPS 2020, 5(2), 14-21
https://doi.org/10.29060/TAPS.2020-5-2/OA2128

Lay Ling Tan & Carmen Jia Wen Kam

Department of Psychological Medicine, Changi General Hospital, Singapore; Clinical Trials and Research Unit, Changi General Hospital, Singapore

Abstract

Aims: Psychiatry residents’ outpatient clinic supervision may be with direct observation of clinical cases (D), without direct observation (WDO) or a mixture of both (M). This study explored residents’ perceptions of clinical teaching effectiveness under these supervision frameworks.

Methods: A survey was conducted amongst residents who completed their third-year training. Cleveland Clinic’s Teaching Effectiveness Instrument (CCTEI) was used to capture their perception of various domains of clinical teaching effectiveness. Content analysis of their qualitative feedback was used to establish the major categories of supervision perception.

Results: 42 out of 60 residents responded. Fewer residents received DO and M compared to WDO. More residents preferred DO. There was no statistical difference in the total CCTEI scores for the three supervision formats. Qualitative content analysis revealed categories surrounding strengths of DO and WDO as well as weaknesses of DO. Although DO allowed timely feedback, addressed patient safety and increased residents’ confidence, it was perceived to be stressful, requiring more resources and inhibited independent learning. WDO was commended for its support of autonomous learning, less anxiety-provoking and contributed to better patient rapport.

Conclusion: There was no statistical difference in clinical teaching effectiveness of the three supervisory frameworks. Previous studies showed DO to be anxiety-provoking with concerns of compromised autonomous learning. This study uncovered similar themes, but residents still preferred DO. It highlighted the residents’ perception of the importance of timely feedback, patient safety and instilling confidence with DO.

Keywords:        Psychiatry, Residents, Supervision, Direct Observation, Ambulatory, Outpatient

Practice Highlights

• There has been little guidance in the literature on best practices in psychiatry supervision.
• Many teaching sites offered WDO, but residents preferred DO with its timely feedback.
• DO was perceived to be stressful, needed more resources and inhibited independent learning.
• WDO was perceived to support autonomous learning, less anxiety provoking and offered better rapport with patients.
• Use of valid assessments may guide the adoption of M, leveraging on the strengths of DO and WDO.

I. INTRODUCTION

The Psychiatry Residency Training in Singapore is a national program which requires our 3^rd year residents to be posted to the outpatient clinics for a year. There are seven teaching sites in the residency program and each site offers its own supervision framework. This may be direct observation (DO) with the supervisor sitting in with the resident for the entire clinic visit or an alternative supervisory model with the resident seeing the new cases independently and discussions with the supervisors followed with no direct observation (WDO) of the consultation process. Some sites offer a mixture of both supervisory framework (M), largely determined by supervisor availability and not intentionally planned and structured.

There have been questions from the faculty, clinical supervisors and residents as to whether DO was a more effective model of supervision. The general perception was that DO would offer better patient care quality and less patient safety concerns. However, there were also concerns about the sustainability of the model of DO in view of the increasing pool of residents. In their brief report, Reardon, May, and Williams (2014) aptly discussed the challenges training directors face in balancing optimisation of patient care, education and reimbursement.

There has been little recent research on the supervision of the psychiatry interview in the non-psychotherapy outpatient setting. Extant literature highlighted advantages of DO, including a more accurate picture of the patient-student relationship; being able to alert students to non-verbal cues and students feeling less threatened since they are not without aid and corroboration (Lewin, 1966). A study by Stein, Karasu, Charles, and Buckley (1975) comparing DO with a verbal report of the interview also revealed that some residents voiced resentment that the treatment was complicated by the supervisors’ interventions and that their inadequacies and inexperience were exposed by the presence of the supervisor.

Supervision has been defined as the provision of monitoring, guidance and feedback on matters of personal, professional and educational development in the context of the doctor’s care of patients (Kilminster & Jolly, 2000). It is a complex activity with different modes of delivery that occurs in a variety of context and is undoubtedly important for the professional development of our residents. Previous work by Stein et al. (1975) suggested that DO was an important factor in the accuracy of patient evaluation and the process of verbal report itself without direct observation tended to influence the supervisor in minimising psychopathology. This will have implications in terms of the quality of patient care.

A recent review of the literature on direct and indirect supervision in psychiatric and other graduate medical education determined that there had been a dearth of research on the impact of direct supervision on the educational experience of psychiatry residents (Galanter et al., 2016). There has been little guidance in the existing literature on best practices in supervision of psychiatry residents with resulting unclear expectations (Newman, Ravindranath, Figueroa, & Jibson, 2016). This survey attempted to compare the residents’ perception of the clinical teaching effectiveness under the various supervision frameworks. This will further enhance our understanding of the variables perceived to be significant in the supervisory process of residents in general psychiatry ambulatory training. It will aid in the program’s consideration of options for optimising patient care and residency education while addressing resource limitations.

II. METHODS

The intent and purpose of the survey was discussed with the Psychiatry Residency Program Director and waiver of informed consent was sought from the Institutional Review Board. Email invitations to the survey detailing the study objectives and questionnaire were sent to all the residents at the end of their third-year ambulatory training over a period of three years. Non-responders (residents who did not respond to the email invitations after two weeks) were given the survey by the resident representative during their weekly residency didactics at the various teaching sites. The residents were posted to teaching sites with different supervision frameworks. Comparisons were made based on the residents’ declaration of the supervision framework offered by each teaching site more than 50% of the time of their ambulatory training. Their current supervisory framework (DO, WDO or M) and their preferred choice of supervision were captured in the analysis.

Feedback and comments on their supervision in the questionnaire were included for qualitative analysis. A deductive content analysis was adopted with development of the categorisation matrix by the principal investigator. Atlas Ti (version 8) software was used to code the data according to the categories.

The Cleveland Clinic’s Teaching Effectiveness Instrument (CCTEI) was used to assess the residents’ perception of the clinical teaching effectiveness under the various supervisory format. The CCTEI has been shown to be a reliable and valid evaluation tool for a wide variety of clinical teaching settings (Copeland & Hewson, 2000). It has 15 questions that used a five-point evaluation scale (see Appendix).

Categorical data was presented as frequency (percentage). Numeric data was tested with the Shapiro-Wilk normality test and presented as median (interquartile range) when the data distribution did not follow normal distribution. The associations between two categorical variables were examined using Fisher’s Exact test. Differences of CCTEI scores among three groups of supervision format were assessed by Kruskal Wallis test. In case of statistically significant difference in Kruskal Wallis test, Bonferroni post-hoc adjustment was used for multiple pairwise comparisons. A two-tailed, p-value of <0.05 was considered statistically significant. Statistical analysis was performed with SPSS statistical software, version 19.0 (IBM Corp. Armonk, NY).

III. RESULTS

Out of the 60 eligible residents, 42 responded to the survey, yielding a response rate of 70%. Most sites offered WDO supervision (45.2%). M was the supervision framework least frequently adopted across the teaching sites (19%) and 35.7% received DO (Table 1). About 80% of the residents were more than three years post-graduation at the time of the survey. There were no statistical differences in the demographic variables and the number of years post-graduation with the different supervision formats (Table 1).

Table 2. Median (interquartile range) of CCTEI Item scores and total score with the different supervision format

When asked for their preferences, most residents opted for DO (20/41, 48.8%) with a fairly equal distribution between M (11/41; 26.8%) and WDO (10/41; 24.3%). Although all but two of the eight residents with less than three years of clinical experience preferred DO and residents with more than three years of clinical experience preferred M supervisory format, the results were not statistically significant (p = .512 and .387 respectively).

Qualitative comments of residents’ perceptions and reasons for their supervision preferences captured in the questionnaire were perused in detail. A deductive content analysis was used to test existing categories and concepts with regards to clinical supervision (Elo & Kyngäs, 2008). A categorisation matrix was developed with three major categories comprising (a) Weaknesses of DO; (b) Strengths of DO; (c) Strengths of WDO. All the data were then reviewed for content and coded for correspondence with the identified structured categories using the Atlas Ti (version 8) software. All the qualitative data was able to fit into the categorisation frame. There were 19 quotations referring to weaknesses of DO; 31 quotations for strengths of DO and 21 quotations for strengths of WDO. Surprisingly, there were no statements made addressing weaknesses for WDO or feedback regarding M supervision. To further improve credibility and transferability, member checking was used. Participants’ comments regarding the developed categories were solicited. There was general agreement with the findings generated from their feedback.

A. Weaknesses of DO

The themes for the category on DO weaknesses were related to it being more stressful, needing more resources and inhibiting independent learning.

1) Stressful Environment: Some residents described that it was stressful to have the supervisor observe them for the entire consultation.

Female Resident 3: “It was nerve-wracking and anxiety-provoking to be watched.”

Female Resident 9: “It was personally very stressful to be observed while clerking a case.”

There was a perceived need for a more conducive learning environment as residents in their junior years of ambulatory training learn to engage their patients and family caregivers in the clinical encounter. The presence of the supervisor with an evaluative function was perceived to contribute to a stressful learning environment.

2) More Resources are Needed: Residents reported that having the supervisor sit in throughout the clinical encounter contributed to wastage of time and resources.

Female Resident 2: “Drains resources.”

Male Resident 8: “Will be too time-consuming and labour-intensive for supervisor and trainee.”

With the perennial challenge of having to reduce the lead time for new referrals to be seen by the outpatient specialist clinics, residents might have the notion that they were contributing to the poor utilization of resources and that more resources would need to be available to cater to both their educational and clinical service needs.

3) Inhibit Independent Learning: There was the perception that DO might potentially affect the residents’ autonomous learning.

Female Resident 2: “Can potentially inhibit the learner’s capacity for individual learning and development of interview style as he/she may feel conscious of the presence of an observer and compelled to conform to the supervisor’s style.”

Male Resident 10: “Having a supervisor in the room all the time can potentially inhibit the learner’s capacity for independent learning.”

The residents were concerned that the supervisor’s constant presence might potentially inhibit their independent learning as they would be closely guided by their supervisor when they encounter any difficulty during the clinical consultation. They also feared that they might be pressured to follow the supervisor’s particular style of interviewing.

B. Strengths of DO

The themes for the category on DO strengths centred around timely feedback, ensuring patient safety and building resident’s confidence.

1) Timely Feedback: Residents valued the immediate feedback achieved through DO.

Female Resident 7: “Able to have real-time discussion of not only management of patient, but also obtain feedback on interviewing skills and my interaction with the patients.”

Female Resident 15: “Would be the ideal scenario as it offers the opportunity to obtain feedback without the clinical interview process and on the spot learning.”

The residents described the opportunities to have their interviewing techniques scrutinized and honed by the supervisors with timely feedback as invaluable and ideal in their ambulatory training.

2) Ensuring Patient Safety: Residents were more assured that they were delivering safe patient care when they were able to consult their supervisors immediately during DO.

Female Resident 3: “Observe my management – To fine-tune my clerking, clinical management skills. Provides supervision for patient safety.”

Male Resident 23: “Discuss management plan together at first visit–Better for patient care.”

Residents endorsed direct observation of their initial interview as an important factor in the accuracy of patient assessment and evaluation. This wouldoffer better patient care quality and less patient safety concerns.

3) Building Residents’ Confidence: DO was considered to be pertinent in boosting the residents’ confidence.

Male Resident 8: “Is especially useful during the initial few weeks/months or for newer trainees to build up confidence.

Male Resident 13: “I think reviewing the case together with consultant is not required unless the resident doesn’t feel confident in managing the case.”

Residents in their junior years of training with less clinical experience would appreciate the reassuring presence of their supervisor and were possibly less conscious of exposing their inexperience and inadequacies.

C. Strengths of WDO

With the category on strengths of WDO, participants cited more autonomous learning, less anxiety and better patient rapport.

1) Autonomous Learning: Residents viewed WDO as helpful in encouraging independent learning.

Female Resident 2: “Allows greater autonomy for independent learning.”

Female Resident 4: “It gave me the independence to make decisions.”

Residents felt that WDO allowed them more opportunity and time for their own reflections about the clinical assessment and to plan their own management plan before discussing with their supervisors.

2) Less Anxiety: WDO was referred to as less stressful compared to DO.

Female Resident 4: “It also allowed me to manage the session without worrying all the time about someone else being present.”

Female Resident 25: “There will be less stress.”

Residents felt that the clinical environment will be more relaxed without the presence of the supervisor and they were better able to focus on engaging their patients and caregivers.

3) Better Patient Rapport: Residents felt that WDO allowed better patient rapport.

Male Resident 19: “Allow resident to develop own style of consult, better rapport with patient.”

Female Resident 34: “Allows the resident to build better rapport with patient.”

Residents perceived that their therapeutic alliance with their patients would be better established if the supervisor were not present in the consult room.

IV. DISCUSSION

This survey was driven by concerns about the sustainability of the model of DO in one of the teaching sites and whether DO is the most effective supervisory approach in our psychiatry ambulatory training. This study found that many of the teaching sites offered WDO as the supervisory model. Clinical supervisors were entrusted with trainees in their third year of residency with different trajectory in terms of their competency milestones. First impressions were usually relied upon to determine competency and skills of the residents (Wood, 2014). Concerns about the accuracy and diminished clinical objectivity of patient evaluation without direct observation were previously highlighted (Stein et al., 1975). This may potentially impact residency training outcomes and patient care.

A survey on how Directors of United States psychiatric residency programs were balancing patient care, education and reimbursement was recently conducted by Reardon et al. (2014). Options were proposed for optimising patient care and resident education while maintaining financial solvency. These included having a process of identifying resident supervisory needs based on training level, resident competency and complexity of patient care (Reardon et al., 2014).

Although maintenance of financial solvency is not expected of our program directors currently, there will be a need to justify the manpower needs for DO for an increasing pool of residents in future. Palesy (2017) studied the use of scaffolding and how learning might be supported in the absence of direct supervision. The consideration of a range of scaffolds provided in the learning environment without rigid stipulations might be able to encourage residents without direct observation to decide on the type and frequency of their interaction with the scaffolding provided (Palesy, 2017). The use of reliable and valid assessments e.g. mini-clinical examinations (mini-cex) may help with determining the readiness of residents for independent clinical practice and better guide the adoption of the mixed supervisory format (Norcini, Blank, Duffy, & Fortna, 2003).

Although residents alluded to WDO’s strengths in offering more autonomous learning, less anxiety and better patient rapport, our study suggested that our psychiatry residents still favoured DO, particularly those with less clinical experiences (less than three years post-graduation). This finding concurred with an earlier report of how DO might be perceived to be less threatening for students (Lewin, 1966). Residents with less clinical experience would appreciate the reassuring presence of a supervisor and possibly less conscious of exposing their inadequacies (Stein et al., 1975). Only Q11 of the CCTEI (coaches me on my clinical/technical skills) was rated as statistically significant amongst the three supervision frameworks, favouring DO. Our residents generally welcome coaching on their psychiatry skills using DO and our findings did not suggest that DO affected their independent learning (CCTEI Q2) and autonomy (CCTEI Q3) in their patient management.

A recent study by Galanter et al. (2016) referred to the traditional supervisory process in psychiatry training without direct observation, where residents were seen to be granted more autonomy and the therapeutic alliance would not be disturbed by having the supervisor in the room. Analysis of the qualitative feedback from residents in our study highlighted their concerns that DO may affect their rapport with patients but at the same time, the presence of a supervisor also helped to boost their confidence and reassure them of patient safety and quality of care. Without DO, there would be a need to have readily available guidance to residents like videoconferencing and use of other technology-facilitated supervision to support their delivery of appropriate care to their patient (Cameron, Ray, & Sabesan, 2015; Hayden, Navedo, & Gordon, 2012).

Our study has limitations. The number of residents who participated in the survey was relatively small although we managed to have a 70% response rate. Self-reported data was used for qualitative analysis with the risk of recall biases. Although independent verification of self-reported data could not be established, there were no incongruent findings. Member checking was also used to improve the credibility of the analysis.

 V. CONCLUSION

Our study did not reveal any statistical difference in the perception of clinical teaching effectiveness of the three supervisory frameworks based on the total CCTEI scores. Previous studies showed DO to be stressful with concerns of too much ‘hand-holding’ and compromised autonomous learning. Our qualitative analysis uncovered similar themes, but residents still preferred DO. The study highlighted the residents’ perceptions of the importance of timely feedback, patient safety and instilling confidence with DO.

Close supervision by DO for all cases may not be feasible but it may be crucial for residents early in their training. There should be flexibility in the supervisory framework for residents to function more independently when they are deemed to be competent and safe. The use of reliable and valid assessments may help with determining the readiness of residents for independent clinical practice and better guide the adoption of the mixed supervisory format. More research is needed to elucidate the determinants of the most effective supervisory model and to develop innovative ways of ensuring a high standard of clinical supervision and patient care.

Notes on Contributors

Dr Tan Lay Ling is Senior Consultant Psychiatrist in the Psychogeriatric Service of the Department of Psychological Medicine of CGH. She is currently appointed as the Associate Program Director for the National Psychiatry Residency Program in CGH. She is also a member of the Clinical Competency Committee of the Residency Program.

Miss Kam Jia Wen Carmen is the Biostatistician of the Clinical Trials and Research Unit of CGH.

Ethical Approval

Submission for review was deemed not required by the SingHealth Institutional Review Board (iSHaRe Ref. No: 201601-00106; CIRB Ref. No: 2016/2083).

Acknowledgements

We would like to acknowledge the contribution of the National Psychiatry Residency Office for its administrative support in this survey and the residents for their willingness to participate.

Funding

We did not receive any funding for this research study.

Declaration of Interest

The main researcher is the Associate Program Director of one of the teaching sites and has a vested interest in ensuring the quality of supervision of residents. The researcher cannot decide on the standardisation of the supervisory framework. There are no other conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Cameron, M., Ray, R., & Sabesan, S. (2015). Remote supervision of medical training via videoconference in northern Australia: a qualitative study of the perspectives of supervisors and trainees. BMJ open, 5(3), e006444. https://doi.org/10.1136/bmjopen-2014-006444

Copeland, H. L., & Hewson, M. G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2), 161-166. https://doi.org/10.1097/00001888-200002000-00015

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107-115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Galanter, C. A., Nikolov, R., Green, N., Naidoo, S., Myers, M. F., & Merlino, J. P. (2016). Direct Supervision in Outpatient Psychiatric Graduate Medical Education. Academic Psychiatry, 40(1), 157-163. https://doi.org/10.1007/s40596-014-0247-z

Hayden, E. M., Navedo, D. D., & Gordon, J. A. (2012). Web-conferenced simulation sessions: a satisfaction survey of clinical simulation encounters via remote supervision. Telemedicine and e-Health, 18(7), 525-529. https://doi.org/10.1089/tmj.2011.0217

Kilminster, S., & Jolly, B. (2000). Effective supervision in clinical practice settings: a literature review. Medical Education, 34(10), 827-840. https://doi.org/10.1046/j.1365-2923.2000.00758.x

Lewin, K. (1966). Psychiatric supervision by direct observation. Academic Medicine, 41(9), 860-864. https://doi.org/10.1097/00001888-196609000-00005

Newman, M., Ravindranath, D., Figueroa, S., & Jibson, M. D. (2016). Perceptions of supervision in an outpatient psychiatry clinic. Academic Psychiatry, 40(1), 153-156. https://doi.org/10.1007/s40596-014-0191-y

Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003). The mini-CEX: a method for assessing clinical skills. Annals of internal medicine, 138(6), 476-481. https://doi.org/10.7326/0003-4819-138-6-200303180-00012

Palesy, D. (2017). Learning in the Absence of Direct Supervision: Person-Dependent Scaffolding. Vocations and Learning, 10(3), 365-382. https://doi.org/10.1007/s12186-017-9176-9

Reardon, C., May, M., & Williams, K. (2014). Psychiatry resident outpatient clinic supervision: how training directors are balancing patient care, education, and reimbursement. Academic Psychiatry, 38(4), 476-480. https://doi.org/10.1007/s40596-014-0091-1

Stein, S. P., Karasu, T. B., Charles, E. S., & Buckley, P. J. (1975). Supervision of the initial interview: A study of two methods. Archives of General Psychiatry, 32(2), 265-268. https://doi.org/10.1001/archpsyc.1975.01760200129013

Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education, 19(3), 409-427. https://doi.org/10.1007/s10459-013-9453-9

*Tan Lay Ling
Changi General Hospital,
2 Simei Street 3, Singapore 529889
Tel: 69365072
E-mail: tan.lay.ling@singhealth.com.sg