Submitted: 17 April 2020
Accepted: 05 August 2020
Published online: 5 January, TAPS 2021, 6(1), 114-118
https://doi.org/10.29060/TAPS.2021-6-1/SC2358

Warren Fong^1,3,4, Yu Heng Kwan², Sungwon Yoon², Jie Kie Phang¹, Julian Thumboo^1,2,4 & Swee Cheng Ng¹

¹Department of Rheumatology and Immunology, Singapore General Hospital, Singapore; ²Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; ³Duke-NUS Medical School, Singapore; ⁴Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Introduction: This study aimed to examine the perception of faculty on the relevance, feasibility and comprehensiveness of the Professionalism Mini Evaluation Exercise (P-MEX) in the assessment of medical professionalism in residency programmes in an Asian postgraduate training centre.

Methods: Cross-sectional survey data was collected from faculty in 33 residency programmes. Items were deemed to be relevant to assessment of medical professionalism when at least 80% of the faculty gave a rating of ≥8 on a 0-10 numerical rating scale (0 representing not relevant, 10 representing very relevant). Feedback regarding the feasibility and comprehensiveness of the P-MEX assessment was also collected from the faculty through open-ended questions.

Results: In total, 555 faculty from 33 residency programmes participated in the survey. Of the 21 items in the P-MEX, 17 items were deemed to be relevant. For the remaining four items ‘maintained appropriate appearance’, ‘extended his/herself to meet patient needs’, ‘solicited feedback’, and ‘advocated on behalf of a patient’, the percentage of faculty who gave a rating of ≥8 was 78%, 75%, 74%, and 69% respectively. Of the 333 respondents to the open-ended question on feasibility, 34% (n=113) felt that there were too many questions in the P-MEX. Faculty also reported that assessments about ‘collegiality’ and ‘communication with empathy’ were missing in the current P-MEX.

Conclusion: The P-MEX is relevant and feasible for assessment of medical professionalism. There may be a need for greater emphasis on the assessment of collegiality and empathetic communication in the P-MEX.

Keywords:            Professionalism, Singapore, Survey, Assessment

I. INTRODUCTION

Medical professionalism is one of the core Accreditation Council for Graduate Medical Education competencies and forms the basis of medicine’s contract with society. Unprofessional behaviour during training of junior doctors has been shown to result in future unprofessional behaviour. Assessment of professionalism not only allows for timely feedback to residents to help them improve, but also allows for development of better curriculum to prevent lapses in medical professionalism. The Professionalism Mini-Evaluation Exercise (P-MEX) had previously been identified as a potential observer-based assessment tool (Kwan et al., 2018), but it has not been validated in a multi-ethnic and multi-cultural Asian context such as Singapore. According to International Ottawa Conference Working Group on the Assessment of Professionalism, professionalism varies across cultural contexts, and therefore cross-cultural validation of the assessment tool for medical professionalism is imperative (Hodges et al., 2011). The current assessment tools adopted in local institutions may not cover the entire continuum of medical professionalism. For example, in the Ministry of Health Holdings (MOHH) C1 form which is currently being used for the assessment of residents on a 6-monthly basis, the assessment of professionalism is summative and consists of only three items (1) Accepts responsibility and follows through on tasks, (2) Responds to patient’s unique characteristics and needs equitably, (3) Demonstrates integrity and ethical behaviour.

We aimed to (1) examine faculty perception of the relevance of the P-MEX for assessment of medical professionalism in the local context, and (2) determine the feasibility and comprehensiveness of the P-MEX as an assessment tool for medical professionalism in Singapore.

II. METHODS

A. Design and Participants

We invited faculty in the SingHealth residency programmes to participate in the study by completing an online anonymous questionnaire in July 2018 to August 2018. Participants were given one week to complete the survey, with three reminder emails sent at one-week, two-weeks and one-month after the deadline for submission. SingHealth Centralised Institutional Review Board approved the conduct of this study (Reference Number: 2016/3009). Implied informed consent was provided by participants before completing the online anonymous questionnaire.

 B. Survey Questionnaire

The P-MEX consists of four domains (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) and 21 sub-domains. Faculty were asked to rate the relevance of each item in P-MEX using a 0-10 numerical rating scale (0 representing not relevant, 10 representing very relevant). The faculty were also asked the following open-ended questions to determine the feasibility and comprehensiveness of the P-MEX- (1) “In your opinion, is a P-MEX form with 21 items too long, making it not feasible for routine use? If so, which items should be removed?” and (2) “In your opinion, are there any missing items (observable actions of a medical professional) that should be included in this form? If so, what new items should be added?” The questionnaire also included additional questions related to demographic characteristics (age, gender, specialty and number of years since becoming a specialist).

 C. Analysis

Items were deemed to be relevant to the assessment of medical professionalism when at least 80% of the faculty gave a rating of ≥8. This was determined by expert judgement and prior literature (Avouac et al., 2011). For the open-ended questions on feasibility and comprehensiveness, responses were categorised and the number of the respondents who deemed the 21-item P-MEX to be not feasible (too long) or not comprehensive (there were missing items that should be included) are presented.

 III. RESULTS

In total, 555 faculty from 33 residency programmes participated in the survey (response rate 44%). The respondents were 59% male, median age 43 years old, age ranged from 30 to 78 years old. Specialists from medical and surgical disciplines made up 39% and 27% of the respondents respectively, with the remaining respondents coming from diagnostic radiology/nuclear medicine, anaesthesiology, paediatrics and emergency medicine (12%, 11%, 6% and 5% of the respondents respectively).

 A. Relevance

Of the 21 items in P-MEX, 17 items were deemed to be relevant (at least 80% of the faculty gave a rating of ≥8). For the remaining four items ‘maintained appropriate appearance’, ‘extended his/herself to meet patient needs’, ‘solicited feedback’, and ‘advocated on behalf of a patient’, the percentage of faculty who gave a rating of ≥8 was 78%, 75%, 74%, and 69% respectively (Figure 1). 

Figure 1: Percentage of faculty (n=555) who rated the item ≥8 on the relevance of the item in assessment of medical professionalism using a 0-10 numerical rating scale (0 representing not relevant, 10 representing very relevant).

B. Feasibility

There were 333 respondents for the question “In your opinion, is a P-MEX form with 21 items too long, making it not feasible for routine use? If so, which items should be removed?”, of which 34% (n=113) felt that there were too many questions in the P-MEX assessment form. The top four items chosen to be removed were “solicited feedback” (n=36), “extended his/herself to meet patient needs” (n=27), “advocated on behalf of a patient” (n=25), and “maintained appropriate appearance” (n=23).  208 (62%) respondents felt that the number of questions in the P-MEX assessment form was appropriate.

C. Comprehensiveness

There were 307 respondents to the question “In your opinion, are there any missing items (observable actions of a medical professional) that should be included in this form? If so, what new items should be added?”, of which 28% (n=85) faculty felt that there were missing items. The most frequently mentioned missing items were regarding assessment of ‘collegiality’ (n=54) and assessment of ‘communication with empathy’ (n=12).

Examples of ‘collegiality’ provided by faculty— “Collaboration with other healthcare professionals in the patients’ best interest”, “Demonstration of collaborative behaviour”

Examples of ‘communication with empathy ‘provided by faculty— “Communicate with empathy and effectively to patient and family, taking into account their level of understanding, education and socioeconomic background”, “Communication skills…should embrace empathy, listening skills, discretion, sensitivity and intelligence… sufficient information, counselling, planning and advice regarding medical condition and options.”

207 respondents (67%) felt that the P-MEX was comprehensive for the assessment of medical professionalism.

IV. DISCUSSION

This study provides preliminary evidence on the relevance, feasibility and comprehensiveness of the P-MEX in the assessment of medical professionalism in an Asian city state. The current study is part of a larger project to culturally adapt and validate the P-MEX. Based on our knowledge, this is the first study to explore the faculty perception on relevance, feasibility and comprehensiveness of the P-MEX in the assessment of medical professionalism in a multi-cultural and multi-ethnic context.

There were four items that were deemed to be less relevant (extended his/herself to meet patient needs, advocated on behalf of a patient, solicited feedback, maintained appropriate appearance). These findings were also similar in a validation study performed in Canada, where the items ‘extended his/herself to meet patient needs’ and ‘advocated on behalf of a patient’ were also frequently marked as ‘not applicable’, suggesting that the two items may be less relevant (Cruess, McIlroy, Cruess, Ginsburg, & Steinert, 2006). Qualitative methods can be used to explore the reasons why these items were deemed to be less relevant. About one-third of faculty felt that P-MEX was too long. Further study is warranted to evaluate the possibilities for shortening the P-MEX to reduce response burden and enhance routine use of the P-MEX.

In addition, our study revealed a need for greater emphasis on the assessment of collegiality. Some faculty felt that ‘collegiality’ was missing in the P-MEX despite the presence of items such as ‘demonstrated respect for colleagues’ and ‘avoided derogatory language’. This suggests that collegiality may encompass actions other than demonstrating respect and avoiding derogatory language in the local context, and further reinforces the emphasis of interprofessional collaborative practice. 

Faculty also felt that there was also a lack of assessment of ‘communication with empathy’ in the P-MEX. The importance of empathetic communication is also supported by a study in Indonesia, a country in the same region, which found that patients considered communication as the most important attribute of medical professionalism (Sari, Prabandari, & Claramita, 2016).

This study has some limitations. The non-response rate raises concern about possible selection bias. Non-responders may have been less enthusiastic about the assessment of medical professionalism. Medical professionalism is affected by socio-cultural factors, therefore the findings from this study may not be entirely generalizable to another socio-cultural context. In addition, we were unable to elucidate the reasons for disagreement with the relevance of some of the items in the P-MEX as many faculty did not provide feedback and comments. Nevertheless, the findings of this study can serve as basis for future research, especially in countries with similar multicultural backgrounds.

V. CONCLUSION

Faculty agreed that most of the items in the P-MEX were relevant in the assessment of medical professionalism. Majority of the faculty also felt that the P-MEX was feasible to be used routinely in the assessment in medical professionalism. There may be a need for greater emphasis on the assessment of collegiality and communication with empathy in the modified P-MEX.

Notes on Contributors

Warren Fong reviewed the literature, designed the study, collected data, analysed data, and wrote manuscript. Yu Heng Kwan reviewed the literature, designed the study, collected data, analysed data, and wrote manuscript. Sungwon Yoon advised the design of study, analysed data, and gave critical feedback to the writing of manuscript. Jie Kie Phang collected data, analysed data, and wrote manuscript. Julian Thumboo advised the design of study, and gave critical feedback to the writing of manuscript. Swee Cheng Ng advised the design of study, collected data, analysed data, and gave critical feedback to the writing of manuscript. All authors have read and approved the final manuscript.

Ethical Approval

Ethical approval for this was granted by the SingHealth Institutional Review Board (Reference Number: 2016/3009).

Acknowledgement

The authors wish to thank all the study participants for contributing to this work.

Funding

This research was supported by SingHealth Duke-NUS Medicine Academic Clinical Programme Education Support Programme Grant (Reference Number: 03/FY2017/P2/03-A47). Funder was not involved in the design, delivery or submission of the research.

Declaration of Interest

The authors declare that they have no competing interests.

References

Avouac, J., Fransen, J., Walker, U., Riccieri, V., Smith, V., Muller, C., … Matucci-Cerinic, M. (2011). Preliminary criteria for the very early diagnosis of systemic sclerosis: Results of a Delphi Consensus Study from EULAR Scleroderma Trials and Research Group. Annals of the Rheumatic Diseases, 70(3), 476-481. doi:10.1136/ard.2010.136929

Cruess, R., McIlroy, J. H., Cruess, S., Ginsburg, S., & Steinert, Y. (2006). The professionalism mini-evaluation exercise: A preliminary investigation. Academic Medicine, 81(10), S74-S78.

Hodges, B. D., Ginsburg, S., Cruess, R., Cruess, S., Delport, R., Hafferty, F., . . . Ohbu, S. (2011). Assessment of professionalism: Recommendations from the Ottawa 2010 Conference. Medical Teacher, 33(5), 354-363.

Kwan, Y. H., Png, K., Phang, J. K., Leung, Y. Y., Goh, H., Seah, Y., . . . Lie, D. (2018). A systematic review of the quality and utility of observer-based instruments for assessing medical professionalism. Journal of Graduate Medical Education, 10(6), 629-638.

Sari, M. I., Prabandari, Y. S., & Claramita, M. (2016). Physicians’ professionalism at primary care facilities from patients’ perspective: The importance of doctors’ communication skills. Journal of Family Medicine and Primary Care, 5(1), 56-60. https://doi.org/10.4103/2249-4863.184624

*Warren Fong
SingHealth Rheumatology,
Senior Residency Programme,
20 College Road,
Singapore 169856
Tel: +6563214028
Email: warren.fong.w.s@singhealth.com.sg

 

Submitted: 17 March 2020
Accepted: 3 April 2020
Published online: 1 September, TAPS 2020, 5(3), 83-87
https://doi.org/10.29060/TAPS.2020-5-3/SC2238

Cristelle Chow¹, Cynthia Lim² & Koh Cheng Thoon³

¹General Paediatrics Service, Department of Paediatrics, KK Women’s and Children’s Hospital, Singapore; ²Nursing Clinical Services, KK Women’s and Children’s Hospital, Singapore; ³Infectious Disease Service, Department of Paediatrics, KK Women’s and Children’s Hospital, Singapore

Abstract

I. INTRODUCTION

Effective doctor-patient communication leads to better compliance, health outcomes and higher doctor and patient satisfaction. Online video-based communication skills courses can be feasible, with learners reporting increased confidence in key communication skills (Kemper, Foy, Wissow, & Shore, 2008). However, these evaluation methods have been limited to the Kirkpatrick levels of “reaction” and “learning”, instead of “behaviour” and “results”, which are more reflective of applied learning.

While in-person communication skills training programs simulate clinical environments, they can have inconsistent delivery because facilitators and standardised patients provide variable training experiences. In order to replace traditional role-play sessions, this study aimed to develop and implement a pilot online communication skills course to provide standardised, video-based scenarios in a self-directed interactive learning format using an evidence-based framework.

Our research questions are as follows:

1. Would an online communication course improve the application of communication skills in real clinical encounters?
2. What is the impact of an online communication course on learner-rated confidence levels in paediatric-specific clinical communication encounters?
3. What are the self-reported aspects of learning that participants of an online communication course experience?

II. METHODS

This course design utilised the Kalamazoo Consensus framework (Makoul, 2001) which included the essential elements of clinical communication: Open the discussion, gather information, understand patient’s perspective, share information, reach agreement and provide closure.

Through Bandura’s social learning theory, people learn through observing others’ behaviour. The attitudes and outcomes of those behaviours then guide subsequent actions. This course therefore utilised videos featuring positive doctor-caregiver interactions, to encourage modelling through observation. The 3-5-minute video scenarios acted by practicing healthcare professionals were based on commonly encountered general paediatric clinical situations.

The course was designed using Articulate© software. “Pop-up” prompts highlighting important clinical or communication points, a pre- and post-test and in-video multiple-choice questions were included to increase learner engagement. To evaluate the impact of the course on learner-rated confidence levels, students were shown a clinical vignette, and asked to rate their self-confidence on a 4-point Likert scale before and after each video. Each video concluded with a summary, emphasising the utilisation of the Kalamazoo Consensus Framework.

Table 1. Caregiver/Patient Feedback Form

To evaluate the self-reported learning points from the course, students were asked upon course completion to provide course feedback, including free-text completion of the phrase: “Things I have learnt include…” To evaluate whether the course improved the application of communication skills in real clinical encounters, caregiver/patient feedback was obtained towards the end of the paediatric clerkship for all students, regardless of course participation (Table 1). This form was modified based on course content from a family feedback instrument utilised in a paediatric setting (Zimmer, Solomon, Siberry, & Serwint, 2008). Implied informed consent was obtained from all participants.

Final year medical students from a five-year Singapore undergraduate medical program were enrolled over one academic year (2017/2018). Alternate batches (2^nd, 4^th, 6^th, 8^th) were enrolled into the course. Each student was provided a unique username and password for course access on any internet-enabled device throughout his/her 2-week paediatric clerkship and course participation was strongly recommended. Students from other batches (1^st, 3^rd, 5^th, 7^th) were analysed as controls. All students integrated into paediatric clinical teams, participated in ward rounds and communicated plans to patients/caregivers.

Statistical analysis was performed using SPSS© Statistics version 25.0 and chi-square analysis was used for patient feedback analysis.

This study was exempted from formal Centralized Institutional Review Board review and implied informed consent was granted by the SingHealth Centralized Institutional Review Board.

III. RESULTS

A total of 146 students were posted to the study institution in academic year 2017/2018 and 88 students were enrolled into the course. There were 80 (90.9%) attempts at the course, of which 76 (95%) students provided course feedback. The median time needed for course completion was 59 minutes. Patient feedback was successfully collected for 94 students, of which 44 (46.8%) attempted the course. Main reasons for unsuccessful collection were fast patient turnovers and patients/caregivers rejecting the request to provide feedback, usually due to perceived insufficient student contact time.

A. Application of Communication Skills – Evaluated via Patient Feedback

Although there were generally no differences in patient feedback between intervention and control groups, the control group was more likely to check with caregivers/patients whether they were agreeable with the management plan (76.0% vs. 56.8%, p<0.05) and provided a summary to the caregiver/patient (74.0% vs. 47.7%, p<0.05). Approaching statistical significance was the finding that caregivers/patients were more likely to feel very comfortable with the intervention compared to the control group (65.9% vs. 48.0%, p=0.062).

B. Course Impact on Self-Reported Confidence Levels

For scenario 1, the median confidence level increased from 3 (“somewhat confident”) to 4 (“very confident”). For the subsequent scenarios, this increased from 2 (“a little confident”) to 3 (“somewhat confident”).

C. Self-Reported Learning Points –Evaluated via Course Feedback

1) Knowledge: The majority of students mentioned learning about the clinical management and discharge advice for gastroenteritis and urinary tract infection, and the need for procedural sedation in uncooperative young children. Students reported that they had learnt general frameworks and principles for communication, and concepts of consent-taking. Students also frequently mentioned “practical”, in terms of “practical knowledge” and “practical tips” for communication.

2) Attitude: Students mentioned that they learnt about the importance of empathy. They also reported important aspects of patient-centred care, such as understanding the parent’s or patient’s perspective to formulate a treatment plan together and ensuring mutual understanding via “checking back to ensure the parent truly understands” and “to have a closed loop at the end of each communication”.

3) Skills: On a broader perspective, students described that they had learnt “how to properly structure communication with a patient’s parents” and “how to better communicate with parents using the various strategies”. Almost all students reflected that they had learnt specific communication skills, particularly with regards to dealing with challenging situations such as “how to approach parents who may not be cooperative/willing to listen to you” and “how to address angry parents” as well as “how to address their concerns and manage their expectations”. Two students also mentioned that they may not have been exposed to similar scenarios in their daily work: “… handle scenarios which are often not taught within lectures.”

IV. DISCUSSION

Computer-based communication courses have shown to improve students’ self-efficacy in performing communication tasks and assessments of students’ perceptions and practice of communication skills (Kemper et al., 2008), which was also demonstrated in this study’s improvement in self-reported confidence levels. It is however, expected that most students would experience increased confidence immediately after receiving new information about an unfamiliar topic.

This study provides an example of how a course that is traditionally delivered face-to-face can be designed to be delivered online, utilising less time and manpower resources while providing standardised teaching instruction in an evidence-based manner.

The qualitative findings in this study have not been replicated elsewhere, and provide an interesting perspective to student course perception. Students gained practical knowledge which is not readily available in clinical clerkships due to patient case variability and gained insight into an applicable framework for future clinical communication encounters. It is possible that the interactive nature of the course increased student presence and participation, resulting in improved learning outcomes in this aspect (Ammenwerth et al., 2019). Empathy, an important professional skill not easily taught but reflected as a learning point, was likely acquired through non-verbal communication demonstrated in the videos. Although it is not guaranteed that self-reported knowledge, skills and attitudes will translate into practice, future e-learning communication courses can be designed as pre-course material for traditional role-play facilitators to enhance learning experiences.

This study’s use of patient feedback provides unique insight into applied learning. Interestingly, the control group fared better in the actions of checking with caregivers/patients about management plan agreement and providing caregivers/patients with a summary. As clerkships also provide opportunities to observe healthcare professionals conducting clinical communication, it is likely that the control group learnt these behaviours from real-life encounters. Caregivers/patients tended to feel more comfortable with the intervention group, which could be explained through unmeasurable, subtle behaviours that the group may have learnt from the course, such as empathy, attentiveness and appropriate body language. Although the use of standardised patients for comparing both groups might have shown different results, it is known that how learners behave in the classroom and with real patients when unobserved is often less reflective of true workplace behaviours (Malhotra et al., 2009).

This study is limited by small participant and patient feedback numbers. Culturally, many patients forget their healthcare providers and experiences. An ideal situation would be direct clinical encounter observation, but due to the Hawthorne effect, a less truthful version of student behaviour may be observed instead.

V. CONCLUSION

Although online video-based communication courses can be used as a standardised teaching tool to improve student self-reported confidence levels and self-perceived knowledge, skills and attitudes, it remains to be proven if they can result in a change in student behaviour. It is likely that on-the-job experiences also contribute to their ability to demonstrate effective communication, which supports the supplementation, rather than the replacement of such practical experiences with online video-based course material.

Notes on Contributors

CC, CL and TKC contributed to the conception and design of the work. CC, CL and TKC also analysed data  and drafted the work . CC, CL and TKC approved the final published version and are agreeable to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CC, CL and TKC collectively contributed equally to this paper.

Ethical Approval

This study was exempted from formal Centralized Institutional Review Board review by the SingHealth Centralized Institutional Review Board (CIRB Ref: 2017/2178).

Acknowledgements

The authors would like to acknowledge the SingHealth Paediatrics Academic Clinical Programme in providing the grant funding for this project.

Funding

The study was funded by the SingHealth Paediatrics Academic Clinical Programme Tan Cheng Lim Fund Grant which was awarded in 2017 (Grant Reference: PAEDACP-TCL/2017/EDU/001).

Declaration of Interest

All authors disclose that there are no potential conflicts of interest, including financial, consultant, institutional and other relationships that could have direct or potential influence or impart bias on the work.

References

Ammenwerth. E., Hackl, W. O., Dornauer, V., Felderer, M., Hoerbst, A., Nantschev, R., & Netzer, M. (2019). Impact of students’ presence and course participation on learning outcome in co-operative online-based courses. Studies in Health Technology and Informatics, 262, 87-90.

Kemper, K. J., Foy, J. M., Wissow, L., & Shore, S. (2008). Enhancing communication skills for paediatric visits through on-line training using video demonstrations. BMC Medical Education, 8, 8.

Makoul, G. (2001). Essential elements of communication in medical encounters: the Kalamazoo consensus statement. Academic Medicine, 76(4), 390-393.

Malhotra, A., Gregory, I., Darvill, E., Goble, E., Pryce-Roberts, A., Lundberg, K., & Hafstad, H. (2009). Mind the gap: Learners’ perspectives on what they learn in communication compared to how they and others behave in the real world. Patient Education and Counseling, 76(3), 385-90.

Zimmer, K. P., Solomon, B. S., Siberry, G. K., & Serwint, J. R. (2008). Continuity-structured clinical observations: assessing the multiple-observer evaluation in a pae1diatric resident continuity clinic. Pediatrics, 121(6), e1633-1645.

*Cristelle Chow
Department of Paediatrics,
KK Women’s and Children’s Hospital
100 Bukit Timah Road,
Singapore 229899
Email: cristelle.chow.ct@singhealth.com.sg

F. Tip 6 – Pay PWDs at Market Rates and Accord Them Identical Contractual Rights

PWD SPs should be remunerated at market rates that are equal to SPs without disability. They also sign the same SP contract and fulfil the same legal obligations. In performing the role of the SP, they are treated as co-workers of the university, not volunteers or patients. This reflects the principles of equality and diversity, as well as the seriousness of their roles as active members of the healthcare and medical education system. If there are certain risks involved in their interaction with students, such risks should be made clear to the PWD SPs, so they can make an informed decision on accepting the job.

G. Tip 7 – Provide Opportunity for PWDs to Give Feedback

PWDs can be a valuable resource in providing feedback to faculty, scenario developers and other SPs. Similarly, they may be able to give insightful feedback to students. It is important to train the PWD SPs on the methods of providing feedback to students. Given their lived experience, they can provide insight into how real patients would respond and react while suggesting ways for trainee healthcare professionals to respond in a more patient-centred manner.

H. Tip 8 – Reflect and Improve

Carrying out an evaluation with the respective stakeholders, whether they are PWD SPs, faculty, or students, is key to the success of an inclusive SP program. This can also ensure quality assurance of the program. The following are several broad questions which can be considered in the evaluation. Firstly, whether the stakeholder faced any challenges during the session. Secondly, whether the scenarios or scripts worked well for PWD SPs to interact with students. Thirdly, whether there are any other ways that the learning experience can be improved. This can provide rich data for the SP program developers to reflect and improve upon the pedagogy. We have received positive feedback from both students and PWDs in this project.

III. CONCLUSION

It is important to empower PWDs and create a dynamic relationship between them and healthcare professionals/
educators. For an inclusive SP program to be effective, educators must change their own mindset about PWDs. We have to switch the lens from viewing them as patients to co-workers. This partnership should be viewed as a reciprocally beneficial one whereby the university and the disability community learn from one another. Through the process of engagement, both educators and students learn from PWD SPs about knowledge that goes beyond the textbook, and the factors that enhance or diminish the quality of healthcare/medical service delivery from individuals who have experienced going through the healthcare/medical system. With time and with more training institutions engaging PWDs as SPs, this can be a potentially viable employment option for PWDs.

Notes on Contributors

Associate Professor May Lim is the Director of the Centre for Learning Environment and Assessment Development (CoLEAD) at the Singapore Institute of Technology, and a faculty in the Health and Social Sciences Cluster teaching occupational therapy.

At the time when this work was done, Mr Goh Zi An Galvyn was a research assistant in the Centre for Learning Environment and Assessment Development (CoLEAD) at the Singapore Institute of Technology.

Associate Professor Tan Bhing Leet is the Deputy Cluster Director (Applied Learning) of the Health and Social Sciences Cluster, and Programme Director of the Bachelor of Science in Occupational Therapy degree programme at the Singapore Institute of Technology.

Ethical Approval

Ethics approval was granted by the Singapore Institute of Technology Institutional Review Board for this project (IRB number: 20150002).

Acknowledgements

The authors thank all faculty, students, PWD and non-PWD standardised patients who were involved in the Singapore Institute of Technology Bachelor of Science in Occupational Therapy degree programme. In addition, we would like to extend our deepest gratitude to Associate Professor Tham Kum Ying, Education Director of Tan Tock Seng Hospital Pre-Professional Education Office and senior lecturers Miss Heidi Tan and Mr Lim Hua Beng from the Singapore Institute of Technology.

Funding

Funding was provided from the Singapore Ministry of Education (MOE Tertiary Education Research Fund grant: R-MOE-A203-A002).

Declaration of Interest

The authors declare no conflict of interest concerning any aspect of this research.

References

Kowitlawakul, Y., Chow, Y., Salam, Z., & Ignacio, J. (2015). Exploring the use of standardized patients for simulation-based learning in preparing advanced practice nurses. Nurse Education Today, 35(7), 894-899. https://doi.org/10.1016/j.nedt.2015.03.004

Lim, S. M., Tan, B. L., Lim, H. B., & Goh, Z. A. G. (2018). Engaging persons with disabilities as community teachers for experiential learning in occupational therapy education. Hong Kong Journal of Occupational Therapy, 31(1), 36-45. https://doi.org/10.1177/1569186118783877

Long-Bellil, L. M., Robey, K. L., Graham, C. L., Minihan, P. M., Smeltzer, S. C., Kahn, P., & Alliance for Disability in Health Care Education. (2011). Teaching medical students about disability: The use of standardized patients. Academic Medicine, 86(9), 1163-1170. https://doi.org/10.1097/ACM.0b013e318226b5dc

Minihan, P. M., Bradshaw, Y. S., Long, L. M., Altman, W., Perduta-Fulginiti, S., Ector, J., … Sneirson, R. (2004). Teaching about disability: Involving patients with disabilities as medical educators. Disability Studies Quarterly, 24(4). https://doi.org/10.18061/dsq.v24i4.883

Wells, T. P. E., Byron, M. A., McMullen, S. H. P., & Birchall, M. A. (2002). Disability teaching for medical students: Disabled people contribute to curriculum development. Medical Education, 36(8), 788-790. https://onlinelibrary.wiley.com/doi/abs/10.1046/j.1365-2923.2002.01264_1.x

*Lim Sok Mui
Singapore Institute of Technology,
SIT@Dover, 10 Dover Drive,
Singapore 138683
Email: may.lim@singaporetech.edu.sg

Published online: 1 June, TAPS 2016, 1(1), 23-25
DOI: https://doi.org/10.29060/TAPS.2016-1-1/SC1009

Richard Hays

University of Tasmania, Australia

Abstract

A curriculum is an important component of a medical program because it is the source of information that learners, teachers and external stakeholders use to understand what learners will experience on their journey to recognition as a medical graduate. While many focus on and debate the content of a medical curriculum, with some suggestions that there should be national curricula for each jurisdiction or even a global curriculum for all medical programs, the curriculum content is only one factor to consider when designing, revising or accrediting a curriculum. Just as important are the alignment with the program’s mission and health workforce needs, the presence of agreed graduate outcomes, the theoretical bases of the curriculum, the prior learning of commencing students, the curriculum implementation models, the assessment of student progress and program evaluation processes. This paper presents a framework for this more holistic approach to reviewing a curriculum, proposing triangulation of information from several sources – documents, websites, learners, teachers and employers – and considering several accreditation standards that impact on curriculum design and delivery.

Keywords:        Curriculum design; curriculum review; accreditation; social accountability; program evaluation

I. BACKGROUND

In medical education the curriculum defines medical programs, guides the teaching by faculty and informs the learning by students of what is required to become a doctor. For basic medical education, the outcome is recognition as a novice practitioner, and for subsequent levels there are more specific outcomes related to particular specialties. The term ‘curriculum’ is defined in the Oxford Dictionary as ‘the subjects comprising a course of study in a school or college’, which suggests an emphasis on the content, whereas learning may depend significantly on how the content is delivered, learned and assessed.

The pace of medical curriculum review has increased globally due to several factors. Several new medical programs have been established, based on growing populations and rising health care standards, particularly in developing nations. Whether purchased from existing institutions or developed locally, new curricula have to be designed and most new programs face either mandatory or voluntary accreditation processes. Demographics are changing, particularly in developed nations, where the population is ageing and living with increasingly complex and chronic health care needs, requiring a larger and differently trained medical workforce (Duckett, 2005).   Many universities are seeking efficiencies in program delivery, because the small group, clinician-led models preferred in medical education are expensive; leaders ask, perhaps not unreasonably, why medical education cannot be provided as effectively by less expensive methods, such as large group lectures supported by on-line resources and more junior faculty. We find ourselves in what might be termed a ‘post-PBL’ environment, where PBL programs have been criticised for gaps or lack of depth in anatomy, pathology and other foundation sciences, even though PBL models were developed in part to address the rapid increase in the knowledge base for medical practice, promoting peer-supported and self-directed learning (Dolmans et al, 2005). Can coping with this knowledge explosion be done differently?

Employers find that some medical graduates are not yet ‘work ready’, able to take responsibility for their actions or contribute to safe patient care without (ex-pensive) supervision and further training. Finally, regulators are becoming more vocal about challenges to the commonly used self-regulation model for the medical profession, amidst increasing complaints and concerns about competence and errors. Although most of these concerns relate to communication skills and professional behaviours of a small minority, regulators are increasing requirements for standards to be met by medical graduates outside of the traditional scientific knowledge domains.   As a result, there are increasing requirements for accreditation or formal recognition of medical programs by regulatory authorities to ensure that programs produce the graduates needed to provide medical care. Arguably, the strongest accreditation systems are conducted by the General Medical Council for the UK, the Australian Medical Council for Australia and New Zealand, and the Liaison Committee for Medical Education (USA and Canada), but many other jurisdictions have, or are developing, strong accreditation processes.   There are also global standards developed and promoted by the World Federation of Medical Education (WFME), which map reasonably well to most standards. While the World Federation of Medical Education is not an accrediting body, there are moves to mandate that accreditation standards and processes must comply with the WFME global standards for graduates to be eligible for recognition across jurisdictional borders (Karle, 2006).

There are therefore two broad categories of curriculum review. The first is that conducted by medical schools, new and old, to develop, maintain or refresh curricula that are current and fit for purpose. This should be a continuous process, with changes based on some kind of evidence, ideally evaluation data. The second category is that conducted by regulatory bodies during accreditation processes, in which the curriculum is always a major focus. For both categories, a broader, more holistic view of a curriculum, rather than just content, should be adopted. This means that a curriculum review should seek information or data from much more than just descriptions of the subject content. This paper presents a framework for achieving this more holistic approach.

II. METHOD

This paper is based on an analysis of the structure of standards and accreditation protocols of the General Medical Council, the Australian Medical Council, the Liaison Committee for Medical Education and the World Federation for Medical Education. In each case medical programs are measured against several standards, where only one standard might specifically address curriculum content, but other standards address delivery, assessment and evaluation. Sources of evidence for a curriculum review may therefore be found when considering almost all standards.

A. A framework for reviewing a medical curriculum

Although a curriculum should be well described in writing, such documents are a single source of information about what is intended. Judgements about curriculum content and process are best made through triangulation of information and data from a combination of potential sources that reflect a wide range of issues, as summarised in Table 1. Most of these sources should be readily accessible, although requires both electronic access (through a guest log in account) and a physical visit to inspect the facilities. Further information, particularly about implementation, can be obtained through observation of aspects of program delivery, such as teaching sessions and clinical examinations.

Constructive alignment of a curriculum, from the vision and mission through curriculum delivery and assessment, is important because it demonstrates that the curriculum is a more holistic, ‘connected’ entity. It shows that curriculum content, process and intended outcomes are planned and designed with an explicit intention to produce a particular kind of graduate. Ideally, the outcomes are the same as those of the accreditation body, although many schools will add some of their own. For example, while all schools in a particular jurisdiction may plan to produce ‘work ready’ graduates safe to enter postgraduate training, some may have additional outcomes relating to elite research performance or to meeting the needs of underserved populations, following the growing international trend towards social accountability (Boelen and Woollard, 2009).

There should be evidence of purposeful, theory-based educational design (Prideaux, 2003). There is a spectrum of pedagogical models, from separate subjects delivered to large groups by lectures, through to highly integrated (vertically and horizontally) programs delivered through interactive, small groups, following a case-based or problem-based learning model. While educators may have a preference for a particular model, all can work, so long as the content, delivery and assessment methods are done well. It is important to design the curriculum content and process to match the learners’ characteristics at entry. For example, school leaver programs tend to be longer and to have adjustment to university life and introductory foundation sciences early, followed by more integrated, clinically-immersed learning, whereas graduate entry programs commence with an assumption that students are ready to commence with the more integrated, clinically-oriented approach.

An additional consideration is cohort size, because interactive, small group models are difficult to deliver unless group size is appropriate (8-10 maximum?). This has implications for the physical facilities and intranet-based Learning Management System (LMS), because small group, interactive learning required larger numbers of tutorial rooms that are appropriately furnished and equipped, and accessible, flexible and interactive repositories of electronic learning resources.

Ideally, all learning outcomes are measurable – this may be a matter of wording – and then form the basis of assessment practices, such as method selection, blueprinting, item bank development and standard setting. It is important that an integrated curriculum has integrated assessment, otherwise students may focus on non-integrated sources (a ‘hidden curriculum’) rather than the curriculum. Finally, there should be evidence of evaluation processes that monitor the curriculum content and delivery. A medical curriculum should be a continuously evolving entity, with decisions for change based on the best available evidence. Such evidence may come from both the routine, annual or semester-based program-wide data on participation, and the more reflexive and exploration of specific questions or concerns that arise during academic years. There should be evidence of evaluation feedback being formally considered, with decisions to make changes and then evidence that the change has taken place and participants advised of the results of the evaluation.

Table 1. Framework for reviewing a medical curriculum

Table 1 includes the potential sources of information that should be sought when a curriculum is reviewed. This demonstrates the potential weakness of reviews based on only documents, because the documents describe what is intended to take place, not necessarily what does take place. Hence speaking with faculty (including part-time clinical teachers), students, employers and regulators can provide different information that describes the curriculum-in-action.   Also important is the direct observation of teaching sessions of various types and of clinical assessment, both in the workplace and in OSCEs. It is not unusual for application to vary widely due to local ‘modifications’, despite apparently similar, ‘standard’ descriptions.

III. CONCLUSION

Reviewing a curriculum should be a continuous activity to maintain currency and fitness for purpose. The review should adopt a more holistic approach that includes curriculum content, delivery and assessment practices, as well as resourcing. This paper presents a framework to guide curriculum reviewers the issues to consider and the potential sources of information on which to base judgements.

Notes on Contributors

Richard Hays is an experienced medical educator with qualifications in both medicine and education. He has contributed to or led the design of several medical education programs and has also conducted formal medical program reviews at approximately 20 institutions in the United Kingdom, Europe and the Asia-Pacific region.

Ethical Approval

Ethical approval is not sought because there is no data presented and no possibility of identification of individual patients or students.

Declaration of Interest

There is no conflict of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Boelen, C. & Woollard, R. (2009). Social accountability and accreditation: a new frontier for educational institutions. Medical Education, 43, 887-894.

Dolmans, D., De Grave, W., Wolfhagen I. & Van Der Vleuten, C. P. M. (2005). Problem-based learning: future challenges for educational practice and research. Medical Education, 39, 732-741.

Duckett, S. (2005). Health workforce design for the 21st century. Australian Health Review Quarterly, 29, 201-210.

Karle, H. (2006). Global standards and Accreditation in medical education: a view from the WFME. Academic Medicine, 81, S43-S48.

Prideaux, D. (2003). ABC of learning and teaching in medicine: Curriculum design. British Medical Journal, 326, 268-270.

 

Published online: 1 June, TAPS 2016, 1(1), 20-22
DOI: https://doi.org/10.29060/TAPS.2016-1-1/SC1011

Fong Jie Ming Nigel¹*, Gan Ming Jin Eugene¹*, Lim Yan Zheng Daniel¹*, Ngiam Jing Hao Nicholas¹*, Yeung Lok Kin Wesley¹* & Tay Sook Muay²

¹Yong Loo Lin School of Medicine, National University of Singapore, Singapore; ²Department of Anaesthesia, Singapore General Hospital, Singapore

* Joint first authors

Abstract

The surgical physical examination is a fundamental part of medical training. We describe our experience with a near-peer teaching program for surgical physical examination skills, which involved senior medical students tutoring junior students starting their clinical rotations. We assessed scores on an Objective Structured Clinical Examination of the abdominal, vascular, and lumps examination before and after teaching. There was improvement in scores for all examinations and overall positive feedback from all students. This suggests that near-peer teaching may be a useful adjunct to faculty-led teaching of clinical skills.

Keywords:         Near Peer Teaching; Physical Examination; Surgery

I. INTRODUCTION

Bedside clinical examination is fundamental to the practice of medicine. All 2^nd year medical students at the National University of Singapore (NUS) undergo a 1 month Clinical Skills Foundation Course at the end of the academic year. This course is their maiden clinical posting and aims to introduce fundamentals of history taking and physical examination before they transit to their clinical years (3^rd to 5^th year). Unfortunately, it is challenging for faculty to effectively impart these skills due to increasing student numbers, busy clinical workloads and time constraints, leading to inadequate opportunities for learners’ active participation (versus passive observation), providing feedback, reflection and discussion (Spencer, 2003).

Near-peer teaching (NPT), using senior students to teach juniors, may supplement faculty-led teaching and ameliorate these difficulties. It has shown success in both case-based learning and teaching physical clinical examination (Blank et al, 2013). Beyond circumventing faculty constraints, NPT may promote a greater degree of active learning, knowledge application, and opportunities to correct misconceptions. Near-peer tutors may better understand learner challenges and share personal experience in overcoming these challenges (cognitive congruence), and promote a conducive, collaborative learning environment (social congruence) (Lockspeiser et al, 2008). Tutors who become role models may also benefit from deeper learning of content (Ten Cate et al, 2007) coupled with the development of higher cognitive skills whilst tutoring – to teach is to learn twice!

We describe a NPT initiative to teach surgical physical examination skills to students who are encountering clinical patients for the first time, in the ward via (i) a practical examination skills workshop and, (ii) the use of a novel course-book developed by the near-peer tutors as a teaching aid.

II. METHODS

Forty second-year students undergoing their maiden clinical posting at the Singapore General Hospital, Singapore, participated in a one-day workshop with IRB approval and written consent (Singhealth Centralised IRB Exemption 2015/2248, 27 March 2015). Students were divided into three groups and participated in three 2-hour long physical examination stations in a round-robin format. The stations were – (1) Abdominal examination, (2) Peripheral vascular examination, and (3) Lumps (skin, neck, breast, and groin). Five final-year students planned the workshop and served as near-peer tutors. A course book that served as a learning aid was written by near-peer tutors and vetted by faculty members. It was designed with the goal of encouraging the student to move beyond knowledge acquisition to application and synthesis of knowledge. Each section of the book introduced the sequence and rationale behind the steps of each surgical physical examination via a series of questions. These questions aimed to facilitate: (1) Understanding of the clinical significance and relationship between examination findings, (2) Clinico-pathological correlation and (3) Reasoning from first principles.

Instruction was modelled on Peyton’s four-step approach. Near-peer tutors explained the background to each examination, demonstrated the examination steps once, and discussed the technique, rationale, and possible findings in each step. Students then practiced on each other under supervision. To consolidate learning, near-peer tutors conducted post-tests based on objective structured clinical examination (OSCE) templates provided by the National University of Singapore, and provided qualitative feedback. Students also undertook identical OSCE pre-tests for comparison, and voluntarily completed anonymous feedback forms that made use of a 5-point Likert scale (Strongly Disagree to Strongly Agree) to evaluate various aspects of the workshop.

Statistical analysis was performed in R. Test scores were percentages of maximum possible score. Paired differences between individual pre-test and post-test scores were analyzed using Bayesian Estimation on weakly informative normal priors. Posterior probability distributions were approximated using Markov Chain Monte Carlo with 100,000 resamples.

III. RESULTS

Table 1. Summary of pre-test and post-test results, and paired difference indicating the improvement from pre- to post-test

  *Two students had to leave early

OSCE scores improved after teaching (Table 1), most markedly in the arterial examination (+49.7, credible interval 41.7-57.8), and also in the lumps (+34.7, credible interval 27.6-41.5) and abdominal examination (+17.3, credible interval 10.0-24.6). The abdominal examination started with the highest mean pre-test scores (73.3%), compared to the arterial (41.6%) and lumps examinations (55.7%). This may reflect greater exposure to the abdominal examination during prior faculty-led tutorials.

Out of the thirty-four students who provided feedback. 33 (97%) students agreed that the workshop helped them to pick up basic physical examination skills, and 32 (94%) were now more confident in performing these examinations on a real patient. Notably, all 34 students (100%) described better understanding of the rationale behind each clinical examination step, as opposed to performing the examinations mechanically; all (100%) found the small-group format conducive for learning. While all (100%) would encourage their juniors to attend this workshop next year, only 12 (35%) expressed interest to be a mentor themselves.

IV. DISCUSSION

We describe a student-initiated NPT initiative that supplements a faculty-led two-week surgical clinical rotation and physical examination teaching for students commencing their maiden clinical posting. This initiative has benefits to both students and near-peer tutors.

With regards to students, this initiative improved OSCE scores and was well received. The marked improvement in OSCE scores in areas less well-covered during faculty-led tutorials suggests that NPT sessions focusing on these gaps may complement and augment faculty-led tutorials particularly well. Physical examination stations with poorer OSCE scores may also highlight and objectively reflect areas in curriculum where faculty should focus on and refine. While it is encouraging that a number of students were keen to be mentors in the future, we are unsure as to why this aspiration was not a unanimous one amongst the entire student group. Possible reasons may include a lack of familiarity with the expectations of clinical teaching and lack of confidence given that this is their maiden clinical exposure.

Although the benefits to near-peer tutors were not formally evaluated, we propose that preparation of the course book and execution of the workshop required them to revisit their pre-clinical knowledge, understand its application to their current clinical knowledge, and synthesize and crystalize all the information to present it effectively, thus reinforcing their own learning.

With regards to the use of teaching aids, near-peer tutor developed examination revision notes for final year students accompanying NPT has been described (Rashid et al, 2011). Our course book, however, is a unique intervention that is primarily aimed at facilitating the building of links between pre-clinical knowledge and first clinical exposure. Future evaluation is necessary to determine the objective benefits of such a teaching aid.

Study limitations include a small sample size, potential observer bias because workshop tutors were OSCE assessors, and the lack of a comparison arm (e.g. Faculty teaching). These limitations can be addressed in the future by: Increasing our sample size by gradually extending subsequent editions of the workshop to all local teaching hospitals, recruiting more near-peers to serve as independent assessors to eliminate observe bias, and designing a study to compare faculty teaching alone against NPT in addition to faculty teaching.

We hope to highlight this valuable, yet under-utilized teaching modality which may be uniquely valuable in addressing the challenges faced in teaching clinical skills. We are optimistic that future studies may detail its academic, non-academic, and logistical benefits to students, near-peer tutors and faculty members alike.

Notes on Contributors

Fong Jie Ming Nigel, Gan Ming Jin Eugene, Lim Yan Zheng Daniel, Ngiam Jing Hao Nicholas and Yeung Lok Kin Wesley are final year medical students at the NUS Yong Loo Lin School of Medicine. They have an interest in medical education and have been involved in coordinating and implementing various near-peer teaching efforts in medical school. They conceived this initiative and were responsible for its implementation and preparation for publication.

Associate Professor Tay Sook Muay is a senior consultant anesthesiologist at the Singapore General Hospital (SGH), and the current Associate Dean for NUS Yong Loo Lin School of Medicine, SGH Campus. She is highly active in local and international teaching programs, education meetings and pedagogical committees. She is the faculty mentor for this near-peer teaching initiative.

Acknowledgements

We wish to acknowledge Cheng Kam Fei, Melissa Tang, and Krissie Chin from the Singapore General Hospital Associate Dean’s Office, for logistical support for the program.

Declaration of Interest

This is an unfunded study. All authors have no potential conflicts of interest.

References

Blank, W. A., Blankenfeld, H., Vogelmann, R., Linde, K., & Schneider, A. (2013). Can near-peer medical students effectively teach a new curriculum in physical examination? BMC Med Educ, 13, 165.

Lockspeiser, T. M., O’Sullivan, P., Teherani, A. & Muller, J. (2008). Understanding the experience of being taught by peers: the value of social and cognitive congruence. Adv Health Sci Educ Theory Pract, 13(3), 361-72.

Rashid, M. S., Sobowale, O., & Gore, D. (2011).  A near-peer teaching program designed, developed and delivered exclusively by recent medical graduates for final year medical students sitting the final objective structured clinical examination (OSCE). BMC Med Educ, 11, 11.

Spencer, J. (2003). Learning and teaching in the clinical environment. BMJ, 326(7389), 591-4.

Ten Cate, O., & Durning, S. (2007). Peer teaching in medical education: twelve reasons to move from theory to practice. Med Teach, 29(6), 591-9.

 

Published online: 3 January, TAPS 2017, 2(1), 25-28
DOI: https://doi.org/10.29060/TAPS.2017-2-1/SC1001

Dipanshi Patel¹, Namrata Baxi², Abhishek Agarwal², Kenyetta Givans¹, Krystal Hunter², Vijay Rajput³ & Anuradha Mookerjee²

¹Cooper Medical School of Rowan University, Camden, New Jersey, United States of America; ²Cooper University Hospital, Camden, New Jersey, United States of America; ³Ross University School of Medicine, Miramar, Florida, United States of America

Abstract

Introduction: In graduate medical education, trainees have different academic and professional growth needs throughout their career, but these needs have not been well studied (Gusic, Zenni, Ludwig & First, 2010). Traditional mentoring programs in many disciplines including medicine, science, law, business and education report individuals with mentors having higher earnings, higher job satisfaction and higher rates of promotion, compared to individuals without mentors (Bussey-Jones et al.,2006; Sambunak, Straus & Marusic, 2010).

Methods: We developed a structured mentoring program in the Department of Medicine in Cooper University Hospital which encourages both academic and professional growth through a major emphasis on academic scholarship. We created a 21 questions survey to evaluate mentee satisfaction towards their assigned mentors. The questions fit into four categories consisting of the mentor’s personal attributes and action characteristics and mentee’s short term and long term career goals. Sixty junior trainees (Post Graduate Year 1-3) and 39 senior trainees (Post Graduate Year 4-7) completed the survey.

Results and Conclusions: Senior trainees were more satisfied with their mentors’ intrinsic qualities (96%) compared to junior trainees (93%), c² (1, N=980) = 5.72, p=0.017. Additionally, senior trainees were more satisfied with their mentors’ action characteristics (95%) compared to junior trainees (91%), c²(1, N=677) = 4.03, p=0.045. Junior trainees had a lower satisfaction rating, compared to their senior colleagues, which might imply that their needs and desires were not properly addressed by their mentors. Both junior and senior trainees identified the lowest satisfaction rates in their mentors’ communication skills and ability to challenge them. This was an area of weakness within the mentorship program which requires further research and attention.

Keywords:        Mentoring; Graduate Medical Education; Assessment

Practice Highlights

• Traditional mentoring programs in many disciplines including medicine, science, law, business, and education report mentees having higher earnings, higher job satisfaction and higher rates of promotion, compared to individuals without mentors (Bussey-Jones et al., 2006; Sambunak et al., 2010).
• There have been many variations to the mentorship framework, but there is a lack in scientific evidence to conclude which aspects of such a program holds the most beneficial characteristics (Bussey-Jones et al., 2006; Gusic et al., 2010).
• The professional and personal development needs for trainees change as they progress in their medical training.
• There is need for faculty development to enhance communication skills between mentor and mentee.

I. INTRODUCTION

Mentoring is an integral part of academic medicine and professional development during graduate medical education (Sambunak et al., 2010). Traditional mentoring programs in many disciplines including medicine, science, law, business, and education report individuals with mentors having higher earnings, higher job satisfaction and higher rates of promotion, compared to individuals without mentors (Bussey-Jones et al., 2006; Sambunak et al., 2010). Unfortunately, mentoring in academic medicine is often undervalued and not well studied (Sambunjak et al., 2006). Additionally, while many trainees and faculty form mentoring relationships independently, there is a lack of formal mentoring of postgraduate trainees in medicine (Sambunjak et al., 2006).

In recent years, many different forms of mentoring programs have been established in medical training. Omary et al. (2008) described multi-leveled mentorship as a way of “mentoring the mentors” which allowed for more well-rounded and thorough relationships. Conversely, Bussey-Jones et al. (2006) describes other organizations that established peer-to-peer mentoring programs which replace the claimed hierarchy and exploitation of traditional mentorship with mutual gain and friendship. However, these programs struggled with conflicts of competition to secure funding, publications, and networking opportunities. Gusic et al. (2010) explored the idea of an informal mentorship that allowed flexibility and fluidity in the mentorship, but later found that formal mentorship with structure and expectations was more beneficial than an informal one (Gusic et al., 2010). There have been many variations to the mentorship framework, but there is a lack of scientific evidence to conclude which aspects of such a program holds the most beneficial characteristics. Our mentoring program is designed to foster relationships between faculty and trainees regarding personal, clinical, and career goals. Academic growth is a major emphasis of this program for those trainee physicians who wish to follow an academic career pathway. With this research, we hope to better understand the dynamic between faculty and trainees in graduate medical education. We hypothesize that there will be no difference in satisfaction rates between junior and senior trainees in regards to their respective mentors.

II. METHODS

A. Participants

Participants consisted of 60 junior trainees (PGY 1-3) and 49 senior trainees (PGY 4-7) who were assigned to clinical facility mentors through a formal mentoring program in the Department of Medicine at Cooper University Hospital, Camden, NJ, for the 2013-2014 academic year.

B. Procedure

We have a formal mentoring program to foster relationships between faculty and trainees in Department of Medicine since 2011. PGY1 trainees were assigned a mentor within the first three months of their internship. Senior trainees were required to seek and identify a professional or academic mentor, preferably across the division and department boundaries. Program Directors and the Division Head of Medical Education helped to identify the mentors and mentees and establish the partnership. A list of mentor/mentee was developed and distributed to all the house staff. Mentor and mentee were required to sign a contract and meet in person for a one hour session at least two times a year at a local restaurant using provided meal vouchers. All meetings between mentor and mentee were recorded in narratives and brief encounter forms. They could continue their relationship outside this program as per their needs.

C. Instrumentation

We conducted IRB approved research by developing an anonymous, structured questionnaire modified based on a validated survey (Archer, Swanwick, Smith, O’Keeffe & Cater, 2013). The survey addressed the quality and satisfaction ratings of the existing mentoring program by identifying 21 areas of mentor qualities and attributes, using a four point agreement scale. The survey questions were grouped into categories that reflected traits which the mentors exhibited or goals which the mentors encouraged. The categories consisted of two competing groups: “personal attributes” versus “action characteristics” and “short-term goals” versus “long-term goals”(Archer et al., 2013) Table 1 lists the questions from the survey and the qualities and traits which correlated with them. Question 6, 11 and 17 from the survey were excluded from data analysis due to low response rates.

Note: Modified from Archer et al., 2013
Table 1. Evaluation of principal mentor attributes

D. Data Analysis

We used Chi Square tests for statistical analyses. We also used reliability testing with Cronbach’s alpha to be sure that the question for each category was internally consistent. We compared the perception of junior trainees and senior trainees regarding their mentors’ personality traits and their mentors’ ability to address their long term and short term goals. We also analyzed the difference in satisfaction between trainees with known research interest versus no research interest.

III. RESULTS

When comparing satisfaction rates, senior trainees (PGY 4-7) were more satisfied (96%) compared to junior trainees (PGY1-3) (93%) regarding their mentors’ inherent qualities, c² (1, N=980) = 5.72, p=0.017. Additionally, senior trainees were more satisfied (95%) compared to junior trainees (91%) regarding their mentors’ actions characteristics, c² (1, N=677) = 4.03, p= 0.045. There were no statistical differences in mentor satisfaction rates irrespective of prior established research interest among trainees. When comparing PGY 1 to PGY 2-7 to identify if there were special needs for the first year junior trainees, there were no differences in satisfaction rates regarding how the mentees perceived their mentors and their career goals. The lowest satisfaction rates among all mentees (PGY 1-7) were regarding their mentors’ communication skills and ability to challenge them for their professional growth and career. Of all mentees, 13% did not believe their mentor showed ability to communicate with clinical supervisors and 14% did not believe their mentors adequately coached them on communication skills. Of all mentees, 11. % did not believe their mentor had the ability to take their supervision beyond a tick box exercise and 13. % did not believe their mentor was able to adequately challenge them.

There was strong reliability for each category of questions. The Crobach’s Alpha =.914 for the 9 items measuring “Inherent characteristic”. The Crobach’s Alpha =.915 for the 7 items measuring “Action characteristics”. The Crobach’s Alpha =.872 for the 4 items measuring “Short-Term”. The Crobach’s Alpha =.892 for the 4 items measuring “Long Term”.

IV. DISCUSSION and CONCLUSION

With this mentorship program, we hoped to foster the scholarly development of apprentices in graduate programs (Bussey-Jones et al., 2006). From the results of our survey, we see that senior trainees (95%) are more satisfied with their mentors learned skills compared to junior trainees (91%). While a mentor’s personal attributes are intrinsic within themselves and their nature, a mentor’s action characteristics (learned skills) can be improved through various training sessions and workshops (Archer et al., 2013). By identifying these areas of weakness and matching them with a plan of action, we can strengthen the mentorship program and provide services to accommodate mentees’ needs. Senior trainees (96%) also found more satisfaction in their mentors’ intrinsic characteristics than junior trainees (93%). It is possible that senior trainees are closer in maturity and life stage to their mentors or have more in common with their mentors compared to junior trainees. This highlights a potential area of improvement since it shows trainees have various needs and goals and we need to better address them. This finding warrants additional investigation to further the scope of the current research.

Junior and senior trainees (PGY 1-7) had lowest satisfaction rates regarding mentors’ coaching abilities in communication skills (13.5%) and mentors’ ability to communicate with clinical superiors (12.5%). Additionally, both junior and senior trainees (PGY 1-7) had the low satisfaction rates regarding mentors’ ability to challenge them (13.1%) and their ability to supervise them beyond a checkbox exercise (11.2%). Both of these components, communication and ability to challenge, are relatively teachable qualities, thus mentors might benefit from having educational workshops, which focus on those components of the mentoring experience.

Limitations which we encountered in our study included obtaining data from only one institution and having a small sample size. Additionally, the method with which the junior and senior trainees were matched with their mentors differed, which might impact the satisfaction rates. This was done because junior trainees are not familiar with the program’s faculty members and are less certain about which subspecialty they are interested in pursuing, compared to senior trainees.

Mentorship is a dynamic dyad interaction with immediate and long term impact. Establishing a mentor program fosters the scholarly development of apprentices in graduate programs (Sambunjak et al., 2006). Our program consisted of junior and senior trainees who might desire different skills and attributes from faculty mentors depending on their career goals and stage in training. Our aim was to identify areas of our mentor program which needed attention and restructuring to ensure a stronger program for the future. This research will help in developing future faculty development and mentoring programs across graduate medical education.

Notes on Contributors

Dipanshi Patel is a third year medical student at Cooper Medical School at Rowan University. She conducted literature review, assisted with data collection and data analysis, and contributed to the construction of short communications manuscript.

Dr. Namrata Baxi, MD is currently doing her specialty training in Nephrology at Robert Wood Johnson University in New Jersey. She collected and analyzed the data for this research while she was a senior post graduate trainee at Cooper University hospital.

Dr. Abhishek Agarwal, MD is currently a Chief resident at Cooper University hospital in the department of Internal Medicine. He assisted in collecting the data and analyzing it.

Kenyetta Givans is a third year medical student at Cooper Medical School at Rowan University, who contributed to the collection of the data.

Ms. Krystal Hunter, MBA is a statistician at Cooper Research Institute, and has contributed to the statistical analysis of this project. She has a valued interest in medical education research.

Dr. Vijay Rajput is Professor and Chair of Medicine at Ross University School of Medicine. He served as senior mentor for this research study. He has an extensive experience in medical education and has a passion for the holistic development of student and faculty.

Dr. Anuradha Mookerjee is Associate Professor of Medicine at Cooper University Hospital, in the Department of Internal Medicine. She helped in the design and implementation of this study. She has a marked interest in medical education and faculty development.

Acknowledgements

We have no additional acknowledgements.

Declaration of Interest

There is no conflict of interest, including financial, consultant, institutional or otherwise for any of the authors involved in this manuscript.

References

Archer, J., Swanwick, T., Smith, D., O’Keeffe, C., & Cater, N. (2013). Developing a Multisource Feedback Tool for Postgraduate Medical Educational Supervisors. Medical Teacher, 35, 245-154.

Bussey-Jones, J., Bernstein, L., Higgins, S., Malebranche, D., Paranjape, A., Genao, I., Lee, B., & Branch, W. (2006). Repaving the Road to Academic Success: The IMeRGE Approach to Peer Mentoring. Academic Medicine, 81, 674-679.

Gusic, M. E., Zenni, E. A., Ludwig, S. & First, L. R. (2010). Strategies to Design an Effective Mentoring Program. The Journal of Pediatrics, 156, 174-176.

Omary, M. B. (2008). Mentoring the Mentor: Another Tool to Enhance Mentorship. Gastroenterology, 315, 13-16.

Sambunak, D., Straus, S. E., & Marusic, A. (2010). A Systemic Review of Qualitative Research on the Meaning and Characteristics of Mentoring. Journal of General Intern Medicine, 25, 72-78.

Published online: 3 January, TAPS 2017, 2(1), 21-24
DOI: https://doi.org/10.29060/TAPS.2017-2-1/SC1017

Helen Naug¹, Natalie Colson¹, Andrew Pearson¹, Eugene DuToit¹, Grace Qi²

¹School of Medical Science, Griffith University, Australia; ²School of Languages Humanities and Social Sciences, Griffith University, Australia

Abstract

As undergraduate allied health programs continue to expand, there is increasing reliance on laboratory tutors, to engage, mentor and teach students. The bioscience laboratory is a specialised, tightly regulated learning environment and a potentially rich learning platform for students, but when the laboratory tutors are inadequately trained, the flow-on effects to the students can produce unfavourable learning outcomes. Traditionally, new tutors attend a generic sessional training workshop offered by the university. Due to the added health & safety considerations and the type of learning (kinaesthetic) that occurs in the laboratory, we identified a need for a specific professional development program for laboratory tutors. Our aim was to develop a program that allowed tutors to work within a framework tailored for laboratory teaching and, in a supportive environment, build teaching skills that promote student learning. Based on Lave’s theory of situated learning (Lave, 1991) and the recognised approach of supported reflective practice (Bell, 2001) we placed opportunities for peer observation at the centre of the program. The framework presented here (3P’s) was developed specifically for laboratory tutors and served as behavioural guidelines for peer observation sessions. Ten participants volunteered for the pilot program which concluded in 2015. Peer observation records and focus group data were gathered to evaluate the pilot program. Tutor feedback has initiated improvements in the program, and due to its overall success, it has been embedded into the recruitment/induction process for laboratory tutors in our department.

Keywords:        Peer Observation, Laboratory Teaching, Bioscience Education, Professional Development

I. INTRODUCTION

In the laboratory teaching environment, creating a safe learning space for inexperienced undergraduate students is a significant responsibility for the laboratory tutor/instructor. While the university provides a generic teaching and learning workshop centrally, we identified the need for a specifically tailored framework for training the laboratory tutors. In response to this need we developed

1) A framework listing the professional qualities of a laboratory tutor (3P’s matrix) and

2) An efficient on-the-job training program (peer observation)

A. Laboratory learning environment

The need for discipline-specific academic training programs is gaining recognition, particularly for laboratory-based learning (Good et al, 2015). Due to the potential hazards encountered in the laboratory environment there is a certain behavioural code of conduct for laboratories that tutors need to teach and model without coming across as ‘authoritarian’. Tutors need to be able to coach students as they learn skills kinaesthetically, and guide them to make connections between the laboratory and lecture content.

Situated learning and reflective practice served as the theoretical underpinnings of our peer observation design. In her theory of situated learning, Lave describes learning as a ‘social phenomenon constituted in the experienced, lived-in world, through legitimate peripheral participation in ongoing social practice’ (Lave, 1991, p64). In addition to situated learning, our approach emphasises reflective practice where activities involving observation ‘… encourage shared critical reflection on real-life teaching experiences-and can lead to transformation of both perspective and practice.’ (Bell, 2001).

This report details the framework and methodology for implementing this program which was piloted in 2014/2015 and, due to its success, is now embedded in our department as a prescribed training program for novice laboratory tutors.

II. METHODS

Formal ethical clearance was obtained before the commencement of the pilot program. Sessional laboratory tutors/instructors were invited to be part of the professional development program at the commencement of the teaching semester in 2014/2015. Ten tutors (4 experienced and 6 novice tutors) volunteered for the program.

A. Induction workshop

As part of the program, a specifically designed two-hour induction workshop was delivered to the participants which highlighted effective teaching practices in the laboratory; it also included an explanation of the peer observation process. Two of the project team members (NC and HN) designed a matrix to create some structure for the tutoring role, termed the 3P’s framework. 3P’s is an acronym for preparation, participation and professionalism, key terms which categorise observable qualities of good laboratory teaching (see Table 1). All the resources (including the 3P’s matrix) were made available to the program participants.

Table 1. 3P’s Matrix

B. Formation of the triad

An excel spreadsheet was electronically shared with the participants who filled in times indicating their availability to be observed, and to observe another tutor or peer. Course convenors also made themselves available to observe the participants as an “expert”. Once a triad was formed (observer, observee and expert) the observation session would proceed.

C. Peer observation process

The peer observation process was modelled on the university’s peer review of teaching program (Martin & Double, 1998) which most of the academic team in this program had experienced first-hand. The modified process we developed for sessional laboratory tutors included 3 episodes:

1) Observee prepares a summary of the teaching session

2) Peer observer and “expert” observer attend the session and observe the tutor and complete the feedback peer observation sheet which includes alignment with the 3P’s matrix and any other comments

3) The triad decide on a time to meet and give feedback (30 minutes) which is then followed by a second observation usually 3-5 weeks later

D. Evaluation

1) Focus group discussion: Five of the participant tutors ( 3 novice and 2 experienced) volunteered to discuss their experience of the program in a 10 minute focus group session which took place the final week of semester after completion of the program. The focus group discussion was recorded, transcribed and analysed for relevant themes. The analytical procedures were based on the Conventional Content Analysis suggested by Hsieh and Shannon (2005).

2) Peer observation sheets: The peer observation sheets were analysed for tutor’s observed alignment to the 3P’s matrix.

III. RESULTS

A. Focus group discussion

1) Theme 1- Receiving feedback:

Participants recognized the need to receive feedback on their performance as tutors and agreed that it was a good initiative:

“In previous years there was never any critical feedback on our performance as tutors… and I always wondered how we know whether our performance is good or bad or how to improve…”

“It’s a two-way process…you get feedback when you are observed, but when I was observing someone I realised that I could learn so much… you don’t just pick up on their mistakes but you can learn new ways of doing things to improve your own teaching”

2) Theme 2- Expectations:

Participants were not sure what to expect from the program.

“I was hesitant about starting the program because I felt that the labs I was going to be observed in I would be very anxious…doubtful in my knowledge and confidence…and I didn’t think I would be able to perform well…but as it turned out I was really calm”

3) Theme 3- Preparation and professionalism:

Participants feel the program helps formalise their teaching and give it meaning.

“ One experience I had in one of the first lab sessions was quite negative in that I couldn’t troubleshoot the software problems the students were having…I spent most of the session trying to understand the software because I hadn’t properly prepared myself on how to use it…through that lab so many students were having problems and I felt helpless as a tutor and disappointed in myself…and I don’t think I would have reflected on that experience so greatly if I hadn’t been part of the program…I would have just taken it as a bad run..”

“I received some critical feedback on health & safety issues that I had disregarded…I simply forgot to take notice…usually I am so strict with health & safety… but I had become complacent…”

“I liked the way we had the matrix as something concrete to focus on…”

B. Peer Observation sheets

Based on the peer observation reports, participants were able to demonstrate moderate or significant evidence of the key attributes that we identified as important for the role of laboratory tutor, as defined in the 3P’s matrix. Only in 8 of the 60 observed teaching sessions did the observed tutor demonstrate insufficient evidence of a key attribute. We can infer from these peer observations that on the whole, the tutors had made an effort to model the attributes, and initiate improvements in their approach to teaching.

IV. DISCUSSION

This report details a professional development program for laboratory tutors which incorporates strategies based on peer observation (situated learning) and feedback sessions (reflective practice). Results show that tutors in the program relied on the 3P’s matrix in preparation for the observed sessions (over 90% of the teaching sessions observed showed alignment with the 3P’s matrix).This indicates that with the 3P’s framework tutors can learn “on the job” and form new practices to improve their teaching; a demonstrable outcome of situated learning. In addition, tutors agree that they “learn something” by observing their peers, another key component of situated learning (Lave, 1991).

Focus group discussions indicate that tutors appreciated the critical feedback regarding their teaching performance which generally resulted in improved performance in the subsequent observations. The feedback sessions encouraged reflective practice, leading to changes in “perspective and practice” (Bell, 2001). Not only did tutors acknowledge errors in their practice but developed confidence in practice initiatives. Such changes in attitude and practice are desirable outcomes in any professional development program..

V. CONCLUSION

This program is innovative as it focuses specifically on the professional development of laboratory tutors. It has fulfilled its primary purpose to create a framework to assist tutors to develop skills for teaching in bioscience laboratories. As we continue to encourage the development of expertise among tutors, experienced academics involved in the program also improve their skills by contributing to what essentially is a community of practice. Future evaluations of the program will incorporate a measurable comparison between the two observation episodes for novice tutors, based on the 3P’s matrix. This will more clearly demonstrate, and help quantify, improvements in performance. The overriding aim of this program is improved learning in undergraduate laboratories, and our future course evaluations will target measurable student feedback on laboratory learning.

Notes on Contributors

Helen Naug PhD is a senior lecturer and academic lead for the common foundation year program for the Health faculty. Natalie Colson PhD is senior lecturer in Genetics and program director for the Bachelor of Biomedical Science program. Andrew Pearson PhD is senior lecturer and first year co-ordinator for the School of Medical Science. Grace Qi is a PhD candidate for the school of languages, humanities and social sciences. Eugene DuToit is associate professor and program director for the Bachelor of Biomedical Science with Honours program.

Declaration of Interest

The authors declare that they have no competing interests.

References

Bell, M. (2001). Supported reflective practice: a programme of peer observation and feedback for academic teaching development. International Journal for Academic Development, 6(1), 29-39.

Good, J., Colthorpe, K., Zimbardi, K., & Kafer, G. (2015). The Roles of Mentoring and Motivation in Student Teaching Assistant Interactions and in Improving Experience in First-Year Biology Laboratory Classes. Journal of College Science Teaching, 44(4), 88-98.

Hsieh, H. & Shannon, S. E. (2005). Three approaches to Qualitative Content Analysis. Qualitative Health Research, 15(9), 1277-1288. doi: 10.1177/1049732305276687

Lave, J. (1991). Situating learning in communities of practice in Resnick, Lauren B. (Ed); Levine, John M. (Ed); Teasley, Stephanie D. (Ed), Perspectives on socially shared cognition. (pp. 63-82). Washington, DC, US: American Psychological Association, xiii, 429 pp. http://dx.doi.org/10.1037/10096-003

Martin, G.A. & Double, J.M. (1998). Developing Higher Education Teaching Skills through Peer Observation and Collaborative Reflection. Innovations in Education and Training International, 35(2) 161-170.

Published online: 3 January, TAPS 2017, 2(1), 16-20
DOI: https://doi.org/10.29060/TAPS.2017-2-1/SC1010

Giat Yeng Khee, Jia Yuan Tay, Sei Keng Koh, Melissa Mee Yin Chow, Wan Chee Ong and Paik Shia Lim

Department of Pharmacy, Singapore General Hospital, Singapore

Abstract

Aim: To assess trainee’s knowledge and practice behavior changes with the revised basic MMS training program.

Methods: Kirkpatrick’s four-level evaluation model was adopted in this study. A questionnaire was used to seek trainees’ feedback on the training program. For learning progression, trainees’ self- perception of their competency pre- and post-training, and results from a multiple-choice questions (MCQ) test were assessed.

Thirty-eight behavioral changes in the patient care delivery and problem solving competency cluster of the General Level Competency Framework (GLF) were compared pre- and 6 months post-training. Evaluation of level-4 results is pending as advanced training is in progress.

Results: Fourteen out of fifteen trainees who had completed the basic MMS training responded to the online questionnaire. Eleven trainees (78.6%) rated basic MMS training program as good or excellent. Positive response was highest in tutorials (84.3%), followed by continuous education sessions (70.6%) and clinic attachments (67.6%). Suggestions for improvement include increasing the frequency and opportunities of hands-on patient interview at clinic attachments.

Sixty-four percent of trainees perceived improvement (p<0.013) in delivering patient care post-training (median=3; IQR: 3-4) than pre-training (median=3; IQR: 2.75-3). All but one trainee passed the MCQ test (median score: 80; IQR: 76-86). A greater proportion of patient care delivery and problem solving behaviors (69.2% and 35.9% respectively) in GLF was graded highest performance at post-training compared to pre-training (35.3% and 14.3% respectively).

Conclusions: The revised training program is well-received by trainees and feedback from trainees provides insights into future improvements in the content and delivery of the program.

Keywords:        Medication Management, Pharmacist, Training, Kirkpatrick

I. INTRODUCTION

Pharmacist-provided Medication Management Service (MMS) aims to improve patients’ outcomes by promoting effective and safe medication use. Essential knowledge and skills in chronic disease management are required in providing good quality pharmaceutical care. An in-house 6-month MMS training program was first developed in January 2013 to impart essential skills of a competent MMS pharmacist. It focused primarily on self-directed learning, using reading materials provided by a sole trainer, amidst the heavy daily workload of trainees. Successful completion of the original training program therefore relied greatly on the trainees’ motivation and commitment. Consequently, completion rate within 6 months as a trained MMS pharmacist was unsatisfactory. Furthermore, the growth of other clinical pharmacy services was affected as the completion of MMS training was a pre-requisite before embarking on other specialized services.

As detailed in Diagram 1, a revised training program which adopted outcome-based education approach was implemented in April 2014. It was divided into basic and advanced phase, which each lasts for 6 months. The basic MMS training program aimed to facilitate learning and enhance competency of pharmacists in medication review and dispensing, while the advanced program aimed to ensure broad- based competency and clinical skills of pharmacists before embarking on pharmacist-provided ambulatory care services. In the revised program, there were significantly more engagement between the trainers and trainees in facilitating trainees’ learning. Prior to this study, a systematic evaluation of the MMS training program has not been conducted. As a first step in evaluating the effectiveness of the revised MMS training program, in this study, we aimed to assess trainees’ knowledge and practice behavior changes following completion of basic MMS training and to identify areas for improvement.

*Topics discussed during tutorials include cardiology, endocrinology, geriatric & nephrology
**Basic MMS training is compulsory for all outpatient pharmacists in SGH whereas the advanced MMS training is optional.
Diagram 1. Original and revised MMS training program in Singapore General Hospital

II. METHODS

In this study, the Kirkpatrick’s four-level evaluation model, developed for measuring training effectiveness, was utilized (Kirkpatrick, 1998). This systematic approach offers multiple levels of evaluation using different data sources and it allows evaluation of both qualitative and quantitative data. All pharmacists who had received basic MMS training in year 2014 were recruited to participate in the study. No ethic approval required based on Singapore Health Services centralized institutional review board. All four- level outcomes were measured as follows:

A. Kirkpatrick level I: Reaction

An online questionnaire, developed by a group of MMS trainers and trainees who were not recruited in the study, was administered to trainee post-basic MMS training. Trainees rated the effectiveness of the basic MMS training program including the various training modalities (tutorials, continuous education sessions and clinic attachments) on a 5-point Likert-scale and provided suggestions for improvement.

B. Kirkpatrick level II: Learning

The same online questionnaire was used to examine the trainees’ perception on the impact of the MMS training program, either positively or negatively affecting their competency in delivering patient care. Results of the multiple-choice questions (MCQ) test administered 6-month post training were examined. The MCQ test that consisted of 50 recall and case-based questions was developed by two MMS trainers, who were the content experts, so as to assess trainees’ clinical knowledge and reasoning. Questions were scrutinised to ensure that they represented an accurate measure of the learning outcomes.

C. Kirkpatrick level III: Behaviour

Changes in observed practice behaviours were assessed using the United Kingdom-adapted General Level Competency Framework (GLF). This is an assessment and developmental tool consisting of 63 behavioural statements distributed into 3 competency clusters: delivery of patient care, problem solving, and professionalism (Rutter et al., 2012). Thirty-eight behavior statements, representing the areas where basic MMS training could have a major influence on, were measured pre- and post-basic MMS training. The performance-level rating for each behaviour (1=rarely; 2=sometimes; 3=usually; 4=consistently) was based on the consistency in demonstration in practice, which was rated according to the assessor’s observation.

D. Kirkpatrick level IV: Results

Currently, the advanced MMS training is still in progress. In advanced MMS training, 15 case reviews by the trainees will be assessed. Trainee with satisfactory achievements in both the final MCQ test and 15 case reviews will be deemed eligible as trained MMS pharmacist.

E. Data Analysis

Descriptive statistics were used to summarize the trainees’ characteristics, reactions to basic MMS training programme and learning outcomes. Behavioural changes pre- and post-basic MMS training were assessed and compared using Wilcoxon signed rank test. SPSS, version 21 (SPSS Inc., an IBM Company Chicago, USA) was used for statistical analysis and a p-value of <0.05 was defined to be statistically significant.

III. RESULTS

Fourteen out of fifteen trainees who had completed the basic MMS training participated in this study. Half of the trainees completed their pharmacy undergraduate program in Singapore while the rest completed their studies overseas.

A. Kirkpatrick level I: Reaction

Fourteen trainees responded to the online questionnaire. Eleven trainees (78.6%) rated basic MMS training program as good or excellent. Positive response was highest in tutorials (84.3%), followed by continuous education (CE) sessions (70.6%) and clinic attachments (67.6%). Suggestions for improvement include increasing the frequency and opportunities of hands-on patient interview at clinical attachments.

B. Kirkpatrick level II: Learning

Nine trainees (64.0%) perceived significant improvement in delivering patient care post-training (median=3; IQR: 3-4) as compared to pre-training (median=3; IQR: 2.75-3) (p<0.013). As for the MCQ test conducted post-training, all but one trainee passed the MCQ test, achieving a median score of 80 (IQR: 76-86).

C. Kirkpatrick level III: Behaviour

Thirteen trainees had completed pre- and post-training GLF assessment for observed practice behaviours. Overall, 23 out of the 38 behaviors (60.5%) were found to have significant improvement at repeated observation post-training. Greater proportion of behaviors in delivering patient care cluster was graded   to perform consistently at highest performance level post-training compared to pre-training (69.2% vs 35.3%). Similar improvement was observed in the problem solving cluster where 14.3% and 35.9% of the behaviors were graded consistent performance at pre- and post-training respectively. There was no significant improvement observed in 3 behaviors as the trainees were practicing at the highest level during the pre-training evaluation. Performance level for other behaviors was maintained over the 6- month training.

IV. DISCUSSION

Assessment of the training program provides valuable information to trainers and its institution in seeking ways to improve the training program. The revised basic MMS training program has likely achieved its aim to provide broad-based training that enhances trainee’s competency in patient care as positive results were observed in trainee’s attitude, knowledge and performance (level 2 and 3). Feedback (level 1) from the trainees was encouraging and indicated that the revised training program with the inclusion of tutorials, CE sessions and clinic attachments, was well-received and trainees had perceived improvement in their competency at providing patient care. Engagement between trainers and trainees in our revised program has been increased, as compared to the original program. This increased engagement between the trainers and trainees may have enhanced the trainees’ learning experiences through regular feedback from the trainers and reflection of their learning process, thus resulting in better performance post-training. From the high rate of positive response from trainees, it demonstrated that such interaction with trainers was in favor to support learning.

Clinic attachment was ranked lowest amongst all training activities as trainees commented that the opportunities for clinic attachments were sparse, with 3 clinic attachments distributed over 6 months. However, increasing the frequency of clinic attachments is not feasible in view of the large pool of trainees and limited training resources. Proposed plans for enhancement of the program include restructuring tutorial or CE sessions to problem-based learning. Simulation cases could be used to facilitate the application of knowledge and prepare trainees better for hands-on experiences during clinic attachments.

There were limitations to our study. First, positive changes observed in attitude, knowledge and performance may not be solely contributed by the MMS training program due to the lack of a control group. Improvement in behaviors observed in GLF assessment could be contributed by other sources of training such as the 5-weekly outpatient clinical group discussion, feedback or suggestions from their GLF assessor, self-directed learning and lecture series from other continuing professional education provided by external parties. Second, pre-training MCQ test was not performed. Therefore, the extent of improvement in trainees’ knowledge level was not established.

V. CONCLUSION

Our assessment shows that the revised basic MMS training program is well-received by trainees and feedback from trainees provides insights into future improvements in the content and delivery of the program. The results of the advanced MMS training are pending, therefore, the effectiveness of the revised MMS program is yet to be evaluated.

Notes on Contributors

Khee Giat Yeng is a board certified pharmacotherapy specialist at Singapore General Hospital where she is involved in the provision of ambulatory care services in the medication management service, rheumatology monitoring clinic and anticoagulation clinic. She is also actively involved in teaching medical students, pre-registration pharmacists, pharmacists and post-graduate students.

Tay Jia Yuan is a pharmacist in Singapore General Hospital and is currently participating in the advanced training program for the medication management service in SGH.

Koh Sei Keng is involved in the provision of anticoagulation services at the ambulatory care clinic in the Singapore General Hospital.

Melissa Mee Yin Chow a senior pharmacist at Singapore General Hospital with a board certification in pharmacotherapy. She is involved with training of pre-registration pharmacists and registered pharmacists. She also provides ambulatory care services in medication management, anti-coagulation and haemodialysis disease management clinics.

Ong Wan Chee is a senior clinical pharmacist at Singapore General Hospital. She is actively involved in the provision of ambulatory care services including medication management service, patient outreach program, inflammatory bowel disease, and rheumatology monitoring clinic. She also teaches the medical students, pharmacists, pre-registration pharmacists and postgraduate students.

Lim Paik Shia is a principal clinical pharmacist, practicing in ambulatory care clinics at Singapore General Hospital. She is actively driving the pharmacist-provided clinics, in particularly medication management service to outpatients and hemodialysis patients. She is also a clinical educator for pharmacists, residents, postgraduate students and pre-registration pharmacists.

Acknowledgements

The authors would like to thank all MMS trainers for their contributions in this study. We would like to acknowledge the guidance provided by Mr Chang Wei Terk who had pioneered the original MMS training program and his instrumental role in the revision of the program.

Declaration of Interest

The authors declare no conflict of interest or any sources of funding for this study.

References

Kirkpatrick, D. L. (1998). Evaluating training programs: The four levels (2nd ed.). San Francisco, CA: Berrett-Koehler.

Rutter, V., Wong, C., Coombes, I., Cardiff, L., Duggan, C., Yee, M. L., Lim, K. W., & Bates, I. (2012). Use of a general level framework to facilitate performance improvement in hospital pharmacists in Singapore. American Journal of Pharmaceutical Education, 76(6), 107.

Published online: 2 May, TAPS 2017, 2(2), 30-33
DOI: https://doi.org/10.29060/TAPS.2017-2-2/SC1013

Akalanka P. Hettihewa, Indika M. Karunathilake & M. N. S. K. Perera

Medical Education Development and Research Centre (MEDARC), Faculty of Medicine, University of Colombo, Sri Lanka

Abstract

Introduction & Objectives: Faculty of Medicine, University of Colombo, Sri Lanka conducts MBBS and BSc physiotherapy degree programs following SPICES model where educators have to play diverse roles in order to fulfil their responsibilities. Looking at them from a student’s perspective will help understand what students perceive as important roles of educators and help the students understand the educators’ different capacities.  An effective medical educator possesses a range of qualities. Looking at them from students’ point of view indicates what they expect from teachers as professionals.

Methods: Harden and Crosby (2000) paper on 12 roles of a medical teacher was the conceptual framework for this study. Based on previous literature a questionnaire was developed with 20 different roles and 15 qualities of an educator. Study followed an analytical cross-sectional design with participation of 188 physiotherapy and medical undergraduates.

Results: Study findings indicated that the students’ perception on importance of different roles of teachers had a close overlap with Harden’s 12 roles. Information provider as a lecturer in class room in clinical settings, in practical settings, developing learning materials for lectures, clinical educators etc. Importance of being an examiner and curriculum planner was rated relatively low by students. Students perceived good communication skills, professional skills, knowledge and respect for patients as most important qualities in an educator.

Conclusion: Findings will help educators understand what learners expect from them and help students understand the different capacities of educators.

Keywords: 12 Roles of a Medical Teacher; Ideal Medical Teacher; Student Perception; Survey

I. INTRODUCTION

The purpose of teaching is to facilitate personal growth and development that impact the professional, social and political aspects of learners (Galbraith, 2004). Since global trend of medical education is moving towards a more humanistic approach to patient care, medical teachers need to become educators, interacting with individual students (McLean, 2001).

Faculty of Medicine, Colombo, Sri Lanka conducts a 5 year MBBS degree programme and a 4 year BSc. Physiotherapy degree programme. Both programmes follow innovative student-centred curricula and incorporate features of SPICES model in their teaching.  This context involves diverse learners, various teaching methods and a range of teaching settings. This student-centred learning environment expects the teachers to guide the learners with diverse abilities, knowledge and different levels of comprehension towards self-directed study. In order to fulfil their responsibilities in this context teachers have to play a variety of roles.

Harden and Crosby (2000) have identified that with rapid changes in medical education the role of the medical teacher is changing.  They have identified 12 roles of the medical teacher falling into six categories.

According to Cross (1995) discrepant perceptions of roles and behaviours in clinical education may have an adverse effect on student learning. So it is important that

students should have an understanding of different roles of the teacher and the faculty should understand the students’ perception on the different roles of a medical educator and how the students perceive the importance of different roles to them as undergraduates.

Successful completion of these demanding and complex roles requires teachers to possess a range of professional and personal qualities. Different studies have identified a variety of characteristics as the best qualities in a medical educator. Looking at them from a student’s point of view will help understand what students perceive as being important and what they expect from their teachers as educators and medical professionals.

The findings of this study will help students understand the different capacities of educators and how to get the maximum aid from their educators in these different capacities. Findings will also help the educators understand what leaners expect from them as teachers and medical professionals.

II. METHODOLOGY

Objectives of this study were to:

1. Identify the medical and physiotherapy undergraduates’ perception on importance of different roles of a medical teacher.
2. Identify the medical and physiotherapy undergraduates’ perception on important qualities in a medical teacher.
3. Compare the differences of students’ perception regarding the importance of different roles in a medical teacher based on their academic year.
4. Compare the differences of students’ perception regarding the importance of different roles in a medical teacher based on students’ programme of study.

This study followed an analytical cross sectional study design; it was carried out in faculty of Medicine, Colombo, Sri Lanka. At a confidence interval of 95% and a 5 % margin of error the target sample was 300 undergraduates representing three academic years of the faculty.

A self-administered questionnaire with 20 roles and 15 qualities of a medical teacher was developed to collect the data. Harden’s (2000) paper on 12 roles of a medical teacher was used as a conceptual frame to identifying the different roles of a medical teacher. Based on these 12 roles, 14 different roles of teachers were identified. To better suit the study setting and participants the learning facilitator role was re-categorized as facilitator of small group discussions (SGD) and facilitator of problem based learning (PBL) sessions, role of clinical or practical teacher was divided as information provider in clinical settings and information provider in practical settings. Other 10 original roles of Harden’s (2000) study were also included. Based on previous literature 6 more roles were added, as tutors, personal advisors, clinical educators, researchers, administrators and community health developers. Based on previous study findings, 15 important personal and professional qualities of a medical teacher were added to the questionnaire.

Students’ perception was obtained through a likert scale, 5 being very important, 4, considerably important to 1 being not important. Participants marked the importance of roles and qualities according to their perception. Data was analysed using the 16th version of SPSS. First and second objectives were analysed using descriptive analyses and third and fourth objectives were analysed using chi square.

III. RESULTS

Total of 188 students participated in the study at a response rate of 62.67% representing three academic years of the faculty.  Out of which 139 were MBBS undergraduates and 159 were females.

According to the responses, the students perceived following roles of medical teachers to be very important. Majority perceived the most important role of a teacher was being an information provider in clinical settings. Following roles are listed according to majority’s perception of their importance.

• Information provider in practical settings
• Information provider as a lecturer in class room
• Clinical Educator
• Developing learning materials for lectures (Presentations/ handouts)
• On-the-job role model (Clinic/ward)
• Role model in teaching setting
• Learning facilitator in SGDs
• Learning facilitator in PBLs
• Examiner of formal examinations
• Curriculum Planner
• Curriculum Evaluator

The roles teachers’ play as researchers, course organizers and resource developers were rated low by the students. Undergraduates’ perceived that the least important role of a medical teacher is of an administrator. Being a mentor and a personal advisor were also rated as less important by the participants.

There was no statistically significant difference found in students’ perception on different roles of a medical teacher based on the participants’ academic year. No statistically significant difference was found in students’ perception for majority of educators’ roles; however physiotherapy undergraduates had a more positive perception towards educators’ role as researchers and course organizers than MBBS undergraduates. Difference in perception was statistically significant (P > 0.05).

Students perceived that the most important qualities of a medical teacher were having, good communication skills, knowledge, professional skills and humanistic approach to patient care. Having good knowledge in technology was rated as the quality with lowest importance and having honesty and integrity were also rated low by the participants.

Figure 1: Undergraduates’ Perceived Importance of Different Roles of a Medical Teacher

IV. DISCUSSION

Students’ perception on importance of different roles of teachers had a close overlap with the 12 roles identified by Professor Harden (2000). However, students didn’t perceive the roles of being a mentor, course organizer, and resource material developer (study guides) as being important as Harden (2000) found. This might be due to the fact that students don’t get an opportunity to observe educators in these roles at the faculty and the responsibilities they have in these roles. Even though these roles have a strong impact on the curriculum and curriculum development educators don’t directly contact with students through these roles so the students might not perceive their importance as much as the faculty does.

Educators can play an important role as a mentor and a personal advisor to the students in their personal and professional lives. However students had rated both these roles as not being “very important” roles of a medical teacher. Can this be due to the fact that the students have not observed their educators in these roles frequently and closely? It may also be due to the fact that with the cultural and social factors involved in the Sri Lankan educational and clinical settings students are not familiar with seeking advice from their educators on personal issues.

Students had identified many of the qualities in the questionnaire as being “very important” for a medical teacher to have. However following medical ethics, honesty and integrity were ranked relatively low by the students. It is an interesting observation and the reasons for this should be identified.

Physiotherapy undergraduates had a more positive perception towards educators’ role as researchers and course organizers than MBBS undergraduates. This could be due to better student to staff ratio in the Physiotherapy training program. The Physiotherapy program has only 30 students per intake every year whereas MBBS program has 200 students in each intake. Therefore the physiotherapy students have more opportunity to work closely with their educators getting more exposure to their teachers involved in variety of different teaching roles as course organizers and researchers than the MBBS students.

V. CONCLUSION

Findings indicate that the participants understand that teachers play a range of important roles other than being just a teacher. They had also identified that to fulfil these roles educators need to possess a range of personal and professional qualities.

Findings from this study will help students understand the different capacities of educators and will also help educators understand what learners expect from them as medical professionals. This will lead to better student-teacher interactions.

Notes on Contributors

Ms. Akalanka Hettihewa is a physiotherapist from Faculty of Medicine, University of Colombo, Sri Lanka, currently employed at the Medical Education Development and Research Centre (MEDARC) Faculty of Medicine Colombo, Sri Lanka.

Dr. Indika Karunathilake is the director of the Medical Education Development and Research Centre (MEDARC) Faculty of Medicine Colombo Sri Lanka.

Dr. M.N.S.K Perera is a MBBS graduate from Faculty of Medicine, Colombo, currently working at the National Hospital Sri Lanka (NHSL) Colombo.

Ethical Approval

The approval for this study was obtained from the Ethics Review Committee, Faculty of Medicine, Colombo, Sri Lanka.

Declaration of Interest

The authors declare that they have no competing interests.

References

Cross, V. (1995). Perceptions of the Ideal Clinical Educator in Physiotherapy Education, Physiotherapy, 81(9), 506-513.

Galbraith, M.W. (2004). Adult learning methods: A guide for effective instruction (3rd ed.). Florida, Krieger publishing Company.

Harden, R.M., & Crosby, J. (2000). AMEE Guide No 20: The good teacher is more than a lecturer, the twelve roles of the teacher, Medical Teacher, 22(4), 334-347.

McLean, M. (2001). Qualities attributed to an ideal educator by medical students: should faculty take cognizance? Medical Teacher, 23(4), 367-370.

*Akalanka P. Hettihewa
110, Sri Dharmarama Rd, Fort, Matara
Tel: +94783641866
Email: akalankahettihewa26@gmail.com

Published online: 5 September, TAPS 2017, 2(3), 24-25
DOI: https://doi.org/10.29060/TAPS.2017-2-3/SC1042

Shahla Siddiqui

Department of Anaesthesia, Khoo Teck Puat Hospital, Singapore

Abstract

There is a gap between the teaching and practice of clinical ethics. Many students learn bioethics at an undergraduate level in lectures and through tutorials. They may also learn case based ethical reflection; however, discussing ethical issues of real ICU patients and families is a novel form of experiential teaching which engages undergraduate students and helps them see practical and quick consideration in very sick patients. Often ICU patients present complex and difficult ethical dilemmas which can demonstrate real life scenarios that students can relate to when learning in a classroom.

Keywords:      Ethics Teaching, Interprofessionalism, Experiential Learning

I. INTRODUCTION

The The Intensive Care Unit is an environment where myriad complex clinical situations present and these are often closely intertwined with ethical dilemmas (Svantesson, Anderzén-Carlsson, Thorsén, Kallenberg & Ahlström, 2008). ‘Ethical dilemmas’ constitute many complex situations which can surround a patient’s clinical care that often involve an apparent mental conflict between moral imperatives, in which to obey one would be opposing the other (Braunack Mayer, 2001). Situations range from triage, surrogate decision making, organ donation, family communication and patient autonomy. An example of this may be a family insisting on non-beneficial life sustaining therapy in an elderly patient. Many times such situations are not explicitly presented to medical students and they rarely get a chance to witness how doctors deal with real complex clinical ethics issues in a situation where time is essence (Oberle & Hughes, 2001). Coping with these problems is crucial to the development of medical students and an exposure to how a multi-disciplinary team of doctors, social workers, nurses and palliative care physicians may help solve hard cases is very useful. The interaction and cooperation between different professions also helps to consolidate the roles of each allied health team and highlights to the medical student an integrated professional attitude (Udén, Norberg & Lindseth, 1992).

The aim of this educational qualitative study was to evaluate the narrative feedback of medical students after being exposed to Ethics rounds in a 14 bedded surgical ICU of a teaching hospital by a multi-disciplinary ICU team.

II. METHODS

After IRB approval, medical students posted to our hospital for an Anaesthesia rotation were invited to participate in the study. A verbal consent was obtained with specific instructions to maintain patient confidentiality. The author who is an intensivist led the rounds of all ICU patients. This included a multi -disciplinary round comprising of nurses, social workers and allied health staff such as a pharmacist, a physiotherapist and a dietician. All staff had been conducting daily ICU rounds on the patients and were familiar with their ethical issues. The students were briefed about ethics issues that may have arisen in each patient’s case and how they were resolved by the team. There was input from the other team members especially the social workers and the nurses. Students were encouraged to reflect and ask questions about each case and offer their own input. Some cases were straightforward and did not have many ethical questions, however some others had complex problems that needed more time. Some issues were resolved already whilst others were ongoing and the students were encouraged to offer solutions based on their past learning. All students belonged to the fourth year of medical school and had received a longitudinal ethics teaching. At the end students were asked to write down an anonymous narrative feedback for the author.

III. RESULTS

Overall the six medical students were very happy to be part of the 1 hour ICU ethics round. They appreciated the real life practical application of the issues facing a team of professionals daily and the aspects of care we need to consider when resolving such issues.

“ I felt that the session was very informative and helped clarify many perceptions we had  towards the ICU.”

It reinforced their view of an inter-professional approach to ICU care:

“The involvement of other healthcare workers of the ICU team helped us better appreciate the multi- disciplinary nature of ICU care and the role each person plays.”

“Having the allied healthcare team there to offer their thoughts also brought a new perspective to patient care for me. I feel that these lessons are more valuable than what we may take away from a didactic lecture alone.”

They were able to appreciate the ethical challenges faced by the team:

“I really enjoyed it and it gave me an insight not only on the logistical side of the ICU but also the ‘heart’ behind it and the ethical dilemmas that the healthcare professionals go through every day.”

A patient perspective and human aspect was also highlighted by the input from nurses, physician and social workers which is different from the often cold clinical picture on rounds:

“It was interesting to learn about the various patients in the ICU and the challenges they themselves face in getting better.”

 “This enabled us to tie in the things that we had heard of during lectures and tutorials, and the many issues that we may have not been exposed first hand to.”

The practical aspects of clinical care was exposed:

“It was also a good way to see the practical aspects of ICU care ….being able to see different ICU patients and hear their stories we got a real feel of the ICU”.

“We also felt easy asking pertinent questions about their management to you.”

IV. DISCUSSION

Experiential learning and case based clinical bioethics teaching is a practical and effective  way of imparting explicit knowledge, skills and role modeling attitudes for medical students (Widdershoven, Abma & Molewijk, 2009). This should not be solely dedicated to ICU or for ethics. Many ethical issues faced in ICUs are commonplace in ward settings as well (Rathert & Fleming, 2008). Effective communication, empathy, cultural nuances and professionalism are all learning that medical students can pick up from such ethics rounds in the ICU.

Notes on Contributors

Shahla Siddiqui is a Senior Consultant and Intensivist as well as a graduate student in medical ethics.

Declaration of Interest

The author declares no conflict of interest.

References

Svantesson, M., Anderzén-Carlsson, A., Thorsén, H., Kallenberg, K., & Ahlström, G. (2008). Interprofessional ethics rounds concerning dialysis patients: Staff’s ethical reflections before and after rounds, Journal of Medical  Ethics, 34(5), 407-13.

Braunack Mayer, A. J. (2001). What makes a problem an ethical problem? An empirical perspective on the nature of ethical problems in general practice. Journal of Medical Ethics, 27 (2), 98-103.

Oberle, K., & Hughes D. (2001) Doctors’ and nurses’ perceptions of ethical problems in end-of-life decisions, Journal of Advanced Nursing, 33(6), 707-15.

Udén, G., Norberg, A., & Lindseth, A. (1992). Ethical reasoning in nurses’ and physicians’ stories about care episodes. Journal of Advanced Nursing, 17(9), 1028-34.

Widdershoven, G., Abma, T. & Molewijk, B. (2009) Empirical ethics as dialogical practice. Bioethics, 23(4), 236-48.

Rathert, C., & Fleming, D. A. (2008). Hospital ethical climate and teamwork in acute care: The moderating role of leaders. Health Care Manage Review, 33(4), 323-31.

*Shahla Siddiqui
Tel: 0065 66022137
Email:  shahlasi@yahoo.com