Submitted: 20 June 2019
Accepted: 12 September 2019
Published online: 5 May, TAPS 2020, 5(2), 5-13
https://doi.org/10.29060/TAPS.2020-5-2/OA2168

Simon Tso

Department of Dermatology, South Warwickshire NHS Foundation Trust, Warwick, United Kingdom

Abstract

Background: The transition experience of graduate-entry medicine degree programme students is less well understood as compared to those from undergraduate-entry medicine degree programmes.

Aim: This thematic analysis study aimed to explore the transition experience of graduate-entry medicine degree programme students at a United Kingdom medical school.

Methods: Twenty-one student volunteers from the University of Warwick 4-year graduate-entry medicine degree programme took part in this study with fourteen participants attended a further follow-up interview. Audio recordings of their semi-structured interviews were transcribed verbatim and analysed thematically.

Results: Results revealed three key transition periods within the University of Warwick Medical School’s graduate-entry medicine degree programme. Learning, professional identity development and managing coping strategies were the three key challenging issues dominating their transition experience. Medical students encountered a range of challenging issues throughout their medical school journey that could be categorised under three conceptual themes: challenges associated with the curriculum, challenges associated with their social role and generic life challenges.

Conclusions: The findings from this study could be useful to educators and medical schools in enhancing their student support services. It could also be useful to prospective and existing medical students in understanding the realities of undertaking a graduate-entry medicine degree programme.

Keywords:        Transition, Graduate-entry, Medical Student, Experience, Challenges, Identity

Practice Highlights

• There are three major transition periods within a graduate-entry medicine degree programme.
• Students could face challenges due to the curriculum, their social role and generic life events.
• Participation in patient care activities and observation of role models developed students’ identity.
  
  

I. INTRODUCTION

The term ‘transition’ originated from the Latin word ‘trānsitiō’ meaning ‘to go across’. This term had been defined by scholars in different ways. Parkes (1971) described psychological transition as the “abandonment of one set of assumptions and the development of a fresh set to enable the individual to cope with the new altered life space.” (p. 103). Levinson (1978) described transition as a “boundary zone between two states of greater stability. It involves a process of change, a shift from one structure to another” (p. 50). Schlossberg, Waters, & Goodman defined transition as “any event or non-event that results in changed relationships, assumptions and roles” (1995, p. 27). In the context of undergraduate students’ transition through their medical school journey, the literature described three key transition periods: the transition into an undergraduate medicine academic programme, the transition from the pre-clinical to clinical phase of an undergraduate medicine academic programme, and the transition into professional practice as newly qualified doctors (Radcliffe & Lester, 2003). Transitions have also been described within specific modules of the undergraduate medicine academic programme, for example, during dissection training (Lamdin, Weller, & Kerse, 2011). The literature reported medical students could encounter a broad range of challenging issues during these transition periods, which could be categorised under five broad themes: dealing with academic or work-related pressures, challenges of professional socialisation, managing uncertainties, concerns about harming patients (such as due to their lack of knowledge or skills), and general life stressors (such as financial pressures and personal life events; Dyrbye, Thomas, & Shanafelt, 2005; Prince, van de Wiel, Scherpbier, can der Vleuten, & Boshuizen, 2000; Rees & Monrouxe, 2011).

Two studies from the literature have transformed our understanding of medical students’ transition experience within their undergraduate medicine academic programmes. Firstly, the study by Radcliffe and Lester (2003) provided a broad overview of the challenges undergraduate-entry medical students could encounter during their medical school transition experience. The authors conducted an interview-based study with 21 fifth year (final year) medical students from the undergraduate-entry medicine degree programme at the University of Birmingham, United Kingdom. The authors analysed the data using a Grounded Theory approach and reported medical students experienced a series of transition periods within their degree programme. These transitions included role transition from secondary education to the first year of their degree programme, role and knowledge transition from preclinical science student status in the second-year to apprentice doctor in the third-year, and their role and knowledge transition on approaching qualification as doctors. They reported most medical students found the transition from secondary education to an undergraduate medicine academic programme stressful as they had to manage the changes to their lifestyle related to starting university, making new friends and competing with their peers. Many medical students found the less didactic teaching methods in their undergraduate-entry medicine degree programme as stressful (Radcliffe & Lester, 2003).

Next, the study by Rapport et al. (2009) offered insights into graduate-entry medical students’ transition experience through their medical school journey. The authors conducted an interview-based study with 44 medical students from the Swansea University Medical School, United Kingdom, to explore the impact of the students’ previous degree programme and life experience on their current degree programme experience. The data was analysed using thematic analysis and the study findings highlighted that prior work and life experiences had a significant and positive impact on their current graduate-entry medicine degree programme experience—findings that were consistent with my own observations (Tso, 2019). Students’ prior experiences had helped them understand complex concepts in professionalism, ethics and reflective learning, as well as how to manage patient encounters (Rapport et al., 2009). The impact of this study could have been greater if the scope of the authors’ investigation were to include graduate-entry medical students’ transition experience into professional practice as doctors.

II. METHODS

This study aimed to make a sociological contribution to understanding the transition experience of medical students from graduate-entry medicine degree programmes. The research question was ‘what are the challenges experienced by graduate-entry medicine degree programme students during their transition through medical school training?’

A two-staged interview-based study was conducted to explore the transition experience of graduate-entry medicine degree programme students at the University of Warwick in the United Kingdom. All registered students from the University of Warwick 4-year graduate-entry medicine degree MB ChB programme were invited to take part in the study via a student society’s electronic mailing list. One electronic mailing reminder was sent to all potential participants. Medical students who have suspended their studies and medical students from other medical schools undertaking placements at the University of Warwick were excluded. There were over 650 registered medical students. A mixed volunteer and quota (by student year group) sampling strategy was used to ensure representation of medical students from across all year groups.

Twenty-one students volunteered to take part in stage one semi-structured interviews each lasting between 30 to 45 minutes. Fourteen of 21 (67%) stage one participants volunteered to attend a follow-up stage two semi-structured interview each lasting between 15 to 25 minutes (took place approximately four to thirteen months following their previous interviews). The stage one and stage two interview questions were summarised in Table 1. All the interviews were audio-recorded using a digital recording device. Audio recordings of their semi-structured interviews were transcribed verbatim. Participants were invited to check and comment on the contents of their interview transcripts.

Published online: 5 May, TAPS 2020, 5(2), 14-21
https://doi.org/10.29060/TAPS.2020-5-2/OA2128

Lay Ling Tan & Carmen Jia Wen Kam

Department of Psychological Medicine, Changi General Hospital, Singapore; Clinical Trials and Research Unit, Changi General Hospital, Singapore

Abstract

Aims: Psychiatry residents’ outpatient clinic supervision may be with direct observation of clinical cases (D), without direct observation (WDO) or a mixture of both (M). This study explored residents’ perceptions of clinical teaching effectiveness under these supervision frameworks.

Methods: A survey was conducted amongst residents who completed their third-year training. Cleveland Clinic’s Teaching Effectiveness Instrument (CCTEI) was used to capture their perception of various domains of clinical teaching effectiveness. Content analysis of their qualitative feedback was used to establish the major categories of supervision perception.

Results: 42 out of 60 residents responded. Fewer residents received DO and M compared to WDO. More residents preferred DO. There was no statistical difference in the total CCTEI scores for the three supervision formats. Qualitative content analysis revealed categories surrounding strengths of DO and WDO as well as weaknesses of DO. Although DO allowed timely feedback, addressed patient safety and increased residents’ confidence, it was perceived to be stressful, requiring more resources and inhibited independent learning. WDO was commended for its support of autonomous learning, less anxiety-provoking and contributed to better patient rapport.

Conclusion: There was no statistical difference in clinical teaching effectiveness of the three supervisory frameworks. Previous studies showed DO to be anxiety-provoking with concerns of compromised autonomous learning. This study uncovered similar themes, but residents still preferred DO. It highlighted the residents’ perception of the importance of timely feedback, patient safety and instilling confidence with DO.

Keywords:        Psychiatry, Residents, Supervision, Direct Observation, Ambulatory, Outpatient

Practice Highlights

• There has been little guidance in the literature on best practices in psychiatry supervision.
• Many teaching sites offered WDO, but residents preferred DO with its timely feedback.
• DO was perceived to be stressful, needed more resources and inhibited independent learning.
• WDO was perceived to support autonomous learning, less anxiety provoking and offered better rapport with patients.
• Use of valid assessments may guide the adoption of M, leveraging on the strengths of DO and WDO.

I. INTRODUCTION

The Psychiatry Residency Training in Singapore is a national program which requires our 3^rd year residents to be posted to the outpatient clinics for a year. There are seven teaching sites in the residency program and each site offers its own supervision framework. This may be direct observation (DO) with the supervisor sitting in with the resident for the entire clinic visit or an alternative supervisory model with the resident seeing the new cases independently and discussions with the supervisors followed with no direct observation (WDO) of the consultation process. Some sites offer a mixture of both supervisory framework (M), largely determined by supervisor availability and not intentionally planned and structured.

There have been questions from the faculty, clinical supervisors and residents as to whether DO was a more effective model of supervision. The general perception was that DO would offer better patient care quality and less patient safety concerns. However, there were also concerns about the sustainability of the model of DO in view of the increasing pool of residents. In their brief report, Reardon, May, and Williams (2014) aptly discussed the challenges training directors face in balancing optimisation of patient care, education and reimbursement.

There has been little recent research on the supervision of the psychiatry interview in the non-psychotherapy outpatient setting. Extant literature highlighted advantages of DO, including a more accurate picture of the patient-student relationship; being able to alert students to non-verbal cues and students feeling less threatened since they are not without aid and corroboration (Lewin, 1966). A study by Stein, Karasu, Charles, and Buckley (1975) comparing DO with a verbal report of the interview also revealed that some residents voiced resentment that the treatment was complicated by the supervisors’ interventions and that their inadequacies and inexperience were exposed by the presence of the supervisor.

Supervision has been defined as the provision of monitoring, guidance and feedback on matters of personal, professional and educational development in the context of the doctor’s care of patients (Kilminster & Jolly, 2000). It is a complex activity with different modes of delivery that occurs in a variety of context and is undoubtedly important for the professional development of our residents. Previous work by Stein et al. (1975) suggested that DO was an important factor in the accuracy of patient evaluation and the process of verbal report itself without direct observation tended to influence the supervisor in minimising psychopathology. This will have implications in terms of the quality of patient care.

A recent review of the literature on direct and indirect supervision in psychiatric and other graduate medical education determined that there had been a dearth of research on the impact of direct supervision on the educational experience of psychiatry residents (Galanter et al., 2016). There has been little guidance in the existing literature on best practices in supervision of psychiatry residents with resulting unclear expectations (Newman, Ravindranath, Figueroa, & Jibson, 2016). This survey attempted to compare the residents’ perception of the clinical teaching effectiveness under the various supervision frameworks. This will further enhance our understanding of the variables perceived to be significant in the supervisory process of residents in general psychiatry ambulatory training. It will aid in the program’s consideration of options for optimising patient care and residency education while addressing resource limitations.

II. METHODS

The intent and purpose of the survey was discussed with the Psychiatry Residency Program Director and waiver of informed consent was sought from the Institutional Review Board. Email invitations to the survey detailing the study objectives and questionnaire were sent to all the residents at the end of their third-year ambulatory training over a period of three years. Non-responders (residents who did not respond to the email invitations after two weeks) were given the survey by the resident representative during their weekly residency didactics at the various teaching sites. The residents were posted to teaching sites with different supervision frameworks. Comparisons were made based on the residents’ declaration of the supervision framework offered by each teaching site more than 50% of the time of their ambulatory training. Their current supervisory framework (DO, WDO or M) and their preferred choice of supervision were captured in the analysis.

Feedback and comments on their supervision in the questionnaire were included for qualitative analysis. A deductive content analysis was adopted with development of the categorisation matrix by the principal investigator. Atlas Ti (version 8) software was used to code the data according to the categories.

The Cleveland Clinic’s Teaching Effectiveness Instrument (CCTEI) was used to assess the residents’ perception of the clinical teaching effectiveness under the various supervisory format. The CCTEI has been shown to be a reliable and valid evaluation tool for a wide variety of clinical teaching settings (Copeland & Hewson, 2000). It has 15 questions that used a five-point evaluation scale (see Appendix).

Categorical data was presented as frequency (percentage). Numeric data was tested with the Shapiro-Wilk normality test and presented as median (interquartile range) when the data distribution did not follow normal distribution. The associations between two categorical variables were examined using Fisher’s Exact test. Differences of CCTEI scores among three groups of supervision format were assessed by Kruskal Wallis test. In case of statistically significant difference in Kruskal Wallis test, Bonferroni post-hoc adjustment was used for multiple pairwise comparisons. A two-tailed, p-value of <0.05 was considered statistically significant. Statistical analysis was performed with SPSS statistical software, version 19.0 (IBM Corp. Armonk, NY).

III. RESULTS

Out of the 60 eligible residents, 42 responded to the survey, yielding a response rate of 70%. Most sites offered WDO supervision (45.2%). M was the supervision framework least frequently adopted across the teaching sites (19%) and 35.7% received DO (Table 1). About 80% of the residents were more than three years post-graduation at the time of the survey. There were no statistical differences in the demographic variables and the number of years post-graduation with the different supervision formats (Table 1).

Table 2. Median (interquartile range) of CCTEI Item scores and total score with the different supervision format

When asked for their preferences, most residents opted for DO (20/41, 48.8%) with a fairly equal distribution between M (11/41; 26.8%) and WDO (10/41; 24.3%). Although all but two of the eight residents with less than three years of clinical experience preferred DO and residents with more than three years of clinical experience preferred M supervisory format, the results were not statistically significant (p = .512 and .387 respectively).

Qualitative comments of residents’ perceptions and reasons for their supervision preferences captured in the questionnaire were perused in detail. A deductive content analysis was used to test existing categories and concepts with regards to clinical supervision (Elo & Kyngäs, 2008). A categorisation matrix was developed with three major categories comprising (a) Weaknesses of DO; (b) Strengths of DO; (c) Strengths of WDO. All the data were then reviewed for content and coded for correspondence with the identified structured categories using the Atlas Ti (version 8) software. All the qualitative data was able to fit into the categorisation frame. There were 19 quotations referring to weaknesses of DO; 31 quotations for strengths of DO and 21 quotations for strengths of WDO. Surprisingly, there were no statements made addressing weaknesses for WDO or feedback regarding M supervision. To further improve credibility and transferability, member checking was used. Participants’ comments regarding the developed categories were solicited. There was general agreement with the findings generated from their feedback.

A. Weaknesses of DO

The themes for the category on DO weaknesses were related to it being more stressful, needing more resources and inhibiting independent learning.

1) Stressful Environment: Some residents described that it was stressful to have the supervisor observe them for the entire consultation.

Female Resident 3: “It was nerve-wracking and anxiety-provoking to be watched.”

Female Resident 9: “It was personally very stressful to be observed while clerking a case.”

There was a perceived need for a more conducive learning environment as residents in their junior years of ambulatory training learn to engage their patients and family caregivers in the clinical encounter. The presence of the supervisor with an evaluative function was perceived to contribute to a stressful learning environment.

2) More Resources are Needed: Residents reported that having the supervisor sit in throughout the clinical encounter contributed to wastage of time and resources.

Female Resident 2: “Drains resources.”

Male Resident 8: “Will be too time-consuming and labour-intensive for supervisor and trainee.”

With the perennial challenge of having to reduce the lead time for new referrals to be seen by the outpatient specialist clinics, residents might have the notion that they were contributing to the poor utilization of resources and that more resources would need to be available to cater to both their educational and clinical service needs.

3) Inhibit Independent Learning: There was the perception that DO might potentially affect the residents’ autonomous learning.

Female Resident 2: “Can potentially inhibit the learner’s capacity for individual learning and development of interview style as he/she may feel conscious of the presence of an observer and compelled to conform to the supervisor’s style.”

Male Resident 10: “Having a supervisor in the room all the time can potentially inhibit the learner’s capacity for independent learning.”

The residents were concerned that the supervisor’s constant presence might potentially inhibit their independent learning as they would be closely guided by their supervisor when they encounter any difficulty during the clinical consultation. They also feared that they might be pressured to follow the supervisor’s particular style of interviewing.

B. Strengths of DO

The themes for the category on DO strengths centred around timely feedback, ensuring patient safety and building resident’s confidence.

1) Timely Feedback: Residents valued the immediate feedback achieved through DO.

Female Resident 7: “Able to have real-time discussion of not only management of patient, but also obtain feedback on interviewing skills and my interaction with the patients.”

Female Resident 15: “Would be the ideal scenario as it offers the opportunity to obtain feedback without the clinical interview process and on the spot learning.”

The residents described the opportunities to have their interviewing techniques scrutinized and honed by the supervisors with timely feedback as invaluable and ideal in their ambulatory training.

2) Ensuring Patient Safety: Residents were more assured that they were delivering safe patient care when they were able to consult their supervisors immediately during DO.

Female Resident 3: “Observe my management – To fine-tune my clerking, clinical management skills. Provides supervision for patient safety.”

Male Resident 23: “Discuss management plan together at first visit–Better for patient care.”

Residents endorsed direct observation of their initial interview as an important factor in the accuracy of patient assessment and evaluation. This wouldoffer better patient care quality and less patient safety concerns.

3) Building Residents’ Confidence: DO was considered to be pertinent in boosting the residents’ confidence.

Male Resident 8: “Is especially useful during the initial few weeks/months or for newer trainees to build up confidence.

Male Resident 13: “I think reviewing the case together with consultant is not required unless the resident doesn’t feel confident in managing the case.”

Residents in their junior years of training with less clinical experience would appreciate the reassuring presence of their supervisor and were possibly less conscious of exposing their inexperience and inadequacies.

C. Strengths of WDO

With the category on strengths of WDO, participants cited more autonomous learning, less anxiety and better patient rapport.

1) Autonomous Learning: Residents viewed WDO as helpful in encouraging independent learning.

Female Resident 2: “Allows greater autonomy for independent learning.”

Female Resident 4: “It gave me the independence to make decisions.”

Residents felt that WDO allowed them more opportunity and time for their own reflections about the clinical assessment and to plan their own management plan before discussing with their supervisors.

2) Less Anxiety: WDO was referred to as less stressful compared to DO.

Female Resident 4: “It also allowed me to manage the session without worrying all the time about someone else being present.”

Female Resident 25: “There will be less stress.”

Residents felt that the clinical environment will be more relaxed without the presence of the supervisor and they were better able to focus on engaging their patients and caregivers.

3) Better Patient Rapport: Residents felt that WDO allowed better patient rapport.

Male Resident 19: “Allow resident to develop own style of consult, better rapport with patient.”

Female Resident 34: “Allows the resident to build better rapport with patient.”

Residents perceived that their therapeutic alliance with their patients would be better established if the supervisor were not present in the consult room.

IV. DISCUSSION

This survey was driven by concerns about the sustainability of the model of DO in one of the teaching sites and whether DO is the most effective supervisory approach in our psychiatry ambulatory training. This study found that many of the teaching sites offered WDO as the supervisory model. Clinical supervisors were entrusted with trainees in their third year of residency with different trajectory in terms of their competency milestones. First impressions were usually relied upon to determine competency and skills of the residents (Wood, 2014). Concerns about the accuracy and diminished clinical objectivity of patient evaluation without direct observation were previously highlighted (Stein et al., 1975). This may potentially impact residency training outcomes and patient care.

A survey on how Directors of United States psychiatric residency programs were balancing patient care, education and reimbursement was recently conducted by Reardon et al. (2014). Options were proposed for optimising patient care and resident education while maintaining financial solvency. These included having a process of identifying resident supervisory needs based on training level, resident competency and complexity of patient care (Reardon et al., 2014).

Although maintenance of financial solvency is not expected of our program directors currently, there will be a need to justify the manpower needs for DO for an increasing pool of residents in future. Palesy (2017) studied the use of scaffolding and how learning might be supported in the absence of direct supervision. The consideration of a range of scaffolds provided in the learning environment without rigid stipulations might be able to encourage residents without direct observation to decide on the type and frequency of their interaction with the scaffolding provided (Palesy, 2017). The use of reliable and valid assessments e.g. mini-clinical examinations (mini-cex) may help with determining the readiness of residents for independent clinical practice and better guide the adoption of the mixed supervisory format (Norcini, Blank, Duffy, & Fortna, 2003).

Although residents alluded to WDO’s strengths in offering more autonomous learning, less anxiety and better patient rapport, our study suggested that our psychiatry residents still favoured DO, particularly those with less clinical experiences (less than three years post-graduation). This finding concurred with an earlier report of how DO might be perceived to be less threatening for students (Lewin, 1966). Residents with less clinical experience would appreciate the reassuring presence of a supervisor and possibly less conscious of exposing their inadequacies (Stein et al., 1975). Only Q11 of the CCTEI (coaches me on my clinical/technical skills) was rated as statistically significant amongst the three supervision frameworks, favouring DO. Our residents generally welcome coaching on their psychiatry skills using DO and our findings did not suggest that DO affected their independent learning (CCTEI Q2) and autonomy (CCTEI Q3) in their patient management.

A recent study by Galanter et al. (2016) referred to the traditional supervisory process in psychiatry training without direct observation, where residents were seen to be granted more autonomy and the therapeutic alliance would not be disturbed by having the supervisor in the room. Analysis of the qualitative feedback from residents in our study highlighted their concerns that DO may affect their rapport with patients but at the same time, the presence of a supervisor also helped to boost their confidence and reassure them of patient safety and quality of care. Without DO, there would be a need to have readily available guidance to residents like videoconferencing and use of other technology-facilitated supervision to support their delivery of appropriate care to their patient (Cameron, Ray, & Sabesan, 2015; Hayden, Navedo, & Gordon, 2012).

Our study has limitations. The number of residents who participated in the survey was relatively small although we managed to have a 70% response rate. Self-reported data was used for qualitative analysis with the risk of recall biases. Although independent verification of self-reported data could not be established, there were no incongruent findings. Member checking was also used to improve the credibility of the analysis.

 V. CONCLUSION

Our study did not reveal any statistical difference in the perception of clinical teaching effectiveness of the three supervisory frameworks based on the total CCTEI scores. Previous studies showed DO to be stressful with concerns of too much ‘hand-holding’ and compromised autonomous learning. Our qualitative analysis uncovered similar themes, but residents still preferred DO. The study highlighted the residents’ perceptions of the importance of timely feedback, patient safety and instilling confidence with DO.

Close supervision by DO for all cases may not be feasible but it may be crucial for residents early in their training. There should be flexibility in the supervisory framework for residents to function more independently when they are deemed to be competent and safe. The use of reliable and valid assessments may help with determining the readiness of residents for independent clinical practice and better guide the adoption of the mixed supervisory format. More research is needed to elucidate the determinants of the most effective supervisory model and to develop innovative ways of ensuring a high standard of clinical supervision and patient care.

Notes on Contributors

Dr Tan Lay Ling is Senior Consultant Psychiatrist in the Psychogeriatric Service of the Department of Psychological Medicine of CGH. She is currently appointed as the Associate Program Director for the National Psychiatry Residency Program in CGH. She is also a member of the Clinical Competency Committee of the Residency Program.

Miss Kam Jia Wen Carmen is the Biostatistician of the Clinical Trials and Research Unit of CGH.

Ethical Approval

Submission for review was deemed not required by the SingHealth Institutional Review Board (iSHaRe Ref. No: 201601-00106; CIRB Ref. No: 2016/2083).

Acknowledgements

We would like to acknowledge the contribution of the National Psychiatry Residency Office for its administrative support in this survey and the residents for their willingness to participate.

Funding

We did not receive any funding for this research study.

Declaration of Interest

The main researcher is the Associate Program Director of one of the teaching sites and has a vested interest in ensuring the quality of supervision of residents. The researcher cannot decide on the standardisation of the supervisory framework. There are no other conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

References

Cameron, M., Ray, R., & Sabesan, S. (2015). Remote supervision of medical training via videoconference in northern Australia: a qualitative study of the perspectives of supervisors and trainees. BMJ open, 5(3), e006444. https://doi.org/10.1136/bmjopen-2014-006444

Copeland, H. L., & Hewson, M. G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2), 161-166. https://doi.org/10.1097/00001888-200002000-00015

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107-115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Galanter, C. A., Nikolov, R., Green, N., Naidoo, S., Myers, M. F., & Merlino, J. P. (2016). Direct Supervision in Outpatient Psychiatric Graduate Medical Education. Academic Psychiatry, 40(1), 157-163. https://doi.org/10.1007/s40596-014-0247-z

Hayden, E. M., Navedo, D. D., & Gordon, J. A. (2012). Web-conferenced simulation sessions: a satisfaction survey of clinical simulation encounters via remote supervision. Telemedicine and e-Health, 18(7), 525-529. https://doi.org/10.1089/tmj.2011.0217

Kilminster, S., & Jolly, B. (2000). Effective supervision in clinical practice settings: a literature review. Medical Education, 34(10), 827-840. https://doi.org/10.1046/j.1365-2923.2000.00758.x

Lewin, K. (1966). Psychiatric supervision by direct observation. Academic Medicine, 41(9), 860-864. https://doi.org/10.1097/00001888-196609000-00005

Newman, M., Ravindranath, D., Figueroa, S., & Jibson, M. D. (2016). Perceptions of supervision in an outpatient psychiatry clinic. Academic Psychiatry, 40(1), 153-156. https://doi.org/10.1007/s40596-014-0191-y

Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003). The mini-CEX: a method for assessing clinical skills. Annals of internal medicine, 138(6), 476-481. https://doi.org/10.7326/0003-4819-138-6-200303180-00012

Palesy, D. (2017). Learning in the Absence of Direct Supervision: Person-Dependent Scaffolding. Vocations and Learning, 10(3), 365-382. https://doi.org/10.1007/s12186-017-9176-9

Reardon, C., May, M., & Williams, K. (2014). Psychiatry resident outpatient clinic supervision: how training directors are balancing patient care, education, and reimbursement. Academic Psychiatry, 38(4), 476-480. https://doi.org/10.1007/s40596-014-0091-1

Stein, S. P., Karasu, T. B., Charles, E. S., & Buckley, P. J. (1975). Supervision of the initial interview: A study of two methods. Archives of General Psychiatry, 32(2), 265-268. https://doi.org/10.1001/archpsyc.1975.01760200129013

Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education, 19(3), 409-427. https://doi.org/10.1007/s10459-013-9453-9

*Tan Lay Ling
Changi General Hospital,
2 Simei Street 3, Singapore 529889
Tel: 69365072
E-mail: tan.lay.ling@singhealth.com.sg

Published online: 5 May, TAPS 2020, 5(2), 32-40
https://doi.org/10.29060/TAPS.2020-5-2/OA2194

Suriyakumar Mahendra Arnold¹, Sepali Wickrematilake², Dinusha Fernando³, Roshan Sampath¹, Palitha Karunapema⁴ & Pasyodun Koralage Buddhika Mahesh⁵

¹Quarantine Unit, Ministry of Health Sri Lanka; ²Regional Director of Health Services Office, Matale, Sri Lanka; ³Regional Director of Health Services Office, Puttalam, Sri Lanka; ⁴Health Promotion Bureau, Ministry of Health, Sri Lanka; ⁵Ministry of Health, Sri Lanka

Abstract

Background: The duties of Public Health Inspectors (PHI) includes those related to food legislation. Effective methods are being explored in providing refresher training for them amidst the constraints of resources.

Objective: To assess the knowledge, attitudes and skills of the PHI on food legislation and to evaluate the effectiveness of a Distance Education (DE) programme in improving these.

Methodology: The study included quasi-experimental educational intervention with a pre- and post- assessments. 105and 109 PHI were recruited for the intervention and control groups. Pre and post interventional assessments on the knowledge, attitudes, self-assessed competencies and performance were done with a self-administered questionnaire and observational checklist. Intervention group was exposed to an educational intervention through a distance education programme. Post-assessments were done following four months of the intervention.

Results: The pre interventional knowledge and performance was poor. PHI had unfavorable views. Post intervention mean knowledge and performance scores revealed that a statistically significant improvement (P<0.001) has occurred in the intervention group. Change in the opinions on usefulness of in-service training on food safety to develop knowledge and skills, was statistically significant (P<0.01) in the intervention group.

Conclusion: The DE method used in the present study was feasible to implement and was effective in significantly improving the knowledge and performance of the PHI.

Keywords:           Distance Education, Continuing Education, Public Health Inspector, Food Legislation

Practice Highlights

• Continues Education is important for development of knowledge and skills of healthcare staff.
• Distance Education can be successfully used for Continues Education for staff geographically scattered.
  

I. INTRODUCTION

Learning is an active, continuous, sequential process. Learning takes place more rapidly when the learner has the opportunity to practice and experience what is being learned in a variety of situations. It is facilitated when it takes place in or near the real situation in which the learner expects to work (Schunk, 2012). There is an increasing recognition of the need for health workers to continue their education throughout their career (Price & Reichert, 2017). The changes that take place in health needs and health care policies necessitate continued learning.

Continuing education (CE) should be an integral part of an employment. Adequate provision should therefore be made for it in terms of manpower, study level and financial reimbursement. Whenever possible, CE must be provided at the place of work and must be relevant to the immediate needs of the health care workers and the community he/she serves (Billett et al, 2015; Cedefop, 2015; Kenny, Ralph & Brown, 2006; Sherman & Chappell, 2018).

Professional and technical education with traditional methods of training are often inflexible, time- and place- constrained and too expensive to meet the demand for training (Asian Development Bank, 2014; Scott, 2010). Efforts like technological advancements and distance education techniques have created an immense impact in molding the education methods (Chay, 2019). Distance education (DE) seems to offer a way of providing flexible, effective job-related training.

To be successful, DE programme must be systematically designed, interactively motivational and suited to the needs of the audience (Balaban-Sali, 2008). Distance education has the potential of delivering more educational opportunities to more people than ever before, at lower average cost and most importantly, at a higher quality than most people can get in other ways (Marcinkiewicz, Cybart & Chrominska-Szosland, 2002).

The problem of updating the knowledge and skills of health personnel already on the job still remains as a challenge (Murray & Christison, 2018). It is almost impossible to have them re-trained at the institutions they occupy with face-to-face contact with the physical presence of trainers. Hence it is important to develop novel programmes in updating and refreshing of the health personnel amidst the feasibility constraints.

Food borne diseases account for a major toll on health. According to the World Health Organization (WHO), up to one-third of the populations of developed countries are affected by foodborne illnesses and the problem is likely to be far widespread in developing countries (World Health Organisation, 2002). The administration and implementation of food laws requires a qualified, trained, and efficient food inspection service (Food and Agriculture Organization of UN & WHO, 2003; Sri Lanka Accreditation Board for Conformity Assessment, 2013). Hence, proper training of food inspectors (PHI) is a prerequisite for an efficient food control system.

The need of providing training to PHI to improve their performance has been identified in the National Health Development Plan (NHDP) of Sri Lanka (Ministry of Health Sri Lanka, 2016). Public Health Inspector is one of the main authorised officers to implement public health laws in Sri Lanka. Of their many duties, PHI play a major role in maintaining food safety in the community. In addition, the food legislation is one area where major responsibility in implementation lies with the health sector. Therefore, educational interventions to improve the implementation could be carried out successfully as a part and parcel of the NHDP of Sri Lanka.

This study was done to assess the knowledge, attitudes and skills of the PHI on food legislation and to evaluate the effectiveness of a DE programme in improving these.

Component	Intervention Group			Control Group
	Pre	Post	Signifi cance	Pre	Post	Significance
	Mean	Mean		Mean	Mean
Enactments on food safety	29.4	61.3	P < 0.001	30.6	32.1	NS
Food labelling	52.4	68.1	P < 0.001	54.2	56.4	NS
Food sampling	24.0	63.4	P < 0.001	24.9	26.8	NS
Meat and fish inspection	54.4	67.6	P < 0.001	53.6	55.6	NS
Legal procedures	40.3	67.8	P < 0.001	41.4	43.5	NS

	Intervention Group	Control Group
Component	Difference between post and pre mean scores	Difference between post and pre mean scores	Significance
Enactments on food safety	+31.9	+1.5	P< 0.001
Food labelling	+15.7	+2.2	P< 0.001
Food sampling	+39.4	+1.9	P< 0.001
Meat and fish inspection	+13.2	+2.0	P< 0.001
Legal procedures	+27.5	+2.1	P< 0.001

Table 4. Comparison of the changes in mean performance scores between the intervention and control groups

IV. DISCUSSION

This is the first documented local study testing a DE method for improving the knowledge, attitude and skills of PHI on food legislation. CE is a process, including the experiences after initial training that helps the personnel to attain competencies relevant to their work (Gaspard & Yang, 2016). Of the methods available, distance education method has been used successfully in many countries to provide CE for health care workers. With the advances in the healthcare system, it is essential that Sri Lanka evaluate the effectiveness of different strategies in updating the primary healthcare workers. Hence this study is an eye-opener for the policy-planners in relation to the in-service training of the health staff.

Pre-intervention assessment results showed that the overall knowledge was poor in both groups. Similarly there was no significant difference in attitudes, self-assessed competencies and performance between the two groups, thus the intervention and control groups were comparable before the intervention. Therefore, these results also discount the selection bias in the present study.

The findings of the pre-intervention assessment pointed to the need of CE as the total knowledge scores obtained was unsatisfactory. Only 17.1% of the study group and 23.9% of the control group have undergone any training on food safety during the past three years. Thus a majority of the PHI have had no opportunity for CE to improve and update their knowledge. The findings also revealed that the knowledge and performance in enactments in food safety, food sampling and legal procedures were poor. These findings were taken into consideration in selecting study topics for the distance education modules. Gap analysis is important in planning CE (Moore, Green, & Gallis, 2009) and identifying of needs has an impact on performance outcome in CE (Cervero & Gaines, 2015).

In training a large number of health care workers at different geographical locations, it is important to maintain uniformity of such training. In addition, the quality of educational materials provided will have an effect on the outcome of the intervention. The present study considered these factors and the services of experts were obtained in developing DE modules which were technically sound and of high quality. Uniformity was maintained by administering the modules in a similar manner at all centres where the training programme was conducted.

Since a time lag between the intervention and the post intervention assessment is needed to ascertain the sustainability of the effectiveness of the intervention, the post-intervention assessment was carried out four months after administering the final module.

In general there was no change in the attitudes towards implementation of food legislation in both intervention and control groups except for the views expressed on usefulness of in-service training. One of the reasons for no change in views in both groups may be due to the fact that there was no significant change in their working environment, resources and support services. However, these factors were outside the control of the present intervention.

CE activities can lead to improved competencies and performance (Moore et al., 2009). This study revealed that DE strategy used here was effective in improving the knowledge and performance significantly in the intervention group.

The intervention used in the present study was carried out with minimal interference to the routine duties of the trainees. Unlike institutional-based training programmes or workshops, trainees did not have to be away from the work place resulting in incurring a minimal cost. In the present educational intervention, printing cost incurred for the three distance education modules was minimal. Thus as a developing country with health care workers scattered throughout the island, due consideration should be given to shift from traditional institutional-based training at the central level to providing continuing education at the local setting using feasible, cost-effective methods such as DE where appropriate.

The study was limited to two districts in two provinces. This study was conducted in two districts as a quasi-experimental study. Since then the probability of a selection bias cannot be totally excluded. Overall picture could have been obtained if this study was conducted as an island-wide study. These were taken into account in the interpretation of findings. Sustainability of the improved performance as a result of the DE programme would have been better monitored if post interventional assessments were carried out at one-year and two-year period.

V. CONCLUSION

The present intervention was effective in improving the knowledge and performance of the PHI in implementing legislation on food safety indicating the effectiveness of providing continues education to PHI using distance education method. As this method proved to be feasible, of low cost and has the ability to provide training without mobilizing the staff, it can be incorporated into the existing training system for health care workers.

Notes on Contributors

Mahendra Arnold, Sepali Wickramatilake, Dinusha Fernando, Roshan Sampath and Palitha Karunapema are Public Health Specialists attached to the Ministry of Health. Pasyodun Koralage Buddhika Mahesh is a Senior Registrar in Community Medicine of the Ministry of Health.

Ethical Approval

Ethical approval was obtained from Ethics Committee, Faculty of Medicine, University of Colombo.

Acknowledgements

The authors would like to acknowledge the Public Health Inspectors who participated in this study.

Funding

This study did not receive any external funding.

Declaration of Interest

The authors do not have any conflict of interest to declare.

References

Asian Development Bank (2014). Innovative Strategies in Technical and Vocational Education and Training for Accelerated Human Resource Development in South Asia. Mandaluyong City, Philippines: Asian Development Bank.

Billett, S., Choy, S., Dymock, D., Smith, R., Henderson, A., Tyler, M., & Kelly, A.(2015). Towards more effective continuing education and training for Australian workers, NCVER, Adelaide.

Balaban-Sali, J. (2008). Designing motivational learning systems in distance education. Turkish Online Journal of Distance Education-TOJDE, 9(3), 149-159.

Cervero, R. M., &  Gaines, J. K. (2015). The impact of CME on physician performance and patient health outcomes: An updated synthesis of systematic reviews. Journal of Continuing Education in the Health Professions, 35(2), 131–138.

Chay, O. M. (2019). Transformation of medical education over the years – A personal view. The Asia Pacific Scolar, 4(1), 59-61

Food and Agriculture Organization of the United Nations [UN], & World Health Organization [WHO] (2003). Assuring Food Safety and Quality: Guidelines for Strengthening National Food Control Systems,_FAO Food and Nutrition Paper 76. A Joint FAO/WHO publication. http://www.fao.org/3/a-y8705e.pdf

Gaspard, J., & Yang, C. (2016). Training needs assessment of health care professionals in a developing country: The example of Saint Lucia. BMC Medical Education, 16, 1-6. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0638-9

Kenny, M., Ralph, S., & Brown, M. (2006) Lifelong learning, professional development and community work. Research in Post-Compulsory Education, 5(1), 51-62.

Marcinkiewicz, A., Cybart, A., & Chrominska-Szosland, D. (2002). Distance learning in postgraduate training of professionals: Example of occupational medicine specialization. Wiad Lek, 55(1), 330-334.

Ministry of Health Sri Lanka. (2016). National health strategic master plan 2016 -2025. Colombo, Sri Lanka: Ministry of Health

Moore, D. E., Green, J. S., & Gallis, H. A. (2009). Achieving Desired Results and Improved Outcomes: Integrating Planning and Assessment Throughout Learning Activities. Journal of Continuing Education in The Health Professions, 29(1), 1–15.

Murray, D. E., & Christison, M. (2018). Online Language Teacher Education: A Review of the Literature. Aqueduto, Norwich.

Price, S., & Reichert, C. (2017). The importance of continuing professional development to career satisfaction and patient care: Meeting the needs of novice to mid- to late-career nurses throughout their career span, Administrative Sciences, 7(2), 17. https://www.mdpi.com/2076-3387/7/2/17.

Schunk, D. H. (2012). Learning Theories, An Educational Perspective (6^th Ed.). Boston, MA: Pearson Education, Inc.

Scott, S. G. (2010) Factors Impacting the Selection of Training-Delivery Systems and Training Methodology of Virginia Training Professionals (doctoral dissertation). Blacksburg, VA: Faculty of the Virginia Polytechnic Institute and State University.

Sherman, L. T., & Chappell, K. B. (2018). Global perspective on continuing professional development. The Asia Pacific Scolar, 3(2), 1-5.

Sri Lanka Accreditation Board for Conformity Assessment. (2013). Specific Criteria for Food Inspection, (Doc No. IB-GL(P)-03). Retrieved from http://www.slab.lk/Support/Publications/Inspection%20Bodies/IB-Specific-Criteria-for-Food-Inspection.pdf

World Health Organization. (2002). WHO Global Strategy for Food Safety: Safer food for better health. Retrieved from https://apps.who.int/iris/handle/10665/42559

*Mahendra Arnold
52/18, Pulinathalarama Road,
Magammana, Ragama, Sri Lanka
Tel: +94714199953
Email: mahendra_arnold@yahoo.com

Published online: 5 May, TAPS 2020, 5(2), 22-31
https://doi.org/10.29060/TAPS.2020-5-2/OA2173

Andrea Thompson¹, Tanisha Jowsey¹, Helen Butler¹, Augusta Connor², Emma Griffiths², Hadley Brown² & Marcus Henning¹

¹University of Auckland, New Zealand; ²Mercy Hospice, Auckland, New Zealand

Abstract

Practice Highlights

• The course studied informs pharmacists’ practice for improved service delivery.
• The course led to more meaningful palliative care conversations.
• Interactive teaching methods supports learner engagement.
• Educators sharing personal experiences supports learning.

I. INTRODUCTION

There have been recommendations in the literature for over 30 years that pharmacists should receive more education around end-of-life issues and care (Dickinson, 2013). While inadequate training and knowledge in palliative care leads to poor palliative care provision (Furstenberg et al., 1998; Vernon, Brien, & Dooley, & Spruyt, 1999), effective palliative education can positively transform care provision (Institute of Medicine, 2015).

Research shows that effective palliative care education for pharmacists can deepen their understanding of their role in symptom and therapy management and psychosocial care during end-of-life stages, including reducing death anxiety among patients (Atayee, Best, & Daniels, 2008; Dickinson, 2013; Needham, Wong & Campion, 2002). Learning about palliative care encourages collaboration and continuity in service provision, and appropriate, timely and individualised care (Dickinson, 2013). Additionally, with extra training, community pharmacists can become more actively involved in their palliative patients’ care, including providing patient education, prescribing advice to physicians and facilitating continuity between healthcare settings (O’Connor, Pugh, Jiwa, Hughes, & Fisher, 2011). Australian community pharmacists report that completing a flexible online palliative care education programme positively impacts their practice (Hussainy, Marriott, Beattie, Nation, & Dooley, 2010). Hussainy et al. (2010) recommend future educational courses to include face-to-face weekly workshops in order to increase participation. In New Zealand, undergraduate pharmacy students receive palliative care training and upon graduation they manage various palliative care needs. To booster pharmacist palliative care knowledge and communication skills, and in response to the call from Hussainy et al. (2010), in 2016, educators at a metropolitan hospice in New Zealand ran a short course for pharmacists. The authors, including the educator participants in this study, were not involved in the course design. Here we explore participant experiences of the course including application of new knowledge/skills to practice.

The seminal pedagogical approaches of Schön (1983), Kolb (1984) and Knowles (1984) are relevant. Learning occurs when professionals reflect on their tacit knowledge and make sense of their experiences; therefore reflective practice is a central pedagogical approach (Schön, 1983). With this in mind, opportunities for reflective practice ought to be integrated into the design of educational initiatives. Kolb’s (1984) experiential learning model suggests learners have a concrete experience and then reflect on it, enabling them to formulate abstract concepts and generalisations. Learners can then try out their new understanding through the process of active experimentation. The need to be self-directed, the role of the learner’s experiences, motivation, and readiness to learn are examples of assumptions embedded in Knowles’ adult learning theory (Knowles, Holton, & Swanson, 2015).

In this paper, we explore the course in terms of teaching delivery and learning. We undertook a small interpretive thematic study to explore the participants’ experiences of the course. It is important for educators to be involved in researching and reflecting on their own teaching (Henning, Hu, Webster, Brown & Murphy, 2015; Steinert, Cruess, Cruess, & Snell, 2005) and we therefore included the teaching staff as participants alongside learner participants in this study.

A. Research Questions

We asked, what are the educator and learner experiences of a short course comprised of workshops and a series of palliative care learning packages, and how have learners changed their practice as a result of the course?

II. METHODS

III. RESULTS

Eight females participated in this study, including five learners (course participants) and three educators. Their reflections on the course that they had attended as either an educator or learner rotated around three core themes: (1) application to practice, (2) learner engagement, and (3) assessment and evaluation (Table 1).

 A. Application to Practice

Learner participants reported that the packages had increased their sense of preparedness for having real-life discussions with patients and family members/carers about death and dying/palliative care options, and ethical issues. Learners reported that this sense of preparedness had encouraged them to engage more deeply in conversations concerning palliative care.

1) Developing an understanding of palliative care: Educator and learner participants said the course enhanced their skills and understanding of communication with patients, their families/carers and other health professionals. Learners particularly valued information concerning the following subjects: the impact of a cancer diagnosis on significant others; medication; pharmacology; symptom and pain management; caring for palliative patients; ethical dilemmas; and dementia. A greater understanding of palliative care meant that learners reported they found they were able to make decisions about pharmaceuticals (without needing to check with another health professional), which they saw as likely to save time while maintaining quality of care. Further, learners’ understanding was influenced by the educators’ passion for teaching in the area of palliative care.

You might be thinking why on earth are they using Dexamethasone to increase this patient’s appetite – that doesn’t sound right. I need to ring the doctor. Whereas now they understand this is normal, this is the dose range, so they’re better equipped to supply the medication because they understand why. They’re not having to constantly ring the GP [general practitioner].

(Educator 1)

They [educators] know their topic inside out and they do their best to pass on … every ounce of knowledge that they have, which is great to see in colleagues. I just found it incredibly, inspiring to have that sort of passion for a topic.

(Learner 2)

2) Developing empathy, listening and communication skills: The learning modules enabled learner participants to develop awareness, empathy, and communication skills–particularly listening skills. They gave specific examples of empathic behaviour when dealing with patients and their families, for example, recognising and attempting to understand the multiple losses and life changes one might experience with a cancer diagnosis. Further, the content and delivery of the learning packages gave learners opportunities to be more prepared to engage in difficult conversations around death and dying.

It’s just being able to have more empathy for the people because you appreciate what they’re going through and what’s happening … just being a bit more available I suppose and realise in the end you spend time listening to what they have to say and trying to do the best for them.

(Learner 1)

And I guess when I’m talking to people, [recognising] that they’re going through a lot of losses and because of their cancer for example, they may have lost their job, I mean, if they can’t work anymore, their role in the family, they may have physical changes and loss around that.

(Learner 3)

The importance of ‘giving special time’ for conversation was an additional skill participants learned through completion of the modules. They recognised the importance of moving beyond the patient’s prescription and taking time to listen and engage in conversation with the patients and/or their families/carers.

Well actually at the time we had a customer–a man whose wife was dying of cancer, and I think instead of taking the prescription and things like that, you went out and took a special time and talked to them and spent a bit of time without actually asking too many specific questions.

(Learner 5)

In addition to communicating with patients and their families/carers, learner participants reported that the delivery of the learning modules helped equip and gave learners confidence to communicate more effectively with other health professionals.

“I deal a lot with rest homes and private hospitals so being able to assist the RNs [registered nurses] and to be able to relate to them.”

(Learner 2)

“And so, I sort of feel a lot more comfortable about that and comfortable talking to the doctors, you know, when something’s happening. So, I’ve learnt that.”

(Learner 4)

B. Learner Engagement

1) Methods of teaching: All participants discussed methods of teaching as having direct impact on their learning. Educator participants discussed teaching methods in more depth and greater frequency than pharmacist learner participants. They identified that when interactive strategies were utilised (videos, small group activities, case scenarios, demonstrations, and brainstorms), the learners absorbed/embedded more information and valued the teaching material more than when the information was presented didactically.

If I have to use a PowerPoint [slide presentation] I will learn it so that it’s behind me and I’m speaking to the audience and using eye contact, engaging from them their interest and whether they’re understanding. And I like to encourage questions to be asked as I’m talking because that then helps to add another layer of explanation. I also like case scenarios and preparing a case scenario or an ethical situation for groups to break off and discuss in their small groups and then to feedback so that each group can learn from everybody else.

(Educator 1)

The exception to this pattern was Educator 3’s observation that didactic teaching was an effective teaching strategy to begin the learning session, as it is a teaching format familiar to learners. Once the class was underway a more engaging method was needed. 

“[Learners] were quite happy to be led initially, [using] didactic teaching. But I don’t think the sessions would have been as effective if we had continued that route.”

(Educator 3)

The relevance of content was also important to both the educator and learner participants. They valued material that was relevant and applicable to the pharmacists’ ‘specific care populations and practices of care.’

I was satisfied that I was able to provide consultant specialist advice and make it real for the pharmacists…. I put in a number of extra slides that were specific to pharmacists and some of the pharmacology of the drugs etcetera.

(Educator 1)

“It was kind of practical stuff [content] that you could easily translate when dealing with people when they came into the pharmacy.”

(Learner 1)

2) Engagement–Interactive style of teaching works: The teaching methods that learner participants had most to say about were interactive ones. This included facilitated whole-group conversation that was supported by small group activities.

There were a lot of different points of view and a lot of different people who were at different levels of experience in different areas and I think having the whole group there that were all pharmacists was really helpful. We all learned from each other.

(Learner 3)

In contrast, when educators discussed interactive teaching, they talked about it in terms of group dynamics and managing the discussions so that all learners had opportunity to voice their opinions.

There’s nothing worse than having somebody that just has to answer every question, has to share everything because they just need to be heard. So, you have to manage that and … manage the person that just sits there and doesn’t say anything.

(Educator 2)

3) Drawing on personal experience: Three educators and two learners reported that some educators drew on their personal experience–such as from the hospice pharmacist setting–to illustrate the relevance of material. When learners discussed their relevant personal experiences, it was equally valued.

“We all learned from each other.”

(Learner 3)

“We even had one pharmacist kind of stand up and say ‘This happened to me and I want you guys to learn from this’, so there was a lot of them talking about their experiences and sharing stuff.”

(Educator 3)

A learner reflected that the course material had value for her in her personal life. 

I’d also gone through it [living and caring for a palliative care patient] with a flatmate early on and I felt that I hadn’t actually coped with it particularly well. So, it [the course] was a little bit for my own good as well.

(Learner 5)

Likewise, an educator made the observation that the course material was applicable to learners in their personal as well as their professional lives.

There was a strong feeling from pharmacists that they were also doing this to learn for their own personal lives because everybody will be touched by palliative care at some point in their lives, whether it be family or friends and it’s helped them to be better equipped with that.

(Educator 1)

The telling of personal experience can be linked with our first theme of Developing empathy, listening and communication skills. It is both telling and listening that comprises effective communication, which is core to pharmacist practices of care.

4) Feeling safe to share and learn: Feeling safe is important to supporting people’s learning. This was explicitly discussed by participants, and implicitly presented in other participants’ accounts.

She [Educator 2] talked to us, she encouraged you to give your opinion or your thoughts as well and you were never made to feel like what you said wasn’t right or was not significant. I thought she was great. It was very much an open forum, so you could relate, add bits in or ask questions and you felt comfortable doing that.

(Learner 4)

Educators were strategic in providing effective learning safe learning environments:

One of the skills of palliative care education is talking about topics that can be really quite difficult for some people if they’ve had a recent loss or they’ve had a situation in their personal life and it’ll trigger, so ground rules are really important to try and keep–to make sure people feel safe. So, you ensure when you start a session that people know that they can share stuff of a personal nature, but that information stays in the room and that it’s not to go outside. And you want people to be respectful.

(Educator 2)

 C. Assessment and Evaluation

1) The role of assessment and evaluation: Educator participants highlighted the importance of assessment to enhance learning. Two assessment methods; reflective activities undertaken during the learning packages and quizzes completed at the end of the packages were formatively assessed methods. Educator participants pointed out there is a mechanism for assessment to be recognised by the Pharmaceutical Society of New Zealand. In this case, successful completion of assessment therefore becomes a summative assessment.

“[The] Pharmaceutical Society approves questions and if participants gain more than 80% then … the Society awards them learning points that contribute to their compulsory continuing education.”

(Educator 1)

Along with the assessment methods described, the Hospice NZ programme gives learners the opportunity to complete an evaluation which includes questions about learning value for each learning package. The inclusion of assessment and evaluation methods fosters learners to think about how process (teaching delivery) and content have contributed to their learning.

Although educators promoted the merits of assessment comprising part of the learning modules, they acknowledged that assessment gauges learning only in the short term. It is not possible to determine the long-term impact on practice from the assessment and evaluation methods currently utilised. Further, participants identified some confusion around the reflective activities.

“We struggle with assessing their learning long-term – what have they taken away a year later? What are they using in their practice? That’s what we haven’t been able to establish.”

(Educator 1)

“Participants appeared confused about the requirements for the pre and post session reflective activities which they were required to complete in the learning packages.”

(Learner 1)

IV. DISCUSSION

Effective palliative education for pharmacists enables participants to understand their role in end of life care, reduces death anxiety, prepares them to relate to people who are dying and facilitates psychological and emotional competence (Dickinson, 2013). It offers people knowledge and confidence to engage in the types of conversations that enrich peoples’ lives by making them feel heard and cared for. Dau Voire said, “be brave enough to start a conversation that matters” (Bravery Sayings & Bravery Quotes, n.d.). Is it bravery for pharmacists to engage patients in conversations about life and death? It absolutely is because the conversation may necessitate reflecting on your own life, mortality, and wishes.

We have shown that pharmacists valued this palliative care course because it developed key skills–and increased their bravery–to engage in deeper, more meaningful palliative care conversations in their professional and personal lives. We discuss the findings in turn.

A. Learning Engagement

Teaching delivery, which is primarily focused on interactive teaching methods such as small group activities, case scenarios, demonstrations, discussions and brainstorms markedly supported learner engagement. They valued opportunities to learn through discussing learner and educator experiences. These findings are consistent with Knowles’ adult learning theory, which suggests that the learners’ experiences ought to be tapped into (Knowles, et al., 2015). Teachers can help learners by using experiential techniques to acknowledge and utilise learners’ experiences through group discussions, activities to foster reflection, simulation exercises, and problem-solving activities (Knowles et al., 2015; Kolb, 1984; Schön 1983). In our view, one of the strengths of the learning packages is that they align with key educational frameworks (Figure 1).

Learners and educators in this study promoted a safe learning environment to optimise participation and learning (Lyons et al., 2015). This is echoed by existing education literature (Brown, 1988; Douglas, 1976). A safe climate, atmosphere or environment enables participants to feel comfortable about discussing personal experiences, including those which are difficult or challenging.

B. Assessment and Evaluation

When reflection is not encouraged in the clinical setting learning opportunities may be lost (Branch & Paranjape, 2002; Schön, 1983). In this course, learning was promoted through reflection and this was valued by learners, although they reported confusion about requirements for reflection activities. The opportunity for completion of modules to contribute to pharmacists’ continuing professional development (CPD) requirements was a key driver for participants completing the summative assessment. It is unlikely that meeting CPD requirements is the sole requirement for engaging in CPD. Further motivators for CPD may be to enhance professional knowledge and competence (Ryan, 2003).

C. Application to Practice

Several elements of the course were reportedly taken forward by participants to change their practices of care which had the flow-on effect of reducing their need to contact another health professional (for example, the patient’s general practitioner) to assist in decision-making processes. This improved the patient’s continuity of care. The value of palliative education is supported by the evaluation of an online programme in palliative care for pharmacists which demonstrated that the programme positively influenced the knowledge, confidence and practice of community pharmacists (Hussainy et al., 2010).

Delivery of the learning packages enabled pharmacists to develop a greater understanding of the impact of life-limiting illnesses and fostered both their development of empathy and ability to communicate with patients and their families and/or carers. The need for developing communication skills was strongly emphasised in a study which examined the community pharmacist’s role in palliative care (O’Connor et al., 2011). Emotional issues were seen as a major source of stress for general practitioners in an earlier study that identified the importance of the need for training and education to support general practitioners in managing emotional responses for palliative care patients and their families (O’Connor, Fisher, & Guilfoyle, 2006).

D. Limitations

Only females consented to participate in this study. We cannot speak to male pharmacist experiences of the course. The number of course participants was small and we had difficulty recruiting people to our study, partly because the interviews were held several months after the conclusion of the course. Our sample size was therefore small and our findings may not be generalisable. To increase study participation, future research of palliative care pharmacy courses could seek to recruit participants on workshop days or within a week of the course conclusion. Two of the authors on this paper were participants in the study. They were not involved in any aspect of the data analysis but they did contribute to the introduction and discussion sections of this article. The study did not establish the longer-term impact of the educational initiative on pharmacists’ practice. This is an opportunity for future research.

V. CONCLUSION

It is clear that effective palliative care education such as the learning packages discussed here is valued by pharmacists and relevant to their practice. Pharmacists in this study found the learning packages enhanced their understanding of palliative care, sharpened their communication skills and bolstered their confidence to engage in deeper conversations with patients and their families/carers and other health professionals. The interactive teaching methods promoted participant engagement and gave them opportunities to share their ideas and personal experiences and to listen to the experiences of others. Improving communication was a key feature for participants in this study. We therefore recommend that community pharmacists continue to be offered effective palliative care education, and that promotion of communication skills remain central to course method and content. An increased focus on critical reflection activities within such courses needs to be encouraged so that pharmacists can make meaning of their experiences and learning opportunities are not lost.

Notes on Contributors

Andrea Thompson is a Professional Teaching Fellow at the Centre for Medical and Health Sciences Education, University of Auckland.

Tanisha Jowsey is a medical anthropologist and a Senior Lecturer at the Centre for Medical and Health Sciences Education, University of Auckland.

Helen Butler was formerly employed at a Hospice in New Zealand and is now a Professional Teaching Fellow in the School of Nursing at the University of Auckland.

Augusta Connor, was formerly a research assistant at a Hospice in New Zealand and now works in health economics in the United Kingdom.

Emma Griffiths works at a Hospice in New Zealand as a specialist palliative care pharmacist.

Hadley Brown was formerly employed at a Hospice in New Zealand and is now self-employed. He has been involved in management within in the New Zealand hospice sector.

Marcus Henning is an Associate Professor at the Centre for Medical and Health Sciences Education, University of Auckland.

Ethical Approval

This study was approved by the University of Auckland Human Participants Ethics Committee (Reference Number: 016800).

Acknowledgements

We would like to acknowledge the participants who shared their experiences of teaching and learning.

Funding

A research assistant who conducted and transcribed the interviews was funded by Mercy Hospice, Auckland, New Zealand.

Declaration of Interest

Two authors were participants in the study. Although they were involved in the design of the study and contributed to writing the paper they were not involved in analysis of the data.

References

Atayee, R., Best, B., & Daniels, C. (2008). Development of an ambulatory palliative care pharmacist practice. Journal of Palliative Medicine, 11(8), 1077-1082. https://doi.org/10.1089/jpm.2008.0023

Branch, T., & Paranjape, T. (2002). Feedback and reflection: Teaching methods for clinical settings. Academic Medicine, 77(12), 1185-1188. https://doi.org/10.1097/00001888-200212000-00005

Bravery Sayings & Bravery Quotes (n.d.). Bravery sayings and quotes. Retrieved from http://www.wiseoldsayings.com/bravery-quotes/

Brown, G. (1988). Effective teaching in higher education. London: Methuen.

Dickinson, G. (2013). End-of-life and palliative care education in US pharmacy schools. American Journal of Hospice and Palliative Medicine, 30(6), 532-535. https://doi.org/10.1177/1049909112457011

Douglas, T. (1976). Groupwork practice. London: Tavistock Publications.

Furstenberg, C., Ahles, T., Whedon, M., Pierce, K., Dolan, M., Roberts, L., & Silberfarb, P. (1998). Knowledge and attitudes of health-care providers toward cancer pain management: A comparison of physicians, nurses, and pharmacists in the state of New Hampshire. Journal of Pain and Symptom Management, 15(6), 335-349. https://doi.org/10.1016/S0885-3924(98)00023-2

Henning, M., Hu, J., Webster, C., Brown, H., & Murphy, J. (2015). An evaluation of Hospice New Zealand’s interprofessional fundamentals of palliative care program at a single site. Palliative & Supportive Care, 13(3), 725-732.

Hussainy, S., Marriott, J., Beattie, J., Nation, R., & Dooley, M. (2010). A palliative cancer care flexible education program for Australian community pharmacists. American Journal of Pharmaceutical Education, 74(2), 1-9. https://doi.org/10.5688/aj740224

Institute of Medicine (2015). Committee on Approaching Death: Addressing, Key End-of-Life Issues. Dying in America: Improving quality and honoring individual preferences near the end of life. Washington, D.C.: National Academies Press.

Knowles, M. (1984). The adult learner: A neglected species (3rd Ed.). Houston: Gulf Pub. Co.

Knowles, M., Holton, E., & Swanson, R. (2015). The adult learner. (8th Ed.). New York: Routledge.

Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, N.J.: Prentice-Hall c1984.

Lyons, R., Lazzara, E. H., Benishek, L. E., Zajac, S., Gregory, M., Sonesh, S. C., & Salas, E. (2015). Enhancing the effectiveness of team debriefings in medical simulation: More best practices. The Joint Commission Journal on Quality and Patient Safety, 41(3), 115. https://doi.org/10.1016/S1553-7250(15)41016-5

Morse, J. & Field, J. (1995). Qualitative research methods for health professionals (2nd Ed.). Thousand Oaks: Sage Publications.

Needham, D. S., Wong, I. C. K., & Campion, P. D. (2002). Evaluation of the effectiveness of UK community pharmacists’ interventions in community palliative care. Palliative Medicine, 16(3), 219-225. https://doi.org/10.1191/0269216302pm533oa

O’Connor, M., Fisher, C., & Guilfoyle, A. (2006). Interdisciplinary teams in palliative care: A critical reflection. International Journal of Palliative Nursing, 12(3), 132-137. https://doi.org/10.12968/ijpn.2006.12.3.20698

O’Connor, M., Pugh, J., Jiwa, M., Hughes, J., & Fisher, C. (2011). The palliative care interdisciplinary team: Where is the community pharmacist? Journal of Palliative Medicine, 14(1), 7-11. https://doi.org/10.1089/jpm.2010.0369

Ryan, J. (2003). Continuous professional development along the continuum of lifelong learning. Nurse Education Today, 23(7), 498-508. https://doi.org/10.1016/S0260-6917(03)00074-1

Saldaña, J. (2016). The coding manual for qualitative researchers (3rd Ed.). London: Sage.

Schön, D. (1983). The reflective practitioner. New York, NY: Basic Books.

Steinert, Y., Cruess, S., Cruess, R., & Snell, L. (2005). Faculty development for teaching and evaluating professionalism: From programme design to curriculum change. Medical Education, 39(2), 127-136.

Vernon, J., Brien, J., Dooley, M., & Spruyt, O. (1999). Multidisciplinary treatment paths and medication management of palliative care patients. Proceedings of the Australian Society of Clinical and Experimental Pharmacologists and Toxicologists, 7, 74.

*Andrea Thompson
University of Auckland,
Building 599, Level 12,
Auckland City Hospital
Private Bag 92019,
Auckland 1142
Tel: +6499231906
Email: andrea.thompson@auckland.ac.nz

Published online: 2 May, TAPS 2017, 2(2), 1-7
DOI: https://doi.org/10.29060/TAPS.2017-2-2/OA1012

Mae Yue Tan¹, Jia Ming Low¹, Kay Choong See² & Marion M Aw^1,3

¹Khoo Teck Puat-National University Children’s Medical Institute, National University Health System, Singapore; ²University Medicine Cluster, National University Hospital, Singapore; ³Department of Paediatrics, Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Abstract

Aim: We aimed to combine self-report measures with physiological measures (using a wrist actigraph) to compare and quantify the difference in activity levels, sleep, fatigue and professional quality of life between residents and faculty.

Methods: All year 1 post-graduate (PGY1) residents and faculty were invited to participate. Participants were required to wear a wrist actigraph for four months, and complete two validated surveys (i.e. Epworth Sleepiness Scale(ESS) and the Professional Quality of Life(ProQoL)) once each at the start and again at the end of the study.

Results: Forty-nine PGY1 residents and eleven faculty participated. Residents logged less sleep in a working day than faculty, with median (range) of 356 (114-630) minutes versus 393 (162-704) minutes (p=0.001). Residents had decreased sleep latency, with a shorter median time to fall asleep (6 versus 7 minutes) (p=0.001). Residents walked more, with median (range) daily steps of 10207.5 (280-26638) versus 8155 (481-31236) (p=0.001). At study start and end, residents (61.5%, 69.2%) and faculty (30%, 44.4%) both reported ESS>10, suggestive of excessive daytime sleepiness (p>0.05). Residents reported higher burnout scores compared to faculty; 34.9% versus 0% (p=0.024) and 23.1% versus 0% (p=0.304), and lower compassion satisfaction scores; 25.6 vs 0% (p=0.095), 37% versus 0% (p=0.039), for both time points.

Conclusion: Although faculty sleep more than PGY1 residents, they are similarly at risk of fatigue. However, faculty experience lower burnout and higher compassion satisfaction scores. Whilst lack of sleep and fatigue can contribute to burnout, faculty likely possess protective mechanisms, which residents have yet to acquire.

 Keywords:        Sleep; Activity; Fatigue; Burnout; Actigraph

Practice Highlights

• PGY1 residents are at risk of greater fatigue, burnout and experience less compassion satisfaction compared to faculty members.
• Fatigue is inevitable in the medical profession. Measures need to be taken and life skills taught to help junior doctors cope with the rigours of work and training, that faculty may have acquired over time.
• Future studies can be done to explore if teaching these strategies early would be beneficial to junior doctors.

I. INTRODUCTION

The training years of a doctor are a stressful period during which one has to work long hours, hold huge responsibilities in the workplace yet has relatively little control over. This is likely experienced for the first time during the first post-graduate year (PGY1) as a doctor. It is a perceived notion that senior doctors are less fatigued than junior residents in the hospital as they have shorter work hours, do not do first-line ward work and do not usually do stay-in calls. Ward work includes interviewing and examining newly admitted patients, creating admission histories, writing admission orders, performing phlebotomy services, following-up  on  laboratory  results, reporting  interim updates to their supervising resident and doing discharge summaries. These duties are physically laborious resulting in longer work hours and potentially less sleep time. We hypothesized that the nature of such duties for PGY1 doctors are linked to increased reports of fatigue and burnout.

We know that fatigue and burnout have detrimental effects on residents, and there might be long-lasting implications on their physical and mental well being (Lourenção, Moscardini, & Soler, 2010; Fletcher, Reed, & Arora, 2009). An observational analysis by Lee et al suggested that increased night duty loading of physicians caused elevated blood pressure and sympathetic tones in a dose-dependent manner (Lee et al., 2016). Studies have reported increased somatic complaints, depression and increased physical dangers in residents who are more fatigued (Veasey, Rosen, Barzansky, Rosen, & Owens, 2002).

An article published in the Annals Academy of Medicine Singapore reported that junior doctors have lower health-related quality of life scores compared to senior doctors (Tong, Tin, Tan, & Lim, 2012). This paper aims to study difference in activity levels, sleep, fatigue and professional quality of life between residents and faculty. Apart from this article, there are no known local data comparing degree of fatigue and burnout between faculty and residents.

Our study aims to combine self-report measures with physiological measures (using a wrist actigraph) to compare and quantify the difference in activity levels, sleep, fatigue and professional quality of life between residents and faculty.

II. METHODOLOGY

A. Participants
All PGY1 residents in the institution were invited to participate. They could be rotating in any of these departments: Medicine, Paediatrics, Orthopaedics, Obstetrics and Gynaecology and Surgery. All the faculty members were recruited from either the Medicine or Paediatric department, who were at least junior consultant and above in rank. Participants were asked if they had known medical conditions. Exclusion criteria were doctors with psychiatric, sleep disorders or cardiac problems. Target recruitment was 60 PGY1 residents, and 20 faculty members. At our institution, work hours for all junior doctors, are governed by ACGME-I (Accreditation Council of Graduate Medical Education- International) duty hour requirements. Depending on which department residents are posted to, they work an on-call rota (approximately 4-6 on-calls per month), or work 5 nights a week on the night float system (approximately once every 2 months). Despite being in different departments and having different on-call schedules, the job scopes for every PGY1 resident are similar as they involve ward work. Faculty members participate in clinical work, education, as well as administrative work in the department. They predominantly do stay-home on-calls.

B. Instruments Used
The study comprised of two components; (1) Wearing a wrist actigraph for a continuous period of four months. (2) Completing two questionnaire surveys; the Epworth Sleepiness Scale (ESS) and the Professional Quality of Life (ProQoL), once each at the start and again at the end of the study. These two surveys were repeated at the start and end of the study. The wrist actigraph (Martin & Hakim, 2011) (FitBit) is a validated alternative to polysomnography for detecting sleep and estimating sleep duration and activity level. Actigraphy (Takacs et al., 2014) uses a single channel that collects data on movement, which is used to infer time spent asleep and awake. It was chosen as a study tool to collect sleep and activity data in our residents as it is less cumbersome than polysomnography, less expensive and can be worn for extended periods of time.

The ProQoL instrument (Proqol, 2016), initially created for therapists, nurses, and humanitarian workers, is a widely used tool to assess positive and negative aspects of caring. It comprises of the positive (Compassion Satisfaction) and the negative (Compassion Fatigue or Secondary Trauma Stress) aspects of helping others who have experienced suffering. Compassion Satisfaction is defined as the pleasure derived from being able to do one’s work well. Higher scores on this subscale represent greater satisfaction with one’s ability to be an effective caregiver. The Compassion Fatigue or Secondary Trauma Stress subscale measures work related secondary exposure to extremely stressful events and higher scores represent greater Compassion Fatigue. The three aspects of the ProQol were scored based on cutoff scores as per the validated scoring, and analysed as categorical data. Compassion Satisfaction scores of ≥ (more or equal to) 57 were defined as low (lower positive aspects of working as a helper), Burnout scores of ≥ 57 were defined high (inefficiency and feeling overwhelmed), and Traumatic scores of ≥ 57 were considered high.

The Epworth Sleep Study Scale (ESS, Epworth sleepiness scale, 2016) is a self-administered questionnaire with eight questions. It provides a measure of a person’s general level of daytime sleepiness, or  their  average  sleep  propensity  in daily life. This simple questionnaire has been validated and has shown reliability as a measure of daytime sleepiness. In the ESS questionnaire, scores of >10 suggest increased daytime sleepiness. Day time sleepiness is defined as a subjective report of difficulty in maintaining the alert awake state, usually accompanied by a rapid entrance into sleep when the person is sedentary (Thorpy, 2012). While sleepiness and fatigue are not synonymous, excessive daytime sleepiness contributes significantly to the feeling of fatigue (Lavidor, Weller, & Babkoff, 2003). We defined severe sleepiness as sleep episodes that are present daily and at times of physical activities that require mild to moderate attention, and this degree of sleepiness is usually associated with mean sleep latency of less than 5 minutes (Mansukhani, Kolla, & Ramar, 2014).

C. Procedure
The National University of Singapore (NUS) Institutional Review Board approved this study. Information about the study was presented to PGY1 residents across all departments during their weekly continuing medical education sessions over four weeks prior to the launch of the study. Posters were placed in the resident’s lounge and lift lobbies around the hospital to recruit residents. Participants provided voluntary written informed consent. Participants were allowed to keep the wrist actigraph on completion of the study.

Study participants were instructed to wear the wrist actigraph on their non-dominant hand for 24 hours per day throughout the study period. The wrist actigraph data were downloaded weekly or every fortnightly into an actigraphy-based scoring software program (Detalytics integrated biometric analytics) which enables calculations of sleep duration, sleep latency (time taken to fall asleep) and sleep efficiency (at least 85% of non-restless sleep). Sleep efficiency is the ratio of total time spent asleep (total sleep time) in an evening to the total amount of time spent in bed (Thorpy, 2012).

D. Data analysis
Data was analyzed using SPSS software. The Mann Whitney test was used for continuous data (e.g number of steps, minutes of vigorous/moderate activity) where there was a non-normal distribution. (Figure 1). This non-parametric test was also chosen because of the disparity between the sample sizes of PGY1 and Faculty group suggesting that the variances between groups to be heterogeneous (hence violating the homogeneity of variance assumption that ANOVA requires).

The chi-square test was used for categorical data (e.g ESS scores, and ProQoL scores were categorically split based on the defined cut-offs). We chose these methods of comparison to compare the activity and sleep between the two groups, and then determining whether these differences in activity and sleep had any trends with the ESS and ProQoL.

PGY1= Post-Graduate Year 1
Figure 1. Histograms showing data distribution for continuous data collect between the two groups.

III. RESULTS

Forty-nine residents and eleven faculty members were recruited. The study was of a 4 month-period, from 1st January 2015 through 30th April 2015. A higher percentage of faculty members (91%) had complete actigraph data throughout the 4-month study duration compared to residents (Table 1a). Of the 49 residents, 22.4% had complete data, and an additional 46.9% had partial data. About 30.6% of residents had less than 1 month worth of data logs from the actigraph device. The response rate for the surveys at both time points of the study are shown in Table 1b. Baseline characteristics of the residents who had complete or partial/incomplete data are shown in Table 2.

Table 1a. Response rate for Activity and Sleep Data Capture

Table 1b. Response rate for Survey Data Capture

Table 2. Baseline characteristics of the residents, between the group who had complete and incomplete data

PGY1 residents have higher levels of activity compared to faculty, with a median (range) of steps of 10207.5 (280-26638) versus 8155 (481-31236) (p=0.001). They also logged significantly longer duration of both moderate and vigorous activity (26 versus 19 minutes, 95 versus 70 minutes, respectively) compared to faculty (Table 3).

In terms of the sleep comparison, PGY1 residents slept less compared to faculty, with a median (range) of 356 (114-630) minutes versus 393 (162-704) minutes (p=0.001). The time taken to fall asleep (sleep latency) was also significantly shorter (6 versus 7 minutes for the residents and faculty respectively). In addition, a higher percentage of residents (36.4% versus 18%) had fallen asleep in less than 5 minutes, correlating with severe sleepiness. Both groups showed similar sleep efficiency (Table 3).

Table 3. Quantitative comparison of activity and sleep measured by wrist actigraph (FitBit)

With regard to the written survey data (Table 4), PGY1 residents had a trend to report higher fatigue scores on the ESS compared to faculty at both time points of the study, although this was not statistically significant. For the ProQoL survey, a significant higher percentage of residents reported higher burnout scores, lower compassion satisfaction scores, and higher traumatic scores. This again was consistent across both time points of the study (Table 4).

Table 4. ESS & ProQoL scores of residents and faculty at beginning and at the end of the study

IV. DISCUSSION

Our study is not without its limitations. There was a relatively high dropout rate within residents, with only a third of the residents wearing the wrist actigraph for the entire duration of the study, and just over half of the cohort completing the end of study surveys. While the population within the resident group was almost homogenous with no difference between the group that had complete and incomplete data, we cannot be entirely certain that the residents whose data was available for analysis are entirely reflective of the whole cohort. They are likely to represent the more motivated residents who comply with the study protocol. We also recognize that use of surveys may be open to recall bias, and possibly the residents who are more fatigued are the ones completing the survey.  As the drop-out rate was fairly high, repeated measures of matched data were not used to analyze the survey data at the two different time points.

Despite these limitations, our study demonstrated several interesting findings, some of which were not entirely expected. We showed that in general, faculty members sleep more and are less physically active compared to residents. However, they are also at risk of excessive daytime sleepiness with a third to half of them reporting ESS scores > 10. Although faculty members report fatigue, they appear to experience less burnout compared to residents. They also have higher compassion satisfaction scores and lower traumatic scores. These suggest that residents derive less satisfaction in their jobs compared to faculty and these findings are consistent with a local study done by Tong et al (Tong, Tin, Tan, & Lim, 2012).

Whilst lack of sleep and fatigue can contribute to burnout, faculty members are likely to have more control over their time and work schedules. In addition, with time and experience, they have also acquired the ability to cope better with the rigors of work (Girard, Choi, Dickey, Wessel, & Austin, 2006). Further studies can be done to identify what these protective factors are that faculty have acquired over the years. These can then be shared with and taught to residents.

Notes on Contributors

Marion M Aw is an Associate Professor in the Department of Paediatrics, Yong Loo Lin School of Medicine (YLL SoM), National University of Singapore. In addition to being the Program Director for Paediatric Residency Training at the National University Health System (NUHS), she also serves as the Associate Chairman, Medical Board (Education) for NUHS.

See Kay Choong is a Consultant Respiratory Physician and Intensivist, and the Program Director of the Research Residency Program, National University Hospital, Singapore. His research interests include healthcare quality improvement, epidemiology and biostatistics and the theory and practice of medical education.

Low Jia Ming and Tan Mae Yue are third year residents in the Department of Paediatrics at the National University Hospital of Singapore (NUH).

Ethical Approval

Standard institutional review board (IRB) procedures have been followed and approval obtained by the authors.

Acknowledgements

We thank Janelle Marie Liwanag and Wang Sha for assistance with data collection and Dr Dimple Rajgor for editing and submission of the manuscript.

Funding

Funding of the FitBit device was from the National University Health System Clinical Quality and Innovation Fund (NUHS CQIF).

Declaration of Interest

Authors have no conflicts of interest, including no financial, consultant, institutional and other relationships that might lead to bias.

References

Epworthsleepinessscale.com. (2016). The Epworth Sleepiness Scale. What the Epworth Sleepiness Scale is and how to use it – Dr Murray Johns. Retrieved 23 January 2016, from http://epworthsleepinessscale.com/about-epworth-sleepiness/

Fletcher, K., Reed, D., & Arora, V. (2009). Systematic review of the literature: resident duty hours and related topics: ACGME. Retrieved 25 April 2016 from https://www.google.com.sg/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjls-2v1KvMAhXNVI4KHUp5Cl4QFggfMAE&url=https%3A%2F%2Fwww.acgme.org%2FPortals%2F0%2FPDFs%2Fjgme-11-00-97-100%5B1%5D.pdf&usg=AFQjCNGZ9-9JKIGAeHzSHyhmjqfWfNt8uA&bvm=bv.120551593,d.c2E.

Girard, D. E., Choi, D., Dickey, J., Wessel, K., & Austin, D. (2006). A mid year comparison study of career satisfaction and emotional states between residents and faculty at one academic medical center. BMC Medical Education, 6(1), 36.

Lavidor, M., Weller, A., & Babkoff, H. (2003). How sleep is related to fatigue. British Journal of Health Psychology, 8(1), 95-105.

Lee, H., Lo, S., Chen, B., Lin, Y., Chu, D., Cheng, T., . . . Guo, Y. (2016). Increased night duty loading of physicians caused elevated   blood  pressure   and  sympathetic  tones  in  a   dose-dependent manner. International Archives of Occupational and Environmental Health, 89(3), 413.

Lourenção, L. G., Moscardini, A. C., & Soler, Z. A. S. G. (2010). Health and quality of life of medical residents. Revista da Associação Médica Brasileira, 56(1), 81-91.

Mansukhani, M. P., Kolla, B. P., & Ramar, K. (2014). International classification of sleep disorders 2 and American Academy of Sleep Medicine practice parameters for central sleep apnea. Sleep Medicine Clinics, 9(1), 1-11.

Martin, J. L., & Hakim, A. D. (2011). Wrist actigraphy. Chest Journal, 139(6), 1514-1527.

Proqol. (2016). Professional Quality of Life. Retrieved 23 January 2016, from http://www.proqol.org

Takacs, J., Pollock, C. L., Guenther, J. R., Bahar, M., Napier, C., & Hunt, M. A. (2014). Validation of the Fitbit One activity monitor device during treadmill walking. Journal of Science and Medicine in Sport, 17(5), 496-500.

Thorpy, M. J. (2012). Classification of sleep disorders. Neurotherapeutics, 9(4), 687-701.

Tong, S. C., Tin, A. S., Tan, D. M., & Lim, J. F. (2012). The health-related quality of life of junior doctors. Annals, Academy of Medicine, Singapore, 41(10), 444.

Veasey, S., Rosen, R., Barzansky, B., Rosen, I., & Owens, J. (2002). Sleep loss and fatigue in residency training: a reappraisal. JAMA, 288(9), 1116-1124.

*Marion M Aw
Department of Paediatrics, National University Health System,
1E Kent Ridge Road,
NUHS Tower Block, Level 12, Singapore 119228
Tel: 65 6772 4112
Email: marion_aw@nuhs.edu.sg

Published online: 1 June, TAPS 2016, 1(1), 9-19
DOI: https://doi.org/10.29060/TAPS.2016-1-1/OA1003

Yu Han Ong¹, Issac Lim¹, Keng Teng Tan², Mark Chan³, Wee Shiong Lim^1,3

¹Health Outcomes and Medical Education Research (HOMER), National Healthcare Group, Singapore; ²Pharmacy Department, Tan Tock Seng Hospital, Singapore; ³Department of Geriatric Medicine, Tan Tock Seng Hospital, Singapore

Abstract

Background: Shared leadership, a team property whereby leadership is distributed among team members, is increasingly salient in interprofessional team-based care. There is currently no validated scale to measure shared leadership in healthcare teams. We aim to describe the developmental and validation of the Clinical Shared Leadership Scale (CSLS) in the context of interprofessional geriatrics care.

Methods: We adapted the CSLS from the Woods (2005) and Carson (2007) scales that were originally used to study management teams. We collected survey data from 115 healthcare professionals who attended interprofessional team meetings (IPTM) in two subacute geriatrics ward. We analyzed internal consistency using Cronbach’s α, as well as construct, convergent, divergent, concurrent and predictive validity using exploratory factor analysis (EFA), inferential statistics and logistic regression.

Results: The 14-item CSLS scale has mean score of 52.32±4.86 (range: 14-70). The scale exhibited high internal reliability (Cronbach’s α=0.76). EFA identified three factors, namely social cohesion, joint involvement, and hierarchical structure (α: 0.76, 0.80 and 0.46 respectively). The good correlation of CSLS total score with internal team environment (ITE) (r=0.78, p<.01) and transactive memory system (TMS) (r=0.65, p<.01) supports convergent validity, whereas poor correlation with task complexity (r=0.17, p=.08) corroborates divergent validity. CSLS total scores increase significantly with increasing number of IPTM attended, higher ITE and higher TMS scores (p<.05), indicating concurrent validity. CSLS total and factor scores, with the exception of factor 3, predicted satisfaction outcomes.

Conclusion: The CSLS is a reliable and valid measure to assess shared leadership in interprofessional team meetings. The third factor, hierarchical structure, merits further study.

Keywords: Clinical Shared Leadership Scale, Interprofessional team, Shared leadership, Validation, Geriatrics

Practice Highlights

• Shared leadership refers to a property among teams whereby leadership is distributed among team members to achieve group or organizational goals.
• The Clinical Shared Leadership Scale (CSLS) is a reliable and valid instrument to assess shared leadership in interprofessional team meetings.
• The three dimensions of CSLS are social cohesion, joint involvement and hierarchical structure.
• The third factor, hierarchical structure, merits further study to ascertain if it also applies in other healthcare settings.

I. INTRODUCTION

Traditionally, the study of leadership in the healthcare context has focused on the characteristics of individual leaders (Bass, 1992). In recent years, interprofessional collaborative work is increasingly required of healthcare professionals to render effective patient care. For instance, in geriatrics care, the professional expertise of different healthcare professionals needs to be tapped upon to effectively and efficiently manage the complex interacting medical, functional and social issues in frail elderly patients (Tan et al., 2012). This has resulted in the concomitant shift in leadership trends from a top-down approach to a dynamic collaborative decision-making approach. Shared leadership, a well-developed concept in business and organizational literature (Pearce & Sims, 2002; Mayo, Meindl & Pastor, 2003; Pearce, 2004; Pearce & Manz, 2005; Pearce, Hoch, Jeppesen, & Wegge, 2009), is thus salient in enhancing our understanding of the leadership dynamics that occur in interprofessional teams. Although several studies (Avolio et.al., 2003; Hiller, Day and Vance, 2006; Hoch, Dulebohn and Pearce, 2010a; Grille & Kauffeld, 2015) have reported validated scales to assess shared leadership, none of these scales have been validated in the healthcare setting.

A. Shared leadership

Shared leadership is defined as ‘a dynamic, interactive influence process among individuals in groups for which the objective is to lead one another to the achievement of group or organisational goals’ (Pearce and Conger, 2003; p.1). It is a process of dynamic delegation (Dow et al., 2013; Klein et al, 2006) in which leaders emerge in teams when their expertise are required, such that leadership is “coproduced” rather than hierarchical or concentrated in formally appointed leaders (Shamir,2007; Carsten & Uhl-Bien, 2013). Rather than following distinct leaders, team members work together interdependently, relying on each other’s expertise to accomplish goals and carry out tasks in lateral team structures (Orchard et al., 2010). Related concepts that have been previously mentioned in the healthcare literature include collaborative leadership and collective leadership (West et al., 2014). This collaborative approach of shared leadership is especially germane to decision-making processes within teams involved in complex care such as interprofessional healthcare teams, whereby the various inter-professional members “share” leadership responsibilities in patient care by bringing to bear unique expertise and experience, depending on the requirements of each situation (Lingard et al., 2012).

Previous studies reported that shared leadership is associated with improved healthcare team outcomes (Kunzle et al., 2010), effective discharge in the acute setting (Pethybridge, 2004), and higher levels of team satisfaction (Steinert, Goebel & Rieger, 2006). However, the process of shared decision-making amongst interprofessional team members may be hampered by inherent hierarchical structures arising from contesting professional identities (Kvarnstro¨m, 2008; Reeves et al., 2010; Dow 2013). In many cases, this was attributed to the prevailing culture of medical leadership in many healthcare teams (Roger, 2012). Interestingly, Lingard et al. (2012) reported that whilst doctors perceived that their teams work in a nonhierarchical manner, this contradicted her observational data as well as perceptions from other professional groups.

Despite the emerging importance of shared leadership and related concepts in the healthcare literature (Dow et al., 2013; Leasure et al., 2013; Lingard et al., 2012; Rogers, 2012; Kunzle et al., 2010; Klein et al, 2006; Steinert, Goebel, & Rieger, 2006), there is currently no validated instrument to assess shared leadership in interprofessional healthcare teams. There is also a significant gap in the understanding of what constitutes shared leadership in the healthcare setting, and its role in bringing about effective interprofessional collaboration. Hence, we developed an instrument, the Clinical Shared Leadership Scale (CSLS), to assess shared leadership as a measure of interprofessional collaborative practice in shared decision-making. Our aim was to validate the CSLS by determining its internal consistency, factor structure, as well as convergent, discriminant, concurrent and predictive validity in interprofessional team meetings in geriatrics care.

II. METHODOLOGY

A. Participants and settings

This is a cross-sectional study of healthcare professionals who attended interprofessional team meetings (IPTM) at two Geriatrics sub-acute care wards of a tertiary teaching hospital in Singapore. The Geriatrics sub-acute wards accept the following categories of general medical elderly patients aged 65 years and above (mean age >80) who are primarily transferred from the acute wards and to a lesser extent, admitted directly from the Emergency Department or outpatient clinics : 1) continued management of medical conditions; 2) short course of functional rehabilitation (less than 2 weeks) to address deconditioning following the acute illness episode; 3) medically stable and awaiting transfer to a step-down care facility; and 4) management and medication titration of behavioural problems from dementia or delirium (Chong, Empensando, Ding, & Tan, 2012). Patients excluded for transfer to the subacute ward include those who are dangerously ill, unstable in parameters, awaiting urgent surgical procedures, and requiring oxygen therapy (except when on long term oxygen therapy for chronic lung conditions). The interprofessional team at the subacute wards comprises doctors, nurses, pharmacists, physiotherapists, occupational therapist, speech therapists, dieticians, care coordinators and medical social workers to meet the complex and interacting medical, functional and psychosocial needs of the patients. Not surprisingly, given the amount of nursing care required by the frail elderly patients, nurses constitute the largest professional group of the ward team.

The IPTM is a platform where interprofessional team members meet biweekly to discuss respective responsibilities in order to coordinate, negotiate and plan for discharge or right siting of frail elderly patients with multiple interacting medical, functional, and/or social issues (Tan et al., 2012). To fulfil its functional role, IPTM not only allows team members to have a shared team mental model in the form of a collective transactive memory system ( Tan et al., 2012), it also serves as a platform for shared leadership in which team members are expected to assume a leading role when their expertise is required in the process of decision-making for patient care. Leadership is thus distributed among different team members at different points of time depending on the situation. Senior geriatricians play an important role in the overall facilitation of IPTMs. This study was approved by the Institutional Review Board of the National Healthcare Group, Singapore.

B. Study procedures

We invited healthcare professionals who attended IPTM in the preceding year to participate in the survey. Using pre-defined categories from an earlier study (Tan et al., 2014), we collected demographic details on age, gender, and clinical roles (doctors, nurses, therapists, social workers/care coordinators, and others). There was no pharmacist in our study because the only pharmacist attached to the geriatrics department is a member of our study team. We measured the level of experience in three ways: years in clinical service, exposure to geriatrics care services, and number of interprofessional team meetings attended. We collected data on satisfaction scores to ascertain the predictive validity of the CSLS as a measure of interprofessional collaborative practice. Participants were asked to rate their satisfaction on a 5-point Likert scale (1 for “very dissatisfied” through 3 for “neutral” to 5 for “very satisfied”) in two areas: 1) overall experience of IPTM (with reference to the last five meetings attended), and 2) working with the interprofessional geriatrics team. We also collected data on shared leadership, internal team environment (ITE) (Carson, Tesluk, & Marrone, 2007), Transactive Memory System (TMS) (Tan et al., 2012) and task complexity (Podsakoff & MacKenzie, 1994).

C. Instruments

1) Shared leadership:

We adapted the Shared Leadership Perception Survey (Wood, 2005), which originally measured the occurrence of shared leadership by pastors within church management teams. We selected this scale because it assesses shared leadership at the team level with respect to: 1) team behaviour, 2) team structure, and 3) team members’ tendency to share leadership. Due to inappropriateness of three items under the “emotional support” subscale (Wood, 2005) in the healthcare setting, we substituted with questions from the “social support” subscale of Carson’s (2007) work on management teams. The initial version of our CSLS comprised nineteen items. After a pilot study for our first draft, the study team in consult with experts in geriatric care reviewed the items for relevance to the geriatrics context. We omitted 4 items that lacked conceptual alignment with shared leadership in the IPTM setting, yielding the 15-item version that was used in the study. Each item was rated on a 5-point Likert scale (1 for “strongly disagree” through 3 for “neutral” to 5 for “strongly agree),

2) Transactive memory system:

A TMS ‘‘is a set of individual memory systems in combination with the communication that takes place between individuals’’ (Wegner, 1986, p. 186). Such a memory system is essential when leading interprofessional   team  meetings because members need to know what information exists, where this information resides, and how to retrieve it to meet the complex medical, functional, and social needs of hospitalized elderly patients. Conceptually, shared leadership concerns collective influence, whereas TMS concerns collective cognition (Carson, Tesluk,& Marrone, 2007). Although the two concepts appear distinct, TMS is likely to be facilitated by shared leadership, through communications and social exchanges (Klimoski & Mohammed, 1994) that occur as team members engage in shared decision-making to solve a common task.

In our study, we used a validated 11-item TMS scale predicated on a two-factor structure, namely team dynamics and team knowledge (Tan et al., 2014). This scale was validated in the interprofessional team meeting context and it measures how knowledge resource is interdependently managed by team members. Each item was scored on a 5-point Likert scale and the possible total score of the scale is 55. The original TMS scale was developed in software management teams (Faraj and Sproull, 2000).

3) Internal team environment:

Previous literature showed that shared purpose (Sims, Hewitt, & Harris, 2015) and communication (Daiker, 2009) are essential for shared leadership and interprofessional work to succeed. According to Carson, Tesluk & Marrone (2007), internal team environement (ITE) is an important predictor for the emergence of shared leadership. In our study, we adopted two subscales, i.e. shared purpose (3 items) and voice (3 items), from the ITE scale described by Carson, Tesluk & Marrone (2007). Shared purpose exists when team members have similar understanding of their team’s goals and take steps to ensure a focus on collective goals. Voice refers to the degree to which team members have input into how the team carries out its goals (Carson, Tesluk & Marrone, 2007). Each item of the ITE scale was scored on a 5-point Likert scale, thus yielding a possible total score of 30.

4) Task complexity:

D’innocenzo et al. (2014) stated that there are three underlying components of task complexity: (a) component complexity, (b) coordinative complexity, and (c) dynamic complexity. In our study, we adapted the task complexity scale (Podsakoff & Mackenzie, 1994) that focuses on component complexity (i.e., the number of distinct acts and information cues needed to be attended to while performing the task) in order to explore how routine were the members’ daily jobs and how it affects leading responsibilities to be shared in teams. Because coordination and communication (i.e. coordinative complexity) among team members are relatively more important in geriatrics care than component complexity (Daiker, 2009), the latter aspect of task complexity measures a different construct from shared leadership in our study setting of geriatrics IPTM.

D. Statistical analysis

Descriptive and analytical statistics were performed using STATA version 12. The level of statistical significance was set at 0.5. To validate the CSLS, we performed the following analysis for both total and factor scores: (1) construct, convergent, discriminant, concurrent, and predictive validity; and (2) internal consistency using Cronbach’s α.

To ascertain construct validity, we conducted exploratory factor analysis (EFA) to determine the factor structure of CSLS, and then performed correlation analysis between the total and derived factor scores. The Kaiser – Meyer – Olkin (KMO) (Kaiser, 1970) statistic was used as a measure of sampling adequacy and the Bartlett test of sphericity was used as a measure of the necessity to perform a factor analysis (Bernstein, Garbin & Teng, 1988). We determined the number of factors to be retained via parallel analysis (Horn, 1965), a more robust and accurate method of factor retention that shows the least variability and was less likely to overestimate the number of factors (Wetzel, 2012). We eliminated items with loadings <0.3.

In convergent validity, we examined the degree to which the operationalization converges on other operationalizations that it theoretically should be similar to. ITE has been shown to be an important antecedent of shared leadership (Carson, Tesluk & Marrone, 2007), whereas TMS measures team dynamics and team expertise that facilitate interprofessional collaborative practice (Tan et al., 2012). Thus, to demonstrate convergent validity, we examined the correlation between CSLS total and factor scores with and the related concepts of ITE and TMS. In comparison, discriminant validity refers to the degree to which the operationalization diverges from other operationalizations that it theoretically should be not be similar to. Hence, to demonstrate discriminant validity, we performed correlation with the component aspect of task complexity scores, which is a distinct aspect from shared leadership.

In concurrent validity, we assessed the operationalization’s ability to distinguish between groups that it should theoretically be able to distinguish between. In the case of shared leadership, this would include related constructs such as level of experience, ITE, and TMS. An earlier study of geriatrics IPTM by Tan et al. (2014) indicated a dose-dependent relationship between team functioning with number of IPTM attended, but not with years in clinical service or exposure to geriatrics service. Thus, for concurrent validity, we compared CSLS total and factor mean scores across number of IPTM attended (66^th percentile cutoffs to derive two categories, ≤15 and >15) via independent sample t-test, and tertile cutoffs of ITE and TMS scores using one-way ANOVA. We also ascertained predictive validity via logistic regression to evaluate the significance of CSLS total and factor scores in predicting satisfaction outcomes, adjusting for number of IPTM attended, IPTM roles, age group and gender. As the dependent variable in logistic regression needs to be categorical, we transformed the satisfaction score into a dichotomous variable that comprises ‘‘not satisfied/neutral’’ and ‘‘satisfied’’ categories.

III. RESULTS

A. Baseline Characteristics (Table 1)

Among 130 healthcare workers invited to take part in the survey, 115 (88% response) completed the survey questionnaire. The majority of the respondents were nurses (42%), predominantly females (90%), had greater than three years of experiences in geriatrics care (47%), and attended more than ten interprofessional team meetings (60%). The mean satisfaction scores for overall interprofessional team meeting experiences and work with IP geriatrics teams were 3.99 (SD = 0.61) and 4.09 (SD = 0.59) respectively. The CSLS mean total score was 52.32 (SD=4.86; total possible score of 70), whilst the mean factor scores were 22.58 (SD=2.77; total possible score of 30), 20.94(SD=2.29; total possible score of 25) and 8.85(SD=1.87; total possible score of 15) for social cohesion, joint involvement, and hierarchical structure, respectively.

Table 1. Baseline characteristics of survey participants (n=115)

Note. IP = Interprofessional; IPTM = Interprofessional Team Meetings

^a Frequencies may not add up to the total number of respondents due to non-response for that question

^b Number (percentage) presented unless otherwise indicated.

^c Items loaded in accordance with factor loading of Table 2: factor 1, social cohesion (6 items); factor 2, joint involvement (5 items); factor 3, hierarchical structure (3 items).

B. Construct validity (Table 2)

Factor analysis was appropriate as the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.78, and the Barlett test of sphericity was 479.31 (p<0.0001). We chose a three-factor solution, as per the optimal number recommended by parallel analysis (Horn, 1965). Both orthogonal (varimax) and oblique (direct oblimin) rotations yielded a three-factor solution with similar items loading on each factor. The item “Team members interject (even if outside an area of personal responsibility) to ensure the team fulfills its obligations” was dropped because of its low loading in all factors. Thus, there were 14 items in the final version of CSLS, which accounted for 51% of variance. The first factor (28.90% of variance) represented social cohesion; the second factor (13.20% of variance) represented joint involvement; and the third factor (9.04% of variance) represented hierarchical structure. There was good correlation between CSLS total and factor scores (r>0.50, p<0.01). Factor 2 was moderately correlated with factor 1 (r=0.42, p<0.01), whereas both were poorly correlated with factor 3 (r=0.04 and 0.21). The overall Cronbach’s alpha for the CSLS is 0.76. Cronbach’s alpha of the first, second and third factors were 0.76, 0.80 and 0.46 respectively.

Table 2. Principal component analysis of the Clinical Shared Leadership scale (varimax rotation)

Note. IPTM = Interprofessional Team Meetings

Boldtype indicates factor to which item is allocated.

^a Item 7 was removed from the final 14-item Clinical Shared Leadership Scale; ^b Reverse-coded items

C. Convergent and Discriminant Validity (Table 3)

Both ITE and TMS, were significantly correlated with CSLS total scores (r: 0.78 and 0.65 respectively, p<0.01), supporting the convergent validity of CSLS. For factor scores, factors 1 and 2 have higher correlation with ITE and TMS (Range of r: 0.45 – 0.70, all p<0.01), compared with factor 3 (Range of r: 0.20 – 0.31, p<0.05). In contrast, task complexity as measured by involvement of multiple professions and routine series of job steps, was poorly correlated with both CSLS total (r: 0.12 and 0.15 respectively, p>0.05) and factor scores (Range of r: 0.05 – 0.16, all p>0.05), thus corroborating the discriminant validity of CSLS.

Table 3. Correlation between 14-item CSLS total and factor scores with internal team environment, transactive memory system, and task complexity scores

*p < 0.05; **p < 0.01

D. Concurrent Validity (Table 4)

CSLS total mean scores showed a significant increase with increasing number of IPTM attended (≤15 vs >15 meetings: 51.14 vs 53.59, p=0.01), higher ITE scores (tertile cutoffs: 50.16 vs 52.06 vs 58.23, p<0.01) and higher TMS scores (tertile cutoffs: 49.53 vs 51.59 vs 56.03, p<0.01). This attests to the concurrent validity of the CSLS. For the factor scores, there was a significant difference across categories for ITE and TMS (p<0.05), although for number of IPTM attended, there was a significant difference only factor 2 with a trend for factor 1.

Table 4. Comparison table of total and factor scores by baseline characteristic, internal team environment and transactive memory system

Note. IPTM = Interprofessional Team Meetings

*p < 0.05; **p < 0.01

E. Predictive validity (Table 5)

In logistic regression adjusting for IPTM attended, IPTM roles, age group and gender, CSLS was a significant predictor of satisfaction working with interprofessional geriatrics teams (OR = 1.37, p<0.01) and satisfaction with overall IPTM experience (OR = 1.37, p<0.01). Similarly, factors 1 and 2 significantly predicted satisfaction outcomes, especially factor 2 in predicting satisfaction working with interprofessional geriatrics teams (OR: 3.09, p<0.01). In contrast, factor 3 did not predict either satisfaction outcomes.

Table 5. Logistic regression for satisfaction outcomes

Note. IP = Interprofessional; IPTM = Interprofessional Team Meetings

^a Odds ratio (95% confidence interval) displayed, adjusted for number of IPTM attended, IPTM roles, age group and gender.

*p < 0.05; **p < 0.01

IV. DISCUSSION

Earlier studies that investigated leadership skills in interprofessional practice focused largely on the characteristics of a formally appointed leader in teams. In line with the broader movement to reconsider the collaborative approach in leadership (Lingard et al., 2012), we drew heavily from previous work in the field of business management and geriatrics care to develop a validated tool to measure shared leadership. As far as we are aware, this is the first study to validate a shared leadership tool in healthcare. We demonstrated the novel use of the 14-item CSLS as a valid and reliable measure of interprofessional collaborative practice in the context of an interprofessional geriatrics team. Our study builds upon the body of evidence by explicating the three key dimensions that underpin shared decision-making processes in interprofessional team meetings, and by providing an assessment tool of shared leadership that can serve as an intermediary outcome to bridge the causal chain between interprofessional practice and patient outcomes (Lim, 2013).

The CSLS is predicated on the three-factor factor structure of: (1) social cohesion; (2) joint involvement; and (3) hierarchical structure. Factor 1 pertains to the emotional bonding and trust among team members whilst factor 2 refers to the collaborative approach through which individual team members with their unique clinical expertise and experience, are involved in task completion. Factor 3, corresponding to the dimension of ‘decentralized interaction among personnel dimension’ in Wood (2005), refers to the existence of hierarchical structure in teams – an important contextual consideration in interprofessional team meetings. Factors 1 and 2 have high mean scores, good internal consistency and are moderately correlated with each other. This supports the conceptual alignment of both factors, such that team members are better able to work collaboratively and effectively when they feel emotionally safe and socially attached to each other. In support of this, the high correlation of factors 1 and 2 with CSLS total scores, corroborates the perceived importance of social cohesion and joint involvement as integral aspects of shared leadership.

In contrast, factor 3 has lower mean scores, moderate internal consistency and is poorly correlated with the other two factors. However, we retained this dimension in our scale for three reasons. Firstly, by assessing hierarchical structure in teams, it measures a valid and fundamental aspect of shared leadership. As opposed to the vertical structure of leadership which is highly reliant on an appointed leader for effective team functioning, shared leadership emphasizes the distribution of leadership in a more “horizontal” team structure (Pearce & Manz, 2005). Secondly, our results support the concurrent validity, discriminant validity, and to a lesser degree, convergent validity of factor 3 as a distinct dimension. Thirdly, the inclusion of factor 3 offers unique and invaluable insights that explicate the complex construct of shared leadership amongst interprofessonal healthcare teams. Of note, the discrepant mean scores between factor 3 with factors 1 and 2, suggest that despite the perceived hierarchical structure of IPTMs, interprofessional team members value joint involvement and social cohesion within the team, an observation which is corroborated by the high satisfaction scores. In view of the above reasons, we submit that hierarchical team structure is an integral aspect of shared leadership that should be assessed, especially in the healthcare setting. Lingard et al. (2012) reported the existence of hierarchical structure in their seminal study of five high-functioning interprofessional healthcare teams working at teaching hospitals in urban Ontario, Canada. They highlighted that deeply embedded traditional health care, education, and medical-legal systems which reinforce the idea of physicians sitting at the top of the hierarchy, could pose a considerable challenge to implementing the ideal “flat” model of shared leadership with absolutely no hierarchy. Rather, by openly recognizing and discussing the tensions between traditional and interprofessional discourses of collaborative leadership, this may help different members of interprofessional teams work together more effectively.

Our results also shed light on important factors that influence shared leadership in interprofessional teams. Shared purpose and voice, subdomains of ITE adopted from Carson, Tesluk & Marrone (2007), were found to be highly correlated with shared leadership. When team members feel recognized within their team, they are more willing to share responsibility and commit to the team’s collective goals. In addition, during shared decision making at IPTMs, it is important that team members can tap upon and retrieve information from their shared memory system to set consensus goals to facilitate patient discharge, as reflected by positive correlation between TMS and shared leadership. Conversely, our results indicated that the component aspect of task complexity is dissociated from shared leadership, indicating both concepts measure different dimensions. This could be because interprofessional team members are proficient when completing tasks within their area of expertise, such that minimal contact with other professional groups is required to solve less complex tasks.

The concurrent validity of CSLS total and factor scores with increasing number of IPTM attended, ITE score, and TMS score, suggests a dose-dependent relationship of shared leadership with these covariates. We found a similar dose-dependent relationship with TMS in IPTMs (Tan et al., 2014). From the perspective of shared leadership, new-comers to the interprofessional team meeting team would initially be unfamiliar with the inherent team roles and team processes. With greater exposure to IPTMs, they will become more familiar with mutual team roles and responsibilities through collaborative knowledge sharing and communication (Kitaygorodskaya, 2006). Increasing familiarity with each other’s role encourages the formation of shared purpose and common mental models among team members, enabling the team to more efficiently and effectively formulate a common discharge plan. Over time, through increasing socialization into team processes and culture, members would then develop deeper bonding and attachment with each other and transcend profession-specific boundaries to norm and perform as a team (Tuckman, 1965), thus enhancing the distribution of leadership in the decision-making process during IPTMs.

Several limitations are worth highlighting. First, the limited number of items (N=3) in Factor 3 contributes to the low reliability (α=.46) of the subscale. Further research is required to better delineate the dimension of “hierarchical structure” among interprofessional teams in different healthcare setting, and what facilitates shared leadership despite the presence of hierarchy within team structures. It is unclear whether hierarchical structure would be more pronounced in certain societies with more deferential cultures, although evidence suggests that hierarchy in healthcare teams is also fairly pervasive across different societies (Reeves et al., 2010; Lingard et al., 2012; Dow et al., 2013). Secondly, the generalizability of the CSLS beyond the context of the IPTMs in a specialized sub-acute geriatrics ward remains to be established. In addition, our study was not designed to examine shared decision making with direct involvement of patients and their caregivers (Col et al., 2011). Nonetheless, by different interprofessional team members representing the unique perspective of the voice of the patient/caregiver, the interests of the patient/caregiver from the medical, functional and psychosocial angles can still be comprehensively considered and integrated in the management plan through shared leadership even though the patient/caregiver may not be directly involved in the IPTMs. Finally, our study through its quantitative nature was not designed to elucidate the underlying reasons for our findings. Further in-depth qualitative inquiry may shed more light into the complexity of the shared leadership construct and its dimensions.

V. CONCLUSION

Our study adds to the understanding on the specific dimensions of shared leadership in interprofessional team meetings in an Asian geriatrics care context. We describe three dimensions of shared leadership, namely social cohesion, joint involvement and hierarchical structure. Although internal consistency for hierarchical structure is moderate, it is retained as a factor because it measures a valid and unique aspect of shared leadership in the healthcare context. Factors that influence shared leadership include internal team environment, transactive memory system, and number of IPTMs attended. Our findings highlight the need for further studies to determine the applicability of the CSLS in other interprofessional settings, and in-depth qualitative research to explicate the dimension of hierarchical structure.

Notes on Contributors

Ms. Yu Han Ong contributed to the conception and design of the study, and the acquisition, analysis and interpretation of data, and drafted the paper. Mr. Issac Lim contributed to the conception and design of the study. Dr. Mark Chan and Ms. Keng Teng Tan contributed to the acquisition of data. Dr. Wee Shiong Lim contributed to the conception and design of the study, acquisition, analysis and interpretation of data and critical revision of the paper. All authors approved the final manuscript for publication.

Ethical Approval

The study was approved by the Institutional Review Board, National Healthcare Group. (NHG DSRB Ref: 2013/00973)

Acknowledgements

This study was supported by FY13 NHG Homer Grant.

Declaration of Interest

The authors declare that they have no competing interests.

References

Avolio, B. J., Sivasubramaniam, N., Murry, W. D., Jung, D., & Garger, J. W. (2003). Development and Preliminary Validation of a Team Multifactor Leadership Questionnaire. In C. L. Pearce, & J. A. Conger (Eds.), Shared Leadership: Reframing the Hows and Whys of Leadership (pp. 143-172). Thousand Oaks: Sage Publications.

Bass, B.M. (1992). Bass and stogdill’s handbook of leadership. New York: Free Press.

Bernstein I.H., Garbin C.P., Teng G.K.(1988). Applied multivariate analysis. Springer: New York.

Carson, J. B., Tesluk, P. E., & Marrone, J. A. (2007). Shared leadership in teams: An investigation of antecedent conditions and performance. Academy of Management Journal, 50(5), 1217–1234.

Carsten, M. K., & Uhl-Bien, M. (2013). Ethical followership: An examination of followership beliefs and crimes of obedience. Journal of Leadership & Organizational Studies: 20(1):45–57.

Chong, M. S., Empensando, E. F., Ding, Y. Y., & Tan, T. L. (2012). A subacute model of geriatric care for frail older persons: The Tan Tock Seng Hospital experience. Ann Acad Med Singapore, 41, 354-61.

Col, N. et al. (2011). Inter-professional education about shared decision making for patients in primary care settings. Journal of Interprofessional Care, 25(6), 409-415.

D’Innocenzo, L., Mathieu, J. E., & Kukenberger, M. R. (2014). A meta-analysis of different forms of shared leadership–team performance relations. Journal Of Management.

Daiker, B. L. (2009). Shared Leadership in a Medical Practice: Keys to Success. The Journal of medical practice management: MPM, 25(2), 111.

Dow, A. W., DiazGranados, D., Mazmanian, P. E., & Retchin, S. M. (2013). Applying Organizational Science to Health Care: A Framework for Collaborative Practice. Academic Medicine, 88(7), 952–957.

Faraj, S., & Sproull, L. (2000). Coordinating expertise in software development teams. Management science, 46(12), 1554-1568.

Grille, A., & Kauffeld, S. (2015). Development and Preliminary Validation of the Shared Professional Leadership Inventory for Teams (SPLIT). Psychology, 06(01), 75–92.

Hiller, N. J., Day, D. V., & Vance, R. J. (2006). Collective Enactment of Leadership Roles and Team Effectiveness: A Field Study. The Leadership Quarterly, 17, 387-397.

Hoch, J. E., Dulebohn, J. H., & Pearce, C. L. (2010a). Shared Leadership Questionnaire (SLQ): Developing a Short Scale to Measure Shared and Vertical Leadership in Teams. Visual Presentation at the Society for Industrial and Organizational Psychology (SIOP) Conference 2010, Atlanta.

Horn, J.L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185.

Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35(4), 401-415.

Kitaygorodskaya, N. (2006). Measurement of team knowledge: transactive memory system and team mental models. In Proceedings of the research forum to understand business in knowledge society (pp. 1-6).

Klein, K. J., Ziegert, J. C., Knight, A. P., & Xiao, Y. (2006). Dynamic delegation: Shared, hierarchical, and deindividualized leadership in extreme action teams. Administrative Science Quarterly, 51(4), 590-621.

Klimoski, R., & Mohammed, S. (1994). Team mental model: construct or metaphor? Journal of management, 20(2), 403-437.

Künzle, B., Zala-Mezö, E., Wacker, J., Kolbe, M., Spahn, D. R., & Grote, G. (2010). Leadership in anaesthesia teams: the most effective leadership is shared. Quality and Safety in Health Care, qshc-2008.

Kvarnström, S. (2008). Difficulties in collaboration: A critical incident study of interprofessional healthcare teamwork. Journal of interprofessional care, 22(2), 191-203.

Leasure, E. L., Jones, R. R., Meade, L. B., Sanger, M. I., Thomas, K. G., Tilden, V. P.,Bowen, J.L., Warm, E. J. (2013). There Is No “I” in Teamwork in the Patient-Centered Medical Home: Defining Teamwork Competencies for Academic Practice. Academic Medicine, 88(5), 585–592.

Lim, W. S. (2013). More About the Focus on Outcomes Research in Medical Education. Academic Medicine, 88(8), 1052.

Lingard, L., Vanstone, M., Durrant, M., Fleming-Carroll, B., Lowe, M., Rashotte, J., Sinclair L. & Tallett, S. (2012). Conflicting messages: examining the dynamics of leadership on interprofessional teams. Academic Medicine, 87(12), 1762-1767.

Mayo, M., Meindl, J. R., & Pastor, J. C. (2003). Shared leadership in work teams: A social network approach. In C. L. Pearce & J. A. Conger (Eds.). Shared leadership: Reframing the hows and whys of leadership (pp.193–214). Thousand Oaks, CA: Sage.

Orchard, C., Bainbridge, L., Bassendowski, S., Stevenson, K., Wagner, S. J., Weinberg, L., Curran V., Loreto L. D. & Sawatsky-Girling, B. (2010). A national interprofessional competency framework. Vancouver, BC: Canadian International Health Collaborative, 2010.

Pearce, C. L. (2004). The future of leadership: Combining vertical and shared leadership to transform knowledge work. The Academy of Management Executive, 18(1), 47-57.

Pearce, C. L., & Conger, J. A. (2003). Shared leadership: Reframing the hows and whys of leadership. Sage Publications.

Pearce, C. L., Hoch, J. E., Jeppesen, H. J., & Wegge, J. (2009). New forms of management: Shared and distributed leadership in organizations. European Journal of Psychological Assessment, 25(4), 285–286.

Pearce, C. L., & Manz, C. C. (2005). The New Silver Bullets of Leadership: Organizational Dynamics, 34(2), 130–140.

Pearce, C. L., & Sims, H. P., Jr. (2002). Vertical versus shared leadership as predictors of the effectiveness of change management teams: An examination of aversive, directive, transactional, transformational, and empowering leader behaviors. Group Dynamics: Theory, Research, and Practice, 6(2), 172–197.

Pethybridge, J. (2004). How team working influences discharge planning from hospital: a study of four multi-disciplinary teams in an acute hospital in England. Journal of Interprofessional Care, 18(1), 29–41.

Podsakoff, P. M., & MacKenzie, S. B., (1994). An examination of the psychometric properties and nomological validity of some revised and reduced substitutes for leadership scales. Journal of Applied Psychology, 79, 702-713.

Reeves, S., Lewin, S., Espin, S., & Zwarenstein, M. (2010). Interprofessional teamwork for health and social care. Oxford: Wiley-Blackwell.

Rogers, R. (2012). Leadership communication styles: a descriptive analysis of health care professionals. Journal of Healthcare Leadership, 4(1), 47-57.

Shamir B. (2007). Introduction: From passive recipients to active co-producers—The roles of followers in the leadership process. In: Shamir B, Pillai R, Bligh M, Uhl-Bien M., eds Follower-Centered Perspectives on Leadership: A Tribute to J. R. Meindl. Stamford, Conn: Information Age Publishing.

Sims, S., Hewitt, G., & Harris, R. (2015). Evidence of a shared purpose, critical reflection, innovation and leadership in interprofessional healthcare teams: a realist synthesis. Journal of Interprofessional Care, 29(3), 209–215.

Steinert, T., Goebel, R., & Rieger, W. (2006). A nurse–physician co‐leadership model in psychiatric hospitals: Results of a survey among leading staff members in three sites. International Journal of Mental Health Nursing, 15(4), 251-257.

Tan, K. T., Adzhahar, F. B. B., Lim, I., Chan, M., & Lim, W. S. (2014). Transactive memory system as a measure of collaborative practice in a geriatrics team: implications for continuing interprofessional education. Journal of interprofessional care, 28(3), 239-245.

Tuckman, B. W. (1965). Developmental sequence in small groups.Psychological bulletin, 63(6), 384-399.

Wegner, D.M. (1986). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G.R. Goethals (Eds.), Theories of group behaviour (pp. 185–208). New York, NY, USA: Springer-Verlag.

West, M., Eckert, R., Steward, K., & Pasmore, B. (2014). Developing collective leadership for healthcare. London: The King’s Fund.

Wetzel, A.P. (2012). Factor analysis methods and validity evidence: A review of instrument development across the medical education continuum. Academic Medicine: Journal of the Association of American Medical Colleges, 87, 1060–1069.

Wood, M. S. (2005). Determinants of shared leadership in management teams. International Journal of Leadership Studies, 1(1), 64–85.

Published online: 1 June, TAPS 2016, 1(1), 3-8
DOI: https://doi.org/10.29060/TAPS.2016-1-1/OA1014

Roger Strasser

Northern Ontario School of Medicine, Lakehead and Laurentian Universities, Canada

Abstract

Background: The Northern Ontario School of Medicine (NOSM) opened in 2005 with a social accountability mandate to contribute to improving the health of the people and communities of Northern Ontario. NOSM recruits students from Northern Ontario or similar backgrounds and provides Distributed Community Engaged Learning in over 90 clinical and community settings located in the region, a vast underserved rural part of Canada. This paper presents outcomes for graduates of NOSM’s undergraduate and postgraduate medical education programs with emphasis on NOSM’s social accountability mandate.

Methods: NOSM and the Centre for Rural and Northern Health Research (CRaNHR) used mixed methods that include administrative data from NOSM and external sources, as well as surveys and interviews of students, graduates and other informants.

Results: 92% of all NOSM students come from Northern Ontario with substantial inclusion of Aboriginal (7%) and Francophone (22%) students. 62% of NOSM graduates have chosen family practice (predominantly rural) training. 94% of the doctors who completed undergraduate and postgraduate education with NOSM are practising in Northern Ontario. The socio-economic impact of NOSM included: new economic activity, more than double the School’s budget; enhanced retention and recruitment for the universities and hospitals/health services; and a sense of empowerment among community participants attributable in large part to NOSM.

Discussion: There are signs that NOSM is successful in graduating doctors who have the skills and the commitment to practice in rural/remote communities and that NOSM is having a largely positive socio-economic impact on Northern Ontario.

Keywords: Social Accountability; Community Engagement; Distributed Medical Education

Practice Highlights

• 92% of all medical students are from Northern Ontario, including 7% Indigenous and 22% Francophone students.
• 62% of all NOSM medical graduates have chosen family practice (predominantly rural) training.
• 69% of the graduates of NOSM’s postgraduate education are practising in Northern Ontario.
• 94% of the doctors who completed undergraduate and postgraduate education with NOSM are practising in Northern Ontario.
• The socio-economic impact of NOSM includes: new economic activity, more than double the School’s budget; enhanced retention and recruitment for the universities and hospitals/health services; and a sense of empowerment among community participants attributable in large part to NOSM.

I. INTRODUCTION

In 1995, the World Health Organization (WHO) defined the “Social Accountability of Medical Schools” as “the obligation to direct their education, research, and service activities towards addressing the priority health concerns of the community, region and the nation that they have a mandate to serve” (Boelen et al 1995). This followed several decades in which innovative schools had focused on connecting with and responding to community needs (Strasser et al 2015). The Network: Towards Unity for Health began in 1979 at the instigation of the WHO with a group of 19 medical schools that implemented community oriented medical education (Schmidt et al 1991). In the 1980s, community-based education and service developed in Africa with the goal of “producing students with a sense of service and a strong inclination toward broad community care and preventive medicine” (Bollag et al 1982).

In 2001, Health Canada, Canada’s federal department of health, along with all of the Canadian medical schools, made a joint commitment to social accountability in the publication, Social Accountability: A Vision for Canadian Medical Schools (Health Canada 2001). When the Northern Ontario School of Medicine (NOSM) was incorporated in 2002, it became the first medical school in Canada established with an explicit social accountability mandate (Tesson et al 2009).

Northern Ontario is geographically vast (>800,000 sq km) with a volatile resource based economy and socio-economic characteristics that differ from the southern part of the province of Ontario. Forty percent of the population of Northern Ontario live in rural and remote areas where there are diverse communities and cultural groups, most notably Indigenous and Francophone peoples. These are minority groups in the rest of Ontario, but comprise a substantial proportion of the population in the North. The health status of people in the region is worse than the province as a whole (Rural and Northern Health Care Panel 2010), and there is a chronic shortage of doctors and other health professionals (Rural and Northern Health Care Panel 2010, Glazier et al 2011) which provided the impetus for the establishment of NOSM. NOSM serves as the Faculty of Medicine of Lakehead University in Thunder Bay (population 120,000) and of Laurentian University in Sudbury (population 160,000). These two universities are over 1,000 km apart and provide teaching, research and administrative bases for NOSM which views the entire geography of Northern Ontario as its campus (Strasser et al 2015).

Uniquely developed through a community consultative process, the holistic cohesive curriculum for the NOSM undergraduate program is: grounded in the Northern Ontario health context; organized around 5 themes (Strasser et al 2009) and relies heavily on electronic communications and interdependent community partnerships to support Distributed Community Engaged Learning (DCEL). In the classroom and in clinical settings, students are learning in context as if they are preparing to practise in Northern Ontario. Through Community Engagement, community members are active participants in various aspects of the School including: the admissions process; as standardized patients; ensuring that learners feel “at home” in their community; and in encouraging an understanding and knowledge of the social determinants of health at the local level. There is a strong emphasis on interprofessional education and integrated clinical learning which takes place in over 90 communities and many different health service settings, so that the students have personal experience of the diversity of the region’s communities and cultures (Strasser et al 2009, Strasser 2010, Strasser et al 2010, Strasser et al 2013).

Integrated Clinical Learning (ICL) involves team teaching and team learning in a variety of clinical and community settings. Patients and families are the central focus of this learning in which teachers may be medical, nursing or other health professionals or the patients/families themselves, and learners may be in a mix of health disciplines and at different levels of undergraduate and postgraduate education. ICL enriches the learning experience for all and enhances capacity in small communities while maintaining high quality patient care and preparing learners to be competent health care team members (Strasser et al 2013).

NOSM was the first medical school in the world in which all students undertake a longitudinal integrated clerkship, the Comprehensive Community Clerkship (CCC) (Couper et al 2011, Strasser et al 2011). Based in family practice, the CCC is the third year of the undergraduate program. Rather than a series of clerkship block rotations, students meet patients in family practice such that “the curriculum walks through the door”. Students follow these patients and their families, including when cared for by other specialists, so as to experience continuity of care in family practice. During the year, students achieve learning objectives which cover the same six core clinical disciplines as in the traditional clerkship blocks. Students live in one of 15 large rural or small urban communities in Northern Ontario, excluding the cities of Sudbury and Thunder Bay. This allows them to learn their core clinical medicine from the family practice, community perspective, while also gaining exposure to community based specialist care.

In addition to undergraduate medical education, NOSM offers postgraduate medical education (residency training) in family medicine/general practice and eight other major general specialties. Like the undergraduate program, these programs recruit residents from Northern Ontario or similar backgrounds and provide Distributed Community Engaged Learning with clinical education in a range of community and clinical settings in the region (NOSM Postgraduate Medical Education Overview 2016).

Ten years since the official opening of the School in 2005, this paper presents outcomes for graduates of NOSM’s undergraduate and postgraduate medical education programs with emphasis on NOSM’s social accountability mandate, including the socio-economic impact that NOSM has had on Northern Ontario.

II. METHODS

The Centre for Rural and Northern Health Research (CRaNHR) is a research centre of both Laurentian and Lakehead Universities and was established in 1992 (About CRaNHR 2016). NOSM and CRaNHR are collaborating in mixed methods studies that track NOSM undergraduate and postgraduate medical

learners, as well as assessing the socio-economic impact of NOSM. These studies use administrative data from NOSM and external sources, as well as surveys and interviews of students, graduates and other informants (Hogenbirk et al 2015b). Ethics approval for these studies was granted by the Research Ethics Boards of Laurentian and Lakehead Universities.

III. RESULTS

Consistent with its social accountability mandate, NOSM seeks to reflect the population distribution of Northern Ontario in each class. Between 2005 and 2013, NOSM received 18,000 applications for 538 places. The selection and admissions process has resulted in 92% of all medical students coming from Northern Ontario with the remaining 8% from other rural and remote parts of Canada, with a substantial inclusion of Indigenous (7%) and Francophone (22%) individuals. This has been achieved without sacrificing academic excellence; the mean grade point average (GPA) of the entry class each year has been 3.7 (out of 4) comparable with that of other Canadian medical schools.

Interviews of the NOSM medical students reveal generally positive experiences with a sense that they value the learning opportunities and feel they are being prepared well for practice in rural settings. Sample comments include: “clinical experiences during (third year) are more substantial than anything in traditional med school experience”; “NOSM creates “generalists” and encourages students to maintain a broad focus”; “rural medicine… that’s where you find the true generalists”; I like how much variety there can be in the doctor’s role”; “we’re better off … we will (learn) more skills in a rural centre”; and “you don’t know it until you live it”.

Between 2009 and 2015, there were 415 medical graduates of whom 257 (62%) chose family practice (predominantly rural) training which is approaching double the Canadian average. Almost all the other graduates (33%) have pursued training in other general specialties such as general internal medicine, general surgery and paediatrics, with a small number (5%) training in subspecialties like dermatology, plastic surgery and radiation oncology. Compared to other Canadian medical schools, NOSM graduates are very successful in matching to residency training, including competitive sub-specialities. Most years all NOSM students match in the first round of the Canadian Residency Matching Service (CaRMS). On occasions when one student has been unmatched, that individual was matched in the second round.

69% of NOSM residency program graduates are practising in Northern Ontario. At this time, they are predominantly family practitioners (FPs) with a small number of other specialists, including in public health, internal medicine and obstetrics/gynaecology. Ninety-four percent of doctors who completed undergraduate and postgraduate education with NOSM are practising in Northern Ontario, including 33% in remote rural communities.

A study of the socio-economic impact undertaken in 2009 found that NOSM makes a substantial contribution to the economy of Northern Ontario, with direct spending in Fiscal Year 2007-2008 (FY07/08) of $36.3 million (all values in Canadian dollars) and an additional $1 million per year spent by undergraduate medical students. Economic contributions were greatest in the university cities of Thunder Bay ($26.7 million) and Sudbury ($30.4 million), and $0.8–$1.2 million accrued to the next 3 largest population centres. All communities might realize an economic contribution of $7300–$103 900 per pair of medical learners per placement. Several of the 59 interviewees remarked that the dollar amount could be small to moderate but had broader economic implications.

In terms of social impact, interviewees reported that NOSM is a source of civic pride and an affirmation of the North’s potential as the region enlarges its knowledge-based economy. According to interviewees, NOSM has enriched the reputation of the universities and affiliated healthcare institutions, thereby enhancing the ability to recruit new doctors, researchers and scientists to the North. Interviewees anticipated that NOSM graduates will ultimately relieve the chronic physician shortage in Northern Ontario. Interviewees also remarked that Francophone and Indigenous students enrolled at NOSM and the School’s commitment to cultural competency training should help alleviate the shortage of doctors serving these population groups.

The most impressive social impact finding was a sense of community empowerment summed up in the phrase “if we can do a successful medical school in Northern Ontario, we can do anything”. The establishment of NOSM and its distributed programs offered opportunities for change and challenges to the status quo. Following the success of NOSM, Laurentian University has established an Architecture School in 2013 and Lakehead University opened a Law School in the same year.

More recently, a further study assessed the impact of NOSM on recruitment and retention of doctors in underserved communities in Northern Ontario with populations up to 11,000 people. The main research question was: “What are the changes (if any) in recruitment and retention expenditures and practices in Northern Ontario underserved communities that have recruited NOSM graduates?” Ten key informants were interviewed from eight communities that were successful in recruiting family doctors who were NOSM graduates. All key informants agreed that NOSM plays a prominent role in the doctor recruitment to underserved communities in Northern Ontario. Five out of eight Northern Ontario communities that previously struggled with a chronic doctor shortages have moved to a more stable situation with a full or almost full complement of FPs over the past five years. In these five communities, the shortage of FPs has decreased from 29 vacant full-time equivalent (FTE) positions to only one FTE vacancy. There is much less dependency on doctors hired on short term contracts (such as locum tenens). “Decreasing desperation” in recruiting doctors has led to a reduction in the amount of financial incentives offered. For example, one community reduced their spending from $200,000 to $50,000 for a four-year return of service agreement. All communities reduced their spending on travel and attendance at career/job fairs in southern Ontario.

IV. DISCUSSION

Ten years since accepting its first students, this paper has presented outcomes for graduates of NOSM’s medical education programs, as well as the impact of NOSM on Northern Ontario communities. Clearly, there are limitations in this research, particularly the relatively small numbers and the comparatively short timeframe. Tracking studies of NOSM graduates will continue and opportunities will be sought to investigate further the School’s economic, social and other impacts (Hogenbirk et al 2015b).

Consistent with its social accountability mandate, the NOSM selection and admissions processes aim to reflect the population distribution of Northern Ontario in each class (Tesson et al 2009). This target approach has been largely successful with 92% of the students coming from Northern Ontario and 40% from remote rural community backgrounds with 22% Francophone and 7% Indigenous. For Indigenous students the target is 10%, however Indigenous applicants make up only 2% of the applicant pool. The Indigenous people themselves recommended against quotas or reserved places because of potential stigma so Indigenous applicants compete for selection according to the same criteria as all other applicants. Support and encouragement is provided to Indigenous applicants in the form of guidance in preparing their applications and special training for the interviews. These interventions are having a positive effect with 7% of the medical students Indigenous drawn from 2% of the applicant pool. This recruitment success has occurred without lowering academic standards and has helped to ensure that NOSM students are well the challenges suited to and opportunities of Distributed Community Engaged Learning.

Students’ comments suggest that they are appreciating the value and making the most of the opportunities presented to them for learning their core clinical medicine in the rural community context. Particularly during the Comprehensive Community Clerkship, students become members of the health team and active contributors to healthcare. This enhances their clinical confidence and competence, and ensures that their clinical knowledge and skills are embedded in the local rural community setting (Hauer 2012, Dube et al 2015).

NOSM graduates have been consistently successful in matching to residency training when compared to other Canadian schools (CaRMS R1 Match Reports). There is a general sense that the residency program directors of Canada are keen to recruit NOSM graduates. Of particular note is the 5% of graduates who have matched to subspecialties. This matching success is important to Northern Ontario because there is a need for subspecialists even though they are required to undertake their postgraduate medical education elsewhere. Also, the 5% of graduates matching to subspecialties indicates that the novel NOSM curriculum model does not preclude students from success in matching to the most competitive subspecialty residency programs. These results clearly contradict the common perception of lower academic standards and lesser quality in rural or community-based schools.

The proportion of NOSM graduates practising in Northern Ontario is greater than might be predicted from experience elsewhere in Canada. For example: 76% of FPs and other specialists enrolled as medical students during 1998 – 2009 were practising in Manitoba after undergraduate and postgraduate education at the University of Manitoba (Raghavan et al 2012); and 49% of Memorial University of Newfoundland’s undergraduate and postgraduate trained FPs and other specialists who graduated as medical students during 1973 – 2008 were practising in Newfoundland with 16% in rural Canada in 2014 (Mathews et al 2015)

The finding that 94% of doctors who completed undergraduate and postgraduate education with NOSM are practising in Northern Ontario is seen as a strong indication that the NOSM model is showing signs of success in addressing the long-standing workforce shortages (Strasser et al 2013, Hogenbirk et al 2016). This perception is confirmed by interviews of key informants in previously underserved small communities which have been successful in recruiting NOSM graduates. These communities have moved from perpetual crisis mode to planning recruitment efforts and have reduced their expenditures on incentive payments and other recruitment activities.

Many of the NOSM graduates practising in Northern Ontario have become faculty members and some have taken on substantial academic leadership roles. This is seen as a positive sign for the future sustainability with NOSM graduates teaching NOSM learners and pursuing academic careers in Northern Ontario.

In addition, NOSM is making a substantial economic contribution to the communities of Northern Ontario not only directly in terms of new economic activity but also indirectly through economic opportunities which are incidental to specific NOSM activities (Hogenbirk et al 2015a). These findings show that when considering the cost of medical education, it is important to look at the whole picture and not just the level of government expenditure per learner. For Northern Ontario, the high level of Ontario government contribution to NOSM is justified by the substantial return on investment for participating communities.

Although defined by WHO 1995, there are many different perceptions and perspectives on what social accountability really is and how to measure its effectiveness. Preston et al (2016) in a study which involved interviewing staff, students and community members of four socially accountable medical schools in two countries found three core elements: “Overall there were three common understandings. Socially accountable medical education was about meeting workforce, community and health needs. Social accountability was also determined by the nature and content of programs the school implemented or how it operated. Finally, social accountability was deemed a personal responsibility. The broad consensus masked the divergent perspectives people held within each school” (Preston et al 2016).

NOSM is a founding member of the Training for Health Equity network (THEnet), a group of health profession schools worldwide which are guided by a social accountability mandate. THEnet members share a core mission to recruit students from, and produce health professionals for under- served communities. Although these schools operate in different contexts and employ somewhat different strategies, they share the following core principles: 1. Health and social needs of targeted communities guide education, research and service programs; 2. Students are recruited from the communities with the greatest health care needs; 3. Programs are located within or in close proximity to the communities they serve; 4. Much of the learning takes place in the community instead of predominantly in university and teaching hospital settings; 5. The curriculum integrates basic and clinical sciences with population health and social sciences; and early clinical contact increases the relevance and value of theoretical learning; 6. Pedagogical methodologies are student, patient and population centred, service-based and assisted by information communication technology; 7. Community-based practitioners are recruited and trained as teachers and mentors; 8. Partnering with the health system to produce locally relevant competencies; 9. Faculty and programs emphasize and model commitment to public service (Palsdottir et al 2008).

THEnet developed, piloted and published an Evaluation Framework for Socially Accountable Health Professional Education (The Training for Health Equity Network 2011, Larkins et al 2013) which provided the core content for the Global Consensus on Socially Accountable Education in 2010 (Global Consensus for Social Accountability of Medical Schools 2010). Subsequently, THEnet has been successful in researching, reporting and advocating for socially accountable education (Strasser et al 2010, Larkins et al 2013, Ross et al 2014, Larkins et al 2015). Following the GCSA, the Association for Medical Education in Europe (AMEE) has adopted social accountability as one of the elements of the ASPIRE: International Recognition of Excellence in Medical Education program (www.amee.org). NOSM was an inaugural recipients of the AMEE Aspire Award of Excellence in Socially Accountable Medical Education.

After 10 years, there are signs that the NOSM model of socially accountable education is providing benefits to Northern Ontario which go beyond simply producing more doctors. NOSM graduates are broadly skilled generalists who are members of the whole health team and are sensitive to the social, cultural, linguistic and geographic diversity of the region. In addition, the majority of these doctors are choosing to provide care in previously underserved communities. There are signs also that NOSM is contributing to other academic developments in Northern Ontario and having a positive broader socio-economic impact on the region. These are all encouraging signs that, guided by social accountability, NOSM is moving in the right direction towards improving the health of the people and communities of Northern Ontario.

Notes on Contributor

Professor Strasser is Dean and CEO, Northern Ontario School of Medicine, Canada since 2002. Previously, he was Professor of Rural Health and Head, Monash University School of Rural Health, Australia. His international roles included Chair of the Working Party on Rural Practice of the World Organization of Family Doctors (WONCA).

Acknowledgements

The author acknowledges the research teams who have contributed to the studies reported in this paper: Raymond W Pong, John C. Hogenbirk, Wayne Warry, Bruce Minore, Mary Ellen Hill, David R Robinson, Patrick Timony, Margaret G French, Oxana Mian, David C. Marsh, Sue Berry, William G. McCready, Lisa Graves, Catherine Cervin, Ken Adams and Joe Lipinski.

Declaration of Interest

The author is an employee of Northern Ontario School of Medicine and has no financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest. The research reported in this article was funded by the Ontario Ministry of Health and Long Term Care.

References

About CRaNHR at: www.cranhr.ca

Boelen C, Heck JE. (1995) Defining and Measuring the Social Accountability of Medical Schools. Geneva, Switzerland: World Health Organization.

Bollag U, Schmidt H, Fryers T, Lawani J. (1982) Medical education in action: Community based experience and service in Nigeria. Med Educ. 16:282–289.

CaRMS, Canadian Resident Matching Service R1 Match Reports at: www.carms.ca/en/data-and-reports/r-1/

Couper I, Worley P, Strasser R. (2011). Rural longitudinal integrated clerkships: lessons from two programs on different continents. Rural and Remote Health 11: 1665. (Online) Available from: www.rrh.org.au

Dube TV, Schinke RJ, Strasser R, Couper I, Lightfoot NE. (2015). Transition processes through a longitudinal integrated clerkship: a qualitative study of medical students’ experiences. Med Educ. 49: 1028–1037

Glazier RH, Gozdyra P, Yeritsyan N. (2011) Geographic Access to Primary Care and Hospital Services for Rural and Northern Communities: Report to the Ontario Ministry of Health and Long-Term Care. Institute for Clinical Evaluative Sciences (ICES). Toronto.

Global Consensus for Social Accountability of Medical Schools. Available at: www.healthsocialaccountability.org

Hauer KE, Hirsh D, Ma I, Hansen L, Ogur B, Poncelet AN, Alexander EK, O’Brien BC. (2012). The role of role: learning in longitudinal integrated and traditional block clerkships. Medical Education. 46: 698–710

Health Canada. (2001) Social Accountability: A Vision for Canadian Medical Schools. Ottawa: Health Canada.

Hogenbirk Robinson JR, Hill ME, Minore B, Adams K, Strasser RP, Lipinski J. (2015a). The economic contribution of the Northern Ontario School of Medicine to communities participating in distributed medical education. Can J Rural Med. 20(1)

Hogenbirk JC, French MG, Timony PE, Strasser RP, Hunt D, Pong RW. (2015b). Outcomes of the Northern Ontario School of Medicine’s distributed medical education programmes: protocol for a longitudinal comparative multicohort study. BMJ Open. 5:7 e008246 doi: 10.1136/bmjopen-2015-008246

Hogenbirk JC, Timony P. French MG, Strasser R, Pong RW, Cervin C, Graves L, (2016) Milestones on the Social Accountability Journey: family medicine practice locations of Northern Ontario School of Medicine graduates. Canadian Family Physician. 62: e138-e145.

Larkins SL, Preston R, Matte MC, et al. Measuring social accountability in health professional education: Development and international pilot testing of an evaluation framework. Medical Teacher 2013; 35:32-45

Larkins S, MichielsenK, Iputo J, Elsanousi S. Mammen M, Graves L, Willems S, Cristobal FL, Samson R, Ellaway R, Ross S, Johnston K,

Derese A, Neusy A-J.. (2015) Impact of selection strategies on representation of underserved populations and intention to practise: international findings. Med Educ. 49: 60–72

Mathews M, Ryan D, Samarasena A. (2015) Work locations in 2014 of medical graduates of Memorial University of Newfoundland: a cross- sectional study. Canadian Medical Association Journal Open. 3(2):E217-E222.

NOSM Postgraduate Medical Education Overview at: www.nosm.ca/postgrad

Palsdottir B, Neusy A-J, Reed G. (2008). Building the Evidence Base: Networking Innovative Socially Accountable Medical Education Programs. Education for Health. 21:2. Available from: www.educationforhealth.net

Preston R, Larkins S, Taylor J, Judd J. (2016). From personal to global: Understandings of social accountability from stakeholders at four medical schools. Medical Teacher. Early Online. DOI: 10.3109/0142159X.2015.1114596

Raghavan M, Fleisher W Downs A, Martin B, Sandham JD. (2012) Determinants of first practice location among Manitoba medical graduates. Can Fam Physician. 58(11):e667-76.

Ross S, Preston R, Lindemann I, Matte M, Samson R, Tandinco F, Larkins S, Palsdottir B, Neusy AJ. (2014). The training for health equity network evaluation framework: A pilot study at five health professional schools. Educ Health. 27:116–126.

Rural and Northern Health Care Panel. (2010) Rural and Northern Health Care Framework/Plan, Stage 1 Report. Final Report. Toronto: Ministry of Health and Long-Term Care. Available at: http://www.health.gov.on.ca/en/public/programs/ruralnorthern/docs/report_rural_northern_EN.pdf.

Schmidt HG, Neufeld VR, Nooman ZM, Ogunbode T. (1991) Network of community oriented educational institutions for the health sciences. Acad Med. 66:259–263.

Strasser R., Lanphear J., McCready W., Topps M., Hunt D., Matte M.. (2009) Canada’s new medical school: the Northern Ontario School of Medicine – social accountability through distributed community engaged learning. Academic Medicine. 84: 1459-1456

Strasser, R. (2010) Community engagement: a key to successful rural clinical education. Rural and Remote Health 10: 1543. (Online). Available from: www.rrh.org.au

Strasser R, Neusy, A-J. (2010) Context Counts: Training Health Workers in and for Rural Areas. Bull World Health Organ. 88: 777 – 782

Strasser R, Hirsh D. (2011). Longitudinal integrated clerkships: transforming medical education worldwide? Medical Education. 45: 436–437.

Strasser R. Hogenbirk JC, Minore B, Marsh DC, Berry S, McCready WG, Graves L. (2013) Transforming health professional education through social accountability: Canada’s Northern Ontario School of Medicine. Medical Teacher. 35: 490-496

Strasser R, Worley P, Cristobal F, Marsh DC, Berry S, Strasser S, Ellaway R. (2015) Putting Communities in the Driver’s Seat: The Realities of Community Engaged Medical Education. Academic Medicine. 90:1466–1470.

Tesson G, Hudson G, Strasser R, Hunt D. (2009) The Making of the Northern Ontario School of Medicine: A Case Study in Medical Education. McGill Queens University Press. Montreal.

The Training for Health Equity Network.(2011) THEnet’s Social Accountability Evaluation Framework Version 1. Monograph I (1 ed.). The Training for Health Equity Network. Available at: www.thenetcommunity.org

Published online: 3 January, TAPS 2017, 2(1), 7-15
DOI: https://doi.org/10.29060/TAPS.2017-2-1/OA1019

Marcus A Henning¹, Christian U Krägeloh², Roger Booth³, Erin M Hill⁴, Julie Chen⁵, Craig S Webster⁶

¹Centre for Medical and Health Sciences Education, University of Auckland, New Zealand; ²Department of Psychology, Auckland University of Technology, New Zealand; ³Faculty of Medical and Health Sciences, University of Auckland, New Zealand; ⁴Department of Psychology, West Chester University, United States of America; ⁵Department of Family Medicine and Primary Care and Institute of Medical and Health Sciences Education, The University of Hong Kong, Hong Kong; ⁶Centre for Medical and Health Sciences Education and Department of Anaesthesiology, University of Auckland, New Zealand

Abstract

At this university, a Biomedical Common Year 1 occurs prior to admission to the medical programme. Students achieving a minimum GPA of 6.0 are eligible for consideration for an admissions interview. The aim of this research was to assess the psycho-educational factors that underpin students’ intention to study medicine. The research question driving the research was, ‘If students have an interest in becoming a future doctor in their premedical course, does this relate to their levels of motivation, competitiveness, perceived stress, health-related quality of life (HRQOL), and grade attainment?’ A total of 339 students (response rate = 25%) who completed a biosciences assessment filled in a survey that asked them to disclose their grade and to respond to a series of questionnaires, namely the Motivated Strategies for Learning Questionnaire, World Health Organisation Quality Of Life questionnaire – New Zealand Version, Perceived Stress Scale, and Revised Competitiveness Index. The findings from the binary logistic regression indicated that several variables predicted students’ career intentions: grade achievement, Perceived Stress, Physical HRQOL, and Environmental HRQOL. Perceived Stress and Physical HRQOL were found to be influential variables that interacted with other variables reducing variability in the model and increasing its predictability. Students with an intention to become a doctor tend to attain higher grades and have better environmental HRQOL scores. Nonetheless, variable interactions suggested that those students with high levels of physical HRQOL and low levels of perceived stress have higher levels of enjoyment regarding competition, self-efficacy, and intrinsic value.

Keywords:       Biomedical and Health Science Students, New Zealand, Career Intention, Motivation, Physical Wellbeing, Competitiveness, Motivation, Academic Achievement

Practice Highlights

1. The impact of students’ intentions to enrol in either medicine or other health science courses were considered with respect to differences in grade achievement, levels of competitiveness, perceived stress, quality of life and motivational beliefs.

2. The findings indicated that those students intending to study medicine: achieved higher grades, indicated higher levels of enjoyment with competition, specified greater environmental health-related quality of life, and suggested higher levels of motivation.

3. Measures of perceived stress and physical health related quality of life interacted with levels of competiveness and motivational beliefs indicating that students with higher physical health related quality of life and lower stress were more competitive and more highly motivated.

4. The findings have implications for selection of students and in inculcating methods of learning that engender the culture of working with heterogeneity, openness and constructive problem solving.

I. INTRODUCTION

The focus of this study was to determine learning and psychosocial variables that are enhanced or diminished in reference to students intending to enrol in medicine versus other health-oriented professions. There have been some studies that have compared medical students with other health professional students in terms of learning strategies and academic achievement. For example, Salamonson, Everett, Koch, Wilson, and Davidson (2009) noted that medical students obtained higher scores than nursing students in peer learning, help seeking, critical thinking, grade point average, and management of time and study environment. However, we found no studies that compared students in their premedical year with respect to their intention to study in medicine versus other health-oriented programmes. In the present study, we sought to increase our awareness of students’ behaviours and their needs in regard to those intending to study medicine versus those who chose other career paths. To explore this potential difference, we chose several operational variables that could define and inform the learning characteristics of aspiring medical students in contrast to those seeking to be in other health-oriented professions.

Our first area of investigation was to determine the impact of grade accomplishment. Grade accomplishment is an important requirement for students to enter into any medical programme (Al Alwan, Al Kushi, Tamim, Magzoub, & Elzubeir, 2013; McManus et al., 2005; Shulruf, Poole, Wang, Rudland, & Wilkinson, 2012; The University of Auckland, 2015b). At the University of Auckland, New Zealand, the requirements for entry into medicine via the Biomedical Common Year 1 programme are defined as: “Following the successful completion of the eight prescribed first year courses, applicants achieving a minimum GPA of 6.0 (B+ average, on a scale of 0.00 to 9.00, where D is 0.00 and A+ is 9.00) are eligible for consideration for an admissions interview” (The University of Auckland, 2015b). The rationale and empirical evidence with regard to academic attainment grades is clear; high achievement in pre-medical courses predicts later achievement in medical courses (Shulruf et al., 2012) and, in the U.K., pre-medical academic achievement scores (A levels) are good indicators of cognitive ability (McManus et al., 2005). Moreover, students, in achieving high A level scores, tend to exhibit “appropriate motivation, commitment, personality, and attitudes, traits that are also beneficial at medical school and for lifelong learning” (McManus et al., 2005, p. 556).

An additional consideration of the present study was whether levels of competiveness could predict career intention. Students who are strategically motivated often recognize that to be successful is to be competitive (Hilliard, 1995) and this is especially true for students focusing on certain disciplines within medicine such as surgery (Glynn & Kerin, 2010). Competiveness is a systems and personality variable and is present when students consider their course of study or later medical specialities (Fincher, Lewis, & Rogers, 1992). Interestingly, and with respect to medical learning, there is evidence suggesting that competiveness declines over time of study and in later years students tend to be more motivated by experiential learning and the need to be a competent doctor (Hilliard, 1995).

Furthermore, there is considerable evidence to suggest that wellbeing and health-related quality of life (HRQOL) are major issues to consider in the medical and health sciences disciplines and this may also impact their career intention (Chang, Eddins-Folensbee, & Coverdale, 2012; Dyrbye et al., 2006; Dyrbye, Thomas, & Shanafelt, 2006; Henning, Krägeloh, Hawken, Zhao, & Doherty, 2010; Paro et al., 2010; Tempski et al., 2012). There are clear indicators as to why students studying medicine may have compromised experiences of HRQOL and some of these may include experiencing excessive competition, sleep deprivation, perceptions of poor teaching, excessive workload, and unmanageable educational schedules (Henning, Hawken, & Hill, 2009; Henning et al., 2010; Tempski et al., 2012).

Lastly, we aimed to determine the impact of motivational beliefs. In our previous work we found that two motivational beliefs were able to discriminate between sub-groups within the medical student cohort, namely test anxiety and self-efficacy (Henning et al., 2011; Henning et al., 2013). Other studies have also found that factors – such as low task value, self-efficacy, enjoyment, boredom and anxiety – defining the concept of motivational beliefs were able to predict academic success in medical students (Artino, La Rochelle, & Durning, 2010; Artino Jr, Hemmer, & Durning, 2011). Non-academic motivations that include altruism and intellectual challenge have also been studied in reference to gender differences and choosing a career in medicine, although no significant difference were found between the two groups with regards these motivational factors (Todisco, Hayes, & Farnill, 1995).

The aim of the present paper was to explore the relationships between these operational variables and to discern whether students aiming to study medicine differed from students not targeting medicine in terms of their learning and psychosocial profile. The research question driving the research was, ‘If students have a definite interest in becoming a future doctor in their premedical course, does this relate to their levels of motivation, competitiveness, perceived stress, HRQOL and grade attainment?’

II. METHODS

A. Participants

Invitations were sent via email to all students (N=1369) enrolled in the bioscience year 1 course of the Biomedical Common Year or Overlapping Year 1 programme at this university (The University of Auckland, 2015a).

B. Procedure

Ethics approval for the study was obtained from the University of Auckland Human Participants Ethics Committee. Students were emailed directly, with two follow-up reminders, about the study and invited to participate in a Google survey. This survey included items pertaining to demographic information and the instruments cited below (Google.com, 2015). The study data collection was anonymous and conducted over a one month period.

C. Measures

Demographic and background details: Several measures pertinent to this paper were obtained: (1) age in years; (2) gender; (3) students were asked whether they would like to pursue a career in medicine (yes/no); and (4) the self-disclosed measures of academic attainment in their mid-year bioscience assessment which were converted to numerical values (0=fail,1=C-,2=C,3=C+,4=B-,5=B,6=B+,7=A-,8=A,9=A+).

Motivated Strategies for Learning Questionnaire: Three of the five sub-scales were used in this study, namely self-efficacy, intrinsic value, and test anxiety (Pintrich & De Groot, 1990).

World Health Organisation Quality Of Life questionnaire (New Zealand Version): This version has 31 items and was developed for measuring HRQOL amongst New Zealanders (Feng, Krägeloh, Billington, & Hsu, 2011; Krägeloh et al., 2013). As with the original WHOQOL-BREF (WHOQoL Group, 1998), which has 26-items, the new version is comprised of four domains: physical HRQOL, psychological HRQOL, social relationships, and environmental HRQOL.

Perceived Stress Scale: This scale is unidimensional with 10 items and measures the level of perceived stress in relation to respondents’ life experiences in certain situations within the last month (Cole, 1999).

Revised Competitiveness Index: This instrument contains 14 items and measures enjoyment of competition and levels of contentiousness or avoidance and dislike of conflict situations (Houston, Harris, McIntire, & Francis, 2002).

All items in the above questionnaires required participants to respond to a 5-point Likert scale.

 D. Data analysis

First, statistics were presented that described the response rate (n/N), and the participants’ demographic and background details. Second, the internal consistency measures for each domain within the measurements used were checked using Cronbach alpha coefficients. In the main analyses and to explore differences between those students who aim to be doctors versus those who do not, a binary logistic regression was conducted with four-blocks: demographic variables and bioscience grade at Block 1, competiveness measures at Block 2, stress and quality-of-life measures at Block 3, and motivation measures at Block 4. Finally, as a post hoc process, the contribution of each independent variable was appraised. Descriptive statistics and t-test mean comparisons were computed for all dependent variables and correlations were conducted on targeted variables.

III. RESULTS

A. Response rate and participant data

A total of 339 (response rate = 25%) students in Biomedical Common Year 1 programme were eligible for this aspect of the research. The average age for all participants was 18.94 years (SD=2.80). More female (n=253, 75%) students responded to the survey than their male (n=86, 25%) peers. The student population statistics generated a higher mean age of students studying this course of study at 19.60 (SD = 2.70) and lower female proportional representation at 65%. Grade attainment was well spread in both the sample and population statistics, with marginally higher proportion of high achieving students responding in the study than those in the main population. These sampling differences, albeit small, may be a source of response bias.

In our sample, 54% (n=183) of participants indicated a preference for medicine. If this sample is representative of the Biomedical Common Year 1 programme population we could extrapolate that 739 out of 1369 students had an interest in study medicine.

B. Questionnaires – internal consistency measures

All measures used in this study were found to be internally consistent, with Cronbach alpha coefficient measures above the recommended cut-off of .70 (Field, 2005). The coefficient measures for Motivational Beliefs variables included intrinsic value (α =.83), self-efficacy (α =.90), and test anxiety (α =.84). The Cronbach alpha scores for the domains of the WHOQOL-BREF, Perceived Stress Scale and Revised Competitiveness Index were physical HRQOL (α =.77), psychological HRQOL (α =.85), social relationships (α =.70), and environmental HRQOL (α =.78). And for the remaining measures: the PSS (α =.86), enjoyment of competition (α =.91) and contentiousness (α =.83).

C. Binary Logistic modelling

First, potential problems with multicollinearity were checked and, overall, the statistics revealed no major concerns with the Variance Inflation Factor (VIF) values for all but one of the independent variables. In general, values were close to 1.00 and Tolerance values were found to be above .20 (Field, 2013). However, the VIF for psychological health-related quality of life was 3.05 and highly correlated with perceived stress (r=-.70, p<.001), hence we removed psychological health-related quality of life from the main model.

As shown in Table 1, at Block 1, only grade predicted student career intention. At Block 2, enjoyment of competition predicted future career intention. At Block 3, perceived stress, physical HRQOL and environmental HRQOL measures significantly added to the prediction career intention. Lastly in Block 4, when the motivation measures were added to the model, no motivational scores predicted career intention.

D. Means comparisons and correlations with targeted variables (post hoc analyses)

To investigate differences further we computed mean comparison statistics for the measures used to predict the dependent variable, career intention. The purpose of Table 2 was to appraise direction of difference and to identify confounders that may be influential in the model. Table 2 includes mean and standard deviation comparisons between the two cohorts under investigation. Table 2 also provides information on confidence intervals and inferential statistics to ascertain the level and significance of difference. According to the findings shown in Table 2, there are several variables that showed significant differences (p<.004, with a Bonferroni adjustment put in place, Field, 2013), and these included grade attainment, enjoyment of competition, environmental HRQOL and motivational beliefs (self-efficacy and intrinsic value). The findings, in Table 2, indicate that students intending to study medicine: obtained higher grades scores, enjoyed competition more, had higher environmental HRQOL, were more self-efficacious and intrinsically valued their studies more.

To investigate the interaction effect of perceived stress and Physical HRQOL as potential influential variables, a series of correlations were conducted with the ‘significant’ variables that emerged from the mean comparisons (Table 2). We noted (Table 3) that Physical HRQOL was positively correlated with all targeted variables, and Perceived Stress was negatively correlated with all variables (p<.001).

Table 2. Descriptive statistics for career intention and the measures used in the binary logistic regression analysis with t-test comparisons for each measure (except gender)

IV. DISCUSSION

The present study surveyed students in their first year of study at an Auckland university. These students were studying in the Biomedical Common Year or Overlapping Year 1 where students have differing career paths in addition to choosing a medical option (The University of Auckland, 2015a). To apply for medicine, a competitive grade average needs to be attained (The University of Auckland, 2015b). In this study, we compared students intending to study medicine versus other health-oriented careers with respect to 12 variables that included grade attainment, demographic measures, level of competition, perceived stress, HRQOL, and motivational beliefs. These variables were included because of their salience in the literature. The results of the binary logistic regression indicated the 12 variables in Block 4 explained 40% of the variance, although only four of the individual variables were found to be significant predictors of career intention, namely grade achievement, perceived stress, physical health-related quality of life, and environmental HRQOL. Also of note was that enjoyment of competition was a significant predictor at Block 3 and indicated significant differences at the individual comparison level between the two groups.

Our first approach incorporated the use of a binary logistic regression to ascertain which of the dependent variables could explain the differences in those students wanting to be a doctor in contrast to those students not intending to study medicine. We then checked the magnitude of difference using a set of independent sample t-tests. The second analyses revealed that two variables, perceived stress and physical HRQOL, yielded non-significant differences and hence could be influential or ‘confounding’ variables. Finally, we computed correlational analyses for perceived stress and physical HRQOL versus those variables that were found to be significantly different on the t-test comparisons. The purpose of this approach was to tease out any interactions produced by perceived stress and physical HRQOL. The outcome being that perceived stress and physical HRQOL were likely moderating variables impacting on, and potentially masking the ‘true’ effect, of environmental HRQOL, self-efficacy, intrinsic value, and grade achievement. These subtle influences will be discussed in more depth in due course.

Grade achievement is clearly a strong variable for determining differences between those intending to be a medical student versus those intending to study other health-oriented programmes. This result was expected given the need to achieve high grades to be considered for admission at the University of Auckland (The University of Auckland, 2015b) and elsewhere (Al Alwan et al., 2013; McManus et al., 2005; Shulruf et al., 2012). The rationale for selecting the grade average criteria is that it will likely predict success of students later in their medical course and career as a practicing doctor (McManus et al., 2005; Shulruf et al., 2012). One area that may extend this argument is that of the academic self-concept (Marsh & Seaton, 2013), which suggests that students have awareness of their academic level and this awareness can assist them in positioning themselves for entry into highly competitive courses like medicine.

Enjoyment of competition is an interesting predictor variable as it has several implications. In their development of the instrument, Houston, Harris, McIntire, and Francis (2002) described enjoyment of competition in terms of students who liked competition, who felt they were competitive by nature, enjoyed competing against others, got satisfaction from competition, and aimed to outperform others. With this in mind, and in considering the Block 3 and individual results, we found that those intending to be medical students scored higher on these attributes that their non-medical peers. Competitiveness could be viewed as a personality trait or a strategy needed to be successful in choosing a career in medicine in order to gain an edge on other students who may have the same goal (Fincher et al., 1992; Hilliard, 1995). In addition, 54% of our sample had a desire to study medicine which may suggest that over 700 students in the population in the Biomedical Common Year 1 may have an interest in studying medicine. We have no idea if this extrapolation is verifiable; although the allocated places of 270 students (Health Workforce New Zealand, 2011) is likely well below those students aiming to study medicine, thus increasing the need to be highly competitive to enter in the medical programme.

The issue for educationalists is how to transform this level of competitiveness to one of collaboration upon transition to medicine (Reagan & Blakemore, 2009). A potential teaching and learning strategy would be to promote group work that engenders the culture of working with heterogeneity, openness and constructive problem solving (Staples, 2008). The question arises as to whether this level of competitiveness is maintained in further years of study after the students are selected into medicine. There is some literature to suggest that this competitive spirit is required later on in the study of medicine, for example when students aim for surgery (Glynn & Kerin, 2010). In addition to the need to be competitive, Glynn and Kerin found that students focussed on surgery because of influences from role models and the need for prestige. Lifestyle and stress were also determinants that predicted students not wanting to pursue surgery. This suggests that stress and lifestyle (which could be aspects of quality of life) impact students’ perceptions of a potentially competitive discipline environment. It is important to note though, that there is evidence to suggest that competiveness declines over time in the medical learning environment and that intrinsic forms of motivation may become more important, i.e., the need to be a good doctor (Hilliard, 1995).

In relation to the Block 4 model, the variables perceived stress, physical HRQOL and environmental HRQOL were able to significantly contribute to the model. At an individual level (Table 2) only environmental HRQOL was found to be significantly different, thus indicating that perceived stress and physical HRQOL are confounding influences that indirectly affect other variables in their prediction of career intention. This was further verified by the correlational analysis (Table 3). Nonetheless, environmental HRQOL was a significant variable in both the model and in the individual analyses. Environmental HRQOL measures feelings of safety, access to health and transportation resources, satisfaction with physical environment and housing, financial and informational resources, and opportunity for leisure activities (WHOQoL Group, 1998). Skevington, Lotfy, and O’Connell (2004) suggest that the concept of environment, as a HRQOL construct, is likely related to levels of satisfaction about accessing the resources within one’s environment that can promote wellbeing. Hence, ample access to these resources may be highly correlated with affluence, suggesting that students intending to study medicine may have greater resource access than students not intending to study medicine. This assumption is consistent with studies showing that the cost of studying medicine is high compared to other courses and many medical students do tend to come from more affluent backgrounds (Collins, Jones, & White, 1993; Ng, Tambyah, & Wong, 2009). It is acknowledged though that there is some qualitative evidence to suggest that year 4 and 5 medical students express similar concerns to other students about their environmental HRQOL, such as worries about the use and access of adequate public transport ( Henning, 2010).

The non-significant variables in the Block 4 model included aspects of competition (enjoyment and contentiousness), social relationships, and motivational beliefs (self-efficacy, intrinsic value, and test anxiety). The results shown in Table 2 suggest that perception of enjoyment of competition and the motivational beliefs variables significantly differ between those students intending to enrol in medicine versus those intending to enrol in other health-oriented courses. The differences between the model and the individual statistics suggest that perceived stress and physical HRQOL are moderating the effects of enjoyment of competition and the motivational beliefs. It may appear that students intending to study medicine are more self-efficacious, see their studies as more intrinsically valuable, enjoy competition more and are potentially less test-anxious.   Nonetheless, our findings appear to show that this is only true for certain conditions or combinations of perceived stress and physical HRQOL (likely low stress and high physical HRQOL).

The major limitation of this study is the response rate of 25%, although this is considered reasonable for an online survey (Nulty, 2008; Sax, Gilmartin, & Bryant, 2003). It is further noted that more female students responded to the survey than male students, and this was expected given that more women study medicine in New Zealand than men and this could be as high as a 65 to 35 percent split suggesting a feminisation of medicine in New Zealand (Fitzjohn, Wilkinson, Gill, & Mulder, 2003; Poole, Moriarty, Wearn, Wilkinson, & Weller, 2009). Nonetheless, it is acknowledged that there is a likely response bias in the results and the study gathered information regarding their intention to study medicine and their self-disclosed grades rather than departmentally recorded grades. However, given the anonymous nature of the study it is unlikely students would intentionally mislead the researchers. Further studies may wish to unpack the influence of perceived stress and physical HRQOL on students’ enjoyment of competition and motivational beliefs Further studies could also consider how students’ academic performance (e.g., bioscience grade) scores affect measures of motivation and HRQOL, which could enhance the interpretation and robustness of the results and predictions on future performance in medicine. Other variables worthy of consideration include evaluating the impact of critical thinking skills, levels of certainty, social or emotional intelligence, and interpersonal skills.

Notes on Contributors

Marcus A. Henning is a Senior Lecturer in the Centre for Medical and Health Sciences Education at the University of Auckland, New Zealand.

Christian U. Krägeloh is an Associate Professor in the Department of Psychology at AUT University, New Zealand.

Roger Booth is an Associate professor in molecular medicine and pathology, in the Faculty of Medical and Health Sciences at the University of Auckland, New Zealand.

Erin M. Hill is an Assistant Professor in the Department of Psychology at West Chester University, United States of America.

Dr Julie Chen is a Family Physician by training, who is engaged in the development, implementation and evaluation of new initiatives in the medical curriculum to encourage early learning of professionalism and humanism in medicine.

Craig Webster is a Senior Lecturer with the Centre for Medical and Health Sciences Education at the University of Auckland, and has degrees in psychology and a PhD in medical human factors.

Acknowledgements

The authors wish to express sincere appreciation to Associate Professor Papaarangi Reid (Tumuaki and Head of Te Kupenga Hauora Māori ), Alistair William Stewart (Faculty Biostatistician), and the Biomedical Common Year 1 students for their valuable input and support.

Declaration of Interest

This is an unfunded study. All authors have no potential conflicts of interest.

References

Al Alwan, I., Al Kushi, M., Tamim, H., Magzoub, M., & Elzubeir, M. (2013). Health sciences and medical college preadmission criteria and prediction of in-course academic performance: a longitudinal cohort study. Advances in Health Sciences Education, 18(3), 427-438.

Artino, A. R., La Rochelle, J. S., & Durning, S. J. (2010). Second‐year medical students’ motivational beliefs, emotions, and achievement. Medical Education, 44(12), 1203-1212.

Artino Jr, A. R., Hemmer, P. A., & Durning, S. J. (2011). Using self-regulated learning theory to understand the beliefs, emotions, and behaviors of struggling medical students. Academic Medicine, 86(10), S35-S38.

Chang, E., Eddins-Folensbee, F., & Coverdale, J. (2012). Survey of the prevalence of burnout, stress, depression, and the use of supports by medical students at one school. Academic Psychiatry, 36(3), 177-182.

Cole, S. R. (1999). Assessment of differential item functioning in the Perceived Stress Scale-10. Journal of Epidemiology and Community Health, 53(5), 319.

Collins, J., Jones, J., & White, G. (1993). Demographic variables in Auckland medical students. The New Zealand Medical Journal, 106(960), 306-308.

Dyrbye, L. N., Thomas, M. R., Huntington, J. L., Lawson, K. L., Novotny, P. J., Sloan, J. A., & Shanafelt, T. D. (2006). Personal life events and medical student burnout: A multicenter study. Academic Medicine, 81, 374-384.

Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Academic Medicine, 81(4), 354-373.

Feng, J., Krägeloh, C., Billington, R., & Hsu, P. (2011). Selection of national items for the New Zealand World Health Organisation Quality of life Questionnaire: Preliminary analyses. In R. Scherman, & C. Krägeloh (Eds.), Walking the talk (pp. 87-95). Auckland, New Zealand: Auckland University of Technology.

Field, A. P. (2005). Discovering statistics using SPSS : And sex, drugs and rock’n’roll (2nd ed.). London: SAGE.

Field, A. P. (2013). Discovering statistics using SPSS : And sex, drugs and rock’n’roll (4th ed.). London: SAGE.

Fincher, R. M., Lewis, L. A., & Rogers, L. Q. (1992). Classification model that predicts medical students’ choices of primary care or non-primary care specialties. Academic Medicine, 67(5), 324-327.

Fitzjohn, J., Wilkinson, T., Gill, D., & Mulder, R. (2003). The demographic characteristics of New Zealand medical students: the New Zealand wellbeing, intentions, debt and experiences (WIDE) survey of medical students 2001 study. Journal of the New Zealand Medical Association, 116, 1183.

Glynn, R. W., & Kerin, M. J. (2010). Factors influencing medical students and junior doctors in choosing a career in surgery. The Surgeon, 8(4), 187-191. 10.1016/j.surge.2009.11.005
Google.com. (2015). Homepage. Retrieved February 13, 2015, from https://www.google.co.nz/forms/about/

Health Workforce New Zealand. (2011, March). An assessment of the utility of graduate-entry medical education programmes for New Zealand. Retrieved January 24, 2015, from
http://www.nzdoctor.co.nz/media/930253/graduate%20medical%20entry%20programme%20report%20final%203%20june.pdf

Henning, M. (2010). Enhancing accessibility for university students with specific learning disabilities using a computerized screening system. The Open Rehabilitation Journal, 3, 34-40. 10.2174/1874943701003020034

Henning, M. A., Hawken, S. J., & Hill, A. G. (2009). The quality of life of New Zealand doctors and medical students: What can be done to avoid burnout? The New Zealand Medical Journal, 122(1307), 102-110.

Henning, M. A., Krägeloh, C., Hawken, S., Zhao, Y., & Doherty, I. (2010). Quality of life and motivation to learn: A study of medical students. Issues in Educational Research, 20(3), 244-256.

Henning, M. A., Krägeloh, C., Hawken, S. J., Doherty, I., Zhao, Y., & Shulruf, B. (2011). Motivation to learn, quality of life and estimated academic achievement: Medical students studying in New Zealand. Medical Science Educator, 21(2), 142-150.

Henning, M. A., Krägeloh, C. U., Manalo, E., Doherty, I., Lamdin, R., & Hawken, S. J. (2013). Medical students in early clinical training and achievement motivation: Variations according to gender, enrolment status, and age. Medical Science Educator, 23(1)(6-15).

Hilliard, R. I. (1995). How do medical students learn: Medical student learning styles and factors that affect these learning styles. Teaching and Learning in Medicine: An International Journal, 7(4), 201-210.

Houston, J., Harris, P., McIntire, S., & Francis, D. (2002). Revising the competitiveness index using factor analysis. Psychological Reports, 90(1), 31-34.

Krägeloh, C. U., Kersten, P., Billington, D. R., Hsu, P. H.-C., Shepherd, D., Landon, J., & Feng, X. J. (2013). Validation of the WHOQOL-BREF quality of life questionnaire for general use in New Zealand: Confirmatory factor analysis and Rasch analysis. Quality of Life Research, 22(6), 1451-1457.

Marsh, H. W., & Seaton, M. (2013). Academic self-concept. International guide to student achievement, 62-63.

McManus, I. C., Powis, D. A., Wakeford, R., Ferguson, E., James, D., & Richards, P. (2005). Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school. BMJ, 331(7516), 555-559. http://dx.doi.org/10.1136/bmj.331.7516.555

Ng, C. L., Tambyah, P. A., & Wong, C. Y. (2009). Cost of medical education, financial assistance and medical school demographics in Singapore. Singapore Medical Journal, 50(5), 462-467. Retrieved from http://smj.sma.org.sg/5005/5005a5001.pdf.

Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: what can be done? Assessment & Evaluation in Higher Education, 33(3), 301-314.

Paro, H. B. M. S., Morales, N. M. O., Silva, C. H. M., Rezende, C. H. A., Pinto, R., Morales, R. R., . . . Prado, M. M. (2010). Health‐related quality of life of medical students. Medical Education, 44(3), 227-235.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33.

Poole, P. J., Moriarty, H. J., Wearn, A., Wilkinson, T., & Weller, J. M. (2009). Medical student selection in New Zealand: looking to the future. New Zealand Medical Journal, 122(1306), 88-100.

Reagan, J., & Blakemore, L. C. (2009). Competitiveness Can Undermine Team Goals. Virtual Mentor, 11(5), 368.

Salamonson, Y., Everett, B., Koch, J., Wilson, I., & Davidson, P. M. (2009). Learning strategies of first year nursing and medical students: a comparative study. International Journal of Nursing Studies, 46(12), 1541-1547.

Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409-432.

Shulruf, B., Poole, P., Wang, G. Y., Rudland, J., & Wilkinson, T. (2012). How well do selection tools predict performance later in a medical programme? Advances in Health Sciences Education, 17(5), 615-626.

Skevington, S. M., Lotfy, M., & O’Connell, K. (2004). The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Quality of Life Research, 13(2), 299-310.

Staples, M. E. (2008). Promoting student collaboration in a detracked, heterogeneous secondary mathematics classroom. Journal of Mathematics Teacher Education, 11(5), 349-371.

Tempski, P., Bellodi, P. L., Paro, H. B. M. S., Enns, S. C., Martins, M. A., & Schraiber, L. B. (2012). What do medical students think about their quality of life? A qualitative study. BMC Medical Education, 12(1), 106.

The University of Auckland. (2015a). Biomedical Common Year or Overlapping Year. Retrieved February 13, 2015, from http://www.science.auckland.ac.nz/en/about/our-programmes/op-biomed-common-year.html

The University of Auckland. (2015b). Entry requirements for Bachelor of Medicine and Bachelor of Surgery (MBChB). Retrieved January 3, 2016, from https://www.fmhs.auckland.ac.nz/ en/faculty/for/future-undergraduates/undergraduate-study-options/mbchb/entry.html#6a24d5a2b8307836303d44e054d0adb1
Todisco, J., Hayes, S., & Farnill, D. (1995). Career motivations of male and female medical students. Psychological Reports, 77(3f), 1199-1202.

WHOQoL Group. (1998). Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological Medicine, 28(03), 551-558.

Published online: 3 January, TAPS 2017, 2(1), 1-6
DOI: https://doi.org/10.29060/TAPS.2017-2-1/OA1007

Diane Kenwright, Wei Dai, Emma Osborne, Tehmina Gladman, Peter Gallagher & Rebecca Grainger

University of Otago Wellington, New Zealand

Abstract

Although the flipped classroom approach has been theorised to encourage active learning, recorded lectures were used as flipped learning activities in the majority of the flipped learning studies, which many still consider to be passive knowledge input. To further promote active learning, the first four modules in a pathology course at University of Otago Wellington were flipped and delivered to fourth year medical students with redesigned active learning activities based on constructive theory of learning. The innovative active learning tasks were delivered via “kuraCloud” (an online-learning platform) and included short video clips, labeling exercises, written questions with immediate feedback, multiple-choice questions with explanatory text and links provided for further information. Students were required to complete all the activities prior to a face-to-face tutorial. A mixed methods design was used in which student engagement was assessed using both survey instruments and focus groups. Results showed that students did not engage with the new flipped learning activities designed to improve active learning, they preferred the passive learning approach with exam- or clinical practice- based core knowledge summarized and delivered to them directly. Three themes emerged: 1. Students considered the time invested in the active learning activities was inefficient. 2. The flipped course challenged students’ sense of predictability. 3. The knowledge construction process was valued as application rather than learning. This might be attributed to time-poor medical students judging the value of the learning task based on the relevance to their immediate learning goal: pass the exam, the new flipped learning activities imposing a heavy cognitive load that impairs the knowledge construction process and the less predictable structure of the flipped learning environment compared to the familiar traditional lectures.

Keywords:        Passive Learning; Active Learning; the Flipped Classroom; Course Structure; Cognitive Load

Practice Highlights

• Students prefer passive learning to active learning.
• Medical students judge what to learn and when to learn based on the perceived value of the task to their learning goals.
• Students prefer structured learning to fragmented flipped learning.
• Constructing knowledge is regarded as application rather than learning.
• Overloaded working memory impedes flipped learning.

I. INTRODUCTION

Passive learning has been theorized to facilitate surface learning or rote learning, rather than deep learning (Ramsden, 1997). When students are “passive learners” they receive information without actively seeking connections between their old knowledge and the new knowledge, leading to poor cognitive engagement in the learning process (Allosopp et al., 2007). Learning has been theorized to be a process of schema (meaningful combination of elements) construction, which can be achieved by bringing elements together during problem solving, integration of new elements into existing schemas, or by acquiring information that has already been schematized (Sweller et al., 1998). The constructivist theory of learning suggests that the most effective learning occurs when knowledge is constructed or reconstructed by individual learners rather than being transmitted directly from instructors (Piaget, 1967; Phillips, 1995). Student engagement may be best achieved during the knowledge construction process, in which students are actively integrating new information into the old information.

Rooted in constructivist theory of learning, the flipped classroom approach has been theorized to be effective in enhancing student engagement by actively involving students in their own knowledge construction processes (Hannafin et al., 1997; Kim et al., 2014; Gilboy et al., 2015; McLaughlin et al., 2013; James et al., 2014). Specifically, the flipped classroom is an innovative pedagogical approach where the instruction happens outside the classroom, while interactive learning activities that encourage higher-order thinking and knowledge application take place inside the classroom (Lage & Treglia, 2000; Strayer, 2012; Bishop & Verleger, 2013). Compared to traditional lectures where knowledge is passively delivered to students, the flipped classroom is composed of active and constructive learning tasks that may promote student engagement in health professional education.

The conventional flipped classroom typically delivers content by videos of lectures uploaded online, which could still be considered passive knowledge input. By replacing the traditional instruction in flipped classrooms with innovative learning activities based on the constructivist theory, students may be actively involved in the information-seeking process during problem solving, which is the key element in active learning (Allen & Tanner, 2005).

In this research a flipped classroom approach using online learning activities with a design based on constructivist theory was used. We hypothesized that active pre-learning activities would enhance the information integration process and improve student engagement.

II. METHODOLOGY

A. Participants

Seventy-five fourth year medical students (mean age 22.7 ± 2.2 years) enrolled in MBChB at University of Otago Wellington, New Zealand, participated this study. Forty-six were female (61%) and 29 were male (39%). Forty-nine were New Zealand European (65%) and 20 were Asian (Chinese and Indian descent) (26%).

B. Design

In 2015 the anatomic pathology teaching at the University of Otago Wellington employed two different delivery styles for the eight modules. The first four modules were delivered with a flipped format: cardiovascular (CVS), central nervous system (CNS), respiratory (R) and gastrointestinal (G). Students first completed activities in an e-learning platform “kuraCloud” <www.kuraCloud.com> were completed by students with knowledge delivery accompanied by multiple-choice questions, short video clips formatted as mini lectures, labeling exercises, quizzes with instant feedback, and written questions with answers given after student attempt, all designed to promote active knowledge construction. The students then attended face-to-face tutorials that included clinical cases with multiple choice questions, pair/share activities and flip card building. This research was approved by the Human Ethics Committee of University of Otago.

C. Evaluation and Assessment

1) Data Collection: This mixed methods study design used paper-based surveys and a focus group. A pre-course survey was administered to students prior to the flipped modules recording demographic information and perceived value of the course. A post-course survey assessing perceived value of the course, intrinsic motivation and time and study environment was administered in class time to students after the four flipped modules were completed, using a selected part of the Motivated Strategies of Learning Questionnaire (MSLQ; Duncan & McKeachie, 2005). All items were completed by students using a seven-point Likert scale (1=strongly disagree, 4=Neutral and 7= strongly agree), and a high value of reliability is observed (Cronbach’s α = 0.902). The post-course survey also requested students provide open-ended responses regarding their usage and reflections of online-learning using kuraCloud, including if learning activities were completed before or after tutorials, how often a video clip was watched thoroughly and whether the correct answer of a question was checked before or after they attempted the question.

Focus group questions were developed based on thematic analysis of open-ended questions in the post-course survey of the fourth-year pathology course (Figure 1). Students were invited to participate in the interviews when completing the end of course evaluation and again by the verbal invitation at the end of a lecture. Six fourth-year students volunteered to participate the focus group, which lasted an hour and was audio-recorded.

2) Data Analysis: Quantitative data analysis was conducted in SPSS 22. Since survey data were ordinal variables, nonparametric tests were used for analysis. The Wilcoxon signed rank test was used to compare responses of perceived value of the course in pre- and post-course survey.

The focus group interview was facilitated by two research assistants. Focus group data were analysed using thematic content analysis, with data transcribed and analyzed thematically by reading and re-reading the transcripts, coding themes emerging from the transcripts together with possible connections between themes (Braun &, Clarke, 2006; Burnard et al., 2008).

Figure 1. Some questions used in the focus group

III. RESULTS

A. Survey

The post-flip course survey showed medium intrinsic motivation in studying with challenging active flipped learning materials for majority of students (M = 4.73, SD = 1.05). Moreover, students’ perceived task value declined compared to the pre-course survey, with less intention to learn more pathology (pre-course median = 6.0, post-course median = 5.0, p = 0.002). Finally, students did not prioritize their study time for this flipped course (M = 4.16, SD= 0.62), with 55.3% of the students saying they did not spend very much time on this course due to their commitment to other academic activities, which they valued more highly.

B. Focus groups

Three major themes were identified regarding the students’ perception on the active flipped learning approach (starting from the most significant theme):

Theme 1: Time invested in the active learning activities was inefficient.

Students regarded the active learning process as extremely time-consuming and inefficient. One student reported: “On constructing your own knowledge – I think that’s a great idea but we’re very time-restricted so we don’t have time to sift through that.” Consistent comments were made by another student: “As a medical student I feel like I’m pressed for time – if there’s something simple…it makes it so much easier. If …we can access that information much quicker and retain it, which will make exam time easier and make us better clinicians …”

Structured information delivered passively was considered to be the most efficient approach to learning: “I think lectures are an easy way to give information quickly. They’re not very tailored, you just run through them quickly. My ideal pathology course would be a lecture that summarizes all the main diseases we should know, gives us a quick overview, summarizes things and points us to where we need to go…”

Theme 2: The flipped course challenged students’ sense of predictability.

Students reported that they perceived the flipped learning as unstructured and this created uncertainty, which made it hard for them to identify the link between the task and the learning goals. One student said “In everything else half of the time they even tell you what you’re meant to be learning. So you walk away going ‘I hope I saw everything I was supposed to’.” Another student said: “There were different links and it doesn’t seem as organised as it could be.”

Theme 3: The knowledge construction process was valued as application rather than learning.

Students conceptualized learning and applying knowledge as two non-overlapping activities. They described learning as acquiring new factual knowledge, which they distinguished from the task of applying this knowledge either in flipped classroom activities or quizzes. One student reported: “Last year…we were given a powerpoint before our pathology tutorials- that was the construction of knowledge, then we went into tutorials and had a quick quiz, then we applied the knowledge.” Similarly, another student commented: “The most helpful thing is to apply knowledge – to be quizzed and to be able to answer, so kuraCloud is good because it has instant feedback. I can type my answer out and compare it to hopefully a good answer and go, I’ve got these points but I missed these points. To me, that’s constructing my knowledge and making me recall it and that’s very useful to me.”

IV. DISCUSSION

Medical students engaged poorly with the flipped learning process used in this undergraduate anatomic pathology course. They preferred the traditional passive teaching method of a lecture providing summarized knowledge over active learning requiring searching for information and building meaningful connections, which they perceived as time-consuming and unstructured. This can be explained by the expectancy-value theory of motivation, which assumes that behavior is a function of the individual expectations and the perceived value of the goal that is being pursued, and the behavior people will choose is one that can achieve the most expected success and value (Fishbein & Ajzen, 1972). In the educational context, students’ engagement is closely related to their understanding of the value of the learning activities towards examinations. For improved engagement in any learning activity, a connection must be built between the learning activities and passing the course (Karaksha et al., 2013; Biggs, 2003). The medical curriculum is crowded and students judged the new active flipped activities a poor investment of their time, despite a sound underlying pedagogical basis. With other academic disciplines taught concurrently, medical students failed to properly engage in the active learning process because the amount of time required to complete the active flipped learning process did not contribute to their immediate learning goal, which was to pass the exam. Since traditional assessment still emphasizes the mastery of independent pieces of knowledge rather than the ability of integrating knowledge, it is very difficult for students to recognize how these innovative learning activities aimed at higher level of Bloom’s taxonomy (Bloom, 1956) can improve their academic performance (Biggs, 2003). Therefore, passive learning, such as traditional lecture format with important knowledge relevant to the exam and clinical practice summarized and directly delivered to students, is still regarded as the most efficient and preferred way of learning.

Another reason why students perceived flipped learning as inefficient may be the heavy cognitive load imposed by flipped learning tasks on kuraCloud. Cognitive load theory assumes that human cognitive architecture consists of a capacity-limited working memory and unlimited long-term memory (Merriënboer & Sweller, 2010). The knowledge construction process can be affected by the working memory resources available for processing new information. To successfully complete the redesigned flipped learning activities in the present study, students needed to synthesize information from multiple sources, including previous knowledge, information provided as hints in kuraCloud and websites out of kuraCloud using the link provided. Students’ working memory might have been overloaded when trying to seek, process and integrate relevant knowledge simultaneously, which may hinder the knowledge construction process and lead to a poor learning experience (Sweller, Merrienboer & Paas, 1998; Merriënboer & Sweller, 2010). Future research should focus on reducing unnecessary cognitive load of online flipped activities in order to facilitate effective learning process, while still maintaining active learning.

Compared to the more structured and task-orientated environment provided by the traditional lectures, the fragmented nature of the flipped classroom might be another possible reason why students prefer traditional passive learning. Specifically, traditional lectures provide a predictable structure to students that allows them to see connections in the knowledge delivered in a familiar way (Strayer, 2012). Whereas in the less-structured flipped learning environment, the orientation varied across flipped learning activities, so that students experienced a high level of unpredictability and uncertainty (Strayer, 2012). This is supported by the qualitative data in the present study, with students less satisfied with the flipped classroom approach in terms of course structure, which made them feel that course objectives were not clear enough and it was hard to see connections among concepts learnt in a meaningful way.

V. IMPLEMENTATIONS AND CONCLUSIONS

Although the flipped classroom has been found to improve students’ motivation and engagement, this study showed there remain challenges related to this transformative process. Students are more comfortable with the traditional passive learning where knowledge has been provided directly to them, that does not demand higher-order thinking. Using an active learning approach that is different from students’ expectations might lead to frustration. To alleviate frustration and convert students from passive learners to active learners, the following points might be considered when implementing the flipped classroom approach:

1. Consideration should be given to students’ workload when designing flipped learning activities (Al-Zahrani, 2015). Medical students are very likely to have crowded curriculum, therefore, it is important not to increase the extra workload. Based on our experience, flipped learning activities that can be completed within half an hour would be appropriate as pre-contact class preparation.
2. The instruction and learning activities should be designed following principles of cognitive load theory. The pre-contact lesson materials should be from a single-source and easily accessible to reduce external cognitive load so that more working memory resources will be available for meaningful and effective learning process (Merriënboer & Sweller, 2010).
3. Learning goals should be precisely summarised for students before flipped learning activities. When students have a clear view about what they are expected to learn and how these learning goals connected to the flipped learning activities, they are more likely to be motivated to engage in the active learning approach (Hidi & Renninger, 2004).
4. Students’ perception of learning and application should be changed prior to implementing the flipped classroom approach. Students are used to the traditional passive learning model where knowledge input comes before knowledge application. It is important to introduce the concept of “learning by doing” (Dewey, 1929; Reigeluth, 2013) to students to reduce the distinction between learning and application before the implementation of the flipped classroom approach.

In conclusion, students failed to embrace the redesigned flipped learning approach and responded with frustration and discomfort, which prevent them from cognitively and emotionally engaging in the active learning process. Passive learning with the main points summarized and presented before application is still regarded as the most efficient way of learning by these time-limited medical students. To facilitate an active learning process, instructors should design the flipped learning activities carefully with consideration given to students’ workload, the cognitive load imposed by outsourced information, the clarity of expected learning goals and the change of students’ conception of learning.

Notes on Contributors

Dr. Diane Kenwright is the Head of Department of Pathology and Molecular Medicine, University of Otago, Wellington. She investigates student engagement with elearning.

Wei Dai is a research assistant of Department of Pathology and Molecular Medicine, University of Otago, Wellington. She is currently doing her Ph.D. in Educational Psychology in Victoria University of Wellington. Her research interest lies in the area of involvement of cognitive resources in the learning process.

Emma Osborne is the Student Learning Advisor of at the University of Otago Wellington. She has a Masters of Education in adult education, with a focus on how people learn in the context of community development projects. Emma is interested in active learning and embedding transferable academic skills into students’ coursework.

Dr. Tehmina Gladman is a member of e-learning team at the University of Otago Wellington. She has tertiary level instructional experience, and instructional design and eLearning training and development experience. Tehmina sees her position as one of facilitating the development of faculty expertise in the use of eLearning tools.

Dr. Peter Gallagher is the medical education advisor at the University of Otago Wellington.

Dr. Rebecca Grainger is a consultant rheumatologist. She teaches chemical pathology and is the course convenor for the undergraduate pathology teaching programme at University of Otago Wellington.

Ethical Approval

Standard institutional review board (IRB) procedures have been followed and approval obtained by the authors.

Acknowledgements

The authors would like to thank Professor Brett Delahunt for reading and commenting on the paper and all the students in year 4 of University of Otago Wellington MBChB 2015 who participated the research.

Declaration of Interest

Authors have no conflicts of interest, including no financial, consultant, institutional and other relationships that might lead to bias.

References

Allsopp, D. H., Kyger, M. M., & Lovin, L. H. (2007). Teaching Mathematics Meaningfully: Solutions for Reaching Struggling Learners. Baltimore, USA: Brookes Publishing Company,

Allen, D., & Tanner, K. (2005). Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biology Education, 4, 262–268.

Al‐Zahrani, A.M., (2015). From passive to active: The impact of the flipped classroom through social learning platforms on higher education students’ creative thinking. British Journal of Educational Technology, 46(6), pp.1133-1148.

Biggs, J. (2003). Setting the stage for effective teaching. Teaching for Quality Learning at University: What the Student Does, 2nd edition. Buckingham, England: SRHE and Open University Press: 56-73

Bishop, J. L., & Verleger, M. A. (2013).The flipped classroom: A survey of the research. In ASEE National Conference Proceedings; Atlanta, GA.

Bloom, B.S. (Ed.). Engelhart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.

Burnard, P., Gill, P., Stewart, K., Treasure, E., Chadwick, B. (2008). Analysing and presenting qualitative data. British Dental Journal, 204(8), 429-32.

Cortina, J.M. (1993). “What is coefficient alpha? An examination of theory and applications”. Journal of Applied Psychology, 78, 98–104. doi:10.1037/0021-9010.78.1.98

Duncan, T. G. & W. J. McKeachie (2005). “The Making of the Motivated Strategies for Learning Questionnaire.” Educational Psychologist, 40(2), 117-128.

Dewey, J. (1929) (1960 edition) The Quest for Certainty: A study in the relation of knowledge and action, London: Putman

Fishbein, M. & Ajzen, I. (1972). Beliefs, attitudes, intentions and behaviour: An introduction to theory and research. Reading, Mass.: Addison-Wesley.

Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing Student Engagement Using the Flipped Classroom. Journal of Nutrition Education and Behavior, 47(1), 109-114.

Hannafin, M., Hill, J. & Land, S. (1997). Student-centred learning and interactive multimedia: Status, issues and implication. Contemporary Education, 68(2), 94–99

Hidi, S. & Renninger K.A. (2004). Interest, a motivational variable that combines affective and cognitive functioning. In Dai, D.Y. & Sternberg, R. J. (2004). Motivation, emotion, and cognition: Integrative perspectives on intellectual functioning and development (pp. 89-115). Mahwah, NJ: Erlbaum.

James, A. J., Chin, C. K., & Williams, B. R. (2014). Using the flipped classroom to improve student engagement and to prepare graduates to meet maritime industry requirements: a focus on maritime education. WMU Journal of Maritime Affairs, 13(2), 331-343.

Kim, M.K., Kim, S.M., Khera, O & Getman, J. (2014). The experience of three flipped classrooms in an urban university: an exploration of design principles. Internet High Education, 22, 37–50

Karaksha, A., Grant, G., Anoopkumar-Dukie, S., Nirthanan, S. N., & Davey, A. K. (2013). Student engagement in pharmacology courses using online learning tools. American Journal of Pharmaceutical Education, 77(6), 125

Lage, M.J., Platt, G.J. & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. Journal of Economic Education, 31(1), p. 30-43.

McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A, Glatt, D.M. Roth, M.T., Gharkholonarehe, N., & Mumper, R. J. (2013). Pharmacy student engagement, performance, and perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77(9), 196. doi: 10.5688/ajpe779196

Piaget, J. (1967). Six Psychological Studies (Elkind, Ed.), New York: Random House.

Phillips, D. C. (1995).The good, the bad, and the ugly: the many faces of constructivism. Educational Researcher, 24(7), 5-12.

Ramsden, P. (1997). The context of learning in academic departments. The experience of learning, 2, 198-216

Reigeluth, C.M. (1999). The elaboration theory: Guidance for scope and sequence decisions. In C.M. Reigeluth (Ed.), Instructional-Design Theories and Models: A New Paradigm of Instructional Theory. (Volume II). Hillsdale, NJ: Lawrence Erlbaum Assoc.

Strayer, J.F. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environment Research, 15, p. 171-193.

Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998).Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296.

Van Merriënboer, J. J., & Sweller, J. (2010). Cognitive load theory in health professional education: design principles and strategies. Medical Education, 44(1), 85-93.

Published online: 2 May, TAPS 2017, 2(2), 24-29
DOI: https://doi.org/10.29060/TAPS.2017-2-2/OA1006

Vindya Perera & Nelun de Silva

South Asian Institute of Technology and Medicine (SAITM), Malabe, Sri Lanka

Abstract

Flipped Classroom Model (FCM) is a method that was introduced to the educational system during the past decade which had shown substantial evidence of changing the traditional classroom learning to a more student centred learning environment. This method offers both students and the faculty, flexibility in teaching leaning activities that encourages a deeper approach to learning. This study was conducted to introduce FCM to undergraduates, to compare its effectiveness in learning clinical microbiology with lectures done in the traditional manner and to evaluate this system through student feedback. Power point presentations of 5 lectures were made accessible to students online and five lectures were done in a traditional manner. Freed lecture time was utilized to conduct discussions on clinical cases and problems. The effectiveness of this approach was determined by comparing the average marks students obtained for answers to questions covering the topics delivered by two methods. Questionnaires were given to students to evaluate their perceptions, effectiveness and experience of FCM versus traditional lectures for learning. Flipped classroom model was shown to be more effective than traditional lectures when the average marks in the final assessment were compared. More than 50% of students agreed on the benefits of FCM.  Majority suggested a combination of FCM and traditional lectures followed by small group discussions would be more beneficial. Newer models of teaching and learning such as FCM would enable teaching and learning in pre and para clinical subjects in a medical curriculum to be more students centred and encourage deep learning.

Keywords:        Flipped Classroom Model; Medical Education; Microbiology

Practice Highlights

• FCM can be adopted as a strategy to implement higher order teaching learning activities without increasing class time.
• This study shows that students performed better in the clinical microbiology assessment given from the topics covered by FCM than traditional lectures.
• Taking into consideration the perceptions of students, a combination model of FCM and traditional lectures is the best option for teaching and learning in clinical microbiology in our institution.

I. INTRODUCTION

Effectiveness of teaching methods that depends on lectures has been a questioned among educators involved in adult education (Barr & Tagg, 1995). For centuries teaching and learning pedagogy has mainly depended on lectures (Bligh, 2000). Even though these traditional face to face lectures provide an efficient method of transferring knowledge, it does not engage students in active learning. Educators are searching for techniques that move learners away from ‘superficial learning’ towards ‘deep learning’ that occur by being involved actively in the learning process (Ritchhart, Church & Morrison, 2011). To encourage the active participation of learners that leads to deep learning, educators use different techniques to shift the teacher centred paradigm towards a learner centred paradigm. One such technique that has become popular in the past decade is the ‘Flipped Classroom’ Model (Roehl, Reddy & Shannon, 2013).

Flipped Classroom model (FCM) is a technique that was introduced by Jonathan Bergmann and Aaron Sams in early 2000 where they delivered their lectures as video records for students to watch at home and the students did more constructive activities during freed class time (Siegle, 2014). According to Bergman and Sams, students need assistance of teachers to answer questions and when solving problems in an assignment but not when listening to a lecture. With the advancement of technology in the new millennium, where students have access to information at the tip of their fingers, the importance of lectures keep diminishing in adult education. Newer technologies like Learning Management System (LMS) software, allow educators to manage their teaching materials in a manner where students can read or listen to lectures whenever and wherever they want. These technologies make it easier for educators to adopt innovative ideas such as flipping the classroom that allow students to access and study lectures at their own pace. Freed class time would allow teachers to engage in techniques that involve students in higher order cognitive work.

Medical education can be improved without increasing the time it takes to earn a medical degree, if lessons are made more comprehensible and memorable and use a learning strategy that is student centred to encourage deep engagement in learning (Heath & Heath, 2007). If the FCM can be applied to teaching and learning in the pre and para clinical sciences in medical schools, class time would be freed up for more interactive lessons. Using LMS to deliver lectures also makes it possible for teachers to monitor each student’s access to the system and to use that knowledge to conduct focused interactions with students using freed lecture time. Students would also welcome more opportunities for case-based, problem based, and team-based exercises; strategies that activate prior knowledge. Teachers would be able to actually teach, rather than merely make a speech (Prober & Heath, 2012). A meta-analysis published by the Department of Education, USA has concluded that “on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction,” with larger effects if the online learning was combined with face-to face instruction (<http://www2.ed.gov 2010>).

This study was conducted to introduce FCM to undergraduates learning clinical microbiology, to compare its effectiveness with traditional lectures and to evaluate this system through student feedback.

II. OBJECTIVES

A. General Objective:
To compare the effectiveness of FCM with lecture based traditional method of learning in Microbiology.

B. Specific Objectives:

1. To obtain and analyse the perceptions of students about FCM
2. To compare the effectiveness of FCM in learning clinical microbiology with lecture based traditional teaching
3. To compare changes in student performance before and after introducing FCM

III. METHODS

The Department of Microbiology, Faculty of Medicine, South Asian Institute of Technology and Medicine (SAITM) introduced this new model in the last six weeks of semester five, when students learn Clinical Microbiology. Of the ten lectures scheduled during this time period, five were randomly selected to be conducted as FCM. Power point presentations (with detailed explanatory notes in the notes section of each slide) of the selected five lectures and relevant quizzes were uploaded on a learning management system, one per week, for students to view, learn and assess themselves. The students were not given curricular time to view the slides, since they were available online throughout the week for them to learn by themselves at their own pace and time. During the freed curricular time in the semester, clinical cases and problems relevant to the lectures given on line were discussed by students in small groups with teachers acting as facilitators. The other half of the lectures were done in the traditional manner face to face. Here too these lectures were followed by student led small group discussion of clinical cases and problems in the same manner. Therefore, to ensure that students were not deprived of equal opportunities for learning in the two methods, both the online and traditional lectures were followed by student led small group discussions. The contents of the clinical cases and their discussions were organized and conducted in a similar manner following online lectures and traditional lectures to prevent any bias that would have arisen due to the research. The only difference in the two methods was that the students were allowed to learn the content on their own in the FCM rather than listening to a didactic lecture. The lecture topics covered by the FCM and by the traditional method were selected randomly. The degree of difficulty of the contents in each lecture topic was similar since they covered the general objectives of the microbiology course and the specific objectives developed for each topic in clinical microbiology. The same teachers were involved for the lectures and the small group discussion during the study period. Ethical clearance was obtained from the Ethical Review Committee of the Faculty of Medicine, SAITM.

A. Analysis of perception of students on FCM
Questionnaires were given to students to evaluate their perceptions on the importance, effectiveness and their experience of the new model versus the traditional model for learning. Feedback from students was collected anonymously  at the end  of the course  in  microbiology but before the final examination. This was done with a questionnaire containing pre-prepared options for students to select, together with a few open ended questions to find out the reasons for their selected options. The responses given were analysed by descriptive statistics.

B. Comparison of the effectiveness of FCM in learning clinical microbiology with lecture based traditional teaching
The performance of students in the final examination in microbiology was analysed. The final examination consisted of a theory paper which included 20 true false MCQ, 4 EMI and 4 SEQ. The practical component consisted of 20 OSPE questions. The questions set for the examination were designed to cover course content in clinical microbiology as well as general and systematic microbiology. MCQ were made to assess mainly the students’ knowledge on general and systematic microbiology, while EMI and SEQ were used to assess the application of their knowledge on clinical microbiology case scenarios and situations. OSPE questions were also made to assess mainly the application of their knowledge on infectious diseases. The setting of questions for the final examination was not done with the intention of covering topics done by the two methods. It was done in the usual manner according to a blue print and taking into consideration past examination questions.

Of the 20 MCQ, 5 questions covered the clinical microbiology lectures and of these, 3 covered topics done by FCM and 2 covered topics done by traditional lectures. Of the 4 EMI with 20 leading questions (5 scenarios for each EMI), 4 scenarios were from clinical microbiology topics done by FCM and 6 scenarios from lecture topics done in the traditional method. Six of the OSPE questions were from topics done by FCM and 8 OSPE questions from topics done traditionally. Of the 4 SEQ given in the theory paper, 1 question was from a topic done by FCM and 1 was by a topic covered by traditional method. The marks obtained for answers to questions set on topics covered by FCM were compared to the marks obtained for answers to questions set on topics delivered as traditional lectures. The mean marks of students for the questions covered by FCM and traditional learning was compared using paired student t test.

C. Comparison of changes in student performance before and after introducing FCM
The average scores of SEQ and OSPE components in the microbiology final examination of the present study batch was compared with the scores obtained in the final examination of the previous batch. In the previous batch all the topics were done by traditional lecture based teaching methods. (Scores obtained for MCQ and EMI were not compared since the formats of these two tools were different between the batches in the two examinations).

IV. RESULTS

A. Analysis of perception of students on FCM
Quantitative feedback was obtained from 50 students by using a questionnaire with pre -prepared options. All the options were analysed independently. Fifty-six percent agreed that FCM was important in medical education while forty-two percent stated it was not important. Fifty-four percent preferred a combination of FCM and traditional lectures followed by small group discussions. Fifty-four percent said their experience in the new model was good and forty percent said it was a waste of time. From the responses given by 50 students, factors that the students pointed out as important for the success of innovative models like FCM were analysed separately. Of the 50 students 28% pointed out that competency in the use of information technology of students was important for the success of FCM. Students’ preference for innovative techniques (18%) and student centred learning methods (18%) were the other factors that were pointed out by the students as factors that contributed to the success of the FCM. The factors that contributed to the failure of FCM were, conventional ideas they have regarding learning (12%); learning as they would have done in school where the teacher is the centre of information and learning is centred around lectures. In addition, 6% of students attributed the failure of FCM to their limited knowledge on information technology hindering them from actively engaging with learning on line and their unwillingness to engage in extra learning activities at home.

B. Comparison of the effectiveness of FCM in learning clinical microbiology with lecture based traditional teaching
In the final assessment in microbiology, the performance of students in areas covered by FCM and traditional learning was compared using a paired student t test. T-test results (p < 0.05) showed a significant difference in the marks obtained for answers to questions covered by FCM compared to those done in the traditional way. The average score of the questions covered by FCM was significantly higher than the average score of the questions covered by traditional lectures.

The difference between the mean score for the Extended Matching Items (EMI) in the final examination that were covered by FCM (mean = 6.85, sd = 1.9) and from the conventional lectures (mean = 5.4, sd = 2.6) was significant (p < 0.001). (Fig. 1)

There was also a significant difference (p < 0.001) between mean marks of answers to Structured Essay Questions (SEQ) covered by FCM (mean = 57.6, sd = 17.5) and from traditional lectures (mean = 41.6, sd = 21.3). (Fig. 2)

The difference between mean marks of Objective Structured Practical Examination (OSPE) covered by FCM (mean = 7.1, sd = 2.851) and conventional lectures (mean= 5.3, sd= 3.5) was also significant (p < 0.001). (Fig. 3)

However, there was no remarkable difference between the true false response MCQ’s asked from FCM (mean = 4.6, sd = 1.4) and from traditional lectures. (mean = 4.5, sd =1.5).

Figure 1.  Bar chart shows the mean marks of answers to EMI from lectures covered by FCM and traditional lectures

Figure 2.  Bar chart shows the mean marks of answers to SEQ from  lectures covered by FCM and traditional lectures

Figure 3.  Bar chart shows the mean marks of answers to OSPE from lectures covered by FCM and traditional lectures

Table 1.  Table shows the comparison of average scores of SEQ & OSPE, of the previous batch and the present study population for lectures covered by traditional lectures & FCM in the study

C. Comparison of the changes in student performance before and after introducing FCM
A difference was observed, when comparing students’ average scores of the present study batch with the previous batch where all the topics were covered by lecture based teaching methods. The average score for topics covered by traditional lectures in the present study batch had not improved when compared to the previous batch (scores were higher in the previous batch) (Table 1). The average score for topics covered by the FCM in the present study batch had improved when compared to the previous batch (Table 1).

V. DISCUSSION AND CONCLUSIONS

Introducing FCM produced a mixed reaction from the students. While appreciating the importance of a novel method, the students in this batch preferred a combination of traditional lectures and FCM. Overall the marks obtained in the majority of the components of the final examination in microbiology indicated that students performed better in areas covered by FCM. Perhaps learning on their own encouraged them to participate actively in the small group discussions that followed during the freed lecture hours in FCM, enabling them to learn more effectively and perform better in these areas. However, noting the students’ preferences, it would be best if the combination model of FCM and traditional lectures are done for learning in clinical microbiology.

In countries like Sri Lanka where students are used to a teacher centred environment for learning in schools with little exposure to student centred learning, introducing novel techniques such as FCM in higher education may be difficult. However, if students had been exposed to student centred learning and information technology courses during their school years or the first year in higher education, educators would have found it easier to implement student centred teaching and learning activities and students would have coped better and welcomed new strategies like FCM.

Innovative techniques such as the Flipped Classroom Model is more likely to address issues that higher education institutes face in the current context of increased demand for higher education and increase of student numbers in medical schools. However, when introducing models like FCM it is important to consider the other factors such as students’ IT knowledge that can influence the success of these models. Especially in countries like Sri Lanka, few students in a batch may not be competent in IT. In such instances the introduction of IT into a foundation course in the medical schools will enhance the usefulness of novel methods of learning dependent on IT and students too will be more willing to use such techniques when they can handle such methods with ease and comfortably. These factors have been rectified to some extent in the medical faculty of SAITM where a foundation course for medicine has been introduced to the new entrants which include IT among other aspects.

In this study although 56% is not an impressive number to show students acceptance of FCM, adopting different strategies like uploading video lectures, including online discussion forums   may improve the model and make it more acceptable to the students.

In the study there was no significant difference in the students’ performance with MCQ questions. This may be due to the fact that MCQ are developed purely to assess the lower level of cognitive domain (knowledge component). Perhaps the increase of interactive learning with application based deep learning that was covered by FCM may improve the students’ performance in examination components that were assessing higher levels in the cognitive domain (SEQ and OSPE).

A difference was observed, when comparing students’ average scores of the present study batch and the previous batch. The improvement of the average score for topics covered by the FCM in the study population can be a result of batch to batch variations. However, since the performance of the study batch for the topics covered by traditional lectures did not show any improvement when compared to the previous batch, the batch to batch variation may be minimal.

Considering that the lectures were done by the same tutors for the two batches and questions given covered the same objectives in the two examinations, the change in the delivery of the lectures could also be a factor that influences the change in the performance of the two batches in the topics covered by FCM.

The study could not confirm that FCM was the sole contributory factor for improvement of students’ performance. However, if students can be introduced to different pedagogical approaches like FCM, the limited time allocated for teaching and learning activities could be used to improve higher order knowledge and skills of students.

Notes on Contributors

Vindya Perera- BVSc (Pera.)- Lecturer in microbiology in the Department of Microbiology, Faculty of Medicine, SAITM with three years of teaching and research. Registered for MPhil in Faculty of Medicine, University of Colombo, Registered Veterinary surgeon in Sri Lanka Veterinary Council.

Nelun de Silva- MBBS. (Patna) MD Medical Micro (Col.)- Professor and consultant in microbiology, Department of Microbiology, Faculty of Medicine, SAITM and Neville Fernando Teaching Hospital, Malabe, Sri Lanka. Accredited Senior Teacher in Higher Education (ASTHE) University of Colombo with accreditation from Staff and Educational Development Association (SEDA) UK.

Acknowledgments

The authors acknowledge the cooperation and commitment of 5th batch of medical students, especially those of Sachitra Wijetunga and Ammar M. Jawfer.

Declaration of Interest

All the authors declared no competing interests.

References

Barr, R., & Tagg, J. (1995). From Teaching to Learning — A New Paradigm For Undergraduate Education. Change: The Magazine of Higher Learning, 27(6), 12-26.

Bligh, D.A. (2000). What’s the use of lectures? San Francisco, CA: Jossey-Bass

Heath, C., & Heath, D. (2007). Made to stick: why some ideas survive and others die. New York: Random House.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices /final report

Prober, C.G., & Heath, C. (2012). Lecture halls without lectures—a proposal for medical education. New England Journal of Medicine, 366(18), 1657-1659.

Ritchhart, R., Church, M., & Morrison, K. (2011). Making thinking visible: How to promote engagement, understanding, and independence for all learners. San Francisco, CA: Jossey-Bass.

Roehl, A., Reddy, S., & Shannon, G. (2013). The flipped classroom: an opportunity to engage millennial students through active learning strategies. (Strategies). Journal of Family and Consumer Sciences, 105(2), 44-49.

Siegle, D. (2014). Technology: Differentiating instruction by flipping the classroom. Gifted Child Today, 37(1), 51-55.

*Vindya Perera
South Asian Institute of Technology and Medicine (SAITM), Malabe, Sri Lanka
Tel: +(94) 11 241 3351, +(94) 11 241 3331
Fax: +(94) 11 241 3332
Email:v.perera83@yahoo.com