Published online: 2 May, TAPS 2018, 3(2), 29-37
DOI: https://doi.org/10.29060/TAPS.2018-3-2/OA1051

Wayne A. de Beer & Helen E. Clark

Waikato District Health Board, Hamilton, New Zealand

Abstract

The New Zealand Curriculum Framework (NZCF) for Prevocational Medical Training identifies a number of procedural skills that prevocational doctors should achieve during their first two years following graduation from medical school. This study aimed to identify the clinical confidence of graduate doctors in performing the list of procedures outlined in the NZCF at two points in time; following completion of undergraduate studies, and the first year of prevocational, preregistration training. An anonymous paper-based survey, consisting of 59 items, was completed by a cohort of PGY-1 doctors (n = 30) twice during 2015, with the first 48 items of the survey rating PGY-1s perceptions of their clinical confidence in performing procedures that fall under the 12 competencies identified in the Procedures and Interventions section of the NZCF. 70.8% of the procedures were rated above 2.0 at the start of the PGY-1 year, indicating that respondents had received teaching in, or viewed the procedure being performed, during undergraduate training. By year-end, procedural skills performance rated above 3.0 (i.e., confident in performing said procedure independently) was achieved in 52% of the listed skills. Low scores occurred in procedures listed under the categories ENT, Ophthalmology, Surgery and Trauma. While ratings of clinical confidence improved in many areas as expected during the PGY-1 tenure, some areas remained low. This highlights an issue that PGY-1 doctors may not be receiving adequate training in certain procedural skills listed as core NZCF competencies during the PGY-1 year.

Keywords:        Prevocational Doctors, Core Competencies, Procedural Skills, Clinical Confidence

I. INTRODUCTION

The New Zealand Medical Council (NZMC) published its New Zealand Curriculum Framework (NZCF) for prevocational medical training in February 2014, with the curriculum implemented in November 2014 (Medical Council of New Zealand [MCNZ] , 2014). The curriculum framework identified the expected learning outcomes for doctors during the first two years of employment following graduation from one of the two New Zealand medical schools. These two years are referred to as the Post-Graduate Year-1 and -2 (PGY-1 / -2) years.

The NZCF is designed to reflect the continuum of learning that starts during undergraduate training and continues during the PGY-1 and -2 years (MCNZ, 2014). The aim of the learning outcomes and, in particular, procedural competence is to promote and ensure patient safety (Patel, Oosthuizen, Child, & Windsor, 2008). To obtain general registration at the end of the PGY-1 year, doctors should have achieved sufficient experience in competently performing a substantive number of the procedures. The PGY-2 year allows for further refinement of procedural skill learning and helps to prepare house officers for vocational training.

The NZCF consists of five sections: Professionalism, Communication, Clinical management, Clinical problems and Conditions, and, Procedures and Interventions. Six overarching outcome statements apply to the execution of the Procedures and Investigations section. These relate the doctor’s ability to provide “safe treatment to patients by competently performing certain procedural and assessment skills” e.g. take informed consent, preparation and post procedure care (MCNZ, 2014). Procedural skills are listed under 12 categories (Table 2). During the PGY-1 year, doctors should achieve competency in 48 identified procedures (NZCF lists 47 procedures, however for measurement purposes we separated female and male bladder catheterisation procedures).

In addition to apprenticeship training achieved during the clinical attachments, various other learning opportunities exist for procedural skills learning during prevocational training. At the organisation where this study took place, six 1.5 hours’ sessions were scheduled for procedural skill learning at a skills simulation centre. PGY-1 doctors were also required to attend an 8-hour advanced life support training session to achieve the New Zealand (NZ) Certificate of Resuscitation (CORE). Formal education sessions provided additional opportunity to teach the theory to support procedural learning.

In this study, the PGY-1 doctors were asked to rate their confidence levels in the performance of several listed procedures. Clinical confidence was defined as an “acquired attribute that provides individuals with the ability to maintain a positive and realistic perception of self and abilities.” (Evans, Bell, Sweeney, Morgan, & Kelly, 2010). It is important to note that ‘clinical confidence’ and ‘clinical competence’ are not necessarily equivalent, with a brief definition of the latter being “the capability to perform acceptably those duties directly related to patient care”. Clinical competence can only be measured by standardised assessment frameworks such as those based on Miller’s pyramid model (Miller, 1990). On the other hand, clinical confidence is a self-assessment, which is not necessarily measurable by standardised tests. Students’ abilities to correctly self-assess have been documented frequently in the medical literature and procedural confidence was identified as an important concept (Fitzgerald, White, & Gruppen, 2003). Two previous studies pointed to procedural confidence as affecting the students’ willingness to engage in the procedure, engage in accurate self-assessment, and to seek external help in performing the procedure (Byrne, Blagrove, & McDougall, 2005; Hays et al., 2002).

The respondents’ ratings of clinical confidence in each of the identified procedures were compared at the start and end of the PGY-1 year. The first rating, at the start of the year aimed to identify their clinical confidence in undertaking procedures following their undergraduate training. This would theoretically reflect the degree to which the two NZ-based undergraduate programmes helped prepare students to learn procedural skills in clinical settings. By assessing their confidence at the end of the PGY-1 year, the authors wanted to assess the direction and degree of any changes in confidence in procedural skills performance because of PGY-1 training.

Benner’s Stages of Clinical Competence was used and adapted to medical training to define different levels of perceived confidence (Benner, 1984). Five statements guided house officers to determine their level of clinical competence in the procedural skills outlined (Table 1).

Table 1. Survey statements based on Benner’s stages of clinical competence

II. METHODS

The Clinical Education & Training Unit (CETU) at Waikato District Health Board (DHB) designed a paper-based survey, consisting of 59 items based on the competencies stated within the NZCF. These items were scored from 0-4; the score reflecting house officers’ perception of their clinical confidence level (Table 1). The first 48 items rated their confidence in performing procedures within the 12 categories identified in Table 2.

Table 2. Clinical skills and procedures item categories

Items 49 – 59 were designed to measure additional skills that fell within categories of leadership, administrative and communicative skills. The results for these items will be discussed in a separate publication.

A review of the Standard Operating Procedures of the Health and Disability Ethics Committee (HDEC) determined that the study did not require formal ethics approval, due to meeting guidelines around health information, human tissue and human participants, as outlined in the HDEC scope summary (Health and Disability Ethics Committee, 2016). Ethical standards were adhered to.

All PGY-1s (n = 30) who commenced working at Waikato DHB in 2015 were asked to complete the survey twice in 2015. Participants were offered the choice of partaking and could withdraw involvement at any stage. The first survey (baseline) was conducted at the start of the 2015 PGY-1 orientation period, while the second survey was conducted at the end of the 4th quarter. Response rate was high; 100% (30 respondents) at baseline, and 83% (25 respondents) at the end of the year (EOY). Survey identification numbers were used to track individual progress while maintaining respondent confidentiality. Demographic data related to gender and medical school attended prior to PGY-1 level was also collected.

III. RESULTS

All survey responses were recorded and analysed. Cronbach’s Alpha was .964 for the baseline survey and .868 for the EOY survey, which showed that the items had high internal consistency at both time points. Differences between individual item means at baseline and EOY were statistically analysed by using the Wilcoxon signed-ranked test.

Table 3 outlines the demographic data of our respondents (where identified). Sixty percent of our PGY-1 doctors were female. Of the group, 63.3% studied at the University of Auckland with 30% coming from the University of Otago.

Table 3. Demographics of respondents (Overall)

Table 4 outlines the mean respondent rating for baseline and EOY survey items that were part of the Clinical Skills and Procedures section. When interpreting the table, the authors concluded that any items that fell below a mean of 2 at baseline were identified as warranting attention. Similarly, items that fell below 3 at the end of the PGY-1 year were identified as potential areas for concern.

Table 4. Baseline and End of Year (EOY) self-rated competence level (clinical skills and procedures)

At the start of the PGY-1 year, the new doctors were most confident in their ability to scrub up, gown and glove (3.52) and this improved at EOY (3.79). This was followed by confidence in performing less invasive procedures like taking nasal/throat swabs and performing venepuncture. At EOY, taking venous or arterial blood, arranging a blood transfusion, performing intravenous cannulation and administering appropriate intravenous electrolytes scored above 3.5 indicating high clinical confidence levels. Male bladder catheterisation also scored highly at EOY (3.75).

Of the 48 clinical procedures listed, 34 (70.8%) were rated above 2.0 indicating that they had received satisfactory skill training in that procedure during undergraduate training. In the EOY survey, 43 out of 48 (90%) procedures were performed above the score of 2.0. However, the authors considered that by the end of the PGY-1 year doctors should be performing at a score of 3 indicating that multiple opportunities for practice of the skill had existed during the PGY-1 year and that they were confident performing the procedure independently. Twenty five of the 48 procedures (i.e. 52%) scored confidence levels above the score of 3. Low scores tended to occur in the following categories; Ear Nose and Throat (ENT), Ophthalmic, Surgical (more specifically, excising simple lesions, deriding and dressing a wound) and Trauma.

Analyses of the baseline and EOY results by gender, and by university attended were also conducted (Table 5). No gender differences were observed at baseline for any of the clinical competencies. However, four items did show significant gender differences in the EOY results. These were: Perform anterior rhinoscopy (p = .031), Administer eye drops (p = .019), Catheterise the female bladder (p = .042) and Perform speculum examination of the vagina and cervix (p = .002). Males rated themselves more competent in the first two items (although low overall), and females rated themselves more competent with the latter two items.

Table 5. Baseline and EOY self-rated competence level by gender and university attended

With regards to university attended prior, statistical significance was shown for four items at baseline. These were: Take blood cultures (p = .022), Perform intravenous cannelation (p = .005), Perform venepuncture (p = .042) and Scrub up, gown and glove (p = .048). In all four cases, the University of Otago graduates rated themselves more competent than their University of Auckland counterparts. By EOY, the difference between the university groups for these four items were non-significant (p > .05). However, three of the nine Otago graduates did not complete the EOY survey and therefore these results should be interpreted with caution.

IV. DISCUSSION

The terms “clinical confidence” and “competence” were employed cautiously in this study recognising that confidence was not necessarily a marker for competence and that only standardised assessment could verify actual competence (Stewart et al., 2000).

When comparing the two surveys, three trends emerged across the grouped categories. These were areas where clinical confidence:
1.  was high at both points i.e. pre- and post-PGY-1 (e.g., cardiopulmonary, diagnostic and surgical),
2.  was not high at baseline, but showed significant improvement by year-end (e.g. intravenous/intravascular) and,
3.  remained low at both baseline and EOY (e.g., ENT, ophthalmic).

Our results indicate that PGY-1 doctors may not be receiving adequate training in the list of procedural skills during the PGY-1 year and it would be imperative that clinical supervisors continue to focus on this attainment during the PGY-2 year. The study showed that they rated their inability to perform 48% of the clinical skills at a level of independence in most settings.

PGY-1 confidence in performing ENT and ophthalmic procedures remained low (<2) throughout the year. This suggested that the undergraduate programme was not adequately addressing the learning of these procedural skills, nor were they having the opportunities during the PGY-1 year to improve their skills in these areas. In contrast, while the students were poorly confident about their intravenous/intravascular skills at baseline, these skills improved during the PGY-1 year to a level of being capable of performing them independently.

Of concern is the drop in clinical confidence in performing speculum examination of the vagina and cervix. While developing a clinical skill is important, maintenance of that skill is equally important during the prevocational years. Further analysis of this item by gender found that PGY-1 males’ clinical confidence dropped from 2.55 to 1.90, whereas females’ confidence levels increased from 3.00 to 3.21. Connick, Connick, Klotsas, Tsagkaraki, and Gkrania-Klotsas (2009) identified procedural confidence as dependent on gender just as it was on being offered the opportunity for gaining experience. This item, and female bladder catheterisation, was rated significantly lower by males at EOY, which may suggest a lack of confidence with gender-specific procedures.

Some differences were found between the two main medical school graduates with respect to four items at the beginning of the PGY-1 year. The subsequent EOY survey indicated that these differences had vanished by year-end. However, it should be noted that a third of the Otago graduates, did not complete the EOY survey, so these results need to be interpreted with caution. It is also difficult to generalise our findings to the wider medical school graduate population given that approximately 150-180 students graduate from Auckland, and 210-230 students from Otago each year. A nationwide study of this sort would however provide insight into both whether the differences we observed are part of a national trend, and whether these differences have tapered off by the end of the PGY-1 year. Such information would provide useful feedback for the institutions involved.

The study did not address the association of clinical confidence in performing clinical skills and the types of clinical attachments completed during the PGY-1 year and whether these influenced the final results. This study also did not measure clinical confidence after the PGY-2 years. A proportion of the PGY-2 doctors are likely to complete clinical attachments in ENT, Ophthalmology and the Emergency Department which may allow for experience in competencies that scored low at the start and end of the PGY-1 year. Given the relatively small size of these departments, it would be unlikely that many PGY-2 doctors will rotate through these departments and therefore experience in performing these procedures would remain low. It is therefore vital that College training programmes that require the competent performance of these procedural skills ensure that vocational trainees receive adequate training (e.g. with the Royal New Zealand College of General Practice (RNZCGP)). A longitudinal-based study, similar in design to the current study could measure changes in clinical confidence at not only the beginning and end of PGY-1; but also at further time points (e.g., at the end of the PGY-2 year and the end of the first year of registrar training). This would provide valuable feedback for the above training colleges.

V. CONCLUSION

The skills survey conducted was designed as a self-assessment tool of how competent PGY-1s felt they were in regard to specific clinical skills and procedures. These procedures are outlined in the NZCF as core procedures and interventions that PGY-1s should be able to perform at the end of the PGY-1 year, while “…recognising the limits of their personal capabilities” (MCNZ, 2014). Our findings show that while this benchmark has been achieved in some fields, there are other areas lacking, which may be due to the lack of exposure in certain specialties in the PGY-1 year. Our concern is that competence in these procedures will remain low through the PGY-2 year and possibly as far as vocational training level, once again due to little practical involvement. This paper, and future longitudinal and / or nation-wide studies may therefore serve to inform current undergraduate curriculum planning at the medical school level, as well as provide feedback to the New Zealand Medical Council on the current level of PGY-1 confidence in the core clinical skills and procedures identified by the NZCF.

Notes on Contributors

Dr Wayne de Beer works as a specialist in Consultation-Liaison psychiatry and work part-time as a Clinical Training Director at the Clinical Education & Training Unit; Waikato Hospital, Hamilton, New Zealand. His focus is largely on the prevocational and vocational medical training periods. Publications have included medical education and psychiatry.

Ms Helen Clark is the Medical Education Officer based in the Clinical Education & Training Unit at Waikato Hospital, Hamilton, New Zealand. Her background includes research and statistical analysis in the fields of medical education and psychology. She has academic publications in both fields.

Ethical Approval

A determination of the need for formal ethical approval was sought from the New Zealand Health and Disability Ethics Committee (HDEC). The study was deemed by HDEC to meet the criteria of observational research, therefore did not require formal ethics approval, due to meeting guidelines around health information, human tissue and human participants, as outlined in the HDEC scope summary (Health and Disability Ethics Committee, 2016). The study was registered with Waikato DHB’s internal research committee.

Acknowledgements

We would like to thank Carol Stevenson, Personal Assistant to the Director of Clinical Training, at the Clinical Education and Training Unit at Waikato DHB, for the development and facilitation of the PGY-1 NZCF Procedural skills and communications Competencies Measure survey tool.

Declaration of Interest

The authors declared no competing interest.

References

Benner, P. (1984). From novice to expert: Excellence and power in clinical nursing practice. Menlo Park: Addison-Wesley.

Byrne, A. J., Blagrove, M. T., & McDougall, S. J. (2005). Dynamic confidence during simulated clinical tasks. Postgraduate Medical Journal, 81(962), 785-788.

Connick, R. M., Connick, P., Klotsas, A. E., Tsagkaraki, P. A., & Gkrania-Klotsas, E. (2009). Procedural confidence in hospital based practitioners: implications for the training and practice of doctors at all grades. BMC Medical Education, 9, 2.

Evans, J., Bell, J. L., Sweeney, A. E., Morgan, J. I., & Kelly, H. M. (2010). Confidence in Critical Care Nursing. Nursing Science Quarterly, 23, 334.

Fitzgerald, J. T., White, C. B., & Gruppen, L. D. (2003). A longitudinal study of self-assessment accuracy. Medical Education, 37, 645-649.

Hays, R. B., Jolly, B. C., Caldon, L. J., McCrorie, P., McAvoy, P. A., McManus, I. C., & Rethans, J. J. (2002). Is insight important? Measuring capacity to change performance. Medical Education, 36(10), 965-971.

Health and Disability Ethics Committee. (2016). Applying for review; does your research require HDEC review? Retrieved from http://ethics.health.govt.nz/applying-review.

Medical Council of New Zealand. (2014). New Zealand Curriculum Framework for Prevocational Medical Training. Retrieved from https://www.mcnz.org.nz/assets/News-and-Publications/NZCF26Feb2014.pdf.

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic medicine, 65(9), S63-7.

Patel, M., Oosthuizen, G., Child, S., & Windsor, J. A. (2008). Training effect of skills courses on confidence of junior doctors performing clinical procedures. New Zealand Medical Journal, 121(1275), 37-45.

Stewart, J., O’Halloran, C., Barton, J. R., Singleton, S. J., Harrigan, P., & Spencer, J. (2000). Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scales. Medical Education, 34(11), 903-909.

*Dr Wayne de Beer
Tel: +64 7 8398899 ext 98399
Fax: +64 21 2232549
Email: Wayne.deBeer@waikatodhb.health.nz

Published online: 2 May, TAPS 2018, 3(2), 25-28
DOI: https://doi.org/10.29060/TAPS.2018-3-2/OA1039

Pairoj Boonluksiri

Hatyai Medical Education Centre, Hatyai Hospital, Songkhla, Thailand

Abstract

Background: Smartphones are used worldwide. Consequently, it does seem to be having an impact on health-related problems if overused. However, it is uncertain whether it is associated with sleep problems or poor learning.

Objective: To determine the association between smartphone overuse and sleep problems in medical students as primary outcome and poor learning as secondary outcome

Methods: A cross-sectional study was conducted in 89 students having their own smartphones, at Hatyai Medical Education Centre, Thailand. The habits of using smartphone were obtained. Smartphone overuse during bedtime was defined as using longer than 1 hour according to Smartphone Addiction Scale (SAS). The primary outcome was napping in a classroom that was defined as a problem if it happened more than 20% of the time attending class. Sleep problems using Pittsburgh Sleep Quality Index (PSQI) and Epworth Sleepiness Scale (ESS) were obtained by self-assessment. Learning outcome measured by grade point average was the secondary outcome. Multivariable analysis was performed for the association between smartphone overuse and sleep problems.

Results: Of all students, 77.5% had sleep problems and 43.6% had napped in the classroom No personal characteristics, daily life behaviours, and physical environments were associated with sleep problems. 70.8% of all students found to over use smartphones during bedtime. The Facebook website was the most popular. Smartphone overuse was significantly associated with poor sleep quality (odds ratio= 3.46) and napping in the classroom (odds ratio=4.09) but not grade point average.

Conclusion: Smartphone overuse during bedtime in medical students is associated with sleep problems but not learning achievement.

Keywords:         Napping in Classroom, Sleep Problems, Smartphone Overuse

Practice Highlights

• The smartphone is a useful and essential tool for communication, it should be used smartly for appropriate purposes.
• Smartphone overuse during bedtime has a significant effect on sleep quality and consequently napping in the classroom.
• No significant association between smartphone overuse and learning outcome was found.

I. INTRODUCTION

Smartphones are now used in everyday life and offer a substantial variety of mobile applications for information, communication, education, and entertainment purposes. Most students also use them for hours and some tend to have the addiction to smartphones. By the Smartphone Addiction Scale (SAS), the incidence of smartphone addiction is as high as 16.9% and the duration of smartphone screen-time increases the incidence of addiction (Haug et al., 2015). Based on the definition of internet addiction, smartphone overuse may disturb users’ daily lives such as learning performance and sleep quality. College students spent almost nine hours daily on their cell-phones (Roberts, Yaya, & Manolis, 2014). Consequently, smartphones or computers have an impact on physical health-related problems such as blurred vision, myofascial pain syndrome at wrists and neck if overused (Ganeriwal, Biswas, & Srivastava, 2013). A previous survey of smartphone screen-time in general population showed that there was a significant association between screen time and poor sleep (Christensen et al., 2016). The prevalence of excessive daytime sleepiness in medical students was reported as high as 30.6% (Ramamoorthy, Mohandas, Sembulingam, & Swaminathan, 2014). But it is uncertain regarding the association between smartphone overuse and sleep problems or poor learning. The objective of this study is to determine the association between smartphone overuse and sleep problems in medical student as primary outcome and poor learning as a secondary outcome.

II. METHODS

A cross-sectional study was conducted in 89 Year-4 and Year-5 medical students at Hatyai Medical Education Centre, Thailand. All students having their own smartphones were included. Fifty-three students (58.4%) were male. Data were obtained using a questionnaire for student characteristics, habits of using a smartphone, self-assessment for sleep quality using Pittsburgh Sleep Quality Index (PSQI), and sleep problems using Epworth Sleepiness Scale (ESS). Recall information was filled out.

Smartphone overuse was defined as using longer than 1 hour during bedtime according to the previous study showing that it increased the incidence of smartphone addiction by Smartphone Addiction Scale (SAS) (Haug et al., 2015). The primary outcome was collected as the incidence of napping in the classroom defined as a problem if it happened more than 20 percent of the numbers of total class attending. The numbers of classroom absences in each student were not counted in the total class attending for the denominator. However, there were only a few students who did not attend completely in every session due to personal reasons such as illness, but they attended more than 90% of all sessions.

Nap means to sleep for a brief period, often during the day; doze or to be unaware of imminent danger or trouble. Learning achievement using grade point average (GPA) in last academic year was a secondary outcome. Data analysis were performed using multivariable logistic regression to find out the association between smartphone overuse among sleep problems and GPA.

III. RESULTS

Most of the students use expensive smartphones. Sixty-five percent of students have phones costing more than 560 USD, 29.4% having phones costing 560-280 USD, the rest having phones costing less than 280 USD. They bought expensive smartphones because of more options and more attractive features. All of 66.7% medical students usually take them all day.

The survey of internet access showed that the top 5 favourite rankings were Facebook, phone calls, non-academic searching, academic searching and LINE chatting. The other option was taking a photograph. The common periods of using smartphone were 9 pm until after midnight, 6pm-9pm, and all day equal 41%, 38%, and 21% respectively. Most students sleep lately. All of 78.6% sleep after midnight.

The mean duration of sleep at night was 2.3+1.1 hours. A total of 77.5% had sleep problems by ESS and 45% of these had poor sleep quality by PSQI. There were 63 cases (70.8%) having smartphone overuse more than 1 hour during bedtime. Consequent napping in the classroom was found as high as 43.6% (range 0-90) and was associated with smartphone overuse during bedtime significantly (p=0.004). No other student characteristics, daily life behaviours, and classroom environments were associated with napping (Table 1).

By multivariable analysis, napping in the classroom and poor sleep problems were associated with smartphone overuse significantly (odds ratio = 4.125 and 3.373 respectively) (Table 2). No significant association between smartphone overuse and short duration of night sleep less than 3 hours was found. The mean GPA was 3.25 (range 2.02-3.91). There was a high rate of smartphone overuse in an honour group with GPA more than 3.50 and low incidence of napping in the classroom. However, there was no statistically significant association between GPA and smartphone overuse.

Table 1. Comparison between napping in the classroom among student characteristics and behaviours by univariate analysis

Table 2. Comparison between smartphone overuse among sleep characteristics and GPA by multivariable analysis

IV. DISCUSSION

Sleep problem is common in students including medical students. The prevalence of sleep problems varies from 17-70% and it was multifactorial such as poor sleep quality, inadequate sleep time, depression, fatigue etc. which is in accordance with this study for poor sleep (Azad et al., 2015).

Medical students in this study had inadequate sleep. Their sleep time duration were less 3-4 times than National Sleep Foundation’s Sleep time duration recommendations suggesting that 18-25 year olds should have a sleep duration about 7-9 hours (Hirshkowitz et al., 2015). Abnormal ESS scores were associated with lower academic achievement (Hamza et al., 2012) however it was not found in this study. Previous studies showed some people having habits at bedtime such as reading, using a smartphone for relaxation. These behaviours have an effect on poor sleep quality and depression.

Smartphone overuse can lead to depression and anxiety, which can, in turn, result in sleep problems (Ahn & Kim, 2015; Alsaggaf, Wali, Alsager, Alkhammash, & Quqandi, 2014; Demirci, Akgönül, & Akpinar, 2015). University students with high depression and anxiety scores should be carefully monitored for smartphone addiction. Side effects of smartphone overuse such as a chronic headache, concentration problem, long-term memory problem, recent memory problem, insomnia and inadequate sleep have been reported among medical students (Abdulmohsen et al., 2016).

Caffeine and alcohol ingestion also affected sleep and daytime sleepiness. Sleep difficulties resulted in irritability and affected lifestyle and interpersonal relationships (Giri, Baviskar, & Phalke, 2013). This study could not find the association between smartphone overuse and learning outcome which is discordant with previous reports (Ahn, & Kim, 2015; Alsaggaf et al., 2014). It might be related to multifactorial and needs further study. The exact percentage of napping in the classroom in this study might be slightly overestimated if all students who attended only 90% of all sessions had no more napping in the classroom when they attended completely.

V. CONCLUSION

Smartphone overuse during bedtime in medical students has an association with sleep problems but not learning achievement.

Notes on Contributors

Pairoj Boonluksiri, a paediatric neurologist, works at Medical Education Centre, Hatyai Hospital, Thailand and is responsible for medical teacher and educator. Assessment is his favourite field in medical education. He has got a certificate of fellowship in medical education (assessment), at the University of Illinois at Chicago, the USA since 2003.

Ethical Approval

The Institutional Review Board for Human Research of Hatyai Hospital approved this study.

Declaration of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Abdulmohsen, A. A. J., Ahmed, R. A. F., Abdulaziz, A. H., Mahdi A. S., Abdulhameed A. L. K., & Sayed I. A. (2016). Patterns of use of ‘smartphones’ among male medical students at King Faisal University (KFU) and its side effects. International Journal of Science and Research, 5(10), 2319-7064.

Ahn, S.Y., & Kim, Y.J. (2015). The Influence of Smartphone Use and Stress on Quality of Sleep among Nursing Students. Indian Journal of Science & Technology, 8(35), 1-6. https://doi.org/10.17485/ijst/2015/v8i35/85943.

Alsaggaf, M., Wali, S., Alsager, A. B., Alkhammash, R., & Quqandi, E. (2014). Sleep disturbances among medical students at clinical years. European Respiratory Journal, 44, 2297.

Azad, M.C., Fraser, K., Rumana, N., Abdullah, A.F., Shahana, N., & Hanly, P.J. (2015). Sleep disturbances among medical students: a global perspective. Journal of Clinical Sleep Medicine, 11(1), 69–74.

Christensen, M. A., Bettencourt, L., Kaye, L., Moturu, S. T., Nguyen, K. T., Olgin, J. E., … Gregory, M. M. (2016). Direct measurements of smartphone screen time: Relationships with demographics and sleep. PLoS ONE, 11(11): e0165331. http:/dx.doi.org/ 10.1371/journal.pone.0165331.

Demirci, K., Akgönül, M., & Akpinar, A. (2015). Relationship of smartphone use severity with sleep quality, depression, and anxiety in university students. Journal of Behavioural Addictions, 4(2), 85–92.

Ganeriwal A. A., Biswas D. A., Srivastava T. K. (2013). The Effects of Working Hours on Nerve Conduction Test in Computer Operators. Malaysian Orthopaedic Journal, 7, 1-6. http://dx.doi.org/10.5704.MOJ.1303.008.

Giri, P. A., Baviskar, M. P., &  Phalke, D. B. (2013). Study of sleep habits and sleep problems among medical students of Pravara Institute of Medical Sciences Loni, western Maharashtra, India. Annals of Medical and Health Sciences Research, 3(1), 51–54.

Hamza, M., Abdulghani, A. A. A., Aorah, S. B. S., Nourah, M. A. S., Alhan, M. A. H. & Ali, I. A. (2012). Sleep disorder among medical students: Relationship to their academic performance. Medical Teacher, 34, S37–S41.

Haug S., Castro R.P., Kwon M., Filler A., Kowatsch T., & Schaub M.P. (2015). Smartphone use and smartphone addiction among young people in Switzerland. Journal of Behavioral Addictions, 4(4), 299–307.

Hirshkowitz, M., Whiton, K., Albert, S. M., Alessi, C., Bruni O., DonCarlos, L., … Adams Hillard, P. J. (2015). National Sleep Foundation’s Sleep time duration recommendations: methodology and results summary. Sleep Health, 1, 40–43.

Roberts, J. A., Yaya, L. H. P. & Manolis, C. (2014). The invisible addiction: Cell-phone activities and addiction among male and female college students. Journal of Behavioral Addictions, 3(4), 254–265.

Ramamoorthy, S., Mohandas, M., Sembulingam, P., & Swaminathan, V.R. (2014) Prevalence of excessive daytime sleepiness (EDS) among medical students. World Journal of Pharmaceutical Research, 3(4), 1819-1826.

*Pairoj Boonluksiri, M.D.
Pediatric Department
Hatyai Medical Education Center
Hatyai Hospital
Songkhla 90110, Thailand
Email: bpairoj@gmail.com

Published online: 2 May, TAPS 2018, 3(2), 19-24
DOI: https://doi.org/10.29060/TAPS.2018-3-2/OA1049

Rebecca Grainger¹, Emma Osborne², Wei Dai¹ & Diane Kenwright¹

¹Department of Pathology and Molecular Medicine, University of Otago Wellington, New Zealand; ²Higher Education Development Centre, University of Otago, New Zealand

Abstract

Cognitively complex assessments encourage students to prepare using deep learning strategies rather than surface learning, recall-based ones. In order to prepare such assessment tasks, it is necessary to have some way of measuring cognitive complexity. In the context of a student-generated MCQ writing task, we developed a rubric for assessing the cognitive complexity of MCQs based on Bloom’s taxonomy. We simplified the six-level taxonomy into a three-level rubric. Three rounds of moderation and rubric development were conducted, in which 10, 15 and 100 randomly selected student-generated MCQs were independently rated by three academic staff. After each round of marking, inter-rater reliability was calculated, qualitative analysis of areas of agreement and disagreement was conducted, and the markers discussed the cognitive processes required to answer the MCQs. Inter-rater reliability, defined by the intra-class correlation coefficient, increased from 0.63 to 0.94, indicating the markers rated the MCQs consistently. The three-level rubric was found to be effective for evaluating the cognitive complexity of MCQs generated by medical students.

Keywords:            Student-generated Multiple-choice Questions, Cognitive Complexity, Bloom’s Taxonomy, Marking Criteria, Moderation of Assessment

Practice Highlights

• Allow enough time for several cycles of moderation between markers, especially when the subject matter is complex. While other researchers have reported reaching a high level of inter-rater reliability swiftly, our research highlights that it can take time for teams to agree on a marking approach for complex, clinically-based questions.
• Guide students to write questions that require the information in the full stem to answer the question. We found that without additional guidance, students often wrote detailed clinical vignettes that were followed by straightforward recall-type questions.
• Minimise levels of complexity included in the rubric. We found three levels of complexity sufficient to make practical distinctions in the quality of students’ questions.

I. INTRODUCTION

Multiple choice questions (MCQs) are used widely in assessing medical education. Well-constructed MCQs can be valid and reliable assessment tools. (McCoubrie, 2004; Schuwirth & Van Der Vleuten, 2004).  From a practical perspective, they are also reusable, easy to administer and easy to grade. While a recognized drawback of MCQs is that they tend to test memorization rather than analytical thinking (Schuwirth & Van Der Vleuten, 2004; Veloski, Rabinowitz, Robeson, & Young, 1999), it is possible to construct MCQs that do test students’ ability to apply knowledge and analyse problems (Khan & Aljarallah, 2011; McQueen, Shields, Finnegan, Higham, & Simmen, 2014; Palmer & Devitt, 2007). Given that students modify their study strategies in accordance with the complexity of thinking they anticipate needing to use in summative assessment (Biggs, 1999; Scouller & Prosser, 1994), one challenge for medical educators is to develop cognitively complex MCQs that will foster the kind of analytical reasoning that students will need in their medical careers.

One facet of improving MCQs is developing clear guidelines for items that require cognitively complex thinking as well as memorization. This requires a framework for classifying the thinking needed to answer MCQs. Bloom’s Taxonomy of Educational Objectives (Bloom, 1956) and the subsequent revision of Bloom’s taxonomy (Krathwohl, 2002) are popular starting points for classifying MCQ (Bates, Galloway, Riise, & Homer, 2014; Buckwalter, Schumacher, Albright, & Cooper, 1981; Khan & Aljarallah, 2011; McQueen et al., 2014; Palmer & Devitt, 2007; Rush, Rankin, & White, 2016). However, the majority of these papers tend to describe the process of rating MCQs using such a taxonomy very briefly, perhaps implying that the act of categorizing questions can be assumed to be intuitive and straightforward. Yet when we attempted to score MCQs using a Bloom-derived taxonomy, we initially found it difficult to translate a theoretical approach to cognitive complexity into a practical marking guide.

Medical students at the University of Otago were tasked with writing case-based MCQs for topics in pathology. The purpose of this task was to engage students in deep, clinically relevant learning in a way that also fulfilled their need for material that prepared them for the end-of-year MCQ examination (Grainger, Dai, Osborne, & Kenwright, 2017). Our research team then developed a rubric to evaluate the cognitive complexity of these student-generated MCQs, and this paper reports this process of rubric development. We initially found a high level of disagreement between markers as to how questions should be scored, evidenced by a low level of inter-rater reliability. Through analyzing the cognitive processes required to answer the questions and revising our marking criteria, we subsequently achieved a high level of inter-rater reliability. This paper argues that assessing MCQs for cognitive complexity based on existing taxonomies is an achievable task for a non-specialist team and reports our process of developing marking criteria as a model for other teams attempting a similar task.

II. METHODS

The student-generated MCQ approach was used in four modules (cardiovascular, central nervous system, respiratory and gastrointestinal) of an anatomic pathology course at the University of Otago. One hundred and six fourth-year medical students were enrolled in the PeerWise platform, in which students create MCQs and answer questions that their peers have created. (University of Auckland, 2016). For each topic, each student was required to create at least two MCQs similar to those found in their end-of-year exam, each comprising a stem (case scenario with question), one correct answer and three or four plausible distractors.

A rubric based on Bloom’s Taxonomy for evaluating the quality of these MCQs was developed over three iterations. The highest level of Bloom’s taxonomy, synthesis, was not included in the rubric as it is not applicable to a pre-defined task such as writing MCQs. In the first round of moderation, 10 out of 201 MCQs were randomly selected and independently rated by three markers. Results were then shared between raters, and one of the raters (EO) identified patterns of agreement and disagreement using summative content analysis of keywords and phrases that indicated the steps the respondent needed to undertake to answer the question (Hsieh & Shannon, 2005). Following this analysis, MCQs that were representative of issues the markers disagreed were circulated between the team members. These questions were used as a starting point for structured conversations where each rater described the process that they had used to mark to the question. Then a subset of 15 out of 331 MCQs, followed by a further 100 out of 678 MCQs were rated, analysed and discussed in the same manner. After each round of moderation, the inter-rater reliability was determined by calculating the intra-class correlation coefficient (ICC) (Bartko, 1976). Three staff participated the rating process. One had content expertise (RG), while the other two had backgrounds in higher education (WD, EO).  The project had ethical approval from the University of Otago Human Ethics Committee (D16/423).

III. RESULTS

After the first round of marking, there was a low level of inter-rater reliability (ICC = 0.543, 95% CI -0.668-0.912), suggesting raters were inconsistent in assigning the MCQs to levels in the six-level rubric. There was high level of agreement among raters about whether certain types of question should be classified as cognitively complex or not. For example, all raters marked questions requiring recall or comprehension of factual knowledge lower than questions required the respondent to make a diagnosis based on a clinical scenario (see Figure 1). However, raters were inconsistent on which level a question should fall within the low-order thinking category (i.e. recall or comprehension) and within the high-order thinking category (i.e. application, analysis or evaluation). As the aim of the task was to foster cognitively complex questions, we condensed Bloom’s recall and comprehension levels into a single level. In line with literature indicating that analysis and evaluation frequently overlapped (Moseley et al., 2005) we condensed these two categories into one level, while retaining the distinction between application and analysis.

The inter-rater reliability slightly increased in the second round of marking using the simplified rubric (ICC = 0.62, 95% CI 0.105-0.869). Content analysis of characteristics of inconsistently marked MCQs showed that marking varied for clinical case-based MCQs. Some MCQs had recall-based questions nested within a stem that superficially featured a clinical case, and markers agreed after discussion that these should be treated as recall/comprehension questions (see Figure 2).

Figure 1. Question testing recall/comprehension without a clinical case in the stem
Note: Questions have been lightly edited for clarity and brevity (abbreviations expanded and extraneous description removed) but otherwise left as written by the students, reflecting understanding of pathology at a fourth year medical student level. Author’s chosen correct answer is indicated in italics.

Figure 2. Recall/comprehension question nested in a clinical stem

There was an unclear boundary between application and analysis/evaluation.  In the subsequent discussion, we agreed that questions where the respondent needed to choose a diagnosis from a straightforward list of symptoms should be classified as application (see Figure 3).

Figure 3. Question testing application of knowledge

We classified as analysis/evaluation questions which required the respondent to combine and interpret multiple forms of information or to anticipate other findings associated with a condition. For example, some questions required the respondent to predict likely test results from presenting symptoms, or combine and weight the importance of different sets of observations (see figure 4). Based on this discussion, specific explanations of each level of the simplified rubric in the context of medical education were generated and incorporated into the rubric (Table 1).

A high inter-rater reliability was shown in the third iteration using the simplified and redefined rubric (ICC= 0.89, 95% CI 0.845-0.923), suggesting that raters were assessing MCQs in a consistent way. Raters also reported improved time efficiency using the new rubric compared to the first two iterations.

Figure 4. Question testing analysis/evaluation of knowledge

Table 1. Rubric with categorization levels and explanations for the cognitive domain

IV. DISCUSSION

Student-generated, cognitively complex MCQs help prepare medical students for examinations which include these question types. This paper addresses the extent to which classifying questions by cognitive level is reliable, valid and practical. It also indicates a need for future research into how best to guide students in developing sophisticated MCQs.

We found our final rubric to be a reliable measure of question complexity, as evidenced by the high level of inter-rater reliability. The difficulties we found in drawing distinctions between levels of complexity were largely consistent with the challenges and possible solutions identified previously. For example, a lack of clarity in the top levels of Bloom’s taxonomy reflects other work suggesting that modelling the higher order skills hierarchically may not be appropriate. One major revision of the taxonomy reverses the order of the upper levels (Krathwohl, 2002) and other critics have suggested that the differences between higher order skills are not clear cut and that ranking these skills is somewhat arbitrary (Moseley et al., 2005). While some have attempted to argue that MCQs can draw on thinking skills at all levels (Bloom, 1956; Young & Shawl, 2013), these appear to either: relate to questions that would only require evaluative thinking if reasoned from first principles in the exam rather than memorized (Young & Shawl, 2013); or be MCQs asked in relation to an extended problem rather than containing all the necessary information within the stem (Bloom, 1956). In developing our rubric, we selected levels of cognition similar to other researchers (Rush et al., 2016; Vanderbilt, Feldman, & Wood, 2013), although we combined comprehension with recall rather than application, as some others have done (Khan & Aljarallah, 2011; Palmer & Devitt, 2007). This suited our purposes in assessing a subject with a very strong applied component, where there was a crucial and clear difference between understanding the salient features of a condition and being able to apply that knowledge to a clinical scenario.  The performance and utility of the rubric will need to be determined in other MCQ sets.

The difficulty we experienced in deciding how complex questions were does not appear to have been reported elsewhere; it is possible that this process is more difficult with highly involved clinical questions or that other authors have chosen not to focus on this area. One paper that does utilize Bloom’s taxonomy in rating student-generated physics MCQs found a high level of inter-rater reliability in marking questions (Bates et al., 2014). Despite this, the authors do note a similar issue to us in that they comment that it was easier to rate lower-order questions than to make distinctions between application and analysis. Here it is likely that the subject material could influence the ease of marking. Bates et al. (2014) rated students’ physics MCQs, and it may be that it was easier to identify, for example, whether single- or multiple-step mathematical calculations were required in these kinds of problems than identify the thought processes associated with clinical scenarios in our research.

In terms of the practicality of our rubric, we found that the clearly redefined rubric was effective in simplifying the rating process and reducing rating time. For non-content experts, the new rubric has enabled them to judge the level of cognitive effort at the same level as a content expert.

A final and not fully resolved question is how best to guide students in writing complex, scenario-based MCQs. Our larger research project found that students tended not to utilise theoretical guidance on using a model such as Bloom’s Taxonomy in developing their MCQs (Grainger et al., 2017). We therefore intend to develop a more concrete, example-based scaffold for item-writing and assess whether students produce a similar quality of questions using this modified guidance.

V. CONCLUSION

Developing a valid and readily useable rubric to assess student-generated MCQs was achievable. A further task is to apply this rubric to new sets of questions to further test its performance and utility.

Notes on Contributors

Dr. Rebecca Grainger is an academic rheumatologist in Department of Pathology and Molecular Medicine of University of Otago Wellington. She is passionate about patient-focused care and medical education. She is responsible for the overall coordination and implementation of the study and assisted preparation for publication.

Emma Osborne is a professional practice fellow in student learning at the University of Otago. Her research interests include e-learning and teaching & learning in medical education. She initiated the process of rubric re-development, conducted qualitative analysis and was responsible for manuscript preparation.

Wei Dai is a research assistant in University of Otago. She is currently a Ph.D. candidate in Educational Psychology. Her research interest lies in the area of student engagement in technology-enhanced learning. She was responsible for the quantitative analysis and manuscript preparation.

Associate Professor Diane Kenwright is the Head of Department of Pathology and Molecular Medicine of University of Otago Wellington. She is a registered pathologist and an enthusiastic medical educator. She approved this research and assisted the preparation for publication.

Ethical Approval

This research has been approved by the Human Ethics Committee of University of Otago (Level B), reference number D 16423.

Acknowledgements

The authors would like to thank all the students in year four of University of Otago Wellington MBChB programme who participated the research.

Declaration of Interest

Authors have no conflicts of interest, including no financial, consultant, institutional and other relationships that might lead to bias.

Funding

This research had no specific grant funding from any funding agency.

References

Bartko, J. J. (1976). On various intraclass correlation reliability coefficients. Psychological Bulletin, 83(5), 762-765. doi: http://dx.doi.org/10.1037/0033-2909.83.5.762.

Bates, S. P., Galloway, R. K., Riise, J., & Homer, D. (2014). Assessing the quality of a student-generated question repository. Physical Review Special Topics – Physics Education Research, 10(2), 020105.

Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18(1), 57-75.

Bloom, B. S. (Ed.) (1956). Taxonomy of educational objectives : the classification of educational goals. Handbook 1, cognitive domain. London, Longman.

Buckwalter, J. A., Schumacher, R., Albright, J. P., & Cooper, R. R. (1981). Use of an educational taxonomy for evaluation of cognitive performance. Academic Medicine, 56(2), 115-121.

Grainger, R., Dai, W., Osborne, E., & Kenwright, D. (2017). Self-generated multiple choice questions engaged medical students in active learning but not in a peer-instruction process. Paper presented at the 14th Asia Pacific Medical Education Conference (APMEC), Singapore.

Hsieh, H.-F., & Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15(9), 1277-1288. doi: https://doi.org/10.1177/1049732305276687.

Khan, M. U. , & Aljarallah, B. M. (2011). Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for Assessing the Cognitive Skills of Undergraduate Medical Students. International Journal of Health Sciences, 5(1), 39-43.

Krathwohl, D. R. (2002). A Revision of Bloom’s Taxonomy: An Overview. Theory Into Practice, 41(4), 212-218. doi: https://doi.org/10.1207/s15430421tip4104_2.

McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26(8), 709-712. doi: https://doi.org/10.1080/01421590400013495.

McQueen, H. A., Shields, C., Finnegan, D. J., Higham, J., & Simmen, M. W. (2014). Peerwise provides significant academic benefits to biological science students across diverse learning tasks, but with minimal instructor intervention. Biochemistry and Molecular Biology Education, 42(5), 371-381. doi: https://doi.org/10.1002/bmb.20806.

Moseley, D., Baumfiled, V., Elliot, J., Gregson, M., Higgins, S., Miller, J., & Newton, D. P. (2005). Frameworks for thinking: A handbook for teaching and learning. Cambridge: Cambridge University Press.

Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Medical Education, 7, 49-49. doi: https://doi.org/10.1186/1472-6920-7-49.

Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Medical Education, 16(1), 250. doi: https://doi.org/10.1186/s12909-016-0773-3.

Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004). Different written assessment methods: what can be said about their strengths and weaknesses? Medical Education, 38(9), 974-979. doi: https://doi.org/10.1111/j.1365-2929.2004.01916.x.

Scouller, K. M., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19(3), 267-279. doi: https://doi.org/10.1080/03075079412331381870.

University of Auckland. (2016). PeerWise. Retrieved from https://peerwise.cs.auckland.ac.nz/index.php.

Vanderbilt, A. A., Feldman, M., & Wood, I. K. (2013). Assessment in undergraduate medical education: a review of course exams. Medical Education Online, 18, 1-5. doi: https://doi.org/10.3402/meo.v18i0.20438.

Veloski, J., Rabinowitz, H., Robeson, M., & Young, P. (1999). Patients don’t present with five choices: an alternative to multiple-choice tests in assessing physicians’ competence. Academic Medicine, 74(5), 539-546.

Young, A., & Shawl, S. J. (2013). Multiple Choice Testing for Introductory Astronomy: Design Theory Using Bloom’s Taxonomy. Astronomy Education Review, 12(1), 1-27. doi: https://doi.org/10.3847/AER2012027.

*Rebecca Grainger
Senior Lecturer
Department of Pathology and Molecular Medicine
University of Otago Wellington
PO Box 7343, 23a Mein St
Newtown
Wellington South 6242
New Zealand
rebecca.grainger@otago.ac.nz
+64 4 385 5541

Published online: 2 May, TAPS 2018, 3(2), 6-18
DOI: https://doi.org/10.29060/TAPS.2018-3-2/OA1046

Joanne Kua, Mark Chan, Jolene See Su Chen, David Ng & Wee Shiong Lim

Department of Geriatric Medicine, Institute of Geriatrics and Active Ageing, Tan Tock Seng Hospital, Singapore

Abstract

Aims: Career counselling is a complex process. Traditional career counselling is unidirectional in approach and ignores the impact and interactions of other factors. The Systems Theory Framework (STF) is an emerging framework that illustrates the dynamic and complex nature of career development. Our study aims to i) explore factors affecting senior residency (SR) subspecialty choices, and ii) determine the suitable utility of the STF in career counselling.

Methods: A prospective observational cohort study of internal medicine residents was done. Surveys were collected at three time points. The Specialty Indecision Scale (SIS) assesses the individual components and expert consensus group derived the questions for the contextual components. We measured burnout using the Mashlach Burnout Inventory. Process influences were assessed via thematic analysis of open-ended question at the 3^rd survey.

Results: 82 responses were collected. There was a trend towards older residents being ready to commit albeit not statistically significant. At year 1, overseas graduands (OR = 6.87, p= 0.02), lifestyle factors (t(29)=2.31, p=0.03, d= 0.91), individual factors of readiness (t(29) = -2.74, p=0.01, d= 1.08), indecisiveness (t(27)= -0.57, p=0.02, d= 0.99) and self- doubt (t(29)= -4.02, p=0.00, d= 1.54) predicted the resident’s ability to commit to SR. These factors change and being married (OR 4.49, p= 0.03) was the only factor by the 3^rd survey. Male residents are more resolute in their choice (OR= 5.17, p= 0.02).

Conclusion: The resident’s choice of SR changes over time. The STF helps in understanding decision-making about subspecialty choices. Potential applications include: i) initiation of career counselling at year 1 and ii) reviewing unpopular SR subspecialties to increase their attractiveness.

Keywords:         Internal Medicine Residents, Career Counselling, Senior Residency

Practice Highlights

• Career decision-making is a complex process and there is a need for a holistic approach to it.
• The Systems Theory Framework consists of a multifaceted range of content (individual and contextual factors) and process influences (change over time, recursiveness, chance) to illustrate the dynamic and complex nature of career development.
• The resident’s choice of senior residency changes over time throughout their residency. The factors that affect their subspecialty choice transit from individual factor of indecisiveness, self-doubt and readiness in year 1 to contextual factor of lifestyle by year 3.
• Male residents are more resolute in their choice of senior residency.
• Reasons for resident’s change of choice of senior residency include i) experience during rotation ii) lifestyle choices and iii) influences from peers and seniors.

I. INTRODUCTION

The choice of senior residency training is an important decision for internal medicine residents. There is a rich array of options ranging from sub-specializing in a chosen field among the many areas in internal medicine, through to continuing as a general internist. This is not an easy process and not surprisingly, there is great interest in examining the factors that influence this decision. Understanding what is important from the perspectives of residents will be helpful for ‘unpopular’ subspecialties as they seek to transform the subspecialty to become more attractive by incorporating factors that residents look for in their choice.

There has been extensive research in the field of career decision-making in medical professionals, with many emphasizing a plethora of factors and their organization (Osipow, 1999). Previous literature reported that 20% of physicians in residency training switched to unrelated specialties and 16% of physicians already in practice changed their specialty identification (Chase, Levine & Weisman, 1980; Holden & Levit, 1978). In recent years, controllable lifestyle factors like personal time for leisure, family and avocational pursuits, control of total weekly hours spent on professional responsibilities, and the presence of burnout have been cited as some of the main reasons for the ultimate choice of subspecialty (Chibnall, Enoch, & Schindler, 2013; Dorsey, Jarjoura, & Rutecki, 2003; Jackson, Nuthalapathy, & Owen, 2004). Other important factors include programme characteristics and the reputation and quality of the training programme (Aagaard, Dedier, & Julian, 2005; Redman, Saltman, & Straton, 1994). However, these studies were mostly done amongst Caucasian medical students and residents and there has hitherto been limited research exploring the relative importance of these factors in Asian societies and even fewer studies looking specifically at internal medicine residents and their subsequent choice of senior residency.

The landscape of Singapore’s postgraduate medical training programme has changed much since the transition to the residency system in 2010. Singapore is one of the first countries outside of the United States to partner with the Accreditation Council for Graduate Medical Education- International (ACGMEI) to introduce this training to our graduates (Huggan, Samarasekara, & Ooi, 2012; Samarasekara, Ooi, & Hooi, 2015). The first batch of residents graduated from junior residency in 2013 and there were anecdotal accounts of “surprises” in choice of senior residency programme, namely those who made decisions that were contrary to prior indicated choices at the start of residency training. While the exact reasons are unclear, some have attributed this observed trend to the younger age at which trainees in Singapore enter residency training. Because medical education in Singapore’s largest medical school is an undergraduate course, residents in Singapore are therefore of a younger age (around 25 years of age) when choosing senior residency fellowship compared to their counterparts in the United States (more than 28 years of age). Thus, the relative ability to decide on their senior residency training and to commit to it upon completion of their residency training may be less firmly established.

Against this background, career counselling assumes salience in helping trainees navigate their career choices. Career counselling comprises both the principles of career theory and counselling theory. Traditional career theory used to focus on specific factors at a definite time point which are relevant to career choice, ignoring the possibility of other factors and the interactions that determine the subsequent career choice. Traditional career counselling is also seen only as a problem solving process where one’s knowledge of self is matched to the knowledge about the world of work, neglecting other important elements like ability, personality and/or aptitude. These have been perceived as being too narrow as it did not encompass other elements of the social or the environmental-societal systems in the process of career decision making (Chen, 2003). This has resulted in the recent emphasis of career counselling on the holistic approach to an individual and also on the effect that multimodal interactions of various elements have on eventual career choice (Patton & McMahon, 2014).

The Systems Theory Framework (STF) (McMahon, & Patton, 1999) (Figure 1) is an emerging framework in career counselling. It is a metatheoretical framework consisting of a multifaceted range of content and process influences to illustrate the dynamic and complex nature of career development. It accommodates not just the traditional perspectives of identifying what works best for the individual but also include a more holistic approach of incorporating many other elements into the decision making process itself. Content influences comprise the interconnecting systems of individual, social and environmental-societal factors (the latter two collectively classified as contextual) while process influences are made up of recursiveness (i.e. interaction between influences), change over time, and chance. Another strength of the STF is that it provides the user with a link between theory and practice, thus bridging the gap inherent in the more theory-centric traditional approaches (McMahon & Patton, 1999). The practical application of STF for guiding career assessment (Kontosh, & Zimmerman, 2007) and counselling (Patton & McMahon, 2006) have been previously described but its utility for understanding career choices in the context of internal medicine senior residency has not been well studied.

Figure 1. The Systems Theory Framework (STF)

Therefore, using the lenses of the STF, our study aims to evaluate factors that affect the choice of senior residency amongst internal medicine residents and to understand the utility of the STF as a potential framework for career decision-making in this context.

II. METHODS

This is an observational cohort study of all the year 1 internal medicine residents entering the National Healthcare Group-Alexandra Health Private Limited (NHG-AHPL) residency programme in 2013. The Internal medicine residency programme in Singapore consists of 3 years of junior residency followed by 3 years of senior residency. Options for senior residency range from general internal medicine, to procedural subspecialties such as cardiology and non-procedural subspecialties such as endocrinology. There are no exclusion criteria. These residents are followed up for 3 years through their junior residency. Anonymity was maintained for all responses. We collected a total of 82 responses over 3 years.

A. Survey

We collected baseline demographics including age, sex, socioeconomic status, location of medical school, marital status at time of choice and number of years post-graduation. We also collected information pertaining to the individual and contextual systems of the STF as well as the choice of their subspecialty. The surveys were administered at three time points: beginning of year 1, beginning of year 2 and end of year 2. We chose to do the third survey at the end of year 2 because we wanted to survey residents before they officially indicate their choice of senior residency at the beginning of year 3. Surveys were done for the entire month to ensure that we capture as many responses as possible.

B. Systems Theory Framework (STF)

The STF explores individual as well as contextual factors which affect one’s choice of career. The individual system is central to the STF, and describes a range of personal influences that can affect career development such as age, beliefs, personality, values and ability. In contrast, contextual factors examine the influence of the social and environmental-societal systems.

To study the individual factors, we adapted the 35-item Specialty Indecision Scale (SIS) (Alexander, Osipow, & Savickas, 1985; Early, Richard, & Savickas, 2007) that looks at 6 major career concerns: readiness, information, identity, barriers, indecisiveness and self-doubt. Readiness assesses whether the resident is ready to make a decision. Information refers to the need for the resident to locate sources of information and guidance to find out more before he can make a career decision. Identity addresses the need to find out more about themselves before exploring suitable specialties. Barriers refer to external conflicts (related to the specialties or persons) that prevent the resident from making a choice. Indecisiveness identifies internal conflicts that prevent the resident from choosing. Self-doubt refers to psychosocial factors like self-confidence or anxiety that prevents decision-making.

Each item is scored on a 4-point Likert scale ranging from 1 (Not at all like me) to 4 (Exactly like me). The mean scale score and scale standard deviation for each cohort are calculated, from which the SIS standard score is then derived using the pre-determined formula: SIS Standard score= 50 + ((Raw Scale Score- Mean Scale Score)/ Scale Standard Deviation) x 10. Scores are considered low if less than 40, moderate if between 40-60 and high if >60. A high score reflects an area of concern that should be addressed during subsequent career counselling.

To study the contextual system, we included factors like programme characteristics, lifestyle, subject matter, and the presence of burnout. The questions in each component are derived through expert group consensus after extensive review of the literature and team discussion. Responses are scored on a 5-point Likert scale ranging from 1 (strongly agree) to 5 (strongly disagree). We measured the degree of burnout in our residents using the Maslach-Burnout Inventory-Human Services Survey (MBI) (Jackson, Leiter & Maslach, 1996). The MBI is a 22-item self-administered questionnaire that assesses the three components of the burnout syndrome, namely emotional exhaustion, depersonalization and reduced personal accomplishment. It has been extensively used amongst healthcare professionals and demonstrates good reliability and validity different cultural settings (Chou, Hecker, & Martin, 2012; Endler, Fuchs, & Messenholl-Strehler, 2011; Endler, Fischer, & Mitsche, 2013; Gillespie, Kalliath, & O’Driscoll, 2000). Responses are collected on a 7-point Likert scale where 0 reflects a statement that the resident has never felt and 6 a statement that the resident feels every day. For the assessment of processes influences, we collected responses at the third time point as to whether there has been a change in their subspecialty choice and the reasons for the change. The respective scales chosen have been based on prior validation and relevance.

C. Outcome Measure

Primary outcome was their readiness to commit to an area of specialization for senior residency training like geriatric medicine, cardiology etc at each time the survey was done.

 D. Statistical Analyses

All statistical analyses were performed using SPSS version 21.0. We performed Chi square test to compare dichotomous variables between groups and the student t-test for unpaired data to compare parametric continuous data. The level of significance was taken to be 5%.

For each of the 3 time-points, we compared demographics, individual factors (SIS scores) and contextual factors between respondents who were ready to commit with those who were not. We performed subgroup analysis comparing the group whose choice of senior residency changed with those whose choice remained constant. For the former group, we also performed thematic analysis of open-ended responses to ascertain the reasons for change.

III. RESULTS

We collected a total of 82 responses over the 3 time points from July 2013 to June 2015. The number of respondents was respectively 31 (64.6% of eligible residents), 30 (62.5%), and 21 (43.8%) for the three surveys (Table 1).

A. Beginning of Year 1

Respondents had a mean age of 26.13 ± 2.33 (23-31) years. There was slightly more females in this cohort (54.8%) and majority of them were single (93.5%) and graduated from local universities (67.7%).

Amongst this cohort, there were 9 who were ready to commit and 22 who weren’t ready to commit (Table 2a). On univariate analyses, those who studied overseas (OR= 6.87, p= 0.02) tended to be more ready to commit than those who studied locally. Looking at the various components of the SIS score, the career concerns of readiness (t(29)= -2.74, p=0.01, d= 1.08), indecisiveness (t(27)= -0.57, p=0.02, d= 0.99) and self-doubt (t(29)= -4.02, p=0.00, d= 1.54) were significantly higher in the group who wasn’t ready to commit. Lifestyle factors seem to play an important role in the choice of senior residency for those who are ready to commit at year 1 (t(29) = 2.31, p=0.03, d= 0.91). There was also a trend towards those who are ready to commit being older (27.11 ± 2.03 years compared to 25.73 ± 2.37, t(29) = 1.53, p=0.14, d=0.60).

^aPost graduate year
^bMeasured by Mashlach Burnout Inventory (Human Services Survey
Table 1. Characteristics of the respondents over the 3 surveys

 

^aPost graduate year
^bMeasured by Mashlach Burnout Inventory (Human Services Survey)
^c P< 0.05
Table 2a. Survey at beginning of year 1

B. Beginning of Year 2

The mean age of respondents was 25.93 ± 1.66 (24-30) years. There was slightly more females (56.7%) and most were still single (93.3%). Compared to the first survey, there were more local graduates (76.7%) and most of them were within one to three years post-graduation from medical school (76.7%).

There were 10 (33.3%) residents who were ready to commit and 20 (66.7%) not ready to commit. There was a non-significant trend towards those who are ready to commit being older (26.70 ± 2.00 years compared to 25.55 ± 1.36 years, t(13.3) = 1.64, p=0.13, d= 0.72). The component of readiness in SIS was significantly higher for those who weren’t ready to commit (t(28) = -2.89, p=0.01, d= 1.12). Programme characteristic was also considered to be important for the group that was ready to commit even though not statistically significant (t(28) = 1.93, p=0.06, d= 0.75).

C. End of Year 2

Respondents had a mean age of 27.48 ± 2.02 (25-33) years. Most were females (71.4%) and the proportion of married residents increased from the previous 2 surveys to 19%. There were also more overseas graduates who responded for this interview (42.9%) and around one third of them are more than 3 years post-graduation from medical school (33.3%).

Eleven (52.4%) residents were ready to commit compared to 10 residents who were not (Table 2b). Interestingly, those who were married tended to be more ready to commit compared to those who are still single (OR= 4.49, p=0.03). Males also tended to be more ready to commit compared to females but this trend was not statistically significant (OR= 3.23, p=0.07). There was a non-statistically significant trend towards lifestyle factor being more important to those who are ready to commit (t(19) = 1.90, p=0.07, d= 0.84).

^aPost graduate year
^bMeasured by Mashlach Burnout Inventory (Human Services Survey)
^c P< 0.05
^d Senior residency
Table 2b. Survey at end of year 2

For the last survey, we also asked the residents whether their choice of senior residency has changed from when they were in first year of residency and out of those who were ready to commit, 72.7% were steadfast in their choice. However, for those who were not ready to commit, 50% had changed their choice since first year of residency.

D. Change of choice of Senior Residency (SR)

Eight residents had a change of SR choice whereas 13 (61.9%) did not (Table 3). The mean age for the latter group was slightly higher (27.77 ± 2.39 years compared to 27.00 ± 1.20 years, t(19) = -0.84 p= 0.41, d=0.38). The only significant finding was that male residents appear to be more resolute in their choices and tended not to change them (OR= 5.17, p= 0.02).

^aPost graduate year
^bMeasured by Mashlach Burnout Inventory (Human Services Survey)
^c P< 0.05
^d Senior residency
Table 3. Comparison of those whose choice of senior residency changed

Analysis of the open-ended responses to reasons for the change in SR revealed 3 main themes: 1) experience and exposure during the rotation, 2) lifestyle choices and life events that altered priorities, and 3) influences from peers and seniors that led to the change in their choice of SR. Rotations through the different postings allowed the residents the opportunity to experience subspecialties they previously had no rotation through as a student. Residents also were able to work with the different consultants within the department who impacted them sufficiently to make them want to join the department. Many residents quoted lifestyle factors as an important consideration, with priorities changing as they progressed through residency and the corresponding seasons in their life. Lastly, interactions with peers who had done the rotation or speaking to seniors who had chosen the subspecialty helped to shape residents’ understanding of the subspecialty and what it actually entailed.

IV. DISCUSSION

Our study contributes to the body of evidence by using the STF to longitudinally examine factors that affect the choice of senior residency programme among internal medicine residents. Our study shows that factors affecting a resident’s subspecialty choice were dynamic and differed throughout the course of their training. For first year residents, individual factors among those who are not ready to commit and lifestyle factors were important determinants in the first year. However, these factors changed in the subsequent 2 surveys such that by the 3^rd survey, only being married appear to influence the resident’s readiness to commit with life-style factors re-emerging as an important factor. The process influences highlighted in the STF are also evident by the recursiveness and dynamic nature over time of career choices, such that residents’ senior residency decisions are influenced by their experience and exposure during the rotation interacting with lifestyle choices or life events, and influences from peers and seniors.

In our study, first year residents who are transitioning into their training are especially vulnerable and struggle more with self-doubt, indecisiveness and readiness components of the SIS. This suggests that the SIS may be a useful tool to delineate individual factors of the STF for career counselling among junior residents. In support of this, other studies have reported that the SIS is a reliable tool with good internal consistency and relates well with subsequent specialty choice (Kantosh, & Zimmerman, 2007). Contrary to other studies where burn-out appears to be a prime motivational factor for the choice of certain subspecialties (Chibnall, Enoch, & Schindler, 2013), our study shows that the presence of burnout did not affect the readiness for the residents to commit to a choice of senior residency. In contrast to earlier studies from the West (Chibnall, Enoch, & Schindler, 2013; Dorsey, Jarjourna, & Rutecki, 2003; Holden, & Levit, 1978; Jackson, Nuthalapaty & Owen, 2004; Redman, Saltman, & Straton, 1994), there is a trend towards older residents being more ready to commit to their SR choice, suggesting that age may be a salient factor in our system where trainees are younger compared to their America’s counterparts.

Taken together, our results support the recent shift in career (subspecialty) counselling away from a once-off objective cognitive problem-solving process, towards a developmental process where the mentor/counsellor interacts to discover the resident’s work personality, define the work competencies, identify information deficits, as well as recognize anxieties and help deal with emotions that may confound the career decision making (Herr, 1997). In this regard, the STF provides a useful over-arching framework to guide subspecialty career counselling in the current era of residency-based training where internal medicine residents enter training at a younger age. It provides a comprehensive framework that incorporates the influence of individual (readiness, identity, barriers, indecisiveness, information and self-doubt) and contextual (programme characteristics, subject matter and lifestyle) factors on career decision-making. By recognizing the recursiveness of these factors with internal and external influences over time, the STF also helps facilitate this circular feedback loop in the longitudinal discussions with residents by allowing constant interaction and review of how their choice of senior residency may change during course of residency training.

Our study had several limitations. Firstly, our sample size was small and limited to only one out of the three residency programmes available in Singapore. Further studies are needed to ascertain if our findings are generalizable to other internal medicine programmes. Secondly, because the identities of residents were not tagged due to consent issues, our study could only capture the cohort effect as opposed to individual changes over time. In spite of that, our study provided interesting and novel insights that pave the way for future studies to employ the STF to further examine subspecialty choice for other senior residency programmes beyond internal medicine.

In summary, our study demonstrated that the factors that affect a resident’s choice of senior residency change over the years. As they mature, individual factors appear to exert a greater influence on their readiness to commit rather than contextual factors. Our pilot study suggests that the STF framework is a useful tool for career counselling for our residents that ought to start as early as year 1 of residency. Future studies should include a bigger sample size with tagged follow-ups to validate the use and effectiveness of the STF framework amongst our residents in Singapore.

Notes on Contributors

Joanne Kua is a consultant geriatrician in Tan Tock Seng Hospital, Singapore with a deep interest in medical education.

Mark Chan is a senior consultant geriatrician and Head of Department of the Department of Geriatric Medicine, Tan Tock Seng Hospital, Singapore.

Jolene See is a resident physician with the Department of Geriatric Medicine, Tan Tock Seng Hospital, Singapore.

David Ng is a senior resident in Geriatric Medicine with Tan Tock Seng Hospital, Singapore.

Lim Wee Shiong is a senior consultant geriatrician with Tan Tock Seng Hospital, Singapore and also an Adjunct Assistant Professor with the Yong Loo Lin School of Medicine.

Ethical Approval

Approval was obtained from the institutional review board of the National Healthcare Group.

Acknowledgements

We would like to acknowledge the NHG- AHPL residency programme for their help in making this study possible.

Declaration of Interest

Authors have no conflicts of interest, including no financial, consultant, institutional and other relationships that might lead to bias.

References

Aagaard, E. M., Dedier, J., & Julian, K. (2005). Factors affecting medical student’s selection of an internal medicine residency programme. Journal of the National Medical Association, 97(9), 1264- 1270.

Alexander, D. E., Osipow, S. H., & Savickas, M. L. (1985).  Measuring Specialty Indecision among career decided students. Journal of Vocational Behaviour, 27, 356-367.

Chase, G. A., Levine, D. M., & Weisman, C. S. (1980). Male and female career patterns: Specialty choices and graduate training. Journal of Medical Education, 55, 813-825.

Chen, C. (2003). Integrating perspectives in career development theory and practice. The Career Development Quarterly, 51, 203-216.

Chibnall, J. T., Enoch, L., & Schindler, D. L. (2013). Association of medical student burnout with residency specialty choice. Medical Education, 47, 173-181.

Chou, H. Y., Hecker, R., & Martin, A. (2012). Predicting nurses well-being from job demands and resources: a cross-sectional study of emotional labour. Journal of Nursing Management, 20, 502-11.

Dorsey, E. R., Jarjoura, D., & Rutecki, G. W. (2003). Influence of controllable lifestyle on recent trends in specialty choice by US medical students. Journal of American Medical Association, 209(9), 1173- 1178.

Endler, P. C., Fuchs, S., & Messenholl-Strehler, E. (2011). Physician burnout in General Practitioners: Reflections upon prevention and treatment. International Journal of Communication, 5(2), 53-60.

Endler, P., Fischer, M., & Mitsche, M. (2013). Burnout in physiotherapists: Use of clinical supervision and desire for emotional closeness of distance to clients. International Journal of Therapy and Rehabilitation, 20(11), 550- 558.

Gillespie, D. F., Kalliath, T. J., & O’Driscoll, M.P. (2000). A test of the Maslach Burnout Inventory in three samples of healthcare professionals. Work and Stress, 14, 35-50.

Herr, E. L. (1997). Perspectives on career guidance and counseling in the 21^st century. International Journal for Educational and vocational guidance, 50, 1-15.

Holden, W. D., & Levit, E. J. (1978). Migration of physicians from one specialty to another. Journal of American Medical Association, 239, 226-231.

Huggan, P. J., Samarasekara, D. D., & Ooi, S. B. S. (2012). The successful, rapid transition to a new model of graduate medical education in Singapore. Academic Medicine, 87, 1268-1273.

Jackson, J. R., Nuthalapaty, & F. S., Owen, J. (2004). The influence of quality of life, academic and workplace factors on residency programme selection. Academic Medicine, 79(5), 417-425.

Kontosh, L. G., & Zimmerman, A. L. (2007). A systems theory approach to career decision-making. Work, 29, 287-293.

Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory (Third Edition). Palo Alto, California: Consulting Psychologists Press.

Osipow, S. H. (1999). Assessing career indecision. Journal of Vocational Behaviour, 55, 147-154.

Patton, W., & McMahon, M. (1999). Career development and systems theory: A new relationship. Pacific Grove, CA: Brooks/ Cole.

Patton, W., & McMahon, M. (2006). The systems theory framework of career development and counselling: connecting theory and practice. International Journal for the Advancement of Counselling, 28(2), 153-166.

Patton, W., & McMahon, M. (2014). Career development and systems theory: Connecting theory and practice. Holland. Rotterdam: Sense Publishers.

Redman, S., Saltman, D., & Straton, J. (1994). Determinants of career choices among women and men medical students and interns. Medical Education, 28, 361-371.

Richard, G. V., Savickas, M. L., Early, L., Calli, J., Englert, C. A. & Bono, J. (2007). Manual for the Specialty indecision Scale. District of Columbia, Washington, D.C.: Association of American Medical Colleges.

Samarasekara, D. D., Ooi, S., & Hooi, S. C. (2015). Medical education in Singapore. Medical Teacher, 37, 707-713.

*Kua Ee Chia Joanne
Address: 11 Jalan Tan Tock Seng Department of Geriatric Medicine Annex 2 Level 3
Tel: 63596474
Email: joanne_kua@ttsh.com.sg

Published online: 4 September, TAPS 2018, 3(3), 24-30
DOI: https://doi.org/10.29060/TAPS.2018-3-3/OA1055

Ong Lynn¹, Chua Khoon Han¹, Soh Jian Yi² & Aw Marion Margaret Hui Yong²

¹Yong Loo Lin School of Medicine, National University of Singapore; ²Department of Paediatrics, Khoo Teck Puat-National University Children’s Medical Institute, National University Health System

Abstract

Background: Teddy Bear Hospital (TBH) was initiated to address children’s fear through role-play. We aim to assess effectiveness of Singapore TBH sessions in decreasing childhood anxiety and enhancing their healthcare knowledge.

Methodology: We performed a cross sectional descriptive study of children (5-8 years) participating in TBH between March and May 2016. Each child completed two multiple-choice questionnaires with pictorial aids. The first survey assessed baseline knowledge and feelings towards healthcare, whilst the second survey assessed the same knowledge and how feelings towards healthcare may have changed after attending TBH.

Results: Data from 334 completed surveys were collected. We excluded 82 children with incomplete data, leaving data from 252 children for analysis; 96 pre-school (38.1%) and 156 primary school (61.9%).

Most children did not have negative feelings towards visiting the doctor or hospital before TBH. Children with positive feelings towards visiting the doctor and hospital increased from 82.5% to 94.4% (p-value <0.001) and 70.2% to 73.4% (p-value 0.035) respectively.

After TBH, majority of children (57.9%) improved scores in knowledge-related questions. Children who answered all knowledge-related questions correctly increased from 81 (32.1%) to 185 (73.4%)  Children in primary school (OR 3.08, 95% CI 1.66 – 5.70) and those who wanted to visit the doctor (OR 3.38, 95% CI 1.08 – 10.55) were more likely to obtain full marks in knowledge-related questions.

Conclusion: We found that most Singaporean children were positive towards healthcare encounters. A TBH experience was able to further increase this number, as well as increase their healthcare knowledge.

Keywords:            Teddy Bear Hospital, Singapore, Hospital Role-play, Childhood Anxiety, Educational Tool

Practice Highlights

• Young children (aged 5 to 8 years) in Singapore have positive feelings towards healthcare encounters.
• TBH is an effective tool in reducing childhood anxiety towards healthcare encounters.
• TBH is an effective educational tool for children to acquire healthcare knowledge.

I. INTRODUCTION

Childhood play is important in normal psychological and socio-emotional skills development (Lewis, 1993; A., Chinekesh, Kamalian, Eltemasi, S., Chinekesh, & Alavi, 2013). Situational play gives children an opportunity for emotional insight and expression of their inner-feelings (Moustakas, 1951), allowing children to develop adaptive coping skills in social environments (Stone & Stark, 2013).

Play therapy is increasingly becoming the platform for assessment and intervention in childhood (Jäger, 2013; Reddy & Hirisave, 2014). In particular, care in hospitalised children also extends to therapeutic play to decrease negative emotions related to hospitalisation (Koukourikos, Tzeh, Pantelidou & Tsaloglidou, 2015). For example, a study of 60 children, aged 3-8 years, who underwent play therapy involving toys were found to have decreased anxiety before their surgery (Ghabeli, Moheb & Nasab, 2014).

Teddy Bear Hospital (TBH) is a project initiated to address children’s fear through hospital role-play, as well as provide medical students with the platform to interact with children in a pseudo-medical context (Kaufman, Modak & Moylan, 2012). Started off as a community project aimed to enhance education of unexpected admissions to reduce fears about hospitals in America (Santen & Feldman, 1994), case-control studies done in different countries such as Israel (Bloch & Toker, 2008), Australia (Kaufman et al., 2012) and Germany (Leonhardt, Margraf-Stiksrud, Badners, Szerencsi & Maier, 2014), has also shown that TBH is effective in reducing children’s anxiety (Chinekesh et al., 2013) about hospitalisation in a simulated environment and enhance their knowledge concerning body, health and disease (Leonhardt et al., 2014).

In Singapore, there are psychological paediatric services such as the Child Life Service of National University Hospital, Singapore which uses therapeutically medical play to prepare children for procedures or surgeries and help them better understand the hospital environment and their medical conditions. However, these services are mostly limited to children with medical conditions seeking healthcare in tertiary institutions such as hospitals. The majority of the population seek healthcare in primary institutions such as polyclinics and private medical practitioners, which the children would be more familiar with.

Since 2013, Singapore medical students have been involved in running TBH sessions for pre-school and primary school children outside of the hospital environment. Medical students will visit different schools which have expressed interest in the TBH session programme. Four concurrent stations will represent different clinical areas in the simulated hospital environment: namely, the family physician clinic, the asthma clinic, the orthopaedics clinic and the operating theatre. Children will learn about various specialties depicted in each station through role-play as young doctors taking care of their teddy bears or stuffed toys. The final station simulated an emergency situation, where the children were taught the important information to convey while calling for an ambulance. Descriptions of each stations are further elaborated in Appendix. The programme is fully run by the medical student volunteers. Parents were not present during the TBH sessions, while school teachers were silent observers.

Although the TBH programme has been ongoing for several years with the aim of reducing childhood anxiety of healthcare environment through role-play, there has been limited international research that have shown results that can supported this aim. Hence, this study aims to assess the effectiveness of TBH in reducing children’s fear of hospital and doctors and its effectiveness in enhancing children’s knowledge of healthcare locally.

II. METHODS

A. Study Design and Data Collection

We performed a cross sectional descriptive study of children taking part in TBH sessions between 28 March and 20 May 2016.  All children aged 5 to 8 years old were recruited from kindergartens and primary schools which TBH sessions were organized. Prior to the TBH sessions and before administering the surveys, descriptions of the TBH session and study were provided to the parents of these children through the schools, and written parental consent was obtained. Exclusion criteria was children without parental consent.

Each child completed two questionnaires containing multiple-choice questions with pictorial aids. The first survey was administered right before the TBH session to assess baseline knowledge and feelings towards healthcare. The second survey was administered immediately after the TBH session to assess knowledge acquired and how their feelings towards healthcare may have changed. All pre- and post-session surveys were matched according to coded-numbers with no child identifiers collected.

We excluded children who had incomplete data, such as missing answers to any of the multiple choice question in the survey. Children who completed the multiple choice component but did not answer the open-ended questions were still included.

B. Definitions of Survey Questions

Questionnaires were modified from previous studies on TBH (Bloch & Toker, 2008; Leonhardt et al., 2014) and developed based on the various stations of the TBH session.

Childhood anxiety towards healthcare services was assessed by questions asking children how they feel when they visit the doctor or hospital from scaled options provided. The options “very happy”, “happy” and “normal” were collapsed into one group defined as “positive feelings”, while “scared” and “very scared” were collapsed into another group defined as “negative feelings”. Willingness of health-seeking behaviour was assessed by children answering “yes” or “no” to the question “When I am sick, I want to see the doctor”.

Impact of TBH in enhancing children’s medical knowledge was assessed using three questions regarding part of the entire content taught during the TBH session. There were four options per question, including the option “don’t know”, which was classified as an “incorrect” answer. Improvement in score after TBH session was defined as “positive impact”, while decrease or no change in score was defined as “no impact”.

We also stratified the responses of participants according to their education level: “preschool” and “primary school”. This was to analyse the potential effect of education level on the attitudes and knowledge of healthcare for children.

C. Statistical Analysis

Data analysis was completed using the Statistical Package for Social Sciences software (version: IBM SPPS Statistics 22.0). Statistical analyses of differences between children’s responses before and after attending TBH were performed using the Chi-squared test for categorical variables. Logistic regression tests were performed to assess associations between education level, baseline feelings towards healthcare and baseline knowledge.

III. RESULTS

A. Study Population and Demographics

There were 411 children who participated in TBH between March and May 2016. Parents of 77 children did not consent for their child to participate in the study. Data from completed surveys of 334 children were collected as described above. We excluded 82 children with incomplete data. In total, data from 252 children were analysed. The children were grouped by their education level, with 96 from pre-school (38.1%) and 156 from primary school (61.9%). The age of the children ranged from 5 to 8 years, with the mean age being 6.37 years. Other demographic data such as gender, past illness and previous medical encounters were not collected for this study.

B. Children’s Feelings towards Healthcare

Before attending TBH, 82.5% and 70.2% children were not afraid to visit the doctor or hospital respectively TBH (Table 1). In addition, 88.5% children indicated they would want to see the doctor when they are sick.  There was a significant association between being not afraid and being willing to see a doctor or visit a hospital when sick OR = 5.03 (CI 2.21 – 11.5, p-value <0.01) and OR = 2.91 (CI 1.33 – 6.39, p-value <0.008) respectively. Children who were willing to visit a doctor were also more likely be willing to visit the hospital (OR = 2.93 [1.50 – 5.71], p-value 0.002).

Education level (pre-school vs primary school) was not significantly associated with any feelings towards healthcare, in terms of wanting to see the doctor (p-value 0.67), being fearful of visiting the doctor (p-value 0.15) or hospital (p-value 0.90).

Children’s feeling of happiness towards visiting the doctor increased from 82.5% to 94.4% (p-value <0.001) when measured on a 5-point scale (1: very happy, 2: happy, 3: normal, 4: scared and 5: very scared), with the median changing from 2 to 1. Children who felt happy to visit the hospital increased from 70.2% to 73.4% (p-value 0.035) when measured on the same 5-point scale (Table 1).

Table 1. Children’s feelings towards visiting the doctor and the hospital on a 5-point scale before and after attending TBH (N = 252)

C. Children’s Healthcare Knowledge

The children’s healthcare knowledge was assessed by three questions (Figure 1). Prior to attending TBH, two-thirds of the children knew that X-rays are taken when an individual falls (65.9%) and the number to call for an ambulance (67.1%), while half of them knew the inhaler puff is used as the treatment for asthma (55.2%).

The number of children who answered each question correctly after attending TBH increased for all three questions (Figure 1). Children who answered all three knowledge-related questions correctly increased from 81 (32.1%) before attending TBH to 185 (73.4%) after attending TBH, with the median value improving from 2 to 3 (p-value <0.001). Overall, healthcare knowledge scores increased in 146 children (57.9%) after attending TBH.

Note: Asthma question: “When I have asthma, the doctor will give me the ______.” X-ray question: “If I fall down, the doctor will ______.”. 995 question: “I need to call the ambulance! The number I call is _______.”

Figure 1. Percentage of children answering each healthcare question correctly before & after TBH (N=252)

D. Factors associated with Children’s Healthcare Knowledge scores

School age of child (p-value <0.001) and wanting to visit the doctor when one is sick (p-value 0.024) were significantly associated with better healthcare knowledge before attending TBH. Children in primary school were three times more likely to obtain full marks in healthcare knowledge questions compared to those in kindergarten (OR 3.08, 95% CI 1.66 – 5.70). Children who wanted to visit the doctor when they were sick were three more times likely to obtain full marks in healthcare knowledge questions as compared to those who did not want to visit doctor (OR 3.38, 95% CI 1.08 – 10.6). These two variables were no longer found to be associated with better healthcare knowledge scores after attending TBH.

E. Children’s Thoughts on Teddy Bear Hospital

Of 252 children, 246 (97.6%) responded “yes” to the statement “I like Teddy Bear Hospital”. Reasons quoted in the free response answers included comments relating to a positive experience (these included “fun”, “nice”, “interesting” or “enjoyable”) (52.8%), knowledge acquisition (“able to learn something new”)(10.3%) and the use of teddy bears or toys during the session (8.3%).

IV. DISCUSSION

To our knowledge, this is the first local study in Singapore that evaluates the impact of TBH on children participants. We observed that majority of children had increased healthcare knowledge after attending TBH. This was in keeping with another German case-control study which examined 131 children from 14 kindergartens (Leonhardt et al., 2014).

We also noted that majority of children felt happy to visit the doctor or the hospital before attending TBH. Despite being relatively high, this number increased further after the TBH session. Our findings were consistent with an Israeli case-control study that recruited 41 preschool children aged 3 to 6.5 years, and 50 age-matched children as controls, which reported that children in the TBH group have significantly lower levels of anxiety than the control group at follow-up (Bloch & Toker, 2008).

From the study results, it appears that while most of the children in Singapore felt happy instead of anxious while visiting the doctor or hospital, TBH is effective locally in further helping to reduce the level of anxiety for children who had taken part in a session. In Singapore, TBH is also effective in enhancing healthcare knowledge of children who took part in a session. These findings are in keeping with the original hypothesis that TBH is effective in both enhancing children knowledge of healthcare and reducing childhood anxiety of doctors and hospitals in Singapore. From this study, we can gather that simulation and role playing as done with TBH in Singapore could potentially be an effective educational tool for teaching children.

There are several limitations to our study. There is the potential for selection bias as we recruited only those children with parental consent. Hence an assumption would be that children whose parents consented to participate would be representative of the whole cohort. Another potential limitation is reporting error. Some of the younger children (mainly pre-schoolers, aged between 5 to 6 years) may have difficulty understanding the questions. We attempted to overcome this by providing pictorial aids which would help these younger children better understand the questions. While there are studies looking at the effect of play therapy (Ghabeli et al., 2014), there is limited long-term data available that looks at the impact of TBH programme, which limits research which the current study can reference too.

A vital aim of TBH in Singapore was to simulate the hospital environment with teddy bears to explore ways to cope with children fears of doctors and hospitals in a controlled setting, which our study has suggested the potential to do so. Role playing with Teddy Bears also can assist us in educating children with the necessary healthcare knowledge to understand common scenarios they might encounter when they visit the doctor hospitals in future. However, while there has been increased interest from schools, organisations and even individuals to take part in TBH programme, the outreach of this community project has been limited by the manpower available as all of the medical students organizing the programme are volunteers.

In the future, more extensive research could be targeted at exploring the exact phobias of children in these situations so that the TBH program locally can be further modified to target these areas to reduce fear and anxiety of children more effectively. It would also be interesting to explore the long-term impact of the TBH programme, to find out whether children who has taken part in TBH really do experience reduced anxiety during their next hospital visit as compared to those who did not participate, which unfortunately we did not have the opportunity to explore in this study. Another potential extension of the current study is to assess the impact that the TBH programme has on children with medical conditions as compared to healthy children, as the former group may have been exposed to prior hospitalization or healthcare situations which were fearful and initially unfamiliar.

V. Conclusion

In conclusion, TBH is an effective educational tool for children to acquire healthcare knowledge as well as reduce their anxiety associated with healthcare encounters. Children in Singapore are in general positive and not fearful of healthcare encounters.

Notes on Contributors

Ong Lynn graduated from Yong Loo Lin School of Medicine, National University of Singapore. She was involved in the study design, data collection, analysis and formal report writing in this research as a final year undergraduate student.

Chua Khoon Han graduated from Yong Loo Lin School of Medicine, National University of Singapore. He was involved in the study design, data collection and formal report writing in this research as a final year undergraduate student.

Soh Jian Yi is a Consultant at the University Children’s Medical Institute, National University Health System. His research interests lies in the fields of allergy and immunology, particularly focusing on food allergy and desensitisation. He also contributes articles regularly to the Straits Times: Mind Your Body on child health.

Marion M. Aw is an Associate Professor with Department of Paediatrics, National University of Singapore and a Senior Consultant at the University Children’s Medical Institute, National University Health System.  She has a keen interest in medical education, and is involved in both Paediatric Undergraduate and Postgraduate training.

Ethical Approval

Ethics approval for the study (NUS-2731) was obtained from the Institutional Review Board of National University of Singapore.

Acknowledgements

The research team extends our greatest gratitude to the following groups of people:

• The 2016 Teddy Bear Hospital Executive Committee for organised Teddy Bear Hospital sessions during March to May 2016 and conducted data collection for the study;
• Medical students from Yong Loo Lin School of Medicine, National University of Singapore and Lee Kong Chian School of Medicine, Nanyang Technological University who volunteered to guide children participants in the role-play during Teddy Bear Hospital sessions; and
• Department of Paediatrics, National University Hospital for contributing financial support for the purchase of equipment required to organise Teddy Bear Hospital sessions.

Declaration of Interest

Marion M. Aw is the clinical mentor for the medical students in charge of the Teddy Bear Hospital, Singapore. Ong Lynn and Chua Khoon Han were both involved in TBH as part of the organising committee during the course of this study.

The remaining author do not have financial, consultant, institutional and other relationships to Teddy Bear Hospital, Singapore that might lead to bias or a conflict of interest.

References

Bloch Y. H., & Toker, A. (2008). Doctor, is my teddy bear okay? The “Teddy Bear Hospital” as a method to reduce children’s fear of hospitalization. Israel Medical Association Journal, 10(8-9), 597-599.

Chinekesh. A, Kamalian. M., Eltemasi, M., Chinekesh, S., & Alavi, M. (2013). The effect of group play therapy on social-emotional skills in pre-school children. Global Journal of Health Sciences, 6(2), 163-167. http://dx.doi.org/10.5539/gjhs.v6n2p163.

Ghabeli, F., Moheb, N., & Nasab, S. D. H. (2014). Effects of toys and preoperative visit on reducing children’s anxiety and their parents’ before surgery and satisfaction with the treatment process. Journal of Caring Sciences, 3(1), 21-28.

Jäger, J. (2013). Facilitating children’s views of therapy: An analysis of the use of play-based techniques to evaluate clinical practice. Clinical Child Psychology and Psychiatry, 18(3), 411-428. http://dx.doi.org/10.1177/1359104512455816.

Kaufman, J., Modak, M., & Moylan, S. (2012) The Teddy Bear Hospital in Australia. Journal of Paediatrics and Child Health, 48(6), 541-542. http://dx.doi.org/10.1111/j.1440-1754.2012.02482.x.

Koukourikos, K., Tzeh, L., Pantelidou, P., & Tsaloglidou, A. (2015). The importance of play during hospitalisation of children. Materia Socio-Medica, 27(6), 438-441. http://dx.doi.org/10.5455/msm.2015.27.438-441.

Leonhardt, C., Margraf-Stiksrud, J., Badners, L., Szerencsi, A., & Maier, R. F. (2014). Does the ‘Teddy Bear Hospital’ enhance preschool children’s knowledge? A pilot study with a pre/post-case control design in Germany. Journal of Health Psychology, 19(10), 1250-1260. https://doi.org/10.1177/1359105313488975.

Lewis, J. M. (1993). Childhood play in normality, pathology, and therapy. American Journal of Orthopsychiatry, 63(1), 6-15.

Moustakas, C. E. (1951) Situational play therapy with normal children. Journal of Consulting Psychology, 15(3), 225-230. http://dx.doi.org/10.1037/h0056804.

Reddy, R. P., & Hirisave, U. (2014). Child’s Play: Therapist’s Narrative. Indian Journal of Psychology Medicine, 36(2), 204-207. http://dx.doi.org/10.4103/0253-7176.130998.

Santen, L., & Feldman, T. (1994). Teddy bear clinics: a huge community project. MCN: The American Journal of Maternal/ Child Nursing, 19(2), 102-106.

Stone, S., & Stark, M. (2013). Structured Play Therapy Groups for Preschoolers: Facilitating the emergence of social competence. International Journal of Group Psychotherapy, 63(1), 25-50. http://dx.doi.org/10.1521/ijgp.2013.63.1.25.

*Ong Lynn
Email: ong.lynn@u.nus.edu
Mobile: +65 8339 8617

Published online: 4 September, TAPS 2018, 3(3), 17-23
DOI: https://doi.org/10.29060/TAPS.2018-3-3/OA1069

Pilane Liyanage Ariyananda

School of Medicine, International Medical University, Malaysia

Abstract

Keywords:            Empathy, Palliative Care, Medical Students, Reflective Reports

Practice Highlights

• Clinical empathy improves healthcare.
• Training medical students and doctors to be empathetic is achievable.
• Exposure of medical students to settings where palliative care is given can be considered a window of opportunity to invoke empathy.
• Reflective reports following visits to settings where palliative care is given can be a useful tool to detect empathetic encounters.

I. INTRODUCTION

Ability to express empathy and sympathy are good qualities that all humans should possess. Empathy is the ability to share someone else’s feelings or experiences by imagining what it would be like to be in their situation, whereas sympathy is understanding and care for someone else’s suffering (McIntosh, 2015). The ability to express empathy is a reflection of emotional intelligence (Goleman, 1995). These qualities should be developed to a high level in care givers like healthcare personnel. During an era where science and technology seem to take precedence in medical care, paying more attention to empathy would help to redress the imbalance (Bleakley, 2005). Studies have shown that empathy helps to develop better therapeutic relationships (a close and consistent bonding that develops between the client and the health care provider) leading to significant outcomes for both psychotherapies and for active and placebo pharmacotherapy (Krupnick et al., 1996). Development of better therapeutic relationships lead to enhancement of physician satisfaction as well (Haas et al., 2000).

There is also evidence to show that training and retraining of physicians are beneficial in enhancing their empathetic behaviour (Bonvicini et al., 2009). Training has also shown to improve competency and expression of empathy and communication, leading to improvement in patient satisfaction (Schrooten & de Jong, 2017). Being empathetic helps to develop better insights to human experience of illness, disability and medical interventions (Gordon, 2014). Therefore, it is desirable to focus attention on ways of developing empathy in medical students.

Empathy plays an important role when looking after patients who are under palliative care. Such patients need the devotion and attention of the care-giver more than the disease of patients. Most patients who are under palliative care do not have complex situations such as those warranting extensive investigations or intensive care and they can be looked after by the usual care-giver such the family physician, without being referred to a dedicated palliative care team (Watson, Lucas, Hoy & Wells, 2009). Therefore, training in palliative care is important for healthcare personnel and the impact of such training improve if included in the undergraduate curriculum (Billings, 1997).

The study was undertaken in the International Medical University (IMU) of Malaysia (Student Information, Regulation & Code of Conduct, 2016). Although the University is a private institution, students have their clinical training in state hospitals where they see a mix of patients from three ethnic groups (Malay, Chinese and Indian). Students communicate with their patients in any of the four languages – Bahasa Malaysia, Mandarin, Tamil or English. IMU students are from a higher and middle income strata with a narrow social and cultural background. In addition, they have received education in schools in the capital or in bigger towns in Malaysia. Therefore, they are likely to have limited experience of values, attitudes, needs and concerns of the less affluent general public.

In the IMU, medical students are not timetabled to undergo dedicated training in clinical empathy. However, sensitization to some concepts of empathy during sessions on ‘Professionalism and Ethics’ is expected to occur within the formal curriculum. During evaluations, such as Mini – CEX, clinical long case or OSCE, little attention gets paid to evaluation of empathetic behaviour, except when the student’s conduct is unprofessional. Generally, in most medical schools, focus on teaching and assessment of medical sciences is given greater priority than on issues dealing with social sciences (Gordon, 2014). A SWOT (Strengths – Weaknesses – Opportunities – Threats) analysis done in the IMU in 2017 showed that the curriculum of most of the programs, including Medicine focused the students’ attention on skills, and not on the holistic care of patients (Babar et al., 2017). IMU medical students are taught to write reflective reports from the first week at the medical school, and their reflective reports are assessed during formative assessments in the Pre-clinical Phase in Semesters 1, 2 & 5 (Sow, 2010). In the Clinical Phase, reflective writing is revisited by the Posting Coordinator, a few weeks prior to their hospice visit in Semester 9 (Final Year), in the Internal Medicine Posting. Medical students are expected to acquire empathetic behaviours through role modelling of good professional conduct of their teachers, mentors, seniors and all healthcare personnel in their clinical working environment (Cruess, 2008). During bedside teaching sessions and ward rounds, attending physicians may elaborate on special ethical and professional aspects regarding patients under consideration, thereby ‘leading by example’. At the same time, episodes of unprofessional conduct of students are pointed out to student/s concerned with attention paid to the sensitive nature of the issues. In addition, exemplary professional behaviour of students is rewarded openly for the benefit of their peers. Some of these encounters may centre on situations requiring empathetic behaviour by healthcare personnel including students (Cruess, 2008). The teachers are expected to use such opportunistic situations to inculcate empathy in students.

Visiting the Hospice and patients under palliative care at home is considered a situation where there is ample room for invoking sympathy and empathy. Taking this background into consideration, the author undertook an analytical study of reflective reports written by IMU final year medical students after their visits to the Hospice and homes of patients under palliative care to find out whether they were able to express any empathetic behaviour.

II. METHODS

During their 6-week posting in Internal Medicine in the 9th semester (final year), students are expected to pay a visit to the hospice in Seremban, Malaysia, and also visit patients in the neighbourhood who are under palliative care, on the same day. These visits were supervised by doctors and nurses from the hospice. Following these visits, they were expected to write a reflective report of about 500 words, in English, regarding an interaction with patients they had seen, and submit the reports to their mentors for scrutiny. As a convenient sampling pilot study, an analysis of 58 consecutive reflective reports by the author’s mentees were reviewed during the period May 2013 to November 2016. In addition to approval from the Institutional Review Board of the IMU (IMU-Joint Committee), permission from students was obtained to perform an analysis of their anonymised narratives.

Table 1. Components of empathy from Morse et al. (1992)

As the study is a qualitative study, a discourse analysis of narrations given in reflective reports was done to identify four themes that will fall into four components of empathy described by Morse et al. (1992) who have pointed out, that the empathetic process can be divided into four components: emotive, moral, cognitive and behavioural. Student reflective reports were analysed to find evidence of reflection that would correspond with any of the categories mentioned by Morse et al. (1992) as shown in table 1 above.

III. RESULTS

Although all 58 students sympathized with the plight of their patients, only 12 of them expressed empathy in their reflective reports. Two examples of expression of sympathy are noted here: “It saddens me to see such a thin and cachectic lady who is bedbound and just counting her seconds away. … I did my best in providing her some reassurance and help her to a more comfortable position in bed” (Reflective report #14). Another student comment: ‘Due to lung involvement, she was breathless, all the time requiring oxygen. The bone metastases also make her writhe in pain, but syrup morphine given every 4 hours has made the symptoms bearable. Although she does not look depressed, she seems to be struggling with all these symptoms. I feel that the visit from Hospice helped her to voice out her concerns, and as appropriate, those will be addressed. This would provide the support that she needs’ (#25). These comments showed that students were emotionally detached from the mental suffering patients with terminal cancer were enduring, with attention being given only to their physical discomforts. These reflections show that students understand patient needs and express sympathy by addressing their needs, though falling short of empathizing as students’ minds do not seem relive the experiences of their patients.

The instances of expression of empathy in these 12 reports (20%) captured under the four components (emotive, moral, cognitive & behavioural) described by Morse et al. (1992) are given below in table 2. Some students expressed more than one component of empathy. Narrations expressing empathy are discussed in the next section.

Table 2. Empathetic responses of students based on components of empathy. Morse et al. (1992). (n = 58)

IV. DISCUSSION

One important reason as to why the majority of students refrained from reporting about empathetic encounters could be that they had resorted to distancing tactics. It has been shown that many doctors do not get close to their patients’ psychological suffering to ensure their own emotional survival (Maguire, 1985).

Examples of reflections extracted to fit each of the above components of empathy from four different reflective reports are given here:

Emotive: “During my hospice visit, I met a cancer patient and during the conversation, I realized that the patient has accepted the fact that this is what life has to offer her and that’s it!  I learnt that a moment of blankness flashes in front of you at the instance of knowing that you have an incurable illness and you are dying. It does not hit you until it really hits you!!” (#55). The above remarks by the student indicate that he understands the plight of the patient who is suffering from cancer and he attempts to share the intrinsic feeling of his patient.

Moral: “Empathy comes from the understanding of the patient which involves a lot of communication with the patient…. These cannot be mastered through textbooks. We need to cure patients… Hear what they hear, see what they see – this offers a better outcome in our management plan” (#18). The student seems to be intrinsically motivated to help the patient as he realizes the importance of better communication in comforting and healing his patient.

Cognitive: ” …patient’s wife as a caretaker was going through the thick and thin with her loved one. It has once again enlightened me that beyond treating the patient physically, we need to understand about the pain and social issues they are going through” (#11). Here, the student is weighing pros and cons from patient’s perspective about suffering that the wife of the patient is going through due the husband’s illness, before reacting to the situation.

Behaviour: “I felt unhappy with the doctor as I felt the way he said it was harsh, and the doctor was not being empathetic with what the family members have to go through. As I was consoling a family member, he told me that what the doctor said has killed his hopes and he was very hurt” (#62). Here, the student is delivering a communicative response to console them after understanding the bad experience that the family has gone through.

The reflective reports that were analysed seem to indicate that there were opportunities to express empathy for students visiting the hospice and the homes of patients who were on palliative care. Although it is hard to observe all student-patient interactions specifically, for explicit empathy, an attempt had been made in this study pick-up whatever they had reflected as empathetic experiences and expressed in writing.

None of the reports had a composite of all four components described by Morse et al. Few reports had two or three components. Although most students did not express a note of empathy in their reports, it does not necessarily imply that they were not empathetic during their interactions with patients as they may not have documented all their feelings. Paucity of documentation of empathetic encounters could have occurred because students themselves may not have been aware that they had gone through such an experience. Due to lack of fluency in writing skills, some students may have confined writing about empathetic encounters to one or two components instead of four, or not written at all.

Failure to include all four components described by Morse et al. does not necessarily mean that there was no empathy. At the same time, one has to bear in mind that not all doctor-patient interactions necessarily involve empathy. It is possible that most students engaged with their patients for a shorter period, thereby giving lesser opportunities for deeper engagement and fewer empathetic opportunities. Furthermore, paucity of documentation of empathetic encounters could be due to lack of fluency in expressing such situations within the word limit of 500 imposed on them.

Often, non-verbal communications manifest as empathy during interactions with patients and these would not get noticed unless such encounters were observed directly or through video recordings. Therefore, observing medical students during such interactions is important to be able to judge whether they actually engaged with patients, empathetically. Although direct observation of students would have been a better way to ascertain such engagement, there are some issues that we need to consider. Firstly, whether there was sufficient trained faculty to oversee them? Secondly, whether observing them too closely could have hampered their rapport building and closer engagement with patients? Thirdly, part of the cognitive component of empathy may not be observable as the thinking process is not always explicit.

Despite these constraints, reflective writing has many advantages as a learning tool. These include, the creation of opportunities to make permanent records of some events that may be forgotten, opening opportunities to prioritise and identify what was important to commit oneself to learning and to take action, helping to breakdown incidents to components to facilitate analysis, helping to take a step back and analyse and evaluate the incident in a balanced manner by considering different perspectives and finally, enabling to share thoughts and feelings (Jennifer, 2006).

Does reflection improve clinical behaviour? There is no evidence to answer this question, because there are no comparative studies between reflective and non-reflective practitioners with respect to the ways they conduct in the clinical environment.  However, reflection and development of self-awareness can pave way to improving clinical practice and guide educational efforts. This is a generalization of concept of reflective thinking as described by Gibbs in his six-stage reflective model which shows how better understanding and deep learning can take place by reflection (Gibbs, 2013). Does reflection improve patient care?  Here again, there is no comparative data to answer the question. However, awareness of uncertainties and reflections about such situations has the potential to improve patient care.

Will reflective writing on patients who are on palliative care enhance the development of empathy in medical students? Although there is presumption in the literature that reflection enhances competence, there is no literature to support or refute it (Mann, Gordon & MacLeod, 2009). Despite absence of such evidence we hope that students improve their clinical behaviour and clinical practice by their empathetic engagement.

Paucity of reporting about empathetic engagement in students’ reflective reports can also be due to various factors such as lack of fluency in writing style, limited information confined to what student wanted to make explicit to the mentor, student’s emotional state at the time of writing and the purpose of the report as well as whether it was written for an academic purpose or not (Jennifer, 2006). Would rewards such as awarding marks for summative assessment for reflective care reports make a difference in the manner and the extent to which students would express empathy in their reports? A reward is likely to create a bias in report writing and reporting of empathetic feelings can be overplayed. It is easier to change attitudes than behaviour. Although, enforcement of reflective report writing may make students more mature and humanistic, students who already possess these qualities are more likely to benefit (Khan, 2008). Students may also find it difficult be empathetic in certain situations due limited real-life experiences thereby making it more challenging to get into ‘patient’s own boots’. However, as they mature and practice as doctors, they can be expected to nurture empathetic behaviour in a habitual manner.

If students were allowed to share their experience following their visit to hospice and homes of patients on palliative care would it nurture the development of empathy? There is no evidence to show that empathy can be further developed by such means. However, when nurses shared their experiences among themselves by storytelling it was shown to promote better understanding and insight in nursing practice (Day, 2012).

Currently, IMU students are scheduled to visit the Hospice in only their final year. Would it be more beneficial to give them an earlier exposure? Patient-centred listening is essential to end-of-life communication and this skill should be emphasized earlier in medical education (Miranda, 2017).  Our students are scheduled to visit the Hospice and patients under palliative care for only one-day in their training.  Should they be given opportunities for more visits and more time to talk to their patients to help deepen understanding and closer engagement? Should students see more patients for the sake of varied experience? Early exposure to varied experiences would allow more opportunities for deep learning by reflection (Kolb, 1984). Student are also likely to enhance empathetic skills if the emphasis during training was around patient-centred clinical encounters (Branch et al., 2001). Successful learning on how to reflect can be better achieved by facilitating the context, safe atmosphere, good mentorship and supervision, peer support and providing time to reflect (Mann et al., 2009). Therefore, feedback from mentors is likely to benefit students by improving skills of reflection. Although skills of reflection can improve with training, it is hard to predict as to when it would change attitudes and behavior. It is noteworthy that English language proficiency as assessed by standardized tests alone were a poor indicator for judging medical communication skills (Chur-Hansen, Vernon-Roberts & Clark, 1997) and this observation may be useful in interpreting reflective reports from a group of medical students with varying competencies in English.

Although this study utilized components of empathy as detailed by Morse et al., several other tools have been cited in the literature to find out evidence of clinical empathy. Some of these are based on video recording of doctor-patient encounters and others are based on questionnaires (Hojat, Axelrod, Spandorfer & Mangione, 2013). Questionnaire-based studies were more likely to find out students’ orientation towards being empathetic rather than empathetic communication (Bylund & Makoul, 2005).

Lack of triangulation seems to be a limitation in this study.  Use of another analytic process such ‘focus group discussion’ could have increased the validity of findings. Whether an element of subjectivity and bias may have occurred as the study was done only by one person is debatable. Greenhalgh (2010) argues that ‘Since I wrote the first edition of this book, inter-rater variability has become less credible as a measure of quality in qualitative research. Appraisers of qualitative research increasing seek to assess the competence and reflexivity of a single researcher than confirm that the findings were checked by someone else (p. 166)’. As discussed earlier, other limitation of the study would include, failure to pen-down all empathetic encounters due to factors such as lack of fluency in written communication skills, word limit on reflective essays, limited time for student-patient interactions and failure to realize that empathetic encounters have actually taken place.

V. CONCLUSION

Building empathy in healthcare is known to improve the quality of care. Training doctors to be empathetic is achievable. Although all students in this study had expressed sympathy, it would have been desirable to see more students expressing empathy. The study showed that reflective reports following visits to the hospice and home visits to patients who are under palliative care were useful documents to find out whether students expressed empathy during their encounters with these patients. Despite limitations with regards to the ability to capture empathetic opportunities through reflective reports due factors such as ‘word limit in essays’, lack of fluency in written communication skills, lack of realization that empathy has actually being invoked and forgetting to pen-down empathetic encounters; the exercise of going through these reports seems to offer a ‘window of opportunity’ to see whether students express empathy. The potential of reflective reports to find out whether medical students invoke empathy during encounters with patients under palliative care needs to be explored further.

Notes on Contributors

Professor Pilane Liyanage Ariyananda contributed to the conception and design of the study, as well as to the acquisition, analysis and interpretation of data, and preparation of the paper. The author is a specialist in Internal Medicine with wide clinical and academic experience and has been a mentor and teacher to medical undergraduates and postgraduates over 35 years.

Ethical Approval

Permission was obtained from the students and the Institutional Review Board (Project ID No.: IMU 383/2017) to analyse and publish narratives after anonymization.

Acknowledgements

I am grateful to IMU of Malaysia for permitting me to analyse student reflective reports and to Professor Sivalingam Nalliah for his insightful comments on the manuscript. The study was not funded.

Declaration of Interest

The author has no competing interests.

References

Babar, M. G., Hasan S, S., Akram, S., Burud, I. A. S., Khalaf, Z. F., Reckelhoff, K. E., … Nalliah, S. (2017). Living with empathy in the 21^st century. In Lim, V. K. E., Lim, P. K. C., & Mak, J-W. (Eds.), Embracing Tomorrow’s Challenges – Continuing IMU’s Journey (pp. 167-212). Kuala Lumpur, Malaysia: International Medical University.

Billings, J. A., Block, S. (1997). Palliative care in undergraduate medical education. Status report and future directions. Journal of the American Medical Association, 278(9), 733-8. https://doi.org/10.1001/jama.1997.03550090057033.

Bleakley, A. (2005). Stories as data, data as stories: making sense of narrative inquiry in clinical education. Medical Education, 39, 534–40. https://doi.org/10.1111/j.1365-2929.2005.02126.x.

Bonvicini, K. A., Perlin, M. J., Bylund, C. L., Carroll, G., Rouse, R. A., & Goldstein, M. G. (2009). Impact of communication training on physician expression of empathy in patient encounters. Patient Education and Counselling, 75(1), 3–10. https://doi.org/10.1016/j.pec.2008.09.007.

Branch, W. T., Kern, D., Haidet, P., Weissmann, P., Gracey, C. F., Mitchell, G., & Inui, T. (2001). Teaching the human dimensions of care in clinical settings. Journal of the American Medical Association, 286(9), 1067-1074. https://doi.org/10.1001/jama.286.9.1067.

Bylund, C. L., & Makoul, G. (2005). Examining Empathy in Medical Encounters: An Observational Study Using the Empathic Communication Coding System. Health Communication, 18(2), 123–140. https://doi.org/10.1207/s15327027hc1802_2.

Chur-Hansen, A., Vernon-Roberts, J., & Clark, S. (1997). Language background, English language proficiency and medical communication skills of medical students. Medical Education, 31(4), 259-63.

Cruess, S. R., Cruess, R. L., & Steinert, Y. (2008). Role modelling – Making the most of a powerful teaching strategy. British Medical Journal, 336, 718-21. https://doi.org/10.1136/bmj.39503.757847.BE.

Day, V. E. (2012). Storytelling: The threads of the nursing tapestry. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses Global. (3502689).

Gibbs, G. (2013). Learning by doing. Oxford, UK: Oxford Centre for Staff and Learning Development, Oxford Brookes University. Retrieved from https://thoughtsmostlyaboutlearning.files.wordpress.com/2015/12/learning-by-doing-graham-gibbs.pdf.

Goleman, D. (1995). Emotional Intelligence. New York, USA: Bantam Books.

Gordon, J. J., & Evans, H. M. (2014). Learning medicines from humanities. In T. Swanwick (2nd Ed.), Understanding Medical Education: Evidence, Theory and Practice (pp. 213-226). England: Wiley Blackwell. Retrieved from
https://books.google.com.sg/books?hl=en&lr=&id=sW9RAgAAQBAJ&oi=fnd&pg=PT19&dq=2014+T.+Swanwick+(2nd+Ed.),+Understanding+Medical+Education:+Evidence,+Theory+and+Practice&ots=Ud8OoXeCPF&sig=1khYvu5rV
NYK4QJ5L-VnL3J40pA#v=onepage&q&f=false”>https://books.google.com.sg/books?hl=en&lr=&idsW9RAgAAQBAJ&
oi=fnd&pg=PT19&dq=2014+T.+Swanwick(2nd+Ed.),+Understanding+Medical+Education:+Evidence,+Theory+and+Pr
actice&ots”>=Ud8OoXeCPF&sig=1khYvu5rVNYK4QJ5L-VnL3J40pA#v=onepage&q&f=false.

Greenhalgh, T. (2010). How to read a paper: The basics of evidence-based medicine. England: Wiley Blackwell BMJ Books.

Haas, J. S., Cook, E. F., Puopolo, A. L., Burstin, H. R., Cleary, P. D. & Brennan, T. A. (2000). Is the Professional Satisfaction of General Internists Associated with Patient Satisfaction? Journal of General Internal Medicine, 15(2), 122–128.

Hojat, M., Axelrod, D., Spandorfer, J., & Mangione, S. (2013). Enhancing and sustaining empathy in medical students. Medical Teacher, 35(12), 996-1001.

International Medical University of Malaysia Student Information, Regulation & Code of Conduct (2016). Retrieved from https://elearn.imu.edu.my/mod/folder/view.php?id=9962.

Jennifer, A. M. (2006). A handbook for reflective practice and professional development (2nd Ed.). United Kingdom: Routledge.

Khan, M. W. (2008). Etiquette-Based Medicine. New England Journal of Medicine, 358(19), 1988-1989.

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, New Jersey: Prentice Hall.

Krupnick, J. L., Sotsky, S. M., Simmens, S., Moyer, J., Elkin, I., Watkins, J., & Pilkonis, P. A. (1996). The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: Findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 64(3), 532-539.

Maguire, P. (1985). Barriers to psychological care of the dying. The British Medical Journal (Clinical research Edition), 291, 1711 – 13. https://doi.org/10.1136/bmj.291.6510.1711.

Mann, K., Gordon, J. & MacLeod, A. (2009). Reflection and reflective practice in health professions education: A systematic review.  Advances in Health Science Education, 14(4), 595–621.

McIntosh, C. (2013). Cambridge Advanced Learner’s Dictionary (4th Ed.). Cambridge, United Kingdom: Cambridge University Press.

Miranda, S. P. (2017). Learning to Drive — Early Exposure to End-of-Life Conversations in Medical Training. The New England Journal of Medicine, 376(5), 413-415.

Morse, J. M., Anderson, G., Bottorff, J. L., Yonge, O., O’Brien, B., Solberg, S. M., & Mcllveen, K. H. (1992). Exploring empathy: A conceptual fit for nursing practice? Image – the Journal of Nursing Scholarship, 24(4), 273-280.

Schrooten, I., & de Jong, M. D. T. (2017). If You Could Read My Mind: The Role of Healthcare Providers’ Empathic and Communicative Competencies in Clients’ Satisfaction with Consultations. Health Communication, 32(1), 111-118. https://doi.org/10.1080/10410236.2015.1110002.

Sow, C. F. (2010). Reflective writing. Retrieved from https://elearn.imu.edu.my/mod/folder/view.php?id=11266.

Watson, M., Lucas, C., Hoy, A. & Wells, J. (2009). Oxford Handbook of Palliative Care (2nd Ed.). Oxford, United Kingdom: Oxford University Press.

*Pilane Liyanage Ariyananda
Clinical Campus, School of Medicine
International Medical University
Jalan Rasah, 70300 Seremban
Negeri Sembilan DK, Malaysia
Tel: +60133850559
Email: ariyananda@imu.edu.my

Published online: 4 September, TAPS 2018, 3(3), 5-16
DOI: https://doi.org/10.29060/TAPS.2018-3-3/OA1054

Marcus A. Henning¹, Christian U. Krägeloh², Roger Booth³, Erin M. Hill⁴, Julie Chen⁵, Craig Webster^1,6

¹Centre for Medical and Health Sciences Education, University of Auckland, New Zealand; ²Department of Psychology, Auckland University of Technology, New Zealand, ³Molecular Medicine & Pathology, Faculty of Medical and Health Sciences, University of Auckland, New Zealand; ⁴Department of Psychology, West Chester University, United States of America; ⁵Department of Family Medicine and Primary Care, Bau Institute of Medical and Health Sciences Education, Li Ka Shing Faculty of Medicine, The University of Hong Kong; ⁶Department of Anaesthesiology, University of Auckland, New Zealand

Abstract

This study highlights the use of multi-factor analytical approaches in the investigation of students’ academic performance and their well-being. The aim of the study was to explore the relationships among physical health, competitiveness, perceived stress, motivation, and academic achievement in pre-medical and health science students. Responses were elicited from 339 students preparing for medical and other health science programmes. Questionnaires obtained information regarding gender, age, grade achievement, perceived stress, motivation, physical health, and competitiveness. Due to the subtle gender differences evident in the literature and substantiated by the findings in this study, two structural equation models were generated, one representing male students and the other female students. In general, the illustrated models showed a positive relationship between physical health and enjoyment of competition. Next, physical health was negatively associated with perceived stress, which showed a negative association with self-efficacy and a positive association with test anxiety. Enjoyment of competition was positively associated with self-efficacy. Grade achievement was positively associated with self-efficacy. The models suggest that students, as a whole, experiencing good physical health report less perceived stress and less test anxiety, and more self-efficacy which are associated with positive grade outcomes. These models give insights into how physical health and learning variables may influence grade outcome providing scope for further research into how these relationships impact learning environments, team behaviours, and professional training.

Keywords:            Biomedical and Health Science Students, New Zealand, Motivation, Physical Health, Competitiveness, Academic Achievement

Practice Highlights

• The derived structural equation models were able to systematically explore the relationships among physical health, competitiveness, perceived stress, motivational beliefs, and consider their impact on grade attainment among first-year pre-medical and health science students.
• The structural equation models indicated that physical health significantly co-varied with enjoyment of competition and that there was a significant negative association with perceived stress.
• Enjoyment of competition formed predictive relationships with self-efficacy and intrinsic value for female students and with only self-efficacy for male students.
• Perceived stress, and the motivational belief variables of self-efficacy, test anxiety, and intrinsic value were identified in this study in relation to the prediction of academic achievement.

I. INTRODUCTION

The learning environment of medical and health sciences students consists of numerous opportunities and challenges. When studying medicine and other health sciences, students are entering a phase in their educational lives that requires them to be self-directed, adult learners adopting evidence-based reasoning (Spencer & Jordan, 1999). However, in this stimulating learning environment challenges emerge, that require them to cope with stress, maintain good physical health, while ensuring that they stay motivated, and achieve good grades (Hojat, Gonnella, Erdmann, & Vogel, 2003). A further important issue that is often ignored is how competitiveness factors may stimulate or impair learning, and this is particularly pertinent in medicine given that medical students are often cited as being highly competitive and aiming for high academic achievement (Hilliard, 1995).

The aim of this study was to examine the interactions between different types of factors linked with learning that are relevant to these first-year biomedical and health science students, many of whom are competing for admission to medical school and other professional programmes. Furthermore, we explored how their perceptions of physical health, perceived stress, motivation and competitiveness could be linked with their academic achievement. This was an exploratory study investigating the relationships between certain key variables targeting learning behaviour, well-being, and grade outcome, with the view to understanding how these variables influence each other to further promote student well-being and academic performance.

A. Physical health

Physical health has been associated with: coping with daily activities, dependence on medicinal substances and medical aids, levels of energy and fatigue, notions of mobility, experiences of pain and discomfort, ability to work, and levels of sleep and rest (World Health Organisation, 1996). One issue often documented in relation to medical students is their problems with maintaining healthy sleep patterns. In their study Samaranayake, Arroll, and Fernando (2014) found that 39% of the medical students that they surveyed reported significant problems with sleep. In addition to sleep issues, Tanaka, Mizuno, Fukuda, Shigihara, and Watanabe (2008) reported that 20% of the medical students surveyed in their study experienced significant levels of fatigue, as measured by the Chalder Fatigue Scale. Of those students who experienced fatigue, Tanaka et al. also found significant issues associated with eating patterns, school attendance, feelings of satisfaction and being attentive to what was taught in the lectures. Eating problems and sedentary behaviours amongst medical students have also been reported elsewhere thus increasing the risk of cardio-vascular problems later in life (Kulkarni, 2016; Rustagi, Taneja, Mishra, & Ingle, 2011). Similar findings on the high prevalence of coronary heart disease risk factors amongst medical students have been further reported (Ibrahim et al., 2014).

B. Perceived stress

Perceived stress is related to how much a person recognizes an incident to be demanding (Cohen, Kamarck, & Mermelstein, 1983).  It has been documented that the experience of undue stress and issues related to the retention of first-year students are common phenomena (Chester, Burton, Xenos, & Elgar, 2013; Friedlander, Reid, Shupak, & Cribbie, 2007). In one study (Park et al., 2012), a feedback loop was indicated when researching the interactions between motivation, grade achievement and stress in medical students. Moreover, Park et al. proposed that a reciprocal relationship occurs between stress and motivation and that motivation is somewhat moderated by both stress and academic performance. One of the practical implications of this study was to develop applicable stress management methods that directly promote motivation. Some of the reasons provided to explain the experience of high levels of stress and issues related to retention include: coping with the transition from high school to university; increased academic demand for self-directed learning approaches; adjusting to a new and larger learning environment; changes in social support systems; fear of failure; fear of anonymity; peer competition; financial strains for students no longer living at home; and transportation changes (Friedlander et al., 2007; Moffat, McConnachie, Ross, & Morrison, 2004).

C. Motivation, competitiveness and gender

In general it can be inferred that medical and health science students are likely to be highly motivated given the demanding admissions requirement for studying and entry into medical school and other professional courses. For example, at the University of Auckland, entry into medicine or other health science degrees is largely contingent upon a high level of academic attainment in their first year of study in the Biomedical Common Year or Overlapping Year 1 programme (The University of Auckland, 2015). Motivation studies in medical education have tended to describe ways of measuring motivation in reference to medical students and trying to understand how medical students learn (Henning et al., 2011; Kusurkar, Ten Cate, Van Asperen, & Croiset, 2011; Lonka et al., 2008; Sobral, 2004). Very few studies have tried to link motivation with academic achievement (Kusurkar, 2012) and of those studies very few have generated convincing arguments regarding such a link (Henning et al., 2011). However, the first-year experience requires cultivation of motivational beliefs, such as self-efficacy, and this is likely to positively influence educational outcomes and academic persistence (Elliott, 2014). Additionally studies in the general educational literature also suggest substantive links between motivation variables, such as self-efficacy, intrinsic value, test anxiety, and academic outcome (Pintrich, 2003; Pintrich & De Groot, 1990; Pintrich & Zusho, 2007).

Less is known about how competition affects academic attainment, although there is a convincing argument linking the experience of competition with stress. For example, there are reports regarding this pre-medical student group in New Zealand as being potentially at risk of lowered life satisfaction and stress-induced difficulties due to the uncertainty of being accepted into medicine and because of the highly competitive environment that precedes acceptance into the medical programme (Samaranayake & Fernando, 2011).

A further, potentially important, demographic factor influencing the experience and expression of the feelings of anxiety, stress and confidence is that of gender. For example, it has been consistently shown that female students are more likely to show high levels of test anxiety and stress, and lower levels of confidence than their male peers (Blanch, Hall, Roter, & Frankel, 2008; Henning et al., 2013; Hojat, Glaser, Xu, Veloski, & Christian, 1999; Papier, Ahmed, Lee, & Wiseman, 2014). In addition, Hibbard and Buhrmester (2010) investigated the two sides of competition, namely competing to win (creating a zero-sum outcome) versus competing to excel, which focuses more on ‘personal development’. In their study, they found that the impact of competitiveness on developing well-being and social relationships was linked to type of competitiveness and gender. In terms of gender, Hibbard and Buhrmester revealed that competing to win created both “internalizing and externalizing problems among females, but mainly related to externalizing problems only among males (p. 420).” In addition they reported that competing to win was linked to dissension in social relationships and problems with developing empathy for females. In contrast, competing to excel, was linked to forming positive social relationships and higher self-esteem for both males and females.

D. Aim of study

The overall aim of the current paper is to explore the relationships among physical health, competitiveness, perceived stress, motivation, and grade attainment in pre-medical and health science students.  More specifically, this was a preliminary study to further explore and understand how these variables may impact each other to better support students with specific interventions, such as developing stress management approaches that promote students’ motivation and physical health when learning to strive in highly competitive courses.

II. METHODS

A. Participants

All students (N=1036) studying in a population health course within the Biomedical Common Year or Overlapping Year 1 programme were invited to participate in this survey. This programme consists of numerous core first-year courses and one general education course, with the aim of preparing students for a Bachelor of Health Science, Bachelor of Pharmacy, Bachelor of Optometry and other programmes (The University of Auckland, 2015). Students aiming for entry into courses such as medicine and other health-related courses attend this programme, and hence the distinction between medical students and health science students has not yet been established and such a distinction eventuates at the end of this year of study.  In addition, to be granted an interview for entry into the Medical programme, students need to attain overall grades of ‘B+’ or greater.

B. Procedure

Students were provided with an email outlining the study and an invitation to participate in the survey by the phase director. Students were informed about the study in late July 2015 and two reminder emails were sent out. The grade for their Population Health paper relates to their course completion in semester 1 (ending June 29, 2015). Therefore, the survey was conducted after course completion and after this grade had been released to students. Anonymity was maintained throughout the process, and the phase director was unaware of who the study participants were. Participants were able to withdraw from the study up to the time of questionnaire submission. They were also able to put their name into a draw to win a $100 supermarket voucher, but their disclosed contact details for the draw were not able to be connected to their responses to the questionnaire. Data were collected online using a Google Forms questionnaire (Google.com, 2015).

C. Measure

1) Demographic and background details

Several measures were obtained from University records: (1) age in years; (2) gender; and (3) academic grade for Population Health (POPLHLTH 111), one of the core courses in the Biomedical Common Year or Overlapping Year 1.

2) Physical Health domain of the World Health Organisation Quality of Life (WHOQOL) Questionnaire

The physical health domain is one of four domains measured by 26-item version of the WHOQOL questionnaire (BREF version) (Krägeloh et al., 2011; Krägeloh et al., 2013). This scale has 7 items measuring: activities of daily living, dependence on medicinal substances and medical aids, energy and fatigue, mobility, pain and discomfort, sleep and rest, and work capacity (World Health Organisation, 1996) All items are presented on a 5-point Likert scale with varying anchors with high scores (after re-scoring negatively worded items) representing higher levels of quality of life.

3) Revised Competitiveness Index

This 14-item instrument measured students’ intention to win in interpersonal situations (Houston, Harris, McIntire, & Francis, 2002). There are two scales which measure enjoyment of competition (9 items) and contentiousness (5 items), with the latter scale measuring aspects of avoidance and dislike of conflict situations. All items are presented on a 5-point Likert scale (1 = never true to 5 = always true) with 9 items reversed scored: Items 4, 6, 7, 8, 10, 11, 12, 13, and 14.

4) Perceived Stress Scale

This 10-item instrument measured the degree to which students perceived their lives to be stressful based on their life experiences in certain situations (Cohen et al., 1983; Cohen, Kamarck, & Mermelstein, 1994). Students were instructed to respond to how often they felt over the last month: upset, in control, nervous, confident or angered.  All items are presented on a 5-point Likert scale (1 = never to 5 = always). Three reversals were instigated for items 4, 5, 7, and 8.

5) Motivated Strategies for Learning Questionnaire

Twenty two items were taken from the motivational beliefs scale, a questionnaire designed and applied by Pintrich and De Groot (1990). Three scales were incorporated measuring aspects of self-efficacy (9 items), intrinsic value (9 items), and test anxiety (4 items). Students were instructed to respond to the items on a 5-point Likert scale (1 = never true to 5 = always true) in reference to their perceptions of their behaviour in the Biomedical Common Year or Overlapping Year 1 programme.

D) Data analysis

First, the response rate (n/N) was calculated as a percentage measure. Second, statistics were presented that described the demographic and academic achievement details of the participants. These descriptive statistics were collected to consider the characteristics of the acquired sample of participants in relation to the student population. Third, the domains within the instruments were appraised in terms of their internal consistency using Cronbach’s alpha coefficients. Lastly, structural equation modelling was incorporated to consider the interrelationships among the four questionnaire measures of physical health, competitiveness, perceived stress, and motivation beliefs. Given the evidence from the literature concerning gender-based differences in these areas of measurement (Blanch et al., 2008; Henning et al., 2013; Hojat et al., 1999; Papier et al., 2014), two structural equation models (SEMs) were conducted for each gender group. The SEMs were conducted using the software AMOS v20. As chi-square values tend to become inflated with increases in sample size (Marsh, Balla, & McDonald, 1988), model fits were evaluated using a combination of goodness-of-fit indices: root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardized root mean square residual (SRMR). Following the accepted guidelines (Hu & Bentler, 1998), model fits were considered acceptable if RMSEA < 0.06, SRMR < 0.08, and CFI close to 0.95 or higher. To ensure that the analysis of the model parameters were stable and that the assumption of multivariate normality was not violated we implemented a Bootstrap method when calculating the confidence intervals and regressions weights (Byrne, 2010). We surmised this was a prudent statistical approach given the male and female models yielded markedly different sample sizes. Consequently, the regression weights were obtained using maximum likelihood bootstrapping with 1,000 samples. Additionally, bias-corrected 95% confidence intervals were calculated for the regression weights

III. RESULTS

A. Response rate and participant data

Three hundred and thirty nine students (n=339) completed the online survey resulting in a response rate of 33%. The average age of the sample was 18.98 years (SD = 2.80) with 75% of respondents being female. Academic achievement grades for the sample were equally spread across all grade allocations (A=31%, B=37%, C=17%, D=2%, other=13%). Table 1 provides demographic and academic achievement details for the sample and population. Some differences were noted, such as the sample group tend to be younger, with a higher proportion of female students, and their achievement scores were higher. However, upon inspection the distributions of the sample data did not indicate any major irregularities.

Table 1. Demographic and academic achievement details of the sample versus the population

The means and standard deviations for the variables of interest (Table 2), namely physical health, perceived stress, and motivational beliefs were comparable with other studies in the area (Henning et al., 2011; Henning et al., 2013; Roberti, Harrington, & Storch, 2006). The competitiveness scores were also well-matched, although trending lower than cited elsewhere (Harris & Houston, 2010; Houston et al., 2002).

Table 2. Means and standard deviations (SD) of the measures used in the study

In reference to internal consistency, all Cronbach measures met acceptable standards (Field, 2005): physical health (α =.77), perceived Stress Scale (α =.86), the competitiveness scales of enjoyment of competition (α =.91) and contentiousness (α =.83), and the motivational beliefs scales for intrinsic value (α =.83), self-efficacy (α =.90), and test anxiety (α =.84).

B. Structural Equation Modelling

Based on previous literature (e.g., Chester et al., 2013; Friedlander et al., 2007; Henning, Krägeloh, Hawken, Zhao, & Doherty, 2010; Hilliard, 1995; Hussain, Guppy, Robertson, & Temple, 2013; Lonka et al., 2008; Papier et al., 2014; Samaranayake & Fernando, 2011), an a priori model was specified that was then tested using structural equation modelling. The criterion variable was the students’ grades on the course Population Health, where scores varied from 0 (fail) to 9 (A+). The outcome variable, ‘Grade’ was assumed to be linked to the variances in the motivational variables test anxiety, self-efficacy, and intrinsic value, which created a potential predictive relationship with perceived stress, and in turn by physical health. Enjoyment of competition and contentiousness were treated as personality trait variables affecting the three motivational belief variables.

Models were tested twice, separately for males and females. The values obtained for RMSEA, SRMRand CFI indicated that the model fits were acceptable. For females, fits were RMSEA = 0.036, CFI = 0.993and SRMR = 0.033. For males, the fits were RMSEA = 0.027, CFI = 0.996and SRMR = 0.056.

Figure 1. Path diagram of the tested structural equation model for female students only. The path coefficients were shown only when statistically significant (p<.05). For non-significant relationships, directive arrows are shown with dashed lines.

Figure 2. Path diagram of the tested structural equation model for male students only. The path coefficients were shown only when statistically significant (p<.05). For non-significant relationships, directive arrows are shown with dashed lines.

During the process of model specification, modification indices were inspected, and as a result, enjoyment of competition was correlated with contentiousness and enjoyment of competition with physical health. Additionally, the error variances of the three motivation variables test anxiety, self-efficacy, and intrinsic value were allowed to be correlated. No other error variances were correlated.

Figures 1 and 2 show the path diagrams for female and male students respectively, for the physical domain. The path coefficients are shown for significant relationships only. The bias-corrected 95% confidence intervals for the regression weights (path coefficients) obtained using the maximum likelihood bootstrapping approach are shown in Table 3.

For female students (Figure 1), physical health was significantly associated with perceived  stress, which significantly explained test anxiety, self-efficacy, and intrinsic value. Contentiousness was not a significant predictor of any of the three motivational beliefs, and enjoyment of competition significantly predicted test anxiety and intrinsic value. Both test anxiety and self-efficacy predicted grade outcome.

For male students (Figure 2), physical health was significantly associated with perceived  stress, which significantly explained test anxiety and self-efficacy, but not intrinsic value. Contentiousness was a significant predictor of test anxiety, and enjoyment of competition significantly predicted test anxiety but not intrinsic value. Only self-efficacy predicted grade outcome.

Table 3. Lower and upper ends of bias-corrected 95% confidence intervals of the regression weights of the directional paths of the SEM. Confidence intervals are shown separately for females and males.

IV. DISCUSSION

The derived SEMs were able to systematically explore the relationships among physical health, competitiveness, perceived stress, motivational beliefs, and consider their impact on grade attainment among first-year pre-medical and health science students. The models considered all potential variable options and measures outlined in the Measures section above. These combinations were informed by prior work conducted in this research area (e.g., Chester et al., 2013; Friedlander et al., 2007; Henning, Krägeloh, Hawken, Zhao, & Doherty, 2010; Henning et al., 2011; Hilliard, 1995; Hussain et al., 2013; Lonka et al., 2008; Papier et al., 2014; Samaranayake & Fernando, 2011). In this study, we have also focused purely on physical health as a hard endpoint outcome variable and focused on the interactions between salient factors rather than creating a binary logistic argument and approach (Henning et al., 2017).

In the emergent model, the variables that resulted in the best fits included the exogenous variables physical health and measures of competitiveness. The process variables encompassed a global sense of perceived stress and motivational beliefs linked to self-efficacy, test anxiety, and intrinsic value. The outcome variable was the acquired Population Health grade on a pre-medical and health science degree course. Hence the model was able to identify and derive links among students’ characteristics, their experience of stress and motivation, and their grade achievement.

A. Exogenous variables: Physical health and competitiveness

Physical health and levels of competitiveness were considered as specific student attributes and this was reinforced by the best-fitted SEMs. The SEMs indicated that physical health significantly co-varied with enjoyment of competition and that there was a significant negative association with perceived stress. This suggests that physical health problems reported by students are likely interacting with stress, although the direction of effect cannot be directly inferred. Stress may impact physical health or vice versa. Nonetheless, physical health is a multidimensional variable and hence further research may need to consider more specific aspects of physical health relevant to this population, such as issues associated with sleep and fatigue (Samaranayake et al., 2014; Tanaka et al., 2008) with potential links with lifestyle choices (Kulkarni, 2016; Rustagi et al., 2011; Tanaka et al., 2008). The indirect effects and powerful predictive relationships between physical health, perceived stress and motivation are evidenced in our findings; however this relationship differed between male and female students with respect to grade achievement. These gender differences will be unpacked later in the discussion.

The notion of competitiveness employed in this study was derived from trait theory (Harris & Houston, 2010). In this study, competitiveness was seen more as a personal characteristic rather than a learned behaviour suggesting that this trait would be enhanced in certain circumstances that reflects a strong intention to succeed in competitive interpersonal situations (Musson, Sandal, & Helmreich, 2004). It is also related to the notion of conscientiousness or a strong sense of persistence in achieving one’s goals (Musson et al., 2004), which in turn relates to the competitive student environment of the biomedical sciences course. In their paper, Houston and colleagues (2002) developed two factors of competitiveness: enjoyment of competitiveness and contentiousness, with the latter concept describing a sense of defiance.

The findings of our study suggest that enjoyment of competition formed predictive relationships with self-efficacy and intrinsic value for female students and with only self-efficacy for male students. For the male data only, contentiousness formed a significant predictive relationship with test anxiety. It is likely that students who enjoy a sense of competition will do better academically, and this is consistent with our belief that students studying in the medical and health sciences are highly competitive and that the learning environment will likely escalate this behaviour (Coles, 1998; Rohe et al., 2006). In one study (Hancock, 2001), students were found to be more test anxious when they are more sensitive to competitive environments, agreeing somewhat with the results of the present study suggesting that the male students were more defiant (less sensitive) and thus less test anxious. One indirect effect on academic achievement was noted for enjoyment of competition only and this was via self-efficacy for both male and female students suggesting a significant interplay between enjoyment of competition and self- efficacy in terms of their predictive relationship with academic achievement.

B. Process variables: Perceived stress and motivation

Four factors were identified in this study in relation to the prediction of academic achievement. These included perceived stress, and the motivational belief variables of self-efficacy, test anxiety, and intrinsic value. According to the SEMs, perceived stress had an antecedent impact on the three motivational belief variables. Perceived stress was positively related to test anxiety and negatively associated with self-efficacy for all students.

Perceived stress can be seen as a generalized form of stress. Cohen and colleagues (1983) developed a global measure describing the impact that situational stressors may have on someone in their everyday life. The findings in our study showed that students who experienced relatively higher levels of everyday life stress are disadvantaged when it comes to developing their motivational beliefs as students. It could be inferred that the environment that one lives in as a person within society inevitably has an impact on the ‘student life’. However, differences were noted in the findings between male and female students, such that for male students there was no predictive relationship between perceived stress and intrinsic value, suggesting that the pre-medical and health science learning environment has differential effects on men and women, or there are differential perceived learning experiences being formed between the two groups. Differences between medical male and female student groups have been reported elsewhere in the literature but at a later level of education (Henning et al., 2013).

The findings further showed that the perceived stress impacted not only test anxiety but also had associations with self-efficacy and intrinsic value, although for male students the association between perceived stress and intrinsic value was not significant. Lee, Bong and Kim (2014) proposed that heightened stress is linked to the deployment of maladaptive strategies and that perceived stress affects educational competency. There is the potential for a feedback process that can create high levels of stress if students have lowered levels of expectancy about doing well and lowered confidence but at the same time value their course of study. In the SEMs in this present study, the optimal models do not account for this feedback mechanism and we would assume there are feedback links between the variables cited. Nonetheless, more research is required to fully explore the shape, direction and magnitude of these connections.

A gender difference was also noted in connection with the variable ‘Test anxiety’ such that test anxiety negatively predicted grade outcomes for female students only. Pintrich and Zusho (2007) stated in their paper that there is little evidence suggesting a difference between males and females in academic achievement. Nonetheless, in a further study, Henning and colleagues (2013) found differences between male and female medical students in their fourth and fifth years of study in reference to self-efficacy and test anxiety, with female students being more test anxious and less self-efficacious.

C. Grade outcome

In the present study, grade outcome was directly associated with the motivational variables test anxiety and self-efficacy for female students and only self-efficacy for male students. This suggests a difference in the way male students report their level of test anxiety or respond to the competitive factors within the learning environment. And this difference may be linked to the notion of sensitivity to the learning environment (Hancock, 2001); however further research is required to unpack this difference.

Similar to the present study, Pintrich and De Groot (1990) found strong associations between the motivational belief variables of self-efficacy, intrinsic value and test anxiety with grade outcome. Similarly, Henning et al. (2011) reported that written grade scores in a medical examination were related to these aspects of motivational beliefs in addition to aspects of self-regulation. However, the current findings showed that intrinsic value does not have a significant association with grade achievement suggesting that further research could investigate whether the role of extrinsic value is significant.

D. Limitations of the study and implications for educational research

The major limitation of this study is the response rate of 33%, although this is considered within the acceptable range for a purposive online survey (Nulty, 2008; Sax, Gilmartin, & Bryant, 2003). Nonetheless, the grade, age and gender distributions appear to be reasonable suggesting that any potential biases in the makeup of the sample are unlikely to change the interrelationships shown in the SEMs, but this potential limitation should nonetheless be acknowledged.

In this study, the illustrative models (Figures 1 and 2) suggest that students, in general, who have good physical health experiences will report less perceived stress, less test anxiety, and more self-efficacy and this then associates with positive grade outcomes. The results suggest that students who experience high levels of physical health will do better academically and this is partly explained because their motivational belief systems are enhanced. There are also subtle differences evident in the findings between the predictive relationships shown for the female students as opposed to their male peers, suggesting that ongoing work needs to be applied in ensuring equitable gender learning experiences. Moreover, physical health was chosen as the illustrative exemplar; however, similar linkages could also be explored in terms of psychological, social and environmental well-being.

From an educational perspective, these results suggest a conundrum, because, once selected into medical school, the educational ethos changes to one of collaboration and interprofessional practice, which could be perceived as antithetical to individual competitiveness (Reagan & Blakemore, 2009). The learning environment selects highly competitive students and then requires them to become collaborative or at least to appear so. A further area of research that will likely stem from this study is the type of competitiveness instilled in this year of study, whether this is in the form of ‘competing to win’ versus ‘competing to excel’ (Hibbard & Buhrmester, 2010).  In this study, we only considered aspects of ‘enjoyment’ and ‘contentiousness’, whereas the study of competitive intention may yield interesting results that may prove to be invaluable in understanding the development of empathy and refining assessment protocols in this pre-medical and health science course, such as instilling more collaborative based assessment. Clearly, more work is needed to tease out these connections, which are likely best studied by longitudinal studies to assess prospective influences on later behaviours as professionals.

V. Conclusion

Physical health and competitiveness characteristics were considered in this paper and the findings clearly show that ‘enjoyment of competition’ is linked with motivational beliefs especially self-efficacy and intrinsic value, and eventually academic outcome. Simply put, students who enjoy competition have more interest in their subject, are more confident, and do better. From a functional perspective, it can be inferred that more competitive students will do better, although there is evidence to suggest that competitiveness leads to performance learning rather than mastery learning at the undergraduate level (Harackiewicz, Barron, Tauer, & Elliot, 2002). Interestingly, our findings showed that competitiveness had a positive relationship with intrinsic value for female students, indicating that the source of their competition may have an intrinsic form which is a positive outcome. However, further research needs to assess the influence of extrinsic value mechanisms and links with mastery versus performance learning (Harackiewicz et al., 2002).

Notes on Contributors

Marcus A. Henning is an associate professor in the Centre for Medical and Health Sciences Education at the University of Auckland, New Zealand.

Christian U. Krägeloh is an associate professor in the Department of Psychology at AUT University, New Zealand.

Roger Booth is an associate professor in molecular medicine and pathology, in the Faculty of Medical and Health Sciences at the University of Auckland, New Zealand.

Erin M. Hill is an assistant professor in the Department of Psychology at West Chester University, United States of America.

Julie Chen is an assistant professor in the Department of Family Medicine and Primary Care and Institute of Medical and Health Sciences Education, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong.

Craig Webster is a senior lecturer in the Centre for Medical and Health Sciences Education at the University of Auckland, New Zealand.

Ethical Approval

Ethics approval for the collection and use of data was obtained from the University of Auckland Human Participants Ethics Committee (Ref. 010641).

Acknowledgements

This paper was presented in part as an e-poster at APMEC conference, Singapore, January 2017 and, for which the authors won a Merit award.

Declaration of Interest

The authors declared no competing interest.

References

Blanch, D. C., Hall, J. A., Roter, D. L., & Frankel, R. M. (2008). Medical student gender and issues of confidence. Patient Education and Counseling, 72(3), 374-381.
https://doi.org/10.1016/j.pec.2008.05.021.

Byrne, B. M. (2010). Structural Equation Modeling with AMOS: Basic concepts, applications and programming (2nd ed.). New York: Routledge.

Chester, A., Burton, L. J., Xenos, S., & Elgar, K. (2013). Peer mentoring: Supporting successful transition for first year undergraduate psychology students. Australian Journal of Psychology, 65(1), 30-37.

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 385-396.

Cohen, S., Kamarck, T., & Mermelstein, R. (1994). Perceived stress scale. Measuring stress: A guide for health and social scientists. Retrieved from
http://mindgarden.com/documents/PerceivedStressScale.pdf.

Coles, R. (1998). The moral education of medical students. Academic Medicine, 73(1), 55-57.

Elliott, D. C. (2014). Trailblazing: Exploring first-generation college students’ self-efficacy beliefs and academic adjustment. Journal of The First-Year Experience & Students in Transition, 26(2), 29-49.

Field, A. P. (2005). Discovering statistics using SPSS: And sex, drugs and rock’n’roll (2nd ed.). London, United Kingdom: SAGE.

Friedlander, L. J., Reid, G. J., Shupak, N., & Cribbie, R. (2007). Social support, self-esteem, and stress as predictors of adjustment to university among first-year undergraduates. Journal of College Student Development, 48(3), 259-274.

Google.com. (2015). Homepage. Retrieved February 13, 2015, from https://www.google.co.nz/forms/about/.

Hancock, D. R. (2001). Effects of test anxiety and evaluative threat on students’ achievement and motivation. The Journal of Educational Research, 94(5), 284-290.

Harackiewicz, J. M., Barron, K. E., Tauer, J. M., & Elliot, A. J. (2002). Predicting success in college: A longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation. Journal of Educational Psychology, 94(3), 562.

Harris, P. B., & Houston, J. M. (2010). A reliability analysis of the revised competitiveness index 1. Psychological Reports, 106(3), 870-874.

Henning, M. A., Krägeloh, C. U., Hawken, S., Zhao, Y., & Doherty, I. (2010). Quality of life and motivation to learn: A study of medical students. Issues in Educational Research, 20(3), 244-256.

Henning, M. A., Krägeloh, C. U., Booth, R., Hill, E. M., Chen, J., & Webster, C. S. (2017). Profiling potential medical students and exploring determinants of career choice. The Asia-Pacific Scholar, 2(1). Retrieved from http://theasiapacificscholar.org/profiling-potential-medical-students-and-exploring-determinants-of-career-choice-2/.

Henning, M. A., Krägeloh, C. U., Hawken, S. J., Doherty, I., Zhao, Y., & Shulruf, B. (2011). Motivation to learn, quality of life and estimated academic achievement: Medical students studying in New Zealand. Medical Science Educator, 21(2), 142-150.

Henning, M. A., Krägeloh, C. U., Manalo, E., Doherty, I., Lamdin, R., & Hawken, S. J. (2013). Medical students in early clinical training and achievement motivation: Variations according to gender, enrolment status, and age. Medical Science Educator, 23(1), 6-15.

Hibbard, D. R., & Buhrmester, D. (2010). Competitiveness, gender, and adjustment among adolescents. Sex Roles, 63(5-6), 412-424.

Hilliard, R. I. (1995). How do medical students learn: medical student learning styles and factors that affect these learning styles. Teaching and Learning in Medicine: An International Journal, 7(4), 201-210.

Hojat, M., Glaser, K., Xu, G., Veloski, J. J., & Christian, E. B. (1999). Gender comparisons of medical students’ psychosocial profiles. Medical Education, 33, 342-349. https://doi.org/10.1046/j.1365-2923.1999.00331.x.

Hojat, M., Gonnella, J. S., Erdmann, J. B., & Vogel, W. H. (2003). Medical students’ cognitive appraisal of stressful life events as related to personality, physical wellbeing, and academic performance: A longitudinal study. Personality and Individual Differences, 35(1), 219-235.

Houston, J., Harris, P., McIntire, S., & Francis, D. (2002). Revising the competitiveness index using factor analysis. Psychological Reports, 90(1), 31-34.

Hu, L.-t., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424-453.

Hussain, R., Guppy, M., Robertson, S., & Temple, E. (2013). Physical and mental health perspectives of first year undergraduate rural university students. BMC Public Health, 13(1), 848.

Ibrahim, N. K., Mahnashi, M., Al-Dhaheri, A., Al-Zahrani, B., Al-Wadie, E., Aljabri, M., . . . Bashawri, J. (2014). Risk factors of coronary heart disease among medical students in King Abdulaziz University, Jeddah, Saudi Arabia. BMC Public Health, 14(1), 411.

Krägeloh, C. U., Henning, M. A., Hawken, S., Zhao, Y., Shepherd, D., & Billington, R. (2011). Validation of the WHOQOL-BREF quality of life questionnaire for use with medical students. Education for Health, 24(2), 545. Retrieved from http://educationforhealth.net/publishedarticles/article_print_545.pdf.

Krägeloh, C. U., Kersten, P., Billington, D. R., Hsu, P. H.-C., Shepherd, D., Landon, J., & Feng, X. J. (2013). Validation of the WHOQOL-BREF quality of life questionnaire for general use in New Zealand: Confirmatory factor analysis and Rasch analysis. Quality of Life Research, 22(6), 1451-1457.

Kulkarni, M. V. (2016). Lifestyle disease risk behaviour among Medical Students in Central India. Panacea Journal of Medical Sciences, 6(2), 92-95.

Kusurkar, R. (2012). Motivation in medical students. Oisterwijk, Netherlands: Uitgeverij BOXPress

Kusurkar, R., Ten Cate, T. J., Van Asperen, M., & Croiset, G. (2011). Motivation as an independent and a dependent variable in medical education: A review of the literature. Medical Teacher, 33(5), e242-e262.

Lee, J., Bong, M., & Kim, S.-i. (2014). Interaction between task values and self-efficacy on maladaptive achievement strategy use. Educational Psychology, 34(5), 538-560.

Lonka, K., Sharafi, P., Karlgren, K., Masiello, I., Nieminen, J., Birgegård, G., & Josephson, A. (2008). MED NORD-A tool for measuring medical students’ wellbeing and study orientations. Medical Teacher, 30(1), 72 – 79.
https://doi.org/10.1080/01421590701769555.

Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391-410.

Moffat, K. J., McConnachie, A., Ross, S., & Morrison, J. M. (2004). First year medical student stress and coping in a problem‐based learning medical curriculum. Medical Education, 38(5), 482-491.

Musson, D. M., Sandal, G., & Helmreich, R. L. (2004). Personality characteristics and trait clusters in final stage astronaut selection. Aviation, Space, and Environmental Medicine, 75(4), 342-349.

Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33(3), 301-314.

Papier, K., Ahmed, F., Lee, P., & Wiseman, J. (2015). Stress and dietary behaviour among first-year university students in Australia: Sex differences. Nutrition, 31(2), 324-330.

Park, J., Chung, S., An, H., Park, S., Lee, C., Kim, S. Y., . . . Kim, K.-S. (2012). A structural model of stress, motivation, and academic performance in medical students. Psychiatry Investigation, 9(2), 143-149.

Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), 667-686.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33-40.

Pintrich, P. R., & Zusho, A. (2007). Student motivation and self-regulated learning in the college classroom. In R. P. Perry, & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective, 731-810. Dordrecht, The Netherlands: Springer.

Reagan, J., & Blakemore, L. C. (2009). Competitiveness Can Undermine Team Goals. Virtual Mentor, 11(5), 368-372.

Roberti, J. W., Harrington, L. N., & Storch, E. A. (2006). Further psychometric support for the 10‐item version of the perceived stress scale. Journal of College Counseling, 9(2), 135-147.

Rohe, D. E., Barrier, P. A., Clark, M. M., Cook, D. A., Vickers, K. S., & Decker, P. A. (2006). The benefits of pass-fail grading on stress, mood, and group cohesion in medical students. Paper presented at the Mayo Clinic Proceedings.

Rustagi, N., Taneja, D. K., Mishra, P., & Ingle, G. K. (2011). Cardiovascular risk behavior among students of a Medical College in Delhi. Indian Journal of Community Medicine, 36(1), 51-53.

Samaranayake, C. B., Arroll, B., & Fernando, A. T. (2014). Sleep disorders, depression, anxiety and satisfaction with life among young adults: a survey of university students in Auckland, New Zealand. The New Zealand Medical Journal (Online), 127(1399), 13-22.

Samaranayake, C. B., & Fernando, A. T. (2011). Satisfaction with life and depression among medical students in Auckland, New Zealand. Women, 20(1341), 17-45.

Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409-432.

Sobral, D. T. (2004). What kind of motivation drives medical students’ learning quests? Medical Education, 38(9), 950-957.

Spencer, J. A., & Jordan, R. K. (1999). Learner centred approaches in medical education. BMJ, 318(7193), 1280-1283.

Tanaka, M., Mizuno, K., Fukuda, S., Shigihara, Y., & Watanabe, Y. (2008). Relationships between dietary habits and the prevalence of fatigue in medical students. Nutrition, 24(10), 985-989. The University of Auckland. (2015). Biomedical Common Year or Overlapping Year. Retrieved from http://www.science.auckland.ac.nz/en/about/our-programmes/op-biomed-common-year.html.

World Health Organisation. (1996). WHOQOL-BREF: Introduction, administration, scoring and generic version of the assessment. Retrieved from
http://www.who.int/mental_health/media/en/76.pdf?ua=1.

*Marcus A. Henning
Centre for Medical and Health Sciences Education
The University of Auckland
Private Bag 92019
Auckland 1142
New Zealand
E-mail: m.henning@auckland.ac.nz
Fax: 0064 9 3737204

Published online: 2 January, TAPS 2019, 4(1), 42-47
DOI: https://doi.org/10.29060/TAPS.2019-4-1/OA2025

Dakshitha P. Wickramasinghe, Isuru S. Almeida & Dharmabandhu N. Samarasekera

Department of Surgery, Faculty of Medicine, University of Colombo

Abstract

Purpose: Medical students face significant psychological stress and adverse life events throughout their student career. The aim of this study was to assess the prevalence of depression among medical undergraduates in a medical faculty in Sri Lanka.

Methods: A cross-sectional study using a self-administrated, validated questionnaire was conducted among 300 undergraduate medical students of the University of Colombo, Sri Lanka. Prevalence of depression was assessed using Beck Depression Inventory (BDI). The data were analysed by the Mann-Witney U-test. P values less than 0.05 were considered statistically significant.

Results: Three hundred students participated in the study (male: female: 111: 189). Overall, 19% (male: female: 25:32) of students were identified as depressed. Depression was most common among 1^st year (25%) students followed by final year (20%) and 3^rd year (12%) students respectively. Depression was more common in students who were living in rented rooms compared to those residing in hostels and home (22%, 19.1% and 13.3%, respectively). Depression was less prevalent in students who participated in extracurricular activities than those who did not (11.9% vs 23.6%, P<0.05, Mann-Whitney U-test). There were 2 students with extreme depression.

Multivariate analysis identified residency outside the city and adverse life events to be associated with a higher risk of depression and extracurricular activities to be associated with a lower risk.

Conclusions: 1^st year and female medical students had a higher prevalence of depression while those who participated in extracurricular activities had lower rates. A system should be established to identify students with depressive features early.

Keywords:            Medical students, Depression, Stress, Sri Lanka

Practice Highlights

• Medical students have a higher prevalence of depression.
• Depression was seen less in students engaging in extracurricular activities.

I. INTRODUCTION

The increasing industrialisation in the 20^th century saw an increase in stress-related mental disorders (Rab, Mamdou, & Nasir, 2008). By the year 2020, it is postulated that anxiety and depression will be only second to heart diseases in terms of disability worldwide (Lopez & Murray, 1998). Medical students are confronted with stressful life events throughout their student career due to the quantum of material to be learnt and frequent academic evaluations (Mosley Jr et al., 1994). In addition, medical schools themselves are said to be stressful environments, which adversely affects students’ academic performance and their physical and psychological wellbeing (Mosley Jr et al., 1994).

Medical students have a higher rate of depression and stress than the general population (Rotenstein et al., 2016). Furthermore undergraduate medical students are the most distressed student group among all undergraduates (Rosenthal & Okie, 2005; Sherina, Rampal, & Kaneson, 2004). Two studies done in the USA using a questionnaire and cut-off values similar to the present study identified depression rates of 21.8% (Haglund et al., 2009) and 23.6% (Ghodasara, Davidson, Reich, Savoie, & Rodgers, 2011) among medical students.

Another study (Kulsoom & Afsar, 2015) recognised that stress and depression were higher in medical students who are preparing for exams. There is evidence that depressive symptoms may worsen as they progress in their career (Newbury-Birch, Lowry, & Kamali, 2002).

In addition, stress can adversely affect the academic performance (Melaku, Mossie, & Negash, 2015; Sohail, 2013), lead to stress-induced disorders (Malathi & Damodaran, 1999) and alcohol and substance abuse (Ball and Bax, 2002; Melaku et al., 2015). Therefore, students who are under extreme stress or depression need medical attention to prevent a cascade of troublesome consequences. Available evidence suggests that many of these psychological problems are under-diagnosed and under-treated, and may eventually lead to psychological morbidity (Tyssen, Vaglum, Grønvold, & Ekeberg, 2001). Previous authors have also shown that mental health deteriorates after beginning medical school career and remains low throughout their training (Rosal et al., 1997), and may even continue later into the internship, postgraduate studies and later life (Ahmed, Banu, Al-Fageer, & Al-Suwaidi, 2009).

The primary objective of this study was to determine the prevalence of depression and stressful life events among medical undergraduates. The secondary objective was to compare these findings with the first year, third year and final year medical students.

II. METHODS

A. Subjects & Setting

A cross-sectional study was conducted at the Faculty of Medicine, University of Colombo, Sri Lanka, from June to August 2016. The undergraduate program spans over 5 years, with 3 academic terms every year. The students learn pre-clinical subjects in the first 3 terms. This is followed by a module based teaching in the 2^nd – 4^th years (e.g. Cardio-vascular system, respiratory system) while simultaneously undergoing clinical training. The final year is almost entirely clinical training.

A self-administered questionnaire was distributed at a teaching activity where 100 students from the first, third and final year batches were present. These students represented the 1^st year, 3^rd year and final year who started their academic studies in 2015,2013 and 2011 respectively. All students returned completed questionnaires (100% response rate) as it was distributed during a common teaching activity and the responses were collected at the end of the teaching session.

The objectives of the study were explained to the participants and returning a completed questionnaire was assumed to be implied consent. To ensure anonymity, the participants were instructed to refrain from including identifiers on the questionnaire.

Students with self-reported longstanding physical/ mental illnesses were excluded from the analysis.

B. Instruments

The self-administered questionnaire included demographic details, history of physical and mental illnesses before entering medical faculty and involvement in extracurricular activities. The students were also asked to complete the Beck Depression Inventory (BDI)(Beck, Steer, & Brown, 1996).

The BDI is a 21-item self-report inventory. It can measure the severity of depression in adolescents and adults. The BDI was chosen to screen for depression in this study because of its well-established validity among medical undergraduates (Mancevska, Bozinovska, Tecce, Pluncevik-Gligoroska, & Sivevska-Smilevska, 2007). Beck, Steer & Carbin (1988) found a mean coefficient alpha of 0.86 and 0.81 for psychiatric and nonpsychiatric subjects, respectively.

The validated questionnaire consisted of 21 questions. The scale consists of items including depressive symptoms with intensities ranging from normal to extreme depression. We adopted the cut-off scores proposed by Mancevska et al. (2007). A cutoff point of 17 and above was used for detection of depression. BDI score of 31-40 was considered as severe depression and a score of above 40 was considered extreme depression.

A validated Sinhala version of the BDI is available (Rodrigo, Kuruppuarachchi, & Pathmeswaran, 2015), but we decided to use the original English version because the undergraduate program is conducted in English.

C. Data analysis

Statistical analysis was done using SPSS® 21.0 statistical software (SPSS Inc., USA) for Windows®. Results of categorical variables were expressed as frequencies and proportions while continuous variables were expressed using means ± standard deviations. Mann-Whitney U-test was used to compare data between groups. Logistic regression was used for multivariate analysis.

A P-value <0.05 was considered to be statistically significant.

III. RESULTS

A. Demographic data

All the students who were invited to participate returned a completed questionnaire (100%). The sample consisted of 300 undergraduate students, including 111 (37%) males and 189 (63%) females from three batches. The mean age of the sample was 22.9 (SD1.89, range 17-28) years (Table 1).

Table 1. Gender and age distribution of the students of the three batches

Of the participants, 202 (67.3%) were not residents of the city where the medical school is located, while 98 (32.7%) were from the city and suburbs. One hundred and thirty-one (43.7%) resided in the faculty hostels while 109 (36.3%) and 60 (20%) stayed in rented accommodation and in their own homes respectively (Table 2).

Table 2. Current residence of the students according to their gender

B. Prevalence of medical student depression

Overall, 57 (19%) of students had a depression score more than 17 in the BDI. Of them, 25 (43.86%) were male and 32 (56.14%) were female. Depression was most prevalent among 1^st-year students followed by the final year and 3^rd-year students, respectively. However, the only statistically significant difference between these years was seen between the 1^st year and 3^rd year students (Chi-square test, p=0.028) (Table 3).

Table 3. Depression rates in three batches

Depression status was then categorized according to the BDI scale (Table 4). Of the 2 students who reported extreme depression, one was from the 1^st year and the other was from the final year.

Table 4. Level of depression according to BDI scale

There was a statistically significant difference in the BDI scores between the students who’ve had positive, negative and no major life events (Kruskal-Wallis test, p=0.001). There was a statistically significant difference between the BDI scores between students with no major life events and negative life events (Mann-Whitney test, U=2561.5, p<0.0001) and between students who’ve had positive vs negative life events (Mann-Whitney test, U=444, p=0.009). There was no statistically significant difference between positive vs no major life events (p=0.8). Students who did not engage in extracurricular activities had a higher likelihood of being depressed (P<0.05). There was no statistically significant difference between residential status or a family history of mental illness with depression among students (Table 5).

Table 5. Rate of self-reported depression among students and their social charasteristics

A logistic regression analysis was performed to identify independent risk and protective factors for depression. The logistic regression model was statistically significant (χ² (3) = 14.5, p = 0.002). However, the model only explained 10.9% (Nagelkerke R²) of the variance in the presence of depression and correctly classified 82.1% of cases. Students who were not originally from the city were 2.3 times more likely to have depression. Students who had an adverse life event also had a risk of depression twice that of other students who did not. Participation in extracurricular activities was associated with a 60% less chance of depression.

IV. DISCUSSION

Medical students worldwide have a higher prevalence of depressive disorders compared to other undergraduates (Dyrbye, Thomas, & Shanafelt, 2006). Medical training aims to prepare graduates for a personally rewarding career (Dyrbye et al., 2006). However, medical students also have high academic expectations and are expected to master an extensive range of knowledge and skills in a highly competitive environment. Therefore, it is understandable that medical students are the most distressed group of students among all undergraduates (Sherina et al., 2004).

Studies on medical students in other parts of the world have also identified a high frequency of depression (Moffat, McConnachie, Ross, & Morrison, 2004; Rab et al., 2008). In a cross-sectional study done in Karachi, Pakistan, Inam, Saqib & Alam (2003) identified that 60% of students had anxiety and depression. Although there is a consensus that medical students have higher scores for depression, there is controversy about how these scores change over the undergraduate studentship. Several studies have shown that students may be the most depressed at the beginning of their course (Inam et al., 2003) – prevalence of depression higher among 1^st and 2^nd year students; depression highest among the 2^nd year students (Ahmed et al., 2009). However, in a recent meta-analysis Rotenstein (2016) failed to identify a difference between pre-clinical and clinical students.

Overall, 19% of the respondents in our sample were depressed by accepted BDI cutoffs. Compared to studies done in the region using BDI, this is similar to China (19% depression) (Pan et al., 2016) while lower than the results of both Hong Kong (48%)(Chan, 1991) and India (45%) (Gupta & Basak, 2013). However, both Chan (1991) and Gupta and Basak (2013) used a lower cut-off score to diagnose depression. We also observed that the rate of depression among our participants is higher than the 6.6% prevalence of depression in general population of Sri Lanka (Ball et al., 2010).

We found higher depression rates among female students than their male colleagues (56.14% vs 43.86%, P>0.05 ). A similar pattern was seen in other many previous studies (Dyrbye et al., 2006; Tjia, Givens, & Shea, 2005). However, a recent meta-analysis did not find the sex to significantly affect the depression scores (Rotenstein et al., 2016).

In addition, 1^st-year students were more depressed than both final year and 3^rd-year students. The same observation was met by other researchers as well (Tjia et al., 2005; Rab et al., 2008) and this may be due to the transition to the new environment (Inam et al., 2003; Rab et al., 2008), the new academic demands or getting mediocre results at exams (Rosenthal & Okie, 2005). Rab et al. (2008) reported that depression was commoner in students with a history of negative life events in the recent past. In addition, those staying in university hostels were more depressed than those living in their own homes. These findings were comparable with our results as well. Multivariate analysis, however, did not find these factors to be independently associated with depression.

We found that students who did not engage in any extracurricular activities were significantly depressed than those who did. Even though the cross-sectional nature of our study prevents us from establishing a causality, participation of extra-curricular activities have been found to reduce school drop-out (Mahoney & Cairns, 1997), lowering substance abuse and improving attitudes towards learning (Darling, 2005), improving school satisfaction (Gilman, 2001) and improve academic performance (Silliker & Quirk, 1997). Therefore, we feel participation in extra-curricular activities should be promoted among university students.

The main limitation of our study was that we did not include other batches apart from 1^st, 3^rd and final year students due to the limited time factor. Also, the study was done in students from one medical school only. Furthermore, the study variables were measured by a self-reported questionnaire which does not allow diagnostic conclusions since some students will not reveal their psychiatric issues. Previous authors have consistently demonstrated a higher overall depression among the western population. However, the data is sparse for South Asia in this regard. Therefore, it was difficult to compare our findings with the data from the south Asian region. A longitudinal study would provide more robust data on how depression scores change as students advance in their training and we are designing such a study.

However, we feel that mechanisms should be established to identify vulnerable students early in their training period and to provide them support. This can be based on the risk factors identified in this study as well as others (Dwyer & Cummings, 2001). In addition, carefully designed curricula aimed at reducing information overload might be helpful in preventing distress among undergraduates arising from the demands of tertiary education.

Simple interventions like physical exercise have been found to reduce depression (Lawlor & Hopker, 2001) and can be easily implemented in Sri Lanka. Steinhardt and Dolbier (2008) reports of successful use of resilience intervention in decreasing symptoms of academic stress. A similar 4-week program could be incorporated into the introductory course all medical undergraduates follow prior to the commencement of the academic program.

V. CONCLUSION

Medical students had a higher prevalence of depression than the general population. Engaging in extracurricular activities was associated with a less chance of depression. To avoid the repercussions, teachers and administrators must explore the reasons, consequences and answers.

Notes on Contributors

Dakshitha W., Isuru S. and Dharmabandhu N. S. designed the study. Dakshitha P. W. and Isuru S. A. collected the data. All authors were involved in data analysis, drafting the manuscript and approving the final manuscript.

Ethical Approval

Ethical approval for the study was obtained from the ethics review committee of the National Hospital of Sri Lanka.

Funding

There is no funding involved for this paper.

Declaration of Interest

None of the authors have any competing interests.

References

Ahmed, I., Banu, H., Al-Fageer, R., & Al-Suwaidi, R. (2009). Cognitive emotions: Depression and anxiety in medical students and staff. Journal of Critical Care, 24(3), e1-e7. http://dx.doi.org/10.1016/j.jcrc.2009.06.003.

Ball, H. A., Siribaddana, S. H., Kovas, Y., Glozier, N., McGuffin, P., Sumathipala, A., & Hotopf, M. (2010). Epidemiology and symptomatology of depression in Sri Lanka: A cross-sectional population-based survey in Colombo District. Journal of Affective Disorders, 123(1), 188-196. http://dx.doi.org/10.1016/j.jad.2009.08.014.

Ball, S., & Bax, A. (2002). Self‐care in medical education: Effectiveness of health‐habits interventions for first‐year medical students. Academic Medicine, 77(9), 911-917.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression inventory-II (pp. 78204-72498). San Antonio, TX: Psychological Corporation.

Beck, A. T., Steer, R. A., & Carbin, M. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8(1), 77-100. http://dx.doi.org/10.1016/0272-7358(88)90050-5.

Chan, D. W. (1991). Depressive symptoms and depressed mood among Chinese medical students in Hong Kong. Comprehensive Psychiatry, 32(2), 170-180.

Darling, N. (2005). Participation in extracurricular activities and adolescent adjustment: Cross-sectional and longitudinal findings. Journal of Youth and Adolescence, 34(5), 493-505. https://doi.org/10.1007/s10964-005-7266-8.

Dwyer, A. L., & Cummings, A. L. (2001). Stress, self-efficacy, social support, and coping strategies in university students. Canadian Journal of Counselling, 25(3), 208-220.

Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Academic Medicine, 81(4), 354-373.

Ghodasara, S. L., Davidson, M. A., Reich, M. S., Savoie, C. V., & Rodgers, S. M. (2011). Assessing student mental health at the Vanderbilt University School of Medicine. Academic Medicine, 86(1), 116-121. https://doi.org/10.1097/ACM.0b013e3181ffb056.

Gilman, R. (2001). The relationship between life satisfaction, social interest, and frequency of extracurricular activities among adolescent students. Journal of Youth and Adolescence, 30(6), 749-767. https://doi.org/10.1023/a:1012285729701.

Gupta, S., & Basak, P. (2013). Depression and type D personality among undergraduate medical students. Indian Journal of Psychiatry, 55(3), 287. https://doi.org/10.4103/0019-5545.117151.

Haglund, M. E., Aan Het Rot, M., Cooper, N. S., Nestadt, P. S., Muller, D., Southwick, S. M., & Charney, D. S. (2009). Resilience in the third year of medical school: A prospective study of the associations between stressful events occurring during clinical rotations and student well-being. Academic Medicine, 84(2), 258-268. https://doi.org/10.1097/ACM.0b013e31819381b1.

Inam, S., Saqib, A., & Alam, E. (2003). Prevalence of anxiety and depression among medical students of private university. Journal of Pakistan Medical Association, 53(2), 44-46.

Kulsoom, B., & Afsar, N. A. (2015). Stress, anxiety, and depression among medical students in a multiethnic setting. Neuropsychiatric Disease and Treatment, 11, 1713-1722. https://doi.org/doi/10.2147/NDT.S83577.

Lawlor, D. A., & Hopker, S. W. (2001). The effectiveness of exercise as an intervention in the management of depression: Systematic review and meta-regression analysis of randomised controlled trials. The British Medical Journal, 322(7289), 763-770. https://doi.org/10.1136/bmj.322.7289.763.

Lopez, A. D., & Murray, C. (1998). The global burden of disease. Nature Medicine, 4(11), 1241-1243. https://doi.org/10.1038/3218.

Mahoney, J. L., & Cairns, R. B. (1997). Do extracurricular activities protect against early school dropout? Developmental Psychology, 33(2), 241-253.

Malathi, A., & Damodaran, A. (1999). Stress due to exams in medical students-a role of Yoga. Indian Journal Of Physiology And Pharmacology, 43, 218-224.

Mancevska, S., Bozinovska, L., Tecce, J., Pluncevik-Gligoroska, J., & Sivevska-Smilevska, E. (2007). Depression, anxiety and substance use in medical students in the Republic of Macedonia. Bratislavské Lekárske Listy, 109(12), 568-572.

Melaku, L., Mossie, A., & Negash, A. (2015). Stress among medical students and its association with substance use and academic performance. Journal of Biomedical Education, 2015, 1-9. https://doi.org/10.1155/2015/149509.

Moffat, K. J., McConnachie, A., Ross, S., & Morrison, J. M. (2004). First year medical student stress and coping in a problem‐based learning medical curriculum. Medical Education, 38(5), 482-491. https://doi.org/10.1046/j.1365-2929.2004.01814.x.

Mosley Jr., T. H., Perrin, S. G., Neral, S. M., Dubbert, P. M., Grothues, C. A., Pinto, B. M. (1994). Stress, coping, and well-being among third-year medical students. Academic Medicine, 69(9), 765-767.

Newbury-Birch, D., Lowry, R., & Kamali, F. (2002). The changing patterns of drinking, illicit drug use, stress, anxiety and depression in dental students in a UK dental school: A longitudinal study. British Dental Journal, 192(11), 646-649.

Pan, X.-F., Wen, Y., Zhao, Y., Hu, J.-M., Li, S.-Q., Zhang, S.-K., … Fu, C. (2016). Prevalence of depressive symptoms and its correlates among medical students in China: A national survey in 33 universities. Psychology, Health & Medicine, 21(7), 882-889. https://doi.org/10.1080/13548506.2015.1127395.

Rab, F., Mamdou, R., & Nasir, S. (2008). Rates of depression and anxiety among female medical students in Pakistan. Eastern Mediterranean Health Journal. 14(1), 126-33.

Rodrigo, A., Kuruppuarachchi, KALA, Pathmeswaran, A. (2015). Validation of the Beck Depression Inventory II among the Sinhalese speaking population in Sri Lanka. Sri Lanka Journal of Psychiatry, 6(2), 20-24.

Rosal, M. C., Ockene, I. S., Ockene, J. K., Barrett, S. V., Ma, Y., & Hebert, J. R. (1997). A longitudinal study of students’ depression at one medical school. Academic Medicine, 72(6), 542-546.

Rosenthal, J. M., & Okie, S. (2005). White coat, mood indigo–depression in medical school. New England Journal of Medicine, 353(11), 1085-1088. https://doi.org/10.1056/NEJMp058183.

Rotenstein L.S., Ramos M.A., Torre M., Segal J.B., Peluso M.J., Guille C., …, Mata, D. A. (2016). Prevalence of depression, depressive symptoms, and suicidal ideation among medical students: A systematic review and meta-analysis. The Journal of American Medical Association, 316(21), 2214-2236. https://doi.org/10.1001/jama.2016.17324.

Sherina, M., Rampal, L., & Kaneson, N. (2004). Psychological stress among undergraduate medical students. Medical Journal of Malaysia, 59(2), 207-211.

Silliker, S. A., & Quirk, J. T. (1997). The effect of extracurricular activity participation on the academic performance of male and female high school students. The School Counselor, 44(4), 288-293.

Sohail, N. (2013). Stress and academic performance among medical students. Journal of College of Physicians and Surgeons of Pakistan, 23(1), 67-71.

Steinhardt, M., & Dolbier, C. (2008). Evaluation of a resilience intervention to enhance coping strategies and protective factors and decrease symptomatology. Journal of American College Health, 56(4), 445-453.

Tjia, J., Givens, J. L., & Shea, J. A. (2005). Factors associated with undertreatment of medical student depression. Journal of American College Health, 53(5), 219-224.

Tyssen, R., Vaglum, P., Grønvold, N. T., & Ekeberg, Ø. (2001). Suicidal ideation among medical students and young physicians: A nationwide and prospective study of prevalence and predictors. Journal of Affective Disorders, 64(1), 69-79.

*Dharmabandhu N Samarasekera
Professor of Surgery and Senior Professor,
Faculty of Medicine, University of Colombo
Address of institution: Department of Surgery,
Faculty of Medicine, P.O. Box 271,
Kynsey Road, Colombo 8, Sri Lanka
Tel: 94 – 112671846
Fax: 94 – 112671846
Email: samarasekera58@yahoo.co.uk

Published online: 2 January, TAPS 2019, 4(1), 34-41
DOI: https://doi.org/10.29060/TAPS.2019-4-1/OA2005

Jill Cheng Sim Lee¹, Xiang Lee Jamie Kee², Sharon Wiener-Ogilvie³, Bernard Su Min Chern^{1, 4} & Chee Yang Chin⁵

¹Division of Obstetrics and Gynaecology, KK Women’s and Children’s Hospital (KKH), Singapore; ²Yong Loo Lin School of Medicine, National University of Singapore (NUS), Singapore; ³NHS Education for Scotland, United Kingdom; ⁴SingHealth Duke-NUS Obstetrics and Gynaecology (OBGYN) Academic Clinical Programme, Singapore; ⁵Department of Cardiovascular Medicine, National Heart Centre Singapore (NHCS), Singapore

Abstract

Background: Resident attrition is costly but literature studying the motivations behind it in obstetrics and gynaecology (OBGYN) is lacking.

Objectives: We aimed to study the relationship between prior specialty-appropriate work experience and risk of attrition amongst OBGYN residents in Singapore, and identify factors placing residents at greater risk of attrition.

Methods: This nationwide mixed methods case control study studied all OBGYN residents in Singapore for the 2011 and 2012 intakes. A pre-piloted questionnaire was used to identify work experience and risk factors related to resident attrition. Structured interviews were conducted with a subgroup of participants chosen for diversity of educational background, work experience, and position towards residency.

Results: 28 of 33 (84.8%) eligible residents participated in this study. Female (40.9% vs. 0%), married (50.0% vs. 22.2%) and Singaporean (38.1% vs.14.3%) residents more often considered attrition from training. Those accepted into residency after graduation were 33% less likely to have considered leaving. No statistical difference was observed between residents with prior work experience and those without (38.9% vs. 20.0%, p=0.417). All interviewees believed that prior experience informs expectations and eases initial learning. Low job satisfaction related to mismatched expectations was the most quoted reason for considering attrition. 85.7% of interviewed residents were concerned about competing work and family demands.

Conclusions: Lack of specialty-appropriate work experience contributes to misinformed training expectations and increased attrition risk. Residents who have considered attrition may still desire career longevity in their specialty. Reduction in mismatched expectations promises to improve job satisfaction and translate to career longevity.

Keywords:            Resident Attrition, Obstetrics and Gynaecology, Work Experience

Practice Highlights

• Resident training is expensive and attrition is stressful to the individual resident, training programme and health service.
• Recognition of risk factors for attrition from the training programme is important to improve identification of at-risk residents and take measures to reduce attrition.
• Mismatched expectations are often the reason for job attrition.
• Prior specialty-appropriate work experience may reduce misperceptions and better prepare residents for transition into the rigour of postgraduate specialty training.

I. INTRODUCTION

20% of obstetrics and gynaecology (OBGYN) residents leave their training programme in the United States (US).(McAlister, Andriole, Brotherton, & Jeffe, 2008) There are concerns that a similar pattern is followed in Singapore where just two years into the adoption of a four-year American-styled residency training programme in 2011, a 5.9% attrition rate (2 of 34) was observed amongst residents, with all departures voluntary. This figure is comparable to the 3.2% average annual attrition rate published by the largest residency sponsoring institution in Singapore across specialties. (Tamil Mannan et al., 2018).

Studies have postulated OBGYN attrition to be a result of workplace demands, such as the high litigation risk and long hours (S. E. Ogbonmwan & D. E. Ogbonmwan, 2010). However, limited studies have formally investigated the motivations behind it. With the training cost for an OBGYN resident being estimated at US$100,000 per year (American College of Obstetrics and Gynecologists, 2014), attrition is an exceedingly costly affair. Identifying factors behind resident attrition can potentially guide program decisions and efforts to retain trainees.

OBGYN, dermatology, paediatric medicine, ophthalmology, orthopaedic surgery, otorhinolaryngology and plastic surgery are seven specialties recognised by the Ministry of Health in Singapore to be highly subscribed specialties where additional training fees are paid by residents in these programmes. Between 2011 and 2018, the ratio of resident intake to applicants in OBGYN in Singapore ranged between 10 to 28.8% with a mean of 22.2%. Recognising applicants at risk of attrition in these popular specialties may help improve resident selection criteria.

Furthermore, although previous work experience in OBGYN has been associated with more confidence in procedures (Ghaem-Maghami, Brockbank, & Bridges, 2006) and is perceived to be useful in the eventual decision of specialisation (W. M. Rozen & G. Rozen, 2006), literature behind the impact of residents’ prior work experience on resident attrition is scarce. Given that in 2011 and 2012, approximately half of all applicants for OBGYN residency in Singapore were medical students with no prior work experience, the relationship between work experience and risk of attrition is potentially an important one.

Therefore, in this study, we aimed to identify factors within the individual resident which placed them at greater risk of attrition, as well as to elucidate the relationship between work experience and risk of attrition.

II. METHODS

A questionnaire was developed following discussion with institutional medical educators, faculty and resident peer leaders to identify relevant study variables with regards OBGYN resident attrition. The questionnaire was piloted twice on clinical and non-clinical healthcare personnel in order to ensure subject relevance. Consideration of leaving the residency programme was used as a proxy for risk of attrition.

In Singapore, only two OBGYN residency programmes exist; SingHealth and National University Hospital Singapore. All OBGYN residents in both sponsoring institutions from the intake of 2011 and 2012 were invited to participate in this study (N=33). They were recruited over a 4-week period in 2014 during weekly national OBGYN teaching sessions and via email.

Questionnaire results were analysed using Fisher’s exact test or independent sample t-test, where appropriate, and descriptive statistics. Gaps in explanations for observed trends were identified and questions addressing these gaps were subsequently developed for a structured face-to-face, individual interview conducted by a single investigator (JL) who is herself an OBGYN resident in Singapore and a resident representative elected by peers in this cohort. JL was excluded from this study cohort to reduce investigator bias.

A mixed pool of participants, comprising residents who had left residency, had considered leaving, or had never considered leaving, and of a variety of educational backgrounds and work experience were identified from the questionnaire and approached to take part in these structured interviews. 15 participants were approached for interview with 7 (46.7%) agreeing to participate. Participants were also selected to have a mixture of gender, age, and marital status.

All interviews were taped and transcribed. Interview data was entered into QSR International NViVo 10 to allow line by line coding for emergence of themes based on the principles borrowed in part from grounded theory and triangulated with quantitative data obtained from the questionnaire.

In accordance with local practices, the study was exempted from SingHealth Centralised Institutional Review Board review. Participants were informed of the study requirements, timeline and aims. Verbal consent was then obtained and confidentiality was maintained throughout. Participation was strictly voluntary. Patients were not involved in this study and no funding was sourced or required for the conduct of this study.

III. RESULTS

A. Questionnaires

28 of 33 (84.8%) residents agreed to participate in the study. Of these, 1 had already left OBGYN residency. The only other resident who had left withdrew her consent to participate due to reasons unrelated to this study. Tables 1 and 2 summarize the demographic data and characteristics of participants, respectively.

Table 1. Demographics of study participants

Figure 1. Influence of prior OBGYN MO experience on resident expectation, satisfaction and regret: Percentage positive responses (Agree or Strongly Agree)

32.1% of residents surveyed had considered leaving the OBGYN residency programme. Female (40.9% vs. 0%), married (50.0% vs. 22.2%) and Singaporean (38.1% vs.14.3%) residents appeared more likely to have considered leaving OBGYN training, although the differences did not amount to statistical significance. Country of medical school did not appear to make a difference to thoughts on attrition.

Table 2. Specialty preference and prior work experience of study participants

42.9% of OBGYN residents who were accepted before graduation from medical school (postgraduate year [PGY] 0) had previously considered leaving OBGYN residency, while only 28.6% of OBGYN residents who were accepted post-graduation (PGY 1 and above) had previously considered leaving the programme. This suggested that those accepted after graduation were 33% less likely to have considered leaving OBGYN residency in this cohort.

No statistical difference was observed between residents with prior work experience as a OBGYN Medical Officer (MO) compared with residents who had no such prior experience with regards to leaving residency (38.9% vs. 20.0%, p=0.417). Notably, there were more participants in this study who had had prior work experience as an OBGYN MO than those who had no prior experience (64.3% vs. 35.7%). 78.6% of participants had prior OBGYN house officer (HO) experience. 30% of participants with no prior OBGYN MO experience also had no OBGYN HO experience prior to application to residency.

When assessed on a 5-point Likert scale about their prior expectations, current satisfaction levels, and feelings of regret towards the residency programme, there did not appear to be a discernible difference in responses between groups with prior work experience and those without (Figure 1).

B. Structured interviews

Table 3 describes the characteristics of interviewees. This study concentrated on themes that were common to all interviews.

1) Prior work experience

All interviewees felt prior OBGYN MO experience enables informed expectations and eases initial learning. OBGYN experiences as students or HOs were deemed not as insightful, providing only superficial specialty-specific work.

Residents with prior experience as both an OBGYN HO and MO noted:

“The house officers’ work scope is very different from that of a medical officer or resident.”

“I think OBGYN is a very subspecialized specialty. It is not like internal medicine or general surgery where you probably have gained more experience as a medical student compared to OBGYN as a medical student. It is good to actually have a non-training medical officer posting, at least one or two (6-month) postings, before actually embarking on the residency programme to make sure that you are actually OK to continue on with the residency programme. I know some of my classmates who had wanted to do OBGYN but after doing some postings here (in OBGYN) they found that it was not their cup of tea. They are doing something else now and are happy with what they are doing right now.”

Similarly, a resident with no prior OBGYN work experience noted:

“I think (prior medical officer experience) makes a difference because I think it makes a big difference for Year 1 and possibly a little bit in Year 2 because if you do not have any experience, then you are essentially catching up with everybody else because everybody else would have had experience. You spend a lot of time catching up and it may not be as easy to do well (in the programme) as compared to other people who have done OBGYN before.”

Table 3. Characteristics of interview participants

2) Mismatched expectations

Those considering attrition among the interview group most commonly attributed their reasons to low job satisfaction related to mismatched expectations of training.

This mismatch was often due to teething problems with a new residency programme. Reasons cited include “the [residency] system [not being] ready yet”, as well as frustrations with “promised […] opportunities” not perceived to be delivered, and thoughts of it “not yet [being] a well thought out programme”.

Having undergone undergraduate medical training outside of Singapore also contributed to these unmet expectations, with one resident noting that “[…] training in UK is different. The culture is all different. So, this isn’t really what I expected.”

Another echoed similar thoughts, saying:

“In the UK, when you become a consultant, your work-life balance actually improves. […]So that’s the perception (I had). (However,) the sad thing about OBGYN (in Singapore) is the work-life balance of a medical officer or resident is a lot better than (that of) a registrar or a consultant.”

These views were corroborated by locally trained residents. They perceived that the OBGYN residency was advantageous to graduates from Singaporean medical schools, most significantly in the earliest stages of residency training.

“People know you since probably medical school. It’s easier to negotiate things and it is probably a better advantage if you are a local graduate to actually go on to do a subspecialty locally.”

“The advantage that it confers is probably just initially during the whole settling in period with being a house officer and medical officer. I think the advantage levels off after a while and it’s really just the initial familiarity that makes us a little bit more competent and efficient earlier on.”

3) Career aspirations

Of note, most interviewees considering attrition still remained hopeful of long public service OBGYN careers, contingent on good leadership and collegiality.

One resident considering attrition stated:

“I think that having a positive work culture and happy work culture contributes a lot. I think I can settle for doing something…like for example I can work for less pay, and work longer hours and work harder for a good boss in a job that I truly feel satisfied in compared to having a cushier time but not being happy.”

Another resident, also considering attrition, expressed:

“(In 10 years’ time,) I don’t really see myself going private. I think I’ll probably just continue on, maybe in general OBGYN or doing admin or teaching.”

 4) Balancing demands of work and family

Almost all interview participants expressed concerns about balancing career and family in the future. Many, as illustrated earlier, recognised the increasing work demands expected of an individual as they progress up the OBGYN hierarchy which may threaten their ideals on work-life balance with many only “hoping to have a family with kid (and) some time for myself”.

Others have observed and reported that:

“My friends are now struggling with family and life. Life means a balance; have time to go out for a meal, have time just to go out. At this moment, they are really struggling with work-life balance issues.”

A male resident added that:

“(In the next 10 years) I will probably want to get married and settle down with children and definitely want to pursue whatever I have learnt over the last 10 years. (I) will definitely continue my practice in OBGYN. I have to support the family and be the breadwinner right?”

IV. DISCUSSION

A) Main Findings

Similar to previous studies, a significant proportion of residents interviewed have considered leaving training. While no significant statistical differences emerged between the two groups (“never considered leaving” vs. “considered leaving”), certain patterns were observed.

This study recognised that residents who were female, married and Singaporean citizens trended towards considering attrition more often than their other counterparts. Despite no statistical significance identified between residents with prior specialty-specific work experience, qualitative data found that all interview participants agreed that such prior experience, especially at a level beyond that of a junior house officer was helpful to better manage their expectations of the training programme as well as eased their initial learning. With mismatched expectations leading to low job satisfaction, it is important to address training expectations prior to entry into OBGYN residency through opportunities such as prior OBGYN medical officer work experience.

B) Strengths and Limitations

This study included residents from all training institutions in Singapore with a participation rate of 84.8%. The high participation rate allowed for the majority of opinions to be studied in this case-control study. As the structured interviews were conducted by a single investigator who was known to all participants, the data may be scrutinised for bias. However, being part of the OBGYN residency programme, JL had a deeper understanding of the complexities of the medical training system beyond that of an independent researcher, allowing her to generate deeper meaning when analysing the qualitative data in this study. Furthermore, her position as a trusted peer-elected representative and guarantee of privacy and anonymity allowed participants to share views forthrightly during the individual structured interviews. To reduce bias, a standardised series of questions developed following analysis of the written questionnaire were asked of all participants during the structured interview and findings from the qualitative interviews were triangulated with the questionnaire data.

We recognise that as Singapore is a small country with small intakes of residents per annum and with only 2 residents having left OBGYN training in this studied cohort, this study was not adequately powered to measure statistical significance between those who considered attrition and those who did not. However, addition of the qualitative component of this study aimed to overcome the limitations of the small resident numbers. In addition, the withdrawal of participation from one of the two residents who had left training resulted in loss of a valuable voice in this study but as over a third of residents have considered the idea of leaving training, the discussion of driving factors for attrition could still be explored thoroughly in this study.

C) Interpretation

Female and married residents appeared more likely to consider leaving residency, possibly due to expectations in the still traditional setting of a nuclear family in Singapore. In Aryeel’s 1993 study of dual-earner Singaporean couples, there was significantly more burnout in wives than in husbands, with stressors stemming from both career and non-career life domains. With the secondary status of women deeply rooted in Singaporean culture, working women are still expected to perform a disproportionate portion of the household chores and childcare, including playing a major role in helping manage their child’s daily schoolwork in the highly competitive Singaporean education system. It is possible that non-career stressors impact one’s performance and enjoyment of work, thereby leading to workplace dissatisfaction, as evidenced by how work-family conflicts have been reported to be a significant positive antecedent of burnout (Bacharach, Bamberger, & Conley, 1991).

Non-Singaporeans appeared less likely to consider attrition than their Singaporean counterparts. 25% of participants were non-Singaporean, a testament to international migration of healthcare professionals. The global phenomenon of migration has been increasingly observed over the last decades, especially from less developed countries to more developed countries. Multiple “push factors” such as low wages, poor working conditions and promotion possibilities, inadequate management support, heavy workloads, limited access to good technology, and even to medicines, have all been recognized as “push factors” motivating health workers to leave their home countries (Connell, Zurn, Stilwell, Awases, & Braichet, 2007). The World Health Organization ranked the Singapore health system sixth for overall performance amongst its member states in The World Health Report 2000 (World Health Organization, 2000). Malaysia ranked 49^th while China trailed in 144^th place. For these non-Singaporean doctors, when selected for residency in a perceivably more successful healthcare system, they have strong motivation to succeed in their new country. This is in contrast with their local counterparts who may perceive such postgraduate training opportunities as a norm and view it more as an expected outcome than a privileged opportunity.

With no statistical difference observed between those with and without prior OBGYN MO experience with regards to attrition in this small study, the influence of work experience is more clearly explored through the qualitative data component. All participants expressed unanimously that prior specialty-specific work experience would be beneficial, irrespective of whether they themselves had gained such experience or not. The two dominant reasons that residents felt that prior experience was helpful were: (1) they felt the learning curve in the first year of OBGYN residency was steep and the prior experience would better prepare them for this hurdle, and (2) they felt it would help them form more realistic expectations and manage their own expectations of residency training and a OBGYN career. This finding challenges the current drive to accelerate postgraduate training and selection of specialty training following graduation from medical school as seen in programmes such as Modernising Medical Careers.

Resident attrition data from SingHealth cites programme mismatch as the leading cause of resident separation from training (Tamil Mannan et al., 2018). Acknowledging that mismatched expectations are a key contributor to attrition risk, it appears that prior work experience, by means of providing exposure and thereby the formation of realistic expectations could help to reduce attrition from unmet expectations. Furthermore, Oraman, Unakıtan and Selen (2011) found that job satisfaction is greater if there is congruency between the expectation and the experience while Shields and Ward (2001) report a 65% higher probability of nurses intending to quit if they had low job satisfaction.

Crucially, considering attrition was not associated with diminished career aspirations. These residents should be identified early in training and provided with necessary support to continue to nurture dedicated clinicians with longevity in the healthcare service. A rigorous and supportive mentoring system utilizing the recognized strength of camaraderie and teamwork within the OBGYN residency could potentially sufficiently address this issue.

V. CONCLUSION

Lack of prior specialty-appropriate work experience is a major contributor to misinformed training expectations and increased inclination towards attrition. Prior specialty-appropriate medical officer experience appears to influence resident attrition as it reduces misperceptions and better prepare residents for the transition into the rigorous reality that is postgraduate specialty training. Reduction in mismatched expectations promises to improve job satisfaction and translate to career longevity. This should be aimed not just at the level of residents but also for retention of trained specialists within the healthcare service. Residents who have considered attrition may still desire career longevity in their specialty. Valuable early work experiences that prepare junior doctors for the demands of training are limited within contemporary training models that emphasize time-efficient specialty training. It is important that non-training medical officer posts continue to be made available to junior doctors to help bridge the expectation gap and transition of pre-residency doctors into specialty training. Follow up research of this cohort upon completion of training will be useful to explore if factors that drive attrition change as the resident progresses through their training years and if prior pre-residency work experience indeed affects career longevity beyond specialty training on long-term follow up.

Notes on Contributors

Dr. Jill Cheng Sim Lee is a senior resident in OBGYN at SingHealth and former Chief Resident for Education within her department. She has a Master of Science in Clinical Education and is involved in undergraduate medical education at Lee Kong Chian School of Medicine, Nanyang Technological University.

Miss Xiang Lee Jamie Kee is a final year medical student at Yong Loo Lin School of Medicine, NUS. She has a keen interest in medical education and is actively involved in running near-peer teaching initiatives at her university.

Dr. Sharon Wiener-Ogilvie is an associate adviser for quality and research in General Practice at NES with prior experience in public health and clinical governance. She holds an MPH and a PhD and has published extensively in medical education, particularly around career choice, and evaluation of educational programmes.

A/Prof. Bernard Su Min Chern is Chairman of Division of OBGYN and Head and Senior Consultant of the Department of Minimally Invasive Surgery at KKH. He is also Chairman of the SingHealth-Duke-NUS OBGYN ACP and was formerly Programme Director of the SingHealth OBGYN Residency Programme.

Dr. Chee Yang Chin is a Consultant Cardiologist at NHCS with a sub-specialty interest in interventional cardiology. He has a Master of Science in Clinical Education and is Core Faculty to both the SingHealth Internal Medicine Residency Programme and Cardiology Senior Residency Programme.

Ethical Approval

In accordance with local practices, the study was exempted from SingHealth Centralized Institutional Review Board review.

Acknowledgements

The authors would like to acknowledge the support and cooperation provided by the residents, faculty and staff at the SingHealth OBGYN Residency Programme during the period of this study. We would also like to thank Ms Gillian Aitken and Dr Michael Ross from the University of Edinburgh for their advice during design and development phases of this study.

Funding

This is an unfunded study.

Declaration of Interest

All authors have no potential conflicts of interest.

References

American College of Obstetricians and Gynecologists (ACOG) (2014). Support the Training of Tomorrow’s Doctors Today Act of 2013. Retrieved from http://www.acog.org/~/media/Departments/Government%20Relations%20and%20Outreach/2014CLCGMEAsk.pdf?dmc=1&ts=20140327T1032213142.

Aryeel, S. (1993). Dual-earner couples in Singapore: An examination of work and nonwork sources of their experienced burnout. Human Relations, 46(12), 1441-1468. http://doi.org/10.1177/001872679304601205.

Bacharach, S. B., Bamberger, P., & Conley, S. (1991). Work-home conflict among nurses and engineers: Mediating the impact of role stress on burnout and satisfaction at work. Journal of Organizational Behaviour, 12(1), 39-53. http://doi.org/10.1002/job.4030120104.

Connell, J., Zurn, P., Stilwell, B., Awases, M., & Braichet, J. M. (2007). Sub-Saharan Africa: beyond the health worker migration crisis? Social Science & Medicine, 64(9), 1876-1891. http://doi.org/10.1016/j.socscimed.2006.12.013.

Ghaem-Maghami, S., Brockbank, E., & Bridges, J. (2006). Survey of surgical experience during training in obstetrics and gynaecology in the UK. Journal of Obstetrics and Gynaecology, 26(4), 297-301. http://doi.org/10.1080/01443610600594740.

McAlister, R. P., Andriole, D. A., Brotherton, S. E., & Jeffe, D. B. (2008). Attrition in residents entering US obstetrics and gynecology residencies: analysis of National GME Census data. American Journal of Obstetrics and Gynecology, 199(5), 574 e571-576. http://doi.org/10.1016/j.ajog.2008.06.081.

Ogbonmwan, S. E., & Ogbonmwan, D. E. (2010). Recruitment and retention in obstetrics and gynaecology in the UK. British Journal of Hospital Medicine (Lond), 71(2), 103-105. http://doi.org/10.12968/hmed.2010.71.2.46490.

Oraman, Y., Unakıtan, G., & Selen, U. (2011). Measuring employee expectations in a strategic human resource management research: Job satisfaction. Procedia – Social and Behavioral Sciences, 24, 413-420. http://dx.doi.org/10.1016/j.sbspro.2011.09.022.

Rozen, W. M., & Rozen, G. (2006). Postgraduate obstetrics and gynaecology experience: The role for early exposure. Australian and New Zealand Journal of Obstetrics and Gynaecology, 46(6), 538-540. http://doi.org/10.1111/j.1479-828X.2006.00656.x.

Shields, M. A., & Ward, M. (2001). Improving nurse retention in the National Health Service in England: The impact of job satisfaction on intentions to quit. Journal of Health Economics, 20(5), 677-701.

Tamil Mannan, S. D., Heng, J. M. J., Andrada, J. A., Choon, P. H., Yeo, H. S. H. & Lim, B. L. (2018, March). Improving resident selection through analysis of attrition rate in SingHealth Residency. Poster presented at the Accreditation Council for Graduate Medical Education Annual Education Conference, Orlando, FL.

World Health Organization. (2000). The World Health Report 2000: Health Systems: Improving Performance. Retrieved from www.who.int/whr/2000/en/whr00_en.pdf.

*Dr Jill C. S. Lee
Email: jill.lee.c.s.@singhealth.com.sg
Division of Obstetrics and Gynaecology
KK Women’s and Children’s Hospital
100 Bukit Timah Road, Singapore 229899
Tel: +65 62255554

Published online: 2 January, TAPS 2019, 4(1), 24-33
DOI: https://doi.org/10.29060/TAPS.2019-4-1/OA1063

Wee Shiong Lim^{1, 2}, Kar Mun Tham³, Fadzli Baharom Adzahar, Han Yee Neo⁴, Wei Chin Wong¹, Issac Lim², Charlotte Ringsted⁵ & Dujeepa D. Samarasekera⁶

¹Department of Geriatric Medicine, Institute of Geriatrics & Active Aging, Tan Tock Seng Hospital, Singapore; ²Health Outcomes and Medical Education Research, National Healthcare Group, Singapore; ³Department of Anaesthesiology, SingHealth, Singapore; ⁴Department of Palliative Medicine, Tan Tock Seng Hospital, Singapore; ⁵Centre for Health Science Education, Faculty of Health, Aarhus University, Denmark; ⁶Centre for Medical Education, National University of Singapore

Abstract

Background: In recent years, the Asia-Pacific region has become a hotbed of activity in medical education. Little is known about the progress in advancing the field through clarification research studies situated within a strong conceptual framework vis-a-vis descriptive (“What was done?”) and justification (“Did it work?”) research purposes. We aimed to determine the trend in clarification as opposed to descriptive and justification research purposes in the Asia-Pacific region over a 5-year period.

Methods: We conducted a scoping review of original research abstracts presented at the 2008, 2010 and 2012 Asia Pacific Medical Education Conferences. All eligible abstracts were classified as descriptive, justification or clarification using the modified Cook (2008) research purpose framework. All abstracts were reviewed by two researchers, with disagreement resolved by consensus. We performed trended Chi-square tests followed by logistic regression adjusted for covariates, to determine the longitudinal trend in clarification studies.

Results: Our sample comprised 517 abstracts (2008:136; 2010:195; 2012:186). There was a significant trend towards an increase in clarification studies from 2008 to 2012 (4.4% vs 8.7% vs 12.9%, p=.001), even after excluding submissions from non-Asian countries. When adjusted for covariates, abstracts in 2012 significantly predicted a clarification research purpose compared to 2008 (OR 2.74, 95% CI 1.04-7.23). A non-descriptive study design also significantly predicted clarification studies (OR 7.07, 95% CI 3.17-15.75).

Conclusions: Our results of a trend towards increased clarification research purpose affirm the longitudinal progress in quality of medical education research in the Asia-Pacific region. Efforts should be made to promote non-descriptive study designs that are undergirded in clarification research purpose.

Keywords:            Medical Education Research, Clarification Research Purpose, Asia-Pacific, Scoping Review

Practice Highlights

• We should aspire towards clarification studies situated within a strong conceptual framework vis-a-vis descriptive (“What was done?”) and justification (“Did it work?”) research purposes.
• There was a significant trend towards an increase in clarification studies from 2008 to 2012.
• A non-descriptive study design significantly predicted clarification studies.
• Efforts should be made to promote non-descriptive study designs that are undergirded in clarification research purpose.

I. INTRODUCTION

In recent years, the Asia-Pacific region has become a hotbed of activity in medical education (Chou, Chiu, Lai, Tsai & Tzeng, 2012; Fan et al., 2012; Bin Abdulrahman, Harden & Patrício, 2012; Samarasekera, Ooi, Yeo & Hooi, 2015). While much emphasis has been placed on improving standards in the scholarship of teaching (Zhang, Lee, Gruppen & Denian, 2013; E. Ahn & D. Ahn, 2014), comparatively less attention is paid to enhancing the quality of the research-related areas of scholarship, namely discovery, integration and application (Boyer, 1990; Obeidat, Alhaqwi & Abdulghani, 2015). With research being a key driver in informing the theory, practice and policy of medical education (Monrouxe & Rees, 2009; Frenk et al., 2010), we need systematic knowledge of the trends in quality of medical education research in the Asia-Pacific region.

In the ascertainment of quality of medical education research, it is imperative to recognize its uniqueness relative to its biomedical counterpart. Monrouxe & Rees (2009) eloquently pointed out that “medical education research is not a poor relation of medical research; it belongs to a different family altogether.” This necessitates an analogous broadening of the debate about quality in medical education research such that progress is judged from the “macroscopic” lenses of its contribution to the development of deeper insights and understanding of phenomenon to inform theory, practice and policy (Dornan, Peile & Spencer, 2009; Eva, 2009), as opposed to the prevalent “microscopic” focus on evidence hierarchies and detailed quality checklists. Such an approach emphasizes the importance of the conceptual framework to guide thinking about an idea, problem, or phenomenon by engendering generalizable knowledge from theories, models, evidence-based best practices or hypotheses (Rees & Monrouxe, 2010; Gibbs, Durning & Van Der Vleuten, 2011), such that research findings can be transferable to future practice and research (Bordage, 2009; Bunniss & Kelly, 2010).

Harnessing this macroscopic perspective, Cook, Bordage & Schmidt (2008) proposed a typology for classifying the purposes of medical education research, namely description, justification and clarification (Table 1). Description studies address the question: “What was done?” whereas justification studies seek to answer the question: “Did it work?” In contrast, clarification studies seek to answer the question: “Why or how did it work?” through the presence of a conceptual framework that can be affirmed or refuted by the study results (Cook et al., 2008; Ringsted et al., 2011). For instance, applying the Cook framework to an educational innovation that utilizes real-life elderly patients (as opposed to simulated patients) for teaching geriatrics assessment skills to junior medical students, a description study merely describes the educational intervention; a justification study compares the intervention group with a control group on pre-determined outcomes to determine if the intervention works; whereas a clarification study may invoke the situated learning theory to explain how the authenticity of exposure to real-life elderly patients complements aging simulation workshops by improving confidence and creating awareness of practical difficulties (Lim, Ng, Natesan, Wong & Tham, 2012). Applying this framework in his seminal study of 105 articles describing education experiments in 6 major journals, Cook et al. (2008) reported that clarification studies were uncommon (12%) relative to justification (72%) and description (16%) studies. A systematic survey of 850 experimental and non-experimental studies on problem-based learning (Schmidt, 2005) and a more recent study of research presentations at a medical education meeting in Mexico (García-Durán, Morales-López, Durante-Montiel, Jiménez & Sánchez-Mendiola, 2011) similarly reported a low rate of clarification studies.

The original iteration of the operationalized Cook framework was limited only to experimental studies (Cook, Beckman & Bordage, 2007; Cook et al., 2008). Expansion of the scope of the Cook framework to explicate trends in research purpose across non-experimental studies is sorely needed, as studies with a purely descriptive design without any underlying conceptual framework have historically constituted a significant proportion of the literature in medical education (Reed et al., 2008; García-Durán et al., 2011). It is therefore important to determine whether calls to enhance the quality of medical education research through more robust study designs beyond cross-sectional descriptive approaches (Gruppen, 2007) have resulted in concomitant shifts in research purpose beyond description. An opportunity to examine the complex relationship between research approach and research purpose, was afforded by the research compass framework first described by Ringsted et al. (2011). In this classification, research approach can be classified into 2 broad categories, namely experimental studies, with the main aim of justification to define appropriate interventions and outcomes, and non-experimental studies. The latter can be further sub-classified into explorative studies such as descriptive, qualitative and psychometric designs; observational studies which include cross-sectional, case-control and prospective cohort designs; and translational studies such as reviews, implementation studies and effect studies.

By successfully integrating the Cook and Ringsted frameworks, a recent systematic review reported that only one-eighth of medical education studies in the Asia-Pacific region have a clarification research purpose (Lim et al., 2017). This study also identified a clear study aim and non-descriptive research approach as important and potentially remediable areas to promote clarification studies. Little is known, however, of the trends in the quality of medical education research in the Asia-Pacific region and the factors which influence these trends. This represents a critical gap in the literature, since the “video” perspective afforded by longitudinal trends, as opposed to the “snapshot” of cross-sectional studies, can offer far richer insights into the impact of existing regional initiatives that promote scholarship in medical education (Lim, Chong & Sahadevan, 2007; Obeidat et al., 2015), such as centres/networks for research, national and regional conferences, and medical education journals.

In order to explore these issues, we conducted a scoping review of experimental and non-experimental original research abstracts presented at Asia Pacific Medical Education Conference (APMEC) from 2008 to 2012. The APMEC is a major regional conference that serves as an accessible “clearinghouse” that provides a timely and comprehensive snapshot of medical education research in the Asia-Pacific region (Samarasekera et al., 2015). Through this review, we aimed to determine the trend in clarification as opposed to descriptive and justification research purposes of APMEC original research abstracts during the study period. This can then serve as the basis from which we can shape a future research agenda to advance the regional progress of medical education research.

II. METHODS

A. Context

The first APMEC was held in 2004. Over the years, the APMEC has established itself as a major regional conference, attracting participants from South-East Asia, other parts of Asia (including Australia and New Zealand), as well as North America and Europe. Because we wanted to ascertain the trend in research purpose before the introduction of the Research in Essential Skills in Medical Education (RESME) course at APMEC in 2013, we studied the 5-year trend using abstracts from the 5^th, 7^th and 9^th APMECs in 2008, 2010 and 2012 respectively. We chose sampling of alternate years in order to make the workload manageable. The themes for the conferences were respectively “Medical education in a flat world”; “Excellence in medical education – quality in healthcare”; and “Towards transformative education for healthcare professionals in the 21^st century – nurturing lifelong habits of mind, behaviour and action.” Whilst the APMEC has traditionally employed the AMRaC (Aims, Methods, Results and Conclusion) abstract format, this was enforced only from 2009 onwards. The National Healthcare Group Institutional Review Board deemed this study exempt from review.

B. Study Eligibility

We studied original research abstarcts from the 5^th, 7^th and 9^th APMECs in 2008,2010 and 2012 respectively. Original research was defined as an educational intervention or trial; implementation of evidence based practice or guidelines; curriculum evaluation with subjective or objective outcomes; evaluation of an educational instrument or tool; surveys; qualitative research; and systematic reviews (Lim et al., 2017). Besides non-orignal research abstarcts, we also excluded abstracts from plenary lectures, workshops, special interest group meetings and discussions. Two authors (TKM and LWS) reviewed all abstracts to determine eligibility; any disagreement was resolved by discussion. Among 728 eligible abstracts, we excluded 211 that were not original research to yield the final sample of 517 abstracts comprising 136 (26.3%), 195 (37.7%), and 186 (36.0%) abstracts from 2008, 2010 and 2012 respectively (Figure 1). Each abstract was then rated independently and in duplicate. Disagreements were resolved by discussion, and if no consensus was reached, via adjudication by a third independent reviewer. Six reviewers were involved in data collection. We first performed a pilot study using randomly selected abstracts from APMEC 2011. After training and harmonization in the pilot phase, the six reviewers achieved good to excellent agreement in the coding (overall percentage agreement: 80-87%; ACI-statistic: 0.73 – 0.82) (Gwet, 1991).

C. Data Collection

1) Research Purpose

We classified research purpose as description, justification or clarification based upon modified definitions of the Cook framework (Table 1). The hallmark of clarification studies is the presence of a conceptual or theoretical framework that sets the stage for the presentation of the specific research question that drives the investigation being reported (Dine, Caelleigh & Shea, 2015). In our study, we classified the presence of a conceptual framework based on 3 crucial elements: 1) A theory, model, or hypothesis that asks “Why or how does it work?” 2) Transferability to new settings and future research; and 3) Confirmed or refuted by the results and/or conclusions of the study (Lim et al., 2017).

By integrating the Ringsted et al. (2011) framework, we expanded the original definitions of Cook et al. (2008) to accommodate both experimental and non-experimental studies. In the process, we were mindful to adhere to the original spirit of the definitions as far as possible (Lim et al., 2017). Firstly, even though the original definition of justification studies merited a comparison group, we waived this requirement for good quality psychometric studies for which we deemed that there was sufficient rigor in the measures of validity and reliability to answer the question “Does this assessment tool work?” Secondly, to address concerns that certain study designs, such as qualitative and observational studies, would be incongruent with a justification purpose, we delinked where appropriate the hierarchy of purpose from description to justification. Thus, a well-conducted observational study underpinned by a conceptual framework which explains the relationship between independent and dependent variables, would still qualify as a clarification study.

Figure 1. Flowchart of study inclusion and exclusion

*Modified from Cook et al., 2008
** Modified from Ringsted et al., 2011
Table 1. Typology of research purpose and research approach

2) Classification of Study Design

Our classification of study design was modified from the “research compass” framework (Table 1). Abstracts were classified into 2 categories: (1) Experimental, defined as any study in which researchers manipulated a variable (also known as the treatment, intervention or independent variable) to evaluate its impact on other (dependent) variables, including evaluation studies with experimental designs; and (2) Non-experimental, defined as all other studies that do not meet criteria for (1). Studies using mixed methods (for instance, an experimental design with a qualitative component) were classified according to the methodology that was deemed to be predominant.

Experimental studies were further sub-classified as experimental, quasi-experimental or pre-experimental according to established hierarchies of research designs (Creswell, 2013). We defined experimental studies by the presence of randomization; examples included factorial design, crossover design and randomized controlled trials. In contrast, for quasi-experimental studies, experimental and control groups were selected without random assignment of participants. Pre-experimental studies, namely single group pre-post and post-only designs, did not have a control group for comparison.

We also sub-classified non-experimental studies as descriptive, qualitative, psychometric, observational, or translational. Descriptive studies typically provide descriptions of phenomena, new initiatives or activities, such as curriculum design, instructional methods, assessment formats, and evaluation strategies (Ringsted et al., 2011). Because pure descriptive study designs may not strictly qualify as research by some authorities, they are ranked by default as lowest in the hierarchy of study designs (Crites et al., 2014). Hence, when two study designs were identified within the same study with one being descriptive, we coded based upon the “higher” non-descriptive study design.

3) Other Variables

We extracted data on other variables which may affect the quality of medical education research, such as presentation category, topic of medical education addressed, professional group being studied, country of the study population, number of institutions involved, Kirkpatrick’s learner outcomes (if applicable), and statement of study intent. We measured learner outcomes on 4 levels based upon Kirkpatrick’s expanded outcomes typology, namely learner reactions (level 1), modification of attitudes/perceptions (level 2a), modification of knowledge/skills (level 2b), behavioural change (level 3), change in organizational practice (level 4a) and benefits to patients or healthcare outcome (level 4b) (Kirkpatrick, 1967; Reeves, Boet, Zierler & Kitto, 2015). A study reporting more than one outcome was rated based upon the highest-level outcome, regardless of whether this was a primary or secondary outcome. The research question is arguably the most important part of any scholarly activity and is framed as a statement of study intent often in the form of a purpose, objective, goal, aim or hypothesis (Dine et al., 2015). We therefore collected data on whether there is an explicit statement of study intent, and if present, its quality as judged by correct location in the aims section; representation of study goals as opposed to mere stating of educational objectives; and completeness of information (i.e. whether any important objective was omitted).

D. Data Analysis

Results were summarized using descriptive statistics. Chi-square tests were used to compare differences in abstract characteristics between the years 2008, 2010 and 2012. To compare trends in research purpose from 2008 to 2012, we conducted trended Chi-square test with post-hoc pairwise comparisons. We also conducted sensitivity analysis by excluding non-Asian studies. Significant variables from bivariate analysis (P<.10) were included in binary logistic regression analysis to estimate the odds ratios (ORs) and 95% confidence intervals of a clarification study purpose associated with these factors. All analyses were performed using SPSS for Windows version 20.0 (SPSS Inc., Chicago, Illinois, USA). Statistical tests were two-tailed and conducted at 5% level of significance.

III. RESULTS

A. Abstract Characteristics

Among 517 eligible APMEC abstracts from 2008, 2010 and 2012, there was no significant difference in terms of presentation category, number of institutions studied, and Kirkpatrick’s learner outcomes (Table 2). In 2012, there were comparatively more abstracts from Singapore and fewer from the rest of South-East Asia, although the trend in number of abstracts from non-Asian countries remained stable. There were also more abstracts from postgraduate medical and non-medical participants in 2012 compared with 2010 (p=0.003). An explicit statement of study intent was absent or unclear in 45.6% of 2008 abstracts before the introduction of the AMRaC format, compared with 26.7% and 36.0% in 2010 and 2012 respectively (p=0.002). Compared with 2008, there was a trend away from descriptive study designs in 2010 and 2012 (2008–12: 66.9% vs 55.4% vs 54.8%, p=0.057).

*P < .01
Table 2. Comparison of abstract characteristics between 2008 and 2012

B. Research Purpose

There was a significant trend towards increase in clarification studies from 2008 to 2012 (4.4% vs 8.7% vs 12.9%; Trended c² = 11.12, p=.001) (table 3), with post-hoc analyses significant for clarification-descriptive (p=0.004) and justification-descriptive (p=0.04) but not clarification-justification (p=0.190) comparisons. The trend remained significant even after excluding 51 submissions from non-Asian countries (5.0% vs 7.4% vs 12.9%; Trended c² = 8.56, p=.003). In contrast, earlier studies that included a systematic survey of 850 experimental and non-experimental studies on problem-based learning (Schmidt, 2005), research presentations at a medical education meeting in Mexico (Garcia-Duran et al., 2011), and 105 articles describing education experiments in 6 major journals (Cook et al., 2008) reported prevalence figures of clarification studies that ranged between 0.4% to 12.0%.

*Trended Chi-square, P<0.01
^Abstracts from the Asia-Pacific Medical Education Conferences
Table 3. Comparison of trends in research purpose among various studies

C. Logistic Regression

Besides year of study, we also included in the regression model four independent variables with P<.10 in bivariate analysis, namely professional group, country of study, presence of clear study aims, and non-descriptive research approach (table 4). When adjusted for these covariates, abstracts in 2012 were still significantly more likely to have a clarification research purpose compared to 2008 (OR 2.74, 95% CI 1.04-7.23). The 2010 vs 2008 comparison was not signifcant (OR 1.63, 95% CI 0.60-4.41). Among the covariates, the only predictor of a clarification research purpose in multivariate analysis was a non-descriptive study design (OR 7.07, 95% CI 3.17-15.75).

*P < .05; **P < .01
Nagelkerke R square: 0.178
Adjusted for: Professional group, country of study, presence of clear study aims
Table 4. Logistic regression predicting likelihood of clarification research purpose

IV. DISCUSSION

Similar to developments elsewhere, there are calls for the medical education community in the Asia-Pacific Region to recognise and build on its existing research strengths in efforts to justify the value and legitimacy of medical education research (Dornan et al., 2009; Bin Abdulrahman, 2012). To our knowledge, this is the first study to report longitudinal trends in quality of medical education research using the Cook et al. (2008) research purpose framework. Our results demonstrated a trend towards increase in clarification studies from 2008 to 2012 even after excluding submissions from non-Asian countries, thus affirming the longitudinal progress in quality of medical education research in the Asia-Pacific region. In addition, we further explicated the observed trends by demonstrating that a non-descriptive research approach independently predicted a 7-fold increased odds of a clarification research purpose in multivariate analysis. The strengths of our study include duplicate review at all stages; clear and detailed description of the methods/procedures involved; adjustment for relevant covariates in multivariate analysis; and high inter-rater agreement among the coders.

We believe that this gradual regional shift in emphasis of research purpose within the Asian-Pacific field towards theory-driven and theory-building clarification research is a healthy and excellent development that is consonant with the wider evidence-based movement for research to inform, influence and impact the practice and policy of medical education (Eva & Lingard, 2008; Bordage, 2009). Theory can help structure and facilitate all aspects of the research process; it can help us generalise our study results beyond their immediate settings, promote collaboration within and between fields, open possibilities for new programs of research, attract funding and ultimately, sustain the virtuous cycle of quality medical education research (Gill & Griffin, 2009; Rees and Monrouxe, 2010; Kuper & Whitehead, 2013). As previously noted, trends in prevalence of clarification studies in our study compared very favourably with the 0.4% – 12.0% range which was reported in earlier cross-sectional studies in non-Asian settings (Schmidt, 2005; Cook et al., 2008; Garcia-Duran et al., 2011). It is reassuring that in our study, the longitudinal increase in clarification studies was accompanied by a corresponding decline in description research purpose. Taken together, this strongly supports the commendable progress made in promoting theory-grounded clarification studies even before the introduction of the RESME course at APMEC 2013. Possible explanations include the catalytic role of ongoing regional initiatives such as centres of excellence, research networks, national and regional conferences, and dedicated medical education journals to promote scholarship in medical education (Obeidat et al., 2015; Lim et al., 2017).

It is important to keep contextual developments in mind when interpreting the observed longitudinal trends in research purpose. For instance, adoption of the Accreditation Council for Graduate Medical Education International framework for postgraduate training in Singapore in 2009 could explain the surge in abstract submissions in the postgraduate medical category from Singapore at APMEC 2012 (Huggan et al., 2012; Samarasekera et al., 2015). Likewise, Interprofessional Core Curricula and Interprofessional Enrichment Activities were incorporated into Medical, Nursing, Dentistry, Pharmacy and Medical Social Work undergraduate programmes at the National University of Singapore (Jacobs et al., 2013), along with concomitant interest in interprofessionalism at the workplace (Tan, Adzahar, I. Lim, Chan, & W.S. Lim, 2014), can explain the encouraging rise in submissions from non-medical professional groups. Similarly, we believe that enforcing the structured format for abstract reporting in 2009 may have been the driving force behind the observed trends of improvement in aims statement. Our results corroborate evidence that structured formats improve the quality of reporting of research abstracts (Wong et al., 2005; Varpio, Amie & Richards, 2016), which is in turn positively associated with superior methodological quality (Cook, Levinson & Garside, 2011) and improved funding for medical education research (Reed et al., 2007). There is thus a case to be made for the consistent use of structured abstracts with relevant and thoughtful headings, for instance, a separate heading for conceptual framework or study hypothesis to spur the development of higher-order clarification studies.

The seven-fold increased odds of a clarification research purpose with non-descriptive research approach, supports the greater use of more rigorous non-descriptive study designs in medical education research (Gruppen, 2007). As opposed to strict adherence to a rigid hierarchy, the best study design would be the design that best answers the research question within a given context (Lim, 2013). Thus, awareness should be raised about the plurality of non-experimental non-descriptive approaches such as qualitative, psychometric, observational and translational research designs, which will encourage the broadening of the repertoire of research questions that can be asked to deepen understanding and advance the field. To borrow an analogy from the biomedical framework of translational science, this is akin to building up the T1 “bench to bedside” basic science evidence which can then inform educational practice and policy, and ultimately, patient care and outcomes (McGaghie, 2010).

Our findings should be interpreted within the limits of our study design. Our decision to sample APMEC abstracts may explain the predominance of abstracts from Singapore and the neighbouring South-East Asian countries. Nonetheless, the distribution of research purpose reported in our study is broadly consistent with existing literature. Future work could explore the generalizability of our results by comparing with other regional and international conferences (e.g. the Association for Medical Education in Europe [AMEE] conference) during the same timeframe. Our use of conference abstracts, which has a significant word constraint as compared to full-length papers, means that the validity of our findings is highly dependent on the reporting quality of the abstracts, such that our results may be more a reflection of the reporting quality rather than the actual quality of research. Notwithstanding, evidence affirming the positive relationship between reporting and ultimate methodological quality lends credence to the validity of assessing conference abstracts as an indirect quality indicator of research (Cook et al., 2011). Moreover, our research involved essential and fairly objective elements of reporting such as study aims and outcomes. Lastly, whilst our choice of timeframe before 2013 was effective in removing the direct influence of the RESME course on quality of APMEC abstracts, it will be prudent and salient to consider prospective studies to further ascertain if the trends in research purpose herein reported will prove sustainable.

V. CONCLUSION

Our results of a trend towards increased rigor of research purpose, even after excluding submissions from non-Asian countries, affirm the steady progress made in the quality of medical education research in the Asia-Pacific region in the last 5 years. We also highlighted the importance of promoting non-descriptive study designs that are undergirded in a clarification research purpose. We applaud recent regional initiatives to enhance the research capabilities of the Asian-Pacific community, such as the Asia Pacific Medical Education Network (APME-Net), the Asian Medical Education Association, regional journals such as The Asia-Pacific Scholar, formal research programs such as Masters and PhD degrees, and consensus statements to prioritize the research agenda. We encourage collaborative efforts with international partners to promote scholarship opportunities; examples include offering research courses such as RESME at regional courses, tie-ups with international journals to publish abstracts of regional conferences, and cross-cultural research to explicate mutual areas of interest involving pedagogy, issues and approaches (O’Sullivan, Stoddard & Kalishman, 2010; Suhoyo, Van Hell, Prihatiningsih, Kuks & Cohen-Schotanus, 2014). These initiatives augur well for fostering a culture of impactful educational research and scholarship in the Asia-Pacific region that aims to inform the theory, practice and policy of medical education through clarification studies.

Notes on Contributors

W. S. Lim conceived the study, and led the design, data collection, analysis, and drafting of the manuscript. K. M. Tham was involved in study selection, data collection and analysis. F. B. Adzahar, H. Y. Neo, W. C. Wong, and I. Lim were involved in the implementation of the study and data collection. C. Ringsted and D. D. Samarasekera were involved in the conception and design of the study, and provided revisions to the manuscript. All authors read and approved the final manuscript.

Ethical Approval

The study was approved by the institutional review board of the National Health Group (NHG DSRB Ref: 2012/00814).

Funding

The study was supported by an educational research grant (NHG-HOMER FY11/A08) from the National Healthcare Group Health Outcomes and Medical Education Research office.

Declaration of Interest

The authors report no declarations of interest.

References

Ahn, E., & Ahn, D. (2014). Beyond accreditation: Excellence in medical education. Medical Teacher, 36, 84-85.

Bin Abdulrahman, K., Harden, R., & Patrício, M. (2012). Medical education in Saudi Arabia: An exciting journey. Medical Teacher, 34, S4-S5.

Bin Abdulrahman, K. A. (2012). The value of medical education research in Saudi Arabia. Medical Teacher, 34, S1-S3.

Bordage, G. (2009). Conceptual frameworks to illuminate and magnify. Medical Education, 43(4), 312-319.

Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. New York, NY: Jossey-Bass Books.

Bunniss, S., & Kelly, D. R. (2010). Research paradigms in medical education research. Medical Education, 44(4), 358-366.

Chou, J. Y., Chiu, C. H., Lai, E., Tsai, D., & Tzeng, C. R. (2012). Medical education in Taiwan. Medical Teacher, 34, 187-191.

Cook, D. A., Beckman, T. J., & Bordage, G. (2007). Quality of reporting of experimental studies in medical education: A systematic review. Medical Education, 41(8), 737-745.

Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128-133.

Cook, D. A., Levinson, A. J., & Garside, S. (2011). Method and reporting quality in health professions education research: A systematic review. Medical Education, 45(3), 227-238.

Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Crites, G. E., Gaines, J. K., Cottrell, S., Kalishman, S., Gusic, M., Mavis, B., & Durning, S. J. (2014). Medical education scholarship: An introductory guide: AMEE Guide No. 89. Medical Teacher, 36(8), 657-674.

Dine, C. J., Caelleigh, A. S., & Shea, J. A. (2015). Chapter 6: Problem statement, conceptual framework, and research question. In Durning, S. J., & Carline, J. D. (Eds.), Review criteria for research manuscripts 2nd ed. (pp. 3-6). Washington, DC: Association of American Medical Colleges.

Dornan, T., Peile, E., & Spencer, J. (2009). In defence of the existing strengths of medical education research. Medical Education, 43, 388–391.

Eva, K. W., & Lingard, L. (2008). What’s next? A guiding question for educators engaged in educational research. Medical Education, 42(8), 752-754.

Eva, K. W. (2009). Broadening the debate about quality in medical education research. Medical Education, 43(4), 294-296.

Fan, A. P., Tran, D. T., Kosik, R. O., Mandell, G. A., Hsu, H. S., & Chen, Y. S. (2012). Medical education in Vietnam. Medical Teacher, 34, 103-107.

Frenk, J., Chen. L., Bhutta, Z. A., Cohen, J., Crisp, N., Evans, T., & Zurayk, H. (2010). Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet, 376, 1923–1958.

García-Durán, R., Morales-López, S., Durante-Montiel, I., Jiménez, M., & Sánchez-Mendiola, M. (2011). Type of research papers in medical education meetings in Mexico: An observational study. Paper presented at the Annual Meeting of the Association for Medical Education in Europe, Vienna, Austria.

Gibbs, T., Durning, S., & Van Der Vleuten, C. (2011). Theories in medical education: Towards creating a union between educational practice and research traditions. Medical teacher, 33(3), 183-187.

Gill, D., & Griffin, A. E. (2009). Reframing medical education research: Let’s make the publishable meaningful and the meaningful publishable. Medical Education, 43(10), 933-935.

Gruppen, L. D. (2007). Improving medical education research. Teaching and Learning In Medicine, 19(4), 331-335.

Gwet, K. (1991). Handbook of Inter-Rater Reliability. STATAXIS Publishing Company.

Huggan, P. J., Samarasekara, D. D., Archuleta, S., Khoo, S. M., Sim, J. H. J., Sin, C. S. P., & Ooi, S. B. S. (2012). The successful, rapid transition to a new model of graduate medical education in Singapore. Academic Medicine, 87, 1268–1273.

Jacobs, J. L., Samarasekera, D. D., Chui, W. K., Chan, S. Y., Wong, L. L., Liaw, S. Y., … & Chan, S. (2013). Building a successful platform for interprofessional education for health professions in an Asian university. Medical Teacher, 35, 343-347.

Kirkpatrick, D. L. (1967). Evaluation of training. In Craig, R. L. & Bittel, L. R. (Eds.), Training and development handbook (pp. 87-112). New York, USA: McGraw-Hill.

Kuper, A., & Whitehead, C. (2013). The practicality of theory. Academic Medicine, 88, 1594–1595.

Lim, W. S., Chong, M. S., & Sahadevan, S. (2007). Utility of the Clinical Dementia Rating in Asian populations. Clinical Medicine & Research, 5, 61-70.

Lim, W. S., Ng, A., Natesan, S., Wong, W. C., & Tham, K. Y. (2012). Situated learning enhances authenticity of learning experience of geriatric assessment skills among junior medical students. Annals Academy Medicine Singapore, 41, S37.

Lim, W. S. (2013). More about the focus on outcomes research in medical education. Academic Medicine, 88(8), 1052.

Lim, W. S., Tham, K. M., Adzahar, F. B., Neo, H.Y., Wong, W. C., & Ringsted, C. (2017). Determinants of clarification research purpose in medical education research: A systematic review. The Asian Pacific Scholar, 2, 8-17.

McGaghie, W. C. (2010). Medical education research as translational science. Science Translational Medicine, 2, 19cm8. https://doi.org/10.1126/scitranslmed.3000679.

Monrouxe, L. V., & Rees, C. E. (2009). Picking up the gauntlet: Constructing medical education as a social science. Medical Education, 43, 196-198.

Obeidat, A. S., Alhaqwi, A. I., & Abdulghani, H. M. (2015). Reprioritizing current research trends in medical education: A reflection on research activities in Saudi Arabia. Medical Teacher, 37, S5-S8.

O’Sullivan, P. S., Stoddard, H. A., & Kalishman, S. (2010). Collaborative research in medical education: A discussion of theory and practice. Medical Education, 44, 1175–1184.

Reed, D. A., Cook, D.A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007). Association between funding and quality of published medical education research. JAMA, 298(9), 1002-1009.

Reed, D. A., Beckman, T. J., Wright, S. M., Levine, R. B., Kern, D. E., & Cook, D. A. (2008). Predictive validity evidence for medical education research study quality instrument scores: Quality of submissions to JGIM’s Medical Education Special Issue. Journal of General Internal Medicine, 23(7), 903-907.

Rees, C. E., & Monrouxe, L. V. (2010). Theory in medical education research: How do we get there?. Medical Education, 44(4), 334-339.

Ringsted, C., Hodges, B., & Scherpbier, A. (2011). ‘The research compass’: An introduction to research in medical education: AMEE Guide No. 56. Medical Teacher, 33(9), 695-709.

Reeves, S., Boet, S., Zierler, B., & Kitto, S. (2015). Interprofessional Education and Practice Guide No. 3: Evaluating interprofessional education. Journal of Interprofessional Care, 29, 305-12.

Samarasekera, D. D., Ooi, S., Yeo, S. P., & Hooi, S. C. (2015). Medical education in Singapore. Medical Teacher, 37, 707-13.

Schmidt, H. G. (2005). Influence of research in practices in medical education: The case of problem-based learning. Paper presented at the Annual Meeting of the Association for Medical Education in Europe, Amsterdam, The Netherlands.

Suhoyo, Y., Van Hell, E. A., Prihatiningsih, T. S., Kuks, J. B. M., & Cohen-Schotanus, J. (2014). Exploring cultural differences in feedback processes and perceived instructiveness during clerkships: Replicating a Dutch study in Indonesia. Medical Teacher, 36, 223-229.

Tan, K. T., Adzahar, F. B., Lim, I., Chan, M., & Lim, W. S. (2014). Transactive memory system as a measure of collaborative practice in a geriatrics team: Implications for continuing interprofessional education. Journal of Interprofessional Care, 28, 239-45.

Varpio, L., Amie, J., & Richards, B. F. (2016). Writing competitive research conference abstracts: AMEE Guide no. 108. Medical Teacher, 38, 1-9.

Wong, H. L., Truong, D., Mahamed, A., Davidian, C., Rana, Z., & Einarson, T. R. (2005). Quality of structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association: A 10-year follow-up study. Current Medical Research Opinion, 21, 467-473.

Zhang, Q., Lee, L. M., Gruppen, L. D., & Denian, B. A. (2013). Medical education: Changes and perspectives. Medical Teacher, 35, 621–627.

*Wee Shiong Lim
11 Jalan Tan Tock Seng,
Department of Geriatric Medicine,
Tan Tock Seng Hospital, Singapore 308433
Telephone: +65 6359 6474
Fax: +65 63596294
Email: wee_shiong_lim@ttsh.com.sg