Submitted: 30 June 2022
Accepted: 31 October 2022
Published online: 4 July, TAPS 2023, 8(3), 26-34
https://doi.org/10.29060/TAPS.2023-8-3/OA2834

Noorjahan Haneem Md Hashim¹, Shairil Rahayu Ruslan¹, Ina Ismiarti Shariffuddin¹, Woon Lai Lim¹, Christina Phoay Lay Tan² & Vinod Pallath³

¹Department of Anaesthesiology, Faculty of Medicine, Universiti Malaya, Malaysia; ²Department of Primary Care Medicine, Faculty of Medicine, Universiti Malaya, Malaysia; ³Medical Education Research & Development Unit, Dean’s Office, Faculty of Medicine, Universiti Malaya, Malaysia

Abstract

Introduction: Examiner training is essential to ensure the trustworthiness of the examination process and results. The Anaesthesiology examiners’ training programme to standardise examination techniques and standards across seniority, subspecialty, and institutions was developed using McLean’s adaptation of Kern’s framework.

Methods: The programme was delivered through an online platform due to pandemic constraints. Key focus areas were Performance Dimension Training (PDT), Form-of-Reference Training (FORT) and factors affecting validity. Training methods included interactive lectures, facilitated discussions and experiential learning sessions using the rubrics created for the viva examination. The programme effectiveness was measured using the Kirkpatrick model for programme evaluation.

Results: Seven out of eleven participants rated the programme content as useful and relevant. Four participants showed improvement in the post-test, when compared to the pre-test. Five participants reported behavioural changes during the examination, either during the preparation or conduct of the examination.  Factors that contributed to this intervention’s effectiveness were identified through the MOAC (motivation, opportunities, abilities, and communality) model.

Conclusion: Though not all examiners attended the training session, all were committed to a fairer and transparent examination and motivated to ensure ease of the process. The success of any faculty development programme must be defined and the factors affecting it must be identified to ensure engagement and sustainability of the programme.

Keywords:           Medical Education, Health Profession Education, Examiner Training, Faculty Development, Assessment, MOAC Model, Programme Evaluation

Practice Highlights

• A faculty development initiative must be tailored to faculty’s learning needs and context.
• A simple framework of planning, implementing, and evaluating can be used to design a programme.
• Target outcome measures and evaluation plans must be included in the planning process.
• The Kirkpatrick model is a useful tool to use in programme evaluation: to answer if the programme has met its objectives.
• The MOAC model is a useful tool to explain why a programme has met its objective.

I. INTRODUCTION

Anaesthesiology specialist training in Malaysia comprises a 4-year clinical master’s programme. At the time of our workshop, five local public universities offer the programme. The course content is similar in all universities, but the course delivery may differ to align with each university’s rules and regulations. The summative examinations are held as a Conjoint Examination. Examiners include lecturers from all five universities, specialists from the Ministry of Health and external examiners from international Anaesthesiology training programmes. The examination consists of a written and a viva voce examination. The areas examined are the knowledge and cognitive skills in patient management.

A speciality training programme’s exit level assessment is an essential milestone for licensing. In our programme, the exit examination occurs at the end of the training before trainees practise independently in the healthcare system and are eligible for national specialist registration. Therefore, aligning the curriculum and assessment to licensing requirements is necessary.

Examiners play an important role during this high-stakes summative examination, making decisions regarding allowing graduating trainees to work as specialists in the community. Therefore, examiners must understand their role. In recent years, the anaesthesiology training programme providers in Malaysia have been taking measures to improve the validity of the examination. These include a stringent vetting process to ensure examination content reflects the syllabus, questions are unambiguous, and the examiners agree on the criteria for passing. However, previous examinations revealed that although examiners were clear on the aim of the examination, some utilised different assessment approaches, which were possibly coloured by personal and professional experiences, and thus needed constant calibration on the passing criteria. In addition, during examiner discussions, different examiners were found to have different skill levels in constructing focused higher-order questions and were not fully aware of potential cognitive biases that may affect the examination results.

These insights from previous examinations warranted a specific skill training session to ensure the trustworthiness of the examination process and results (Blew et al., 2010; Iqbal et al., 2010; Juul et al., 2019, Chapter 8, pp. 127-140; McLean et al., 2008). The examiners and the Specialty committee were keen to ensure that these issues were addressed with a training programme that complements the current on-the-job examiner training.

II. METHODS

An examiner training module was developed using McLean’s adaptation of Kern’s framework for curriculum development: Planning, Implementation and Evaluation (McLean et al., 2008; Thomas et al., 2015). A conceptual framework for the examiner training programme was drawn up from the programme’s conception stage to the evaluation of its outcome, as illustrated in Figure 1 (Steinert et al., 2016).

Figure 1: The conceptual framework for the examiner training programme and evaluation of its effectiveness

A. Planning

Three key focus areas were identified for the training programme: (1) Performance Dimension Training (PDT); (2) examiner calibration with Frame-Of-Reference Training (FORT); as well as (3) identifying factors affecting the validity of results and measures that can be taken to prevent them.

1) Performance dimension training (Feldman et al., 2012): The aim was to improve examination validity by reducing examiner errors or biases unrelated to the examinees’ targeted performance behaviours. Finalised marking schemes outlining competencies to be assessed required agreement by all the examiners ahead of time. These needed to be clearly defined and easily understood by all the examiners, and consistency was key to reducing examiner bias.

2) Examiner calibration with Frame-of-Reference Training (FORT) (Newman et al., 2016): Differing levels of experience among all the participants meant that there were differing expectations and performances among them. The examiner training programme needed to assist examiners in resetting expectations and criteria for assessing the candidates’ competencies. This examiner calibration was achieved using pre-recorded simulated viva sessions in which the participants rated candidates’ performances in each simulated viva session and received immediate feedback on their ability and criteria for scoring the candidates.

3) Identifying factors affecting the validity of results (Lineberry, 2019): Factors that may affect the validity of examination results may be related to construct underrepresentation (CU), where the results only reflect one part of an attribute being examined; or construct-irrelevant variance (CIV), where the results are being affected by areas or issues other than the attribute being examined.

An example of CU is sampling issues where only a limited area of the syllabus is examined, or an answer key is limited by the availability of evidence or content expertise.

Examples of CIV include the different ways a concept can be interpreted in different cultures or training centres, ambiguous questions, examiner cognitive biases, examiner fatigue, examinee language abilities, and examinees guessing or cheating. The examiner training programme was designed with the objectives listed in Table 1.

B. Implementation

The faculty intervention programme was designed as a one-day online programme to be attended by potential examiners for the Anaesthesiology Exit Examination. The programme objectives were prioritised from the needs assessment and designed based on Tekian & Norcini’s recommendations (Tekian & Norcini, 2016). Due to time constraints, training was performed using an online platform closer to the examination dates after obtaining university clearance on confidentiality regarding assessment issues.

The structure and contents of the examiner training programme are outlined in Table 2 and is further elaborated in Appendix A.

Based on the objectives, the organisers invited a multidisciplinary group of facilitators. The group consisted of anaesthesiologists, medical education experts in assessment and faculty development, and a technical and logistics support team to ensure efficient delivery of the online programme.

A multimodal approach to delivery was adopted to accommodate the diversity of the examiner group (gender, seniority, subspeciality, and examination experience). Explicit ground rules were agreed upon to underpin the safe and respectful learning environment. The educational strategy included interactive lectures, hands-on practice using rubrics created and calibration using video-assisted scenarios. The programme objectives were embedded and reinforced with each strategy. Pre- and post-tests were performed to help participants gauge their learning and assist the programme organisers in evaluating the participants’ learning.

This would be the first time such a programme was held within the local setting. Participants were all anaesthesiologists by profession, were actively involved in clinical duties within a tertiary hospital setting and consented to participate in this programme. As potential examiners, they all had prior experience as observers of the examination process, with the majority having previous experience as examiners as well.

The programme was organised during the peak of the COVID-19 pandemic and was managed on a fully online platform to ensure safety and minimise the time taken away from clinical duties. In addition, participants received protected time for this programme, a necessary luxury as anaesthesiologists were at the forefront of managing the pandemic.

C. Evaluation

The Kirkpatrick model (McLean et al., 2008; Newstrom, 1995;) was used to evaluate the programme’s effectiveness described and elaborated in Figure 2.

Figure 2: The Kirkpatrick model, elaborated for this programme

The MOAC model (Vollenbroek, 2019), expanded from the MOA (Marin-Garcia & Martinez Tomas, 2016) model by Blumberg & Pringle (Blumberg & Pringle, 1982) was used to examine factors that contributed to the effectiveness of the programme. Motivation, opportunity, ability, and communality are factors that drives action and performance.

III. RESULTS

Eleven participants attended the programme. These participants were examiners for the 2021 examinations from the university training centres and the Ministry of Health, Malaysia. Only one of the participants would be a first-time examiner in the Exit Examination. Four of the would-be examiners could not attend due to service priorities.

A. Level 1: Reaction

Seven of the eleven participants completed the programme evaluation form, which is openly available in Figshare at https://doi.org/10.6084/m9.figshare.20189309.v1 (Tan & Pallath, 2022). All of them rated the programme content as useful and relevant to their examination duties and stated that the content and presentations were pitched at the correct level, with appropriate visual aids and reference materials. The online learning opportunity was also rated as good.

All seven also aimed to make behavioural changes after attending the programme, as indicated below. Some of the excerpts include:

“I am more cognizant of the candidates’ understanding to questions and marking schemes”

“Yes. We definitely need the rubric/marking scheme for standardisation. Will also try to reduce all the possible biases as mentioned in the programme.”

“Yes, as I will be more agreeable to question standardisation in viva examination because it makes it fairer for the candidates.”

The participants also shared their understanding of the importance of standardisation and examiner training and would recommend this programme to be conducted annually. They agreed that the examiner training programme should be made mandatory for all new examiners, with the option of refresher courses for veteran examiners if appropriate.

B. Level 2: Learning

All 11 participants completed the pre-and post-tests. The data supporting these findings of this is openly available in Figshare at https://doi.org/10.6084/m9.figshare.20186582.v1 (Md Hashim, 2021). The participants’ marks in both tests are shown in Appendix B. The areas that showed improvement in scores were identifying why under-sampling is a problem and methods to prevent validity threats. Understanding the source of validity threat from cognitive biases showed a decline in scores (question 2 with scores of 11 to 8 and question 3 with scores of 10 to 8), respectively.

Comparing the post-test scores to pre-test scores, four participants showed improvement, four showed no change (one of the participants answered all questions correctly in both tests) and three participants showed a decline in test scores.

C. Level 3: Behavioural Change

Six participants responded to the follow-up questionnaire, which is openly available in Figshare at https://doi.org/10.6084/m9.figshare.20186591.v2 (Md Hashim, 2022). This questionnaire was administered about a year after the examiner training programme and after the completion of two examinations. Only one respondent did not make any self-perceived behavioural change while preparing the examination questions and conducting the viva examinations. Two respondents did not make any changes while marking or rating candidates.

The specific changes in the three areas of behavioural change that were consciously noted by the respondents were explored. Respondents reported increased awareness and being more systematic in question preparation, making questions more aligned to the curriculum, preparing better quality questions, and being more cognizant of candidates’ understanding of the questions.

They also reported being more objective and guided during marking and rating as the passing criteria were better defined and structured.

Regarding the conduct of the viva examination, respondents shared that they were better prepared during vetting and felt it was easier to rate candidates as the marking schemes and questions were standardised and could ensure candidates could answer all the required questions to pass.

D. Level 4: Results

The examiners who attended the training programme were able to prepare questions as blueprinted and were able to identify areas to be examined and provided recommended criteria for passing each question. This has led to a smooth vetting process and examination.

E. Factors Affecting Effectiveness

Even though the programme was not attended by all the potential examiners, those who did were committed to the idea of a fairer and more transparent examination process. This formed the motivation aspect of the model.

In terms of opportunity, protected training time is important, followed by prioritising the content of the training material according to the most pressing needs.

The ability aspect encompassed the abilities of the facilitators and participants. To emphasise the learning process, credible trainers were invited to this programme to facilitate the lectures and experiential learning sessions. In this aspect, the Faculty Development team comprised an experienced clinician, a basic medical scientist, and an anaesthesiologist, all with medical education qualifications and were vital in ensuring the success of this programme. The whole team was led by the Chief Examiner who focused on the dimensions to be tested and calibrated, while simultaneously managing the expectations of the examiners and their abilities to give and accept feedback. Communication and the skill to be receptive to the proposed changes were also crucial to make the intervention work.

In terms of communality, all the participants were of similar professional backgrounds and shared the common realisation that this training programme was essential and would only yield positive results. Hence this ensured the programme’s overall success.

IV. DISCUSSION

The progressive change seen in this attempt to improve the examination system is aligned with the general progress in medical education. Training of examiners is important (Holmboe et al., 2011), as it is not the tool used for assessment, but rather the person using the tool, that makes the difference. As it is difficult to design the ‘perfect tool’ for performance tests and redesigning a tool only changes 10% of the variance in rating (Holmboe et al., 2011; Williams et al., 2003), educators must now train the faculty in observation and assessment. It is not irrational to extrapolate this effect on written and oral examinations. Holmboe et al. (2011) also share the reasons for a training programme for assessors, which are changing curriculum structure, content and delivery and emerging evidence regarding assessment, building a system reserve, utilising training programmes as opportunities to identify and engage change agents and allow the faculty to form a mental picture of how changes will affect them and improve practice. Enlisting the help and support of a respected faculty member during training will promote the depth and breadth of change.

Khera et al. (2005) described their paediatric examination experiences, in which the Royal College of Paediatrics and Child Health defined examiners’ competencies, selection process and training programme components. The training programme included principles of assessment, examination design, writing questions, interpersonal skills, professional attributes, managing diversity, and assessing the examiners’ skills. They believe these contents will ensure the assessment is valid, reliable, and fair. As Anaesthesiology examiners have different knowledge levels and experiences, it had been crucial to assess their learning needs and provide them with appropriate learning opportunities.

In the emergency brought on by the COVID-19 pandemic, online training was the safest and most feasible platform for conducting this programme. Online faculty development activities have the perceived advantages of being convenient, flexible, and allowing interdisciplinary interaction and providing an experience of being an online student(Cook & Steinert, 2013). Forming the facilitation team together with the dedicated technical and logistics team and creating a chat group prior to conducting the programme were key in anticipating and handling communication and technical issues (Cook & Steinert, 2013).

Though participants were engaged and the results of the workshop were encouraging, the programme delivery and the content will be reviewed based on the feedback received. The convenience of an online activity must be balanced with the participant engagement and facilitator presence of a face-to-face-activity. Since the results of both methods of delivery differs (Arias et al., 2018; Daniel, 2014; Kemp & Grieve, 2014), the best solution may to ask the participants what would best work for them, as they are adult learners and experienced examiners. The programme must be designed with participants involvement, with opportunities to participate and engaging facilitators and support teams that would be able to support the participants’ learning need (Singh et al., 2022).

At the end of the programme, the effectiveness of the programme was measured by referencing the Kirkpatrick model. The Kirkpatrick model (Newstrom, 1995; Steinert et al., 2006) was the most helpful in helping us identify the success of the intervention, which included behavioural change. Measuring behavioural change and impact on the examination results, organisational changes and changes in student learning may be difficult and may not be directly caused by a single intervention (McLean et al., 2008). The key, is perhaps to involve examiners, students and other stakeholders in the evaluation process, using various validated tools, and to ensure that the effort is ongoing, with sustained support, guidance and feedback (McLean et al., 2008).

To explain the overall effectiveness of the programme (with regards to reaction, learning and behavioural change), the MOAC model (Vollenbroek, 2019) expanded from the original MOA model was used. The MOAC model not only describes factors that affect an individual’s performance in a group, but also the group behaviour.

Motivation is an important driving force of action, and members are more motivated when a subject becomes relevant on a personal level, leading to action. The motivation to be informed and to improve has led to active participation in the knowledge sharing session, processing new information presented in the programme and adopting changes learnt during the programme. Presence of a group of motivated individuals with the same goals supported each other’s learning.

Opportunity, especially time, space and resources, must be allocated to reflect the value and relevance of any activity. Work autonomy, allows professionals to engage in what they consider relevant or important, and be accountable for their work outcomes. Facilitating conditions, for example, technology, facilitators, and a platform to practise what is being learnt are also important aspects of opportunity. Allowing protected time with the appropriate facilitating conditions, indicates institutional support and has enabled participants to fully optimise the learning experience.

Ability positively affects knowledge exchange and willingness to participate. Having prior knowledge improves a participant’s ability to absorb and utilise new knowledge. The programme participants, being experienced clinical teachers and examiners are fully aware of their capabilities and are able to process and share important information. Experienced faculty development facilitators who are also clinical teachers and examiners were able to identify areas to focus and provide relevant examples for application.

Communality is the added dimension to the original MOA model. Participants of this programme are members in a complex system, who already know each other. Having shared identity, language and challenges have allowed them to develop trust while pursuing the common goal of improving the system they were working in. This facilitated knowledge sharing and behavioural change.

The limitation in our programme is the small sample size. However, we believe that is important to review the effectiveness of a programme, especially with regards to behavioural change, and to share how other programmes can benefit from using the frameworks we shared. The findings from this programme will also inform how we conduct future faculty development programmes. With pandemic restrictions lifted, we hope to conduct this programme face-to-face, to facilitate engagement and communication.

V. CONCLUSION

For this faculty development programme to succeed, targets for success must first be defined and factors that contribute to its success need to be identified. This will ensure active engagement from the participants and promote the sustainability of the programme.

Notes on Contributors

Noorjahan Haneem Md Hashim designed the programme, assisted in content creation, curation and matching learning activities, moderated the programme, and conceptualised and wrote this manuscript.

Shairil Rahayu Ruslan participated as a committee of the programme, assisted as a simulated candidate during the training sessions, as well as contributed to the conceptualisation, writing, and formatting of this manuscript. She also compiled the bibliography and cross-checked the references for this manuscript.

Ina Ismiarti Shariffuddin created the opportunity for the programme (Specialty board and interdisciplinary buy-in, department funding), prioritised the programme learning outcomes, chaired the programme, and contributed to the writing and review of this manuscript.

Woon Lai Lim participated as a committee member of the programme and contributed to the writing of this manuscript.

Christina Phoay Lay Tan designed and conducted the faculty development training programme, and reviewed and contributed to the writing of this manuscript. She also cross-checked the references for this manuscript.

Vinod Pallath designed and conducted the faculty development training programme, and reviewed and contributed to the writing of this manuscript.

All authors verified and approved the final version of the manuscript.

Ethical Approval

Ethical approval was applied for the follow-up questionnaire that was distributed to the participants, which was approved on the 6th of May 2022 (Reference number: UM.TNC2/UMREC_1879). The programme evaluation and pre- and post-tests are accepted as part of the programme evaluation procedures.

Data Availability

De-identified individual participant data collected are available in the Figshare repository immediately after publication without an end date, as below :

https://doi.org/10.6084/m9.figshare.20189309.v1

https://doi.org/10.6084/m9.figshare.20186582.v1

https://doi.org/10.6084/m9.figshare.20186591.v2

The authors confirm that all data underlying the findings are freely available for view from the Figshare data repository. However, the reuse and resharing of the programme evaluation form, pre- and posttest questions, as well as followup questionnaire, despite being easily accessible from the data repository, should warrant a reasonable request from the corresponding author out of courtesy.

Acknowledgement

The authors would like to acknowledge Dr Selvan Segaran and Dr Siti Nur Jawahir Rosli from the Medical Education, Research and Development Unit (MERDU) for their logistics and technical support in all stages of this programme; Professor Dr Jamuna Vadivelu, Head, MERDU for her insight and support; Dr Nur Azreen Hussain and Dr Wan Aizat Wan Zakaria from the Department of Anaesthesiology, UMMC and UM, for their acting skills in the training videos; and the Visibility and Communication Unit, Faculty of Medicine, Universiti Malaya for their video editing services.

Funding

There is no funding source for this manuscript.

Declaration of Interest

There are no conflicts of interest among the authors of this manuscript.

References

Arias, J. J., Swinton, J., & Anderson, K. (2018). Online vs. face-to-face: A comparison of student outcomes with random assignment. E-Journal of Business Education & Scholarship of Teaching, 12(2), 1–23. https://eric.ed.gov/?id=EJ1193426

Blew, P., Muir, J. G., & Naik, V. N. (2010). The evolving Royal College examination in anesthesiology. Canadian Journal of Anesthesia/Journal canadien d’anesthésie, 57(9), 804-810. https://doi.org/10.1007/s12630-010-9341-1

Blumberg, M., & Pringle, C. D. (1982). The missing opportunity in organizational research: Some implications for a theory of work performance. The Academy of Management Review, 7(4), 560–569. https://doi.org/10.2307/257222

Cook, D. A., & Steinert, Y. (2013). Online learning for faculty development: A review of the literature. Medical Teacher, 35(11), 930–937. https://doi.org/10.3109/0142159X.2013.827328

Daniel, C. M. (2014). Comparing online and face-to-face professional development [Doctoral dissertation, Nova Southeastern University]. https://doi.org/10.13140/2.1.3157.5042

Feldman, M., Lazzara, E. H., Vanderbilt, A. A., & DiazGranados, D. (2012). Rater training to support high-stakes simulation-based assessments. Journal of Continuing Education in the Health Professions, 32(4), 279–286. https://doi.org/10.1002/chp.21156

Holmboe, E. S., Ward, D. S., Reznick, R. K., Katsufrakis, P. J., Leslie, K. M., Patel, V. L., Ray, D. D., & Nelson, E. A. (2011). Faculty development in assessment: The missing link in competency-based medical education. Academic Medicine, 86(4), 460–467. https://doi.org/10.1097/ACM.0b013e31820cb2a7

Iqbal, I., Naqvi, S., Abeysundara, L., & Narula, A. (2010). The value of oral assessments: A review. The Bulletin of the Royal College of Surgeons of England, 92(7), 1–6. https://doi.org/10.1308/147363510×511030

Juul, D., Yudkowsky, R., & Tekian, A. (2019). Oral Examinations. In R. Yudkowsky, Y. S. Park, & S. M. Downing (Eds.), Assessment in Health Professions Education. Routledge. https://doi.org/10.4324/9781315166902-8

Kemp, N., & Grieve, R. (2014). Face-to-face or face-to-screen? Undergraduates’ opinions and test performance in classroom vs. online learning. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.01278

Khera, N., Davies, H., Davies, H., Lissauer, T., Skuse, D., Wakeford, R., & Stroobant, J. (2005). How should paediatric examiners be trained? Archives of Disease in Childhood, 90(1), 43–47. https://doi.org/10.1136/adc.2004.055103

Lineberry, M. (2019). Validity and quality. Assessment in Health Professions Education, 17-32. https://doi.org/10.4324/9781315166902-2

Marin-Garcia, J. A., & Martinez Tomas, J. (2016). Deconstructing AMO framework: A systematic review. Intangible Capital, 12(4), 1040. https://doi.org/10.3926/ic.838

McLean, M., Cilliers, F., & Van Wyk, J. M. (2008). Faculty development: Yesterday, today and tomorrow. Medical Teacher, 30(6), 555–584. https://doi.org/10.1080/01421590802109834

Md Hashim, N. H. (2021). Pre- and Post-test [Dataset]. Figshare. https://doi.org/10.6084/m9.figshare.20186582.v1

Md Hashim, N. H. (2022). Followup Questionnaire [Dataset]. Figshare. https://doi.org/10.6084/m9.figshare.20186591.v2

Newman, L. R., Brodsky, D., Jones, R. N., Schwartzstein, R. M., Atkins, K. M., & Roberts, D. H. (2016). Frame-of-reference training: Establishing reliable assessment of teaching effectiveness. Journal of Continuing Education in the Health Professions, 36(3), 206–210. https://doi.org/10.1097/CEH.0000000000000086

Newstrom, J. W. (1995). Evaluating training programs: The four levels, by Donald L. Kirkpatrick. (1994). San Francisco: Berrett-Koehler. 229 pp., $32.95 cloth. Human Resource Development Quarterly, 6(3), 317-320. https://doi.org/10.1002/hrdq.3920060310

Singh, J., Evans, E., Reed, A., Karch, L., Qualey, K., Singh, L., & Wiersma, H. (2022). Online, hybrid, and face-to-face learning through the eyes of faculty, students, administrators, and instructional designers: Lessons learned and directions for the post-vaccine and post-pandemic/COVID-19 World. Journal of Educational Technology Systems, 50(3), 301–326. https://doi.org/10.1177/00472395211063754

Steinert, Y., Mann, K., Anderson, B., Barnett, B. M., Centeno, A., Naismith, L., Prideaux, D., Spencer, J., Tullo, E., Viggiano, T., Ward, H., & Dolmans, D. (2016). A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Medical Teacher, 38(8), 769-786. https://doi.org/10.1080/0142159x.2016.1181851

Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher, 28(6), 497–526. https://doi.org/10.1080/01421590600902976

Tan, C. P. L., & Pallath, V. (2022). Workshop Evaluation Form [Dataset]. Figshare. https://doi.org/10.6084/m9.figshare.20189309.v1

Tekian, A., & Norcini, J. J. (2016). Faculty development in assessment : What the faculty need to know and do. In M. Mentkowski, P.F. Wimmers (Eds.), Assessing Competence in Professional Performance across Disciplines and Professions (1st ed., pp. 355–374). Springer Cham. https://doi.org/10.1007/978-3-319-30064-1

Thomas, P. A., Kern, D. E., Hughes, M. T., & Chen, B. Y. (2015). Curriculum development for medical education : A six-step approach. John Hopkins University Press. https://jhu.pure.elsevier.com/en/publications/curriculum-development-for-medical-education-a-six-step-approach

Vollenbroek, W. B. (2019). Communities of Practice: Beyond the Hype – Analysing the Developments in Communities of Practice at Work [Doctoral dissertation, University of Twente]. https://doi.org/10.3990/1.9789036548205

Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). SPECIAL ARTICLE: Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine, 15(4), 270–292. https://doi.org/10.1207/S15328015TLM1504_11

*Shairil Rahayu Ruslan
50604, Kuala Lumpur,
Malaysia
03-79492052 / 012-3291074
Email: shairilrahayu@gmail.com, shairil@ummc.edu.my

Submitted: 23 August 2022
Accepted: 3 January 2023
Published online: 4 July, TAPS 2023, 8(3), 15-25
https://doi.org/10.29060/TAPS.2023-8-3/OA2871

Iroro Enameguolo Yarhere¹, Tudor Chinnah² & Uche Chineze³

¹Department of Paediatrics, College of Health Sciences, University of Port Harcourt, Nigeria; ²Department of Anatomy, University of Exeter, United Kingdom; ³Department of Education and Curriculum studies, University of Port Harcourt, Nigeria

Abstract

Introduction: This study aimed to compare the paediatric endocrinology curriculum across Southern Nigeria medical schools, using reports from learners. It also checked the learners’ perceptions about different learning patterns and competency in some expected core skills.

Methods: This mixed (quantitative and qualitative) study was conducted with 7 medical schools in Southern Nigeria. A multi-staged randomized selection of schools and respondents, was adopted for a focus group discussion (FGD), and the information derived was used to develop a semi-structured questionnaire, which 314 doctors submitted. The FGD discussed rotation patterns, completion rates of topics and perceptions for some skills. These themes were included in the forms for general survey, and Likert scale was used to assess competency in skills. Data generated was analysed using statistical package for social sciences, SPSS 24, and p values < 0.05 were considered significant

Results: Lectures and topics had various completion rates, 42.6% – 98%, highest being “diabetes mellitus”. Endocrinology rotation was completed by 58.6% of respondents, and 58 – 78 % perceived competency in growth measurement and charting. Significantly more learners, 46.6% who had staggered posting got correct matching of Tanner staging, versus learners who had block posting, 33.3%, p = 0.018.

Conclusion: Respondents reported high variability in the implementation of the recommended guidelines for paediatric endocrinology curriculum between schools in Southern Nigeria. Variabilities were in the courses’ completion, learners’ skills exposure and how much hands-on were allowed in various skills acquisitions. This variability will hamper the core objectives of human capital development should the trend continue.

Keywords:          Paediatric Endocrinology Curriculum, Perception, Compliance, Completion Rate, Learners

Practice Highlights

• Medical and dental council of Nigeria has a recommended benchmark for minimum academic standards in all medical schools to which total compliance is expected.
• Evaluation of paediatric endocrinology curriculum content and training methods was conducted using reports from learners.
• Variability in the content, and training methods of the intended competency were reported across medical schools.
• Compliance rate of the recommended curriculum was less than 50% in some contents and some learners reported low skill performance training.
• The lack of uniformity can prevent achievement of the overarching objective of the curriculum in Nigeria with wide variations in competence among graduating doctors.

I. INTRODUCTION

The primary aim of the Medical and Dental council of Nigeria (MDCN) undergraduate curriculum is “to train doctors and dentists who can work effectively in a health team to provide comprehensive health care to individuals in any community in the nation, and keep up to date on issues of global health” (Federal Ministry of Health of Nigeria, 2012). In Nigeria today, there are 49 federal, 59 states and 111 private universities, and 44 of these have full or partially accredited medical schools and while these schools have a prescribed curriculum, some are not following explicitly (Federal Ministry of Health of Nigeria, 2012). This curriculum advocates for universities to develop syllabus to meet the benchmark for minimum academic standards (BMAS) across schools, however there is no uniform template developed for assessing graduates to know how their competence converge as is applicable in United States of America (USA), Canada and United Kingdom (UK) (Santen et al., 2019; Shah et al., 2020; Sosna et al., 2021).  Diabetes mellitus, thyroid disorders, puberty, rickets and growth abnormalities are topics included in the MDCN paediatric curriculum under endocrinology which learners are expected to acquire competence in cognitive and psychomotor skills to diagnose and treat or refer appropriately children presenting with these diseases.

A. Problem

Most deaths from diseases in Nigeria and other resource-limited countries are consequent upon general public ignorance of disease, late presentation to the health care systems, poverty and lack of funds to access healthcare facilities and reduced knowledge of some disease patterns by the healthcare providers (Yarhere & Nte, 2018). Addressing the gaps in reduced knowledge can be done by developing competency-based curriculum for all graduating doctors to have as near-similar competence as possible but achieving this may not be feasible. Training activities are not uniform throughout medical schools in Nigeria and elsewhere, and depend on schools’ vision, mission and objectives, and the structures and processes put in place. There are barriers to positive implementation across schools including but not limited to individual school’s determination of what is relevant in the curriculum, access to the materials needed to teach the curriculum content and getting trainers to use these curriculums (Polikoff, 2018). The lack of uniformity of curriculum across universities may not be contending issues, but when the graduating doctors have varying degree of competencies in skills and cognition, then a template for imparting uniform and up to date knowledge and to evaluate this is needed to find ways of reducing the variability (McManus, 2003; McManus et al., 2020; Rimmer, 2014).

The curriculum uniformity across schools is one way of improving competency and thus, healthcare standards, and there is need to explore this uniformity or diversity within the paediatric undergraduate training. In some countries, there is a uniform board certification examination before doctors can practice and this is also done for doctors immigrating into these countries (Hohmann & Tetsworth, 2018; Puri et al., 2021; Tiffin et al., 2017; van Zanten et al., 2022) but Nigeria is exempt from this uniform exit examination. This uniform exit board examination makes these schools align course contents, and therefore reduces the variabilities between medical schools and undergraduate training.

B. Curriculum Evaluation for Change or Improvement

Curriculum evaluation is a means by which educators understand whether the curriculum used to train learners is working as intended, and whether there is need to change the entire programme or redesign aspects (Burton & McDonald, 2001; Ornstein & Hunkins, 2009). It is also a way of identifying deficiencies in training syllabus across universities, (Rufai et al., 2016) or whether compliance to a curriculum is being achieved (Grant, 2014; Olson et al., 2000). Kirkpatrick’s curriculum evaluation method is widely acceptable in medical education using the 4 steps; learners’ reaction or satisfaction, knowledge, behavioural changes and results or impact, and in Nigeria, for paediatric endocrinology, this has not been done (Alsalamah, 2021; Bates, 2004).

Universities have variabilities in organisation, students’ numbers in classes, duration of specific posting, posting types and whether the courses are elective or core. In medical schools in Nigeria, paediatric postings are undertaken in the 5^th or 6^th year of a 6-year programme. While some stagger the posting to be done within the last 2 years, others do theirs in the 5^th or the 6^th year exclusively, and the extent of these variabilities and how they affect the training processes and products has not been evaluated in Nigeria and this can be done using learners’ or graduates’ perceptions.

The aim of this research was to evaluate learners’ report and perception of some aspects of the paediatric endocrinology curriculum contents and learning methods across Southern Nigeria medical schools. Endocrinology was taken from the paediatric course to reduce the volume of information to be analysed.

II. METHODS

This was a cross sectional study design with qualitative and quantitative data analyses, evaluating learners’ report and their perception of the curriculum being used by various medical schools in Southern Nigeria to deliver the MDCN paediatric endocrinology curriculum. Survey was conducted across 10 medical schools in Southern Nigeria that have learners who have either completed their final year, or are doing their internship. Two steps were used to retrieve the information needed; a focus group discussion of sampled learners, and a questionnaire survey sent out to randomly selected respondents and these 2 methods complemented each other. The focus group discussion was used to explore in depth, the minds of the respondents and what they perceived was being done well and what needed to be changed in the syllabus in their respective schools. The questionnaire survey was then used to collect reports and perceptions from a wider set of learners who had completed their paediatric posting within the past 6 – 12 months. Some of these were already doing their internship and others were in their final year in preparation for their final examinations.

Sample size for respondents will be calculated using the formula:

N = (Z score)² x SD x (1 – SD) 

                       (CI)²

Z score = 1.96, SD (standard deviation of the mean) = estimated at ± 0.5, Confidence interval = 0.05

= 384 respondents, with an attrition rate of 10% will be added 10% of 384 = 38

384 + 38 = 422 respondents.

A. Sampling Technique

Multi-staged sampling technique was used to determine the schools, and respondents that participated in the study. There are 29 Southern Universities with medical / health colleges and 16 of these had more than 50 learners in their final year or had graduated. Ten schools were randomly selected using the excel formula [= rand ()], and a proportionate stratified sampling was done using the matriculation numbers of the students in each school to arrive at 422 respondents. Total number of learners that studied paediatrics in various institutions was 800; Ibadan 150, Port Harcourt, 128, Lagos 128, Niger delta University 69, UNN Enugu 128, University of Benin 128, Others 69. From the total number of learners in each school, selected learners and interns were sent the questionnaire using their email addresses. Selection for the FGD was done using simple random sampling from each school and these were sent separate emails with details for the meeting.

B. Focus Group Discussions Process

Focus group discussion was conducted with the respondents using zoom video platform, and the process lasted for 2 hours, 30 minutes. Ten learners’ representatives from the selected schools were contacted for this FGD, however, 7 (70%) agreed to participate after several email reminders.  The interview was semi-structured with a flexible topic guide, which covered issues relating to the respondents’ views and opinions on the curriculum in paediatric endocrinology; description of posting type in each school, whether block, or staggered, topics received and/or completed, perception of their competence in a key psychomotor skill. The focus group interview discussions were recorded in the zoom meeting platform and transcribed verbatim. The data were analysed using the thematic framework content analyses method. The themes generated were categorised into; 1. Lecture contents and completion rate, 2. Types of paediatric rotation and posting, 3. Skill competence acquisition and clinical postings. Their perceptions about these themes were also sought and discussed. The transcription of the groups’ discussions was reviewed by IY and TC to help categorise the data and pull-out important quotes used.

C. Questionnaire Survey

Following thematic analyses of the FGD, the themes generated were converted to questions in a survey for a larger sample population. Themes generated were the type of paediatric posting, rotations through units in the departments and paediatric endocrinology topics, training methods and competency acquired. Demographic characteristics of responders such as level/year of study, age, gender and university of study were collected. The respondents were also asked to select topics from a poll, included in their paediatric endocrinology syllabus, with result in Figure 1, and to state the various methods used to learn growth and growth disorders in their schools. A means of assessing cognitive (recall) skills of the learners was conducted using animated pictures of Tanner staging and matching-type multiple choice, and the responses were crossmatched with the type of posting learners were exposed to, i.e. block posting or staggered posting. Tanner staging was chosen as it cuts across general paediatrics and endocrinology as part of growth and puberty (endocrinology).

Data retrieved were analysed statistically by using chi-square test, and Pearson correlation for categorical variables. The level of competence perceived by learners in height measurement and charting on growth chart was retrieved using 5-point Likert scale (where 1 = not competent; 2 = low competence; 3 = neutral; 4 = competent; and 5 = proficient). The association between level of competence and whether learners rotated through paediatric endocrinology was checked using Pearson’s correlation test. For all statistics, p value < 0.05 was considered significant.

D. Ethics

The research commenced after the Research Ethics committee of the University of Port Harcourt granted approval (UPH/CEREMAD/REC/MM80/056). Verbal informed consent was obtained from the participants during the focus group discussion, who also gave consent for video and recording of the process. Informed consent was also obtained from all participants who filled and submitted the online survey. The focus group discussants received N3,000 ($10) for internet data only as monetary compensation.

III. RESULTS

There were 314 learners from the 422 calculated sample size, responded to the questionnaire survey, giving a response rate of 74.4%. There were more final year respondents than early career doctors and more of the respondents were in the age bracket 20 – 24 years, with a mean of 25.02 ±2.71 years. The male: female ratio was 1:1.01, and the data that support the findings of this study are available in Figshare at https://doi.org/10.6084 /m9.figshare.20730937.v1 (Yarhere et al., 2022).

RESPONDENTS	Frequency	Percentage
Year of study
Early career doctor (graduate/intern)	130	41.4	p = 0.002
Final year	184	58.6
University attended (calculated cohort)
University of Port Harcourt (63)	62	19.7
Niger Delta University (54)	54	17.2
University of Ibadan (76)	50	15.9
University of Benin (65)	44	14.0
University of Lagos (65)	40	12.7
University of Nigeria (65)	42	13.4
Other western Universities (34)	22	7.0
Age
20-24	140	44.6
25-29	162	51.6
>=30	12	3.8
Mean	25.02 ± 2.71
Gender
Male	152	48.4	p = 0.612
Female	162	51.6

Table 1. Demographic characteristics of all respondents and the universities attended

There were basically 2 modes of paediatric posting in the institutions sampled; 4-months block posting where respondents have a month of didactic lectures and 3 months of clinical rotations through various units in the Paediatric departments, and 4 months of staggered rotations with junior and senior postings in the clinical classes. While some learners rotated through all the units (core and electives) in the departments, some went through core units, emergency and neonatal units, and 2 other units randomly selected for the respondents by the departments.

C. Learners’ Responses to Rotation through Paediatrics and Posting Types

Participant 2 shared:

The way it works in University of xxxx, we rotate through 2 elective postings with core (CHEW and SCBU) postings in the junior and senior postings. These elective postings are randomly selected by the department (meaning heads or coordinators). I did neurology and gastroenterology in my junior posting and haemato-oncology and I really can’t remember the other one in my senior posting.

 “Will I be wrong to say you did not see a patient with Diabetic keto acidosis?”

Facilitator

“I saw a child with diabetic keto acidosis in the ward but it wasn’t my unit managing the patient. I only went to the ward to do some other thing.”

Participant 2 

“If you were given the opportunity to design a curriculum or programme for your university, will you prefer what is being practiced now, or will you rather have every student go through every unit and get titbits from each unit?”

Facilitator

Participant 2 responded:

Yes, I will prefer that situation where you get to be exposed to every unit in the department but …. emmm, that creates a problem because you may be in a unit for a week, and no patient comes in but the next group rotating to the unit gets to see many patients. I would want to suggest that perhaps, instead of focusing on more of clinical posting, that a unified tutorial class which will expose everyone to the core diseases in the various disciplines.

Table 2 corroborates the information given by the focus group discussants. Testing the competency outcome in either method can give some estimated guess as to which is better, however, there are several confounding factors that will not allow fair comparison (See Table 3).

Though there were differences in the mode of paediatric postings where staggered or block, c² = 4.59, p = 0.032. the difference in proportion of respondents who had core and selected elective posting as against all units posting was not significant, c² = 0.318, p = 0.573.

In the 2×2 table above where recall was tested in the learners based on their paediatric posting type, higher percentage of those who had staggered posting got the correct matching of Tanner stage, and the difference was significant, c² = 5.630, p = 0.018. However, the total number of respondents with the correct response was low.

D. Perception of Core Competency Skill in Growth Measurement and Charting by Learners

One of the most important courses in paediatrics is growth and development and training future medical doctor to acquire skills and competence in growth and management is a key component of the BMAS. While growth measurement may seem easy to the uninformed, the whole task is daunting especially in children with complex growth abnormalities and malformation, and for more complex skills like arm span. Which of the more complex skills should the learner be expected to be competent in, will be debated in an expert forum of trainers.

“So, did you do anthropometric measures?”

Facilitator

Participant 1 shared:

Yes, anytime we clerk a patient, we must check the weight and height and interpret using age-appropriate charts, but we did not plot them in the charts. We carry the age-appropriate chart and interpreted our patients, as this is a requirement.

Using the chart may not be emphasised by all paediatric lecturers, so learners can be smart to know those lecturers who will request this skill from them during the clerkship period or the unit rotations.

“We did not quite get the concept of mid parental height, height percentile, it was just mentioned in passing. I never saw a severely short child that needed growth hormone. I was only told by a classmate of mine.”

Participant 3

The charting and interpretation of weight and height measurements of children was not done in all schools as shown in Table 4 below, which tells that only 65.8% of total respondents were taught interpretation of measured and charted growth parameters. The level of competence in these tasks will also be varied as seen in Appendix 1. Two hundred and thirty-eight (75.8%) learners perceived they had competency/ proficiency in height measures using stadiometer, and 44.6 % of the learners with these perceptions actually had paediatric endocrinology clinical rotation (Appendix 1).

Bone age and orchidometers are used to assess skeletal maturation and puberty, which are advanced for the undergraduate learners and certainly not compulsory, but some respondents were taught with the tools showing the variabilities in contents and skills delivery between these schools. From Table 4 above, framers of the syllabus for endocrinology aspect of paediatrics curriculum are unlikely to include use of orchidometer and bone age during the undergraduate paediatric endocrinology rotation as the skill is complex, and not necessary for their level of development.

IV. DISCUSSION

This study has highlighted differences in course contents and training methods across medical schools in Southern Nigeria. While many schools have used the BMAS prescribed by the MDCN, the syllabus used are different and the intended learning outcomes are diverse based on the respondents’ reports. Some learners reported not having diabetes lectures in their school through no fault of theirs, as lecturer rescheduled the lectures and never gave them. While learners have the responsibility to attend lectures, trainers are also obligated to be present at their scheduled lectures or transfer this to their teacher-assistants, or use technologies (Grant, 2014; Ruiz et al., 2006). Some learners had little participation in the Emergency Room, others participated fully in DKA management, learning empathy, specialised skills and communication. The intended competencies to be acquired can be achieved through shadowing and participation, bed-side teaching, and tutorial to improve the cognitive and psychomotor skill, and these opportunities must be created for them in experiential settings (Ryan et al., 2020; Shah et al., 2020).

More learners had staggered postings, going through junior and senior paediatric postings in what may be considered as integrated learning departing from the traditional method (Patel et al., 2005; Watmough et al., 2006, 2009). In the staggered posting type of rotation, we noticed that not all learners went through paediatric endocrinology unit posting, and like one of the discussants said, they would rather everyone went through each unit getting bits of everything and having opportunity to study specific and prevalent diseases in paediatric units rather than leaving them with the possibility of not learning important disorders. As it is not always possible to encounter specific diseases like DKA during entire posting in the schools that use staggered posting types, the likelihood of exposure was higher in schools that had block posting from the FGD conducted, but this did not translate to better retention of skills or cognitions as depicted in the Tanner staging matching question.

Having learners train in all special postings may not be the best approach in undergraduate medicine because the specialised skills may not be utilised in general practice and even in general paediatrics should the learners plan paediatric specialisation (Bindal et al., 2011). While some trainers may argue that all information and skill should be taught to the learners, the time to acquire and achieve mastery may be short for the learners (Jensen et al., 2018; Offiah et al., 2019). This study can be referenced in curriculum designing and implementation so the framers understand what society needs should be filled at any time. The concept of cognitive overload has actually reduced the duration of core specialty in clinical medicine while increasing the duration for others with emphasis on psychomotor, affective skills and professionalism. Some medical schools have core paediatric posting of 7 – 8 weeks, but Nigeria is still fixed with the traditional 3 – 4 months. In some schools in South Africa, the clinical posting is run as modular block for 3 years, with paediatric curriculum running from year 4 through year 6 (Dudley & Rohwer, 2015). With the long duration in the Nigeria curriculum, skills competencies are still deficient, so there is need to revamp the curriculum to make it more competency driven. It is excusable that more sophisticated competence like use of orchidometer were not known by more than half the learners, but if some were taught, the level of confidence in these skills at this stage of their learning should also be assessed as was done for diabetes by George et al. (2008).

Medical schools in Nigeria and other countries will have to continually evolve and produce curricula that are competency based, using problem-based learning, simulations, mannikin training for skills as is done in other countries (Watmough et al., 2006). Diabetes, thyroid, ambiguous genitalia with congenital adrenal hyperplasia, short stature and calcium disorders are common in Nigeria and should be taught in structured and integrated formats. Integrated curriculum where skills are graded from simple to complex can also be tested e g, skills of height measurements and charting using the stadiometer and growth charts can be taught in the 1^st clinical year, and then the mid parental heights, target height calculation and bone age may be taught in the 2^nd and 3^rd clinical years. (Brauer & Ferguson, 2015; Grant, 2014).

A. Strength of the Research

Articulating the perceptions of learners is not always easy as they are varied and subjective, but getting them to come together, discuss and give suggestions on how curriculum can be designed and achieved increases the strength of this research. There was no sense of victimisation of the learners as many had already graduated from their schools, and the discussants admitted to not missing classes, or clinical learning. They spoke freely, with courtesy to others and there was little or no argument among them.

B. Limitations of the Research

As this research is based on past experiences of the cognitive and psychomotor skills achieved during the learners’ training period, the possibility of recall bias is high, and respondents may underestimate or exaggerate their skills. Using respondents who had just concluded their paediatric postings was an attempt at reducing this limitation. The best time to evaluate a programme is usually soon after the programme has been concluded however, as there has been no report of this type of evaluation, there was need to embark on it and make recommendations.

V. CONCLUSION

Respondents reported high variability in the implementation of the recommended guidelines for paediatric endocrinology curriculum between schools in Southern Nigeria. Variabilities were in the courses’ completion, learners’ skills exposure and how much hands-on were allowed in various skills acquisitions. This variability will hamper the core objectives of human capital development should the trend continue.

A. Area of Future Research

Noting the differences exist between schools, curriculum strategists and implementation teams in universities should commission a DELPHI study by experts, where core competencies and objectives for paediatric endocrinology will be agreed on and sent to the regulatory bodies for endorsement and implementation.

Notes on Contributors

IY conceived, designed, planned, executed and conducted interviews and the research. He also collected the data, analysed it and wrote the manuscript.

TC helped in designing the methodology for the data colllection and analyses, and reviewed the manuscript.

CU gave critical appraisal of the manuscript and all authors have approved the final manuscript.

Ethical Approval

The research ethics committee of the Univeristy of Port Harcourt gave ethical approval before the start of the study with the number: UPH/CEREMAD/REC/MM80/056.

Data Availability

The data supporting this research is available for publication purposes, without editing. Data can be shared only with express permission from the corresponding author as deposited in Figshare repository, using the private url:

https://figshare.com/articles/dataset/Copy_of_CURRICULUM_STUDENTS_xls/21154396

Acknowledgement

We acknowledge the early career doctors and final year students who participated in the online survey especially the selected ones who took part in the focus group discussion.

Declaration of Interest

Authors declare that there are no conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.

Funding

There was no funding for this survey.

References

Alsalamah, A., & Callinan, C. (2021). Adaptation of Kirkpatrick’s four level model of training criteria to evaluate training programmes for head teachers. Education Science, 11(116), 1-25. https://doi.org/10.3390/educsci11030116

Bates, R. (2004). A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning, 27, 341-347. https://doi.org/10.1016/j.evalprogplan.2004.04.011

Bindal, T., Wall, D., & Goodyear, H. M. (2011). Medical students’ views on selecting paediatrics as a career choice. European Journal of Pediatrics, 170(9), 1193-1199. https://doi.org/10.1007/s00431-011-1467-9

Brauer, D. G., & Ferguson, K. J. (2015). The integrated curriculum in medical education: AMEE Guide No. 96. Medical Teacher, 37(4), 312-322. https://doi.org/10.3109/0142159X.2014.970998

Burton, J. L., & McDonald, S. (2001). Curriculum or syllabus: Which are we reforming? Medical Teacher, 23(2), 187-191. https://doi.org/10.1080/01421590020031110

Dudley, L. D., Young, T. N., Rohwer, A. C., Willems, B., Dramowski, A., Goliath, C., Mukinda, F. K., Marais, F., Mehtar, S., & Cameron, N. A. (2015). Fit for purpose? A review of a medical curriculum and its contribution to strengthening health systems in South Africa. African Journal Health Profession Education, 7(1), 81-84. https://doi.org/10.7196/AJHPE.512

Federal Ministry of Health of Nigeria, Health Systems 20/20Project. (2012). Nigeria undergraduate medical and dental curriculum template. Health systems 20/20 Project, Abt Associates Inc.

George, J. T., Warriner, D. A., Anthony, J., Rozario, K. S., Xavier, S., Jude, E. B., & Mckay, G. A. (2008). Training tomorrow’s doctors in diabetes: Self-reported confidence levels, practice and perceived training needs of post-graduate trainee doctors in the UK. A multi-centre survey. BMC Medical Education, 8, Article 22. https://doi.org/10.1186/1472-6920-8-22

Grant, J. (2014). Principles of curriculum design. In T. Swanwick, K. Forrest, B. C. O’Brien (Eds.), Understanding medical education evidence, theory and practice Sussex, UK: Wiley Blackwell, 31-46.

Hohmann, E., & Tetsworth, K. (2018). Fellowship exit examination in orthopaedic surgery in the commonwealth countries of Australia, UK, South Africa and Canada. Are they comparable and equivalent? A perspective on the requirements for medical migration. Medical Education Online, 23(1), Article 1537429. https://doi.org/10.1080/10872981.2018.1537429

Jensen, J. K., Dyre, L., Jørgensen, M. E., Andreasen, L. A., & Tolsgaard, M. G. (2018). Simulation-based point-of-care ultrasound training: a matter of competency rather than volume. Acta Anaesthesiology Scandanavia, 62(6), 811-819. https://doi.org/10.1111/aas.13083

McManus, I. C. (2003). Medical school differences: beneficial diversity or harmful deviations. BMJ Quality and Safety in Health Care, 12(5), 324-325. https://doi.org/10.1136/qhc.12.5.324

McManus, I. C., Harborne, A. C., Horsfall, H. L., Joseph, T., Smith, D. T., Marshall-Andon, T., Samuels, R., Kearsley, J. W., Abbas, N., Baig, H., Beecham, J., Benons, N., Caird, C., Clark, R., Cope, T., Coultas, J., Debenham, L., Douglas, S., Eldridge, J., . . . Devine, O. P. (2020). Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise. BMC Medicine, 18(1), Article 136. https://doi.org/10.1186/s12916-020-01572-3

Offiah, G., Ekpotu, L. P., Murphy, S., Kane, D., Gordon, A., O’Sullivan, M., Sharifuddin, S. F., Hill, A. D. K., & Condron, C. M. (2019). Evaluation of medical student retention of clinical skills following simulation training. BMC Medical Education, 19(1), Article 263. https://doi.org/10.1186/s12909-019-1663-2

Olson, A. L., Woodhead, J., Bekow, R., Kaufman, N., & Marshal, S. (2000). A national general pediatric clerkship curriculum: The process of development and implementation. Pediatrics, 160(S1), 216 -222. https://doi.org/10.1542/peds.106.S1.216

Ornstein A, H. F., & Hunkins, F. P. (2009). Curriculum: Foundations, principles and issues (5th Ed.). Pearson.

Patel, V. L., Arocha, J. F., Chaudhari, S., Karlin, D. R., & Briedis, D. J. (2005). Knowledge integration and reasoning as a function of instruction in a hybrid medical curriculum. Journal of Dental Education, 69(11), 1186-1211. https://www.ncbi.nlm.nih.gov/pubmed/16275683

Polikoff, M. S. (2018). The challenges of curriculum materials as a reform lever Evidence Speaks Reports, 2, 58

Puri, N., McCarthy, M., & Miller, B. (2021). Validity and reliability of pre-matriculation and institutional assessments in predicting USMLE STEP 1 success: Lessons from a traditional 2 x 2 curricular model. Frontiers in Medicine (Lausanne), 8, Article 798876. https://doi.org/10.3389/fmed.2021.798876

Rimmer, A. (2014). GMC will develop single exam for all medical graduates wishing to practise in UK. BMJ, 349, g5896. https://doi.org/10.1136/bmj.g5896

Rufai, S. R., Holland, L. C., Dimovska, E. O., Bing Chuo, C., Tilley, S., & Ellis, H. (2016). A national survey of undergraduate suture and local anesthetic training in the United Kingdom. Journal of Surgical Education, 73(2), 181-184. https://doi.org/10.1016/j.jsurg.2015.09.017

Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of E-learning in medical education. Academic Medicine, 81(3), 207-212. https://doi.org/10.1097/00001888-200603000-00002

Ryan, A., Hatala, R., Brydges, R., & Molloy, E. (2020). Learning with patients, students, and peers: Continuing professional development in the solo practitioner workplace. Journal of Continuing Education in the Health Profession, 40(4), 283-288. https://doi.org/10.1097/CEH.0000000000000307

Santen, S. A., Feldman, M., Weir, S., Blondino, C., Rawls, M., & DiGiovanni, S. (2019). Developing comprehensive strategies to evaluate medical school curricula. Medical Science Educator, 29(1), 291-298. https://doi.org/10.1007/s40670-018-00640-x

Shah, S., McCann, M., & Yu, C. (2020). Developing a national competency-based diabetes curriculum in undergraduate medical education: A Delphi study. Canadian Journal of Diabetes, 44(1), 30-36. https://doi.org/10.1016/j.jcjd.2019.04.019

Sosna, J., Pyatigorskaya, N., Krestin, G., Denton, E., Stanislav, K., Morozov, S., Kumamaru, K. K., Jankharia, B., Mildenberger, P., Forster, B., Schouman-Clayes, E., Bradey, A., Akata, D., Brkljacic, B., Grassi, R., Plako, A., Papanagiotou, H., Maksimović, R., & Lexa, F. (2021). International survey on residency programs in radiology: similarities and differences among 17 countries. Clinical Imaging, 79, 230-234. https://doi.org/10.1016/j.clinimag.2021.05.011

Tiffin, P. A., Paton, L. W., Mwandigha, L. M., McLachlan, J. C., & Illing, J. (2017). Predicting fitness to practise events in international medical graduates who registered as UK doctors via the Professional and Linguistic Assessments Board (PLAB) system: a national cohort study. BMC Medicine, 15(1), Article 66. https://doi.org/10.1186/s12916-017-0829-1

van Zanten, M., Boulet, J. R., & Shiffer, C. D. (2022). Making the grade: licensing examination performance by medical school accreditation status. BMC Medical Education, 22(1), Article 36. https://doi.org/10.1186/s12909-022-03101-7

Watmough, S., Garden, A., & Taylor, D. (2006). Does a new integrated PBL curriculum with specific communication skills classes produce Pre Registration House Officers (PRHOs) with improved communication skills. Medical Teachers, 28(3), 264-269. https://doi.org/10.1080/01421590600605173

Watmough, S., O’Sullivan, H., & Taylor, D. (2009). Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews. BMC Medical Education, 9, Article 64. https://doi.org/10.1186/1472-6920-9-64

Yarhere, I., Chinnah, T., & Uche, C. (2022). Learners’ report and perception of differences in undergraduate paediatric endocrinology curriculum content and delivery across Southern Nigeria. [Data set]. Figshare. https://doi.org/10.6084/m9.figshare.20730937.v1

Yarhere, I. E., & Nte, A. R. (2018) A ten-year review of all cause paediatric mortality in University of Port Harcourt Teaching Hospital, Nigeria (2006–2015). Nigerian Journal of Paediatrics, 45(4), 185-91.

*Iroro Enameguolo Yarhere
East/West Road,
PMB 5323 Choba,
Rivers State, Nigeria
+2347067987148
Email: iroro.yarhere@uniport.edu.ng

Submitted: 16 May 2022
Accepted: 3 January 2023
Published online: 4 July, TAPS 2023, 8(3), 5-14
https://doi.org/10.29060/TAPS.2023-8-3/OA2813

Bikramjit Pal¹, Aung Win Thein², Sook Vui Chong³, Ava Gwak Mui Tay⁴, Htoo Htoo Kyaw Soe⁵ & Sudipta Pal⁶

¹Department of Surgery, Manipal University College Malaysia, Melaka, Malaysia; ²Department of Surgery, Manipal University College Malaysia, Melaka, Malaysia; ³Department of Medicine, Manipal University College Malaysia, Melaka, Malaysia; ⁴Department of Surgery, Manipal University College Malaysia, Melaka, Malaysia; ⁵Department of Community Medicine, Manipal University College Malaysia, Melaka, Malaysia; ⁶Department of Community Medicine, Manipal University College Malaysia, Melaka, Malaysia

Abstract

Introduction: The practice of high-fidelity simulation-based medical education has become a popular small-group teaching modality across all spheres of clinical medicine. High-fidelity simulation (HFS) is now being increasingly used in the context of undergraduate medical education, but its superiority over traditional teaching methods is still not established. The main objective of this study was to analyse the effectiveness of HFS-based teaching over video-assisted lecture (VAL)-based teaching in the enhancement of knowledge for the management of tension pneumothorax among undergraduate medical students.

Methods: A cohort of 111 final-year undergraduate medical students were randomised for this study. The efficacy of HFS-based teaching (intervention group) and VAL-based teaching (control group), on the acquisition of knowledge, was assessed by single-best answer multiple choice questions (MCQ) tests in the first and eighth week of their surgery posting. Mean and standard deviation (SD) for the total score of MCQ assessments were used as outcome measures. ANCOVA was used to determine the difference in post-test MCQ marks between groups. The intragroup comparison of the pre-test and post-test MCQ scores was done by using paired t-test. The P-value was set at 0.05.

Results: The mean of post-test MCQ scores were significantly higher than the mean of pre-test MCQ scores in both groups. The mean pre-test and post-test MCQ scores in the intervention group were slightly more than those of the control group but not statistically significant.

Conclusion: There was a statistically significant enhancement of knowledge in both groups but the difference in knowledge enhancement between the groups was insignificant.

Keywords:           High-Fidelity Simulation, Video-Assisted Lecture, Simulation-Based Medical Education (SBME), Randomized Controlled Trial (RCT), Medical Education, Pre-test and Post-test Knowledge Assessments

Practice Highlights

• An RCT study to evaluate the effectiveness of HFS over video-assisted lecture teaching method.
• HFS seems to be not superior than VAL-based teaching for knowledge acquisition and retention.
• HFS may be used judiciously when the objectives are mainly knowledge based.
• Further research may determine curricular areas where HFS is superior and worth adopting.

I. INTRODUCTION

High-Fidelity Simulation (HFS) is an innovative healthcare education methodology that involves the use of sophisticated life-like mannequins to create a realistic patient environment. HFS can be considered an innovative teaching method that aids students in translating knowledge and psychomotor skills from the classroom to the actual clinical setting. Kolb’s Experiential Learning Cycle (Kolb, 1984) provides a basis for the integration of active learning of simulation with conventional teaching methods for a comprehensive learning experience in undergraduate medical education. HFS-based education is potentially an efficacious pedagogy that is now available for teaching. The usefulness of HFS has been recognized by the Accreditation Council of Graduate Medical Education (Accreditation Council of Graduate Medical Education [ACGME], 2020). HFS has the added benefit of increasing students’ confidence and their ability to care for the patients at the bedside (Kiernan, 2018). HFS-based education and video-assisted lecture-based teaching are both effective in achieving factual learning. Despite the increasing acceptance of HFS, there are limited studies to compare the usefulness of HFS with conventional teaching methods for factual learning among undergraduate medical students. At present, the different research studies have not provided enough evidence to establish HFS-based teaching’s superiority over traditional educational methods in the acquisition and retention of knowledge. There is inconsistent and variable outcome regarding the effectiveness of HFS on student learning (Yang & Liu, 2016). HFS-based education is both time-consuming and resource intensive. Its long-term merits in retaining knowledge and translating it into enhanced patient care need further research. As educators, we need to systematically evaluate the expensive newer teaching-learning modules like HFPS for their effectiveness by using rigorous research methodology and protocols. This is to ensure that we are providing the best learning opportunities conceivable for the students. Previous studies were mostly done in North America and, therefore, the generalisability of these results is guarded and might not be applicable in the context of Europe and Asia due to many differences in academic and curriculum aspects (Davies, 2008). The purpose of this study was to establish the feasibility of the use of HFS to deliver critical care education to final-year medical students and to find its efficacy in the enhancement of knowledge when compared to video-assisted lectures. The study compared the effectiveness of two methods of teaching pedagogy in the enhancement of knowledge acquisition using pre-test and post-test MCQ. This study was designed to provide insights that may be applied to the future development and improvement of HFS-based education among undergraduate medical students and its possibility of integrating it into course curricula.

II. METHODS

A. Study Design

Randomized Controlled Trial (RCT) with parallel groups and 1:1 allocation. Please see Appendix 1 for the Flow Chart.

B. Sample Size

G*Power software was used to calculate the sample size (Faul et al., 2007). Based on the preliminary RCT study of our institute done with the same protocol in 2018, the calculated sample size was 114 with a power of 0.95 for this study.

C. Inclusion and Exclusion Criteria

All male and female final-year undergraduate medical (MBBS) students in our institute were recruited after obtaining their written informed consent. All final-year students in the institute consented to the study. The participants were between the ages of 22-26 years.

The total number of participants recruited was 123.

The number of participants dropped out was 12 (9.77%).

Out of 111 participants who completed the study, 61 (54.95%) were female and 50 (45.05%) were male.

The study was conducted in the Clinical Skills Simulation Lab of Melaka Manipal Medical College (presently known as Manipal University College Malaysia).

The study period was from March 2019 to February 2020 (12 months).

D. Interventions

1) Description of HFPS-based teaching: It was an interactive session using a high-fidelity patient simulator demonstrating the management of tension pneumothorax by performing Needle Decompression on METIman (Pre-Hospital) following the Advanced Trauma Life Support Manual developed by the American College of Surgeons (ATLS Subcommittee et al., 2013).

2) Description of Hi-fidelity simulator: METIman Pre-Hospital HI-Fidelity Simulator (MMP-0418) was used for the simulation sessions. It was a fully wireless, adult High-Fidelity Patient Simulator (HFPS) with modelled physiology. It comes with extensive clinical features and capabilities designed specifically for learners to practice, gain experience, and develop clinical mastery in a wide range of patient care scenarios.

3) Description of video-assisted lecture-based teaching: It was a small group interactive session delivered face-to-face to the participants using a recorded video clip demonstrating the management of tension pneumothorax by performing Needle Decompression on METIman (Pre-Hospital) following the Advanced Trauma Life Support Manual developed by the American College of Surgeons (ATLS Subcommittee et al., 2013).

E. Outcome

The tool for measurement of knowledge was an identical set of single-best answer A-type MCQs. These MCQs were used for both Pre-test and Post-test knowledge assessments. MCQs were constructed based on the teaching sessions to assess their learning outcome.

The efficacy of HFPS-based teaching when compared to video-assisted lecture-based teaching is enhancement of knowledge for management of tension pneumothorax.

F. Recruitment

The students were recruited in the study during their final year surgical posting.

G. Randomisation

A cohort of 12 to 14 students from each rotation was randomised into intervention (HFPS-based teaching) and control (video-assisted lecture-based teaching) groups following random sequence generation method.

A computer-generated random sequence number was developed from randomizer.org. The independent randomiser was a biostatistician who did not participate in the delivery of interventions. The allocated interventions were then sealed in a sequentially numbered, opaque envelope.

Block randomisation with a block size of two was used to assign the students into intervention and control groups.

H. Implementation

A biostatistician generated the allocation sequence. One independent investigator enrolled the participants, and another independent investigator assigned the participants to interventions. The outcome assessor and the biostatistician were kept blinded to the randomisation.

I. Procedure for Data Collection

The participants who gave consent were enrolled in the study. Each session was conducted with a group of 12 to 14 participants. On the first day, the participants were briefed about the sessions and expected learning outcomes. As part of the briefing process, they were explained the confidentiality of the HFPS, the video-assisted lecture sessions and the ethical issues involved. All the participants were introduced to the high-fidelity patient simulator (METIman) in the clinical lab set-up to make them aware of its functions and familiarise them with the handling of the mannequin. An assurance was given to the students that the training course was not part of the evaluation process for the surgical curriculum. The briefing was followed by the first knowledge assessment (Pre-test MCQ) of all the participants. Pre-test MCQ was designed to collect the score of initial background knowledge about tension pneumothorax and its management following the ATLS protocol. The module for the aetiology, pathophysiology and clinical presentation of tension pneumothorax and its steps of management following the ATLS protocol was part of their final year course curriculum. It was taught before they participated in the study. After the Pre-test MCQ session, they were randomized into intervention and control groups consisting of 6 to 7 participants each. For the intervention group, an independent investigator used the high-fidelity simulator (METIman Pre-hospital) to demonstrate the diagnosis and management of tension pneumothorax (Needle Decompression) in an emergency setting. The demonstration time was 20 minutes followed by hands-on training for another 20 minutes. For the control group, a recorded video clip of the identical facilitated simulation session on the diagnosis and management of tension pneumothorax (Needle Decompression) was shown by another investigator. The video demonstration lasted for 20 minutes. This session was followed by a 20-minute interactive discussion session with the faculty. All the participants in both groups were apprised of the importance of aetiology, pathophysiology and clinical presentation in arriving at the diagnosis and management of tension pneumothorax during these interactive teaching sessions. The participants were encouraged to explore how they would manage the stated clinical situation through discussion. The faculty were instructed to emphasize the teaching points related to the outcome of the study. The total duration for both types of teaching was 40 minutes. There were no more additional hands-on practice or video-assisted lecture sessions for the participants during the course of the research study. In the seventh/eighth week, both the intervention and the control groups again participated in the second knowledge assessment (Delayed Post-test MCQ) to assess their gain and retention of knowledge. Delayed Post-test MCQ assessment may minimise the recall bias and test their retained memory better.

Both Pre-test and Post-test knowledge assessments comprised 20 MCQs which were to be completed in 20 minutes. The single-best answer A-type MCQs with five options of answers were prepared following the guidelines framed by the National Board of Medical Examiners (Case & Swanson, 2001). For each correct response, a score of one point was awarded. No negative marking was awarded for incorrect response. Based on the learning objectives, the MCQs were constructed by 6 experts in the field of Surgery, Medicine and Medical Education who were not part of this research study. The MCQs covered the items on pathophysiology, diagnosis, and management of tension pneumothorax, and assessed for knowledge comprehension and knowledge application. The order of the questions was changed between the Pre-test and the Post-test. The MCQ answer sheets were scanned by Konica Minolta FM (172.17.5.12) scanner and graded by using Optical Mark Recognition (OMR) software (Remark Office OMR, version 9.5, 2014; Gravic Inc., USA). Before the main study, a preliminary study involving 56 students was conducted to explore the time management, feasibility, acceptability, and validation of the MCQs (Pal et al., 2021). In the preliminary study, the Pre-test and the Post-test were administered in the first week and the fourth week respectively to note the short-term retention of knowledge. This study is an extension of the preliminary study with a different cohort of students where the Pre-test and the Delayed Post-test were administered in the first week and the seventh/eighth week respectively to determine the medium-term retention of knowledge. The MCQs were reviewed based on the feedback from the preliminary study on the appropriateness of the content, clarity in wording, and difficulty level. The difficulty index and the bi-serial correlation for item discrimination of all MCQs were checked. The value between 30 and 95 in the difficulty index and the bi-serial correlation value > 0.2 were chosen as the accepted standard for this study. 

At the end of the study, the participants in the intervention group were provided with access to the identical video-assisted lecture sessions as designed for the control group. Similarly, the participants in the control group were provided with access to the same HFS sessions. This is to ensure parity between the groups for their professional development of knowledge.

J. Statistical Analysis

SPSS software (version 25) was used for data analysis. The descriptive statistics such as frequency and percentage for categorical data and the mean and standard deviation for the total score of the assessments were calculated. ANCOVA was used to determine the difference in post-test MCQ marks between intervention and control groups with pre-test MCQ marks as a covariate. Intragroup comparison of pre-test and post-test MCQ marks was also done by calculating paired t-test. For intergroup comparison, the effect size – Partial Eta Squared was calculated in ANCOVA. Cohen’s dz was calculated for the comparison of dependent means. The level of significance was set at 0.05 and the null hypothesis was rejected when P < 0.05. We measured the scale-level content validity index (SCVI) and item-level content validity index (ICVI) for the validity and Cronbach alpha for the internal consistency (reliability) of the MCQs. The average values of SCVI and ICVI were 0.94 & 0.89 respectively. The value of Cronbach’s alpha was 0.78.

III. RESULTS

The data that support the findings this RCT study are openly available at https://doi.org/10.6084/m9.figshare.19932053 (Pal et al., 2022).

A. General Data Analysis

There was no difference in the highest Pre-test scores achieved by the participants in both intervention and control groups. The lowest scores recorded in the intervention group were better than the control group in both Pre-test and Post-test. There was a negligible difference between the highest Post-test scores among control and intervention groups (See Table 1).

Test score	Intervention	Control
PRE-TEST
Mean (SE)	12.31 (0.34)	12.23 (0.36)
95% CI for Mean	11.64 – 12.98	11.50 – 12.96
Min – Max	6.0 – 18.0	6.0 – 18.0
POST-TEST
Mean (SE)	13.65 (0.27)	13.60 (0.30)
95% CI for Mean	13.12 – 14.19	12.98 – 14.20
Min – Max	8.0 – 18.0	7.0 – 17.0
Table 1. Highest, lowest and unadjusted mean MCQ scores among intervention and control groups
SE – Standard Error                CI – Confidence Interval Min – Minimum                      Max – Maximum

There was a statistically significant difference between pre-test and post-test MCQ scores among the intervention and control groups. The mean of post-test MCQ scores was significantly higher than the mean of pre-test MCQ scores in both intervention and control groups. The effect size was moderate in both groups (See Table 3).

IV. DISCUSSION

Multiple studies have revealed slight to the modest enhancement of knowledge in simulation-based medical education (SBME) when compared to other instructional teaching methods (Cook et al., 2012; Gordon et al., 2006; Lo et al., 2011; Ray et al., 2012; Ten Eyck et al., 2009). Notwithstanding the increasing popularity of SBME, there is little evidence to conclude that it is superior to other small-group teaching modalities for the acquisition of knowledge (Alluri et al., 2016). The common perception is that knowledge lies at the lowest level of competence in Miller’s model of clinical acumen (Miller, 1990), but it is also important to note that knowledge is the basic foundation of competence and proficiency (Norman, 2009). Theoretically, SBME is advantageous for assessment of both knowledge and skills but there are few studies which directly evaluated the effectiveness of HFS in the assessment of knowledge (McGaghie et al., 2009; Rogers, 2008).

The mean scores of both Pre-test and the Post-test were higher in the intervention group in this study. In comparison, our preliminary study demonstrated that the control group had higher mean MCQ marks than the intervention group in Pre-test whereas at Post-test, the intervention group had higher mean MCQ marks than the control group (Pal et al., 2021).

In our study, there is significant enhancement of knowledge (P < 0.001) in both modes of teaching which corroborates the findings of Alluri et al. (2016). Their RCT study demonstrated that the participants in both the simulation and lecture groups had improved post-test scores (p < 0.05). The comparison of Pre-test and Post-test MCQ scores in our preliminary study also revealed significant higher mean MCQ scores at Post-test than Pre-test in both intervention and control groups (Pal et al., 2021). A study by Couto et al. (2015) showed improved post-test scores in both methods. Similar results were noted in the studies by Chen et al. (2017) and Vijayaraghavan et al. (2019). The finding of a study by Hall (2013) showed a slight increase in post-test scores in both the HFPS and control groups.

A systematic review by La Cerra et al. (2019) revealed that HFS was superior to other teaching methods in improving knowledge and performance. Significant higher scores for participants in the HFS group in the studies by Larsen et al. (2020) and Solymos et al. (2015) demonstrated that HFS may be superior to conventional teaching methods for factual learning. In another study by Bartlett et al. (2021), HFS showed a significant long-term gain in knowledge over traditional teaching methods, but short-term knowledge gain was insignificant. Our study revealed that the Post-test MCQ score was higher in the HFS group but after adjustment of pre-test scores, there was no significant difference in knowledge gain between the control and intervention groups. The findings were similar in our preliminary study where  the intervention group had higher mean change score of MCQ scores than the control group but it was not statistically significant (Pal et al., 2021).

On the other hand, there was no significant knowledge improvement in both simulation and traditional teaching methods as observed in the studies (Corbridge et al., 2010; Kerr et al., 2013; Moadel et al., 2017). The findings of Alluri et al. (2016) also showed no difference in knowledge gain between simulation and lecture-based teaching. The studies by Morgan et al. (2002) and Tan et al. (2008), demonstrated equal efficacy between simulation and conventional lectures. The findings of a study by Kerr et al. (2013) demonstrated that SBME was not beneficial in acquisition and retention of knowledge. There was no significant improvement in knowledge after simulation-based education as revealed by the findings of three RCTs (Cavaleiro et al., 2009; Cherry et al., 2007; Kim et al., 2002).

Despite simulation being effective in acquisition of knowledge, it may not be the most efficient modality when compared to other traditional educational methods (Bordage et al., 2009). There is ample evidence that SBME usually leads to enhancement of knowledge and skills among undergraduate students but its superiority over other conventional teaching methods is yet to be defined (Nestel et al., 2015).

A. Limitations

There is a possibility of potential biases in the form of design, recruitment, sample populations and data analysis that could have influenced the findings. Due to randomization in blocks of two, the allocation of participants may be predictable which may result in selection bias. The confounding factors such as communication between the different groups of students prior to the second MCQ assessment, participants’ recall memory and preparation for the post-test after 7 – 8 weeks need to be considered. As it was a single-centre study which included final-year medical students only, the validity of the findings may not be applicable to other settings.

V. CONCLUSION

Conventional teaching modalities and HFS, when used in conjunction with bedside teaching, may complement clinical practice, leading to higher retention of knowledge. Therefore, more studies are required to measure the efficacy of simulation for a better understanding of the differences that it can make in the acquisition of knowledge. Our study revealed that the efficacy of high-fidelity simulation-based teaching was not superior to video-assisted lecture-based teaching in terms of knowledge acquisition and retention. The substantially higher cost and maintenance associated with HFS need to be considered before planning a teaching-learning activity. It may be used judiciously with conventional teaching when the objectives are mainly knowledge-based. More studies are required to determine its effectiveness and further evaluation as a teaching-learning tool in medical education.

Notes on Contributors

Bikramjit Pal was involved in Conceptualization, Formal Analysis, Literature Review, Methodology, Project administration & Supervision, Data Analysis and Writing (original draft & editing).

Aung Win Thein was involved in Formal Analysis, Literature Review, Methodology, Supervision and Writing (review & editing).

Sook Vui Chong was involved in Literature Review, Methodology, Supervision and Writing (review & editing).

Ava Gwak Mui Tay was involved in Formal Analysis, Literature Review, Supervision and Writing (review & editing).

Htoo Htoo Kyaw Soe was involved in Formal Analysis, Methodology, Data curation, Statistical Analysis and Validation.

Sudipta Pal was involved in Literature Review, Methodology, Formal Analysis, Data curation and Writing (review & editing).

Ethical Approval

Ethical approval was duly obtained from the Ethical Committee / IRB of Manipal University College Malaysia. Informed consent was taken from all the participants. All information about the participants was kept confidential.

Approval number: MMMC/FOM/Research Ethics Committee – 11/2018.

Data Availability

The data that supports the findings of this RCT study are openly available at Figshare repository, https://doi.org/10.6084/m9.figshare.19932053.v2 (Pal et al., 2022).

Acknowledgement

The authors would like to acknowledge the final year MBBS students of Manipal University College Malaysia who had participated in this research project, the faculty of the Department of Surgery, the lab assistants and technicians of Clinical Skills Lab and the Management of Manipal University College Malaysia.

Funding

The researchers had not received any funding or benefits from industry or elsewhere to conduct this study.

Declaration of Interest

The researchers had no conflicts of interest.

References

Accreditation Council for Graduate Medical Education. (2020, July 1). Program requirements for graduate medical education in general surgery. https://www.acgme.org/globalassets/pfassets/programrequirements/440_generalsurgery_2020.pdf.

Alluri, R. K., Tsing, P., Lee, E., & Napolitano, J. (2016). A randomized controlled trial of high-fidelity simulation versus lecture-based education in preclinical medical students. Medical Teacher, 38(4), 404–409. https://doi.org/10.3109/0142159X.2015.1031734

ATLS Subcommittee, American College of Surgeons’ Committee on Trauma, & International ATLS working group. (2013). Advanced trauma life support (ATLS®): The ninth edition. Journal of Trauma and Acute Care Surgery, 74(5), 1363–1366. https://doi.org/10.1097/TA.0b013e31828b82f5

Bartlett, R. S., Bruecker, S., & Eccleston, B. (2021). High-fidelity simulation improves long-term knowledge of clinical swallow evaluation. American Journal of Speech-Language Pathology, 30(2), 673–686. 

Bordage, G., Carlin, B., Mazmanian, P. E., & American College of Chest Physicians Health and Science Policy Committee (2009). Continuing medical education effect on physician knowledge: Effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest, 135(3 Suppl), 29S–36S. https://doi.org/10.1378/chest.08-2515

Case, S. M., & Swanson, D. B. (2001). Constructing written test questions for the basic and clinical sciences. 3rd ed. Philadelphia: National Board of Medical Examiners.

Cavaleiro, A. P., Guimarães, H., & Calheiros, F. (2009). Training neonatal skills with simulators? Acta Paediatrica, 98(4), 636–639.    

Chen, T., Stapleton, S., Ledford, M., & Frallicciardi, A. (2017). Comparison of high-fidelity simulation versus case-based discussion on fourth- year medical student performance. Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health, 18(5.1). https://escholarship.org/uc/item/5k73f4qc

Cherry, R. A., Williams, J., George, J., & Ali, J. (2007). The effectiveness of a human patient simulator in the ATLS shock skills station.  Journal of Surgical Research, 139(2), 229–235. https://doi.org/10.1016/j.jss.2006.08.010

Cook, D. A., Brydges, R., Hamstra, S. J., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hatala, R. (2012). Comparative effectiveness of technology-enhanced simulation versus other instructional methods: A systematic review and meta-analysis. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 7(5), 308–320. https://doi.org/10.1097/SIH.0b013e3182614f95

Corbridge, S. J., Robinson, F. P., Tiffen, J., & Corbridge, T. C. (2010). Online learning versus simulation for teaching principles of mechanical ventilation to nurse practitioner students. International Journal of Nursing Education Scholarship, 7(1), Article 12. https://doi.org/10.2202/1548-923X.1976

Couto, T. B., Farhat, S. C., Geis, G. L., Olsen, O., & Schvartsman, C. (2015). High-fidelity simulation versus case-based discussion for teaching medical students in Brazil about pediatric emergencies. Clinics (Sao Paulo, Brazil), 70(6), 393–399. https://doi.org/10.6061/clinics/2015(06)02

Davies, R. (2008). The Bologna process: the quiet revolution in nursing higher education. Nurse Education Today, 28(8), 935–942. https://doi.org/10.1016/j.nedt.2008.05.008

Ellis, P. (2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge University Press. https://doi.org/10.1017/CBO9780511761676

Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/bf03193146

Gordon, J. A., Shaffer, D. W., Raemer, D. B., Pawlowski, J., Hurford, W. E., & Cooper, J. B. (2006). A randomized controlled trial of simulation-based teaching versus traditional instruction in medicine: A pilot study among clinical medical students. Advances in Health Sciences Education , 11(1), 33–39. https://doi.org/10.1007/s10459-004-7346-7

Hall, R. M. (2013). Effects of high fidelity simulation on knowledge acquisition, self-confidence, and satisfaction with baccalaureate nursing students using the solomon-four research design [Doctoral dissertation, East Tennessee State University]. East Tennessee State University Higher Education Commons. https://dc.etsu.edu/etd/2281

Kerr, B., Hawkins, T. L., Herman, R., Barnes, S., Kaufmann, S., Fraser, K., & Ma, I. W. (2013). Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Medical Education Online, 18(1), Article 21312. https://doi.org/10.3402/meo.v18i0.21312

Kiernan, L. C. (2018). Evaluating competence and confidence using simulation technology. Nursing, 48(10), 45–52. https://doi.org/10.1097/01.NURSE.0000545022.36908.f3

Kim, J. H., Kim, W. O., Min, K. T., Yang, J. Y., & Nam, Y. T. (2002). Learning by computer simulation does not lead to better test performance on advanced cardiac life support than textbook study. The Journal of Education in Perioperative Medicine, 4(1), Article E019. 

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Prentice Hall. 

La Cerra, C., Dante, A., Caponnetto, V., Franconi, I., Gaxhja, E., Petrucci, C., Alfes, C. M., & Lancia, L. (2019). Effects of high-fidelity simulation based on life-threatening clinical condition scenarios on learning outcomes of undergraduate and postgraduate nursing students: A systematic review and meta-analysis. BMJ Open, 9(2), Article e025306. https://doi.org/10.1136/bmjopen-2018-025306

Larsen, T., Jackson, N., & Napolitano, J. (2020). A comparison of simulation-based education and problem-based learning in pre-clinical medical undergraduates. MedEdPublish, 9(1), Article 172.

Lo, B. M., Devine, A. S., Evans, D. P., Byars, D. V., Lamm, O. Y., Lee, R. J., Lowe, S. M., & Walker, L. L. (2011). Comparison of traditional versus high-fidelity simulation in the retention of ACLS knowledge. Resuscitation, 82(11), 1440–1443. https://doi.org/10.1016/j.resuscitation.2011.06.017

McGaghie, W. C., Siddall, V. J., Mazmanian, P. E., Myers, J., & American College of Chest Physicians Health and Science Policy Committee (2009). Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: Effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest, 135(3 Suppl), 62S–68S. https://doi.org/10.1378/chest.08-2521

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63–S67. https://doi.org/10.1097/00001888-199009000-00045

Moadel, T., Varga, S., & Hile, D. (2017). A prospective randomized controlled trial comparing simulation, lecture and discussion-based education of sepsis to emergency medicine residents. Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health, 18(5.1). https://escholarship.org/uc/item/0132981t

Morgan, P. J., Cleave-Hogg, D., McIlroy, J., & Devitt, J. H. (2002). Simulation technology: A comparison of experiential and visual learning for undergraduate medical students. Anesthesiology, 96, 10–16. https://doi.org/10.1097/00000542-200201000-00008

Nestel, D., Harlim, J., Smith, C., Krogh, K., & Bearman, M. (2015). Simulated learning technologies in undergraduate curricula: An evidence check review for HETI.

Norman, G. (2009). The American College of Chest Physicians evidence-based educational guidelines for continuing medical education interventions: a critical review of evidence-based educational guidelines. Chest, 135(3), 834–837. https://doi.org/10.1378/chest.09-0036

Pal, B., Chong, S. V., Thein, A. W., Tay, A. G., Soe, H. H., & Pal, S. (2021). Is high-fidelity patient simulation-based teaching superior to video-assisted lecture-based teaching in enhancing knowledge and skills among undergraduate medical students? Journal of Health and Translational Medicine, 24(1), 83-90. https://doi.org/10.22452/jummec.vol24no1.14

Pal, B., Thein, A. W., Chong, S. V., Tay, A., Htoo, H., & Pal, S. (2022). A randomized controlled trial study to compare the effectiveness of high-fidelity based teaching with video-assisted based lecture teaching in enhancing knowledge [Dataset]. Figshare. https://doi.org/10.6084/m9.figshare.19932053

Ray, S. M., Wylie, D. R., Shaun Rowe, A., Heidel, E., & Franks, A. S. (2012). Pharmacy student knowledge retention after completing either a simulated or written patient case. American Journal of Pharmaceutical Education, 76(5), 86. https://doi.org/10.5688/ajpe76586

Rogers, D. A. (2008). The role of simulation in surgical continuing medical education. Seminars in Colon and Rectal Surgery, 19(2), 108-114. https://doi.org/10.1053/j.scrs.2008.02.007

Solymos, O., O’Kelly, P., & Walshe, C. M. (2015). Pilot study comparing simulation-based and didactic lecture-based critical care teaching for final-year medical students. BMC Anesthesiology, 15, Article 153. https://doi.org/10.1186/s12871-015-0109-6

Tan, G. M., Ti, L. K., Tan, K., & Lee, T. (2008). A comparison of screen-based simulation and conventional lectures for undergraduate teaching of crisis management. Anaesthesia and Intensive Care, 36(4), 565–569.

Ten Eyck, R. P., Tews, M., & Ballester, J. M. (2009). Improved medical student satisfaction and test performance with a simulation-based emergency medicine curriculum: a randomized controlled trial. Annals of Emergency Medicine, 54(5), 684–691. https://doi.org/10.1016/j.annemergmed.2009.03.025

Vijayaraghavan, S., Rishipathak, P., & Hinduja, A. (2019). High-fidelity simulation versus case-based discussion for teaching bradyarrhythmia to emergency medical services students. Journal of Emergencies, Trauma, and Shock, 12(3), 176–178. https://doi.org/10.4103/JETS.JETS_115_18

Yang, Y., & Liu, H. P. (2016). Systematic evaluation influence of high-fidelity simulation teaching on clinical competence of nursing students. Chinese Nursing Research, 30(7), 809–814. https://caod.oriprobe.com/articles/47628779/Systematic_evaluation_influence_of_high_fidelity_s.htm

*Bikramjit Pal
RCSI & UCD Malaysia Campus (RUMC),
4 Jalan Sepoy Lines,
Georgetown, Penang, 10450, Malaysia
+6042171908-1908 (Ext)
Email: bikramjit.pal@rcsiucd.edu.my

Submitted: 15 October 2022
Accepted: 3 January 2023
Published online: 4 July, TAPS 2023, 8(3), 1-4
https://doi.org/10.29060/TAPS.2023-8-3/GP2903 

Galvin Sim Siang Lin¹, Yook Shiang Ng² & Chan Choong Foong³

¹Department of Dental Materials, Faculty of Dentistry, Asian Institute of Medicine, Science and Technology (AIMST) University, Malaysia; ²Conservative Dentistry Unit, School of Dental Sciences, Universiti Sains Malaysia, Health Campus, Malaysia; ³Medical Education and Research Development Unit (MERDU), Faculty of Medicine, Universiti Malaya, Malaysia

Abstract

Introduction: Dental materials science is one of the core courses for Malaysian undergraduate dental programmes, which has been primarily taught through a series of didactic lectures during the preclinical phase. In accordance with the newly revised national competency statement, Malaysian dental education is moving toward competency-based education, hence a reformation of the dental materials science curriculum is warranted.

Methods: Curriculum design including common teaching, learning and assessments methods for the Malaysian dental materials science were described and analysed. The common practices were reviewed and compared with latest national guidelines and literature.

Results: There is yet an initiative to establish a national curriculum for dental materials science. The use of traditional teaching strategies for this course also needs to be revamped from a teacher-centred to a student-centred approach. Furthermore, faculty members are facing significant challenges because the revamp requires them to explore cutting-edge pedagogical methods and develop appropriate learning opportunities, environments, resources, and assessments. Several recommendations are proposed, such as mapping the existing dental materials science curriculum to identify gaps, incorporating more hands-on sessions, implementing an integrated curriculum, introducing various formative and summative assessments, as well as recruiting faculty members with different areas of educational expertise.

Conclusion: It is hoped that this article offers a clearer pathway for Malaysian dental educators to pioneer new insight and transform the existing dental materials science curriculum.

I. INTRODUCTION

Malaysian dental curriculum is historically underpinned by the primary goal in nurturing clinically competent dentists to provide safe and high-quality oral healthcare treatment to the public. Although the curriculum has witnessed an array of revisions over the years, little has been done for dental materials science courses. In the present article, the authors outline an overview of the dental curriculum in Malaysia and describe how dental materials science courses are currently delivered. Subsequently, challenges and compelling needs are highlighted to envision the future curriculum.

II. OVERVIEW OF DENTAL CURRICULUM IN MALAYSIA

In May 1972, the Universiti Malaya established the country’s first dental school to offer a Bachelor of Dental Surgery (BDS) programme. Nowadays, dentistry has become one of the popular university programmes for high school leavers in Malaysia and the number of applicants has increased significantly over the past decades. Thirteen dental schools have been established of which six are public institutions and the remaining seven are private institutions. All dental schools in Malaysia operate independently as siloed institutions; there is no national standardised curriculum or licensing examination. Currently, each of these dental schools offers a five-year undergraduate dental programme leading to the conferment of BDS or Doctor of Dental Surgery (DDS), and all programmes are required to be accredited by the Malaysian Dental Council (MDC).

In general, undergraduate dental programmes in Malaysia are divided into two phases: preclinical and clinical phases. The preclinical phase encompasses the first two years of the undergraduate programmes. Undergraduate dental students are taught fundamental dental and medical concepts along with operative practical and laboratory skills through simulation exercises on mannequins or dummy heads. In the subsequent three years of the clinical phase, students are given the opportunities to manage and provide treatments to patients. At the end of the five-year programmes, dental students are expected to be clinically competent and practise evidence-based dentistry with ethics and professionalism.

III. DENTAL MATERIALS SCIENCE CURRICULUM IN MALAYSIA

A. Curriculum Content

Undergraduate dental programmes in Malaysia offer dental materials science courses as one of the core disciplines in the second year of preclinical phase. Dental materials science is a course that integrates underlying principles of chemical engineering and materials science into the practice of contemporary dentistry (Qazi et al., 2019). Typically, dental materials science is introduced as a stand-alone course distributed across several modules over the preclinical dental phase. The course usually starts with an introduction to dental materials and the properties of commonly used dental materials. In some dental schools, the course is divided into discipline-based such as dental materials used in conservative dentistry, prosthodontics, paediatric dentistry, orthodontics, or periodontics; whereas some schools divide the course into clinical-based and laboratory-based dental materials. Among the course content covered in clinical-based dental materials include amalgam, dental composite resin, glass ionomer cement and impression materials. On the other hand, denture-based polymers, dental ceramics, metal alloys, gypsum products as well as casting and investment materials are covered under laboratory-based dental materials. Additionally, most dental schools in Malaysia design and develop their dental materials science courses based on Bloom’s and Simpson’s taxonomies, with the bulk of the curriculum content lying within cognitive C1 (Remember) to C3 (Apply) and psychomotor P2 (Set) to P4 (Mechanism) levels.

B. Teaching, Learning and Assessment Methods

Dental materials science courses are usually delivered through a series of didactic lectures. Students are also exposed to practical hands-on sessions to examine the properties of commonly used dental materials and manipulate various dental materials. For instance, some dental schools in Malaysia introduce hands-on practical to evaluate the mechanical and viscoelastic properties of various dental materials as well as to mix clinical and laboratory-based dental materials, such as dental gypsum products, dental polymers, dental cement, and dental impression materials.

Nevertheless, since each dental school utilises a distinct set of course learning outcomes, it is infeasible to make a direct comparison between them. Moreover, students are required to meet the desired course learning outcomes upon completion of the courses. Most dental schools use either formative, summative, or a combination of both assessments to evaluate their students, though solely evaluating students based on formative assessment is rare. Students also might be evaluated through quizzes, presentation seminars, assignments, written assessments, and practical assessments such as objective-structured practical examinations (OSPE). Examples of written assessments include multiple-choice questions (MCQ) and short-answer questions (SAQ).

IV. CHALLENGES AND THE TURNING POINT

In June 2021, the Malaysian Dental Dean Council organised a workshop and proposed a revised national competency statement for future graduates in Malaysia, which was later endorsed by the Malaysian Dental Council in July 2021 (Malaysian Dental Council [MDC], 2021). The national competency statement was revised to align with the Malaysian Qualifications Framework version 2.0 competency-based education. Specifically, for dental materials science, the cognitive and psychomotor-related clusters have stated that future dental graduates should be able to justify the selection of dental materials based on the science and applications and related environmental issues (Cognitive: C4 Analyse) and to manipulate commonly used dental materials (Psychomotor: P5 Complex or overt response). Subsequently, Malaysian dental schools are prompted to revise their existing dental materials science curriculum to attain the intended learning outcomes. It is necessary to reform the dental materials science curriculum that is going to lead the attainment of cognitive and psychomotor competencies. Yet, there is no standardised national curriculum for dental materials science courses in Malaysia, despite there were a few attempts by faculty members from several dental schools.

Furthermore, pedagogical methods require improvement as dental materials science courses are often delivered via didactic lectures, whereby students claimed as a ‘dry’ subject (Soni et al., 2021). Implementation of traditional pedagogical methods does not only result in a lack of applications of pertinent knowledge, but these methods also reduce students’ interest and learning efficacies for the courses. Hence, the traditional pedagogical methods used to deliver dental materials science must be revamped to shift from teacher-centred approach to student-centred approach. There is a need to adopt diverse pedagogical methods in encouraging active students’ engagement during learning activities. This shift has posed a significant challenge to faculty members because not only do they need to explore innovative pedagogical methods that promote active learning, but they need to devise appropriate learning opportunities, learning locations, learning resources, and alternative assessments for students.

Different educational backgrounds of faculty members who teach dental materials courses are also a topic of discussion. Some Malaysian dental schools recruit experts with postgraduate degrees either in dental materials, conservative dentistry, prosthodontics, or restorative dentistry, while some schools might employ experts with a background in chemistry or material sciences. Students may miss the fundamental knowledge (if taught by BDS qualified) or fail to relate the clinical applications (if taught by scientists). At the present, teaching qualifications are encouraged but not mandatory for faculty members.

V. RECOMMENDATIONS FOR TOMORROW’S DENTAL MATERIALS SCIENCE CURRICULUM

The revised national competencies statement represents national advocacy for transformation in the dental curriculum. Hence, curriculum mapping is an initial step in transforming tomorrow’s dental materials science curriculum. Following curriculum mapping, dental educators can identify the gaps in the existing curriculum, suggest components to be maintained, revised, or removed with the purpose to achieve the desired cognitive and psychomotor competencies as listed in the national statement. Consequently, it also symbolises the significance to spur the call for the establishment of a designated dental education department in designing and developing comprehensive dental programmes in Malaysia. In hindsight, one may say that this notion served as the impetus for Malaysia to accomplish competency-based dental education in the 21st century.

Didactic lectures place a focus on the transmission of knowledge passively from lecturers to students. Students may find it difficult to understand the basic premises and practical applications of materials science and engineering due to its interdisciplinary nature in the field of dentistry. As students are required to comprehend and justify the selection of various dental materials, hands-on experience in their pre-clinical education is of utmost importance. For instance, practical sessions on manipulating different materials may be arranged following theoretical lectures. Thus, the ability to actively apply the theories of materials science to clinical content is what constitutes mastery in dental materials science courses, which goes beyond how well a student can recall and repeat factual information. In addition, to incorporate more hands-on sessions, the dental materials science curriculum should consider diverse pedagogical methods including but not limited to concept mapping, flipped classroom, micro-teaching, jigsaw, small-group discussions, and team-based learning. Although these methods are experimented in the literature, there is limited published evidence in Malaysia (Bhat et al., 2021).

The current discipline-based curriculum in Malaysian dental education consists of a stack of separate courses with their compartmentalised scopes and syllabus. Students may find it difficult to correlate theoretical knowledge and clinical application due to the lack of integration between preclinical and clinical courses. An integrated competency-based curriculum could overcome this dilemma by incorporating dental materials science into the list of courses that demand vertical and horizontal integration. Integrated curriculum as a visionary change could be regarded as one of the first instances in Malaysian dental education, where dental materials science is no longer viewed as a stand-alone course or under the aegis of a particular dental speciality. In fact, it will be divided into several subtopics that are integrated into several dental specialities across the preclinical and clinical phases. For instance, topics like amalgam and dental composite resin can be incorporated in conservative dentistry courses, whilst removable prosthodontics courses can cover dental materials like resin polymer and gypsum. As students are transitioning into clinical years, dental ceramics can be integrated into fixed prosthodontics courses.

A well-designed dental materials science curriculum must take assessments into consideration since it reflects how the documented curriculum is related to the outcomes (e.g., the student’s learning experiences, course learning outcomes and learning opportunities). If the dental materials science curriculum is to be revised to accommodate horizontal and vertical integrations, then a myriad of assessments should be utilised. Interdisciplinary care and evidence-based treatments with sound analytical and communication skills, are necessary for the successful and efficient delivery of tomorrow’s dental education. This justifies the need for assessment systems to be thorough and concrete enough to evaluate every stage of students’ progression. Although dental schools in Malaysia have been utilising written assessments (e.g., multiple choices questions, short answer questions) as the main assessment tool in dental materials science courses, alternative assessments such as modified Direct Observation of Procedural Skills (DOPS), peer- and self-assessments could be introduced in accordance with an integrated curriculum (Ferris & O’Flynn, 2015). These alternative assessments are valid and reliable in determining competencies and offering exceptional chances to combine summative and formative assessments. It is expected that students would be capable of recognising knowledge and skills, and they would value opportunities for improvement in their learning (i.e., self-reflection).

Moreover, Malaysian dental schools should consider recruiting faculty members with different areas of educational expertise in teaching dental materials science courses. Diversity of teaching staff will enhance and enrich the learning experiences of students. Experts with backgrounds in chemistry and materials science could involve in the teaching of fundamental principles of dental materials, while those with dental degrees may involve in translating the knowledge into practical applications. It is also worth noting that no single faculty member is recommended to cover the whole teaching and learning syllabus. Last, teaching qualifications are highly recommended for faculty members to apply and keep abreast with the latest instructional design. Nevertheless, the movement toward competency-based education (and moving away from traditional discipline-based or requirement-based education) would require more education research and academic discussions in Malaysia to rationalise and select appropriate teaching, learning and assessment methods as well as intellectual vibrancy, academic support, research, scholarship, and educational management in dental materials science courses.

VI. CONCLUSION

Malaysia is on the verge of advancing towards competency-based dental education along with the introduction of a diversity of teaching, learning and assessment methods. The path to change is not without obstacles and ambiguity. Thus, comprehensive education research and academic discussions among dental educators in Malaysia involved in the teaching of dental materials science are warranted to pioneer new insight to transform the existing dental materials science curriculum.

Notes on Contributors

GSSL and CCF was involved in conception and design of the study. GSSL and YSN reviewed the literature, collected the data, and wrote the original draft. CCF edited the original draft. All authors have read and approved the final manuscript.

Funding

No funding is required for this paper.

Declaration of Interest

All authors have no conflicts of interest.

References

Bhat, S., Madiyal, A., & Babu, G. S. (2021). Innovative teaching methods in dental education. Gülhane Medical Journal, 63, 8-13. https://doi.org/10.4274/gulhane.galenos.2020.1181  

Ferris, H., & O’Flynn, D. (2015). Assessment in medical education; What are we trying to achieve? International Journal of Higher Education, 4(2), 139-144. https://doi.org/10.5430/ijhe.v4n2p139

Malaysian Dental Council. (2021). Competencies of new dental graduates, Malaysia-V2-2021. Retrieved September 23, 2022, from https://mdc.moh.gov.my/uploads/competencies_of_new_dental_grad_20222.pdf

Qazi, H. S., Ashar, A., & Ahmad, S. A. (2019). Impact of an innovative approach of teaching science of dental materials on the learning experiences of undergraduate students. Pakistan Armed Forces Medical Journal, 69, 582-588.

Soni, V., Kotsane, D. F., Moeno, S., & Molepo, J. (2021). Perceptions of students on a stand-alone dental materials course in a revised dental curriculum. European Journal of Dental Education, 25, 117-123.

*Galvin Sim Siang Lin
Department of Dental Materials,
Faculty of Dentistry,
Asian Institute of Medicine,
Science and Technology (AIMST) University,
08100, Bedong, Kedah, Malaysia
Email: galvin@aimst.edu.my

Submitted: 27 December 2022
Accepted: 13 March 2023
Published online: 4 July, TAPS 2023, 8(3), 70-71
https://doi.org/10.29060/TAPS.2023-8-3/LE2938

Hirofumi Kanazawa¹ & Ikuo Shimizu^2,3

¹Shinshu University School of Medicine, Matsumoto, Japan; ²Safety Management Office, Shinshu University Hospital, Matsumoto, Japan; ³Department of Medical Education, Chiba University Graduate School of Medicine, Chiba, Japan

Medication errors are one of the most serious problems in healthcare, and their occurrence is due to a wide variety of causes. While almost all drug-related medical errors are potentially avoidable, the main obstacles are primarily health system vulnerabilities and human error. To remove those obstacles, the World Health Organization (WHO) suggested providing more education (World Health Organization, 2017), specifically by participating in medical care. Graduation competencies in many countries, which include activities about medication and patient safety, are consistent with the statement. In addition, it is known that more prescribing errors occur in the first postgraduate year. Learning about prescriptions during medical studentships must be enhanced.

Nevertheless, we are concerned that medication safety education is still insufficient in the current undergraduate curriculum because of license-based regulation. It is considered difficult for clinical students to incorporate prescribing and ordering medication as part of medical studentship. Such license-based regulation is typical in Asia, where authoritarian attitudes are strong. We would like to point out that invasive clinical procedures are handled differently from prescriptions, despite the fact that such procedures are also medical practices that can involve risk. While medical students have the opportunity to conduct invasive medical procedures in the workplace, they scarcely prescribe medication before graduation.

It is time for medical educators to take the WHO statement seriously and organise more opportunities to learn how to prescribe medications safely. We can transfer learning strategies from invasive clinical procedures, even in the context of strict regulations. For example, we can implement more simulation practices. Although the WHO Curriculum Guide for Patient Safety does not yet include scenario simulation, research suggests that education with appropriate feedback can be very effective (Motola et al., 2013). Simulation-based prescription practice, especially in situations where medication errors are likely to occur, would allow for focused practice in a zero-risk environment. Another concern is that there have been relatively few published studies on the effectiveness of medication safety in undergraduate education, including simulation. High-fidelity simulators are available in many medical schools, but the usability is still limited because they do not cover a wide variety of situations to use medications. A possible breakthrough can be the implementation of virtual or mixed reality environment. Enhancing the reality of prescribing and administrating medications through these educational strategies could be a very useful tool to apply not only in Asia but also in other contexts.

Notes on Contributors

Hirofumi Kanazawa conceptualised and wrote the manuscript.

Ikuo Shimizu wrote and supervised the manuscript.

All authors discussed and contributed to the final manuscript.

Funding

This work was supported by JSPS KAKENHI Grant Number 21H03161.

Declaration of Interest

There is no conflict of interest.

The main idea of this article was presented at The 17th Annual Congress of Japanese Society for Quality and Safety in Healthcare on November 27, 2022.

References

Motola, I., Devine, L. A., Hyun, S. C., Sullivan, J. E., & Issenberg, S. B. (2013). Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82. Medical Teacher, 35(10), e1511-e1530. https://doi.org/10.3109/0142159X.2013.818632

World Health Organization. (2017, May 15). Medication Without Harm – Global Patient Safety Challenge on Medication Safety. https://www.who.int/publications/i/item/WHO-HIS-SDS-2017.6

*Ikuo Shimizu
Safety Management Office,
Shinshu University Hospital,
Matsumoto, Japan
3-1-1, Asahi, Matsumoto City,
Nagano, Japan, 390-8621
+81 263 37 3359
Email: qingshuiyufu@gmail.com

Submitted: 22 July 2022
Accepted: 5 December 2022
Published online: 4 April, TAPS 2023, 8(2), 93-96
https://doi.org/10.29060/TAPS.2023-8-2/CS2849

Mian Jie Lim¹, Jeremy Choon Peng Wee², Dana Xin Tian Han² , Evelyn Wong­²

¹SingHealth Emergency Medicine Residency Programme, Singapore Health Services, Singapore; ²Department of Emergency Medicine, Singapore General Hospital, Singapore

I. INTRODUCTION

Singapore raised its Disease Outbreak Response System Condition (DORSCON) from yellow to orange on the 7^th of February 2020 after it is first reported unlinked community COVID-19 case on the 6^th of February 2020.

The Department of Emergency Medicine (DEM) of Singapore General Hospital (SGH) hosted clinical rotations for medical students (MS) from Duke-NUS Medical School. Their clinical rotations lasted four weeks, during which MS were expected to achieve competence in history taking, physical examination, formulating a management plan, and performing minor procedures.

The local curriculum relies heavily on clinical rotations, during which MS directly contacts patients to fulfil their learning objectives. During DO, clinical postings were postponed, and the term break was brought forward (Ashokka et al., 2020). Restricting MS from entering hospitals and having face-to-face interactions with patients resulted in significant changes to clinical learning. Learning sessions involving direct contact that could not be held over remote platforms were cancelled.

We aimed to discover how MS felt excluded from clinical teaching during the COVID-19 pandemic and how teaching could be improved to support their learning.

II. METHODS

A mixed method of qualitative content analysis and quantitative analysis was performed for this study. Purposive sampling was conducted among all the Duke-NUS MS whose clinical postings to SGH DEM were affected during the COVID-19 outbreak. The link for the online survey form was sent via email or WhatsApp® to 60 MS. Their preferences in learning were assessed by a simple descriptive quantitative analysis of the responses to multiple-choice questions and were followed by an open-ended question whereby template analysis was performed. The consent of each participant was obtained as part of the online survey. The sample of the survey is attached in Appendix A.

III. RESULTS

A. Quantitative Result

Twenty-five MS (42%) were keen to participate in all learning activities at all department areas during normal times if they were trained to wear adequate personal protective equipment (PPE). Twenty-one MS (35%) felt that all learning activities should only be in safe areas of the department with appropriate PPE. Twelve MS (20%) were not keen on having any patient contact.

B. Qualitative Results

1) What other areas of improvement department could introduce about learning during the current pandemic?

Identified main themes, subthemes, and quotes from the open-ended questionnaire questions are presented in Table 1.

Table 1. Themes, subthemes and medical students’ responses

IV. DISCUSSION

A multi-centre quantitative study was done in the USA by Harries et al. (2021) which showed that 83.4% of MS agreed to return to the clinical environment. These results were comparable with our study’s quantitative result, which showed that 77% of MS were willing to return to the clinical environment during a pandemic.

Spencer et al. (2000) showed that direct patient contact is essential for developing clinical reasoning, communication skills, professional attitudes, and empathy. The findings of this study showed that MS felt that not having direct clinical contact with patients during the pandemic had adverse effects on their learning. Although actual patient contact is desired, this may be countered by the risk of COVID-19 infection, reflected in the first theme of “balance of training needs with infection control” only if adequate risk reduction can patient contact be achieved within a low-risk environment.

MS are all adult learners and should be given liberty on the need for full clinical exposure to patients. However, barring MS from clinical areas also occurred in other countries, especially with the shortage of PPE during the initial emergence of COVID-19 (Rose, 2020).

PPE and infection control training for MS is crucial to ensure clinical teaching can be conducted safely during a pandemic outbreak. Norton et al. (2021) suggested that to avoid injury to patients and healthcare providers and reduce COVID-19-related anxiety among MS, better PPE and infection control training was essential.

A. Limitation

As our study is a mixed qualitative content analysis and quantitative analysis study of a single centre, it is therefore affecting the generalisability of our results. More multi-centre research could be conducted to understand MS’s opinions across Singapore better.

Secondly, another limitation is that our data was collected via an online survey, and we could not follow up with in-depth questions, unlike face-to-face interviews.

The study was done during the early phase of the COVID outbreak and was designed as an initial exploratory study. The results could be used to establish initial information that can help guide future studies.

V. CONCLUSION

From this study, it is essential that medical education needs to be more versatile in future pandemics and consider MS’s opinions. We recommend the incorporation of PPE and infection control training in the undergraduate curriculum, as well as the set-up of an effective online learning platform.

Notes on Contributors

Lim Mian Jie performed the literature review and the template analysis of the data and wrote the manuscript.

Wee Choon Peng Jeremy developed the methodological framework, performed the template analysis of the data, and gave critical feedback to the writing of the manuscript.

Dana Han Xin Tian submitted the CIRB application, recruited the participants, and collected the data.

Evelyn Wong conceptualised the study, collected the data, advised on the study’s design, and gave critical feedback to the writing of the manuscript.

All the authors have read and approved the final manuscript.

Ethical Approval

IRB exemption for this study was obtained (SingHealth CRIB reference number 2020/2134).

Acknowledgement

The authors would like to acknowledge the contributions of all the participants.

Funding

No funding is required for this study.

Declaration of Interest

There is no conflict of interest.

References

Ashokka, B., Ong, S. Y., Tay, K. H., Loh, N. H. W., Gee, C. F., & Samarasekera, D. D. (2020). Coordinated responses of academic medical centres to pandemics: Sustaining medical education during COVID-19. Medical Teacher, 42(7), 762–771. https://doi.org/10.1080/0142159X.2020.1757634 

Harries, A. J., Lee, C., Jones, L., Rodriguez, R. M., Davis, J. A., Boysen-Osborn, M., Kashima, K. J., Krane, N. K., Rae, G., Kman, N., Langsfeld, J. M., & Juarez, M. (2021). Effects of the COVID-19 pandemic on medical students: A multi-centre quantitative study. BMC Medical Education, 21(1), Article 14. https://doi.org/10.1186/s12909-020-02462-1

Norton, E. J., Georgiou, I., Fung, A., Nazari, A., Bandyopadhyay, S., & Saunders, K. E. A. (2021). Personal protective equipment and infection prevention and control: A national survey of UK medical students and interim foundation doctors during the COVID-19 pandemic. Journal of Public Health, 43(1), 67–75. https://doi.org/10.1093/pubmed/fdaa187

Rose, S. (2020). Medical student education in the time of COVID-19. JAMA, 323(21), 2131. https://doi.org/10.1001/jama.2020.5227

Spencer, J., Blackmore, D., Heard, S., McCrorie, P., McHaffie, D., Scherpbier, A., Gupta, T. S., Singh, K., & Southgate, L. (2000). Patient-oriented learning: A review of the role of the patient in the education of medical students. Medical Education, 34(10), 851–857. https://doi.org/10.1046/j.1365-2923.2000.00779.x

*Lim Mian Jie
Singapore General Hospital,
Outram Road,
169608, Singapore
Email: mianjie.lim@mohh.com.sg

Submitted: 9 May 2022
Accepted: 11 October 2022
Published online: 4 April, TAPS 2023, 8(2), 89-92
https://doi.org/10.29060/TAPS.2023-8-2/CS2806

Vidya Kushare^1,2, Bharti M K¹, Narendra Pamidi¹, Lakshmi Selvaratnam¹, Arkendu Sen¹ & Nisha Angela Dominic³

¹Jeffrey Cheah School of Medicine & Health Sciences (JCSMHS), Monash University, Malaysia; ²Department of Anatomy, Faculty of Medicine, Universiti Malaya, Malaysia; ³Clinical School Johor Bahru (CSJB), Monash University, Malaysia

Figure 1. Student responses in evaluating impact (based on a 5-point Likert scale) of virtual integration of relevant anatomy in the episiotomy workshop

Qualitatively, the responses to open-ended questions were grouped as either most or least beneficial. Most beneficial to the students was that it helped them to revise and correlate relevant anatomy, consolidate and highlight the important concepts. Least beneficial to students were the non-clinical aspects, overlapping content between the uploaded lecture video and the real time zoom session, insufficient models and the lack of online engagement. Overall, the students responded positively towards this learning approach.

IV. DISCUSSION

Based on student feedback, more than 90% responded positively towards this virtual integrated approach of reviewing relevant anatomy during the hands-on workshop. This just-in-time’ review approach, even when conducted virtually, allows them to focus on applying only pertinent knowledge to the hands-on session and subsequently when dealing with real-time episiotomy repair on patients.

The limitations of the study include the internet network bandwidth at the two distant sites and the restrictions posed by the ongoing COVID-19 pandemic. Replacing the live anatomy demonstrations, time constraints, social distancing and the use of face shields/face masks made online interactions more challenging.

V. CONCLUSION

This is an ongoing project, requiring further evaluation to assess the impact of this pre-internship training strategy on key procedural skills learning and future practice that is expected in obstetrics.

To conclude, incorporating relevant, refresher anatomy sessions into clinical teaching, even when held virtually, can benefit students to review the core concepts of basic sciences and apply it to clinical practice. This allows for the development of clinical skill competency and ultimately safe patient care.

Notes on Contributors

Vidya Kushare initiated and designed the project, conducted the virtual anatomy review sessions, prepared the video, the quiz and the feedback questionnaire, performed the data collection and data analysis, wrote the manuscript and presented this in a conference.

Bharti M K was involved in the design of the project, conducted the virtual anatomy review sessions, prepared the video and edited the manuscript.

Narendra Pamidi was involved in the design of the project, editing the manuscript and providing references.

Lakshmi Selvaratnam was involved in the planning and development of the project, providing references, providing feedback, writing and editing the manuscript.

Arkendu Sen was involved in the design of the project, providing feedback, editing the manuscript and providing references.

Nisha Angela Dominic initiated and designed the project, conducted the hands-on workshop sessions, prepared the quiz and the feedback questionnaire, performed the data collection, editing the manuscript and providing references.

All the authors have read and approved the final manuscript.

Acknowledgement

The authors acknowledge the technical teams from both campuses: Mr Mah, Ms Nurul, Ms Zafrizal & Mr Abisina.

Funding

The authors received no financial support for this study.

Declaration of Interest

The authors have no conflict of interest.

References

Jurjus, R. A., Lee, J., Ahle, S., Brown, K. M., Butera, G., Goldman, E. F., & Krapf, J. M. (2014). Anatomical knowledge retention in third-year medical students prior to obstetrics and gynecology and surgery rotations. Anatomical Sciences Education, 7(6), 461–468. https://doi.org/10.1002/ase.1441

Malik, A. S., & Malik, R. H. (2010). Twelve tips for developing an integrated curriculum. Medical Teacher, 33(2), 99–104. https://doi.org/10.3109/0142159x.2010.507711

Rajan, S. J., Jacob, T. M., & Sathyendra, S. (2016). Vertical integration of basic science in final year of medical education. International Journal of Applied & Basic Medical Research, 6(3), 182–185. https://doi.org/10.4103/2229-516X.186958

Wijnen-Meijer, M., van den Broek, S., Koens, F., & ten Cate, O. (2020). Vertical integration in medical education: The broader perspective. BMC Medical Education, 20(1). https://doi.org/10.1186/s12909-020-02433-6

Zumwalt, A. C., Marks, L., & Halperin, E. C. (2007). Integrating gross anatomy into a clinical oncology curriculum: The oncoanatomy course at Duke University School of Medicine. Academic Medicine, 82(5), 469–474. https://doi.org/10.1097/acm.0b013e31803ea96a

*Vidya Kushare
Jln Profesor Diraja Ungku Aziz,
50603 Kuala Lumpur,
Wilayah Persekutuan, Malaysia
+60162440142
Email: vidyakusharee@gmail.com / vidyakushare@um.edu.my

Submitted: 4 May 2022
Accepted: 16 August 2022
Published online: 4 April, TAPS 2023, 8(2), 86-88
https://doi.org/10.29060/TAPS.2023-8-2/SC2804

Sok Mui Lim^1,2 & Chun Yi Lim^2,3

¹Centre for Learning Environment and Assessment Development (CoLEAD), Singapore Institute of Technology, Singapore; ² Health and Social Sciences, Singapore Institute of Technology, Singapore; ³Department of Child Development, KKH Women’s and Children’s Hospital, Singapore

I. INTRODUCTION

Interactive oral assessment has been identified as a form of authentic assessment that enables students to develop their professional identity, communications skills, and helps promote employability (Sotiriadou et al., 2020). It simulates authentic scenarios where assessors can engage students in genuine and unscripted interactions that represents workplace experiences (Sotiriadou et al., 2020). Unlike written examinations, interactive oral questions are not rigidly standardised as students and assessors role-play using workplace scenarios, enabling students to respond to the conversational flow and achieve authenticity (Tan et al., 2021). Using Villarroel et al. (2018) four-step ‘Model to Build Authentic Assessment’, this paper will present the use of oral interactive with first year occupational therapy students. This is within the context of a module named “Occupational Performance Across Lifespan” and students learn about children’s developmental milestones.

II. METHODS

The first step of the Model by Villarroel et al. (2018) is to consider the workplace context. It is important to identify key transferable skills that are needed at typical workplace scenarios. In the job of occupational therapists, they need to meet with caregivers and address their concerns. The key transferable skills include determining whether there is delay in a child’s developmental milestones, communicating with empathy and articulating practical suggestions for caregivers. Thinking critically and communicating persuasively and empathetically, especially in dynamic situations, are important graduate attributes for our students to prepare themselves for the clinical workforce.

The second step of the Model is to design authentic assessment, which involves (1) drafting a rich context; (2) creating a worthwhile task; and (3) requiring higher order skills. In our assessment, students were given a scenario and asked to discuss developmental milestones with parents, identify whether there were areas of concerns from what was reported and provide suggestions if appropriate.

To do this, we trained standardised “actors” / “parents” to share their concerns and correspond with the student individually. As the assessment took place during the pandemic, we used Zoom for corresponding, like therapists conducting teleconsultations. To promote employment opportunities, we included persons with disability as standardised parents. The students were unaware of the disability such as spinal cord injury, as it was conducted on an online platform. We followed the guide on inclusion of persons with disabilities as standardised patients (Lim et al., 2020). The academic staff took the role of the examiner and focused on listening in to the answers provided and writing down feedback for each student.

The third step involves developing the assessment criteria and standard in the form of rubrics and familiarising students with them. To prepare the students, five weeks before the actual assessment, we explained what oral interactive assessments were and introduced the rubrics. They watched videos of one high performing student and one who struggled from previous cohort (with permission sought). They discussed what went well and where the gaps were, followed by pairing up to practice. This helped the students to understand the expected standard, visualise how the oral interactive will take place and learn to evaluate. Three weeks before the assessment, students were given some mock scenarios to practice, and suggestions from the previous cohort on how best to prepare for the assessment.

The fourth step relates to feedback. Feedback can enable students to judge future performances and make improvements within the context of individual assessment. After the assessment, each student was given individual written feedback. The cohort was given group feedback on what they did well and some of the common mistakes. Students who needed more detailed feedback were also given the opportunity to be coached by the Module Lead. At coaching-feedback sessions, the student will watch their video, pause, coached on what they notice, what was done well, and how they can do differently in future. Such feedback sessions are viewed as a coachable moment for educators to develop students in their competency (Lim, 2021).

III. RESULTS

We conducted oral interactive assessments with persons with disability as standardised parents for two cohorts of students (n>200). From the anonymous module feedback, we learnt that students appreciated the assessment as it has real world relevance and enable them to gain professional skills. Some appreciated the opportunity to experience what it felt like interacting with caregivers at their future workplaces. We also noted some students expressed they were more anxious preparing for the oral interactive compared to other forms of assessments. Students shared that they prepared the assessment by remembering the developmental milestones and practise verbalising the concepts out loud with their peers.

IV. DISCUSSION

Students need time to be prepared for a new form of assessment as they may be more familiar with pen and paper examination or report. A few recommendations are suggested:  

1) To reduce their anxiety, early preparation is important. Performance anxiety was a common stumbling block. Supporting students in learning strategies to manage performance anxiety can help.

2) For the assessment conversation to be natural, it is important to train the standardised actors on reactions for hit and missed responses from the students.

3) To maintain integrity of the assessment, different scenarios of similar level of difficulties were needed. Educators emphasised the value of learning from the assessment and individualised feedback, such that experience itself becomes intrinsically rewarding.

4) The educator plays the role of the examiner and concentrates to note down the quality of the answers and writes down feedback for each student.

5) Scaffolding students for continuous practice towards workplace competence is important. It is recommended to plan other authentic assessments in the later years of the curriculum such as OSCE.

V. CONCLUSION

Oral interactive assessment provides students with the opportunity to practice and be assessed on workplace competency. While students find themselves more anxious in preparing, they appreciate the real-world relevance and the opportunity to gain professional skills. It is worthwhile to spend effort in designing the assessment in detail, planning authentic scenarios, and preparing students for the experience. As an educator, it is rewarding to witness students developing the ability to demonstrate their competency in a professional manner.

Notes on Contributors

Associate Professor Lim Sok Mui (May) contributed to the conception, drafted and critically revised the manuscript.

Dr Lim Chun Yi contributed to the execution of the assessment,  drafting and reviewing the manuscript.

All authors gave their final approval and agree to be accountable for all aspects of the work.

Acknowledgement

We would like to acknowledge the help of Mr Lim Li Siong, Dr Shamini d/o Logannathan and Miss Elisa Chong for their help in supporting the oral interactive assessments and Miss Hannah Goh for assisting to proofread this manuscript.

Funding

There is no funding involved in the preparation of the manuscript.

Declaration of Interest

The authors declare no conflict of interest.

References

Lim, S. M. (2021, May 27). The answer is not always the solution: Using coaching in higher education. THE Campus Learn, Share, Connect. https://www.timeshighereducation.com/campus/answer-not-always-solution-using-coaching-higher-education

Lim, S. M., Goh, Z. A. G., & Tan, B. L. (2020). Eight tips for inclusion of persons with disabilities as standardised patients. The Asia Pacific Scholar, 5(2), 41-44. https://doi.org/10.29060/TAPS.2020-5-2/SC2134

Sotiriadou, P., Logan, D., Daly, A., & Guest, R. (2020). The role of authentic assessment to preserve academic integrity and promote skill development and employability. Studies in Higher Education, 45(11), 2132–2148. https://doi.org/10.1080/03075079.2019.1582015

Tan, C. P., Howes, D., Tan, R. K., & Dancza, K. M. (2021). Developing interactive oral assessments to foster graduate attributes in higher education. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2021.2020722

Villarroel, V., Bloxham, S., Bruna, D., & Herrera-Seda, C. (2018). Authentic assessment: Creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5): 840–854. https://doi.org/10.1080/02602938.2017.1412396

*May Lim Sok Mui
Singapore Institute of Technology,
10 Dover Drive,
Singapore 138683
+65 6592 1171
Email: may.lim@singaporetech.edu.sg

Submitted: 31 August 2022
Accepted: 27 September 2022
Published online: 4 April, TAPS 2022, 8(2), 83-85
https://doi.org/10.29060/TAPS.2023-8-2/PV2874

Janet Grant^1,2

¹Centre for Medical Education in Context (CenMEDIC), United Kingdom; ²University College London Medical School, United Kingdom

I. TO BEGIN WITH MY VIEW

Medical education is a social science which addresses how people learn and teach medicine. The practice of education and training is therefore fundamental to its epistemology, whereby knowledge, and so scholarship, derives from practice. Where that practice is subject to social, contextual and cultural factors, we must question whether the tenets that are put forward are generalisable beyond the context from which they were derived (Fendler & Cole, 2006). This lack of automatic generalisability has implications for both the scholarship of the medical educationalist, and for the relationship between medical educationalist and teacher. Where educational practice is primary, and is contextually informed, then the teacher, the practitioner of medical education, must be the leader in developing scholarship, while the medical educationalist can support that development by enabling each teacher, context or culture, to tell their own truth well (Grant & Grant, 2022). 

II. WHY IS THIS MY VIEW?

A. Scholarship and the Primacy of Practice

The term ‘scholarship’ implies special knowledge that is derived from research or academic analysis. While we can argue that learning has a research basis in educational and cognitive psychology, the same cannot be said of teaching. We can think, for example, of the churn of new teaching methods (sometimes erroneously presented as new ideas about learning) that sweep into medical education, find little evidence of consistent effect, and fade into the ever-expanding menu of teaching options. We can think of problem-based, task-based, case-based, resource-based, peer-assisted, blended, team-based, and e-learning, the flipped classroom, and more broadly, active learning and learner-centred learning. And there are more, changing with fashion and social values.

Are these changes based on generalisable knowledge derived from robust research? Although there might be published papers, they rarely constitute a consistent body of scholarly knowledge that enables claims about predictable effects of different teaching methods in different contexts. That is the nature of social science (Fendler & Cole, 2006). It is this lack of generalisability of the social practice of teaching that places the epistemology of medical education not in theories or fashion, but in widely variable, and contextually tailored, practice.

Although the practice of teaching is socially bound, we can say that the fundamental cognitive basis of learning, how knowledge is effectively organised in memory and accessed when needed, is the same for everyone. Knowing that short-term, working memory should not be overloaded, and that long-term memory should be well-organised with structured, generalisable and accessible knowledge, is the simple baseline against which a medical educationalist can ensure that teaching and learning methods are designed and judged. Many which demand complex processes (sometimes termed ‘learner-centred’) would fail that test.

There is a parallel literature demonstrating that the social, personal and interpersonal processes that cause knowledge to be stored and used effectively, and motor and cognitive skills to develop, are different depending on culture, content and context. Teaching that seems applicable and relevant in one cultural or content context may not apply in another. So it seems important to begin from practice, observe successes and problems, and build theories, if that seems helpful.

These uncertainties underpin our practice-based epistemology, where the teacher is the key person. Accordingly, we have argued (Grant & Grant, 2022) that medical education is not an academic discipline at all, but is an examination of instrumental practice, trying to relate educational activities to purposes, making its means relate to its ends, and making decisions about that on the basis of context and judgement. 

This view places the teacher at the heart and in the vanguard of relevant medical education development. This is social science where generalisable scholarship in teaching is difficult to attain. So, there is an ethical responsibility borne by those who claim to know what effective teaching is. 

This leads to the next question.

B. What might be the Relationship Between the Teacher and the Medical Educationalist?

Definitionally, I take a medical educationalist to be someone who claims special expertise by virtue of, for example, having completed a Master’s degree in health professions education. Some teachers have done this too, but most have not. Teachers, here, are the subject specialists who actively help others to acquire necessary knowledge and skill.

What might be the relationship between these two?

To answer this, we turn to Lawrence Stenhouse, a British educational thinker who sought to promote an active role for teachers in educational research and curriculum development. Stenhouse argued that the teacher might lead quality development, becoming an ‘extended professional’, supported by trained technical expertise: ‘It is not enough that teachers’ work should be studied: they need to study it themselves’ (Stenhouse, 1976, p143).

In this endeavour, the medical education specialist is a resource, knowing the theories and fashions, and their critiques, summarising where there is and is not evidence, guiding analysis, offering options in relation to the teacher’s practice. The teacher is an equal partner in this ‘mutually supportive co-operative research’ (Stenhouse, 1976, p159), learning to be a researcher, simply because knowledge comes from and is tested in its performance. The medical educationalist will be a crucial support in this process.

To hold this supporting role demands being critically informed about medical education theory and practice. Medical education seems replete with largely unexamined terms such as ‘adult learning’, ‘learner-centred’ or the oxymoronic ‘passive learning’; or with handy mnemonics, and frameworks that have ever-decreasing academic credibility such as ’learning styles’. Medical educationalists must be more securely rooted in the critical approach of social science, beyond the constantly metamorphosing rhetoric of medical education. That authenticity will be gained in equal partnership with teachers. 

Stenhouse’s position is unequivocal: the expert is the teacher, the practitioner who understands the individual context. The ‘teacher as researcher’ was Stenhouse’s ground-breaking view of the basis of rational educational development (Stenhouse, 1976, pp. 142-165).

How different is the implication of this view of the teacher, not as a person to be studied or developed, but the person who should be the scholar, reaching, and sharing, their own conclusions in their own classroom. Agency then belongs to the teacher who enacts the curriculum.

In this model, the role of the medical education specialist is to provide knowledge of developmental potential, and of how to develop practice-based, contextual scholarship around methods of reflective action research, perhaps. The medical educationalist is no longer the primary source of knowledge, or the impartial researcher, but is the means of supporting authentic practice development, helping each teacher to find their own truth.

C. And What of the Scholarship of Teaching?

The literature on the scholarship of teaching addresses its derivation in research and reflection on practice, and its use in theory building and educational development. In that literature, the meaning of scholarship in relation to actual teaching is ill-defined.

The importance of this for medical education is that scholarship can easily be thought of as the domain of those who have taken medical education as their speciality, rather than the domain of the teacher who is primarily a scientist or a clinician. This creates a particular relationship where ideas such as ‘faculty development’ suggest that the scholarship of teaching is garnered elsewhere and then shared with the teacher.

But I have argued that the scholarship of teaching will come from the experience of the teachers. Others argued, before me, that knowledge comes from social practice, and then returns to serve and enhance that practice (Mao, 1937). In that, there must be a mechanism for gathering that knowledge derived from social practice and returning it to practice. This may be the role of the medical educationalist, or of medical educationalists collectively, pooling their knowledge gained through working with teachers, reflecting their experience.

This role of gathering together knowledge generated in practice, is especially important in these days when the controversial idea of ‘globalisation of education’ often passes without critique. But ‘Globalisation initiatives must be tempered by ‘cultural humility’ in recognition of the likelihood that, rather than there being one exclusive, universal and ‘superior’ model, there may be many models of effective teaching and learning in medical education around the world’ (Wong, 2011, p. 1218). For Wong, in opposition to the neo-institutionalist, perhaps neo-colonialist, view, ‘…the culturalist perspective focuses on the enduring ability of different cultures and ways of knowing to re-interpret, transform and hybridise education practices to best suit local context’ (Wong, 2011, p. 2010).   

This view recognises those contextual imperatives: scholarship must derive from the domain of the teacher, supported, not driven, by the medical education specialist. This is true both of ideas on teaching methods, and of the theoretical and conceptual frameworks that shine and fade in parallel.

In this view, the teacher would become an extended professional who has ‘a capacity for autonomous professional self-development through a systematic self-study, through the study of the work of other teachers and through the testing of ideas by classroom research procedures’ (Stenhouse, 1976, p. 144). In other words, scholarship reverts to the teacher. Support for that scholarship belongs to the medical education specialist, working by the practitioner’s side, in the classroom, enabling that person to advance the contextual practice of medical education.

Note on Contributor

Janet Grant wrote the script, discussed it with Leo Grant and Professor Ahmed Rashid, and wrote the final version.

Acknowledgement

I would like to thank Leo Grant of CenMEDIC, London, and Professor Ahmed Rashid, of University College London Medical School, for their comments on this paper which helped me to express my personal view so much better than I could have done by myself.

Funding

There was no funding support accorded for this study.

Declaration of Interest

The author declares that there is no conflict of interest.

References

Fendler, L., & Cole, J. (2006). Why Generalisability is not Generalisable. Journal of Philosophy of Education, 40(4), 437–449. https://doi.org/10.1111/j.1467-9752.2006.00520.x

Grant, J., & Grant, L. (2022). Quality and constructed knowledge: Truth, paradigms, and the state of the science. Medical Education. https://doi.org/10.1111/medu.14871

Mao, Z. (1937). On Practice. https://www.marxists.org/reference/archive/mao/selected-works/volume-1/mswv1_16.htm

Stenhouse, L. (1976). An Introduction to Curriculum Research and Development. Heinemann.

Wong, A. K. (2011). Culture in medical education: Comparing a Thai and a Canadian residency programme. Medical Education, 45(12), 1209–1219. https://doi.org/10.1111/j.1365-2923.2011.04059.x

*Janet Grant
27 Church Street,
Hampton, Middlesex
TW12 2EB,
United Kingdom
Email: janet@cenmedic.net

Submitted: 15 July 2022
Accepted: 21 September 2022
Published online: 4 April, TAPS 2023, 8(2), 80-82
https://doi.org/10.29060/TAPS.2023-8-2/PV2842

Kevin Tan^1,2, Yang Yann Foo² & Nigel Choon Kiat Tan^1,2

¹Office of Neurological Education, Department of Neurology, National Neuroscience Institute, Singapore; ²Duke-NUS Medical School, Singapore

A program director of a one-year-old Singapore surgical residency programme reads a publication about a new model of feedback. The paper describes how a US medical school successfully trialled and implemented this new feedback model. Excited, she then tries to implement this new model in her residency programme. Unfortunately, this fails to change faculty and resident behaviour, with disgruntled faculty and residents, and poor take-up by the various surgical departments within her programme. Disappointed, she stops using the new feedback model after a year.

What happened? Why would an educational intervention about feedback, published as part of Scholarship of Teaching and Learning (SoTL) (Steinert, 2017), and successfully implemented in a US medical school, fail to take root in a Singapore surgical residency programme? Might failure to consider context have contributed?  A review of the publication showed that while descriptions of the feedback model and the educational outcomes were rich, descriptions of the medical school environment and the broader educational context of the US were sparse.

Might a richer description of context in the publication have helped readers understand the social and educational milieu from which the novel feedback model developed? And with that understanding of context, might a subsequent analysis of contextual differences between the publication and the residency programme’s dissimilar contexts have helped avoid this education setback? Fundamentally, did the lack of contextual descriptions lead to a myopic view of the educational intervention?

Let’s first examine SoTL, which is defined as “the description and dissemination of effective and novel teaching methods and strategies, in a research presentation or publication” (Steinert, 2017).  While standards for SoTL in health professions education (HPE) have been proposed (Glassick, 2000), including the need to describe goals, preparation, methods and results, there is scant mention of the need to describe the context within which the novel methods or strategies were operationalised or implemented. So while SoTL remains effective for disseminating novel teaching methods, the variable extent to which context is described (Bates & Ellaway, 2016) may result in challenges in implementing such methods in a different environment; key contextual enablers for successful implementation may have been inadequately described within the HPE SoTL literature. In contrast, the general education literature has long been aware of the importance of context in SoTL (Felten, 2013). There is therefore a blind spot in the HPE SoTL literature.

We next examine context. While we highlight rich descriptions of context for the value it brings to SoTL, we pause to reflect: how do we define context? Context can be difficult to define. A scoping review (Bates & Ellaway, 2016) concluded that one perspective was context as a “surrounding”, much like the layers of an onion, with a particular context playing a role as a mechanism influencing education outcomes. Employing these twin perspectives of “context as an environment surrounding an education activity”, and “context as a mechanism” (Bates & Ellaway, 2016) influencing said activity, we can then view context as surrounding and influencing the educational method, its implementation and its outcomes.

Given the many elements within the context that may influence outcomes, how do we then systematically identify and dissect these disparate elements? The analogy of an onion with surrounding layers (Bates & Ellaway, 2016) led us to consider Bronfenbrenner’s Ecological Systems theory (EST) (Bronfenbrenner, 1986). In EST, multiple systems (micro-, meso-, exo-, macro- and chrono-), much like layers of an onion, influence an individual’s learning. EST can be used to identify, dissect, and categorise contextual influences, and determine if they enable or inhibit educational activities.

In our scenario, the original SoTL work did not fully describe the context. Let us now imagine that the situation was clarified by us writing to the authors to learn more about their context. We are then rewarded with a rich, three-page description of their context. Using EST to dissect the differences between the US context of the intervention, versus the Singapore context of the residency programme, we now realise there were differences at multiple EST system levels, for example:

1. Microsystem: medical students vs residents as learners and feedback recipients, university faculty vs clinician faculty as feedback providers
2. Mesosystem: uniprofessional vs multiprofessional peers and colleagues, undergraduate vs postgraduate curricula
3. Exosystem: university vs clinical training environment, academic workload vs clinical workload
4. Macrosystem: cultures of medical school vs residency, cultures of university vs medical profession, societal cultures of the US vs Singapore
5. Chronosystem: historical perceptions of feedback and utility of feedback in the US vs Singapore

 

With these different EST system levels in mind, one can identify enablers and inhibitors to successful implementation of the published feedback model in Singapore:

1. Microsystem: residents and/or clinician faculty may be busy or distracted by concurrent clinical duties, thus less willing or able to deliver actionable feedback using the model, vs university lecturers who had dedicated time for feedback sessions
2. Mesosystem: while feedback was institutionalised in the US medical school as a longitudinal aspect of the curriculum since 10 years ago, allowing easier integration of a new model into a mature curricular element, adding a new feedback model into a one-year-old programme’s curriculum and implementing it added more stress to a new programme still in flux
3. Exosystem: the US medical school had several resources that the local programme did not. The American researchers had many dedicated teaching rooms for feedback provision to the medical students. In contrast, the surgical residents had to compete with other residents and users for fewer rooms in the local hospital that were also used for multiple clinical, administrative and research purposes. The university also had a mature e-portfolio system where faculty and students could review goals, milestones and progress to facilitate feedback provision, while the new residency programme did not.
4. Macrosystem: feedback was viewed positively by university faculty and students as a key learning activity, with the school taking pride in providing actionable feedback as part of its culture and values. The school’s Dean also publicly affirmed support for the new feedback model. In contrast, the new residency faculty were still unused to providing structured feedback, or inviting reflection as part of feedback; some even viewed feedback as a chore rather than as a vehicle for learning and improvement. The nascent feedback culture of the residency faculty had not fully taken root yet, unlike in the US school.
5. Chronosystem: Historical perceptions of feedback differed in the US vs Singapore, with feedback considered valuable for learning and improvement in the US. In Singapore however, feedback was viewed by some senior surgical faculty members as being useful only when mistakes were made by residents, whereupon forceful negative feedback was given by faculty to the resident in the name of patient safety, rather than for learning. These views from the local senior faculty were informed by their prior experiences as trainees in earlier training systems, leading to their rejection of the new feedback model as being “soft” and compromising patient safety.

 

With a rich description of context, and using EST as a tool, one can now see how the different system layers surround and envelope the faculty, residents and their feedback interaction. One can also see how contextual differences in these system layers (in the US vs Singapore) influenced the success or failure in implementation of the new feedback model. If rich contextual information was provided in the SoTL literature at the start then this information, considered with EST, might have helped the residency programme director avoid the implementation failure.

Successful understanding and application of SoTL in HPE thus relies not only on the six goals espoused by Glassick (Glassick, 2000), but also requires adequate descriptions of context. Readers can then understand contextual differences, use EST to compare and contrast it to their context, identify differences at various EST system layers and determine the potential influence of these differences.

Conversely, the general education literature emphasises that SoTL should be “grounded in context” (Felten, 2013). Felten explicitly states “… all SoTL is rooted in particular classroom, disciplinary, institutional, and cultural contexts” and that “any measure of good practice must account for both the scholarly and the local context where that work is being done” (Felten, 2013). The primacy of context is stated, clearly and unambiguously.

In summary, while we have made progress in SoTL in HPE, we have not adequately considered context in our SoTL guidance (Glassick, 2000) compared to our general education colleagues (Felten, 2013). This underemphasis on context may result in sparse descriptions of context in the HPE SoTL literature, leading HPE readers to be myopic and failing to see the myriad contextual influences affecting understanding and translation of the described SoTL methods to the reader’s context. If we had richer descriptions of context in the SoTL literature, however, we can then use the ‘context lenses’ to clearly view the surrounding layers that influence education outcomes (Bates & Ellaway, 2016). Finally, with visual clarity, we can then dissect and analyse these layers via mapping them to systems levels using EST (Bronfenbrenner, 1986), so that effective translation and implementation of the described SoTL methods can take place. It is time to correct our myopia by collectively advocating for the rich descriptions of context in our HPE SoTL literature.

Notes on Contributors

Dr Kevin Tan reviewed the literature and developed the manuscript. Dr Foo Yang Yann reviewed the literature and gave critical feedback to the writing of the manuscript. Dr Nigel Choon Kiat Tan reviewed the literature and gave critical feedback to the writing of the manuscript. All authors have read and approved the final manuscript.

Funding

The research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of Interest

Authors have no conflict of interest, including financial, institutional and other relationships that might lead to bias.

References

Bates, J., & Ellaway, R. H. (2016). Mapping the dark matter of context: A conceptual scoping review. Medical Education, 50(8), 807-816. https://doi.org/10.1111/medu.13034

Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22(6), 723-742. https://doi.org/10.1037/0012-1649.2 2.6.723  

Felten, P. (2013). Principles of Good Practice in SoTL. Teaching & Learning Inquiry: The ISSOTL Journal, 1(1), 121-125. https://doi.org/10.2979/teachlearninqu.1.1.121

Glassick, C. E. (2000). Boyerʼs expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Academic Medicine, 75(9), 877-880. https://doi.org/10.1097/00001888-200009000-00007

Steinert, Y. (2017). Scholarship in medical education. International Journal of Education and Health, 1(1), 3-4. https://doi.org/10.17267/2594-7907ijhe.v1i1.1657

*Kevin Tan
Office of Neurological Education,
Department of Neurology,
National Neuroscience Institute
11 Jalan Tan Tock Seng,
Singapore 308433
Email: kevin.tan@singhealth.com.sg