Submitted: 6 April 2024
Accepted: 10 December 2025
Published online: 1 April, TAPS 2025, 10(2), 57-64
https://doi.org/10.29060/TAPS.2025-10-2/OA3332

Chatchai Kreepala¹, Srunwas Thongsombat², Krittanont Wattanavaekin³, Taechasit Danjittrong⁴, Nattawut Keeratibharat⁵ & Thitikorn Juntararuangtong¹

¹School of Internal Medicine, Institute of Medicine, Suranaree University of Technology, Thailand; ²Department of Orthopedics, Faculty of Medicine, Prince of Songkla University, Thailand; ³Department of Surgery, Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Thailand; ⁴Department of Anesthesiology, Chulabhorn Hospital, Thailand; ⁵School of Surgery, Institute of Medicine, Suranaree University of Technology, Thailand

Abstract

Introduction: Factors affecting decision-making duration in MCQs can enhance assessment effectiveness, ensuring they accurately measure the intended objectives and address issues related to incomplete exams due to time constraints. The authors aimed to explore the aspects of medical student’s perspective regarding the factors influencing their decision making on MCQ assessments.

Methods: A mixed-methods explanatory sequential design was utilised. Initial surveys were conducted using percentages, mean and non-parametric analysis obtained via online questionnaires from the sample group: all 2^nd – 5^th year medical students from SUT, Thailand. The validity of the questionnaires was verified by three independent reviewers (IOC=0.89). This was followed by semi-structured group interviews to explore student’s perspective on the factors affecting their decision. Qualitative analysis was conducted to explore detailed information until data saturation was achieved.

Results: Data from the quantitative analysis identified four factors that students believe affect the duration of the exam: the total word count of each question, test difficulty, and images in tests. Meanwhile, the qualitative analysis provided additional insights on factors such as the examination atmosphere affecting their decisions.

Conclusion: This report indicated that data acquired from a comprehensive reading question should be distinguished from those requiring decisive reading. Apart from text length, question taxonomy-such as recall or application- and questions with given images and tables should be considered as factors determining time allocation for an MCQ. Future research based on these results should aim to develop a mathematical formula to calculate exam duration, accounting for question difficulty and length.

Keywords:           MCQ, Medical Assessment, Medical Education, Testing Time Estimation, Qualitative Research, Students’ Perspective

Practice Highlights

• The multiple-choice question (MCQ) stands as one of the objective assessment methods, widely regarded as the most utilised form of assessment.
• The word-length effect has been proposed to determine the length of each examination.
• Educational theories on decision-making have posited that decision-making is a dynamic process stemming from prior experiences.
• The authors were interested in exploring the aspects of the medical student’s perspective about the factors affecting their decision on MCQs answering.

I. INTRODUCTION

The multiple-choice question (MCQ) stands as one of the available objective assessment methods, widely regarded as the most utilised form of assessment, particularly within the fields of medical sciences and technology. Evidence suggests that the recall of short words often surpasses that of longer words (Tehan & Tolan, 2007). This observation is frequently analysed within the framework of a working memory model and the role of the phonological loop in immediate recall. However, the word-length effect has also been observed in delayed tests and in lists that surpass the memory span, thereby challenging the working memory interpretation of the phenomenon. Three alternative interpretations of the word-length effect have been proposed to explain how an exam length should be determined (Arif & Stuerzlinger, 2009; Kumar et al., 2021).

Educational theories on decision-making have posited that decision-making is a dynamic process stemming from prior experiences (Phillips et al., 2004) and meaningful learning (Foley, 2019). As a result, the ability to comprehend text while reading does not automatically equate to reading for decision-making or answering questions. From the literature, the context of factors influencing medical students’ decisions on MCQs includes 1) Length or number of words: The time students need to read to gather information before making a decision on an answer (Arif & Stuerzlinger, 2009). 2) Difficulty of the questions: analytical thinking, especially calculations are involved, may increase decision-making time. This depends on the students’ prior learning experiences before the exam (González et al., 2008). 3) Language comprehension: since exams in medical schools are often in English, non-native speakers may take longer to read and understand the questions (Schenck, 2020). 4) Visuals and tables: these serve as symbols that help students retrieve information from their prior learning experiences more easily (Ziefle, 1998). It is certain that teachers want academic assessment tests, such as MCQs, to be used to distinguish between high-performing and low-performing students and to assess the knowledge and understanding they have acquired. However, these objectives may be undermined by issues such as students running out of time and resorting to guessing. This inevitably reduces the reliability of the test.

The authors were interested in exploring medical student’s aspect regarding factors affecting their decision on MCQs answering. Previous studies focused on duration required for question comprehension and understanding but not for analysis. These were also mostly done in native Englisher speakers. This study builds upon previous studies but with an emphasis on factors affecting non-native English speakers’ decision making after analysis of the provided questions to answer MCQs in English. This research should be approached from the perspective of the student to obtain appropriate data. Semi-structured qualitative interviews were analysed in conjunction with quantitative data to identify and clarify the reasons and factors that students believe influence their performance on exams.

II. METHODS

A. Study Population

The research participants were second to fifth-year Thai medical students who had taken MCQ tests during their preclinical and clinical years between the academic years 2021-2022. Questionnaires were sent to all students without sampling.

To minimise data artifacts caused by recall bias, the online questionnaires were distributed the first week after each MCQ test before the study to the students who completed the exams. All examinations in this study were computer-based, closed book, single best answer MCQs written in English. The participants were non-native English speakers of Thai nationality (as detailed in Definition of Terms). An online survey or questionnaire-based study was used to collect information from participants in this study. If the data was unsaturated, triangulated data from a group of interviews consisting of students from different rotations was included to receive as much information from students’ perspectives as possible.

B. Study Design and Data Collection

The authors employed a mixed method study comprising a quantitative approach and a sequential, explanatory approach. The literature review unveiled several factors influencing MCQ test duration, including the number of questions, question types (recall or comprehension), subject matter difficulty, calculation items, and picture identification, as outlined in the questionnaire (O’Dwyer, 2012). 

An online survey or questionnaire-based study was used to collect information from participants with minimal disruption to their learning activities. The quantitative research section was managed by CK, NK and TJ.  Students completed the questionnaire once, based on their experiences in medical school. This, therefore, necessitates the researcher to summarise the responses and, if required, categorise interviews into groups according to their year of study. Open-ended questions were included in the last section of the questionnaire. The open-ended questions about the factors that, in the student’s opinions, were helpful information about the other factors affecting MCQ time (Lertwilaiwittaya et al., 2019). Survey research was employed as a quantitative method, while semi-structured group interviews were utilised in qualitative data collection to gather insights from medical students’ perspectives. The interview questions were designed to investigate whether students possessed any additional insights regarding the factors influencing MCQ test duration (Carnegie Mellon University, 2019; Schenck, 2020; Wang, 2019).

There were three sections in the questionnaire. Part I consisted of the instruction and informed consent. Part II consisted of general information of the participants, including sex, age, and academic year. Part III consisted of the questionnaires covering all four constructive domains previously mentioned (the domains affecting MCQ time from the literature included: 1) the number and total word count, 2) English language questions, 3) calculation questions, 4) the analytical thinking questions and open-ended questions about the factors that, in the students’ opinions, were helpful information about the other factors affecting MCQ time. After the questions in Part One were completed, they would be taken away so that the researchers would not be able to identify whose students have answered Part II and Part III.

To prevent neutral opinions from students, each questionnaire item featured a four-point Likert scale corresponding to levels of agreement: ‘Strongly disagree,’ ‘Disagree,’ ‘Agree,’ and ‘Strongly agree.’ The researcher wanted clear opinion whether the students were trending towards which side, hence the four-point Likert scale to prevent neutral opinion which may complicate statistical analysis. Validity of the questionnaires were verified by three independent reviewers with an Index of Item-Objective Congruence (IOC) value of 0.89.

Semi-structured group interviews were adopted into this study as insufficient flexibility is provided by a structured interview, whereas unstructured interviews would be too flexible. Semi-structured group interviews were the combination of formal and informal interviews focusing on personal experience; this often leads to unexpected results, enhancing the quality of data collected.

These interviews would take place after class by independent interviewers without any conflict of interest. Two facilitators were present in each session, CK facilitated the conversation and NK contributed ideas. The two facilitators were known by the student participants as faculty members, but they were not actively engaged in their academic learning.  Audio and written recording would be coded then decoded by the researchers (SK, KW and TD).

The interview would take around 30-45 minutes per group, with each group consisting of five to eight people. Analysis would be done after the first three groups using relevant domain analysis and further analysis done after new interviews until data saturation was achieved.  Coding, theme identification, and triangulation would be undertaken following the analysis and evaluation of the quantitative and qualitative data of which the analysis could be extrapolated to form a conclusion of the study. In this study, the open-end question would be analysed, and the semi-structured interview would be done.

Triangulation helped to provide meaning and helped to gain broader and more precise understanding. It could help increase validity. Triangulation was undertaken following the analysis and evaluation of the quantitative and qualitative data of which the analysis was extrapolated to form a conclusion of the study.

C. Definition of Terms

1) Multiple choice question (MCQ): This paper exclusively focused on the Single Best Answer (SBA) Multiple Choice Questions (MCQs), which were structured as questions followed by 4 or 5 potential answers, with only one correct response per question (Coughlin & Featherstone, 2017).

2) Taxonomy MCQ: MCQs were formulated based on two assumptions: that they could be categorised into higher or lower orders according to Bloom’s taxonomy (Stringer et al., 2021). This study sought to comprehend students’ approaches to questions by examining variances in their perceptions of the Bloom’s level of MCQs regarding their knowledge and confidence. The authors employed Bloom’s taxonomy in this study, classifying questions as “recall,” “comprehension,” and “application” (Stringer et al., 2021).

3) Non-native English speakers: The term non-native English speakers was defined as those students who spoke a language other than English domestically. Non-native English speakers were inclusive of both competent bi-literate and limited English proficiency students. In addition, it is also defined as students who learn the language as older children or adults (Cassels & Johnstone, 1984).

D. Statistical Analysis

Statistical analyses were performed for quantitative analysis with SPSS Statistics for Windows, Version 18.0 (SPSS Inc., Chicago, Illinois, USA). Information in the quantitative section was elaborated and displayed in and counts percentage. The qualitative data was analysed by code grouping of text fragments based on content. Subsequently, the codes were reorganised and grouped, main themes and subthemes were identified, and illustrative quotations were selected. The authors assigned other three medical teachers to undertake independent coding of the transcripts for each interview. The final coding and discussions continued until the frameworks were agreed upon and new themes were derived (CK, SK, KW and TD).

III. RESULTS

A. Demographic Information

The questionnaire was done online by the participants from second to fifth-year medical students in the academic year 2021-2022. There were 93 second-year medical students, 92 third-year medical students, 92 fourth-year medical students, and 93 fifth-year medical students, respectively, with 370 participants in total. It was found that there were 298 respondents (a return rate of 81%). 73 second-year medical students (78% response rate) answered the questions, while 70 third year (76%), 75 fourth year (81%), and 80 fifth year medical students (86%) answered the questions respectively as shown in Table 1.

D. The Number Tests and Total Word Count

The exam questions, according to some students, were challenging and time-consuming, and the answer options were likewise lengthy. It was shown that not only the number of tests, but the length of each test item also affected the testing time.

Quote: Student B1F*; “The questions were too long. I can’t complete them in time.”

Quote: Student A2M*; “If there are too many questions in the exam, I wouldn’t be able to finish it”

* student’s code

English Language Questions and Examiners (Native Versus Non-Native English Speakers): The respondents, who were not native English speakers, believed that the English-language test took longer to finish than the Thai-language test. Accordingly, they decided to guess or answer each question slowly since they could not understand the questions. They believed that the English-language tests took longer to finish than the Thai-language tests. Accordingly, they decided to guess or answer each question slowly since they could not fully understand the English questions.

Quote: Student D1F*; “I’m not good at reading English. Sometimes I just have to guess on the exam.”

Quote: Student C1M*; “The language in the test is too hard to understand.”

* student’s code

E. Test Difficulty Determining Time Allocation

For the analysis of coding, grouping, and generating themes, the author found that the medical students paid attention to the difficulty level of the questions which affected the decision to answer the questions.

1) The Calculation and Analytical Thinking: The calculation and analytical thinking tests took students longer to read. Additionally, students believed that examinations they had never taken before or exams that required knowledge application took longer to complete, such as exams that included questions requiring the students to diagnose patients by themselves which occasionally left them unsure of how to respond.

Quote: Student C2M*; “Calculation tests take a long time to get the answers.”

* student’s code

2) Recall Question Leads to Quick Answers: Students commented that recall-type questions, including tests from previous academic years, contained duplicated sentences, pictures, or messages from textbooks that students remembered. This led to students being able to complete the test in a short thinking time.

Quote: Student K1M*; “If the teacher copied the exact words from the course sheet, I would remember and answer questions quickly.”

Quote: Student L1M*; “If the questions are the same as in the sheet provided, I can answer them.”

*student’ code

This information indicated that the taxonomy of the test (recall -compression-application) had a large effect on decision time. Applied questions, not direct or calculated questions, required more attention and time for decision-making when compared to comprehensive questions (questions about knowledge understanding). In contrast, recall questions required the least decision-making time.

F. The Visual Image and Atmosphere of the Examination: the New Derived Domains Recognised by Qualitative Analysis

1) Questions with images, graphs, or tables serve as key guides for decision-making: The students thought that the exams that consisted of graphs and tables helped them understand the questions and were better than the questions that only had descriptions. That would lead to less time consumed.

Quote: Student L2M*; “If the test got the exact same summary table from the book, I could remember and get the answers right away.”

* student’s code

2) The Atmosphere of the Examination: The environment and atmosphere of the exam were also mentioned. The student’s response time was slowed down by the distractions during the exam. The environment such as brightness, temperature, and examination devices affected the concentration of the students.

Quote: Student H1F*; “The atmosphere in the exam venue, noise, and the air quality in the room affect the exam results.”

*student’s code

IV. DISCUSSION

The results revealed that students perceived lengthy exam content or a large number of questions as time-consuming, particularly when exams were conducted in English. Studies indicated that English speakers could read up to 150 words per minute (Trauzettel-Klosinski et al., 2012). However, for non-native English speakers, the expected reading time for exams was longer. Hence, using the English reading rate as a basis for determining exam duration was deemed unsuitable for Thai students, given that English was not their primary language of communication. When compared with a previous study (Trauzettel-Klosinski et al., 2012), the increased duration may result from decision making, thus this implicates reading for decision making requires more time than reading for the context which is cumulatively longer for non-English native speakers.

Qualitative findings indicated that irrespective of the exam duration set by the administering professor, students generally completed exams within the allotted time frame. This often entailed guessing answers towards the end of the exam period, as students might not have adequate time to complete the exam thoroughly. It was observed that students tended to resort to guessing exam questions approximately five minutes before the exam conclusion, thereby minimising threats to validity posed by guessing due to time constraints during exam (Foley, 2019).

There may be limitations if the exam questions contain lengthy content that cannot be comprehended and decided upon within one minute. Furthermore, the difficulty level of the exam questions is often established as a passing criterion, prioritising validity considerations in terms of content format and achieving the intended objectives. Moreover, students naturally desire to obtain the highest possible score on the exam, regardless of the level of difficulty or length of the exam. Therefore, it is important for students to manage their time effectively to ensure they can complete all the exam questions within the given timeframe.

The qualitative results indicated that regardless of the exam duration set by the administering professor, students ultimately would complete the exam within the allotted time frame. Additionally, students agreed that application and calculation questions on the exam require more time to read and decide upon, as opposed to questions with figures and tables that aided in faster decision making. Based on these findings, it could be concluded that comprehensive reading rates may not be a reliable indicator of decision-making reading rates, particularly in the context of medical school exams. Therefore, studying decision-making reading rates within the context of medical school exams was crucial.

The researcher, therefore, examined the domain and specific factors on the characteristics of the MCQ test. Additionally, the study scope was limited to English tests administered to non-native English speakers and onsite computer-based tests, thereby eliminating unrelated factors that could impact exam duration. The analysis yielded the following results: Firstly, factors positively correlated with exam duration (negatively correlated with decision-making) included the number of questions, total word count, calculation questions, and analytical thinking questions. Secondly, factors negatively correlated with exam duration (positively correlated with decision-making) were recall questions, questions with provided images, and tables.

A factor contributing to longer reading times for decision-making purposes was when the exam contained a higher proportion of application or calculation questions, comprising over 33% of the exam questions, as evidenced by qualitative data from students. Therefore, analysing exam completion time based on reading comprehension data for decision-making purposes is not recommended. Moreover, it should be noted that these factors present internal threats to validity, but they can be managed to ensure that examination tools are effectively used and aligned with intended objectives. Incorporating data from research can lead to the identification of new themes related to factors influencing examination time.

Five constructive domains were identified: 1) the number and total word count, 2) positive difficulty factors (application/calculation questions), 3) negative difficulty factors (recall questions), 4) examiners (non-native English speakers or not), and 5) pictures/symbols in tests.

A distinctive aspect of this study was its targeted focus on Thai medical students who were non–native English speakers. While many studies have examined MCQ performance across broad and diverse populations, this research concentrated on a specific demographic, enabling a more in-depth exploration of how cultural and linguistic factors influence test-taking behaviour. The study uniquely combined quantitative survey data with qualitative insights from semi-structured group interviews. While some research utilised either quantitative or qualitative methods, this study’s integration of both provided a more holistic understanding of student perspectives and experiences (Lertwilaiwittaya et al., 2019). This methodological triangulation strengthened the validity of the findings by cross-verifying quantitative data with qualitative insights. In contrast to many existing studies that focused predominantly on performance metrics (such as scores or pass rates), this research examined the cognitive processes and decision-making strategies students employed while answering MCQs. It investigated how elements like question difficulty, language comprehension, and prior experiences shaped students’ approaches to test questions—a dimension less frequently explored in previous literature.

In conjunction with examination-related factors, students also recognised the importance of considering the test environment within the examination room which was a new finding found using qualitative analysis from this research. This was crucial for promoting student concentration and facilitating accurate response selection in line with assessment tool objectives. It aligned with existing literature, which suggested that the test environment poses a construct irrelevant threat to the validity of educational measurement. The findings from this study may have led to future research on developing a mathematical formula to tailor the exam duration for different sets of questions. This would have involved analysing factors such as the number of words, length, difficulty, and the presence of images and tables in the exam. Additionally, the impact of language proficiency on reading and decision-making time should have been considered, as there may have been differences between native and non-native speakers. The study suggested that the future research direction should include diverse populations of non-native English speakers from different countries and educational contexts. This could help identify whether the findings are consistent across various cultural backgrounds and educational systems. Moreover, conducted longitudinal studies should be used to track students’ performance and decision-making processes over time. This approach could provide insights into how experiences and familiarity with MCQs influence their strategies and confidence levels throughout their medical education.

A major limitation of this research was the variation in learning experiences, exam-taking skills, and analytical thinking among medical students at different year levels, which might lead to differing opinions. Therefore, the researcher needed to conduct qualitative analysis to examine the reasons behind these differences. However, the diversity of experiences might also introduce bias due to varying familiarity with different types of exams. The online format restricted the depth of responses, as students often did not fully articulate their thoughts without immediate follow-up questions, which limited the richness of the qualitative data. Additionally, the focus on Thai medical students constrained the applicability of the findings to other populations or contexts, thereby limiting broader conclusions about non-native English speakers in different educational settings.

V. CONCLUSION

Based on the student’s perspective, data showed questions with lengthy content required more time whilst those with tables or diagrams required less time. This report indicated that the data acquired from a comprehensive reading examination should be distinguished from a decisive reading examination.

In addition to the number of questions and the length of text, factors that should be positively correlated with the duration of the exam include the number of questions, word count, calculation-based questions, and analytical thinking questions. These factors should be considered for additional time allocation beyond the regular exam duration, particularly when the proportion of analytical thinking questions exceeds one-third of the total question set. On the other hand, recall questions, as well as questions accompanied by images and tables, should be taken into account to ensure a balanced distribution of exam time, as they can be answered more easily and quickly in terms of decision-making compared to general questions.

Notes on Contributors

CK conceived of the presented idea, developed the theory, and performed the computations and discussed the results and contributed to the final manuscript. ST, KW, and TD. discussed the results and wrote the manuscript with support from CK, NK, and TJ, designed the model and the computational framework and analysed the data.

Ethical Approval

All participants voluntarily signed a consent form prior to participating in the study. The participation protocol was approved by the Human Research Ethics Committee, Suranaree University of Technology (Issue # EC-64-102).

Data Availability

Institutional ethical clearance was given to maintain the data in the secure storage of the principal investigator of the study. The data to this study may be provided upon reasonable request to the corresponding author. A preprint of our manuscript, which is not peer-reviewed, is available at https://www.researchsquare.com/article/rs-3019852/v1

Acknowledgement

The authors would like to thank the participants of this study, the medical students in the Institute of Medicine, Suranaree University of Technology. Without their passionate participation and input, the validation survey could not have been successfully conducted.

Funding

This work was supported by the Grant of Suranaree University of Technology (contract number SUT-602-64-12-08(NEW)).

Declaration of Interest

The authors have no conflicts of interest to disclose.

References

Arif, A. S., & Stuerzlinger, W. (2009). Analysis of text entry performance metrics [Conference presentation]. 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH), Canada. https://doi.org/10.1109/TIC-STH.2009.5444533

Cassels, J., & Johnstone, A. (1984). The effect of language on student performance on multiple choice tests in chemistry. Journal of Chemical Education, 61(7), 613. https://doi.org/10.1021/ed061p613

Coughlin, P., & Featherstone, C. (2017). How to write a high-quality multiple-choice question (MCQ): A guide for clinicians. European Journal of Vascular and Endovascular Surgery, 54(5), 654-658. https://doi.org/10.1016/j.ejvs.2017.07.012

Carnegie Mellon University. (2019). Creating exams Eberly Center. https://www.cmu.edu/teaching/assessment/assesslearning/creatingexams.html

Foley, B. P. (2019). Getting lucky: How guessing threatens the validity of performance classifications. Practical Assessment, Research, and Evaluation, 21(1), 3. https://doi.org/10.7275/1g6p-4y79

González, H. L., Palencia, A. P., Umaña, L. A., Galindo, L., & Villafrade M, L. A. (2008). Mediated learning experience and concept maps: A pedagogical tool for achieving meaningful learning in medical physiology students. Advances in Physiology Education, 32(4), 312-316. https://doi.org/10.1152/advan.00021.2007

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77, S85-S89. https://doi.org/10.1016/j.mjafi.2020.11.007

Lertwilaiwittaya, P., Sitticharoon, C., Maikaew, P., & Keadkraichaiwat, I. (2019). Factors influencing the National License Examination step 1 score in preclinical medical students. Advances in Physiology Education, 43(3), 306-316. https://doi.org/10.1152/advan.00197.2018

O’Dwyer, A. (2012). A teaching practice review of the use of multiple-choice questions for formative and summative assessment of student work on advanced undergraduate and postgraduate modules in engineering. All-Ireland Journal of Teaching and Learning in Higher Education, 4(1). https://doi.org/10.21427/D7C03R

Phillips, J. K., Klein, G., & Sieck, W. R. (2004). Expertise in judgment and decision making: A case for training intuitive decision skills. Blackwell Handbook of Judgment and Decision Making, 297, 315. https://doi.org/10.1002/9780470752937.ch15

Schenck, A. (2020). Examining the influence of native and non-native English-speaking teachers on Korean EFL writing. Asian-Pacific Journal of Second and Foreign Language Education, 5(1), 2. https://doi.org/10.1186/s40862-020-00081-3

Stringer, J., Santen, S. A., Lee, E., Rawls, M., Bailey, J., Richards, A., . . . Biskobing, D. (2021). Examining Bloom’s taxonomy in multiple choice questions: Students’ approach to questions. Medical Science Educator, 31(4), 1311-1317. https://doi.org/10.1007/s40670-021-01305-y

Tehan, G., & Tolan, G. A. (2007). Word length effects in long-term memory. Journal of Memory and Language, 56(1), 35-48. https://doi.org/10.1016/j.jml.2006.08.015

Trauzettel-Klosinski, S., Dietz, K., & Group, I. S. (2012). Standardized assessment of reading performance: The new international reading speed texts IReST. Investigative Ophthalmology & Visual Science, 53(9), 5452-5461. https://doi.org/10.1167/iovs.11-8284

Wang, A. (2019, July 15). How to determine the best length for your assessment. Pear Deck Learning. https://www.peardeck.com/blog/how-to-determine-the-best-length-for-your-assessment

Ziefle, M. (1998). Effects of display resolution on visual performance. Human Factors, 40(4), 554-568. https://doi.org/10.1518/001872098779649355

*Assoc. Prof. Chatchai Kreepala, M.D.
Institute of Medicine
Suranaree University of Technology
Thailand
+66(93)3874665
Email: chatchaikree@gmail.com

Submitted: 17 April 2024
Accepted: 18 December 2025
Published online: 1 April, TAPS 2025, 10(2), 65-70
https://doi.org/10.29060/TAPS.2025-10-2/OA3336

Rachael Tufui Masilomani¹, Sophaganie Jepsen¹, Maria Lourdes Villaruel¹, Aying Wang¹, Alena Kotoiwasawasa¹, Lusiana Naikawakawavesi¹, Norman Bartolome¹, Claudia Paterson², Andrew Hill² & Maria Concepcion Bartolome¹

¹Basic Clinical Medicine, Fiji National University, Fiji; ²Department of Surgery, Middlemore Hospital, University of Auckland, New Zealand

Abstract

Introduction: The MBBS programme at Fiji National University reduced its teaching weeks from 18 to 14 weeks in 2018. The purpose of this study was to assess student perceptions of learning and teachers following the reduction in the number of teaching weeks from 18 to 14 weeks.

Methods: A questionnaire was created using a modified Dundee Ready Education Environment Measure (mDREEM) tool (23 items). This was comprised of two subscales; Students Perception of Learning (SPL – 12 items) and Students Perception of Teachers (SPT – 11 items). This was circulated to Year 5 MBBS students through an online survey in 2020.

Results: The response rate was 96%. The students regarded their educational environment as positive in both length of teaching weeks, with an overall mDREEM mean score of 63.29 in 18 weeks and 62.03 in 14 weeks. No statistically significant differences were found between 18 week and 14 week scores across mDREEM scores, SPL scores or SPT scores. The highest scoring item over both was ‘teachers are knowledgeable’.

Conclusion: A positive perception was noted for both lengths of teaching weeks. Reducing the teaching weeks from 18 to 14 did not statistically significantly reduce students’ perception of their educational environment. Items identified with low scores will give a window of opportunity for lecturers and to improve these areas. Future studies may explore the use of the five subscales of the DREEM tool and this study can be integrated into further evaluations of educational environment at Fiji National University.

Keywords:           Medical Students, Educational Learning Environment, DREEM Tool, Fiji National University, Teaching, Medical Education

Practice Highlights

• mDREEM scores did not significantly differ between 18 weeks and 14 weeks of teaching.
• The highest scoring item over both weeks was ‘teachers are knowledgeable’.

I. INTRODUCTION

The Fiji National University (FNU) was founded in 2010 by the merging of six academic institutions in the Fiji Islands, including the Fiji School of Medicine (FSM). FNU has continued the FSM’s tradition of educating and training a diverse population of students from Fiji and neighbouring Pacific Island nations.  The Bachelor of Medicine and Bachelor of Surgery (MBBS) programme is a six-year course at FNU. As part of the academic year, each pre-clinical year group have a teaching week block course. During this time, students receive 2-4 hours of scheduled lectures, 2-hour tutorials twice weekly, as well as 2 hours of clinical skills and 2 hours of anatomy, 2 hours of laboratory sessions and a health centre attachment 4 hours a week.

In 2018, FNU reduced the teaching week block course in the MBBS programme from 18 weeks to 14 weeks. The teaching weeks were shortened due to the decision to move specialty courses such as Psychiatry and Public Health to their respective clinical blocks from Years 4 to 6. This adaptation was challenging for the lecturers, who had to reformat their teaching sessions, in addition to providing resources onto an online Moodle platform. It is well established that the educational environment plays an important role in determining the academic success of medical students (Prosser et al., 1999; Ramsden 2003). Therefore, it is important to evaluate the impact of any major changes to the educational environment, such as a reduction in teaching weeks from 18 weeks to 14 weeks.

Previous research has demonstrated that the duration of clinical rotations has been able to be decreased without adversely affecting the academic success of medical students. For example, one group demonstrated that a shortened four-week clinical rotation in Obstetrics and Gynaecology provided enough opportunity for final year medical students to undertake a quality improvement project in the curriculum (Kool et al., 2017).

The Dundee Ready Education Environment Measure (DREEM) is a quantitative tool used to evaluate students’ perceptions of the educational environment in medical schools. The DREEM tool consists of 50 items, each rated on a scale from 0 to 4. It has five domains, allowing for a maximum score of 200. Higher scores indicate a positive perception of the learning environment (Roff et al., 1997).

A systematic review published in 2012 analysed 40 studies reporting DREEM scores from 20 countries. This review demonstrated that DREEM had been used internationally for various purposes, including diagnostic assessments and comparative studies across different groups (Miles et al., 2012). Five studies focused on investigating the impact of a changed curriculum, which was our area of interest (Demirören et al., 2008, Edgren et al., 2010, O’Brien et al., 2008, Riquelme et al., 2009, Till et al., 2004). We identified three key themes: 1) the DREEM tool was able to highlight areas of concern and/or remediation among students (Riquelme et al., 2009 Till et al., 2004), 2) DREEM scores were different in different phases of medical education, with year 3 students having the highest scores, and year 5 students having the lowest scores (Demirören et al., 2008, Riquelme et al., 2009), and 3) the DREEM tool identified clusters of students based on how positively they perceived the curriculum (O’Brien et al., 2008).

The aim of this research was to compare students’ perception of learning and teachers between 18 weeks and 14 weeks, using a modified DREEM (mDREEM) tool which utilises two of the five domains from the original DREEM tool. The authors’ hypothesis was that students’ perception of learning and teaching would be reduced in with a decrease in teaching weeks to 14 weeks compared to 18 weeks.

II. METHODS

A. Participants

An online survey was developed by the authors. This was distributed via email to eligible participants using Google Forms from 12^th December to 17^th December 2020. Eligible participants included the Year 5 MBBS cohort of 2021, as this cohort was exposed to both 18 week and 14 week teaching programmes. Participation in the survey was optional and results were anonymous. Submission of a completed survey was taken as providing informed consent to participate in this research. Full ethics approval was provided through FNU’s College Human Health Ethics Committee (ID: 292.20). Facility approval was granted to conduct the research.

B. mDREEM Tool

The authors selected two of the five domains of the DREEM tool to be included in the survey. The selected two domains included students’ perceptions of learning (SPL) and students’ perceptions of teachers (SPT). The rationale for using only these two domains was that they were the two domains of interest for the teachers, and it was thought that a survey with less questions would be more likely to be filled to completion by more of the students. Using a Likert scale, each item was scored from 0 to 4, with 4 = strongly agree, 3 = agree, 2 = not sure, 1 = disagree and 0 = strongly disagree. Six of the 23 statements in the mDREEM tool were negative statements: 1) the teaching over-emphasised factual learning; 2) the teaching is too teacher-centred; 3) teachers ridicule the students; 4) the teachers are authoritarian; 5) teachers get angry in class and 6) the students irritate the teachers. These were scored in a reverse manner.

The mDREEM tool had a maximum score of 92. The SPL domain included 12 items, with a maximum score of 48. The SPT domain included 11 items, with a maximum score of 44. This research used the following guide to interpret the overall scores:

• 0–23 = Very poor environment
• 24 – 46 = A large number of problems in the environment
• 47–69 = More positive than negative environment
• 70–92 = Excellent

C. Statistical Analysis

Analysis of the collected data was by using R version 4.3.1. Mean scores were reported with standard deviations. Paired t-tests were performed to compare aspects of the mDREEM scores over 18 weeks and 14 weeks, with a statistical significance threshold of p<0.05. Reliability analysis of the mDREEM tool was assessed using Cronbach’s Alpha test, where >0.7 was deemed acceptable internal consistency.

III. RESULTS

Seventy-eight out of eighty-one (96%) MBSS Year 5 students participated in the online survey. Fifty-one (65%) were females and 51% of participants were aged between 23 and 25 years old. Fijian of Indian descent students made up the majority of the responders (67%) followed by other ethnicities (18%) and i-Taueki students (15%). The majority of students resided at the FNU Hostel (69%) while 28% lived within Suva and only 3% lived outside Suva.

Table 1 illustrates the 23 individual mDREEM items with mean scores across 18 weeks and 14 weeks. The item ‘the teaching overemphasised factual learning’ scored the lowest for both 18 and 14 weeks. The item ‘teachers are knowledgeable’ scored the highest for both 18 and 14 weeks.

Items	mDREEM item	Code	18 weeks		14 weeks
			Mean	SD	Mean	SD
1	I am encouraged to participate in class	SPL	3.13	0.91	2.78	1.03
2	The teaching is often stimulating	SPL	2.79	1.02	2.65	0.94
3	The teaching is student-centred	SPL	2.97	0.95	2.79	0.92
4	The teaching helps to develop my competence	SPL	3.06	0.82	2.63	0.92
5	The teaching is well focused	SPL	2.67	1.00	2.55	1.03
6	The teaching helps to develop my confidence	SPL	2.88	1.01	2.79	0.94
7	The teaching time is put to good use	SPL	2.81	1.12	2.60	1.00
8	The teaching over-emphasised factual learning	SPL	1.85	0.92	1.91	0.79
9	I am clear about the learning objectives of the course	SPL	2.77	1.02	2.78	0.98
10	The teaching encourages me to be an active learner	SPL	2.86	1.16	2.97	0.88
11	Long term learning is emphasised over short term learning	SPL	2.59	1.13	2.56	0.98
12	The teaching is too teacher-centred	SPL	2.85	0.80	2.74	0.78
13	The teachers are knowledgeable	SPT	3.18	0.83	3.13	0.80
14	The teachers are patient	SPT	3.01	0.73	2.74	0.89
15	The teachers ridicule the students	SPT	2.49	0.97	2.56	0.97
16	The teachers are authoritarian	SPT	2.49	0.96	2.58	0.91
17	The teachers have good communication skills	SPT	3.04	0.90	2.99	0.71
18	The teachers are good at providing feedback to students	SPT	2.54	1.15	2.86	0.96
19	The teachers provide constructive criticisms	SPT	2.85	0.90	2.92	0.84
20	The teachers give clear examples during class	SPT	2.78	0.91	2.79	0.84
21	The teachers get angry in class	SPT	2.63	1.01	2.63	0.85
22	The teachers are well prepared for their classes	SPT	2.87	0.90	2.92	0.81
23	The students irritate the teachers	SPT	2.23	0.95	2.23	0.83

The mean SPL scores over 18 weeks and 14 weeks were 33.23 (SD 7.38) and 31.74 (SD 7.52), respectively, out of a maximum of 48 (SPL 18 weeks: 69.33%; SPL 14 weeks: 66.13%). The mean SPT scores over 18 weeks and 14 weeks were 30.06 (SD 6.34) and 30.28 (SD 5.74), respectively, out of a maximum of 44 (SPT 18 weeks: 68.32%; SPT 14 weeks: 68.82%). The mean mDREEM scores over 18 weeks and 14 weeks were 63.29 (SD 12.58) and 62.03 (SD 12.01), respectively, out of a maximum of 92 (mDREEM 18 weeks: 68.80%; mDREEM 14 weeks: 67.42%). These results are presented in Table 2.

	SPL 18	SPL 14	SPT 18	SPT 14	mDREEM 18	mDREEM 14
Mean	33.23	31.74	30.06	30.28	63.29	62.03
SD	7.38	7.52	6.34	5.74	12.58	12.01

Table 2. Descriptive summary statistics for 18 and 14 teaching weeks

The mean difference in SPL scores between 18 weeks and 14 weeks was 1.48. This difference was not statistically significant (t (77) = 1.61, p = 0.11). The mean difference in SPT scores between 18 weeks and 14 weeks was -0.22, and this was also not statistically significant (t (77) = -0.43, p = 0.67). The mean difference in overall mDREEM scores between 18 weeks and 14 weeks was 1.27, which was also not statistically significant (t (77) = 1.04, p = 0.30).

The reliability analysis for both 18 and 14 teaching weeks found a Cronbach’s Alpha Test of 0.58 for SPL, which was less than the threshold of 0.7 and 0.84 for SPT, which was greater than the threshold of 0.7. For mDREEM, the Cronbach’s Alpha Test was 0.77, which was greater than the threshold of 0.7 and confirmed acceptable internal consistency for the mDREEM tool.

IV. DISCUSSION

This study has demonstrated that a reduction in teaching from 18 to 14 weeks did not result in a significant decrease in mDREEM, SPL or SPT scores among Year 5 MBBS students at FNU in 2020. This finding refuted the authors’ hypothesis of a reduction in teaching weeks leading to student dissatisfaction of the educational environment.

Both 18 weeks and 14 weeks scored within the range of 47 to 69 out of 92, indicating a ‘more positive than negative environment’ based on predefined thresholds outlined in the methods section. The ‘excellent’ threshold of 70+ out of 92 was not obtained, indicating room for improvement. In particular, the item ‘teachers are knowledgeable’ scored the highest for both 18 and 14 weeks indicating that students perceived their teachers to have high levels of knowledge despite the reduction in teaching weeks. In contrast, ‘the teaching overemphasised factual learning’ was identified as the most significant negative statement and provides an area of focus for the lecturers.

One group used the DREEM tool to assess curriculum changes in an Irish medical school and that found that the new curriculum was associated with students reporting higher mean DREEM scores (Finn et al., 2014). While FNU’s curriculum change did not result in higher mDREEM scores, the maintenance of mDREEM scores is encouraging and provides a framework for ongoing work towards improving students’ perception of their educational environment.

This finding was similar to a previous study reviewing shortened medical education rotations previously discussed in the introduction section of this paper (Kool et al., 2017). Given the concordance with other similar research findings, the authors are of the belief that the results from this study are largely generalisable to MBBS students and lecturers worldwide, despite only being a single-centre study.

A major strength of this study was the high response rate of 96%. This ensures that data collected as part of this research is representative of the population of interest as compared to several studies with lower response rates (Al-Ansari et al., 2015; Ali et al., 2012; Hyde et al., 2018). Moreover, the results are comparable to the high response rates of other DREEM studies conducted internationally (Alraawi et al., 2020; Stormon et al., 2019; Till et al., 2004).

One limitation of this study was using only two domains of the DREEM tool, neglecting the three domains of Students’ Academic Self-Perception (SAP), Students’ Perception of Atmosphere (SPA) and Students’ Social Self-Perception (SSP). The authors selected SPT and SPL as the two key domains for this research, and thought that by selecting the two most relevant domains, that this would shorten the questionnaire and improve completion of the questionnaire amongst participants. Future research at FNU should trial the use of all five domains of the DREEM tool to assess whether this provides further insights into how teaching weeks can be improved for MBBS students. It will be of interest to see whether response rates are reduced with the use of the full DREEM tool in a questionnaire. Furthermore, future surveys should investigate perspectives of MBBS students over a range of year groups, given previous research suggesting that Year 3 MBBS students have higher DREEM scores than Year 5 MBBS students. A second limitation of this study was that the alpha value for SPL failed to achieve the threshold of >0.7, making it concerning that this domain was unable to achieve acceptable internal consistency. However, the authors note that SPT and mDREEM both achieved acceptable internal consistency. A final limitation of this study was the fully quantitative nature of the survey – the authors did not provide an option for students to add comments to this survey. Future surveys should provide an option for students to add comments, in order to provide more insights into the perspectives of MBBS students.

V. CONCLUSION

In conclusion, the students rated FNU’s MBBS programme educational environment positively. A reduction in teaching weeks from 18 to 14 did not result in a statistically significant decrease in SPL, SPT or mDREEM scores. This study identified valuable information for the authors regarding the improvement of educational environment for medical students. Utilisation of these results to guide educational development in these areas needing improvement will be of help in shaping the delivery of quality education to medical students. In addition, this research may open a door for further studies to investigate challenges faced by tutors and link it to the perceptions of medical students in their educational environment. Likewise, this study is important for future studies in evaluating the educational climate for FNU and other local and international universities.

Notes on Contributors

Dr Maria Bartolome is the corresponding author for this research. She is a PBL (problem based learning) lecturer at the School of Medicine, Fiji National University. She was involved in conceptualisation, methodology, investigation, formal analysis, and writing the original draft.

Dr Norman Bartolome is a PBL lecturer at the School of Medicine,  Fiji National University. He was involved in conceptualisation, methodology, investigation, and writing the original draft.

Dr Alena Kotoiwasawa is a PBL lecturer at the School of Medicine,  Fiji National University, and was involved in  conceptualisation, methodology, and investigation.

Dr Rachael Masilomani is a former PBL lecturer at the School of Medicine,  Fiji National University. She was involved in conceptualisation, methodology, investigation, and formal analysis.

Dr Lusiana Naikawakawavesi is a PBL lecturer at the School of Medicine,  Fiji National University, and was involved in conceptualisation, methodology, and investigation.

Dr Maria Villareul is a PBL lecturer at the School of Medicine, Fiji National University. She was involved in conceptualisation, methodology, and investigation.

Dr Sophaganie Jepson is a PBL lecturer at Fiji National University. She was involved in conceptualisation, methodology, investigation.

Dr Aying Wang is a former PBL Lecturer at Fiji National University. He was involved in conceptualisation, methodology, investigation.

Dr Claudia Paterson is a research fellow at The University of Auckland, New Zealand. She was involved in formal analysis, reviewing and editing.

Professor Andrew Hill is a colorectal surgeon and Professor of Surgery at Middlemore Hospital, Auckland, New Zealand. He was involved in supervision, reviewing and editing.

Ethical Approval

Ethical approval was provided through FNU’s College Human Health Ethics Committee (CHHREC) – reference ID: 292.20.

Data Availability

The corresponding author is able to provide researchers access to our anonymised dataset, on reasonable request.

Acknowledgements

The authors acknowledge the efforts of the students in participating in this study.

Funding

No funding was used for this study.

Declaration of Interest

The authors do not have any conflicts of interest to disclose.

References

Al-Ansari, A. A., & El Tantawi, M. M. A. (2015). Predicting academic performance of dental students using perception of educational environment. Journal of Dental Education, 79(3), 337–344. https://doi.org/10.1002/j.00220337.2015.79.3.tb05889.x

Ali, K., Raja, M., Watson, G., Coombes, L., & Heffernan, E. (2012). The dental school learning milieu: Students’ perceptions at five academic dental institutions in Pakistan. Journal of Dental Education, 76(4), 487–494. https://doi.org/10.1002/j.0022-0337.2012.76.4.tb05281.x

Alraawi, M. O., Baris, S. A., Ahmari, N. A., Alshadidi, A. B., Abidi, N. A., & Al Moaleem, M. (2020). Analyzing students’ perceptions of educational environment in new dental colleges, Turkey using DREEM Inventory. Bioscience Biotechnology Research Communications, 13(2), 556–564. http://dx.doi.org/10.21786/bbrc/13.2/29

Demirören, M., Palaoğlu, Ö., Kemahlı, S., Özyurda, F., & Ayhan, İ. H. (2008). Perceptions of students in different phases of medical education of educational environment: Ankara University Faculty of Medicine. Medical Education Online, 13, Article 8. https://doi.org/10.3885/meo.2008.Res00267

Edgren, G., Haffling, A. C., Jakobsson, U., McAleer, S., & Danielsen, N. (2010). Comparing the educational environment (as measured by DREEM) at two different stages of curriculum reform. Medical Teacher, 32(6), e233–e238. https://doi.org/10.3109/01421591003706282

Finn, Y., Avalos, G., & Dunne, F. (2014). Positive changes in the medical educational environment following introduction of a new systems-based curriculum: DREEM or reality? Irish Journal of Medical Science, 183(2), 253–258. https://doi.org/10.1007/s11845-013-1000-4

Hyde, S., Hannigan, A., Dornan, T., & McGrath, D. (2018). Medical school clinical placements – The optimal method for assessing the clinical educational environment from a graduate entry perspective. BMC Medical Education, 18, Article 7. https://doi.org/10.1186/s12909-017-1113-y

Kool, B., Wise, M. R., Peiris-John, R., Sadler, L., Mahony, F., & Wells, S. (2017). Is the delivery of a quality improvement education programme in obstetrics and gynaecology for final year medical students feasible and still effective in a shortened time frame? BMC Medical Education, 17, Article 91. https://doi.org/10.1186/s12909-017-0927-y

Miles, S., Swift, L., & Leinster, S. J. (2012). The Dundee Ready Education Environment Measure (DREEM): A review of its adoption and use. Medical Teacher, 34(9), e620–e634. https://doi.org/10.3109/0142159X.2012.668625

O’Brien, A. P., Chan, T. M. F., & Cho, M. A. A. (2008). Investigating nursing students’ perceptions of the changes in a nursing curriculum by means of the Dundee Ready Education Environment Measure (DREEM) Inventory: Results of a cluster analysis. International Journal of Nursing Education Scholarship, 5(1), Article 25. https://doi.org/10.2202/1548-923X.1503

Prosser, M., & Trigwell, K. (1999). Understanding learning and teaching: The experience in higher education. McGraw-Hill Education.

Ramsden, P. (2003). Learning to teach in higher education (2nd ed.). Routledge.

Riquelme, A., Oporto, M., Oporto, J., Méndez, J. I., Viviani, P., Salech, F., Chianale, J., Moreno, R., & Sánchez, I. (2009). Measuring students’ perceptions of the educational climate of the new curriculum at the Pontificia Universidad Católica de Chile: Performance of the Spanish translation of the Dundee Ready Education Environment Measure (DREEM). Education for Health, 22(1), Article 112.

Roff, S., McAleer, S., Harden, R. M., Al-Qahtani, M., Ahmed, A. U., Deza, H., Groenen, G., & Primparyon, P. (1997). Development and validation of the Dundee Ready Education Environment Measure (DREEM). Medical Teacher, 19(4), 295–299.

Stormon, N., Ford, P. J., & Eley, D. S. (2019). DREEM-ing of dentistry: Students’ perception of the academic learning environment in Australia. European Journal of Dental Education, 23(1), 35–41. https://doi.org/10.1111/eje.12384

Till, H. (2004). Identifying the perceived weaknesses of a new curriculum by means of the Dundee Ready Education Environment Measure (DREEM) Inventory. Medical Teacher, 26(1), 39–45. https://doi.org/10.1080/01421590310001642948

*Dr Maria Concepcion Bartolome
Basic Clinical Medicine Department
School of Medical Sciences (SMS)
College of Medicine, Nursing and Health Sciences (CMNHS),
Fiji National University
Hoodless House, Brown St. Suva, Fiji Islands
+679 3311700
Email: maria.bartolome@fnu.ac.fj

Submitted: 5 June 2024
Accepted: 30 October 2024
Published online: 1 April, TAPS 2025, 10(2), 71-81
https://doi.org/10.29060/TAPS.2025-10-2/OA3424

Mary Xiaorong Chen¹, Meredith Tsz Ling Yeung¹, Nur Khairuddin Bin Aron², Joachim Wen Jie Lee³ & Taylor Yutong Liu⁴

¹Health and Social Sciences Cluster, Singapore Institute of Technology, Singapore; ²Rehabilitation Department, Jurong Community Hospital, Singapore; ³Rehabilitation Medicine, Singapore General Hospital, Singapore; ⁴Clinical Support Services Department, National University Hospital, Singapore

Abstract

Introduction: Transitioning from a novice physiotherapist (NPT) to an independent practitioner presents significant challenges. Burnout becomes a risk if NPTs lack adequate support for learning and coping. Despite the importance of this transition, few studies have explored NPTs’ experiences in Singapore. This study aims to investigate the transitional journey of NPTs within this context.

Methods: Conducted as a descriptive phenomenological study, researchers collected data through semi-structured online interviews with eight NPTs from six acute hospitals across Singapore. Simultaneous data analysis during collection allowed for a reflexive approach, enabling the researchers to explore new facets until data saturation. Thematic analysis was employed and complemented by member triangulation.

Results: The challenges NPTs encountered include seeking guidance from supervisors, managing fast-paced work and patients with complex conditions. Additionally, NPTs grappled with fear of failure, making mistakes and self-doubt. They adopted strategies such as assuming responsibility for learning, developing patient-focused approaches, and emotional resilience. However, a concerning trend emerged with the growing emotional apathy and doubts about their professional choice.

Conclusion: This study provides a nuanced understanding of the challenges faced by NPTs during their transition. The workplace should be viewed as a learning community, where members form mutual relationships and support authentic learning. Recommendations include augmenting learning along work activities, fostering relationships, ensuring psychological safety, and allowing “safe” mistakes for comprehensive learning.

Keywords:           Novice Physiotherapist Transition in Practice, Clinical Learning and Supervision, Mentoring, Emotional Resilience and Support, Safe Learning Environment

Practice Highlights

• Gradual assumption of responsibilities helps Novice Physiotherapists (NPTs) build competence.
• Learning should be augmented along with work activities.
• It is important to support NPTs to overcome the fear of failure and self-doubt.
• NPTs’ ability to negotiate learning and emotional resilience are essential.
• Trusting relationships and a safe learning environment are essential to NPTs’ learning.

I. INTRODUCTION

Novice Physiotherapists (NPTs) are physical therapy graduates with two years or less of clinical practice, and during this transition to independent practitioners in clinical settings, they face significant challenges (Martin et al., 2020; Wright et al., 2018). Despite the expectation  of competence, concerns persist regarding NPTs’ abilities in various aspects of their practice.

It was reported that the persistent challenges faced by NPTs include managing workload, handling patients with complex conditions, seeking adequate guidance, and navigating relationship dynamics (Latzke et al., 2021; Mulcahy et al., 2010). One critical issue is the oversight of NPTs’ “new” status, leading to their assignment of patient loads comparable to experienced practitioners. Consequently, NPTs find themselves under tremendous stress in managing patients with complex conditions and diverse sociocultural backgrounds beyond their abilities (Stoikov et al., 2021; Wells et al., 2021). Workloads and time constraints hinder the development of meaningful connections between NPTs and supervisors, affecting teaching and coping abilities (Rothwell et al., 2021). In the busy clinical environment, NPTs cannot solely rely on their assigned supervisors, the support from senior colleagues around them along their developmental journey is necessary. Unfortunately, studies suggest that inadequate support and guidance from senior colleagues exacerbate these challenges (Forbes et al., 2021; Jones et al., 2021; Phan et al., 2022; Stoikov et al., 2020; Te et al., 2022).

Additionally, as NPTs are inexperienced, communicating with patients, their families, and other healthcare professionals present a significant hurdle in clinical decision-making (Atkinson & McElroy, 2016). The pressure to make informed clinical decisions, drawing upon extensive knowledge and experience, contributes to job-related stress and feelings of inadequacy among NPTs (Adam et al., 2013).

Job stress-related symptoms, including exhaustion, self-doubt, and depression, further impact NPTs’ well-being. These symptoms, akin to burnout, result from a mismatch between the worker’s performance and job expectations (Brooke et al., 2020; Pustułka-Piwnik et al., 2014). Studies reveal that burnout affects approximately 65% of physiotherapists in Spain (Carmona-Barrientos et al., 2020). This is a concern as burnout was found to be correlated positively with intentions to leave the profession (Cantu et al., 2022), leading to low morale, and compromised patient service quality (Evans et al., 2022; Lau et al., 2016).

Studies suggest that ill-prepared PTs may feel inadequate and lack confidence in making decisions which can negatively influence their clinical management and support for patients’ needs. For example, PTs who lack the ability to adopt a person focused approach might not be able to manage patients with chronic lower back pain effectively (Gardner et al., 2017). Furthermore, such impacts are subtle, difficult to pinpoint, and can result in poor care quality, low patient satisfaction and staff morale (Gardner et al., 2017; Holopainen et al., 2020; Marks et al., 2017).

In Singapore, the healthcare system is bifurcated into public and private sectors. Public hospitals, which fall under government ownership (Ministry of Health, 2023), are pivotal in delivering healthcare services. These hospitals are organised into three distinct clusters, each serving specific regions within the country. Table 1 for a comprehensive list of public hospitals categorised by their respective clusters.

Submitted: 25 January 2024
Accepted: 20 December 2025
Published online: 1 April, TAPS 2025, 10(2), 46-56
https://doi.org/10.29060/TAPS.2025-10-2/OA3228

Chhaya Divecha¹, Miriam Simon² & Ciraj Mohammed³

¹Department of Paediatrics, College of Medicine and Health Sciences, National University of Science and Technology, Oman; ²Department of Psychiatry and Behavioural Science, College of Medicine and Health Sciences, National University of Science and Technology, Oman; ³Department of Medical Education, College of Medicine and Health Sciences, National University of Science and Technology, Oman

Abstract

Introduction: Paediatric milestones provide a structured method for observing and monitoring a child’s progress and should be part of core paediatric curriculum. However, a literature review reveals that primary care physicians and pediatricians feel inadequate about their knowledge and practice of developmental paediatrics, thus exposing the lacunae in training.

Methods: An intervention was planned amongst final-year medical undergraduate students in Oman during their paediatric rotation. A 90-minute multimodal active learning module incorporating diverse learning orientations was planned and administered as a skill-lab session. Its effectiveness in learner motivation, engagement, and faculty participation was evaluated using a questionnaire based on the ICAP (Interactive, Constructive, Active, and Passive) framework, administered to students at the end of the session.

Results: Responses of the 62 participants indicated a significant association between their overall experience and tasks related to the active, constructive, and interactive elements of the module (p=0.001). The faculty’s role in facilitating the session significantly contributed to students’ overall experience (p=0.000). On linear regression, active, constructive, and interactive components of the module were moderate to high predictors of the participants’ overall learning experience.

Conclusion: It was beneficial to base the teaching module on established learning theories. Active learning strategies proactively fostered student engagement and self-directed learning during the session. Faculty played an important role in planning and customising the content, flow, and delivery to maximise meaningful learning. Such interactive collaboration, especially for theoretical concepts in medicine, enables better student engagement, providing enhanced opportunities for learning, practice, and feedback.

Keywords:           Active Learning, Child Development, Undergraduate Medical Education, Student Engagement, ICAP Framework

Practice Highlights

• Active learning strategies can foster student engagement in teaching paediatrics.
• The use of interactive collaboration for theoretical concepts in medicine is effective.
• The role of faculty is crucial to maximise meaningful student learning.
• Utilisation of learning theories to design modules is beneficial for successful content delivery.

I. INTRODUCTION

Clinical curriculum while preparing students for patient care frequently focuses on abnormal pathologies with normal development or physiology often being neglected or underemphasised (Densen, 2011). Developmental and behavioral paediatrics are integral components of pediatric clinical practice. Paediatric milestones provide a structured method to monitor a child’s progress with a comprehensive understanding of development across all domains (gross motor, fine motor, language, and social) and thus must be adequately addressed in the core paediatrics curriculum. However, review of literature reveals that primary care physicians and paediatricians feel inadequate about their knowledge and practice of developmental paediatrics, thus exposing the gaps in education and training (Bauer et al., 2009; Beggs et al., 2005; Bright et al., 2019; Chödrön et al., 2021; Comley et al., 2008; Honigfeld et al., 2012).

Large group didactic classrooms often involve passive reception, leading to lack of engagement among learners (Huggett & Jeffries, 2021). Undergraduate medical curriculum is transitioning from a passive, teacher-centered approach to a learner-centered, active learning strategy, which has demonstrated enhanced students’ understanding, retention of complex concepts, improved student motivation, and overall course satisfaction (McCoy et al., 2018; O’Sullivan et al., 2022). Innovative active learning approaches on developmental milestones largely employ live visits to community resource sites along with instructional videos, reflective reports, observing live parent-child pairs, and use of short video clips (Clark et al., 2012; Comley et al., 2008). In addition, interactive DVDs containing animated cartoons and questions, live interviews, case vignettes, case write-ups, stimulus videos, observation experiences, discussion groups, field trips, and personal experiences have been documented as teaching-learning strategies (Fox et al., 2007; Leiner et al., 2011). Digital resources such as “Beyond Milestones”, developed as free online material for medical professionals using real-life developmental assessments, have shown improved scores on knowledge, observational expertise, confidence, and learner satisfaction (Connolly et al., 2014).

Though observing children in real-life scenarios (including wards, outpatient departments, well-baby clinics, schools, community centers, etc.) creates opportunities for interactive and authentic learning, restrictions in availability of exposure, time, and faculty, especially during the COVID pandemic have compounded the woes of clinical training. The aforementioned reasons prompted us to develop an active learning module using the ICAP framework to introduce developmental milestones in the paediatric curriculum. This framework identifies four modes of cognitive engagement in active learning: Interactive (I), Constructive (C), Active (A), and Passive (P). Passive modes of cognitive engagement involve receipt of information, compared to active modes which require learners to physically manipulate information provided (Chi & Wylie, 2014). Engagement further increases in the constructive mode as students generate diagrams, questions, etc., and is maximum in the interactive mode where peers collaborate and co-construct knowledge through the process of questioning and responding during a conversation. Research reveals that learning achievement is lowest at P and increases in the order of A, C, and I (Chi & Wylie, 2014). Given that the ICAP framework involves both interactive and active learning, we hypothesised that its application to the education of developmental milestones would further promote and expand learning and performance of undergraduate medical students. Additionally, it would help identify and address gaps in their knowledge and understanding of developmental milestones.

In this study, we developed an innovative learning module for developmental milestones using the ICAP model. The module was active, interactive, experiential, and grounded in the major theories of learning (behaviorism, cognitivism, constructivism, humanism, and social learning theories) to maximise opportunities for learning. This pilot study aimed at testing the effectiveness of the module in terms of learner motivation, engagement, and faculty participation.

II. METHODS

The current study was conducted at a private medical college in Oman. It was a cohort study with a quantitative survey and qualitative component. Final-year undergraduate medical students doing their 6-week clinical rotation in Paediatrics were introduced to the teaching module during their skill-lab session (in groups of 6 to 8 students). Verbal consent was obtained for learner feedback.

The learning session was planned and conducted by faculty researchers with expertise in paediatrics, developmental psychology, and medical education. The learning outcomes of this module on developmental milestones were:

1. To identify age-appropriate milestones in children from birth to 5 years of age.
2. To apply knowledge of milestones for various domains (gross motor, fine motor, language, and social/cognitive) to assess development in various age groups (birth to 5 years).
3. To differentiate between normal and delayed development in children.

The specific learning outcomes were addressed through various strategies as summarised in Figure 1.

Figure 1. Flow chart depicting the sequence of activities mapped with specific learning objectives

Figure 2. Sample of bingo ticket with milestones

F. Scaffolding

The faculty would wrap up the case and summarise major learning points thus consolidating the knowledge about milestones. Throughout the learning session, the facilitator provided cognitive scaffolding by leading the students through different questions, prompts, tasks, and structured interactions enabling them to learn more about developmental milestones. Students had opportunities to work with their existing knowledge and build further on it through the various learning activities, group discussions and faculty-led facilitation.

G. Reflection on Action

Students were divided into groups (2 or 3 students per group) to reflect on their entire learning experience using Borton’s model of reflection (Rolfe, 2014). Their reflections elucidated the “what”- their experience of the activities, “so what”- how the module improved on their prior knowledge and understanding about the topics and “now what”- providing suggestions for improvement and preference for similar active learning strategies in future sessions. Individual sharing within groups was followed by sharing between groups via their representatives.

H. Data Collection and Analysis

Students were asked to complete a feedback form about their learning experience during the activity via an anonymised electronic feedback form administered at the end of the session. The form had 15 items evaluating their learning experience – 11 quantitative (Likert scale-based) and 4 qualitative questions. The quantitative feedback responses were analysed based on the four domains of the ICAP model- questions were framed for Interactive, Constructive, Active, and Passive engagement of students during the active learning strategies. There were 4 items (Max score=20) about the interactive components of the module, 3 items (Max score=15) based on constructive elements, and two items each (Max score=10) on the active and passive engagement of students during the module. In addition, 4 quantitative items assessed feedback on faculty involvement and students’ overall experience.

Data was analysed using IBM’s Statistical Package for Social Studies (SPSS 22; IBM Corp., Armonk, NY, USA). Descriptive statistical methods such as percentages, mean and standard deviation were employed. Cronbach’s alpha method was used to assess internal consistency/test reliability. The Shapiro-Wilk method was used to test normality. Inferential statistical methods such as the Spearman’s correlation analysis and Chi-square test were used to explore significant associations between variables. Linear regression was also used to explore various predictors of the participants’ learning experience. Thematic analysis was performed for qualitative feedback.

III. RESULTS

A. Overview of Study Participants

A total of 62 students from the final clinical year of the MD program participated in this study; 91% were females and 9% were males. The average age of participants was 24.4 years (SD=0.707).

Results indicate high internal consistency for the survey items developed. Full scale (15 items) Cronbach’s alpha value of 0.808 was obtained. Results of the Shapiro-Wilk test of normality for all survey items (p=0.000) indicate that participant’s responses were not normally distributed. Non-parametric tests were thus employed for data analyses.

B. Analysis of Learner Feedback

In general, participants agreed that the session on developmental milestones was enjoyable.

The preference for game-based interactive/collaborative learning was high. The learner responses categorised item wise are provided in Table 1.

Item	Strongly Agree	Agree	Can’t say	Disagree	Strongly Disagree	Mean	SD
The pre-reading material on developmental milestones was easily understandable	59.7%	35.5%	4.8%	0	0	4.548	0.591
The session established clear learning outcomes and objectives.	91.9%	8.1%	0	0	0	4.919	0.274
The case-based activity on developmental milestones provided relevant opportunity to witness theory in practice.	83.9%	16.1%	0	0	0	4.839	0.370
Presentation of real-world contexts followed by discussion in groups helped me learn better.	88.7%	9.7%	1.6%	0	0	4.871	0.383
The game-based activity enabled me to build an emotional connection to learning and the subject matter.	82.3%	16.1%	0	1.6%		4.790	0.516
This session provided me opportunities for feedback and practice	83.9%	16.1%	0	0	0	4.839	0.370
I enjoyed the session on developmental milestones as it actively engaged me with the course material through case study and discussion.	87.1%	12.9%	0	0	0	4.871	0.338
The session assisted us in organising our thoughts, reflecting on our understanding, and finding gaps in our reasoning.	74.2%	21%	4.8%	0	0	4.694	0.560
I prefer similar sessions to learn theoretical concepts in paediatrics and medicine.	88.7%	9.7%	1.6%	0	0	4.871	0.383
Today’s session was well-planned and helped me work on my own to accomplish learning goals independently.	82.3%	17.7%	0	0	0	4.823	0.385
The faculty supported students in the learning process during the session.	98.4%	1.6%	0	0	0	4.984	0.127
The faculty regulated the level of information and maintained my ability to be responsible for my learning.	87.1%	11.3%	1.6%	0	0	4.855	0.398
The session made me identify psychosocial issues associated with developmental delays	69.4%	21%	8.1%	1.6%	0	4.581	0.714
I found myself motivated, engaged and self-directed during the entire session.	82.3%	14.5%	3.2%	0	0	4.790	0.483
Using the interactive online module to explore developmental milestones helped improve my learning in this area.	66.1%	25.8%	6.5%	1.6%	0	4.565	0.692

Table 1. Analysis of survey responses based on dimensions of the ICAP Model

 

C. Analysis of Feedback on Faculty Involvement and Overall Experience

The mean score on the participants’ feedback on the involvement of faculty members during the session was 9.839 (SD=0.450) and their overall experience was 9.710 (SD=0.686).

D. Association among Various Survey Dimensions

Results indicated significant associations between learner’s overall experience and tasks related to the active component of the session (p=0.000), constructive engagement (p=.000), and interactive collaboration with peers (p=0.001). Results also indicated significant associations between the interactive and constructive components of the session (p=0.000), faculty’s role in facilitating the session and students’ overall experience (p=0.000), and faculty involvement with the passive component (designing the initial reading material) (p=0.000).

E. Spearman’s Correlation

Spearman’s correlation across the various survey dimensions based on the ICAP model indicated high positive inter-dimension correlation. A significant correlation was also seen between the survey dimensions and the full-scale scores.

F. Linear Regression

Linear regression analysis was carried out to explore various predictors of the participants’ learning experience. The active (R²=0.438), constructive (R²=0.718), and interactive components (R²=0.644) are moderate to high predictors of the participants’ overall learning experience.

G. Thematic Analysis of Qualitative Items

The following themes and elaborations emerged on analysis of student feedback relating to their experience during the active learning module on developmental milestones which are summarised in Table 2.

Themes	Elaboration of theme	Samples of students’ feedback
(i) Elevated learning experience	Participants expressed positive feedback regarding all components of the active learning session. They enjoyed the game-based activities, technology-integrated tasks, while at the same time enhancing their knowledge on developmental disorders from a clinical perspective.	“The bingo game brought a surprisingly fun twist to a boring topic and the competitiveness of the group as well as the doctors fun proctoring made it an overall pleasant and enjoyable experience. In short, this was a session I’m glad I got out of bed for.” “The smart board table was an active way of remembering.” “Encourage us to learn more and attend to skill lab. It was wonderful.” “A lot better than what I thought I would feel compared to a normal session. Confident to answer any questions related to developmental milestones.”
(ii) Efficacious collaborative interactions	Students reported increased interest in learning as the session involved working together with their peers. Participants felt that continual interaction throughout the session strengthened learning.	“The group interactions really helped me retain the information. It was a fun experience and something new and out of the ordinary.” “Interactions made the information much easier to understand.” “It was interactive which made it interesting.”  “I interacted throughout the session, learned quite a lot of information, very helpful.”
(iii) Reflective outlook to patient care	Participants appreciated the inclusion of humanism in the learning experience. The incorporation of reflective practice in patient and caregiver experiences is a vital component that may impact future outcomes related to patient care.	“Playing the video which wasn’t pure scientific as usual, it is emotional also, so it touches both the doctor and the human inside me.” “Watching the video of cerebral palsy and then reflecting over why development is very important.”
(iv) Supportive learning environment	Student feedback highlighted the presence of a positive, non-judgmental environment that ensured improvement of learning in a relaxed/ stress-free setting.	“We were able to make mistakes and learn from them without the fear of being criticised” “It was very fun and interactive. No pressure was there and not scary.” “It was interactive and very useful and most important comfortable and didn’t feel like we were being pressured and that surely helped us learn way better.” “Very happy and I enjoyed it a lot. My favourite session since the beginning of the year… no pressure was put on us, so we were able to actually learn instead of being terrified.”

Table 2. Thematic analysis of qualitative feedback

IV. DISCUSSION

Monitoring a child’s development over time via milestones is a core part of paediatric curriculum and practice, as it enables early identification of delay or atypical progress, allowing timely referral. As an alternative to traditional learning through lectures as practiced routinely, we utilised active learning strategies that addressed multiple learning orientations for enhanced student engagement and learning.

A. Designing the Module and Active Learning Strategies based on Learning Theories

The developmental milestone module designed to involve multimodal active learning strategies has been described in the methods section. The session began with a novel “lid-opener” showing a real-life video of a child with developmental delay and the psychosocial impact of the disability on the child and family, followed by an active learning strategy; think pair and share (TPS). We explored the utility of “cognitive” orientations to learning by positioning learners to seek and understand the structure of knowledge for meaningful learning. TPS has been known to enhance the process of clinical learning by allowing students to experience different viewpoints on a particular problem and express as well as to listen to others as compared to a traditional classroom lecture (Ganatra et al., 2021; Linsenmeyer, 2021).

A significant proportion (91%) of students agreed that using the interactive module to explore milestones helped improve their learning on the topic which is also reflected in the qualitative feedback (Table 2). The interactive smart board allowed students to explore milestones in a self-directed activity and enabled higher-order cognition through active engagement with the course content. Technology in learning is stimulating for millennial learners and allows them to use online novel educational tools to maneuver their learning process (George & Dreibelbis, 2021). The wide availability of mobile devices, laptops, smart boards, etc. provides the teacher opportunities to use these new technologies effectively to transform learning into a more collaborative, personalised, and empowering experience that is rooted in connectivism- the learning theory of the digital era. We employed a multimodal approach integrating diverse activities such as videos, interactive smart boards, and gamification (via the BINGO game). Integrating many information sources nurtures the learner’s ability to reflect on connections between fields, ideas, and concepts, a core skill linked to the theory of connectivism. (Goldie, 2016).

Case-based learning (CBL) effectively links theory and practice and prepares students for actual clinical application of knowledge through inquiry (Thistlethwaite et al., 2012). The hypothetico-deductive reasoning inherent to a “constructivist” framework was applied while designing this approach (Kalinowski & Pelakh, 2024). It increases the learner’s engagement and motivation for learning through creativity, challenge, interest, and enjoyment afforded through the case-study method. An important part of CBL involves problem-solving through free discussions and with often no correct or incorrect answers, thus broadening the students’ reasoning process (Thistlethwaite et al., 2012). We used a CBL trigger in the form of a video demonstrating an assessment of a child’s development, allowed students analyse the video individually followed by buzz group discussions to share their understanding. All our students unanimously agreed that the case-based activity gave them ample opportunity to witness theory in practice. Buzz groups also provide an opportunity to apply contextual thinking to actual practice and develop collaborative skills. Studies have shown that buzz groups improve clinical reasoning and learning, promote teamwork, increase motivation to learn, improve academic performance through a conducive learning environment, and provide a chance for all members to participate and share their opinions within their sub-group (Abbasi et al., 2017; Balslev et al., 2015; Shrivastava, & Shrivastava, 2018). The elements of exploration and experimentation ensured that the theory of constructivism rooted in the connection of new knowledge to pre-existing knowledge was effectively utilised in the CBL process. Co-creation of this knowledge with peer interactions can be attributed to social learning theories of cognitivism and constructivism.

Selective and purposeful gamification has been known to increase learner motivation and engagement and, ultimately, learning (Rutledge et al., 2018). This was evident in our study where a vast majority (98%) of students agreed that the game-based activity helped them build an emotional connection with the process of learning and concepts being learned, which is also reflected in the qualitative feedback (Table 2). Game-based learning (GBL) builds emotional connections by immersing learners in engaging and interactive environments. These environments evoke emotions such as curiosity, excitement, and empathy, which enhance motivation and deepen the learning experience (Loderer et al., 2020). Based on constructivist theory, the game-based activity integrated their knowledge of age-appropriate milestones with a fun-based BINGO theme. The instant feedback that served as formative assessment in this segment had a behaviorist orientation that is specific and quantifiable.

B. Student Engagement using Multimodal Active Learning Strategies

Studies have noted a preference for multi-modal learning strategies by medical students which have been shown to improve academic performance, and promote deeper learning and enquiry (Alkhasawneh et al., 2008; James et al., 2011; Lujan & DiCarlo, 2006; Nicholson et al., 2016). Different learners have different learning styles and use of a variety of strategies can engage learners synergistically. We used various active learning strategies to facilitate student engagement as shown in Figure 3 which incorporated the ICAP framework of Interactive, Constructive, Active, and Passive activities. In addition, the use of learning theories to rationalise the framing of learning objectives, selection of specific teaching-learning methods/strategies, and design of appropriate evaluation strategies ensured student engagement and meaningful learning.

Figure 3. Summary of active learning strategies used in the module based on ICAP framework

The module had more activities involving interactive and constructive domains as compared to the active and passive modes. It is known that student learning and engagement improve as they move from passive to active to constructive to interactive modes (Chi & Wylie, 2014). Significant association between overall experience and interactive collaboration with peers (p=0.001) as well as constructive engagement (p=0.000) was observed. The feedback was also structured to measure student engagement in all four behaviors. We observed high mean scores in interactive and constructive domains. Learners agreed on enough opportunities for feedback, and practice and appreciated the engagement through case studies and discussions. Effective use of multiple active learning strategies thus enabled self-directed learning and students felt that they could accomplish the learning goals independently. The results underpin the utility of behaviorism and humanism as orientations for the learning process in such activities.

C. Individual and Collaborative Learning

The combination of active learning strategies facilitated engagement at both individual and collaborative levels. Students were involved in individual learning through pre-reading, lid-opener, discourse on development, case-based learning, and game (BINGO). These activities provided opportunities to self-regulate and moderate their learning. It also allowed them to organise their thoughts and reflect on their understanding. Active learning has a positive impact on memory and knowledge transfer and hence, individual self-studying before discussion improves the effectiveness of collaboration (Beggs et al., 2005). Activities initiated with an individual thinking process were balanced by subsequent group discussions. Collaborative activities in the form of Think pair share (TPS) after lid-opener, Buzz groups after CBL learning, and interactive table enabled constructive interaction among students and exchange of information and concepts. The collaborative activities helped students work together towards problem-solving, observe both their own and another’s learning process, discover different ways of approaching a situation, and find gaps in their reasoning. It also helped to build on their team skills, leadership, clinical competence, and interpersonal communication which are core attributes of professionalism in our curriculum (Branch, 2015).

D. Faculty Involvement

Though activities in the module were planned to encourage self-directedness, faculty role in the preparation and implementation of these strategies cannot be undermined. We adopted an amalgamation of humanistic and social cognitive orientation to strengthen the relationship between quality of instructional design and outcomes, thereby influencing learning directly. Perhaps for the same reasons we found a significant association between faculty facilitation with overall learning experience (p=.000). Students felt that the faculty supported them through the learning process throughout the session (100%) and regulated information besides allowing students to be responsible for their learning (98%). There was also a significant association between faculty involvement and the passive component (pre-reading material, discourse) (p=0.00); students agreed that pre-reading material was easily understandable (95%) and clear objectives and outcomes laid down before the session facilitated their understanding (100%). It must be noted that faculty can personalise the module based on the strengths, interests, cultural competencies, and time restraints of the student. Thus, rather than delivering a “one size fits all” module, the faculty can customise the content based on the student’s needs and limitations.

Involving various sensory processes enables better processing and retention of information; thus, enhancing the learning process (Friedlander et al., 2011). As more methods are employed, they are likely to engage more senses thus improving learning. Neuroscientific rationale for constructivist cognition may be seen as a possible explanation (Dennick, 2016).

While the use of active learning techniques in undergraduate medical education is not entirely novel, our approach is distinctive in that we’ve designed an entirely active learning module addressing various orientations to student learning, which ensures individual and collaborative student engagement. The study’s primary advantage is the creation and application of a targeted educational resource to teach developmental paediatrics to undergraduate students. This not only addresses the gap in effectively integrating developmental paediatrics into the core paediatrics curriculum but also demonstrates the feasibility of using a fully active learning approach in other clinical topics. Our detailed methodology aims to assist other educators in transforming their content into interactive teaching modules. The module not only meets the institutional requirement for increasing active learning sessions in the clinical years but also addresses a broader need by offering a framework and learning strategies that can be effectively applied to different topics and courses.

We could only assess student engagement based on their perception, i.e. level 1 of the New World Kirkpatrick Model (reaction), and have not quantified their ability to learn, understand, and apply their learning which constitutes higher levels of the model (Liao & Hsu, 2019). Additional limitations include the fact that this is a pilot study, tailored to a specific context and curriculum, which may restrict its generalisability. It also does not directly compare with other learning methods and lacks long-term tracking of students.

V. CONCLUSION

Paediatric topics such as developmental paediatrics are delivered mainly through didactic orientations and fail to capture student engagement leading to poor comprehension. Our study demonstrates that medical students enjoy sessions involving multimodal active learning strategies, particularly while discussing theoretical concepts that provide opportunities for practice and feedback. Heterogeneous learning strategies which underpin various learning theories and constructs have been shown to increase student motivation and engagement, thus contributing towards retention and deep learning. The faculty have an important role in planning such modules to customise the content and delivery for successful student engagement and effective learning thereafter. The framework and active learning strategies presented in the module can be applied more broadly to other subjects adapting to the needs of other undergraduate faculty in their teaching, thus making active teaching strategies easily transferable. We recommend future research be planned to include pre- and post-session assessments or a crossover study with a control group for comprehensive evaluation. Furthermore, the implementation of active learning strategies to create entirely active modules in other courses within the undergraduate medical program can be explored to assess its potential for broader applicability.

Notes on Contributors

Dr. Chhaya Divecha, Associate Professor of Paediatrics was involved in the conceptualisation and design of the study, literature search, data collection and drafting the manuscript.

Dr. Miriam Simon, Associate Professor of Behavioral Science was involved in the design of the study, analysis, interpretation of results and drafting the manuscript.

Dr. Ciraj Mohammed, Professor of Medical Education was involved in the design of the study and revised the manuscript for scientific content.

All authors have read and approved the final manuscript.

Ethical Approval

Approval to conduct this study was obtained from the institution’s Ethics and Biosafety Committee (NU/COMHS/EBC0036/2022).

Data Availability

Data will be made available by the authors on acceptance of the manuscript.

Acknowledgement

The authors wish to thank the participants for their cooperation in the study.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interests.

References

Abbasi, F., Mofrad, M. N., Borhani, F., & Nasiri, M. (2017). Evaluation of the effect of training by buzz group method on nursing diagnostic skills of nursing students. Research Journal of Pharmacy and Technology, 10(1), 213-218.  https://doi.org/10.5958/0974-360x.2017.00045.2

Alkhasawneh, I. M., Mrayyan, M. T., Docherty, C., Alashram, S., & Yousef, H. Y. (2008). Problem-based learning (PBL): Assessing students’ learning preferences using VARK. Nurse Education Today, 28(5), 572–579. https://doi.org/10.1016/j.nedt.2007.09.012

Balslev, T., Rasmussen, A. B., Skajaa, T., Nielsen, J. P., Muijtjens, A., De Grave, W., & Van Merriënboer, J. (2015). Combining bimodal presentation schemes and buzz groups improves clinical reasoning and learning at morning report. Medical Teacher, 37(8), 759–766. https://doi.org/10.3109/0142159X.2014.986445

Bauer, S. C., Smith, P. J., Chien, A. T., Berry, A. D., & Msall, M. (2009). Educating pediatric residents about developmental and social–emotional health. Infants & Young Children, 22(4), 309-320. https://doi.org/10.1097/IYC.0b013e3181bc4da0

Beggs, S., Sewell, J., Efron, D., & Orkin, C. (2005). Developmental assessment of children: A survey of Australian and New Zealand paediatricians. Journal of Paediatrics and Child Health, 41(8), 444–448. https://doi.org/10.1111/j.1440-1754.2005.00664.x

Branch, W. T. (2015). Teaching professional and humanistic values: Suggestion for a practical and theoretical model. Patient Education and Counseling, 98(2), 162–167. https://doi.org/10.1016/j.pec.2014.10.022

Bright, M. A., Zubler, J., Boothby, C., & Whitaker, T. M. (2019). Improving developmental screening, discussion, and referral in pediatric practice. Clinical Pediatrics, 58(9), 941–948. https://doi.org/10.1177/0009922819841017

Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520 .2014.965823

Chödrön, G., Barger, B., Pizur-Barnekow, K., Viehweg, S., & Puk-Ament, A. (2021). “Watch Me!” Training increases knowledge and impacts attitudes related to developmental monitoring and referral among childcare providers. Maternal and Child Health Journal, 25(6), 980–990. https://doi.org/10.1007/s10995-020-03097-w

Clark, B., Andrews, D., Taghaddos, S., & Dinu, I. (2012). Teaching child development to medical students. The Clinical Teacher, 9(6), 368–372. https://doi.org/10.1111/j.1743-498X.2012.00581.x

Comley, L., Janus, M., Marshall, D., & Niccols, A. (2008). The early years: Child development in undergraduate medical school training. Canadian Family Physician, 54(6), 876.e1-4.

Connolly, A. M., Cunningham, C., Sinclair, A. J., Rao, A., Lonergan, A., & Bye, A. M. (2014). ‘Beyond Milestones’: A randomised controlled trial evaluating an innovative digital resource teaching quality observation of normal child development. Journal of Paediatrics and Child Health, 50(5), 393–398. https://doi.org/10.1111/jpc.12485

Dennick, R. (2016). Constructivism: Reflections on twenty-five years teaching the constructivist approach in medical education. International Journal of Medical Education, 7, 200–205. https://doi.org/10.5116/ijme.5763.de11

Densen, P. (2011). Challenges and opportunities facing medical education. Transactions of the American Clinical and Climatological Association, 122, 48–58.

Fox, G., Katz, D. A., Eddins-Folensbee, F. F., & Folensbee, R. W. (2007). Teaching development in undergraduate and graduate medical education. Child and Adolescent Psychiatric Clinics of North America, 16(1), 67-94. https://doi.org/10.1016/j.chc.2006.07.006

Friedlander, M. J., Andrews, L., Armstrong, E. G., Aschenbrenner, C., Kass, J. S., Ogden, P., Schwartzstein, R., & Viggiano, T. R. (2011). What can medical education learn from the neurobiology of learning? Academic Medicine, 86(4), 415–420. https://doi.org/10.1097/ACM.0b013e31820dc197

Ganatra, S., Doblanko, T., Rasmussen, K., Green, J., Kebbe, M., Amin, M., & Perez, A. (2021). Perceived effectiveness and applicability of Think-Pair-Share including storytelling (TPS-S) to enhance clinical learning. Teaching and Learning in Medicine, 33(2), 184–195. https://doi.org/10.1080/10401334.2020.1811094

George, D. R., & Dreibelbis, T. D. (2021). Active learning in ‘The cloud’: Using “social” technologies to expand the medical classroom. In A. Fornari & A. Poznanski (Eds.), How-to guide for active learning (pp. 85-99). Springer.

Goldie, J. G. (2016). Connectivism: A knowledge learning theory for the digital age? Medical Teacher, 38(10), 1064–1069. https://doi.org/10.3109/0142159X.2016.1173661

Honigfeld, L., Chandhok, L., & Spiegelman, K. (2012). Engaging pediatricians in developmental screening: The effectiveness of academic detailing. Journal of Autism and Developmental Disorders, 42(6), 1175–1182. https://doi.org/10.1007/s10803-011-1344-4

Huggett, K. N., & Jeffries, W. B. (2021). Overview of active learning research and rationale for active learning. In A. Fornari & A. Poznanski (Eds.), How-to guide for active learning (pp. 1-7). Springer.

James, S., D’Amore, A., & Thomas, T. (2011). Learning preferences of first year nursing and midwifery students: Utilising VARK. Nurse Education Today, 31(4), 417–423. https://doi.org/10.1016/j.nedt.2010.08.008

Kalinowski, S. T., & Pelakh, A. (2024). A hypothetico-deductive theory of science and learning. Journal of Research in Science Teaching, 61(6), 1362–1388. https://doi.org/10.1002/tea.21892

Leiner, M., Krishnamurthy, G. P., Blanc, O., Castillo, B., & Medina, I. (2011). Comparison of methods for teaching developmental milestones to pediatric residents. World Journal of Paediatrics, 7(2), 161–166. https://doi.org/10.1007/s12519-011-0269-5

Liao, S. C., & Hsu, S. Y. (2019). Evaluating a continuing medical education program: New world Kirkpatrick model approach. International Journal of Management, Economics and Social Sciences, 8(4), 266-279.

Linsenmeyer, M. (2021). Brief activities: Questioning, brainstorming, think-pair-share, jigsaw, and clinical case discussions. In A. Fornari & A. Poznanski (Eds.), How-to guide for active learning (pp. 39-66). Springer.

Loderer, K., Pekrun, R., & Plass, J. L. (2020). Emotional foundations of game-based learning. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning (pp. 111–151). MIT Press.

Lujan, H. L., & DiCarlo, S. E. (2006). First-year medical students prefer multiple learning styles. Advances in Physiology Education, 30(1), 13–16. https://doi.org/10.1152/advan.00045.2005

McCoy, L., Pettit, R. K., Kellar, C., & Morgan, C. (2018). Tracking active learning in the medical school curriculum: A learning-centered approach. Journal of Medical Education and Curricular Development, 5, 2382120518765135. https://doi.org/10.1177/2382120518765135

Nicholson, L. L., Reed, D., & Chan, C. (2016). An interactive, multi-modal anatomy workshop improves academic performance in the health sciences: A cohort study. BMC Medical Education, 16, Article 7. https://doi.org/10.1186/s12909-016-0541-4

O’Sullivan, S., Campos, L. A., & Baltatu, O. C. (2022). “Involve Me and I Learn”: Active learning in a hybrid medical biochemistry first year course on an American-style MD program in the UAE. Medical Science Educator, 32(3), 703–709. https://doi.org/10.1007/s40670-022-01545-6

Rolfe, G. (2014). Reach touch and teach: Terry Borton. Nurse Education Today, 34(4), 488–489. https://doi.org/10.1016/j.nedt.2013.11.003

Rutledge, C., Walsh, C. M., Swinger, N., Auerbach, M., Castro, D., Dewan, M., Khattab, M., Rake, A., Harwayne-Gidansky, I., Raymond, T. T., Maa, T., & Chang, T. P. (2018). Gamification in action: Theoretical and practical considerations for medical educators. Academic Medicine, 93(7), 1014–1020. https://doi.org/10.1097/ACM.0000000000002183

Shrivastava, S. R., & Shrivastava, P. S. (2018). Exploring the scope of introducing buzz group in teaching community medicine to undergraduate medical students. MAMC Journal of Medical Sciences, 4(2), 109.

Thistlethwaite, J. E., Davies, D., Ekeocha, S., Kidd, J. M., MacDougall, C., Matthews, P., Purkis, J., & Clay, D. (2012). The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Medical Teacher, 34(6), e421–e444. https://doi.org/10.3109/0142159X.2012.680939

Woolfson, L. (2004). Family well-being and disabled children: A psychosocial model of disability-related child behaviour problems. British Journal of Health Psychology, 9(1), 1–13. https://doi.org/10.1348/135910704322778687

*Dr. Miriam Simon
Department of Psychiatry and Behavioral Science
College of Medicine and Health Sciences
National University of Science and Technology
PO Box 391; PC 321
Sohar, Sultanate of Oman
+96826852039
Email: miriamsimon@nu.edu.om

Submitted: 2 April 2024
Accepted: 12 September 2024
Published online: 7 January, TAPS 2025, 10(1), 27-35
https://doi.org/10.29060/TAPS.2025-10-1/OA3247

Estivana Felaza¹, Ardi Findyartini^1,2, Rita Mustika¹, Jenny Bashiruddin¹, Joedo Prihartono³, Lucia RM Royanto⁴ & Subha Ramani⁵

¹Department of Medical Education, Faculty of Medicine Universitas Indonesia, Indonesia; ²Medical Education Centre, Indonesia Medical Education and Research Institute (IMERI), Faculty of Medicine Universitas Indonesia, Indonesia; ³Department of Public Health, Faculty of Medicine Universitas Indonesia, Indonesia; ⁴Faculty of Psychology Universitas Indonesia, Indonesia; ⁵Brigham Women and Child Hospital, Harvard Medical School, USA

Abstract

Introduction: Professional identity formation requires students’ ability to reflect, which can be supported by constructive feedback from teachers. Feedback practice faces many challenges, including those related to sociocultural aspects. Feedback can be optimised by implementing a model suitable to the attributes of students, teachers, and the clinical environment. This study aimed to develop a suitable feedback model and test its effectiveness in promoting reflection.

Methods: The model was designed based on the theoretical framework and identified feedback-related attributes in FMUI. Expert reviews and cognitive interviews were conducted before the model was tested. The model’s effectiveness was tested using a quasi-experimental design involving 74 students in the Paediatric Module. Students were asked to reflect on their mini-CEX experience after receiving feedback from teachers, and the depth of reflection was compared between students in the intervention and control group. The depth of reflection was measured using Kember scoring, and bivariate analysis was conducted using SPSS 20.0.

Results: The model consists of 5 steps including: Rapport building, Acknowledge students’ strengths, Identify aspect(s) that need improvement, Share teachers’ experiences, and Establish a plan to improve. Deep reflection was more frequently found in the intervention group. Mann-Whitney test showed that the difference between groups was statistically significant (Z=2.964, p=0.003), indicating that reflection in the intervention group was deeper than the control group.

Conclusion: The feedback model formulated based on feedback-related attributes in FMUI was named the RAISE model. Upon testing, students receiving the model could reflect more deeply compared to those receiving the standard model.

Keywords:           Feedback, Clinical, Reflection, Sociocultural

Practice Highlights

• Constructive feedback is needed for medical students to optimally form their professional identity, especially in clinical settings
• The RAISE model which was developed based on feedback-related attributes can be implemented in clinical education
• The RAISE model consists of five steps: Building rapport, Acknowledge students’ strengths, Identify aspect(s) to improve, Share teachers’ experience, and Establish a plan to improve
• The RAISE model promotes students’ ability to reflect on their experiences

I. INTRODUCTION

Medical students’ professional identity is mostly formed during their clinical education (Cruess et al., 2015). Professional identity formation is a long and complex process that occurs through the integration of values and norms of the profession. Socio-constructivist theory can explain how learning occurs in clinical settings through the involvement of students in a medical community of practice. As students immerse themselves in the clinical environment, they engage in numerous interactions with their surroundings. The interactions expose them to professional values, which are gradually internalised as students develop their new identity as medical doctors (Cruess et al., 2015).

Optimal professional identity formation requires students’ ability to reflect on their experiences. Reflection enables students to extract meaning from their hectic day-to-day activities, and create plans to continuously improve themselves (Mann et al., 2009). The ability to reflect is also essential in their future profession as doctors. Reflection was defined as critically analysing one’s own experience to reach new and deeper comprehension and insight. Experience become the object to reflect, and the process of recalling and analysing the experience was conducted with the intention of improving future performance (Mann et al., 2009). Reflection also becomes the media to process feelings and emotions, and create plans to improve performance, which leads to improved clinical decision-making ability (Rozental et al., 2021). The importance of reflective practice made it necessary to provide students with opportunity to become skilful in conducting reflection.

The analysis of reflection should include many perspectives to get comprehensive views and insights. One of the ways to support reflection is through the provision of constructive feedback (Husebo et al., 2015; Sandars, 2009). Feedback has been shown to be beneficial in helping students improve their performance (Choi et al., 2020; Johnson et al., 2020; Trehan et al., 2015). Information given through feedback assists students in identifying the improvement needed and the strategies they can use. Feedback can initiate reflections by pointing out the gap between the intended outcome and the current performance; and it can also strengthen reflection by confirming students’ judgment and promoting insights into their performance (Nicol & MacFarlane-Dick, 2006).

Many problems are still found in the practice of feedback in clinical settings, such as the message not being specific enough and the limited time available for interaction. In addition, the Indonesian culture adds certain unique challenges to feedback practice, as identified in our previous study (Felaza et al., 2023). These challenges can be associated with the cultural dimensions of high-power distance and collectivism in Indonesian society. Teachers, experienced and have expertise, are believed by students to know what is best. In a busy clinical environment, the hierarchical gap creates a preference of both teachers and students to engage in directive feedback. In receiving feedback, students tend to appear receptive even when not fully agreeing with the feedback. Being a collectivistic society, group approaches in seeking, providing, and processing feedback practices were commonly found. Students share their feedback experiences with their peers, resulting in collective perception of teacher that they had interacted with (Felaza et al., 2023). Similar challenges have been reported in other studies conducted in Indonesia and Thailand (Areemit et al., 2020; Suhoyo et al., 2018).

Considering the importance of feedback in facilitating students’ professional identity formation, and the way feedback practice is affected by sociocultural aspects, there is a need to develop a feedback model suitable to the attributes of students, teachers, and the clinical environment in Indonesia. This study aims to develop a feedback model relevant to our setting and assess its effectiveness in supporting students’ reflective ability. This study proposes two research aims:

1. To develop a feedback model suitable for the Indonesian clinical education setting.
2. To evaluate the effectiveness of the feedback model in supporting students’ reflective ability.

 

II. METHODS

This quantitative research was conducted using a design-based research (DBR) approach, comprising the following steps of identifying the problem, designing a solution, evaluating its effectiveness in a real-life setting, and reflecting on the outcomes (Scott et al., 2020). The first step of our DBR was the identification of feedback-related attributes in FMUI, in a study conducted in September 2023. The next steps, described in detail below, included development of a feedback model within our institution based on data from the previous study, followed by evaluation of its effectiveness in promoting reflection.

A. Feedback model development

The development of the model was guided by a theoretical framework from literature and findings of students’, teachers’, and environmental attributes in our setting. Socio-constructivism, as the theoretical framework, perceives learning as a process of making meanings from experiences through social interactions. Discourses between teacher and students exposes students to thought process and professional norms and values which later can be internalised within themselves. The attributes identified were environmental attributes (limited time, collectivistic, hierarchical, and the need for a safe environment), teachers’ attributes (busy, expert, and having the skill to provide feedback), and students’ attributes (dependent on feedback, less able to read between the lines, perceived to be less resilient, appear to be receptive of feedback, critical, and values teacher’s experience). These findings have been published on September 2023 (Felaza et al., 2023). Along with the socio-constructivist approach as the theoretical framework, these attributes would become the foundation in the development of the model.

Review by experts and cognitive interviews were conducted to ensure the model’s readiness for implementation. Review panel included experts on medical education and communication, program coordinator, and module coordinators. The model was sent via e-mail, and experts were asked to provide feedback on each step and the overall applicability of the model. Cognitive interviews involved clinical teachers from surgery-related and non-surgery-related specialties as future users of the model. After receiving explanation of the model, the teachers practiced the model on their clinical teaching activities. Interviews were held with each teacher to gain input on the experience. Inputs from expert review and cognitive interviews were used to refine the model. The final version of the model was taught to clinical teachers in Paediatric Module as a preparation to test the model. The model was tested during Mini-Clinical Evaluation Exercise (Mini-CEX) by comparing its effectiveness with the Feedback Sandwich method as the standard model being used in FMUI. Mini-CEX is a workplace-based assessment which evaluate students’ performance of clinical skills in real setting. The assessment was originally intended for formative purposes as the clinical teacher observes the performance and provide feedback for students (Hejri et al., 2019). The sandwich model was found to be the common method used for feedback provision in FMUI, and it starts with informing students of the positive aspects of their performance, followed by the negative ones, and closed with another positive aspect (Jug et al., 2019).

B. Testing the effectiveness of the model

The design used was quasi-experimental, involving 5^th-year FMUI medical students during their clinical rotation in Paediatric Module. There were 37 students in each group and the data collection took place in May 2022–March 2023. Students in the intervention group received feedback using the new model, while their peers in the control group received feedback using the Sandwich model (Jug et al., 2019). Upon completing data collection in the control group, authors conducted training for teachers on how to utilise the new model, followed by data collection in the intervention group.

Data was collected by asking students to record their voices as they reflect by thinking out loud. FMUI students have been introduced to reflection since the beginning of their academic years, and have experienced completing reflective writing assignments in several modules. For this study, they were instructed to reflect on their experience during mini-CEX. The depth of the reflection was then scored using Kember’s scoring. The scoring was developed by Kember based on Mezirow’s four levels of reflection, and it has been adapted to Bahasa Indonesia (Soemantri et al., 2022). Mezirow divided reflection to 4 levels based on its depth, starting from habitual action which showed no reflection, thoughtful action/understanding, reflection, and critical reflection (Kember et al., 2000). The scoring process was conducted separately by the first author (EF) and two co-authors (AF & RM) and later compared in team discussions to reach an agreed score. Bivariate analysis to test to compare the depth of the reflection between the intervention and the control group was conducted using SPSS 20.0.

III. RESULTS

The criteria for the feedback model were developed based on attributes found from the previous stage (as shown in Table 1 and Table 2).

Table 1. Attributes of Factors Affecting Feedback and The Criteria of the Feedback Model

Table 2. Steps Proposed and How it Accommodates the Criteria of the Feedback Model

The model was named RAISE as an abbreviation of the first letter in each step. The RAISE model was reviewed by an expert panel, consisting of 3 experts in medical education, 2 experts in communication, and 4 clinical program coordinators. Overall, the RAISE model was perceived as suitable to be implemented in the clinical settings by the expert panel (Figure 1). The crucial aspects that needed to be ensured were teachers’ comprehension of the reasoning behind each step, and the adjustability of the model to be applied in limited time and variability of contexts. Rapport building was considered an essential initial step to start feedback interaction. For the second step, it was advised that teachers ask the student to state positive aspects of performance first before confirmation based on their judgment. The focus of the third step was the areas for improvement, a term that experts emphasised. The step which was initially named ‘invite student to reflect’ was changed into ‘identify aspect that needs improvement’. This revision was made to prevent the misconception that reflection was exclusively covered in the 3^rd step. In the fourth step, teachers share their experiences from which students can learn and develop ideas to improve their performance. The information shared should be relevant and can be delivered in a manner that is non-threatening and relevant for the students. The final step in the RAISE model focuses on the action plan for improvement. The agreed plan should be feasible and ready to be implemented.

Figure 1. RAISE Model After Expert Review

The model was then utilised by four clinical teachers in various teaching sessions, and cognitive interviews were conducted to gain insight into the experiences. They reported that the model was relatively easy to implement and able to accommodate the limited time available. The model helped develop rapport between teacher and students and enable the two of them to interact more effectively. Continuation between each step was seamless and the conversation did not have to be interrupted due to the movement from one step to another.

The final version was then taught to 22 clinical teachers in August 2022. The training was organised by the first author and two of the co-authors, who had expertise in medical education and experience in conducting faculty development. Various activities were held in the training (video appraisal, discussions, role play, and reflection) to ensure teachers’ ability to perform feedback provision using the RAISE model.

The final version was compared with the Sandwich model, the standard model being used in FMUI, to evaluate its effectiveness. The comparison between the steps of the two models can be seen in Table 3. As shown in the comparison, the strength of the RAISE model lies in its effort to establish rapport from the start of the feedback conversation, its way of facilitating reflection, and the opportunity it provides for teachers to share their experiences.

Table 3. Comparison Between the RAISE Model and the Standard Model

The effectiveness of the model was then tested during mini-CEX in Paediatric Module by comparing the depth of reflection in the experimental and control group. There were 37 students in each group with majority of students in experimental and control groups (30 and 28 students respectively) were female and belonged in the year 2018 batch.

Students reflected on the feedback they received, and the depth of the reflection was rated using the Kember score with the score ranging from 1-4 (1: no reflection; to 4: deep reflection). The distribution of the depth of students’ reflection is shown in Table 4.

Table 4. Distribution of the Depth of Students’ Reflection

The distribution showed most of the students’ reflections in the intervention group were scored 3 based on the Kember score (26 students or 70.2%); while the score in the control group was almost evenly divided between 2 (16 students or 43.2%) and 3 (17 students or 45.9%). Deep reflection was more frequently found in the intervention group (9 students or 24.4%) compared to the control group (4 students or 11.9%).

Mann-Whitney test was used to compare the depth of students’ reflection between the intervention and the control group. The test showed that the mean rank in the intervention group was 30.61 and in the control group was 44.39. The difference between the two groups was statistically significant (Z = 2.964, p = 0.003), indicating that students’ reflections in the intervention group were deeper than their peers in the control group.

IV. DISCUSSION

This study aimed to develop a culturally relevant feedback model that nurtured reflective ability of medical students and to evaluate its effectiveness. The results indicated that students receiving feedback using the RAISE model could reflect more deeply compared to their peers receiving feedback using the standard model. The RAISE model was designed to enable feedback practice in an interactive manner, while feedback using Sandwich model was more directive. The RAISE model starts by building rapport between teacher and student in order to create a safe environment which helps student to optimally conduct reflection. The 2^nd and 3^rd steps of the model guide students’ reflection systematically, covering both the positive and negative aspects of the performance. On the 4^th step, teachers’ sharing of their experiences helps to enrich students’ reflection, enabling them to learn from the experience, as well as strengthening and further maintaining the rapport between them. The final step concludes the interaction with a plan that students can implement to improve. Each of these steps portrays how reflection is guided and supported by the teacher through the feedback interaction.

The RAISE model was developed based on the socio-constructivist theory that describes learning as a way of constructing meaning from experiences. In clinical settings, students are exposed to a variety of experiences to build comprehension, acquire skill proficiency, and integrate professional norms and values. Effective feedback interactions between teachers and students facilitate students’ reflective process and support them in identifying effective ways to improve their performance (Wenger, 2020). Through these interactions, students are exposed to the teacher’s thought process, as well as their professional norms and values. The process that starts as inter-mental between teacher and student gradually becomes internalised into intra-mental within the student (Amineh & Asl, 2015; Tappan, 2010). This process enables teachers to guide students to reach their zone of proximal development (ZPD). Students are supported at each stage of their performance through the concept of scaffolding, and the level of support given would be decreased gradually as students gain the ability for independent clinical practice (Fani & Ghaemi, 2011).

Our previous study highlighted the need for rapport development and the act of sharing teachers’ own experiences in developing the feedback model (Felaza et al., 2023). Rapport was considered essential to create a safe environment for students. Interactions that empowered students in an authentic and safe environment, and teachers as role models in practicing reflection, were needed for creating reflective habits among students (Gathu, 2022; Mann et al., 2009). A sense of safety in feedback interaction was built upon a good relationship between teacher and students, mutual goals, and students’ perception of the teacher’s credibility (Sargeant et al., 2017). In the RAISE model, the safe environment has been developed since the beginning by building rapport between teacher and students. When students reflect, they would need to be open in analysing their experience. Destructive feedback would create reluctance for students to share their reflections (Chan & Lee, 2021; Khoiriyati & Sari, 2021). The model started by asking students’ perceptions of their performance, and their previous experience with similar situations; encouraging them to be open and reflective. The questions were relatively easy to answer and able to lighten the situation. Safety was continuously built through constructive dialogue between teachers and students which created an environment that was safe for reflection (MacNeil et al., 2020; Telio et al., 2015). Mutual goals also can implicitly be seen in students’ perception of the teacher’s intention. When the students believe that feedback was given to help them improve, they would be more willing to openly reflect (Blakey et al., 2019; Eva et al., 2012).

The reflection process is initiated when the condition cannot be handled simply by the automatic action the individual usually conducts or habitual act (Mann et al., 2009). The initiation of reflection might also be triggered by feedback. The feedback information made the individuals realise the quality of their performance, highlighting the strength and weaknesses that requires improvement (Husebo et al., 2015; Mann et al., 2009). When students receive feedback that is aligned with their perception regarding the quality of their performance, reflection usually occurs faster. However, for feedback information that is not aligned, students need to first manage the negative emotion it might evoke, analyse the message, and contemplate what action plan should be taken (Sargeant et al., 2009).

Teachers, perceived as knowledgeable figures, have numerous valuable experiences they can share. By incorporating the act of sharing experience, every feedback interaction would provide lessons from which students can extract meaningful understanding. This particular approach has not been seen explicitly in other feedback models. Other than enriching students’ comprehension, the sharing also provides a sense of relatedness with the teacher, and a role model on how to become reflective (Burgess et al., 2020). A role model in practicing reflection was an important aspect of strengthening students’ motivation. Teachers who practice self-reflection confirm to students that such practice can be done in their profession as doctors. Teachers’ reflective behaviour also showed the students that admitting weaknesses was not an embarrassing act, and on the contrary, it was an important skill to master to improve (Gishen & Chakrabarti, 2022). In the RAISE model, the opportunity for teachers to demonstrate reflection was mostly done in the 4^th step. In this step, teachers can share their experiences that was relevant to the situation the students faced. Upon sharing the experience, the teacher can explain the lesson learned from that experience. Students receiving feedback using the RAISE model claimed that this step exposed them to teachers’ openness and minimised the power distance between them. Students also felt more motivated by listening to the experiences. Sense of relatedness was one of the factors that increased intrinsic motivation. The teachers’ willingness to share their experiences creates a connection between them and students, and strengthens students’ internal motivation (Kusurkar et al., 2011).

Other than being influenced on a systemic level by cultural aspects, such as power distance and collectivism; reflection is also affected in individual level by students’ and teachers’ reflective ability, teachers’ ability in facilitating reflection, and the relationship between teacher and students (Chan & Lee, 2021). The RAISE model supports teachers in facilitating students’ reflection through the use of reflective questions and the opportunity to show students how reflection was practiced by themselves as they shared their experiences. The rapport built during the interactions also strengthens the teacher-student relationship which further enables students to be more open and trusting in conducting their reflection.

Students’ reflective ability was crucial in their education process and later as they became member of the medical profession. Experiences in the clinical learning environment, which depicts the authentic professional world that students would enter, expose students to norms and values. Through reflection, students create meanings from these experiences which facilitate the formation of their professional identity as medical doctors (Winkel et al., 2017).

This study showed that implementation of the RAISE model, a feedback model developed based on the feedback-related attributes in Indonesian settings, enabled students to reflect more deeply on their performance. Findings from the study suggested that the RAISE model was suitable for supporting students’ reflection in a setting with high power distance and collectivistic background. However, it should be noted as a limitation of this study that reflection was an internal process and the effect might not be visible immediately. The changes in comprehension, perspectives, and even students’ attitude occurs gradually along the reflection process that was taking place. Therefore, further research is needed to observe the effect of reflection which might require longitudinal observation of students’ performance. The study involved a single institution in Jakarta, and therefore research and certain adjustments might be needed before the model can be implemented in different settings.

V. CONCLUSION

The feedback model formulated based on the feedback-related attributes in our clinical setting was named the RAISE model. It consists of 5 steps, which are Rapport building, Acknowledge student’s strengths, Identify aspects that need improvement, Share teacher’s experience and tips, and Establish a plan to improve. Our data indicate that students receiving feedback using the RAISE model were able to reflect more deeply compared to their peers receiving the standard model. We conclude that the model’s ability to create a safe environment, build rapport between teachers and learners, and teachers’ role-modelling of reflective practice can encourage students to be more self-reflective of their clinical performance.

Notes on Contributors

Estivana Felaza, Ardi Findyartini, Rita Mustika, Jenny Bashiruddin, Joedo Prihartono, Lucia RM Royanto, and Subha Ramani contributed to all processes of this research, and have read and approved this manuscript.

Dr. dr. Estivana Felaza, MPdKed is a Lecturer in Faculty of Medicine, Universitas Indonesia. She reviewed the literatures, designed the study, performed data collection and analysis, and wrote the manuscript.

Prof. dr. Ardi Findyartini, PhD is a Professor of Medical Education in the Faculty of Medicine, Universitas Indonesia. She developed the methodological framework for this research, involved in scoring the students’ reflections (data), provided feedback on the data collection, data analysis, and the manuscript writing process.

Dr. dr. Rita Mustika, M.Epid is an expert in Medical Education and a Lecturer in the Faculty of Medicine, Universitas Indonesia. She developed the methodological framework for this research, was involved in scoring the students’ reflections (data), provided feedback on the data collection, data analysis, and manuscript writing process.

Prof. Dr. dr. Jenny Bashiruddin, SpTHT-KL(K) is a Professor in Otolaryngology and an expert in Medical Education. She provided feedback on the theoretical and methodological framework of this research, data collection, data analysis, and the manuscript writing process.

Dr. dr. Joedo Prihartono, MPH is an expert in Statistics and a Senior Lecturer in Community Medicine, Faculty of Medicine, Universitas Indonesia. He provided feedback on the methodological framework of this research, data collection, data analysis, and the manuscript writing process.

Dr. Lucia RM Royanto, M.Si, Sp.Ed is an expert in Educational Psychology and a Senior Lecturer in the Faculty of Psychology, Universitas Indonesia. She provided feedback on the theoretical framework of this research, data collection, data analysis, and the manuscript writing process.

Dr. Subha Ramani, MBBS, MPH, PhD, FAMEE is an expert in Medical Education and an Associate Professor of Medicine in Harvard Medical School. She provided feedback on the theoretical and methodological framework of this research, data collection, data analysis, and the manuscript writing process.

Ethical Approval

The study was approved by the Research Ethics Review Committee of the Faculty of Medicine, Universitas Indonesia (approval number: KET-1114/UN2.F1/ETIK/ PPM.00.02/2021). The subjects of this study were informed verbally and in writing of the purpose, the methods, and the confidentiality of the data being collected. Each subjects’ involvement in this research were voluntary, and consent were signed upon receiving the necessary information.

Data Availability

Datasets generated and/or analysed in this study are available upon request via electronic mail to the corresponding author.

Acknowledgement

The authors would like to acknowledge the contributions of the students and clinical teachers who willingly involved and cooperated in this study. 

Funding

No funding was obtained for this study.

Declaration of Interest

All authors have no declaration of interest.

References

Amineh, R. J., & Asl, H. D. (2015). Review of constructivism and social constructivism. Journal of Social Science, Literature & Language, 1(1), 9–16.

Areemit, R. S., Cooper, C. N., Wirasorn, K., Paopongsawan, P., Panthongviriyakul, C., & Ramani, S. (2020). Hierarchy, “Kreng Jai”, and feedback: A grounded theory study exploring perspectives of clinical faculty and medical students in Thailand. Teaching and Learning in Medicine, 33(3), 235–244. https://doi.org/10.1080/10401334.2020.1813584

Blakey, A. G., Smith-Han, K., Anderson, L., Collins, E., Berryman, E., & Wilkinson, T. (2019). “They cared about us students”: Learning from exemplar clinical teaching environments. BMC Medical Education, 19, 119–128. https://doi.org/10.1186/ s12909-019-1551-9

Burgess, A., van Diggele, C., Roberts, C., & Mellis, C. (2020). Feedback in the clinical settings. BMC Medical Education, 20(Suppl 2), 458–463. https://doi.org/10.1186/s12909-020-02280-5

Chan, C. K. Y., & Lee, K. K. W. (2021). Reflection literacy: A multilevel perspective on the challenges of using reflections in higher education through a comprehensive literature review. Educational Research Review, 32(2), Article 100376. https://doi.org/10.1016/j.edurev.2020.100376

Choi, S., Oh, S., Lee, D. H., & Yoon, H. S. (2020). Effects of reflection and immediate feedback to improve clinical reasoning of medical students in the assessment of dermatologic conditions: A randomized controlled trial. BMC Medical Education, 20, 146–154. https://doi.org/10.1186/s12909-020-02063-y

Cruess, R. L., Cruess, S. R., Boudreau, J. D., Snell, L., & Steinert, Y. (2015). A schematic representation of the professional identity formation and socialization of medical students and residents: A guide for medical educators. Academic Medicine, 90(6), 718–725. https://doi.org/10.1097/acm.0000000000000700

Eva, K. W., Armson, H., Holmboe, E., Lockyer, J., Loney, E., Mann, K., & Sargeant, J. (2012). Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning process. Advance in Health Science Education, 17, 15–26. https://doi.org/10.1007/s10459-011-9290-7

Fani, T., & Ghaemi, F. (2011). Implications of Vygotsky’s zone of proximal development in teacher education. Social Behavioral Science, 29, 1549–1554. https://doi.org/10.1016/j.sbspro.2011.11. 396

Felaza, E., Findyartini, A., Mustika, R., Bashiruddin, J., Royanto, L. R. M., Prihartono, J., & Ramani, S. (2023). Deeper look into feedback practice in an Indonesian context: Exploration of factors in undergraduate clinical setting. Korean Journal of Medical Education, 35(3), 263–273. https://doi.org/10.3946/kjme.2023.264

Gathu, C. (2022). Facilitators and barriers of reflective learning in postgraduate medical education: A narrative review. Journal of Medical Education and Curriculum Development, 9, 1–8. https://doi.org/10.1177/23821205221096106

Gishen, F., & Chakrabarti, R. (2022). Medical students’ perceptions of reflective practice in the undergraduate curriculum. MedEd Publish, 12, 53–67. https://doi.org/10.12688/mep.19211.3

Hejri, S. M., Jalili, M., Masoomi, R., Shirazi, M., Nedjat, S., & Norcini, J. (2019). The utility of mini clinical examination exercise in undergraduate and postgraduate medical education: A BEME review: BEME guide no. 59. Medical Teacher, 42(2), 1–18. https://doi.org/10.1080/0142159x.2019.1652732

Husebo, S. E., O’Reagen, S., & Nestel, D. (2015). Reflective practice and its role in simulation. Clinical Simulation in Nursing, 11, 368–375. http://doi.org/10.1016/j.ecns.2015.04.005

Johnson, C. E., Weerasuria, M. P., & Keating, J. L. (2020). Effect of face-to-face verbal feedback compared with no alternative feedback on the objective workplace task performance of health professionals: A meta-analysis and systematic review. BMJ Open, 10(3), Article e030672. https://doi.org/10.1136/bmjopen-2019-030672

Jug, R., Jiang, X., & Bean, S. M. (2019). Giving and receiving effective feedback. Archives of Pathology and Laboratory Medicine, 143, 244–250. https://doi.org/10.5858/arpa.2018-0058-RA

Kember, D., Leung, D. Y. P., Jones, A., Loke, A. Y., MacKay, J., Sinclair, K., Tse, H., Webb, C., Wong, F. K. Y., Wong, M., & Yeung, E. (2000). Development of a questionnaire to measure the level of reflective thinking. Assessment and Evaluation in Higher Education, 25(4), 381–395. http://doi.org/10.1080/713611442

Khoiriyati, A., & Sari, N. K. (2021). Reflective practice on nursing students: A qualitative study. Indonesian Nursing Journal of Education and Clinic, 6(2), 201–209. http://doi.org/10.24990/in jec.v6i2.391

Kusurkar, R., Croiset, G., & Ten Cate, O. (2011). Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory. Medical Teacher, 33, 978–982. https://doi.org/10.3109/01 42159x.2011.599896

MacNeil, K., Cuncic, C., Voyer, S., Butler, D., & Hatala, R. (2020). Necessary but not sufficient: Identifying conditions for effective feedback during internal medicine residents’ clinical education. Advances in Health Science Education, 25, 641–654. https://doi.org/10.1007/s10459-019-09948-8

Mann, K., Gordon, J., & MacLeod, A. (2009). Reflection and reflective practice in health professions education: A systematic review. Advances in Health Science Education, 14, 595–621. https://doi.org/10.1007/s10459-007-9090-2

Nicol, D., & MacFarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090

Rozental, L., Meitar, D., & Karnielli-Miller, O. (2021). Medical students’ experiences and needs from written reflective journal feedback. Medical Education, 55, 505–517. https://doi.org/10.1111/medu.14406

Sandars, J. (2009). The use of reflection in medical education: AMEE Guide no 44. Medical Teacher, 31, 685–695. https://doi.org/10.1080/01421590903050374

Sargeant, J. M., Mann, K. V., van der Vleuten, C. P., & Metsemakers, J. F. (2009). Reflection: A link between receiving and using assessment feedback. Advances in Health Science Education, 14, 399–410. https://doi.org/10.1007/s10459-008-9124-4

Scott, E. E., Wenderoth, M. P., & Doherty, J. H. (2020). Design-based research: A methodology to extend and enrich biology education research. CBE Life Science Education, 19(3), 1–12. https://doi.org/10.1187/cbe.19-11-0245

Soemantri, D., Mustika, R., & Greviana, N. (2022). Inter-rater reliability of reflective writing assessment in an undergraduate professionalism course in medical education. Education in Medicine Journal, 14(1), 87–97. http://doi.org/10.21315/eimj2022.14.1.8

Suhoyo, Y., Schönrock-Adema, J., Emilia, O., Kuks, J. B. M., & Cohen-Schotanus, J. (2018). Clinical workplace learning: Perceived learning value of individual and group feedback in a collectivistic culture. BMC Medical Education, 18, 79–84. https://doi.org/10.1186/s12909-018-1188-0

Tappan, M. (2010). Sociocultural psychology and caring pedagogy: Exploring Vygotsky’s hidden curriculum. Educational Psychology, 33(1), 23–33. http://doi.org/10.1207/s15326985ep3301_2

Telio, S., Ajjawi, R., & Regehr, G. (2015). The “Educational Alliance” as a framework for reconceptualizing feedback in medical education. Academic Medicine, 90(5), 609–614. https://doi.org/10.1097/acm.0000000000000560

Trehan, A., Barton-Vaness, A., Carty, M. J., McCulloch, P., & Maruthappu, M. (2015). The impact of feedback of intraoperative technical performance in surgery: A systematic review. BMJ Open, 5, Article e006759. http://doi.org/10.1136/bmjopen-2014-006759

Wenger, E. (2020). Communities of practice and social learning systems: The career of a concept. Organization, 7(2), 225–246. http://doi.org/10.1177/135050840072002

Winkel, A. F., Yingling, F., Jones, A. A., & Nicholson, J. (2017). Reflection as a learning tool in graduate medical education: A systematic review. Journal of Graduate Medical Education, 5, 430–439. https://doi.org/10.4300/jgme-d-16-00500.1

*Estivana Felaza
Faculty of Medicine Universitas Indonesia,
Jalan Salemba Raya No 6,
Jakarta 10430, Indonesia
Email: estivanafelaza@gmail.com;
estivana.felaza01@ui.ac.id

Submitted: 13 June 2024
Accepted: 14 October 2024
Published online: 7 January, TAPS 2025, 10(1), 36-47
https://doi.org/10.29060/TAPS.2025-10-1/OA3438

Sivapalan Sanchayan¹, Sisira Dharmaratne², Shalika Pathirana³, Pavithra Godamunne³ & Madawa Chandratilake³

¹Department of Medical Education, Faculty of Medicine, University of Jaffna, Sri Lanka; ²Translational Health Research Institute, Western Sydney University, Australia; Department of Medical Education, Faculty of Medicine, University of Kelaniya, Sri Lanka

Abstract

Introduction: Selection for basic medical training is highly contextual. The use of cognitive aptitude tests, which commonly supplement measures of prior academic achievement in the selection process of medical schools internationally, is rarely reported from resource-constrained settings in South Asia. We report on the psychometric properties of an aptitude test designed based on the UCAT format, administered to first-year medical undergraduates to determine its utility.

Methods: The aptitude test was administered online to first-year medical students shortly after their admission to two medical faculties in Sri Lanka (n=328). The reliability of the test was determined using Cronbach’s alpha. Overall and subtest scores were computed, and the scores of different demographic groups were compared using the t-test. Factor analysis of the subtests was performed.

Results: The internal consistency of the test was 0.63. The difficulty and discrimination indices were within the acceptable range. The mean score of the aptitude test (AT) was 70.9/100 (SD 8.88). The mean score for females was higher than for males (p=0.04). No statistically significant differences in AT scores were observed between different ethnicities or religions. The entry academic scores demonstrated a weak correlation with the overall AT score (Pearson’s correlation coefficient r=0.27), verbal reasoning (r=0.24), the human body (r=0.19), quantitative reasoning (r=0.18), and situational judgement (r=0.128). Factor analysis indicated items that need revision.

Conclusion: The results indicate the potential value of aptitude tests in Sri Lanka and in similar jurisdictions where selection criteria have yet to expand beyond prior academic performance. A way forward in introducing such tests has been outlined.

Keywords:           Medical Student Selection, Medical Student Admission, Selection Criteria

Practice Highlights

• Aptitude tests assess cognitive and non-cognitive attributes desirable for healthcare professionals.
• The use of aptitude tests is rarely reported from resource constrained settings in South Asia.
• Sri Lanka uses only the national high school examination performance for medical school admission.
• This study indicates the potential value of aptitude tests in Sri Lanka and in similar jurisdictions.

I. INTRODUCTION

Selection for basic medical training is highly contextual and influenced by educational, geographical, socioeconomic, and political factors (Dharmaratne & Ponnamperuma, 2020; Patterson et al., 2018). High academic achievement has long been considered an appropriate criterion for ranking candidates for medical school (McManus et al., 2013; Patterson et al., 2017). However, this approach has been contested due to the low predictive validity of pre-university performance for medical school success (McManus et al., 2013; Prideaux et al., 2011).

Aptitude tests have emerged to assess cognitive and non-cognitive traits desirable for medical and healthcare professions (Gliatto et al., 2016; Greatrix & Dowell, 2020; Powis et al., 2021). Despite evidence on their potential to improve selection processes (Bala et al., 2022; Greatrix et al., 2021), most resource-constrained jurisdictions, including Sri Lanka, rely solely on prior academic achievement (Soemantri et al., 2020; University Grants Commission, 2023). This study represents an initial step in Sri Lanka toward introducing aptitude tests and reports on the psychometric properties of a test developed to measure cognitive attributes.

The Sri Lankan medical school admission system uses national high school examination results (G.C.E. AL) as the only criterion for admission (Dharmaratne & Ponnamperuma, 2020; University Grants Commission, 2023). The results of three AL subjects (Chemistry, Physics, and Biology) are considered, and an average Z score is used to rank individuals (University Grants Commission, 2023). A minimum of two credit passes and one simple pass is required for eligibility. A few candidates with foreign examination results equivalent to the G.C.E. AL Examination and meeting equitable minimum requirements are also admitted (University Grants Commission, 2023).

G.C.E. AL examination results have shown a mild correlation with subsequent undergraduate performance, with a maximum predictive correlation coefficient of 0.37 (de Silva et al., 2004, 2006; Hewage et al., 2011; Mettananda et al., 2006). These low figures may underestimate the true relationship, as range restriction among high-achieving candidates can affect the statistical accuracy of predictive validity measures (Zimmermann et al., 2017). Studies in the UK, addressing range restriction, found that prior academic performance is the best predictor of subsequent performance, accounting for 65% of the variance in undergraduate performance (McManus et al., 2013). However, this leaves 35% of the variance unexplained.

Aptitude tests are used worldwide to support medical student selection. The United Kingdom Clinical Aptitude Test (UKCAT), now named UCAT, has been used for nearly two decades (Greatrix & Dowell, 2020). The UCAT consists of 200 items in five subtests delivered over two hours (Paton et al., 2022). Results of the UKCAT and UCAT have been used to discriminate between high-achieving students during selection and predict undergraduate and postgraduate performance (Bala et al., 2022; MacKenzie et al., 2016; McManus et al., 2013; Paton et al., 2022).

However, there are additional qualities beyond academic readiness (prior academic performance and cognitive traits) that are necessary qualities of medical professionals that include desirable behaviours and attitudes that could indicate successful development as healthcare practitioners in the future (Patterson et al., 2018). The Ottawa consensus statement recommends that best practice selection should involve the use of selection criteria based on multi-source multi-method job analysis studies (Patterson et al., 2018). This necessitates the use of other selection processes such as structured interviews or multiple mini interviews (MMI).

This study aims to evaluate the psychometric properties of an aptitude test based on the UCAT format, administered to first-year medical undergraduates at two medical faculties in Sri Lanka. Specifically, the objectives are to evaluate the psychometric properties of the test, determine the relationship between Z scores at the AL examination and aptitude test scores, and compare the performance of different demographic groups in the aptitude test.

II. METHODS

A cross-sectional observational study was conducted in the faculties of medicine at the universities of Kelaniya and Jaffna, Sri Lanka. Including these two faculties allowed for the recruitment of an adequate number of students from the three main ethnic groups (Sinhala, Tamil, and Sri Lankan Moors) and a wider range of Z scores. All first-year students in 2023 from these faculties were invited to participate. Students admitted through selection criteria other than Sri Lankan advanced level performance were excluded. We obtained a sample size of 328 students who consented to participated in the study (response rate of 80.8%).

The aptitude test comprised 50 items across six subtests: abstract reasoning (8 items), the human body (8 items), quantitative reasoning (8 items), decision-making (8 items), verbal reasoning (8 items), and situational judgment (10 items). Except for the “the human body” subtest, all others were modelled after the UKCAT examination. The “human body” subtest was designed to assess advanced-level knowledge of human biology from the Sri Lankan school curriculum. Test items were selected by a panel of experts in psychology and medical education from the Faculty of Medicine, University of Kelaniya with the help of disability studies experts who use a similar test for selection to their course. These experts, being native Sri Lankans, ensured the items were culturally relevant and conceptually and semantically equivalent (Hambleton & Zenisky, 2010). Similar methods were used to finalise the Sinhala and Tamil versions of the test, translated by competent translators. Appendix 1 shows sample test items from each subtest. The test was piloted on 15 students and identified issues were corrected.

The test was administered via the online learning management system (LMS) of the respective faculties under examination conditions, with invigilators ensuring adherence to university examination protocols. Students had one hour to complete the test, available in English, Sinhala, and Tamil. Participants could choose the language they were most comfortable with to simulate AL examination conditions and to minimise interpretation errors.

Descriptive statistics and internal consistency (Cronbach’s alpha) were calculated for the test and each subtest and item analysis (Paniagua & Swygert, 2016) was performed. Correlations between the aptitude test score, subtest scores, and Z scores were calculated using Pearson’s correlation coefficient, and relationships to other demographic factors were analysed using Student’s t-test. Factor analysis was performed using R statistical package.

Written informed consent was obtained from the students before the test administration. Ethics approval was granted by the ethics review committee of the Faculty of Medicine, University of Kelaniya, and permission for the study was obtained from the deans of the respective medical faculties.

III. RESULTS

328 students participated in the study, yielding an overall response rate of 80.8%. The descriptive statistics of these students are presented in Table 1. The male-to-female ratio of the sample (31.7% to 68.3%) closely mirrors the demographics of the national cohort of medical undergraduates from the previous academic year (34% to 66%), as reported by the University Grants Commission, Sri Lanka (University Grants Commission, 2022).

Table 1. Descriptive statistics of the demographic variables of the sample

The overall mean aptitude test (AT) score was 70.9 (SD 8.88), with a median of 72. Female students had a higher mean AT score than male students (p=0.044, Table 2), while there was no statistically significant difference in the Z scores between genders (p=0.122). No significant differences in mean AT scores were found between ethnicities and religions. The Faculty of Medicine, University of Kelaniya, had a higher mean AT score (73.7, SD 8.7) than the Faculty of Medicine, Jaffna (68.8, SD 8.4, p=0.000). Similarly, the mean Z score at Faculty of Medicine, University of Kelaniya (2.10, SD 0.07) was higher than at Jaffna (1.89, SD 0.13, p=0.000).

Table 2. Comparison of means of AT scores and Z scores between males and females

The internal consistency of the test was 0.628 and the subscales ranged between 0.2 to 0.47 (Table 3).

Table 3. Data regarding the subtests of the aptitude test

Difficulty and discrimination indices for each test item were calculated (Table 4). The difficulty index represents the proportion of students who answered each item correctly (Paniagua & Swygert, 2016). The mean difficulty index was 70.9% (SD 23.5), ranging from 19.2% to 99.4%. The discrimination index was calculated as the difference between the proportion of students who answered an item correctly in the top group (those who obtained the top 27% in the overall score) and bottom group (bottom 27% scorers) and measures an item’s ability to differentiate between students who performed best and those who performed worst on the test (Paniagua & Swygert, 2016). The mean discrimination index was 19% (SD 12) and ranged from 0 to 50%. There was only one item with the lowest discrimination index of 0. It had a difficulty index of 99.4% indicating that most students had scored it correctly. Items that adversely affected the reliability of the subtests were identified (Appendix 2).

Table 4. Distribution of the difficulty indices of the aptitude test questions

The correlation analysis revealed several key findings regarding the relationship between Z and AT scores. The correlation coefficient between Z score and AT score was 0.268 (p=0.000), indicating a weak correlation. The coefficient of determination (r²) was calculated to be 0.072, suggesting that only 7.2% of the variability in AT scores could be explained by Z scores. This signifies a limited association between Z scores and overall AT performance. Further analysis was conducted on the correlation between Z scores and subtest scores. There was a weak but statistically significant correlation between Z scores and the subtests of verbal reasoning (r=0.24, p=0.000), human body (r=0.19, p=0.001), quantitative reasoning (r=0.18, p=0.001), and situational judgment (r=0.128, p=0.021). However, the correlations between Z scores and the subtests of abstract reasoning and decision-making were not statistically significant.

Factor analysis was performed using the most prominent two factors of each of the six subtests separately as they each measure well defined and different cognitive domains. The factor loading of abstract reasoning subtest is shown in figure 1. “Question 3” shows a strong positive loading on Factor 1, suggesting that this item heavily measures the trait most represented by this factor whilst “Question 5” shows a slight negative loading on Factor 1, indicating it measures a trait that is perhaps inversely related to what is captured by this factor. “Question 1” and “Question 2” are both located positively on Factor 2 but with minimal loading on Factor 1. This suggests that they are measuring traits more aligned with Factor 2. The items “Question 1”, “Question 2″ and “Question 3”, appear to be critical for measuring distinct aspects of abstract reasoning as shown by their stronger loadings. “Question 5” shows a strong negative loading on Factor 2, suggesting that it measures a trait that contrasts with what Factor 2 represents. “Question 4”, ” Question 6″, “Question 7”, and “Question 8” are clustered around the origin. This indicates that these items have weaker loadings on both factors, meaning they may not strongly measure the traits represented by either factor, or they could be measuring a balanced mix of both traits. While these questions contribute to the test, they might need to be reviewed to enhance their discriminative power or clarity in measuring specific abstract reasoning skills.

Figure 1. Two factor loading of the abstract reasoning subtest

The factor loadings of the other five subtests are shown in Appendix 3. Regarding the human body subtest, Factor 1 might represent knowledge or understanding of structural aspects of the human body, given the strong loading by “Question 11” and reasonable loading by other items. This factor could reflect knowledge regarding human biology whilst Factor 2 seems to capture a different dimension, suggested by the negative loading of “Question 15”. In quantitative reasoning, most questions are clustered near the centre with a slight spread along Factor 1, indicating moderate influence by this factor across the items. “Question 19” and “Question 22” are slightly farther along Factor 1 suggesting a stronger influence by the trait measured by this factor. Regarding the subtests of decision making, verbal reasoning and situational judgment, Factor 1 and Factor 2 seemed to be capturing different dimension of these subtests thereby suggesting the need for revision of their items.

IV. DISCUSSION

This study aimed to design an aptitude test and investigate its psychometric properties among first-year medical students from two selected faculties of Medicine in Sri Lanka. The results indicate that overall internal reliability of the aptitude test was marginal (Cohen et al., 2018). However, the reliability of the UKCAT has been 0.86 or above overall and between 0.58 – 0.86 for the subtests (MacKenzie et al., 2016). These findings demonstrate the value of evaluating and reviewing aptitude tests locally. Further, reviewing test items that adversely affected the internal consistency of the subtests for clarity and improved alignment with the construct, could improve the overall validity. It is likely that the low number of test items has further contributed to the low-reliability score (Taber, 2018). The UKCAT consists of 200 test items, as opposed to the 50 items in the aptitude test employed by the current study. The item analysis indicated that most of the questions were easy, with 60% having a difficulty index of more than 70%. The discriminating capacity of the items was low (mean discrimination index was 19%). This could be attributed to the aptitude test being administered to students who have performed well in the advanced level examination and already have been selected to medical school.

There was a weak correlation between the aptitude test score and the Z score (0.27), indicating that the test assessed a significantly different aspect of cognitive skills compared to traditional AL examinations. This observation encourages exploring how the aptitude test may be incorporated into the selection process. Different selection authorities use aptitude scores differently (Greatrix & Dowell, 2020). Overall, among the universities which employ UKCAT, the weightage given for school leaver level academic performance in the selection process has increased, while the use of aptitude test scores as an absolute or relative cutoff to be achieved also has increased over the period of 2008-2015 (Greatrix & Dowell, 2020). However, empirical evidence on a best way of using UKCAT or UCAT in the selection process is not available (Greatrix & Dowell, 2020). Therefore, implementing a valid aptitude test as a supplementary test and exploring predictive validities of such a test across a national sample is likely to provide more robust evidence of the utility of aptitude tests in the Sri Lankan context.

This study used an aptitude test developed and deployed systematically and modelled on a widely tested international aptitude test format for medical school selection. The sample was representative of demographics of the national cohort of medical students and included all three major ethnic groups and students with a wide range of Z scores. However, the limited number of test items may have offset these strengths. The authors did not have access to participants’ raw scores at the advanced level examination, which may have provided a more appropriate measure of performance for statistical manipulation. In addition, the aptitude test was administered to students who have already been selected to medical school, whereas ideally, it should have been administered to medical school applicants. Despite the limitations, the aptitude test returned an acceptable internal reliability score, and further analysis demonstrated a way forward.

V. CONCLUSION

This study was the first time in the literature that an aptitude test was evaluated in medical education settings in the Sri Lankan context. Using an aptitude test to select students for medical school in Sri Lanka and similar settings, where admission processes have not diversified, is potentially valuable. However, further studies are necessary to establish evidence to support the adoption of such a test in the Sri Lankan and other similar resource-constrained contexts where prior educational achievement primarily informs selection decisions. In the Sri Lankan context, further evaluating the psychometric properties and the predictive validity of an improved aptitude test using a national cohort is recommended as the way forward.

Notes on Contributors

Dr. Sivapalan Sanchayan contributed to the development of the aptitude test, administering the test, collection of data, analysis and writing of the manuscript.

Dr. Sisira Dharmaratne contributed to the conceptualisation, study design, development of the aptitude test, data analysis and writing of the manuscript.

Ms. Shalika Pathirana took part in developing and administering the test, data collection and reviewing the manuscript.

Dr. Pavithra Godamunne contributed to the conceptualisation, study design, development of the aptitude test, data collection, analysis and revision of the manuscript.

Prof. Madawa Chandratilake took part in the conceptualisation, study design, data analysis and revision of the manuscript

Ethical Approval

Ethical clearance (P/223/12/2018) was obtained from the Faculty of Medicine, University of Kelaniya, Sri Lanka.

Data Availability

The datasets generated and/or analysed during the current study are not publicly available to preserve the confidentiality of the participants in this study but are available from the corresponding author on reasonable request.

Acknowledgement

We acknowledge the contribution of Ms. B. N. Jayasena (former lecturer, Department of Disability Studies, Faculty of Medicine, University of Kelaniya) and Ms. L. K. E. Chathurika (former lecturer, Department of Disability Studies, Faculty of Medicine, University of Kelaniya) in the development of the aptitude test. We also acknowledge the contribution of Dr. Rasiah Thayakaran (Research fellow in Health Informatics, Institute of Applied Health Research, University of Birmingham) in performing factor analysis.

Funding

The research received no specific grant from any funding agency.

Declaration of Interest

The authors have no conflicts of interest to disclose.

References

Bala, L., Pedder, S., Sam, A. H., & Brown, C. (2022). Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis. Medical Teacher, 44(4), 401–409. https://doi.org/10.1080/0142159X.2021.1998401

Cohen, L., Manion, L., & Morrison, K. (2018). Research Methods in Education. Routledge.

de Silva, N. R., Pathmeswaran, A., & de Silva, H. J. (2004). Selection of students for admission to a medical school in Sri Lanka. The Ceylon Medical Journal, 49(3), 81–85. https://doi.org/10.4038/cmj.v49i3.3245

de Silva, N. R., Pathmeswaran, A., de Silva, N., Edirisinghe, J. S., Kumarasiri, P. V., Parameswaran, S. V., Seneviratne, R., Warnasuriya, N., & de Silva, H. J. (2006). Admission to medical schools in Sri Lanka: Predictive validity of selection criteria. The Ceylon Medical Journal, 51(1), 17–21. https://doi.org/10.4038/cmj.v51i1.1371

Dharmaratne, S., & Ponnamperuma, G. (2020). Selection for health professions education. In M. Chandratilake & A. Olupeliyawa (Eds.), 14 topics in health professions education (pp. 120–132). College of Medical Educationists, Sri Lanka.

Gliatto, P., Leitman, I. M., & Muller, D. (2016). Scylla and charybdis: The MCAT, USMLE, and degrees of freedom in undergraduate medical education. Academic Medicine, 91(11), 1498–1500. https://journals.lww.com/academicmedicine/Fulltext/ 2016/11000/Scylla_and_Charybdis__The_MCAT,_USMLE,_and_Degrees.16.aspx

Greatrix, R., & Dowell, J. (2020). UKCAT and medical student selection in the UK – what has changed since 2006? BMC Medical Education, 20(1), 1–12. https://doi.org/10.1186/s12909-020-02214-1

Greatrix, R., Nicholson, S., & Anderson, S. (2021). Does the UKCAT predict performance in medical and dental school? A systematic review. BMJ Open, 11, Article e040128. https://doi.org/10.1136/BMJOPEN-2020-040128

Hambleton, R. K., & Zenisky, A. L. (2010). Translating and adapting tests for cross-cultural assessments. In D. Matsumoto, F. J. R. van de Vijver (Eds.), Cross-cultural research methods in psychology (pp. 46–70). Cambridge University Press.

Hewage, S. N., Salgado, L. S. S., Fernando, G. M. O., Liyanage, P., Pathmeswaran, A., & De Silva, N. R. (2011). Selection of medical students in Sri Lanka: Time to re-think criteria? Ceylon Medical Journal, 56(1), 22–28.

MacKenzie, R. K., Cleland, J. A., Ayansina, D., & Nicholson, S. (2016). Does the UKCAT predict performance on exit from medical school? A national cohort study. BMJ Open, 6, Article e011313. https://doi.org/10.1136/BMJOPEN-2016-011313

McManus, I. C., Dewberry, C., Nicholson, S., & Dowell, J. S. (2013). The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools. BMC Medicine, 11, Article 244. https://doi.org/10.1186/1741-7015-11-244

McManus, I. C., Dewberry, C., Nicholson, S., Dowell, J. S., Woolf, K., & Potts, H. W. W. (2013). Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: Meta-regression of six UK longitudinal studies. BMC Medicine, 11, Article 243. https://doi.org/10.1186/1741-701 5-11-243

Mettananda, D. S. G., Wickramasinghe, V. P., Kudolugoda Arachchi, J., Lamabadusuriya, S. P., Ajanthan, R., & Kottahachchi, D. (2006). Suitability of selection criteria as a measure of outcome of medical graduates: University of Colombo. The Ceylon Journal of Medical Science 49, 1–12.

Paniagua, M. A., & Swygert, K. A. (2016). Constructing written test questions for the basic and clinical sciences (4th ed.). National Board of Medical Examiners.

Paton, L. W., McManus, I. C., Cheung, K. Y. F., Smith, D. T., & Tiffin, P. A. (2022). Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study. BMJ Open, 12, Article e056129. https://doi.org/10.1136/bmjopen-2021-056129

Patterson, F., Cousans, F., Edwards, H., Rosselli, A., Nicholson, S., & Wright, B. (2017). The predictive validity of a text-based situational judgment test in undergraduate medical and dental school admissions. Academic Medicine, 92(9), 1250–1253. https://doi.org/10.1097/ACM.0000000000001630

Patterson, F., Roberts, C., Hanson, M. D., Hampe, W., Eva, K., Ponnamperuma, G., Magzoub, M., Tekian, A., & Cleland, J. (2018). 2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions. Medical Teacher, 40(11), 1091–1101. https://doi.org/10.1080/0142159X.2018.149 8589

Powis, D. A., Munro, D., & Bore, M. R. (2021). Is the UCAT appropriate for selecting undergraduate medical students? Medical Journal of Australia, 214(2), 77–78. https://doi.org/10.5694/mja2.50906

Prideaux, D., Roberts, C., Eva, K., Centeno, A., Mccrorie, P., Mcmanus, C., Patterson, F., Powis, D., Tekian, A., & Wilkinson, D. (2011). Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 conference. Medical Teacher, 33(3), 215–223. https://doi.org/10.3109/0142159X.2011.551560

Soemantri, D., Karunathilake, I., Yang, J. H., Chang, S. C., Lin, C. H., Nadarajah, V. D., Nishigori, H., Samarasekera, D. D., Lee, S. S., Tanchoco, L. R., & Ponnamperuma, G. (2020). Admission policies and methods at crossroads: A review of medical school admission policies and methods in seven Asian countries. Korean Journal of Medical Education, 32(3), 243–256. https://doi.org/10.3946/KJME.2020.169

Taber, K. S. (2018). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/ s11165-016-9602-2

University Grants Commission. (2022). Sri Lanka university statistics 2022. Ministry of Higher Education. https://www.ugc.ac.lk/downloads/statistics/stat_2022/Chapter%202.pdf

University Grants Commission. (2023). Admission to undergraduate courses of the universities in Sri Lanka – 2022/2023. Ministry of Higher Education. https://www.ugc.ac.lk/downloads/admissions/Handbook_2022_23/student_handbook_english.pdf

Zimmermann, S., Klusmann, D., & Hampe, W. (2017). Correcting the predictive validity of a selection test for the effect of indirect range restriction. BMC Medical Education, 17, Article 246. https://doi.org/10.1186/s12909-017-1070-5

*S. Sanchayan
Faculty of Medicine,
University of Jaffna,
Adiyapatham Road,
Kokuvil West, Kokuvil, Sri Lanka
Email: sanch.s6@gmail.com,
sanchayan@univ.jfn.ac.lk

Submitted: 15 November 2023
Accepted: 1 August 2024
Published online: 7 January, TAPS 2025, 10(1), 17-26
https://doi.org/10.29060/TAPS.2025-10-1/OA3146

Eugene Kua¹, Sinead Kado⁴, Valsan Philip Verghese⁵, Pauline Wake⁶ & Amy Gray^1,2,3

¹Murdoch Children’s Research Institute, The Royal Children’s Hospital, Melbourne, Australia; ²The Department of Paediatrics, Melbourne Medical School, University of Melbourne, Melbourne, Australia; ³The Royal Children’s Hospital Melbourne, Parkville, Victoria, Australia; ⁴Division of Health Professions Education, School of Allied Health, University of Western Australia, Perth, Western Australia; ⁵Department of Paediatric Infectious Diseases, Christian Medical College, Vellore, India; ⁶School of Medicine and Health Sciences, University of Papua New Guinea, National Capital District, Port Moresby, Papua New Guinea

Abstract

Introduction: COVID-19 pandemic disruptions to medical educational programs have been felt worldwide. However, little is understood about the experience of Low to Middle Income Countries’ (LMIC) and the impact of ensuing disruptions on medical education. We aimed to develop a collective understanding of this through the perspectives of individual LMIC educators, exploring how they continued to deliver undergraduate and postgraduate medical education in the context of pandemic barriers, by reflecting on their successes and shortcomings.

Methods: A phenomenological study was conducted with ten purposefully recruited educators involved in medical education from mainly the Asia-Pacific region. In-depth interviews via web-conferencing were conducted, and data analysed using Braun & Clarke’s (2006) framework. 

Results: Four major themes were identified: ‘Navigating Change, ‘Altered and Divided Teaching Roles and Spaces’, ‘Curricula Impact’ and ‘Challenges and Affordances of Remote Learning’. Educators described major disruptions to delivering medical education, particularly clinical education. Remote learning was the mainstay of facilitating continued education, often requiring investment into infrastructure that was non-existent pre-pandemic.

Conclusion: LMIC educators’ pandemic response have been robust in pursuit of continued medical education. Necessary adoption of online teaching approaches has provided avenues to close healthcare education gaps by facilitating improved reach and quality of medical education in these regions. Building on this requires strategies that meet LMICs areas of need, focusing on capacity building that sustains and grows new pedagogical practices.  We must ensure educational advances are equitable and accessible for all, identifying and supporting countries and contexts at risk of being left behind.

Keywords:           Low to Middle Income Countries, COVID-19, Medical Education, Remote Learning, Qualitative

Practice Highlights

• Educational ramifications from COVID-19 were most acutely discernible within the clinical domain.
• LMICs demonstrated resilient responses, despite their comparatively limited resource availability.
• The pandemic necessitated remote learning and heralded investment into infrastructure.
• Remote learning may provide a means of narrowing health disparities.
• COVID-19 practices of remote learning should be leveraged and supported in LMICs.

I. INTRODUCTION

Declared a global pandemic by the World Health Organisation in March 2020, COVID-19 has had significant global influences, impacting the social fabric and operations of everyday living (Cucinotta & Vanelli, 2020). Governments and institutions were required to re-examine and enforce changes to social procedures in light of the need for infection control, whilst also finding ways to minimise the negative impacts of disruptions to healthcare provision and medical education (Kaul et al., 2021).

The United Nations Secretary-General warned that COVID-19 disruptions to education threaten to cause a ‘generational catastrophe’, widening existing inequalities and stifling what established and evolving progress has been made (UNESCO, 2020). Medical education was not exempt. Indeed, in a global survey of 424 universities and higher education institutes by the International Association of Universities in early 2020, 59% had ceased campus activities and closed, with indications that LMICs were being hit harder, struggling and lagging in their responses and abilities to adapt compared to high income countries (HIC) counterparts (Marinoni et al., 2020).

Adaptations, challenges and impacts to medical education during the pandemic are well described in HICs (Dedeilia et al., 2020; Gallagher & Schleyer, 2020; Gill et al., 2020; Kachra & Brown, 2020; Kaul et al., 2021; Rose, 2020). LMIC narratives in comparison have been relatively isolated, and often represented by single institutions (Adesunkanmi et al., 2021; Aslan & Sayek, 2020; Cecilio-Fernandes et al., 2020; Fiorillo & Javed, 2021; Kalayasiri & Wainipitapong, 2021; Kanmounye & Esene, 2020; Sahi et al., 2020; Sandal et al., 2021; Tokuç & Varol, 2020). Collectively, what has happened to medical education in LMICs, particularly in contexts with fever resources, already stretched healthcare systems and concerns for slower transitions (Aishat et al., 2020; Cecilio-Fernandes et al., 2020; Gill et al., 2020; Sandal et al., 2021)?

This study aimed to answer this question by highlighting the experiences of educators in LMICs during the COVID-19 pandemic, through distilling common themes across various contexts. Through understanding their narratives of successes and shortcomings, we hope to advocate for future needs and highlight opportunities.

II. METHODS

A. Study Design

We conducted a qualitative study using a phenomenological approach involving semi-structured interviews to evaluate the ‘lived experience’ of medical educators in LMICs as they aimed to change, adapt and sustain the delivery of medical education during the pandemic. Interviews were conducted between 7^th July and 21^st October 2021. We sought to capture a range of contexts rather than an in-depth experience of a single site.

B. Inclusion and Exclusion

LMIC educators involved in undergraduate and postgraduate medical programs were the target population. We included educators from tertiary (pre-vocational) and clinical (post-graduate) settings in primary roles of teaching, curriculum development and education oversight. We defined LMIC using the economic bands of low, lower-middle or upper-middle as described by The World Bank for the financial years of 2020 and 2021(World Bank Group, 2021).

C. Data Collection and Analysis

Participants were identified using existing professional networks through purposeful (Côté & Turgeon, 2005; Giacomini & Cook, 2000) and convenience sampling (Etikan et al., 2016). Subsequently, snowball sampling was used whereby existing participants identified future participants from their own professional network. Recruitment was ongoing during the analysis phase until data saturation was achieved. Data saturation was determined through continuous analysis of data and iterative development and review of themes until no new concepts were interpreted.

Invitations to participate were sent via email including a written description of the research goals, study design, and a consent form. Ten consenting participants were scheduled for an interview at a time convenient to them. 

Consistent with phenomenological research (Van Manen, 2016), data was collected via in-depth semi-structured interviews.  Individual interviews were conducted using a guide comprising open ended questions in combination with prompts to ensure consistency of scope while maintaining flexibility to explore experiences raised by participants (Table 1). Key areas of focus included the impacts of the pandemic on medical education at undergraduate and postgraduate levels, strategies used to adapt education, their outcomes and the factors influencing success, and future opportunities.

All interviews were conducted via a password-protected, video-conferencing platform (Zoom, https://zoom.us/) by the principal investigator (EK) who had no pre-existing relationships with participants. Video interviews were conducted and audio-recorded for verbatim transcription. Initial impressions were recorded in the researcher’s journal during or immediately after the interview. Participants were given the opportunity to review written copies of their transcripts on request. There were no withdrawals or repeated interviews.

Thematic analysis of data drew upon both essentialist and constructionist approaches described by Braun and Clarke (Braun & Clarke, 2006), illustrating the lived realities whilst also exploring the impact of the COVID-19 pandemic on institutional operations and subsequent educational outcomes. Interview data was coded by the principal investigator and subsequently reviewed by the chief investigator (AG) to enhance reliability and reduce biases. Theme generation occurred using an inductive approach whereby theoretical concepts were derived from data without a pre-existing framework (Braun & Clarke, 2006). Themes were developed, analysed and refined through iterative consensus meetings between the principal and chief investigators. A selection of interviewees (PW, VV, SK) were subsequently invited to peer review findings to ensure their voices were represented. To ensure trustworthiness of findings, actions taken to address credibility, dependability, confirmability, transferability and reflexivity are outlined in Table 2.

Table 1. Question guide

Table 2. Ensuring trustworthiness

III. RESULTS

A total of 10 educators participated, representing perspectives from eight countries: Vietnam, India, Indonesia, Fiji, Solomon Island, Papua New Guinea, People’s Democratic Republic (PDR) of Lao and Nigeria. Educators had backgrounds in paediatrics, community medicine, internal medicine, anaesthesiology, public health/hospital management and educational leadership.

Four major themes were identified. These were ‘Navigating Change’, ‘Altered and Divided Teaching Roles and Spaces’, ‘Curricular Impact’ and ‘Challenges and Affordances of Remote Learning’.

A. Navigating Change

Educators described operating in a volatile landscape, where the fluctuations of the COVID-19 pandemic forced a dynamic interplay between health policy, health systems and educational institutions. Efforts to control infection rates meant lockdowns, mask wearing, travel restrictions, prevention of mass gatherings, curfews and social distancing precautions were universal experiences. Geographical variability existed in responses within countries to local COVID-19 severity, resulting in a heterogenous distribution of restrictions, and impacts on education.

“Because they [other Pacific Island nations] shut their borders so early, I think most of them have carried on as normal… So don’t think it’s been such a big impact for the other smaller Pacific Islands.”

(Fiji)

Frequent changes in hospital policy, operations and organisational structure meant educators and learners were operating in dynamic environments, at the mercy of redirection of workforce redistributions and space divisions. The dependence of medical curriculum on healthcare environments meant changes invariably influenced the provision of education, with clinical learning hit hardest. How this occurred varied between LMICs over time waves of the pandemic.

“So what happened is you have less number of inpatients, even the consults that we got called to see reduced… And now our (COVID-19) numbers have started coming down so we are going back to normal. So it’s like a cycle as far as numbers and clinical teaching is concerned.”

(India)

B. Altered and Divided Teaching Roles and Spaces

Educators described periods of restricted access to university campuses. Continuing face-to-face teaching was difficult, with examples of existing architectural spaces poorly adaptable to COVID-19-safe measures.

“…ventilation is not very good…it’s not set up for social distancing. Students are backed up in a room …20 centimetres distance between each other.”

(Indonesia)

Clinically, most educators described a spectrum of hospital access, from periods marked by complete cancelation of bedside teaching, to limited access necessitating modified clinical activities, reduced time and crowd limitations.

The move towards remote learning resulted in the delivery of education from home spaces. Here educators described major challenges regarding infrastructure. Successful initiatives included faculty-initiated needs surveys with subsequent provision of required infrastructure for the home. Alternatively, some universities provided spaces fully equipped with remote teaching equipment for use.

 “…we have a survey to all of the lecturers, what they need so that they can give the lecture from their home.”

(Indonesia)

Social distancing necessitated investment into new infrastructure. In PNG, educators described procuring screens to facilitate telehealth to aid medical practice both internally within hospitals and remotely with peripheral healthcare centres. Such technology would then be used for enhancing educational experiences.

“So we now have screens in all the classrooms where we can connect directly to internet ….So I can honestly tell you, we used to ask for it, it never happened. And when COVID-19 came it forced them to make it happen and we’re happy.”

(Papua New Guinea)

For educators with clinical responsibilities, pandemic medicine pulled human resources away from education, driven by re-deployment and personal illness. Capacity was impacted further by fatigue and psychological stress. Educators reported the absence of international trainees normally present to buffer workforce. However, educators employed by universities felt at times underused on the clinical side.

 “The teaching is still ongoing, but it’s not regular …for example, for the Department of Infectious Disease, the teacher has to take responsibility of the COVID-19 ward. So at the time bedside teaching is not that easy to do.”

(Lao PDR)

C. Curricular Impact: Deficiencies in Clinical Content, Assessment and Trusting the New Workforce, and Compensation in Delivery

1) Deficiencies in clinical content: Theoretical content lent itself well to remote learning approaches and was delivered without much disruption to academic calendars. In contrast, clinical education, dependent on availability of clinical spaces and patients, suffered and often occurred over reduced time frames in settings where the content landscape was different and deficient.

“So by the end of January, we called the students in for clinics, and they had their paediatric posting …. I think we had to cram everything into a period of approximately four weeks …it was a struggle to make sure that we had those cases.”

(India)

Cancellation of elective surgical cases resulted in a heavy skew towards emergency cases. Common medical presentations became rare and patient numbers shrank due to travel restrictions and reduced health-seeking behaviour, fostering both clerical and procedure poor environments. At both undergraduate and post-graduate levels, educators lamented this loss of clinical breadth, opportunities and authentic workplace experiences normally afforded to learners. Many expressed concerns about the effects on job-preparedness and skill acquisition of new graduates and those in specialty training.

“…my students couldn’t see like severe malnutrition or TB with spondylosis. Actually, we still have a lot, but I don’t know where these patients are! They disappeared!”

(Indonesia)

“For the current batch of interns, if someone says there’s no spleen, no liver palpable, you don’t trust. You know, it’s quite possible that they’ve missed it completely, because the only exposure they’ve had is anywhere between two to four weeks…”

(India)

The COVID learning environment was not without its own merits. Educators described those at the forefront of COVID-19 care experiencing increased learning opportunities and greater depth of learning via involvement and active management of cases on the clinical floor. Furthermore, it was also viewed by some as a valuable experience for future pandemic preparedness.

“The pandemic will teach the students what we have never taught …they become better clinicians than us when they are exposed to the similar kinds of situations in the future.”

(Papua New Guinea)

2) Assessment and trusting the new workforce: The aforementioned impacts of COVID-19 had flow-on effects on assessments resulting in postponement, modification, replacement by other assessment modalities or cancellation. Irrespective of this, ensuring a continued cycle of new graduates or subspecialists was paramount, particularly in areas of need both locally and regionally, and meant that regardless of the quality and quantity of education learners received, transitions (e.g., undergraduate to postgraduate, postgraduate to subspeciality) were pushed through. Whilst this ensured continuity of workforce, educators voiced concerns regarding clinical competencies of COVID-era cohorts.

“…we can’t delay graduation because we need interns. We need the workforce. …if we don’t conduct exams and we don’t take the next batch … after three years, we actually have nobody.”

(India)

3) Compensation in delivery: Endeavoring to minimise faced shortcomings, educators compensated by trialing and implementing various low to high resource strategies. Delivering education by aligning judicious use of space, crowd management, session timetabling and utilising windows of ‘normality’ was commonplace. Collaborative interdisciplinary relationships were fostered to account for reduced patient pools available for clinical learning/interactions and simulation was used in a couple of settings to make up for the paucity in procedural opportunities.

“We were sharing cases between units. If we had one heart disease, that child would have three clinics in a day by three different sets of people.”

(India)

Evaluating risk of disease versus the benefits of learning, vaccinated students in Papua New Guinea (PNG) who consented were able to continue clinical clerkship schedules with a degree of normality. Recognising a need for increased senior supervision of the post COVID-era workforce, periods of extended internship were also discussed allowing for a longer period of supervised upskilling.

“So what we’ve said to the other clinicians is that don’t worry, everyone’s in the same boat after COVID-19. So the Ministry of Health has said, okay, we’re going to make it a two year internship now, so that they get bit more supervision.”

(Fiji)

D) Challenges and Affordances of Remote Learning

1) Challenges: Educators described limited pre- COVID-19 experiences with remote learning, with little to no incorporation of online teaching in medical education curricula. Furthermore, pedagogical knowledge was limited, with educators often learning how to teach via process of trial and error and little feedback. Time to self-educate was identified as a limiting factor. For those who received support, utility was mixed from helpful to lacking expert guidance.

“…we don’t get any training, on how to use the IT and the tools that can help us teach. This is something that I had to learn on my own, to figure out how to do. So I guess that’s the other barrier, not having that support to help with the students’ virtual learning.”

(Fiji)

Educators describe initial apprehension amongst colleagues with regards to online education, rooted in discomforts around change and pitted against the inertia of entrenched concepts of what constitutes proper teaching. Many had persistent concerns about quality of education provided by online methods and expressed preferences for face-to-face learning – ‘…nothing is equal to the human touch’. Common challenges included limited ability to interact with students, gauge understanding and assess competency. Video cameras were often off due to challenges with bandwidth, and lecture slides were commonly viewed on small smartphone screens.

“But I’ve definitely got the negative people, you know, the cohort of the negatives who decide this is just too hard, you shouldn’t be teaching online …they keep quoting to me Osler …‘you can’t learn medicine without books, but neither can you learn just from the books, you need to have some patient contact.’”

(Fiji)

Despite most learners having some sort of device that could facilitate e-learning, internet reliability was a constraint, alongside prohibitive costs of required data. Acknowledging existing socioeconomic inequities affecting access, provisions for infrastructure, data and reliable connectivity were provided by some educational institutes for learners and teachers.

2) Affordances: For educators, the ability to rehearse and edit lectures was felt to improve the quality of teaching. Many appreciated flexibilities in scheduling and abilities to extend educational sessions beyond the limitations of time and place normally associated with face-to-face teaching. Increased geographical reach provided opportunities for individuals and communities, particularly those limited by location, to access education and expertise previously not available. Accessibility to both domestic and international expertise also helped improve learner engagement.

“… technology has become something that’s sort of changed in our practice …and still plays a vital role in in medical education in our country. And we are now able to reach people we’ve never reached before; we could not reach before. And that’s a good thing.”

(Papua New Guinea)

The ability to bridge international borders provided further opportunities to support and improve programs of domestic education and assessment through international collaboration. Through existing professional relationships, the anaesthetics department in PNG was able to organise fortnightly online teaching sessions to assist with the burden of teaching, as well as invite subspecialty specialists via videoconference to help examine their trainee candidates in viva assessment exams.

IV. DISCUSSION

This qualitative study investigated the impact of COVID-19 on medical education in LMICs from the perspective of medical educators who described how the pandemic transformed their educational landscapes. Curricular impacts were felt most in the clinical domain, and educators conveyed concerns regarding the quality of education during this time and what this meant for the capabilities of future healthcare workers and service provision. In the face of social distancing and contextual barriers unique to LMIC settings, educators looked towards remote learning as a largely successful bridging solution despite some difficulties in establishing and supporting this approach. In recognising remote learning affordances as we emerge from the pandemic, positive changes can be and have been made – how we continue to best support this is currently unclear.

Medical educators early on anticipated challenges related to educational continuity during COVID-19 (Rose, 2020), with narratives from study participants reflecting both the projected impacts and consequential experiences reported by other individual LMICs (Adesunkanmi et al., 2021; Aslan & Sayek, 2020; Cecilio-Fernandes et al., 2020; Connolly & Abdalla, 2022; Fiorillo & Javed, 2021; Kalayasiri & Wainipitapong, 2021; Kanmounye & Esene, 2020; Sahi et al., 2020; Sandal et al., 2021; Tokuç & Varol, 2020). Our study adds by highlighting baseline gaps in pedagogical knowledge of more modern educational techniques, exemplified by the struggles encountered during implementation of remote learning. Where needs for educator instruction were recognised, the extent to which these occurred was heterogeneous, relying on funding and available expertise, connections or partnerships.

Educators without existing collaborations were and continue to be relatively isolated in educational endeavours, whereas those with existing partnerships were enabled to capitalise on opportunity. Collaborations between LMIC educators are powerful initiating tools for grassroots change and for local educators to play leading roles in the development and sustainment of their own competencies and capacities (Al Shorbaji et al., 2015).  Common challenges faced by educators are best addressed by connecting, sharing and learning from each other. This can be facilitated by prioritising ongoing investment in communities of practice that ensures sustainability through fostering continued learning, growth and independence (Ramani et al., 2020). We found that despite societal schisms engendered by the pandemic, proximity among individuals and communities traditionally separated have perhaps closed owing to the pronounced emphasis on online interactions, enabling collaborations that otherwise may not have occurred.

Whilst it would be safe to assume that barriers introduced by pandemics are not general considerations in curriculum design, the COVID-19 pandemic has highlighted how deficiencies in the use of modern educational practices and the consequent severer interruptions to medical education in LMICs compared to HICs, have the potential to widen existing gaps in health outcomes (Chahine et al., 2018; Dattani et al., 2013; Hunter & Reddy, 2013; Marinoni et al., 2020; Mossialos et al.; Roser & Ritchie, 2013). The compensations and affordances – the success stories – described here by LMIC educators serve to highlight how familiarity with technology-assisted education can safeguard education particularly in times of social disorder.

COVID-19 crisis has thus also been a transformative moment for education in LMICs, particularly with the widespread adoption of remote learning, previously under-exploited in many regions (Al Shorbaji et al., 2015). Where the benefits of remote learning have been well described – cost effectiveness, accessibility, portability, coverage, scalability, adaptability and diversity (Al Shorbaji et al., 2015; Barteit et al., 2020; Papapanou et al., 2022) – its potential for ‘revolutionary’ change for medical education in LMICs depends on how changes are taken forward beyond the pandemic. There is an opportunity to close pre- and post-COVID-19 educational and health gaps through the affordances of remote learning via improving educational equity in regions where access to ongoing professional development and education programs have been limited both in quantity and quality (Barteit et al., 2020; Mack et al., 2017). In doing so we must consider context-specific strategies that foster fidelity, adaptability, longevity and accessibility (McLean et al., 2017).

Momentum is key.  We know that attitudes in the broader literature about the longevity of remote learning practices post-COVID 19 are mixed (Adesunkanmi et al., 2021; Motte-Signoret et al., 2021). Indeed, gaps in technology-aided education could widen as countries where remote learning was done well continue to consolidate and innovate such approaches, while those who struggled to adapt may revert to more traditional practices. Nevertheless, there is room for optimism as educators in our study, particularly those of the younger generation, were enthusiastic about the place of remote learning in the future of medical curricula. How then do we support these individuals in capitalising, championing, and sustaining forward momentum? We provide six considerations.

A. Invest in Technology Infrastructure

Prioritise investing in technology infrastructure to ensure reliable and equitable access to remote learning resources. This includes internet connectivity, hardware and spaces, and allocating funds to support remote learning initiatives.

B. Pedagogical Training

Pedagogical training and support that meets educators in LMICs to upskill and adapt teaching methods for remote learning practice, emphasising ongoing efforts to enhance local expertise.

C. Innovating Curriculum

Develop long-term strategies for integrating technology-assisted education into curricula. Remote learning should not be seen as a temporary solution but as a valuable tool for enhancing educational quality, accessibility and ultimately quality healthcare provision.

D. Foster Collaboration

Encourage collaboration and sharing between LMICs, and with HICs on knowledge, resources, and best practices in education. These partnerships can help bridge knowledge gaps and provide mentorship opportunities that link experienced educators with younger generations to champion change.

E. Communities of Practice

Development of communities of practice among educators particularly among LMIC counterparts. These groups can serve as platforms for sharing experiences, challenges, and innovative solutions, thus promoting ongoing learning and growth, which in turn promotes self-sustainability, local expertise and resourcefulness.

F. Stakeholder Engagement

Engage stakeholders at individual, institutional, national, and international levels to commit to and support remote learning initiatives. Collaboration between governments, educational institutions, and non-governmental organisations is essential.

This study has limitations. Whilst our aim is to provide an initial window of understanding into the collective experience of LMICs, we acknowledge that this study is non-exhaustive in capturing all variations in experience. Educators were recruited via convenience, purposeful and snowballing sampling resulting in an Asia-Pacific sample bias. Furthermore, we appreciate that not all medical educational institutes nor specialty departments are necessarily equal in resources and varied experiences may exist domestically. However, the key themes we identified were broad, recurred across all interviews and may be applicable to other LMIC jurisdictions through reader generalisability, that is, where themes resonate with the others. Another limitation was that student perspectives were not included our study. We acknowledge that educational experiences exist in duality between educator and learner and that understanding both will be paramount in developing high fidelity educational strategies (Kachra & Brown, 2020; Zaharias & Poylymenakou, 2009).

V. CONCLUSION

This study gives insight into the degree and scope of disruptions the COVID-19 pandemic had on the medical education landscape and the adaptive changes to medical educational systems implemented by educators in LMIC settings. Despite reduced capacity and infrastructure, LMICs demonstrated resourcefulness and robustness in continuing medical education in a time of health crisis. Our study brings to light the enthusiasm and commitment educators in LMICs have in improving the education that they provide.  With the quality of education intrinsically tied to health outcomes there remains a risk that countries that did not succeed in embedding online learning and teaching approaches will now fall behind on the educational landscape.  There is not just an opportunity but an onus to build on educational progress triggered by COVID-19, particularly that of remote learning methods, and ensure it is grown and sustained, but at the same time identify and assist those at risk of falling behind. 

Notes on Contributors

Dr. Eugene Kua contributed to study design, interviewing of study participants, coding of data, subsequent thematic analysis and paper write up.

Dr. Sinead Kado contributed by means data analysis through peer review, with supplementary contributions in the form of additional content enhancements to the paper.

Prof. Valsan Philip Verghese contributed by means data analysis through peer review, with supplementary contributions in the form of additional content enhancements to the paper.

Dr. Pauline Wake contributed by means data analysis through peer review, with supplementary contributions in the form of additional content enhancements to the paper.

A/Prof. Amy Gray contributed to study design, subsequent thematic analysis and paper write up and review.

Ethical Approval

This project was approved by The Royal Children’s Hospital Melbourne Human Research Ethics Committee (reference number 75258).

Data Availability

No consent was given to share transcript data.

Funding

No funding was provided in conducting this research.

Declaration of Interest

There is no competing interest to be declared.

References

Adesunkanmi, A. O., Ubom, A. E., Olasehinde, O., Wuraola, F. O., Ijarotimi, O. A., Okon, N. E., Ikimalo, J. I., Fasubaa, O. B., & Adesunkanmi, A. R. K. (2021). Impact of the COVID-19 pandemic on surgical residency training: Perspective from a low-middle income country. World Journal of Surgery, 45(1), 10-17. https://doi.org/10.1007/s00268-020-05826-2

Aishat, T. O., Oluwayemisi, T. F., Eniola, I. A., Omolola, O. A., Oluwatobiloba, D. O., & Adeseye, M. A. (2020). Challenges with medical education in Nigeria in the COVID-19 era. Pan African Medical Journal, 37(223). https://doi.org/10.11604/pamj.2020.37.223.26418

Al Shorbaji, N., Atun, R., Car, J., Majeed, A., Wheeler, E. L., Beck, D., Belisario, J. M., Cotič, Ž., George, P. P., & Hirvonen, H. (2015). eLearning for undergraduate health professional education: A systematic review informing a radical transformation of health workforce development. World Health Organisation.

Aslan, D., & Sayek, İ. (2020). We need to rethink on medical education for pandemic preparedness: lessons learnt from COVID-19. Balkan Medical Journal, 37(4), 178-179. https://doi.org/10.4274/balkanmedj.galenos.2020.2020.4.002  

Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020). Evaluation of e-learning for medical education in low- and middle-income countries: A systematic review. Computers & Education, 145, Article 103726. https://doi.org/10.1016/j.compedu.2019.103726  

Braun, V., & Clarke, V. (2006). Using thematic analysis in pyschology. Qualitative Health Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa  

Cecilio-Fernandes, D., Parisi, M., Santos, T., & Sandars, J. (2020). The COVID-19 pandemic and the challenge of using technology for medical education in low and middle income countries [version 1]. MedEdPublish, 9(74). https://doi.org/10.15694/mep.2020.000074.1

Chahine, S., Kulasegaram, K., Wright, S., Monteiro, S., Grierson, L. E. M., Barber, C., Sebok-Syer, S. S., McConnell, M., Yen, W., De Champlain, A., & Touchie, C. (2018). A call to investigate the relationship between education and health outcomes using big data. Academic Medicine, 93(6), 829-832. https://doi.org/10.1097/acm.0000000000002217

Connolly, N., & Abdalla, M. E. (2022). Impact of COVID-19 on medical education in different income countries: a scoping review of the literature. Medical Education Online, 27(1), 2040192. https://doi.org/10.1080/10872981.2022.2040192  

Côté, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education. Medical Teacher, 27(1), 71-75. https://doi.org/10.1080/01421590400016308

Cucinotta, D., & Vanelli, M. (2020). WHO declares COVID-19 a pandemic. Acta bio-medica : Atenei Parmensis, 91(1), 157-160. https://doi.org/10.23750/abm.v91i1.9397  

Dattani, S., Spooner, F., Roser, M., & Ritchie, H. (2013). Child and Infant Mortality. Our World in Data. Retrieved January 9, 2023, from https://ourworldindata.org/child-mortality [Online Resource]

Dedeilia, A., Sotiropoulos, M. G., Hanrahan, J. G., Janga, D., Dedeilias, P., & Sideris, M. (2020). Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo, 34(3 Suppl), 1603-1611. https://doi.org/10.21873/invivo.11950

Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1-4.

Fiorillo, A., & Javed, A. (2021). Education and training in psychiatry in low‐ and middle‐income countries during and after the COVID‐19 pandemic. Asia Pacific Psychiatry, 13(4), Article e12500. https://doi.org/10.1111/appy.12500  

Gallagher, T. H., & Schleyer, A. M. (2020). “We signed up for this!” — student and trainee responses to the Covid-19 pandemic. New England Journal of Medicine, 382(25), Article e96. https://doi.org/10.1056/NEJMp2005234  

Giacomini, M. K., & Cook, D. J. (2000). Users’ guides to the medical literature: XXIII. Qualitative research in health care A. Are the results of the study valid? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 284(3), 357-362. https://doi.org/10.1001/jama.284.3.357  

Gill, D., Whitehead, C., & Wondimagegn, D. (2020). Challenges to medical education at a time of physical distancing. Lancet, 396(10244), 77-79. https://doi.org/10.1016/S0140-6736(20)31368-4

Hunter, D. J., & Reddy, K. S. (2013). Noncommunicable diseases. New England Journal of Medicine, 369(14), 1336-1343. https://doi.org/10.1056/NEJMra1109345  

Kachra, R., & Brown, A. (2020). The new normal: Medical education during and beyond the COVID-19 pandemic. Canadian Medical Education Journal, 11(6), e167-e169. https://doi.org/10.36834/cmej.70317  

Kalayasiri, R., & Wainipitapong, S. (2021). Training of psychiatry and mental health in a low- and middle-income country: Experience from Thailand before and after COVID-19 outbreak. Asia-Pacific Psychiatry, 13(4), Article e12493. https://doi.org/10.1111/appy.12493  

Kanmounye, U. S., & Esene, I. (2020). COVID-19 and neurosurgical education in Africa: Making lemonade from lemons. World Neurosurgery, 139, 732-733. https://doi.org/10.1016/j.wneu.2020.05.126

Kaul, V., Gallo de Moraes, A., Khateeb, D., Greenstein, Y., Winter, G., Chae, J., Stewart, N. H., Qadir, N., & Dangayach, N. S. (2021). Medical education during the COVID-19 pandemic. Chest, 159(5), 1949-1960. https://doi.org/10.1016/j.chest.2020.12.026

Mack, H. G., Golnik, K. C., Murray, N., & Filipe, H. P. (2017). Models for implementing continuing professional development programs in low-resource countries. MedEdPublish, 6(1).

Marinoni, G., Van’t Land, H., & Jensen, T. (2020). The impact of COVID-19 on higher education around the world. IAU global survey report, 23, 1-17.

McLean, M., McKimm, J., & Gibbs, T. (2017). Medical education in difficult circumstances: A global responsibility to contribute. Medical Teacher, 39(1), 4-6.

Motte-Signoret, E., Labbé, A., Benoist, G., Linglart, A., Gajdos, V., & Lapillonne, A. (2021). Perception of medical education by learners and teachers during the COVID-19 pandemic: A cross-sectional survey of online teaching. Medical Education Online, 26(1), 1919042. https://doi.org/10.1080/10872981.2021.1919042  

Papapanou, M., Routsi, E., Tsamakis, K., Fotis, L., Marinos, G., Lidoriki, I., Karamanou, M., Papaioannou, T. G., Tsiptsios, D., Smyrnis, N., Rizos, E., & Schizas, D. (2022). Medical education challenges and innovations during COVID-19 pandemic. Postgraduate Medical Journal, 98(1159), 321. https://doi.org/10.1136/postgradmedj-2021-140032

Ramani, S., McKimm, J., Thampy, H., O’Sullivan, P. S., Rogers, G. D., Turner, T. L., Chisolm, M. S., Kusurkar, R. A., Hays, R., Fornari, A., Kachur, E. K., Wilson, K. W., Filipe, H. P., & Schumacher, D. J. (2020). From clinical educators to educational scholars and leaders: Strategies for developing and advancing a career in health professions education. The clinical teacher, 17(5), 477-482.

Rose, S. (2020). Medical student education in the time of COVID-19. JAMA, 323(21), 2131-2132. https://doi.org/10.1001/jama.2020.5227

Sahi, P. K., Mishra, D., & Singh, T. (2020). Medical education Amid the COVID-19 pandemic. Indian Pediatrics, 57(7), 652-657. https://doi.org/10.1007/s13312-020-1894-7  

Sandal, S., Boyarsky, B. J., & Cantarovich, M. (2021). The higher impact of the COVID‐19 pandemic on resident/fellow training in low‐ and middle‐income countries. Transplant International, 34(5), 988-989. https://doi.org/10.1111/tri.13853  

Tokuç, B., & Varol, G. (2020). Medical education in Turkey in time of COVID-19. Balkan Medical Journal, 37(4), 180-181. https://doi.org/10.4274/balkanmedj.galenos.2020.2020.4.003 

UNESCO. (2020). Policy brief: education during COVID-19 and beyond. https://unsdg.un.org/sites/default/files/2020-08/sgpolicy_brief_covid-19_and_education_august_2020.pdf

Van Manen, M. (2016). Researching lived experience: Human science for an action sensitive pedagogy. Routledge.

World Bank Group. (2021). World Bank Country and Lending Groups – World Bank Data Help Desk. Retrieved December 16. 2021 from https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups

Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: beyond functional usability. International Journal of Human–Computer Interaction, 25(1), 75-98. https://doi.org/10.1080/10447310802546716  

*Eugene Kua
Royal Children’s Hospital,
50 Flemington Rd,
Parkville VIC 3052
(+61) 411 224 125
Email: ekua12@gmail.com

Submitted: 3 April 2024
Accepted: 22 July 2024
Published online: 1 October, TAPS 2024, 9(4), 33-39
https://doi.org/10.29060/TAPS.2024-9-4/OA3282

Meltem Saygılı¹, Aysun Kandemir Türe¹ & Şirin Özkan²

¹Department of Health Management, Kırıkkale University, Kırıkkale, Türkiye; ²Department of Medical Services and Techniques, Vocational School of Health Services, Uludag University, Türkiye

Abstract

Introduction: Many societal changes threaten the sustainability of health systems. Entrepreneurs play a significant role in creating sustainable innovations necessary for development and structural change. This study aimed to evaluate how university students studying health sciences perceive individual entrepreneurship during the COVID-19 pandemic and to explore the impact of COVID-19-related hopelessness on their perception of entrepreneurship.

Methods: Six hundred eighty-one undergraduate students from the faculty of health sciences in Türkiye, participated in the study. The Individual Entrepreneurship Perception Scale and Beck Hopelessness Scale were used to collect data.

Results: The results revealed that the students’ perception of individual entrepreneurship was high, while their hopelessness levels were moderate. In addition, a moderate negative relationship was found between the students’ hopelessness levels and their perception of individual entrepreneurship. The regression analysis showed that the students’ hopelessness levels and sub-dimensions (hope, loss of motivation, and future expectation) accounted for 26.3% of the total variance in individual entrepreneurship perception. Increased hopelessness regarding the sub-dimensions of hope, loss of motivation, and expectations for the future decreased their individual entrepreneurship perceptions.

Conclusions: These findings suggest that individual entrepreneurship perception among health sciences students could be negatively impacted during extraordinary periods related to health crises like pandemics. Developing hope, motivation, and expectations for the future is crucial to enhancing individual entrepreneurship perception among health sciences students.

Keywords: Individual Entrepreneurship Perception, Hopelessness, Health Sciences, University Students, COVID-19 Pandemic

Practice Highlights

• University education plays a critical role in developing students’ perceptions of entrepreneurship.
• Students’ perceptions of entrepreneurship are affected by changing environmental conditions.
• The COVID-19 pandemic has led to a significant increase in awareness of Healthcare Entrepreneurship.
• The COVID-19 pandemic has negatively affected students’ expectations and hopes for the future.
• Health sciences students experienced the psychological effects of the pandemic more intensely.

I. INTRODUCTION

Entrepreneurs are described as individuals who “implement bold new ideas from the outset, taking on the risks of the business or venture that enables innovation to happen” (Suryavanshi et al., 2020). Entrepreneurship education seeks to provide students with in-depth knowledge, motivation, and skills to increase entrepreneurial intentions’ success in various settings (Sampene et al., 2023). In developing economies such as Türkiye, government efforts to improve unemployment conditions are directed towards developing students’ positive attitudes towards entrepreneurship and increasing their interest in self-employment.

Universities are crucial in developing students’ entrepreneurial intentions (Bagheri, 2018). It is critical to examine and develop the entrepreneurial potential of university students trained to fulfill important roles in the field of health services in the future. This is because students and newly graduated healthcare professionals with entrepreneurial skills are now needed to develop innovative healthcare delivery models that can improve the quality of healthcare services, improve treatment outcomes, and reduce the cost of care (Fashami et al., 2021; Mohamed et al., 2023). It is stated that healthcare is one of the main industrial areas where user entrepreneurship takes place worldwide (Schiavone et al., 2020) and the pandemic environment brings unique opportunities for entrepreneurs, especially in healthcare. However, research shows that fear and worry due to the COVID-19 pandemic are making the future workforce anxious and hopeless about their professional futures (Duplaga & Grysztar, 2021; Tao et al., 2023). In such an environment, it is predicted that university students, especially those who are trained to work in health service delivery, may act in different ways in their decisions about their professional future.

Because an individual’s perception of current conditions can significantly, positively, or negatively affect the formation of entrepreneurial intentions and the likelihood of starting an enterprise (Krichen & Chaabouni, 2022). Considering the increasing entrepreneurship opportunities in healthcare and the negative effects of the COVID-19 pandemic on healthcare services, it has become important to address the entrepreneurship perceptions of students who will serve in this field in the future and to evaluate the hopelessness experienced. When the literature was examined, no other research addressing entrepreneurship in the context of hopelessness was found. Therefore, we believe that our research findings will make a valuable contribution to filling this gap in the literature.

A. Universities, Health Sciences Education and Entrepreneurship

Recent years have focused on the role of universities in industrial competitiveness and their impact on social change (Hart & Rodgers, 2023), and universities in teaching and initiating entrepreneurship have emerged as leading actors. Recent research examining students’ attitudes toward entrepreneurship reveals that universities have a critical role in developing entrepreneurial spirit and perceptions of entrepreneurship in students (Krichen & Chaabouni, 2022; Liao et al., 2022; Sampene et al., 2023). Moreover, university students now value real experience more and prefer universities that offer entrepreneurship opportunities and experiences (Taneja et al., 2024).

Universities providing education in healthcare have traditionally emphasised vocational training and unfortunately, there are very few university programs with curricula that focus on innovation and entrepreneurship in healthcare (Suryavanshi et al., 2020). Consequently, most newly graduated physicians and other healthcare professionals start their careers without the knowledge, experience, and courage to integrate their medical knowledge with the healthcare business world, and thus are unable to commercialise new healthcare delivery methods (Boore & Porter, 2011).

B. Hopelessness Experienced by Students Due to the COVID-19 Pandemic and Perception of Individual Entrepreneurship

Students are one of the groups that experience the psychological effects of the pandemic most intensely (Marahwa et al., 2022). Studies conducted with students during the pandemic period showed that students experienced higher levels of anxiety and depressive symptoms due to uncertainties about the future and the negative effects of the pandemic on their educational lives (Alnıaçık et al., 2021; Hajinasab et al., 2022; Kaplan Serin & Doğan, 2023; Zhang et al., 2021). Moreover, it was also determined that students’ hopelessness levels were higher in the first wave of the pandemic and during the quarantine period and decreased with normalisation (Tao et al., 2023). In addition, it has been found that students who are educated in the field of health and have clinical experience are more anxious and hopeless than undergraduate students who are educated outside the field of health (Pretorius, 2021). Although entrepreneurship is a future-oriented orientation and is perceived as the ability to turn negative situations into opportunities, it is estimated that the psychological effects of COVID-19 may negatively affect entrepreneurial intention in students. Therefore, this study aims to examine the effect of hopelessness experienced by health sciences faculty students during the COVID-19 pandemic on their perception of individual entrepreneurship.

II. METHODS

A. Sampling and Participants and Data Collection

The study was conducted with university students studying at the Faculty of Health Sciences in Türkiye. The faculty consists of six departments, namely Physiotherapy and Rehabilitation, Health Management, Social Work, Nutrition and Dietetics, Child Development and Nursing, and has 2335 students. The sample size required for the study was calculated using the formula (n=[z²xp(1-p)/d²]/1+[z²xp(1-p)/d²N]) and it was determined that a minimum of 337 participants would be sufficient with a 5% error level and 95% confidence level. Participants were selected using a simple random sampling technique and volunteerism was taken as a basis in obtaining the data. The study was conducted between April and May 2022 and 681 valid questionnaires were returned from the participating students.

Data were collected using the socio-demographic information form, Individual Entrepreneurship Perception Scale, and Beck Hopelessness Scale. The Individual Entrepreneurship Perception Scale is a 5-point Likert scale consisting of 31 items and six sub-dimensions: planning, locus of control, self-confidence, communication, motivation, and self-discipline. This scale was developed by Incik and Uzun in 2017 to assess university students’ perceptions of entrepreneurship. The reliability coefficients (Cronbach’s Alpha) of the sub-dimensions of the Individual Entrepreneurship Perception Scale and the total scale were found to be 0.80, 0.84, 0.75, 0.75, 0.75, 0.72, 0.75, 0.60 and 0.92, respectively (Incik & Uzun, 2017). For this study, it was determined that the Cronbach Alpha values of the Individual Entrepreneurship Perception Scale ranged between 0.69 and 0.93 (Table 2).

The Hopelessness Scale was initially developed by Beck et al. in 1974 and subsequently adapted into Turkish by Durak and Palabıyıkoğlu in 1994. This scale comprises 20 items divided into three sub-dimensions: expectations for the future, loss of motivation, and hope. Each question in the scale requires a ‘yes’ or ‘no’ response, and the maximum achievable score is 20. An increase in the overall score denotes a higher level of hopelessness. The reliability coefficients for the sub-dimensions of the scale were calculated to be 0.78, 0.72, and 0.72, respectively (Durak & Palabıyıkoğlu, 1994). Table 2 presents the Cronbach’s alpha coefficients for the sub-dimensions and the scale, which were found to be 0.75, 0.77, 0.74, and 0.88, respectively.

B. Statistical Analysis

The statistical analyses used were performed in SPSS v23.0. Descriptive analyses such as percentage, mean, standard deviation, reliability analysis (Cronbach’s Alpha), and normality tests were performed to obtain the results. To test the normal distribution, kurtosis, and skewness values were calculated for each scale’s sub-dimensions and total scores. It was determined that the values obtained were in the range of -1.5 to +1.5 and the data showed a normal distribution (Tabachnick & Fidell, 2013). The results of the analysis are presented in Table 2. In addition, multiple linear regression analysis was used to investigate the relationship between hopelessness and students’ perception of individual entrepreneurship.

III. RESULTS

The study involved students with an average age of 21.3 years, ranging from 18 to 44 years old. 86% of the participants were female students, and the majority (52.3%) reported a grade point average of 3 or above (with a scale of 0 to 4). 68.3% of the students lived in the city center with their families. Among the students’ mothers, 37.7% had a primary school graduate degree, and 81.4% were not working. On the other hand, 32.2% of the students’ fathers had a high school graduate degree, and 78.4% were actively working. Most of the students (68.3%) stayed in dormitories. Regarding career plans after graduation, most students expressed their interest in working in a health institution (public or private). At the same time, only 15.6% said they wanted to start their own business. 15% of the students were still undecided about their work plans (Table 1).

Submitted: 21 February 2024
Accepted: 16 July 2024
Published online: 1 October, TAPS 2024, 9(4), 26-32
https://doi.org/10.29060/TAPS.2024-9-4/OA3255

Aletheia Chia¹, Menghao Duan¹ & Sashikumar Ganapathy^2,3

¹Paediatric Medicine, KK Women’s and Children’s Hospital, Singapore; ²Department of Emergency Medicine, KK Women’s and Children’s Hospital, Singapore; ³Paediatric Academic Clinical Programme, Duke-NUS Medical School, Singapore

Abstract

Introduction: Mentoring is an essential component of post-graduate medical training programs worldwide, with potential benefits for both mentors and mentees. While factors associated with mentorship success have been described, studies have focused on intrapersonal characteristics and are largely based in Western academic programs. Mentorship occurs in a broader environmental milieu, and in an Asian context, cultural factors such as respect for authority, hierarchy and collectivism are likely to affect mentoring relationships. We aim to explore the lived experience of mentors within an Asian postgraduate medical training program, and thus identify challenges and develop best practices for effective mentoring.

Methods: 14 faculty mentors from a post-graduate paediatric residency program were interviewed between October 2021 to September 2022. Data was collected through semi-structured one-on-one interviews, with participants chosen via purposeful sampling. Qualitative analysis was done via a systematic process for phenomenological inquiry, with interviews thematically coded separately by 2 independent reviewers and checked for consistency.

Results: 4 main thematic concepts were identified: “professional, but also personal”, “respect and hierarchy”, “harmony and avoidance of open conflict” and the “importance of trust and establishing a familial relationship”. Mentors also highlighted the value of structure in Asian mentoring relationships.

Conclusion: Cultural factors, which are deeply rooted in social norms and values, play an important role in shaping mentoring relationships in an Asian context. Mentoring programs should be tailored to leverage on the unique cultural norms and values of the region in order to promote career growth and personal development of trainees and mentors.

Keywords:           Medical Education, Graduate Medical Education, Professional Development

Practice Highlights

• Cultural factors are key in shaping Asian mentoring relationships.
• This includes being ‘professional, but also personal’, ‘respect and hierarchy’, ‘harmony and avoidance of open conflict’ and the “importance of trust and establishing a familial relationship’.
• Mentoring programs should be tailored to leverage on the unique local cultural norms and values.

I. INTRODUCTION

Mentoring is an essential component of post-graduate medical training programs worldwide. Mentorship is a reciprocal, interdependent relationship between a mentor (often a faculty member who is senior and experienced) and a mentee (beginner or protégé in the field) (Sambunjak et al., 2006). Benefits for mentees include aiding career preparation, development of clinical and communication skills, independence, and preventing burnout (Flint et al., 2009; Ramanan et al., 2006; Spickard et al., 2002). Mentors derive satisfaction from aiding the next generation, motivation for ongoing learning and institutional recognition (Burgess et al., 2018).

Variables associated with mentoring success have been described. Key components identified by mentors and mentees are communication and accessibility, caring personal relationship, mutual respect and trust, exchange of knowledge, independence and collaboration, and role modelling (Eller et al., 2014). Personality differences, lack of commitment, conflict of interests and mentor’s lack of experience can contribute to unsuccessful mentoring relationships (Straus et al., 2013).

However, mentorship occurs in a broader environmental milieu. Sambunjak (2015) described an ecological model of mentoring in academic medicine, with a first societal level of cultural, economic and political factors; a second institutional level of system- and organisation-related factors, and a third level of intrapersonal and interpersonal characteristics. Studies on mentorship have mainly focused on the latter and are situated in Western academic programs. In an Asian context, cultural factors such as respect for authority, hierarchy and collectivism may affect mentoring relationships (Chin & Kameoka, 2019). Trainees may show more deference to their mentors, and mentors may be more directive than collaborative. An Asian study surveying Doha’s postgraduate paediatric program found 75% mentees unsatisfied in their mentoring relationship (Khair et al., 2015).

We aim to explore the lived experience of mentors within an Asian postgraduate medical training program, and thus identify the challenges faced by trainees and mentors and develop best practices for effective mentoring.

II. METHODS

A. Study Design

This qualitative study is based on an interpretive phenomenological approach of participants’ lived experiences in their mentoring relationships. Through close examination of individual experiences, phenomenological analysis seeks to capture the meaning and common features, or essences, of an experience (Starks & Trinidad, 2007).

Semi-structured interviews were conducted. The interview guide was designed to follow a pre-determined structure whilst allowing for flexibility in probing. It was based on insights from literature on key socio-cultural determinants of successful mentoring relationships. Data was collected until saturation, with no new themes emerging.

B. Setting

We studied a paediatric residency program of a tertiary academic centre in Singapore, with 47 residents and 180 faculty members.

A formal mentorship program (Figure 1) has been in place since 2010. Residents indicate preferred faculty mentors at the start of residency, and are advised to consider specialty of interest, characteristics, and gender. Matches are subject to availability, review by the residency program, and mentor acceptance. Residents have one formal mentor throughout the 6 years unless the mentorship is terminated by mutual agreement between mentor and mentee.

Figure 1. Mentorship program structure, with suggested meeting timings and requisite forms. Meetings are required minimally 6-monthly and are scheduled on an ad-hoc basis by the mentor and mentee.

C. Participants

Purposive sampling to identify mentors in the residency program who would provide comprehensive and relevant insights. Considerations included age, gender, race, and years of mentorship and faculty experience. Study members and their mentors were excluded.

Study information sheets were provided to participants with assurance of confidentiality, and written informed consent obtained from each participant. The study was approved by the SingHealth Institution Review Board.

D. Analysis

Qualitative analysis was done via a systematic process for phenomenological inquiry (Creswell & Creswell, 2022), whereby statements were analysed and categorised into clusters of meaning that represent phenomenon of interest. Transcripts were interpreted independently by 2 reviewers (AC, MD) and reviewed by a 3rd study member (SG). Iterative data analysis and collection was performed, with coding done after each interview to identify new themes and inform further interviews.

III. RESULTS

We interviewed 14 mentors from October 2021 to September 2022. 8 were male and 6 were female. 12 were Chinese, 1 Indian, and 1 of other ethnicity. This was representative of faculty demographics. Mentors had two to eleven years of mentorship experience within the program, and one to five existing and prior mentees.

Mentors described their lived experiences in their mentoring journey, providing insights into key values and their relationships’ evolution. 4 main thematic concepts were identified: “professional, but also personal”, “respect and hierarchy”, “avoidance of open conflict” and the “importance of trust”. Mentors also highlighted the value of structure in Asian mentoring relationships.

A. Professional, but also Personal

All mentors agreed that the relationship was predominantly professional, with their key role being that of professional and career guidance. They described their roles as:

“Guidance through difficult decisions or challenges” (#1), “leaning the real world of medicine” (#2), “driving professional development” (#12) and providing “timely and wise advice to support the journey” (#13)

Relationships “predominantly focused on professional or educational aspects… as that’s what it was meant to be” (#10), and were “mainly limited to career-related matters (#11)”.

However, many also identified personal connection as key. While the focus was primarily professional, awareness of personal or emotional aspects aided in understanding their mentors to further professional development and psycho-emotional growth. This included sharing of family lives, and emotional difficulties faced at work.

As the journey progresses it becomes a lot more about the psycho-emotional aspect, and about their mental health and personal well-being. (#1)

A lot of time is spent discussing family issues. If we knew more about the personal life of our mentee it’s so much easier to tailor the advice based on the individual’s unique circumstances. (#3)

A minority of mentors kept their relationship strictly professional and preferred not to talk about aspects outside of work, as it was ‘easier’ (#10) and shared concerns of ‘overstepping certain norms’ (#11).

B. Respect and Hierarchy

Respect was a key factor brought up when exploring the socio-cultural aspects of mentoring in our Asian community. Mentors varied in their opinion as to the extent that this resulted in a hierarchical relationship, and if this had a negative or positive impact on the relationship.

All agreed that respect is a key value in mentoring relationships:

Culturally there’s a large part to play as we’re taught to respect our elders. (#1)

Respecting elders – definitely it’s more prominent in our Asian culture. (#2)

Many mentors highlighted that this resulted in a hierarchical relationship. This manifested in the way senior doctors were addressed strictly by title, polite communication, and consideration of what would be ‘proper’ to discuss or ask a mentor to do.

The hierarchical kind of mindset is still very strong, and is something that is not necessarily healthy. (#4)

You would always see your mentor as someone higher than you. It’s similar to the way in our Asian context we see our parents. a certain sense of distance (#11).

The way medicine is a 师傅徒弟 kind of thing (‘master and disciple’) (#13)

Many shared that this could be a barrier to open communication with juniors wanting to “respect and agree” with their mentors (#14), slowing the growth of some relationships.

No matter how much honesty and trust there is. If they want to say something that their mentor is not happy to hear, or strikes them as being a bit rude or disrespectful – they won’t say it. (#1)

Our culture does say to respect your senior, don’t argue and don’t disagree with your senior. Sometimes they’re not very vocal, ‘ok sir ok sir’. And then later you find out they have certain issues. (#9)

One mentor felt that hierarchy did not play a large part in his mentoring relationships. This was possibly personality related, describing himself as naturally “quite informal”.

Mentors also highlighted factors that mitigated the hierarchical nature of their relationship. This included time, and setting clear boundaries and goals of the relationship.

When we give… a clear boundary and aim with no go zones, then culture may not necessarily be that important anymore (#10)

A minority of mentors felt that hierarchy and respect was not a limiting factor in their relationships:

If the primary aim is having someone to offer you guidance and a different point of view, even if the mentee sees you as someone who is not equal, you can still have that effectiveness. (#11)

C. Harmony, Avoiding Open Conflict and Confrontation

Another socio-cultural concept highlighted was the avoidance of confrontation. While some of this was linked to avoiding disagreements given the hierarchical nature of the relationship, avoiding open conflict and striving for harmony was also a key factor.

Rather than openly bringing up something, to avoid being confrontational we have evolved other means of trying to work our way through that conflict. There is a conscious and deliberate effort to avoid open and confrontational conflict. (#3)

When I was in the UK, they really questioned their mentors quite a lot – almost like a quarrel. That kind of questioning style may not be that well received in our own culture. (#2)

When mentees had differing opinions from their mentors “they would rather not talk about the topic again, or just ask someone else” in order to preserve the relationship (#1).

Within our program, this resulted in difficulties in exiting the relationship to avoid “offending” the mentor:

When the mentor-mentee relationship is breaking down, culturally it can be more difficult for mentees to request to swap. That’s very detrimental to both the mentor and the mentee in the long run. (#1)

This also manifested in avoiding overly ‘emotional’ discussions, with discussion often being more “superficial”, “reserved” (#7) and “factual” (#5) in nature.

Conversely, one mentee shared that younger mentees being of a “younger generation” were more open to speaking their mind, and that this would continue to evolve.

D. Importance of Trust and Establishing a Familial Relationship

In exploring key values for successful mentoring relationships, many highlighted the importance of trust and building up an established relationship.

Chemistry and compatibility when starting out was key. Mentors often felt more comfortable if there was a pre-existing relationship they had their mentors and had “shared commonalities and chemistry”. Honesty and trust were key in enabling the relationship to progress. This included respecting each other’s confidentiality. Relationships without trust was difficult as mentors “had to keep guessing what they want”, and “whatever you plan may not be the real goals of what they actually want” (#2). Over time, establishing the relationship made it easier to confide in each other, overcoming boundaries brought on by hierarchy.

It’s about forming relationships before you can start reflecting with the person. Over time we get to know each other, and seeing that what is shared is truly kept private and confidential. Once we have trust among each other it (reservations) doesn’t become a barrier. (#7)

There must be a certain comfort and trust level before one readily does share vulnerabilities. (#2)

This can be enabled by being approachable, and creating safe environments where mentors can share their difficulties without consequence. However, this could be compromised if mentors have to take up a supervisory role or be involved in remediation processes.

Mentors who developed close and trusting relationships with their mentees described it as familial in nature. This could be as a big brother or sister who would give advice to their younger siblings in non-threatening and neutral ways. It was also described by one mentor as parental in nature.

One interviewer highlighted that whilst Asian cultural factors may limit mentoring, there were also potential benefits:

We must find the best of both worlds. The independence that the Western systems have is good, but Asians tend to be better at teamwork and team spirit. (#13)

E. Value of Structure in an Asian Mentorship Relationship

Many mentors highlighted the value of having a framework for their mentoring relationship. Formalisation of the relationship and having a structure provided a foundation for discussions and enabled them to set boundaries. This prevented it from becoming awkward or “random and situation-based” (#15), and also helped faculty who were “still learning the whole journey of mentoring” (#7).

When we don’t know what to talk about it becomes quite awkward and uncomfortable. But if in the Asian context the mentor brings to it some structure, and they respect that structure, that structure is helpful. (#10)

A minority of mentors felt having a framework was too rigid or unnecessary.

The structure must be there to guide the mentors, but the mentors chosen must also be of a certain maturity so they can find their own way. We must not be too prescriptive or rigid. (#13)

IV. DISCUSSION

In this study, we explored the lived experiences of mentors within an Asian paediatric postgraduate training program. Existing studies have explored characteristics of effective and ineffective mentor relationships, but less is known about the impact of sociocultural factors. Key thematic concepts identified such as “respect and hierarchy” and “avoidance of open conflict” highlighted the importance of cultural factors in shaping mentoring relationships in an Asian context. These are deeply rooted in social norms and values of the region.

Hierarchy is a fundamental aspect of many Asian cultures, where individuals are expected to show respect and deference to their ‘elders’ or those in positions of authority. This was also observed in other Asian communities. A study in postgraduate medicine in Japan found that mentees had an inner desire to “respect the mentor’s ideas”, with both mentees and mentors embracing “paternalistic mentoring” (Obara et al., 2021). In our interviews, this was most apparent in the way mentees addressed their mentors: by title and respectfully. On a deeper level, this was a barrier to open communication. Open sharing was identified as crucial for a constructive mentoring relationship (Burgess et al., 2018), with the lack of it a cause of failed mentoring relationships (Straus et al., 2013).  The willingness to share personal experiences by both mentors and mentees is key for effective mentoring and career growth. Additionally, this is not conducive to fostering creativity and innovation, which are increasingly important in the medical profession.

Communication was also affected by avoidance of open conflict and confrontation. Asian cultures have been described as collectivist, where the needs of the group take precedence over that of the individual, and intragroup harmony is paramount (Chin & Kameoka, 2019). In mentoring relationships, this translates to prioritising a successful and harmonious relationship over personal goals. Indirect communication styles are also more common in many Asian cultures. This has been described as high-context communication, whereby “most of the information is either in the physical context or internalised in the person, while very little is in the coded, explicit, transmitted part of the message” (Hall, 1976). Relying on indirect language nonverbal cues rather than explicitly stating one’s thoughts and feelings can hinder open communication.

Hierarchy and a lack of open communication may result in mentors taking on the role of advisors or coaches rather than true mentors. While there is no universal definition of mentorship, key features are that of a long-term dyadic relationship that encompasses educational, training and professional aspects that is personal and reciprocal (Sambunjak & Marusic, 2009). This is in contrast to tutors or coaches that primarily exhibit educational functions, or counsellors that exhibit personal functions. If the mentor-mentee relationship if influenced by hierarchical norms, mentors may be seen as figures of authority rather than partners in development. Cultural respect for authority figures and an emphasis on conformity may also discourage mentees from questioning or having open conversations with their mentors, limiting mutual learning.

Challenges with hierarchy and communication can be overcome with the aid of a structured program, and eventually establishment of trust and ‘familial’ relationships.  A structured program can guide mentors and mentees in having open communication. In an Asian context, mentors may initially play a more authoritative role in guiding and directing their mentees with the aid of a structured guide, from which more two-way communication may open up as the relationship becomes more established. Whilst desirable mentors have characteristically been described as not “bossy” or authoritative (Sambunjak & Marusic, 2009), a study of Japanese physician-scientist mentor-mentees viewed more paternalistic mentoring as favourable (Obara et al., 2021).  However, this will need to be individualised, as a highly directive mentoring style may not be well-suited to those who prefer a more collaborative and participatory mentoring relationship. Communication and learning styles may also continue to evolve with as incoming trainee physicians belong increasingly to Generation Z (1997-2012) instead of Generation Y/Millennials (1981-1996). A study of the mentorship experiences of Gen Z women medical students by Li et al (2024) described how current society had afforded them more opportunities for empowerment and expression, and emphasised the importance of tailored mentorship that considered the mentee’s identify and intersectionality.

Having mutual respect and trust were also key. The mentee and mentor having a pre-existing relationship and familiarity helped, and was more common in our context given that mentees could indicate their mentor of interest. Mutual respect and having a personal connection were also identified as key components in effective mentoring relationships by Eller (2014) and Straus (2013).

Whilst we had initially hypothesised that Asian sociocultural concepts would limit mentorship relationships to be largely professional, mentors shared that mutual respect, trust, and time enabled the relationship to also extend to sharing of personal matters and psychosocial wellbeing. Successful relationships were even described as ‘familial’, with a sense of fulfilment from both parties. A family-like relationship and a sense of loyalty to the mentor and organisation was also described in Japanese mentoring relationships (Obara et al., 2021). Such relationships may be more common in more collectivist cultures. These can be furthered by fostering a sense of community amongst mentees and mentors, such as through group activities, peer support, and shared learning experiences.

A. Limitations

This study was conducted in one of the two paediatric training centres in Singapore. Future studies should expand to other postgraduate programs to improve applicability of the results.

The investigators were participants in the program as mentees or mentors, with potential for bias in analysis. To minimise this, transcripts were analysed independently by two investigators followed by review by the third investigator. While our study focused on the lived experience of mentors, examining the perspective of mentees would be able to provide a more balanced and comprehensive understanding of mentoring relationships and highlight gaps where they can be better supported, and should be considered in future studies.

Our study did not delve into gender dynamics. Female medical trainees may face unique challenges, and male mentors may be stereotypically less nurturing and more process-oriented. Existing studies are varied: a survey of American cardiologists found sex concordance to be beneficial (Abudayyeh et al., 2020), whereas Jackson (2003) did not find same-gender matching to be important in an US academic program. In our initial interviews, gender did not come up as a significant factor and was hence not a focus subsequently. The role of gender in our program may have been minimised by a balanced gender ratio, with 59% of faculty female.

B. Future Research and Practical Implications

Given the significant influence of sociocultural factors on mentoring relationships, mentoring programs should be tailored to reflect the unique cultural norms and values of the region. In Asian cultures, this would include methods to reduce hierarchy, ensuring accessibility to mentors, and having a structured program. Training on mentorship for mentors and mentees would be beneficial to promote characteristics of effective mentoring relationships, and should include a focus on culturally sensitive mentoring with a recognition of how culturally-shaped beliefs can affect mentorship. This is particularly important in multicultural societies where cross-cultural mentorship is more common.

V. CONCLUSION

Cultural factors play an important role in shaping mentoring relationships in an Asian context. Whilst such these may be limiting to a degree, these can be also be leveraged on to further effective mentoring programs. Mentoring programs should be tailored to reflect the unique cultural norms and values of the region to promote career growth and personal development of trainees and mentors.

Notes on Contributors

AC, MD and SG contributed to study conception and design. Participant interviews were conducted by AC. Analysis and thematic interpretation were done by AC, MD with review by SG. All authors were involved in drafting the manuscript and reviewing it critically, and all read and approved the final manuscript.

Ethical Approval

The study was approved by the SingHealth Institution Review Board (IRB number 2021/2542).

Data Availability

The data of this qualitative study are not publicly available due to confidentiality agreements with the participants. 

Funding

No funding was received for conducting this study. 

Declaration of Interest

The authors declare that they have no competing interests.  

References

Abudayyeh, I., Tandon, A., Wittekind, S. G., Rzeszut, A. K., Sivaram, C. A., Freeman, A. M., & Madhur, M. S. (2020). Landscape of mentorship and its effects on success in cardiology. JACC: Basic to Translational Science, 5(12), 1181-1186. https://doi.org/10.1016/j.jacbts.2020.09.014

Burgess, A., van Diggele, C., & Mellis, C. (2018). Mentorship in the health professions: A review. The Clinical Teacher, 15(3), 197-202. https://doi.org/10.1111/tct.12756

Chin, D., & Kameoka, V. A. (2019). Mentoring Asian American scholars: Stereotypes and cultural values. American Journal of Orthopsychiatry, 89(3), 337-342. https://doi.org/10.1037/ort0000 411

Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods approaches. SAGE Publications.

Eller, L. S., Lev, E. L., & Feurer, A. (2014). Key components of an effective mentoring relationship: A qualitative study. Nurse Education Today, 34(5), 815-820. https://doi.org/10.1016/j.nedt. 2013.07.020

Flint, J. H., Jahangir, A. A., Browner, B. D., & Mehta, S. (2009). The value of mentorship in orthopaedic surgery resident education: The residents’ perspective. The Journal of Bone and Joint Surgery, 91(4), 1017-1022. https://doi.org/10.2106/JBJS.H.00934

Hall, E. T. (1976). Beyond culture. Anchor Press/Double Day.

Jackson, V. A., Palepu, A., Szalacha, L., Caswell, C., Carr, P. L., & Inui, T. (2003). “Having the right chemistry”: A qualitative study of mentoring in academic medicine. Academic Medicine, 78(3), 328-334. https://doi.org/10.1097/00001888-200303000-00020

Khair, A. M., Abdulrahman, H. M., & Hammadi, A. A. (2015). Mentorship in pediatric Arab board postgraduate residency training program: Qatar experience. Innovations in Global Health Professions Education. https://doi.org/10.20421/ighpe2015.6

Li, C., Veinot, P., Mylopoulos, M., Leung, F. H., & Law, M. (2024). The new mentee: Exploring Gen Z women medical students’ mentorship needs and experiences. The Clinical Teacher, 21(3), e13697. https://doi.org/10.1111/tct.13697

Obara, H., Saiki, T., Imafuku, R., Fujisaki, K., & Suzuki, Y. (2021). Influence of national culture on mentoring relationship: A qualitative study of Japanese physician-scientists. BMC Medical Education, 21(1), 300. https://doi.org/10.1186/s12909-021-02744-2

Ramanan, R. A., Taylor, W. C., Davis, R. B., & Phillips, R. S. (2006). Mentoring matters: Mentoring and career preparation in internal medicine residency training. Journal of General Intermal Medicine, 21(4), 340-345. https://doi.org/10.1111/j.1525-1497.20 06.00346.x

Sambunjak, D. (2015). Understanding wider environmental influences on mentoring: Towards an ecological model of mentoring in academic medicine. Acta Medica Academica, 44(1), 47-57. https://doi.org/10.5644/ama2006-124.126

Sambunjak, D., & Marusic, A. (2009). Mentoring: What’s in a name? JAMA, 302(23), 2591-2592. https://doi.org/10.1001/jama. 2009.1858

Sambunjak, D., Straus, S. E., & Marusic, A. (2006). Mentoring in academic medicine: A systematic review. JAMA, 296(9), 1103-1115. https://doi.org/10.1001/jama.296.9.1103

Spickard, A., Gabbe, S. G., & Christensen, J. F. (2002). Mid-career burnout in generalist and specialist physicians. JAMA, 288(12), 1447-1450. https://doi.org/10.1001/jama.288.12.1447

Starks, H., & Trinidad, S. B. (2007). Choose your method: A comparison of phenomenology, discourse analysis, and grounded theory. Qualitative Health Research, 17(10), 1372-1380. https://doi.org/10.1177/1049732307307031

Straus, S. E., Johnson, M. O., Marquez, C., & Feldman, M. D. (2013). Characteristics of successful and failed mentoring relationships: A qualitative study across two academic health centers. Academic Medicine, 88(1), 82-89. https://doi.org/10.1097/ACM.0b013e31827647a0

*Dr Aletheia Chia
Department of Paediat,
KK Women’s and Children’s Hospital
100 Bukit Timah Road
Singapore 229899
Email: aletheia.chia@mohh.com.sg

Submitted: 8 January 2024
Accepted: 2 July 2024
Published online: 1 October, TAPS 2024, 9(4), 14-25
https://doi.org/10.29060/TAPS.2024-9-4/OA3212

Hema Suryavanshi¹, Santosh R Patil², Kaladhar Reddy Aileni³ & Mohmed Isaqali Karobari⁴

¹Department of Oral Pathology and Microbiology, Chhattisgarh Dental College & Research Institute, India; ²Department of Oral Medicine and Radiology, Chhattisgarh Dental College & Research Institute, India; ³Department of Preventive Dentistry, College of Dentistry, Jouf University, Kingdom of Saudi Arabia; ⁴Dental Research Unit, Centre for Global Health Research, Saveetha Institute of Medical and Technical Sciences, India

Abstract

Introduction: Oral health education is essential for adolescents, yet traditional methods face challenges in engaging this demographic. This study evaluates the effectiveness of a multimedia-based dental education program among high school students. This study was carried out to assess the impact of a comprehensive multimedia-based dental education program on oral health knowledge and practices among high school students.

Methods: A total of 228 students were recruited and randomly assigned to experimental (n=114) and control (n=114) groups. The experimental group received a four-week multimedia program, while the control group followed the standard curriculum. Outcome measures included oral health knowledge scores, brushing frequency, flossing habits, dietary choices, and observational assessments. Statistical analysis employed descriptive statistics, paired t-tests, and analysis of covariance (ANCOVA).

Results: The multimedia-based program led to a significant increase in oral health knowledge scores in the experimental group compared to the control group (Mean Change: 14.6, p < 0.001). Moreover, the experimental group demonstrated higher brushing frequency (p < 0.001), increased flossing habits (p < 0.001), and improved dietary choices (p < 0.05). Observational assessments showed higher adherence to oral hygiene practices in the experimental group (p < 0.001).

Conclusion: The multimedia-based dental education program effectively improved oral health knowledge and practices among high school students. This approach holds promise for scalable and engaging oral health education strategies.

Keywords:           Multimedia-based Education, Oral Health, Adolescents, Dental Education, Health Promotion, Preventive Dentistry

Practice Highlights

• Integrated multimedia elements enhanced participant engagement, utilising videos, e-learning modules, and pamphlets.
• Positive shifts in oral health practices observed, with increased brushing and flossing frequencies in the experimental group.

I. INTRODUCTION

Oral health education serves as a cornerstone in empowering adolescents with the knowledge and skills necessary to maintain optimal oral hygiene. By educating young individuals about the importance of regular brushing, flossing, and dental check-ups, as well as the impact of dietary choices on oral health, preventive practices can be instilled early on. This not only helps in preventing common dental problems like cavities and gum disease but also promotes overall health and quality of life (Nakre & Harikiran, 2013).

Furthermore, addressing oral health education during adolescence is strategic as it provides an opportunity to intervene during a critical developmental stage. By targeting adolescents with effective educational interventions, it becomes possible to shape behavior and instill healthy habits that can last a lifetime. This proactive approach not only benefits the individual’s oral health but also reduces the burden on healthcare systems by minimising the need for costly and invasive dental treatments later in life (Tadin et al., 2022).

The increasing prevalence of dental problems presents a significant public health challenge, as untreated oral diseases can lead to pain, discomfort, and even serious complications such as tooth loss, systemic infections, and impaired overall health (Texas Dental Association, 2008). Moreover, dental problems impose a substantial economic burden on healthcare systems, with costs associated with treatment, emergency care, and lost productivity. In light of these challenges, effective educational interventions are crucial for curbing the burden of oral diseases (Ghoneim et al., 2022). Oral health education programs aim to raise awareness about the importance of oral hygiene, preventive measures, and regular dental check-ups. By providing individuals with the knowledge and skills necessary to maintain good oral health practices, these interventions can empower them to take proactive steps to prevent dental problems and minimise their impact (Kassebaum et al., 2017). This study addresses this need by investigating the impact of a multimedia-based dental education program on the oral health knowledge and practices of high school students.

Adolescence represents a critical period for establishing lifelong health behaviors, and oral health is no exception (National Institutes of Health, 2021). Unfortunately, global studies indicate a concerning prevalence of dental issues among adolescents, including dental caries, gingivitis, and poor oral hygiene practices (Agbelusi & Jeboda., 2006, Zhang et al., 2021). Such issues not only impact immediate health but can also lead to long-term consequences, emphasising the urgency of effective oral health education.

Traditional oral health education methods often face challenges in engaging and effectively conveying information to adolescents. Conventional classroom lectures and textbook-based approaches may struggle to capture the attention of this demographic, potentially limiting the efficacy of such interventions. However, there is evidence to suggest that traditional oral health education has been effective in improving oral health knowledge (Angelopoulou et al., 2015), reducing plaque accumulation, and promoting oral health in adolescents (Gousalya et al., 2022). On the other hand, it is important to note that traditional models of oral health education have been criticised for their didactic delivery and failure to acknowledge the individual context and motivations of the recipients (Ford & Farah., 2012).

As technology becomes increasingly integrated into daily life, leveraging multimedia platforms for health education emerges as a promising strategy to enhance engagement and knowledge retention (George et al., 2014).

The primary objective of this study was to assess the effectiveness of a comprehensive multimedia-based dental education program in improving oral health knowledge and promoting positive oral hygiene practices among high school students. By employing a well-structured intervention and rigorous evaluation, this research aims to contribute valuable insights that can inform the design and implementation of future oral health education initiatives.

The significance of this study lies in its potential to inform evidence-based oral health education strategies for adolescents, especially in regions facing resource constraints. If successful, the multimedia-based approach could serve as a scalable and cost-effective model for widespread implementation in school settings. Ultimately, the outcomes may contribute not only to the improvement of oral health outcomes among adolescents in the study setting but also to the development of best practices with broader implications for global oral health promotion.

II. METHODS

A. Intervention

A comprehensive multimedia-based dental education program was developed by a team of experienced dental educators, multimedia designers, and content developers. The program included visually engaging educational videos covering various aspects of oral health, interactive e-learning modules to reinforce key concepts, and informative pamphlets providing supplementary written material.

Throughout the intervention period, the research team closely monitored the attendance and participation of students in both the experimental and control groups. Attendance records, completion rates of e-learning modules, and participation levels in interactive sessions were meticulously documented to assess the program’s adherence.

1) Program feedback and modification: Continuous feedback sessions were held with both students and educators to evaluate the effectiveness of the multimedia-based dental education program. Adjustments and modifications were made in real-time based on feedback received, ensuring the program remained dynamic and responsive to the specific needs and preferences of the participants.

2) Quality assurance: To maintain consistency and quality across program delivery, educators underwent training sessions, and periodic evaluations were conducted to assess their adherence to the prescribed curriculum and teaching methodologies.

B. Study Design

A randomised controlled trial was conducted involving 228 high school students, aged 14 to 18, recruited from four local high schools in Burhanpur, India, after obtaining permission from the school authorities. Institutional Review Board (IRB) approval with approval number GGSCDS&RC/2022/IEC/168 was obtained prior to the start of this study. Students and their parents/guardians were provided with detailed information about the study, and informed consent was obtained from all participants. The study was conducted between January 2022 and December 2023.

A sample size calculation was conducted prior to the initiation of the study to ensure adequate statistical power. Based on previous literature and anticipated effect sizes, the calculation indicated that a sample size of 228 participants would provide sufficient power to detect significant differences in oral health knowledge and practices between the experimental and control groups. Factors such as an alpha level of 0.05, a power of 0.80, and an effect size estimate derived from similar interventions were taken into consideration during the calculation. This approach ensured that the study was adequately powered to detect meaningful differences in the outcomes of interest.

Random assignment of participants to the experimental and control groups was performed using a computer-generated randomisation sequence. The sequence was generated by an independent researcher not directly involved in the implementation of the study. Participants were assigned to the experimental or control group based on their identification numbers, ensuring an equal chance of allocation to either group. This randomisation process helped minimise selection bias and ensure that any differences observed between the groups could be attributed to the intervention rather than systematic differences in participant characteristics.

Given that participants were recruited from multiple local high schools in Burhanpur, India, cluster randomisation was employed to minimise contamination between groups. High schools were considered as clusters, and randomisation was carried out at the school level to prevent potential contamination of intervention effects between students within the same school. This approach helped maintain the integrity of the study design and reduce the risk of cross-group contamination, thereby enhancing the internal validity of the findings.

To address potential biases related to academic performance, we collected demographic information from all participants, including indicators of academic achievement. This information allowed us to assess any confounding variables during data analysis. Additionally, randomisation was employed to ensure that participants with varying academic backgrounds were equally distributed between the experimental and control groups, minimising the impact of academic performance on study outcomes.

1) Experimental group (n=114): Prior to the intervention, informed consent was obtained from both students and their parents/guardians. The multimedia-based dental education program was delivered to the experimental group during regular school hours. The intervention spanned four weeks, with carefully structured sessions designed to cover topics such as oral anatomy, proper brushing and flossing techniques, the impact of diet on oral health, and the importance of regular dental check-ups. Each week, students participated in interactive discussions, watched educational videos, completed e-learning modules, and received informational pamphlets to reinforce the acquired knowledge. To ensure engagement and understanding, the program incorporated quizzes, group activities, and open discussions facilitated by trained dental educators.

2) Control group (n=114): The control group received the standard oral health education provided as part of the existing school curriculum. The standard curriculum included traditional classroom lectures, textbook readings, and basic demonstrations of oral hygiene practices. No additional multimedia resources were introduced to the control group to maintain a clear distinction from the experimental group.

C. Development of Survey Instrument

The survey instrument utilised for baseline assessment underwent a rigorous validation process to ensure its reliability and validity in measuring participants’ oral health knowledge. The validation process included several key steps aimed at enhancing the quality and accuracy of the survey instrument.

The survey items were delved based on an extensive review of existing literature on oral health knowledge among adolescents. Content experts in the fields of dentistry, public health, and education were consulted to ensure that the survey items adequately covered essential concepts related to oral health.

Prior to the commencement of the main study, a pilot test of the survey instrument was conducted with a small group of high school students similar to the study population. Feedback from the pilot test participants was carefully analysed, and adjustments were made to the wording and clarity of the survey items based on their input.

The survey instrument underwent thorough review by a panel of experts comprising professionals from diverse backgrounds, including dentistry, education, and survey design. The expert panel assessed the relevance, comprehensibility, and appropriateness of the survey items for the target population, providing valuable feedback for further refinement.

D. Outcome Measurements and Analysis

In this study, a comprehensive set of outcome measures was employed to assess the impact of the intervention on participants’ oral health knowledge and practices. The study commenced with a baseline assessment, during which demographic data, oral health history, and participants’ knowledge of oral health practices were collected. Throughout the four-week intervention period, participants’ oral health practices were evaluated weekly through self-reports and observations, ensuring continuous monitoring of their adherence and engagement with the program. Following the intervention, a post-intervention assessment was conducted, and participants’ oral health practices were monitored for an additional two weeks to assess the sustainability of behavior changes beyond the intervention period.

1) Pre-intervention assessment: Before the initiation of the intervention, a thorough baseline assessment of participants’ oral health knowledge was conducted using validated surveys (Appendix A). These surveys covered a wide spectrum of oral health concepts, including the anatomy of teeth and gums, the significance of fluoride, and common dental diseases. Individual knowledge scores were calculated, establishing a baseline measure for each participant, with the maximum score for the knowledge assessment set at 100 (Appendix B).

2) Post-intervention assessment: Following the four-week intervention period, participants from both the experimental and control groups underwent a post-intervention survey (Appendix C). This survey mirrored the pre-intervention assessment and allowed for a direct comparison of changes in knowledge. Individual knowledge scores were recalculated, and the difference between pre- and post-intervention scores was used to quantify the impact of the educational program.

3) Brushing frequency: Participants’ brushing habits were assessed through a combination of self-reporting and observation. Each participant maintained a detailed log, recording the frequency and duration of their toothbrushing activities. Trained researchers also conducted periodic observations to validate self-reported data, enhancing the reliability of the findings.

4) Flossing habits: Similar to brushing, participants self-reported their flossing habits, detailing the frequency and thoroughness of their flossing routine. Periodic checks were conducted to corroborate self-reported data, ensuring the accuracy of the information gathered.

5) Dietary choices: The study assessed participants’ dietary habits related to oral health, including the consumption of sugary snacks and beverages. Dietary logs and self-reporting were used, and nutritional assessments were conducted to evaluate participants’ awareness of the relationship between diet and oral health.

Trained researchers conducted periodic observations to assess participants’ oral health practices in a naturalistic setting. This included direct observations of toothbrushing and flossing routines, as well as an assessment of participants’ adherence to recommended oral hygiene practices.

Throughout the study, continuous monitoring and quality checks were implemented to ensure the accuracy and reliability of the outcome measures. Any deviations or unexpected trends in the data were promptly investigated, and corrective actions were taken as needed. This approach ensured the robustness of the study’s outcome assessments.

Descriptive statistics were employed for demographic data. Paired t-tests were used to compare pre- and post-intervention knowledge scores. Analysis of covariance (ANCOVA) was conducted to assess the impact of the intervention on oral health practices, controlling for baseline differences. IBM SPSS Statistics software (version 22.0) was utilised for data analysis, with a predetermined significance level set at p < 0.05.

III. RESULTS

The demographic characteristics table illustrates a well-balanced study population, with both the experimental and control groups having similar age distributions (16.2 ± 1.0 vs. 16.1 ± 0.9) and gender representation (Male/Female: 56/58 vs. 55/59). Baseline knowledge scores align closely, with mean scores of 45.2 (experimental) and 44.8 (control) and standard deviations of 8.3 and 8.1, respectively (Table 1).