{"id":71839,"date":"2022-09-26T08:57:16","date_gmt":"2022-09-26T00:57:16","guid":{"rendered":"https:\/\/medicine.nus.edu.sg\/taps\/?post_type=issues&#038;p=71839"},"modified":"2023-01-03T09:04:06","modified_gmt":"2023-01-03T01:04:06","slug":"comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool","status":"publish","type":"issues","link":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/","title":{"rendered":"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool"},"content":{"rendered":"<p>Submitted: 13 January<span lang=\"EN-GB\"> 2022<\/span><br \/>\r\nAccepted: 9 May 2022<br \/>\r\nPublished online: 4 October<span lang=\"EN-GB\">, TAPS 2022, <\/span><span lang=\"EN-GB\">7(4), 35-49<\/span><br \/>\r\n<a href=\"https:\/\/doi.org\/10.29060\/TAPS.2022-7-4\/OA2699\">https:\/\/doi.org\/10.29060\/TAPS.2022-7-4\/OA2699<\/a><\/p>\r\n<p style=\"text-align: justify;\"><span lang=\"EN-GB\">Yuan Kit Christopher Chua<sup>1*<\/sup>, Kay Wei Ping Ng<sup>1*<\/sup>, Eng Soo Yap<sup>2,3<\/sup>, Pei Shi Priscillia Lye<sup>4<\/sup>, Joy Vijayan<sup>1<\/sup>, &amp; Yee Cheun Chan<sup>1<\/sup> <\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify;\"><i><sup><span lang=\"EN-GB\">1<\/span><\/sup><\/i><i><span lang=\"EN-GB\">Department <\/span><\/i><i>of Medicine, Division of Neurology, National University Hospital Singapore, Singapore;<\/i><i> <sup><span lang=\"EN-GB\">2<\/span><\/sup><span lang=\"EN-GB\">Department<\/span><\/i><i> of\u00a0Haematology-oncology, National University Cancer Institute Singapore, Singapore; <sup>3<\/sup>Department of Laboratory Medicine, National University Hospital Singapore, Singapore; <sup>4<\/sup>Department of Medicine, Division of\u00a0Infectious Diseases,\u00a0National University Hospital Singapore, Singapore<\/i><\/p>\r\n<p style=\"text-align: justify;\"><i>*Co-first authors<\/i><\/p>\r\n<p style=\"text-align: center;\"><b><span lang=\"EN-GB\">Abstract <\/span><\/b><\/p>\r\n<p style=\"text-align: justify;\"><b>Introduction:<\/b> In-class engagement enhances learning and can be measured using observational tools. As the COVID-19 pandemic shifted teaching online, we modified a tool to measure the engagement of instructors and students, comparing in-person with online teaching and different class types.<span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify;\"><b>Methods:<\/b> Video recordings of in-person and online teachings of six identical topics each were evaluated using our \u2018In-class Engagement Measure\u2019 (IEM). There were three topics each of case-based learning (CBL) and lecture-based instruction (LLC). Student IEM scores were: (1) no response, (2) answers when directly questioned, (3) answers spontaneously, (4) questions spontaneously, (5) initiates group discussions. Instructor IEM scores were: (1) addressing passive listeners, (2) asking \u22651 students, (3) initiates discussions, (4) monitors small group discussion, (5) monitoring whole class discussions.<span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify;\"><b>Results:<\/b> Twelve video recorded sessions were analysed. For instructors, there were no significant differences in percentage time of no engagement or IEM scores when comparing in-person with online teaching. For students, there was a significantly higher percentage time of no engagement for the online teaching of two topics. For class type, there was overall less percentage time of no engagement and higher IEM scores for CBL than LLC. <span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify;\"><b>Conclusion:<\/b> Our modified IEM tool demonstrated that instructors\u2019 engagement remained similar, but students\u2019 engagement reduced with online teaching. Additionally, more in-class engagement was observed in CBL. \u201cPresenteeism\u201d, where learners were online but disengaged was common. More effort is needed to engage students during online teaching. <span lang=\"EN-GB\"><\/span><\/p>\r\n<p><b><span lang=\"EN-GB\">Keywords:<\/span><\/b>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <i><span lang=\"EN-GB\">Engagement, Observational Tool, Online Learning, E-learning, COVID-19, Medical Education, Research<\/span><\/i><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Practice Highlights<\/span><\/strong><\/p>\r\n<ul>\r\n\t<li style=\"text-align: justify;\"><span lang=\"EN-GB\"> <\/span>Lectures to large class (LLC) and case-based learning (CBL) are associated with lower levels of student engagement when conducted on a virtual platform.<span lang=\"EN-GB\"><\/span><\/li>\r\n\t<li style=\"text-align: justify;\"><span lang=\"EN-GB\"> <\/span>Instructors\u2019 engagement during online teachings remained similar to that of in-person teachings.<span lang=\"EN-GB\"><\/span><\/li>\r\n\t<li style=\"text-align: justify;\"><span lang=\"EN-GB\"> <\/span>LLC is associated with reduced student engagement than CBL.<span lang=\"EN-GB\"><\/span><\/li>\r\n<\/ul>\r\n<p style=\"text-align: center;\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">I. INTRODUCTION<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\"><span>Educational theories suggest that learning should be an active process. According to social constructivist theory, learning can be better achieved by social interactions in the learning environment<\/span><span>\u00a0<\/span><span>(Kaufman, 2003). Active learning strategies fostering the students to interact with each other and the instructor<\/span><span>\u00a0<\/span><span>such as<\/span><span>\u00a0<\/span><span>discussions, talks, questions, may yield desirable learning outcomes in terms of knowledge, skills, or attitudes<\/span><span> <\/span><span>(Rao<\/span><span> &amp; <\/span><span>DiCarlo, 2001).<\/span><span>\u00a0<\/span><span>Therefore, using in-class learner engagement as an important keystone of active learning strategies is known to stimulate and enhance the learner\u2019s assimilation of content and concepts<\/span><span>\u00a0<\/span><span>(Armstrong &amp; Fukami, 2009; Watson et al., 1991).<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>There is good evidence for the importance of engagement in online learning and use of an engagement metric has been advocated to better understand student online interactions to improve the online learning environment<\/span><span>\u00a0<\/span><span>(Berman &amp;<\/span><span>\u00a0<\/span><span>Artino,<\/span><span>\u00a0<\/span><span>2018). While medical literature suggests that virtual education games foster engagement<\/span><span>\u00a0<\/span><span>(McCoy et al., 2016), the level of engagement and learning fostered by online methods for group discussion and teaching is unknown. Teleconferencing is among some of the methods suggested for maintaining education during the COVID-19 pandemic (Chick et al., 2020).<\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Possible methods of quantifying student engagement include direct observation and student self-report. O\u2019Malley et al. (2003) has published a validated observation instrument called STROBE to assess in-class learner engagement in health professions without interfering with learner activities. This observation instrument is used to document observed dichotomized types of instructor and student behaviors in 5-minute cycles and quantify the number of questions asked by the instructor and students in different class subtypes.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">This instrument as well as revised forms of this instrument has since been used as \u201cin-class engagement measures\u201d to compare instructor and student behaviors in different class types (Alimoglu<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">et al., 2014; Kelly et al., 2005).<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">In our institution, a hybrid curriculum of case-based learning as well as lecture-style courses is used to teach the<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">post graduate year one<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(PGY-1)<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">interns. We had video recordings of these courses performed in-person prior to the COVID-19 pandemic. With the advent of the pandemic, these courses were shifted onto Zoom teleconferencing platform, but delivered by the same instructors, in the same class format.<\/span><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">We therefore aimed to determine and compare in-class learning engagement levels via observing instructor and student <\/span><\/span><span>behaviours<\/span><span> <span lang=\"EN-US\">in different platforms of learning (either observed online or in-person retrospectively via video recording) delivered by the same instructor before and during the COVID-19 pandemic. We also aimed to compare instructor and student behaviours in different class types (either case-based learning or lecture style instruction). To do this, we planned to modify a known in-person observational tool for student engagement \u2013 \u201cSTROBE\u201d (O\u2019Malley et al., 2003) for use in analysing and recording the behaviours of students in both online and in-person teaching. <\/span><\/span><\/p>\r\n<p style=\"text-align: center;\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">II. METHODS<\/span><\/strong><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">A. Observed Class Types<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">In this study, we observed two different class types, case-based learning<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(CBL), as well as lecture-based instruction to teach basic medical\/surgical topics to a large classroom<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(LLC)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">of PGY-1 interns. Video recordings of these in-person teachings were made in 2017. Both these class types were replicated<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">in the same format<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">on an online Zoom teleconferencing platform and were delivered by<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">nearly<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">all<\/span><\/span><span><span lang=\"EN-US\"> of <\/span><\/span><span><span lang=\"EN-US\">the same tutors<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">using the same content and<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Powerpoint<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">slides<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">during the COVID-19 pandemic in 2020.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">We aimed to view the 2017 video-recordings of the in-person<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">teachings and<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">compare them with the 2020<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">online teaching of PGY-1 interns.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span>Written consent was obtained from the tutors and implied consent from the students. Students were informed beforehand via email that the sessions were going to be observed and they were again reminded at the start of each session where they had the chance to opt out. Subsequently, all student feedback and observation scores were amalgamated and de-identified. <\/span><span><span lang=\"EN-US\">This study was approved by the institution\u2019s ethics board.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Three topics each of case-based learning as well as lecture-style instruction were selected in chronological order as scheduled for students.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Each topic of instruction<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">was allotted up to<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">a maximum of<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">90 minutes of time, but the instructor could<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">choose to end the class earlier if the session<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">was completed.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Description of both class types are below.<\/span><\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">1) Description of case-based learning in large classroom<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">The content of the learning was designed by the instructor, and consisted of clinical cases involving patient scenarios, where the main pedagogy was problem-solving and answering case-based questions relating to the patient scenario<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(e.g.,<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">diagnosis, reading clinical images or electrocardiograms, creating an investigation or treatment plan). Each case would typically take about 15 to 20 minutes to complete, and there would typically be five to six cases. Students were expected to answer the questions, and the instructor gave feedback on the answers and provided additional information,<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">sometimes via additional<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">Powerpoint<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">slides. Class discussions were encouraged where students were encouraged to debate and discuss with each other over their classmates\u2019 answers.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">The titles of the case-based learning were \u201cECG \u2013 tachydysrhythmias\u201d, \u201cApproach to a confused patient\u201d<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">and<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">\u201cApproach to chest pain\u201d.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">2) Description of lecture in large classroom<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">This is a typical lecture-style instruction performed with participation of around 86 PGY1-interns and one instructor. The instructor delivers information via a<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">Powerpoint<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">slide presentation and rarely adds clinical case-based questions<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">into the slides to invite<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">student discussion.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">The titles of the lectures were \u201cCardiovascular health \u2013 hypertensive urgencies\u201d, \u201cTrauma \u2013 chest, abdomen and pelvis\u201d and \u201cStroke\u201d.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">B. Instructor and Student Characteristics<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">The instructors all had at least ten years of teaching experience in medical education, and all had been teaching the same topics to the PGY-1 interns for at least the last five years.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">Student feedback scores on their teaching activities have been<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">satisfactorily high (mean 4.63<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">for 2019, the year prior to the shift to<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">online learning<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">for the pandemic).<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">All<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">the<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">tutors<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(except for one instructor who taught \u201cStroke\u201d)<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">had taught the same topics using the same content and<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">Powerpoint<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">slides in 2017 via in-person teaching which was caught on camera.<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">The students were all PGY-1 interns, who have been asked by the institution to attend at least 70% of a mandatory one-year long teaching program where they are given weekly instruction on various medical or surgical topics. The teaching program commences from May of each year. There were 86 PGY-1 interns commencing their rotations in our institution and attending the teaching program from May 2020.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">There were 75 PGY-1 interns attending the teaching program in the video recordings caught in 2017.<\/span><\/span><span><span lang=\"EN-US\"><\/span><\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">C. Observation Tool<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">A revised form of STROBE<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(O\u2019Malley et al., 2003)<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">was used to analyze and record the behaviors of the instructor and students in classes, to provide a more objective third-person measure of student engagement. The original STROBE tool was an instrument that was developed to objectively measure student engagement across a variety medical education classroom settings. The STROBE instrument consists of 5-minute observational cycles repeated continuously throughout the learning session with relevant observations recorded on a data collection form. Within each cycle, observers record selected aspects of behavior from a list of specified categories that occur in each interval recorded. Observations include macrolevel elements such as structure of class, major activity during time, and a global judgment of the proportion of class members who appear on task, as well as microlevel elements such as instructor\u2019s behavior and the behaviors of four randomly selected students. Observers also record who the behaviours of instructors and students were directed at. After which, observers tally the number of questions asked by the students and instructor in the remainder of the 5 minutes. The revision of this tool was made by the 3<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">Clinician-educators<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">from the research team<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(CYC, YES, KN), having discussed what kind of instructor and student behaviors were considered as \u201cactive student engagement\u201d, keeping the main statements and principles of the original STROBE tool.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">The scale was modified to make it<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">suitable for use in an online learning setting, where the observers may not be able to observe the student\u2019s body language cues when the student does not turn on his\/her video function.<\/span><\/span><span>\u00a0We called this modified scale our \u2018In-class engagement measure\u2019. The modified scales were as follows:<\/span><span lang=\"EN-US\"><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">A 5-item list of instructor and student behaviors was therefore created and rated from 1 to 5 each, with different scales for instructor and student. For the student behavior scale, each item was to show progressively increasing levels of interaction, and perceived engagement, both with the instructor and with each other. For the instructor behavior list, each item was also about progressively interactive behaviors by the instructor<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">to<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">get the students to engage.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">We called these scales our \u201cIn-class Engagement Measure (IEM)\u201d.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">The scales were as follows:<\/span><\/span><span><\/span><\/p>\r\n<p><span>Student:<\/span><span>\u00a0<\/span><\/p>\r\n<ol>\r\n\t<li><span><\/span><span>No response even when asked<\/span><\/li>\r\n\t<li><span><\/span><span>Answers only when directly questioned<\/span><span><\/span><\/li>\r\n\t<li><span><\/span><span>Answers questions spontaneously<\/span><span><\/span><\/li>\r\n\t<li><span><\/span><span>Speaks to instructor spontaneously<\/span><span><\/span><span>g.,<\/span><span><\/span><span>Poses questions, discusses concepts<\/span><span><\/span><\/li>\r\n\t<li><span>Speaks to instructor and 1 or more other student during a discussion<\/span><\/li>\r\n<\/ol>\r\n<p><span><\/span><\/p>\r\n<p><span>Instructor:<\/span><span>\u00a0<\/span><\/p>\r\n<ol>\r\n\t<li><span><\/span><span>Talking to entire class while all the students are passive receivers\u00a0<\/span><span><\/span><\/li>\r\n\t<li><span><\/span><span>Telling\/asking to one or a group of students, or teaching\/showing an application on a student<\/span><\/li>\r\n\t<li><span><\/span><span>Starting or conducting a discussion open to whole class, or assigning some students for some learning tasks<\/span><\/li>\r\n\t<li><span><\/span><span>Listening\/monitoring actively discussing one or a group of students<\/span><\/li>\r\n\t<li><span>Listening\/monitoring actively discussing entire class<\/span><\/li>\r\n<\/ol>\r\n<p><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>For the student<\/span><span>\u00a0<\/span><span>behaviour<\/span><span>\u00a0<\/span><span>list, we also sub-categorized the student<\/span><span>\u00a0<\/span><span>behaviour<\/span><span>\u00a0<\/span><span>item<\/span><span>\u00a0<\/span><span>\u201c1\u201d,<\/span><span>\u00a0<\/span><span>where<\/span><span>\u00a0<\/span><span>\u201c1*\u201d was<\/span><span>\u00a0<\/span><span>defined as no response when a question was posed to a specific student<\/span><span>\u00a0<\/span><span>and not just the whole class, where the student-in-question would have his\/her name called by the tutor.\u00a0<\/span><\/p>\r\n<p><i><\/i><span><i>D. Observation Process<\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>Drawing from the described process for the STROBE observation tool<\/span><span> <\/span><span>(O\u2019Malley et al., 2003)<\/span><span><span lang=\"EN-US\">, as well as other described modifications of the STROBE tool (Alimoglu<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">et al., 2014), we used the same observation units and cycles.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Modifications to the original described process for the STROBE observation tool was made to make it suitable for not being in-person when observing a large group of students and their instructor.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Three observers<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">from the research team<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(CYC, YES, KN)<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">observed and recorded the instructor and student behaviors for the three case-based learning and three lecture-style learning conducted live online in 2020, and as a video recording of in-person teaching in 2017. A total of 12 lectures were therefore analyzed. One observation unit was a 5-minute cycle. The 5-minute cycle would proceed as such: The observer would write the starting time of the cycle and information about the class (number of students, title of session). The observer would select a student from the class and observe that student for 20<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">seconds and<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">mark the type of engagement observed according to the IEM scale created. As the observers were not in-person for the teaching at either the 2017 video recording, and for the 2020 online learning, students who responded to the instructor or posed questions were marked at the same time by all the three observers.<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">The 5-minute cycle would consist of four 20-second observations of individual learners, so marking of student engagement would be performed four times within that cycle with different students in<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">succession. The observer would also observe the instructor for that 5-minute cycle and similarly mark the instructor\u2019s behavior once for that 5-minute cycle. For the remainder of the modified STROBE cycle, the observer would tally the number of questions asked by all the students and the instructor.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Observers independently and separately observed and marked the students\u2019 and instructors\u2019 behaviors. Due to the lack of in-person observation, students who responded or posed questions during the session were uniformly chosen for marking by the three observers. If a student had already been marked once during that cycle, the same student was not used for remaining three observations within the same cycle. At the end of the marking, two observers (KN and YES) compared their<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">scores<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">for both students and instructor.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">The marks given by the third observer (CYC) was used to validate the final score<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">awarded and<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">used as the tiebreaker when<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">there was a discrepancy in the marks given by the first<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">two observers.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p><i><\/i><span><i>E. Collation of Post Teaching Survey Feedback<\/i><\/span><i><\/i><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Apart from the data derived from our modified observational tool, we also reviewed data from surveys conducted by the educational committee after each of these teaching sessions (see Table 1). These were general surveys used to solicit student feedback on the teaching sessions. They were distributed in-person in 2017, with the same forms distributed to the students online in 2020. \u00a0Responses from the students were in response to five statements, with scoring 0 to 5 (1 for Strongly disagree, 2 for disagree, 3 for neither agree nor disagree, 4 for agree, and 5 for Strongly agree). These feedback forms had an overall feedback score marked by the student, as well as a score marked by the student in response to a question assessing for self-reported engagement \u2013 \u201cThe session was interactive and engaging\u201d. The other questions were \u201cThe session has encouraged self-directed learning and critical thinking\u201d, \u201cThe session was relevant to my stage of training\u201d, \u201cThe session helped me advance my clinical decision-making skills\u201d, and \u201cThe session has increased my confidence in day-to-day patient management\u201d.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Means of the feedback scores were taken as a qualitative guide, and we analyzed the overall feedback scores (\u201cOverall feedback score\u201d in Table 1), and the scores in response to the question assessing for self-reported engagement (\u201c<\/span><\/span>Self-reported engagement feedback score\u201d in Table 1)<span><span lang=\"EN-US\">.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">F. Statistical Analyses<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Descriptive statistics were used to determine frequencies and median number of questions asked, as well as mean student feedback scores and absolute duration of each teaching session. Fisher exact test was also performed to analyze the differences in scores between different lectures and case-based learning, and the scores in the 2017 in-person learning versus that of the 2020 online learning. For analysis of the scores, we dichotomized our scores using the cut-off of \u201c1\u201d, or our first item on the behavior list for both students and instructors, as we felt that the first item reflected an extreme non-participation for both student and instructor, which<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">if left to continue, can result in negative learning and teaching behaviors.<\/span><\/span><\/p>\r\n<p style=\"text-align: center;\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">III. RESULTS<\/span><\/strong><\/p>\r\n<p><span><i><\/i><\/span><span><i><span lang=\"EN-US\">A. Class Types, Characteristics, Feedback Scores<\/span><\/i><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">A total of 12 sessions were observed, consisting of in-person and online teaching sessions of six topics (Table 1). There were 3 topics of CBL and LLC each. Duration of the class sessions range from 30-55 minutes for the in-person sessions and 40-90 minutes for the online sessions. Total number of PGY-1 students eligible to attend the in-person teaching sessions in 2017 was 82, and 86 for the online teaching sessions in 2020. Student attendance for the in-person sessions ranged from 11 (13.4%) to 31 (37.8%) and that for the online session ranged from 28 (32.6%) to 77 (89.5%). Median (range) of feedback scores for in-person sessions were 4.57 (4.25 to 4.72) vs 4.32 (4.04 to 4.61) for online sessions. Median (range) of self-reported engagement scores for in-person sessions were 4.55 (4.25 to 4.79) vs 4.34 (4.00 to 4.67) for online sessions (Table 1).<\/span><\/span><\/p>\r\n<p>&nbsp;<\/p>\r\n<p align=\"center\"><span lang=\"EN-GB\">Table 1. Class types and characteristics (*Different tutors, but using same content)<\/span><span lang=\"EN-US\"><\/span><\/p>\r\n<p><i><\/i><span><i><span lang=\"EN-US\">B. Instructors\u2019<\/span><\/i><\/span><span><i><span lang=\"EN-US\"><\/span><\/i><\/span><span><i><span lang=\"EN-US\">Engagement<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><i><span lang=\"EN-US\">Behaviour<\/span><\/i><\/span><span><span lang=\"EN-US\"> <\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">1) Comparing in-person vs online teaching<\/span><\/i><\/span><span>:<\/span><span><i> <span lang=\"EN-US\">Percentage time during which there is no engagement\/interaction<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><i><span lang=\"EN-US\">(or scoring \u201c1\u201d on the IEM score).<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><span lang=\"EN-US\">This ranges from 0-80% for in-person teaching vs 0-100% for online teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table 2A). For each topic, there is no significant difference between percentage time of no engagement.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">Most frequent IEM<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><i><span lang=\"EN-US\">scores.<\/span><\/i><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Most frequent IEM scores<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">for each<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">5-minute<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">segment<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">were<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">3 for in-person teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(48.9%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">and online teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(52.9%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2B).<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">2) Comparing CBL vs LLC: Percentage time during which there is no engagement\/interaction.<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><span lang=\"EN-US\">This ranges from 0-23.1% for CBL vs 50-100% for LLC<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table 2A).<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">Most frequent IEM scores.<\/span><\/i><\/span><span><i><span lang=\"EN-US\">\u00a0<\/span><\/i><\/span><span><span lang=\"EN-US\">Most frequent IEM score was 3 for CBL (77.3%) and 1 for LLC (71.4%).<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2B)<\/span><\/span><span>.<\/span><\/p>\r\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png\" alt=\"\" class=\"wp-image-71841 aligncenter\" width=\"602\" height=\"225\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-1024x382.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-768x286.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A.png 1291w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/p>\r\n<p align=\"center\">Table 2A. Comparison of instructors\u2019\u00a0behaviour\u00a0showing percentage time with no engagement\u00a0(scoring \u201c1\u201d on the IEM score)<\/p>\r\n<p align=\"center\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2B-300x104.png\" alt=\"\" class=\"alignnone  wp-image-71842\" width=\"588\" height=\"204\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2B-300x104.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2B-1024x356.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2B-768x267.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2B.png 1070w\" sizes=\"auto, (max-width: 588px) 100vw, 588px\" \/><\/p>\r\n<p align=\"center\">Table 2B. Numbers (percentages) of a particular IEM score received for a 5-minutes segment of teaching &#8211; for instructors<\/p>\r\n<div>\r\n<p style=\"text-align: justify;\"><span><i>C. <\/i><\/span><span><i><span lang=\"EN-US\">Students\u2019 Engagement Behaviour<\/span><\/i><\/span><span>\u00a0<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">1) Comparing in-person vs online teaching: Percentage time during which there is no engagement\/interaction.<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><span lang=\"EN-US\">This ranges from 0-95% for in-person teaching vs 78.8-100% for online teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table 3A). There is<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">significant difference in percentage time of no engagement in two topics (ECG, chest pain), where there is higher percentage of no engagement time with online teaching.<\/span><\/span><span>\u00a0<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">Most frequent IEM scores.<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><span lang=\"EN-US\">Most frequent IEM scores were 1 for both in-person teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(63.8%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">and online teaching<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(85.1%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">3B).<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">2) Comparing CBL vs LLC:<\/span><\/i><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><i><span lang=\"EN-US\">Percentage time during which there is no engagement\/interaction.<\/span><\/i><\/span><span><i><span lang=\"EN-US\">\u00a0<\/span><\/i><\/span><span><span lang=\"EN-US\">This ranges from 0-81.9% for CBL vs<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">84.4-100% for LLC<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(Table 3A).<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><i><span lang=\"EN-US\">Most frequent IEM scores.<\/span><\/i><\/span><span><i><span lang=\"EN-US\">\u00a0<\/span><\/i><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Most frequent IEM scores were 1 for both CBL (65.3%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">and LLC <\/span><\/span><span><span lang=\"EN-US\">(<\/span><\/span><span><span lang=\"EN-US\">91.8%)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">3B)<\/span><\/span><span>.<\/span><\/p>\r\n<p><span><i><span lang=\"EN-US\">Presence of 1* scores, where<\/span><\/i><\/span><span><i><span lang=\"EN-US\"> <\/span><\/i><\/span><span><i>\u201c1*\u201d was defined as no response when a question was posed to a specific student called by name.<\/i><\/span><span><i> <\/i><\/span><span><span lang=\"EN-US\">There was no 1* IEM score for in-person teaching for either CBL or LLC, and 8.4% (12\/143) of the \u201c1\u201d responses were 1* for online-teaching for CBL and 6.5% (6\/92) of the \u201c1\u201d responses were 1* for LLC.<\/span><\/span><span><\/span><\/p>\r\n<\/div>\r\n<p align=\"center\" style=\"text-align: left;\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3A-300x111.png\" alt=\"\" class=\" wp-image-71843 aligncenter\" width=\"524\" height=\"194\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3A-300x111.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3A-1024x380.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3A-768x285.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3A.png 1290w\" sizes=\"auto, (max-width: 524px) 100vw, 524px\" \/><\/p>\r\n<p align=\"center\">Table 3A. Comparison of students\u2019 behaviour showing percentage time with no engagement (scoring \u201c1\u201d on the IEM score)<\/p>\r\n<p align=\"center\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3B-300x104.png\" alt=\"\" class=\"alignnone  wp-image-71844\" width=\"516\" height=\"179\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3B-300x104.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3B-1024x356.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3B-768x267.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-3B.png 1065w\" sizes=\"auto, (max-width: 516px) 100vw, 516px\" \/><\/p>\r\n<p align=\"center\">Table 3B. Numbers (percentages) of a particular IEM score received for a 5-minutes segment of teaching &#8211; for students<\/p>\r\n<p><span><i><\/i><\/span><span><i><span lang=\"EN-US\">D. Number of Questions Asked Per 5-minute Cycle<\/span><\/i><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Median number of questions asked by instructors ranged from 0-2 for in-person teaching and 1-3 for online teaching (See Appendix 1). These range from 1-3 for CBL vs 0-1 for LLC. <\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Median number of questions asked by students in all sessions were 0.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\">The results for this study can be derived from the dataset uploaded onto the online repository accessed via https:\/\/doi.org\/10.6084\/m9.figshare.18133379.v1 (Chua et al., 2022).<\/p>\r\n<p style=\"text-align: center;\"><span><span lang=\"EN-GB\"><\/span><\/span><span lang=\"EN-GB\"><strong>IV. DISCUSSION<\/strong><span><\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>We modified<\/span><span>\u00a0<\/span><span>the known STROBE instrument (O\u2019Malley et al., 2003) to create an<\/span><span> <\/span><span>observational tool \u201cIEM\u201d<\/span><span>\u00a0which could be used <\/span><span>to quantify instructor and student engagement<\/span><span> <\/span><span>despite the observer not being present in-person.<\/span><span> Our IEM scores were derived by taking scores that were in agreement when independently scored by two main observers (YES and KN). The third observer (CYC) was used as the validator of the scores by the two main observers. <\/span><span><span lang=\"EN-US\">When there was a discrepancy in the scores awarded by the two observers, the score which<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">was in agreement<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">with the score awarded by CYC was used. To give an indication of the IEM tool\u2019s effectiveness where the observer is not present in-person, we postulated that our modified IEM score should still demonstrate the well-documented difference in engagement between lecture-style learning and case-based learning sessions<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Kelly et al., 2005). Our modified IEM score did indeed show more frequent higher scores as expected for case-based learning sessions<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Tables<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2B<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">and 3B).<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">We also compared our IEM scores with the students\u2019 self-reported engagement scores (Table 1) that had been collected as part of student feedback. The general correlation in the trend of observed IEM scores with that of the students\u2019 self-reported engagement scores also suggest the usefulness of our modified STROBE tool in situations where the observer is not present in-person, although this needs to be further validated in prospective studies.<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>Our initial study hypothesis was that students may find themselves more engaged<\/span><span> <\/span><span>in online teaching sessions<\/span><span>\u00a0<\/span><span>and open to posing questions to the instructor and their peers, due to the presence of the \u201cchat\u201d, \u201clikes\u201d and \u201cpoll\u201d functions available on the Zoom tele-conferencing platform, which may be more familiar to a younger generation accustomed to using social media. We<\/span><span>\u00a0<\/span><span>had postulated that live online lectures would encourage further engagement from students who would not otherwise participate in-person, due to the less intimidating online environment where they can ask and answer questions more anonymously<\/span><span> <\/span><span>(Kay<\/span><span> <\/span><span>&amp;<\/span><span> <\/span><span>Pasarica,<\/span><span> <\/span><span>2019; Ni, 2013).<\/span><span> In an Asian-pacific context, video conferencing had been found to be able to improve access for participation for more reticent participants who prefer written expression, through alternative communication channels like the &#8216;chat box&#8217;, although there was a potential trend to reduced engagement. (Ong et al., 2021).<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span>Our data, shows, that Zoom teleconferencing<\/span><span>\u00a0<\/span><span>during the COVID-19 pandemic can be associated with<\/span><span> <\/span><span>reduced<\/span><span> <\/span><span>student engagement.<\/span><span> <\/span><span>The percentage time where there was no engagement was significantly higher with online sessions (Table 3A) and the<\/span><span> <\/span><span>most frequent IEM score was lower<\/span><span> <\/span><span>(1 for online vs 3 for in-person),<\/span><span> <\/span><span>for CBL sessions<\/span><span> <\/span><span>(Table 3B).<\/span><span> <\/span><span>This phenomenon<\/span><span> <\/span><span>in medical education during the COVID-19 pandemic<\/span><span> <\/span><span>has previously been described. Using student and instructor feedback, students were more likely to have reduced engagement during virtual learning<\/span><span> <\/span><span>(Longhurst et al., 2020; Dost et al., 2020), and would have increased difficulties maintain focus,<\/span><span> <\/span><span>concentration<\/span><span> <\/span><span>and motivation during online learning <\/span><span>(<\/span><span>Wilcha,<\/span><span> <\/span><span>2020).<\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">Our data also suggests that for the instructor to even try to achieve close to the same levels of engagement as before, a longer duration of time<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">was spent by each instructor per topic when executing CBL<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Table 1).<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">This may include time where the instructor needs multiple attempts at questioning and discussion before there is a student response. It is also possible that for in-person learning, the instructor relies greatly also on non-verbal cues (e.g.,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">body language, nods of the head, collective feel of the room) to determine if a question has been satisfactorily answered, and therefore can move on quicker than when on a Zoom platform where one cannot see most, or even every student.<\/span><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">The higher number of attendees for online learning compared to<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">in-person<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">attendance<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(see Table 1)<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">highlights<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">one of the strengths of online learning, which is where online learning is more easily accessible<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">for students who would save on time getting to a designated lecture<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">room and<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">provides flexibility for students to enter and exit<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Dost et al., 2020). Unfortunately, this also likely encourages the phenomenon of \u201cpresenteeism\u201d, where students are not focused on the learning session, but instead engage<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">in other tasks simultaneously,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">e.g.,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">reading or composing emails, or completing work tasks instead of having dedicated protected teaching time. Resident learners have been described to<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">participate in nearly twice as many non-teaching session related activities per hour during an online session than when in-person<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Weber<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">&amp;<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Ahn,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2020). This has likely contributed to the number of 1* scores we had, where the student has logged into the Zoom platform, but is not available to even respond in the negative when called upon to answer a question. This presenteeism, however, is not just a problem for online learning, but even for in-person learning, where pretending to engage has been found to be a significant unrecognized issue<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(Fuller et al.,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2018).<\/span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">The main implication that our study highlights that to improve student engagement when using online learning, a face-to-face platform cannot simply be transposed into a virtual platform. It had been suggested that engagement during live virtual learning could be enhanced with the use of interactive quizzes with audience polling functions<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Morawo<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">et al., 2020) and possibly other methods such as \u201cgamification\u201d (Nieto-Escamez &amp; Roldan-Tapia, 2021). Our instructors for the CBL sessions had used both poll functions and live questioning for their sessions, but without increased success in engagement. Smaller groups are likely required<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">to enhance student engagement, but this would lead to the need for increased time and teaching manpower. Increasing the opportunity for interaction via a virtual platform would also require the need to create additional online resources, which would take up more faculty time where creating new resources can take at least three times as much work compared to a traditional format<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Gewin,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2020). Online resources would need to be modified in such a way that increases student autonomy to increase student engagement in medical education<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Kay<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">&amp;<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Pasarica,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2019).<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span>Our study also shows that as a first step, in time and resource-limited settings, a case-based approach to teaching would be more ideal to enhance student engagement than lecture style teaching.<\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">A culture of accountability also needs to be fostered within the online teaching sessions, where students need to be educated on how Zoom meetings can be more enriching when cameras are on<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Sharp et al., 2021). PGY-1 interns, as recent graduates, also need to be educated<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">on the aspect of professionalism<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">when entering<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">the<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">medical<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">work force, where they can be called upon to answer questions<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">during meetings or conferences.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">When initial<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">questions are not voluntarily answered, our tutors often practice \u201ccold-calling\u201d, which can help keep learners alert and ready<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">(Lemov,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">2015).<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">Unfortunately, these evidence-based teaching methods that work well when the student is in-person,<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">ultimately will fail if online students are not educated on their need to be accountable to the instructor<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">or their peers.<\/span><\/span><span><\/span><\/p>\r\n<p style=\"text-align: justify;\"><span><span lang=\"EN-US\">This study has several limitations. Firstly, the level of student engagement may also be affected by external factors, such as a different physical learning environment, class size and avenues of communication. The stresses of the on-going pandemic may also have affected student engagement, as a decrease in quality of life and stress would negatively impact student motivation<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">(Lyndon et al.,<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">2017). Secondly, the topics for lecture to large class and case-based learning were not identical as these topics were picked in chronological order and there were no topics in the curriculum that had material for both the lecture and case-based learning class types. This difference in topics may have potentially contributed to confounding when we try to make direct comparisons between the two class types, although, we have attempted to mitigate this by including a variety of topics in each class type. Thirdly,<\/span><\/span><span><span lang=\"EN-US\">\u00a0<\/span><\/span><span><span lang=\"EN-US\">the improved student engagement and feedback scores for in-person learning may also have had some bias given the smaller student size for in-person learning. It is also possible that only the more motivated, and hence more likely to be engaged students, would turn up for in-person learning. Fourthly, due to the online nature as well as the retrospective viewing of the video recordings, the observers were not present in-person to observe the non-verbal cues of the students or instructors. The tool, however, was modified to<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">take into account<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span><span lang=\"EN-US\">only the verbal output that could be observed online or via video recording.<\/span><\/span><span><span lang=\"EN-US\"> <\/span><\/span><span>Lastly, our IEM tool will benefit from more studies and research to further confirm its validity in observing students when the observer is not present in-person. <\/span><\/p>\r\n<p style=\"text-align: center;\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">V. CONCLUSION<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">Lectures are associated with reduced student engagement than case-based learning, while both class types are associated with lower levels of student engagement when conducted on a virtual platform. Instructor levels of engagement, however, remain about the same. This highlights that <span><span lang=\"EN-US\">a face-to-face platform cannot simply be transposed into a virtual platform,<\/span><\/span> and it is important to address this gap in engagement as this can lower faculty satisfaction with teaching and ultimately result in burnout. Blended teaching or smaller group teaching as the world turns the corner in the COVID-19 pandemic may be one way to circumvent the situation but is also constrained by faculty time and manpower. <span>Our study also shows that as a first step, in time and resource-limited settings, a case-based approach to teaching would be more ideal to enhance student engagement than lecture style teaching.<\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Notes on Contributors<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">Dr Ng Wei Ping Kay and Dr Chua Yuan Kit Christopher are co-first authors and contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. They contributed to drafting and revising the work and approved the final version to be published. They agree to be accountable for all aspects of the work.<\/p>\r\n<p style=\"text-align: justify;\">Dr Lye Pei Shi Priscillia contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. She contributed to drafting and revising the work and approved the final version to be published. She agrees to be accountable for all aspects of the work.<\/p>\r\n<p style=\"text-align: justify;\">Dr Joy Vijayan contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.<\/p>\r\n<p style=\"text-align: justify;\">Dr Yap Eng Soo contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.<\/p>\r\n<p style=\"text-align: justify;\">Dr Chan Yee Cheun contributed to conceptual development, acquisition, analysis, and interpretation of data for the work. He contributed to drafting and revising the work and approved the final version to be published. He agrees to be accountable for all aspects of the work.<span lang=\"EN-GB\"><\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Ethical Approval<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">I confirm that the study has been approved by Domain Specific Review Board (DSRB), National Healthcare Group, Singapore, an institutional ethics committee. DSRB reference number: 2020\/00415.<\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Data Availability<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">The data that support the findings of this study are openly available in Figshare at <a href=\"https:\/\/doi.org\/10.6084\/m9.fig%20share.18133379.v1\">https:\/\/doi.org\/10.6084\/m9.fig share.18133379.v1<\/a>.<\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Acknowledgement<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">We would like to acknowledge Ms. Jacqueline Lam for her administrative support in observing the recordings and online-teaching.\u00a0<\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Funding<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\"><span lang=\"EN-GB\">There was no funding for this research study.<\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Declaration of Interest<\/span><\/strong><\/p>\r\n<p><span lang=\"EN-GB\">The authors report no conflicts of interest, including financial, consultant, institutional and other relationships that might lead to bias or a conflict of interest.<\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">References<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify;\">Alimoglu, M. K., Sarac, D. B., Alparslan, D., Karakas, A. A., &amp; Altintas. (2014). An observation tool for instructor and student behaviors to measure in-class learner engagement: A validation study. <i>Medical Education Online<\/i>, <i>19<\/i>(1), 24037. <a href=\"https:\/\/doi.org\/10.3402\/meo.v19.24037\">https:\/\/doi.org\/10.3402\/meo.v19.24037<\/a><\/p>\r\n<p style=\"text-align: justify;\">Armstrong, S. J., &amp; Fukami, C. V. (2009). <i>The SAGE Handbook of Management Learning, Education and Development<\/i>. SAGE Publications Ltd. <a href=\"https:\/\/www.doi.org\/10.4135\/9780857021038\">https:\/\/www.doi.org\/10.4135\/9780857021038<\/a><\/p>\r\n<p style=\"text-align: justify;\">Berman, N. B., &amp; Artino, A. R. J., (2018). Development and initial validation of an online engagement metric using virtual patients. <i>BMC Medical Education<\/i>, <i>18<\/i>(1), 213. <a href=\"https:\/\/doi.org\/10.1186\/s12909-018-1322-z\">https:\/\/doi.org\/10.1186\/s12909-018-1322-z<\/a><\/p>\r\n<p style=\"text-align: justify;\">Chick, R. C., Clifton, G. T., Peace, K. M., Propper, B. W., Hale, D. F., Alseidi, A. A., &amp; Vreeland, T. J. (2020). Using technology to maintain the education of residents during the COVID-19 Pandemic. <i>Journal of Surgical Education<\/i>, <i>77<\/i>(4), 729\u2013732. <a href=\"https:\/\/doi.org\/10.1016\/j.jsurg.2020.03.018\">https:\/\/doi.org\/10.1016\/j.jsurg.2020.03.018<\/a><\/p>\r\n<p style=\"text-align: justify;\">Chua, Y. K. C., Ng, K. W. P., Yap, E. S., Lye, P. S. P., Vijayan, J., &amp; Chan, Y. C. (2022). <i>Evaluating online learning engagement<\/i> (Version 1) [Data set]. Figshare. <a href=\"https:\/\/doi.org\/10.6084\/m9.figshare.18133379.v1\">https:\/\/doi.org\/10.6084\/m9.figshare.18133379.v1<\/a><\/p>\r\n<p style=\"text-align: justify;\">Dost, S., Hossain, A., Shehab, M., Abdelwahed, A., &amp; Al-Nusair, L. (2020). Perceptions of medical students towards online teaching during the COVID-19 pandemic: A national cross-sectional survey of 2721 UK medical students. <i>BMJ Open<\/i>, <i>10<\/i>(11), e42378. <a href=\"https:\/\/doi.org\/10.1136\/bmjopen-2020-042378\">https:\/\/doi.org\/10.1136\/bmjopen-2020-042378<\/a><\/p>\r\n<p style=\"text-align: justify;\">Fuller, K. A., Karunaratne, N. S., Naidu, S., Exintaris, B., Short, J. L., Wolcott, M. D., Singleton, S., &amp; White, P. J. (2018).<span> <\/span>Development of a self-report instrument for measuring in-class student engagement reveals that pretending to engage is a significant unrecognized problem. <i>PLOS ONE<\/i>, <i>13<\/i>(10), e0205828.<span> <\/span><a href=\"https:\/\/doi.org\/10.1371\/journal.pone.0205828\">https:\/\/doi.org\/10.1371\/journal.pone.0205828<\/a><\/p>\r\n<p style=\"text-align: justify;\">Gewin, V. (2020). Five tips for moving teaching online as COVID-19 takes hold. <i>Nature<\/i>, <i>580<\/i>(7802), 295\u2013296. <a href=\"https:\/\/doi.org\/10.1038\/d41586-020-00896-7\">https:\/\/doi.org\/10.1038\/d41586-020-00896-7<\/a><\/p>\r\n<p style=\"text-align: justify;\">Kaufman, D. M. (2003). Applying educational theory in practice. <i>BMJ, 326<\/i>(7382), 213\u2013216. <a href=\"https:\/\/doi.org\/10.1136\/bmj.326.7382.213\">https:\/\/doi.org\/10.1136\/bmj.326.7382.213<\/a><\/p>\r\n<p style=\"text-align: justify;\">Kay, D., &amp; Pasarica, M. (2019). Using technology to increase student (and faculty satisfaction with) engagement in medical education. <i>Advances in Physiology Education<\/i>, <i>43<\/i>(3), 408\u2013413. <a href=\"https:\/\/doi.org\/10.1152\/advan.00033.2019\">https:\/\/doi.org\/10.1152\/advan.00033.2019<\/a><\/p>\r\n<p style=\"text-align: justify;\">Kelly, P. A., Haidet, P., Schneider, V., Searle, N., Seidel, C. L., &amp; Richards, B. F. (2005). A comparison of in-class learner engagement across lecture, problem-based learning, and team learning using the STROBE classroom observation tool. <i>Teaching and Learning in Medicine<\/i>, <i>17<\/i>(2), 112\u2013118. <a href=\"https:\/\/doi.org\/10.1207\/s15328015tlm1702_4\">https:\/\/doi.org\/10.1207\/s15328015tlm1702_4<\/a><\/p>\r\n<p style=\"text-align: justify;\">Lemov, D. (2015). <i>Teach like a champion 2.0: 62 techniques that put students on the path to college.<\/i> (2nd ed.). Jossey-Bass.<\/p>\r\n<p style=\"text-align: justify;\">Longhurst, G. J., Stone, D. M., Dulohery, K., Scully, D., Campbell, T., &amp; Smith, C. F. (2020). Strength, weakness, opportunity, threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and Republic of Ireland in response to the Covid-19 pandemic. <i>Anatomical Sciences Education<\/i>, <i>13<\/i>(3), 301\u2013311. <a href=\"https:\/\/doi.org\/10.1002\/ase.1967\">https:\/\/doi.org\/10.1002\/ase.1967<\/a><\/p>\r\n<p style=\"text-align: justify;\">Lyndon, M. P., Henning, M. A., Alyami, H., Krishna, S., Zeng, I., Yu, T.-C., &amp; Hill, A. G. (2017). Burnout, quality of life, motivation, and academic achievement among medical students: A person-oriented approach. <i>Perspectives on Medical Education<\/i>, <i>6<\/i>(2), 108\u2013114. <a href=\"https:\/\/doi.org\/10.1007\/s40037-017-0340-6\">https:\/\/doi.org\/10.1007\/s40037-017-0340-6<\/a><\/p>\r\n<p style=\"text-align: justify;\">McCoy, L., Pettit, R. K., Lewis, J. H., Allgood, J. A., Bay, C., &amp; Schwartz, F. N. (2016). Evaluating medical student engagement during virtual patient simulations: A sequential, mixed methods study. <i>BMC Medical Education<\/i>, <i>16<\/i>, 20. <a href=\"https:\/\/doi.org\/10.1186\/s12909-016-0530-7\">https:\/\/doi.org\/10.1186\/s12909-016-0530-7<\/a><\/p>\r\n<p style=\"text-align: justify;\">Morawo, A., Sun, C., &amp; Lowden, M. (2020). Enhancing engagement during live virtual learning using interactive quizzes. <i>Medical Education<\/i>, <i>54<\/i>(12), 1188. <a href=\"https:\/\/doi.org\/10.1111\/medu.14253\">https:\/\/doi.org\/10.1111\/medu.14253<\/a><\/p>\r\n<p style=\"text-align: justify;\"><span>Ni, A. Y.<\/span><span> <\/span><span>(2013).<\/span><span> <\/span><span>Comparing the effectiveness of classroom and online learning: Teaching research methods. <\/span><span><i>Journal of Public Affairs Education<\/i><\/span><span>,<\/span><span> <\/span><span><i>19<\/i><\/span><span>(2),<\/span><span> <\/span><span>199-215. <\/span><a href=\"https:\/\/doi.org\/10.1080\/15236803.2013.12001730\">https:\/\/doi.org\/10.1080\/15236803.2013.12001730<\/a><\/p>\r\n<p style=\"text-align: justify;\">Nieto-Escamez, F. A., &amp; Roldan-Tapia, M. D. (2021). Gamifica- tion as online teaching strategy during COVID-19: A mini-review. <i>Frontiers in Psychology<\/i>, <i>12<\/i>, 648522. <a href=\"https:\/\/doi.org\/10.3389\/fpsyg.2021.648552\">https:\/\/doi.org\/10.3389\/fpsyg.2021.648552<\/a><\/p>\r\n<p style=\"text-align: justify;\">O&#8217;Malley, K. J., Moran, B. J., Haidet, P., Seidel, C. L., Schneider, V., Morgan, R. O., Kelly, P. A., &amp; Richards, B. (2003). Validation of an observation instrument for measuring student engagement in health professions settings. <i>Evaluation &amp; the Health Professions<\/i>, <i>26<\/i>(1), 86\u2013103. <a href=\"https:\/\/doi.org\/10.1177\/0163278702250093\">https:\/\/doi.org\/10.1177\/0163278702250093<\/a><\/p>\r\n<p style=\"text-align: justify;\"><span>Ong, C. C. P., Choo, C. S. C., Tan, N. C. K., &amp; Ong, L. Y. (2021). Unanticipated learning effects in videoconference continuous professional development. <i>The Asia Pacific Scholar<\/i>, <i>6<\/i>(4), 135-141. <\/span><a href=\"https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/SC2484\">https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/SC2484<\/a><span><\/span><\/p>\r\n<p style=\"text-align: justify;\">Rao, S. P., &amp; DiCarlo, S. E. (2001). Active learning of respiratory physiology improves performance on respiratory physiology examinations. <i>Advances in Physiology E<\/i><i>ducation<\/i>, <i>25<\/i>(2), 55\u201361. <a href=\"https:\/\/doi.org\/10.1152\/advances.2001.25.2.55\">https:\/\/doi.org\/10.1152\/advances.2001.25.2.55<\/a><\/p>\r\n<p style=\"text-align: justify;\">Sharp, E. A., Norman, M. K., Spagnoletti, C. L., &amp; Miller, B. G. (2021). Optimizing synchronous online teaching sessions: A guide to the &#8220;new normal&#8221; in medical education. <i>Academic Pediatrics<\/i>, <i>21<\/i>(1), 11\u201315. <a href=\"https:\/\/doi.org\/10.1016\/j.acap.2020.11.009\">https:\/\/doi.org\/10.1016\/j.acap.2020.11.009<\/a><\/p>\r\n<p style=\"text-align: justify;\">Watson, W. E., Michaelsen, L. K., &amp; Sharp, W. (1991). Member competence, group interaction, and group decision making: A longitudinal study.<span> <\/span><em>Journal of Applied Psychology, 76<\/em>(6), 803\u2013809.<span> <\/span><a href=\"https:\/\/doi.org\/10.1037\/0021-9010.76.6.803\">https:\/\/doi.org\/10.1037\/0021-9010.76.6.803<\/a>\u00a0<\/p>\r\n<p style=\"text-align: justify;\">Weber, W., &amp; Ahn, J. (2020). COVID-19 conferences: Resident perceptions of online synchronous learning environments. W<i>estern Journal of Emergency Medicine<\/i>, <i>22<\/i>(1), 115\u2013118. <a href=\"https:\/\/doi.org\/10.5811\/westjem.2020.11.49125\">https:\/\/doi.org\/10.5811\/westjem.2020.11.49125<\/a><\/p>\r\n<p style=\"text-align: justify;\">Wilcha, R. J. (2020). Effectiveness of virtual medical teaching during the COVID-19 crisis: Systematic review. <i>JMIR Medical Education<\/i>, <i>6<\/i>(2), e20963. <a href=\"https:\/\/doi.org\/10.2196\/20963\">https:\/\/doi.org\/10.2196\/20963<\/a><\/p>\r\n<p><span lang=\"EN-US\"><span lang=\"EN-NZ\"><span lang=\"EN-AU\">*<span lang=\"EN-GB\">Chua Yuan Kit Christopher<\/span><br \/>\r\n<span lang=\"EN-GB\">5 Lower Kent Ridge Road,<\/span><br \/>\r\n<\/span><span lang=\"EN-AU\"><span lang=\"EN-GB\">National University Hospital,<\/span><br \/>\r\n<span lang=\"EN-GB\">Singapore 119074<\/span><br \/>\r\n<\/span><\/span><\/span><span lang=\"EN-US\"><span lang=\"EN-NZ\"><span lang=\"EN-AU\"><span lang=\"EN-GB\">+65 7795555<\/span><br \/>\r\n<span lang=\"EN-GB\">Email: christopher_chua@nuhs.edu.sg<\/span><\/span><br \/>\r\n<\/span><\/span><\/p>","protected":false},"featured_media":0,"parent":0,"menu_order":3,"template":"","issues_category":[12],"archive_category":[],"issue_type":[25],"volume_category":[61873],"class_list":["post-71839","issues","type-issues","status-publish","hentry","issues_category-original-articles","issue_type-past-issue","volume_category-volume-7-number-4-october-2022"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar\" \/>\n<meta property=\"og:description\" content=\"Submitted: 13 January 2022 Accepted: 9 May 2022 Published online: 4 October, TAPS 2022, 7(4), 35-49 https:\/\/doi.org\/10.29060\/TAPS.2022-7-4\/OA2699 Yuan Kit Christopher Chua1*, Kay Wei Ping Ng1*, Eng Soo Yap2,3, Pei Shi Priscillia Lye4, Joy Vijayan1, &amp; Yee Cheun Chan1 1Department of Medicine, Division of Neurology, National University Hospital Singapore, Singapore; 2Department of\u00a0Haematology-oncology, National University Cancer Institute [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/\" \/>\n<meta property=\"og:site_name\" content=\"The Asia Pacific Scholar\" \/>\n<meta property=\"article:modified_time\" content=\"2023-01-03T01:04:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"29 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/\",\"name\":\"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar\",\"isPartOf\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png\",\"datePublished\":\"2022-09-26T00:57:16+00:00\",\"dateModified\":\"2023-01-03T01:04:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A.png\",\"contentUrl\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A.png\",\"width\":1291,\"height\":481},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/medicine.nus.edu.sg\/taps\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/\",\"name\":\"The Asia Pacific Scholar\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/","og_locale":"en_US","og_type":"article","og_title":"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar","og_description":"Submitted: 13 January 2022 Accepted: 9 May 2022 Published online: 4 October, TAPS 2022, 7(4), 35-49 https:\/\/doi.org\/10.29060\/TAPS.2022-7-4\/OA2699 Yuan Kit Christopher Chua1*, Kay Wei Ping Ng1*, Eng Soo Yap2,3, Pei Shi Priscillia Lye4, Joy Vijayan1, &amp; Yee Cheun Chan1 1Department of Medicine, Division of Neurology, National University Hospital Singapore, Singapore; 2Department of\u00a0Haematology-oncology, National University Cancer Institute [&hellip;]","og_url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/","og_site_name":"The Asia Pacific Scholar","article_modified_time":"2023-01-03T01:04:06+00:00","og_image":[{"url":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"29 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/","url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/","name":"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool - The Asia Pacific Scholar","isPartOf":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website"},"primaryImageOfPage":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage"},"image":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage"},"thumbnailUrl":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A-300x112.png","datePublished":"2022-09-26T00:57:16+00:00","dateModified":"2023-01-03T01:04:06+00:00","breadcrumb":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#primaryimage","url":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A.png","contentUrl":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2022\/09\/OA2699-Table-2A.png","width":1291,"height":481},{"@type":"BreadcrumbList","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/comparison-of-in-class-learning-engagement-of-pgy-1-interns-during-online-and-in-person-teachings-using-a-modified-classroom-observational-tool\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/medicine.nus.edu.sg\/taps\/"},{"@type":"ListItem","position":2,"name":"Comparison of in-class learning engagement of PGY-1 interns during online and in-person teachings using a modified classroom observational tool"}]},{"@type":"WebSite","@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website","url":"https:\/\/medicine.nus.edu.sg\/taps\/","name":"The Asia Pacific Scholar","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues\/71839","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues"}],"about":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/types\/issues"}],"wp:attachment":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/media?parent=71839"}],"wp:term":[{"taxonomy":"issues_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues_category?post=71839"},{"taxonomy":"archive_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/archive_category?post=71839"},{"taxonomy":"issue_type","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issue_type?post=71839"},{"taxonomy":"volume_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/volume_category?post=71839"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}