Learning experiences of pre-clinical medical students in virtual problem-based learning amidst the COVID-19 pandemic

  • Share
Print Article   Download PDF

Number of Citations:

Submitted: 19 February 2021
Accepted: 17 June 2021
Published online: 4 January, TAPS 2022, 7(1), 33-43
https://doi.org/10.29060/TAPS.2022-7-1/OA2493

Chan Choong Foong*, An Jie Lye*, Che Rafidah Aziz, Wei-Han Hong, Vinod Pallath, Jessica Grace Cockburn, Siti Nurjawahir Rosli, Kuhan Krishnan, Prahaladhan Sivalingam, Noor Filzati Zulkepli & Jamuna Vadivelu

Medical Education & Research Development Unit (MERDU), Faculty of Medicine, University of Malaya, Malaysia

*Joint first authors

Abstract

Introduction: Medical schools universally responded by migrating teaching and learning to virtual learning environments (VLE) due to the impact of the COVID-19 pandemic. The use of virtual problem-based learning (PBL) in lieu of face-to-face sessions seems to be an appropriate response, but its effectiveness was understudied. The study compared the learning experiences of pre-clinical medical students at the beginning and completion of the virtual PBL.

Methods: The study was conducted at the University of Malaya, a public-funded university in Malaysia. A 12-item questionnaire was developed and validated to assess the learning experiences of students conducting virtual PBL sessions. Principal component analysis and test for internal consistency suggested that the questionnaire is valid and reliable. The questionnaire was administered to pre-clinical students (Year 1 and Year 2) twice: at the beginning and the end of the virtual PBL implementation. Their responses were compared for the domains “learning”, “confidence” and “concern”.

Results: Three hundred and forty-four pre-clinical students were recruited but only 275 students (80%) responded to both the initial and final questionnaires. Based on the responses, the learning experiences of students generally improved by the completion of the virtual PBL implementation. Students were most convinced that they obtained and understood the information given during the virtual PBL. However, they continued to be worried about passing the clinical examination and content acquisition.

Conclusion: The study supports the feasibility of virtual PBL as an acceptable alternative to replace face-to-face PBL during the COVID-19 pandemic.

Keywords:           Problem-based Learning, COVID-19, Undergraduate Medical Education, Virtual Learning Environment

Practice Highlights

  • The study compared the learning experiences of pre-clinical medical students in virtual Problem Based Learning (PBL).
  • The learning experiences of students in virtual PBL generally improved.
  • Students obtained and understood the information given during the virtual PBL.
  • Students continued to be worried about passing the clinical examination and content acquisition.
  • Virtual PBL could be an acceptable alternative to replace face-to-face PBL during the pandemic.

I. INTRODUCTION

The novel coronavirus disease, COVID-19, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and characterised by pneumonia-like symptoms, was identified on January 7, 2020. The World Health Organization declared a pandemic on March 11, 2020, due to the rapid global increase in reported cases (World Health Organization, 2020). The first COVID-19 positive case in Malaysia was reported on January 25 and the first fatal case on March 17 of 2020 (Shah et al., 2020). A Movement Control Order (MCO) was issued by the Malaysian Government on March 18 to reduce the spread of COVID-19 in Malaysia (Shah et al., 2020). As mandated by the MCO, schools, and universities were closed to reduce the transmission of COVID-19.

The order to close the universities greatly affected the undergraduate medical program, particularly because students spend a large portion of time in physical classrooms, medical clinics, and face-to-face discussions. The University of Malaya was not isolated in this challenge, as the outbreak of COVID-19 affected medical schools worldwide (Ferrel & Ryan, 2020; Khalil et al., 2020; Reyna, 2020; Yusoff et al., 2020). Medical schools universally responded by migrating teaching and learning to virtual learning environments (VLE) to maintain planned 2019-2020 academic calendars as much as possible (Alkhowailed et al., 2020; Khalil et al., 2020). Most of the theoretical teaching and learning activities, such as lectures, seminars, and small group problem-based learning (PBL), were transferred to digital platforms to accommodate social distancing practices amidst the COVID-19 pandemic (Abbas, 2020; Alkhowailed et al., 2020). Online learning has increasingly complemented traditional face-to-face teaching in medical education in recent years (Ifediora, 2019), but the COVID-19 pandemic has expedited this transition in only months (Abedi & Abedi, 2020; Alkhowailed et al., 2020; Ebner et al., 2020; Tabatabai, 2020). Digitalisation has made the studies available for students regardless of their geographical locations and local circumstances (Laamanen et al., 2021).

It was a necessary assumption by medical education administrators that virtual PBL sessions would be an acceptable alternative to the planned face-to-face PBL sessions during the course of the COVID-19 pandemic. This included student participation from external locations using online platforms, such as Microsoft Teams and Zoom (Coiado et al., 2020), or customized information technology packages (Musal et al., 2017). This strategy ensured physical distancing practices were followed, as students remained isolated, while they were still able to actively participate in PBL discussions. Despite the change to a VLE, tutors and students were asked to adapt to the virtual environment to ensure that students could still practice communication, clinical reasoning, and presentation skills (Joseph et al., 2016). Additionally, course outcomes and student expectations, including positive interactions and participation, remained the same as those of face-to-face PBL sessions. However, the effect of the transition to virtual PBL sessions in the middle of a term on the development of these competencies was unknown. Research is therefore essential to assess the usefulness of those alternative plans for the COVID-19 pandemic (Foong, 2020). There are a few attempts to assess the effectiveness of fully online education amidst the COVID-19 pandemic, but these studies were generic to large-group learning, limited attempts explained the impacts on small-group interactive learning such as PBL (Hew et al., 2020; Schlenz et al., 2020).

Conveniently, administrators could draw on the results from studies that have reported on the implementation, effectiveness, and experiences of conducting planned virtual PBL sessions (Car et al., 2019; Musal et al., 2017; Valaitis et al., 2005). Virtual PBL sessions have shown to be effective as students have reported increased flexibility for learning, improved ability to process content, and improved access to valuable learning resources (Coiado et al., 2020; Valaitis et al., 2005). A systematic review also suggests that virtual PBL is not only as effective as traditional PBL but that it may be more effective at improving skills (Car et al., 2019). However, students have also reported technical difficulties, perceived increased workload, and asynchronous communication (Coiado et al., 2020; Musal et al., 2017; Valaitis et al., 2005). Overall, the use of virtual PBL in lieu of face-to-face sessions is an appropriate response, particularly facing the challenges of the COVID-19 pandemic.

Medical schools have globally responded to the COVID-19 pandemic by continuing programming by offering PBL in VLEs (Coiado et al., 2020; Khalil et al., 2020). It is necessary to measure the effectiveness and success of a transition to virtual PBL, particularly in environments where students were unfamiliar with the VLE at the beginning of their PBL sessions. The success of this transition may also indicate the success of the continued delivery of medical education using VLEs in other courses as well. This study aimed to assess the learning experiences of pre-clinical medical students on the implementation of the virtual PBL, in replacing the traditional in-person learning space. We compared the experiences of pre-clinical medical students at the beginning and completion of the virtual PBL since student perception is a measure of effectiveness and success of the curricula (Degtjarjova et al., 2018). This study helped to understand students’ experiences of an unexpected transition to the virtual PBL during the COVID-19 pandemic.

A. Theoretical Framework

We used the ACTIONS model (Access, Costs, Teaching and learning, Interaction and user-friendliness, Organisational issues, Novelty, and Speed) by Tony Bates to develop the VLE in a timely and resource-efficient manner (Boettcher & Conrad, 1999). We prioritised the consideration in Access and Costs since the University of Malaya was unable to provide unlimited internet subscriptions and access to all students and lecturers. This is, of course, required to establish an effective VLE. Teaching and learning and Interaction and user-friendliness were also important factors of the framework that were considered before finally selecting Microsoft Teams and OneNote. The software was able to support both the user demand and could host large and small group sessions. Finally, Microsoft Teams and OneNote were also able to satisfy the Organisational issues and Novelty and Speed properties of the VLE. The University of Malaya was able to provide appropriate information technology (IT) support by expanding online resources to be synchronised, aligned, and integrated. Ultimately, the university managed to provide a centralised platform for teaching and learning through Microsoft Teams and OneNote.

II. METHODS

The curriculum team, faculty development team, IT team, and programme evaluation team at the Medical Education Unit were involved in designing, implementing, and evaluating the virtual PBL.

A. Designing the Virtual PBL

The University of Malaya had been practicing face-to-face PBL weekly. There were eight students assigned to a group, and the discussion was facilitated by a trained tutor. In a PBL room, students discussed at a round table and scribed their points of discussion on the whiteboards. When the virtual teaching and learning modality commenced, students and tutors conducted their PBL in a virtual learning space using Microsoft Teams, and they used Microsoft One Note as the virtual whiteboard. All the required materials, such as the handbooks, were made available online. Students participated in the virtual PBL sessions at their primary residences (off-campus).

B. Setting up Microsoft Teams, Microsoft One Note (Virtual Whiteboard), and Users Onboarding

The customisation of the features was performed by the IT team. They created, set up and, organised the virtual classrooms and relevant resources (e.g., user distribution list, naming structure). The instruction was given to students to register for a Microsoft Teams account. Every student was supplied with an installation guide, a user manual, and a video guide by the IT team. Meanwhile, the user manual for tutors provided additional information. Selected features (e.g., raise the hand, initiate chat, share screen, scribe) were introduced to promote interactions with students. Tutors were also given operational access to record sessions. The recordings were stored in the Microsoft Streams for students to use for revision.

C. Training Tutors and Students

After the user manuals were distributed, training sessions and workshops were conducted to show the practical use of basic online interactive tools and the various features of Microsoft Teams and OneNote. In addition to this, a short session on experiencing virtual PBL was provided for tutors. Questions and concerns raised during each training session were resolved immediately by the IT team.

D. Implementation and Monitoring

The Medical Education Unit integrated student evaluation and tutor experiences in continually monitoring and improving the virtual PBL. The curriculum team monitored each of the allocated virtual PBL sessions to ensure the sessions were conducted as scheduled. Next, the faculty development team conducted a series of interviews with 20 tutors about the VLE. The information from these interviews was integrated into the future training of tutors. The faculty development team found that the tutors were able to navigate the VLE. The general attitude towards the VLE improved as tutors became more familiar with the platform. There were several reported issues, like poor connectivity and difficulty in navigating Microsoft Teams, these were all resolved by the IT team without affecting the delivery of the PBL materials. Finally, student feedback, through standardised evaluation forms, was collected by the programme evaluation team. The feedback was provided to tutors, and they were expected to integrate this feedback into the facilitation of subsequent virtual PBL sessions.

E. Assessing Learning Experiences of Students

A questionnaire was developed to measure the learning experiences of students in a transition to virtual PBL from face-to-face sessions. Items were designed using several resources: (1) the existing teaching criteria of the medical programme (e.g. based on the programme’s learning objectives) (Foong et al., 2015), (2) concerns raised by faculty lecturers prior to the implementation of virtual PBL (e.g. motivation, obtaining and understanding the information), (3) previous instruments in measuring the effectiveness of PBL (e.g. able to apply knowledge) (Yeo & Chang, 2017), (4) online learning (e.g. learn when I want) (Ngampornchai & Adams, 2016) and (5) education environment  (e.g. confidence to pass the examination) (Roff, 2005). A pilot study was not possible due to limited time and the lack of a subject pool with experience with virtual PBL. As such, the programme evaluation team consisted of an educationist and two evaluation officers who reviewed the items. Ultimately, the instrument consisted of 13 items, with 5-point Likert-scale responses (1=strongly disagree, 2=disagree, 3=unsure, 4=agree, 5=strongly agree).

A prospective design was used to evaluate any change in experience between the beginning of implementation and the completion of all the virtual PBL sessions. The University of Malaya Research Ethics Committee (UM.TNC2/ UMREC-998) approved the instrument and design before the study was initiated. The questionnaire was distributed online to 344 pre-clinical (Year 1 and Year 2) medical students. Digital consent was obtained from students before they started to answer the online questionnaires. The first (initial) questionnaire was distributed to students one week after the implementation of the virtual PBL, May 4, 2020. One week later, students were sent one reminder. The same students were asked to complete the same questionnaire (final) a second time, after the completion of 13 virtual PBL sessions (i.e., two learning blocks). An invitation asking for participation was sent on July 27, 2020. Responses were initially opened for two weeks, and it was extended for another two weeks due to a low response rate. Initial and final questionnaire responses were matched using participant identities, after which responses were anonymised.

F. Data Analysis

All data were entered and analysed using IBM SPSS Statistics for Windows version 23.0. First, the construct validity and reliability of the questionnaire were assessed. A principal component analysis (PCA) was conducted on the 13-item instrument with varimax rotation. Overall, the Kaiser-Meyer Olkin (KMO) should be ≥ 0.7, and Bartlett’s Test measure of sampling adequacy should be significant at p= 0.05, to verify the appropriateness of using PCA (Bowling, 2009). For each component to exist, it should demonstrate an eigenvalue ≥ 1 in the Scree plot (Campbell et al., 2002). For items to be considered in components, the average communality after extraction should be greater than ≥0.6 for a sample size that exceeds 250 (Field, 2009). In addition, to consider the inclusion of an item for a component, the item should demonstrate factor loading ≥0.5 (Hair et al., 2009); an item with cross-loadings ≥0.5 should be excluded (Maskey et al., 2018). In terms of reliability, internal consistency using Cronbach’s alpha measure was analysed for the overall instrument and its domains. A Cronbach’s alpha coefficient of ≥0.5 would indicate an adequate internal consistency (Bowling, 2009; Verma et al., 2010). The corrected item-total correlation should be ≥ 0.2, and the effect of deleting an item on the Cronbach’s alpha coefficient is considered (Verma et al., 2010).

Second, descriptive statistics (e.g., frequencies, percentages, mean, standard deviation) were calculated for the demographics and each item. Next, the data followed non-normal distribution, so Wilcoxon signed-rank tests were used to compare item scores of the same students (paired samples) between the beginning and at the end of virtual PBL implementation. Third, the total score of the 12-item questions was compared between different years of study, genders, geographical locations, and online course experiences using the Mann-Whitney U test, respectively for initial and final questionnaires. Similarly, the total score was compared between different ages and household incomes using the Kruskal Wallis test, respectively for initial and final questionnaires.

III. RESULTS

A. Response Rate

Three hundred and forty-four students (N=344) (in 43 pre-assigned groups) began virtual PBL sessions when teaching and learning resumed on April 19, 2020. Students completed 13 PBL cases representing two learning blocks. Out of the 344 students, 293 students completed and submitted the initial questionnaire distributed at the beginning of the virtual PBL, and 317 students completed the final questionnaire distributed after completing the virtual PBL sessions. The number of students asked to participate was the same for both surveys. Upon matching the identities of respondents, 275 students (80%) responded to both the questionnaires, and whose data was used for further analysis. The remaining surveys were disregarded.

B. Psychometric Properties of the Questionnaire

A principal component analysis of 550 questionnaires produced the following results. During initial analysis, one item, ‘Overall, I believe that PBL using Microsoft Teams could replace face-to-face PBL sessions’, was excluded due to its factor loadings <0.5 in all components (Table 1). In the final analysis, the KMO measure for sampling adequacy was 0.85, and Bartlett’s test of sphericity produced an approximate chi-square of 4073.55, p< 0.01. Three components had eigenvalues >1.0 respectively, and in combination explained 71.42% of the variances (component 1 represents “Learning” = 27.80%; component 2 represents “Confidence” = 26.74%; component 3 represents “Concern” =16.89%). All the 12 items had factor loadings ≥0.5 and did not have cross-loading ≥0.5. An average communality after extraction of 0.65 was produced (max=0.90; min=0.55).

Internal consistency was measured using Cronbach’s alpha coefficient. Overall, the questionnaire produced a high and satisfactory coefficient (α= 0.897). The coefficient for each domain was also satisfactory such that the ‘Learning’ subsection produced a value of 0.869, the ‘Confidence’ subsection produced a value of 0.856, and the ‘Concern’ subsection produced a value of 0.900. All items had corrected item-total correlation coefficients of ≥ 0.2; while if any item was to be removed, the Cronbach’s alpha coefficient will be reduced (Table 1). In conclusion, the questionnaire is valid and reliable.

 

No

Statement

Factor Loadings

Cronbach Alpha (Overall or by Domain)

Corrected Item-Total Correlation

Cronbach Alpha if Item Deleted

All items

0.897

‘Learning’ Domain

L1

I am able to learn anytime or anywhere that I want through PBL using Microsoft Teams

0.86

0.869

0.670

0.846

L2

I am able to learn according to my pace or ability through PBL using Microsoft Teams

0.84

0.692

0.842

L3

I am motivated to participate in PBL using Microsoft Teams for my studies

0.65

0.690

0.843

L4

I am able to obtain adequate information based on the learning objectives through PBL using Microsoft Teams

0.61

0.702

0.843

L5

I am able to understand the given information through PBL using Microsoft Teams

0.61

0.689

0.846

L6

Advantages of PBL using Microsoft Teams outweigh its disadvantages

0.52

0.623

0.860

‘Confidence’ Domain

‘C1

I am confident to pass my clinical examination after participating in PBL using Microsoft Teams

0.84

0.856

0.665

0.836

C2

I am confident to pass my knowledge-based written examination after participating in PBL using Microsoft Teams

0.81

0.741

0.799

C3

I am confident to perform skills that I have gained through PBL using Microsoft Teams

0.78

0.741

0.799

C4

I am confident to apply knowledge that I have gained through PBL using Microsoft Teams

0.66

0.667

0.832

‘Concern’ Domain

W1

(Reversed) I am worried that I might acquire less content through PBL using Microsoft Teams, as compared to face-to-face PBL sessions

0.94

0.900

0.818

NA

W2

(Reversed) I am worried that I might acquire the content slower through PBL using Microsoft Teams, as compared to face-to-face PBL sessions

0.92

0.818

NA

NA=Not Applicable

Table 1. Psychometric properties of the questionnaire

A. Demographics

The distribution of demographics was representative of the Year 1 and Year 2 pre-clinical student population based on student records (Table 2).


Note: aThe Malaysian government categorises its population into three income groups. T20 represents the top 20% income earners; M40 represents the middle 40% income earners; B40 represents the bottom 40% income earners.

Table 2. Demographics

A. Learning Experiences of Students

Student responses were assessed for their strength in agreement with each item in the initial and final questionnaires. The trends in response for items from each domain were also compared.

 

For the “Learning” domain, students reported learning experiences to be more positive than negative on a scale from 1.00 to 5.00 (Luciani et al., 2015) (Table 3). Many students indicated agreement for L1 to L5 (medianL1-L5=4.00), except for L6 (medianL6=3.00). In addition, students’ perceptions on learning through virtual PBL were significantly improved for items L2 (learn according to pace/ability) (p=.015), L3 (motivated) (p=.000), L4 (obtain adequate information) (p=.004), L5 (understand the information) (p=.003), and L6 (advantages outweigh disadvantages) (p=.041).

 

For the “Confidence” domain, students responded positively on average to all the items relating to confidence (medianC2-C4=4.00) except C1 (pass the clinical examination) (medianC4=3.00) (Table 3). Confidence of students was significantly higher at the end of the virtual PBL implementation in terms of C1 (pass the clinical examination) (p=.001), C2 (pass the knowledge-based written examination) (p=.000), and C4 (apply the gained knowledge) (p=.000); the increase in C3 (perform gained skills) was not significant (p=.067). Nevertheless, it is noted that students, while more agreed in the final questionnaire, remained mostly neutral towards their confidence to pass the clinical examination after participating in PBL using Microsoft Teams at the end of the implementation (medianinitial=3.00, medianfinal=3.00).

 

For the “Concern” domain, the results are summarised in Table 3. Their concerns in terms of W1 (might acquire less content) and W2 (might acquire content slower) slightly reduced, but the differences were not significant at p=0.05. Many students indicated neutrality, at the beginning and the end of the implementation (medianW1-W2=3.00). Further investigation into this trend showed that the initial median of 3.00 for W1 had a broad distribution of responses across the scale (12.4% students strongly disagreed, 21.1% students disagreed, 24.4% students were neutral, 28.0% students agreed, and 14.2% students strongly agreed). Similarly, the initial median of 3.00 for W2 was a result of 12.4% students strongly agreed, 20.4% disagreed, 23.6% students were neutral, 30.2% students agreed, and 13.5% students strongly agreed. These trends were similar for W1 and W2 from the final questionnaire where the distribution of response for W1 and W2 ranged from 11.3% to 26.5%. In summary, students’ concern was broadly represented in both the initial and final questionnaires.

 

Last, there were no significant differences in the total score comparing between different years of study, ages, geographical locations, and online course experiences, in both the initial and final questionnaires. However, the post-hoc test for the Kruskal Wallis test reported that students in households with a higher income (T20) had significantly higher total score than the lower-income students (B40) in both the initial (medianT20=43.0, medianB40=39.0, p=0.034) and final questionnaire (medianT20=46.0, medianB40=40.0, p=0.008). Whereas, male respondents indicated significantly more agreement in the initial questionnaire (medianmale=42.0, medianfemale=40.0, p=0.048); there was no significant difference between gender by the completion of PBL sessions. 

Items

Descriptive Statistics

Wilcoxon Signed-Ranked Tests

Initial Questionnaire

 

Median (IQR)

Final Questionnaire

 

Median (IQR)

 

N

Mean Rank

Sum of Ranks

Z

p

L1

4.00(2)

4.00(2)

Negative Ranks

73

76.10

5555.00

-1.582

.114

Positive Ranks

87

84.20

7325.00

Ties

115

 

 

L2

4.00(2)

4.00(2)

Negative Ranks

61

82.25

5017.50

-2.431

.015

Positive Ranks

98

78.60

7702.50

Ties

116

 

 

L3

4.00(1)

4.00(2)

Negative Ranks

55

74.89

4119.00

-3.868

.000

Positive Ranks

102

81.22

8284.00

Ties

118

 

 

L4

4.00(1)

4.00(1)

Negative Ranks

52

70.04

3642.00

-2.896

.004

Positive Ranks

88

70.77

6228.00

Ties

135

 

 

L5

4.00(0)

4.00(1)

Negative Ranks

51

68.38

3487.50

-3.017

.003

Positive Ranks

87

70.16

6103.50

Ties

137

 

 

L6

3.00(2)

3.00(1)

Negative Ranks

73

81.04

5916.00

-2.043

.041

Positive Ranks

96

88.01

8449.00

Ties

106

 

 

C1

3.00(1)

3.00(2)

Negative Ranks

67

84.04

5630.50

-3.330

.001

Positive Ranks

109

91.24

9945.50

Ties

99

 

 

C2

3.00(1)

4.00(1)

Negative Ranks

47

80.29

3773.50

-4.642

.000

Positive Ranks

111

79.17

8787.50

Ties

117

 

 

C3

3.00(1)

4.00(1)

Negative Ranks

67

82.83

5549.50

-1.832

.067

Positive Ranks

95

80.56

7653.50

Ties

113

 

 

C4

4.00(1)

4.00(1)

Negative Ranks

52

74.81

3890.00

-3.690

.000

Positive Ranks

99

76.63

7586.00

Ties

124

 

 

W1 (Reversed)

3.00(2)

3.00(2)

Negative Ranks

86

94.38

8117.00

-.303

.762

Positive Ranks

96

88.92

8536.00

Ties

93

 

 

W2 (Reversed)

3.00(2)

3.00(2)

Negative Ranks

84

100.73

8461.00

-.579

.562

Positive Ranks

104

89.47

9305.00

Ties

87

 

 

Table 3. Item scores for initial and final questionnaire

 IV. DISCUSSION

The rapid and widespread transmission of SARS-CoV-2 affected every nation and aspect of society, including medical education (Abbas, 2020; Rose, 2020; Sandhu & de Wolf, 2020; Wong, 2020).  A universal response to reduce the spread of the virus was to implement physical distancing practices and elimination of any unnecessary interaction between contacts (Reyna, 2020). As such, competency-based medical programs, like the University of Malaya medical programme was particularly affected. The programme is designed to reflect clinical practice and to develop medical competencies, so group discussion and meeting in person is common practice (Frank et al., 2010). However, due to the closure of university campuses, medical schools were forced to respond by modifying the programme to be hosted in VLEs (Alkhowailed et al., 2020). An example of this comes from the University of Malaya who decided to transition PBL sessions to a VLE hosted by Microsoft Teams.

As with other medical schools (Coiado et al., 2020; Newman & Lattouf, 2020; Rose, 2020; Yusoff et al., 2020), the University of Malaya decided to offer the programme virtually, as the most effective means to continue programming with minimal disturbance. In this study, students’ experiences were generally improved by the end of the virtual PBL implementation. The following might be the contributing factors. First, a systematic approach using the ACTIONS framework was taken to develop an appropriate VLE. The Medical Education Unit chose Microsoft Teams and One Note as the most suitable online platform to deliver their courses and ensured that the key features of the traditional PBL sessions were translated to the VLE and virtual PBL sessions. Before implementing the virtual PBL, the faculty development team prepared guidebooks, conducted workshops, and answering questions to students and tutors. A strategy to identify and resolve problems as they arose was also implemented through tutor interviews and student evaluations.

The ability to translate students’ ability to learn in a VLE is a necessary component of virtual PBL particularly during the COVID-19 pandemic (Reyna, 2020). As such, we asked students about their opinions towards learning in the VLE. In general, students indicated a positive experience towards participating and learning from virtual PBL. Another important measure of educational success is the ability of students to translate their new knowledge into practice, applications, and assessments (Car et al., 2019; Robinson et al., 2015). Taken as a whole, students felt confident in their abilities to use the knowledge gained during the virtual PBL sessions.

Students were also asked about their concerns towards acquiring the appropriate amount of content and pacing of course material (items W1 and W2). The average response from students indicated a neutral opinion about these worries and there was no significant difference between the first and final questionnaires. Interestingly, the responses are broadly distributed across the scale for both items and in both the initial and final questionnaires. This may indicate that ‘concern’ may be interpreted differently between students as the level of tolerance to uncertainties varies among individuals (Hillen et al., 2017).

We also discovered some interesting trends between specific students in this study with their experiences conducting virtual PBL. Interestingly, students associated with the highest household income reported significantly higher scores in both the initial and final questionnaires. Higher household income may be indicative of accessibility to learning resources or increased confidence in performance (Thomson, 2018). Further investigation into this is necessary to understand whether the transition to VLE may provide inequitable education for all students. Additionally, males responded significantly higher than females in the initial questionnaire, but similarly in the final questionnaire. It would be interesting to investigate whether this trend is related to societal determinants of attitudes, which suggest that males are more technology-driven (Vázquez-Cano et al., 2017). Though interestingly, other studies suggest that there is no difference between the genders and attitudes towards e-learning (Gormley et al., 2009) or even show that males were less motivated to participate in PBL (Joseph et al., 2016). Regardless of the motivation for the responses, both genders had similar perceptions after completing the virtual PBL sessions.

In addition to the findings above, the questionnaire also provided insight into areas of improvement for the virtual PBL. Students were overwhelmingly positive towards the use of a VLE for PBL, suggesting that these sessions could remain virtual in the future. The biggest areas to improve are the confidence in passing clinical exams and the worry about content acquisition. More emphasis should be placed on the clinical applicability of the PBL sessions so that students may better see how to use the material in practice. Additionally, students must be reassured that participating in virtual PBL sessions is equivocal to in-person sessions in terms of the amount and pacing of content delivery.

A. Limitations of the Study

First, the study was conducted in a single institution, which limited the generalisability of its results. Second, a standard for expectations in conducting virtual PBL during the COVID-19 pandemic is not available (Schlenz et al., 2020). In this study, improvements in mean scores could only be considered as positive changes instead of meeting the standard of expectations, as there is no benchmark on the satisfactory mean score.  Last, more investigations are needed before virtual PBL can be considered as an effective option post-pandemic. One of the future investigations would be accessibility to the internet and bandwidth in conducting virtual PBL.

 V. CONCLUSION

A transition to virtual PBL was necessary during the MCO in Malaysia during the COVID-19 pandemic. In this study, the advantages of virtual PBL outweigh its disadvantages, as an acceptable alternative. Other medical schools could consider using virtual PBL to replace face-to-face PBL, and it would minimise disruption in medical training during the COVID-19 pandemic. Our experiences and lessons learnt may be helpful for medical schools moving towards digitising their PBL.

Notes on Contributors

VP, KK, and PS designed and developed the VLE. CCF, AJL, CRA designed and conceptualised this study with input from WHH, VP, and JV. CCF, AJL, and CRA worked on the acquisition and analysis of data. All the authors interpreted the data and drafted portions of the manuscript related to their roles in the Medical Education Unit. The manuscript has been read, revised, and approved for submission by all authors.

Ethical Approval

The authors had obtained ethical approval from the University of Malaya Research Ethics Committee (UM.TNC2/ UMREC-998) before the study was initiated.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request. The data are not publicly available due to privacy and ethical concerns.

Acknowledgement

The authors would like to thank the students for participating in the study.

Funding

The authors received no financial support for this research.

Declaration of Interest

The authors declare that they have no competing interests.

References

Abbas, A. M. (2020). COVID-19 pandemic and medical education in a developing country. American Journal of Biomedical Science & Research, 9(4), 289-290. https://doi.org/10.34297/AJBSR.2020.09.001408

Abedi, M., & Abedi, D. (2020). A letter to the editor: The impact of COVID-19 on intercalating and non-clinical medical students in the UK. Medical Education Online, 25(1), 1771245.

https://doi.org/10.1080/10872981.2020.1771245

Alkhowailed, M. S., Rasheed, Z., Shariq, A., Elzainy, A., El Sadik, A., Alkhamiss, A., Alsolai, A. M., Alduraibi, S. K., Alduraibi, A., & Alamro, A. (2020). Digitalization plan in medical education during COVID-19 lockdown. Informatics in Medicine Unlocked, 20, 100432. https://doi.org/10.1016/j.imu.2020.100432

Boettcher, J., & Conrad, R.-M. (1999). Faculty guide for moving teaching and learning to the web. League for Innovation in the Community College.

Bowling, A. (2009). The psychometric properties of the older people’s quality of life questionnaire, compared with the CASP-19 and the WHOQOL-OLD. Current Gerontology and Geriatrics Research, 2009, 1-12. https://doi.org/10.1155/2009/298950

Campbell, S., Braspenning, J., Hutchinson, A., & Marshall, M. (2002). Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care, 11(4), 358-364. https://doi.org/10.1136/bmj.326.7393.816

Car, L. T., Kyaw, B. M., Dunleavy, G., Smart, N. A., Semwal, M., Rotgans, J. I., Low-Beer, N., & Campbell, J. (2019). Digital problem-based learning in health professions: Systematic review and meta-analysis by the digital health education collaboration. Journal of Medical Internet Research, 21(2), e12945. https://doi.org/10.2196/12945

Coiado, O. C., Yodh, J., Galvez, R., & Ahmad, K. (2020). How COVID-19 transformed problem-based learning at Carle Illinois College of Medicine. Medical Science Educator, 30(4), 1353-1354. https://doi.org/10.1007%2Fs40670-020-01063-3

Degtjarjova, I., Lapina, I., & Freidenfelds, D. (2018). Student as stakeholder: Voice of customer in higher education quality development. Marketing and Management of Innovations, 2, 388-398. https://doi.org/10.21272/mmi.2018.2-30

Ebner, M., Schön, S., Braun, C., Ebner, M., Grigoriadis, Y., Haas, M., Leitner, P., & Taraghi, B. (2020). COVID-19 epidemic as e-learning boost? Chronological development and effects at an Austrian University against the background of the concept of “e-learning readiness”. Future Internet, 12(6), 94. https://doi.org/10.3390/fi12060094

Ferrel, M. N., & Ryan, J. J. (2020). The Impact of COVID-19 on Medical Education. The Cureus Journal of Medical Science, 12(3), e7492. https://doi.org/10.7759%2Fcureus.7492

Field, A. (2009). Discovering statistics using SPSS (3rd ed.). SAGE publications.

Foong, C. C. (2020). Preliminary measures in the COVID-19 pandemic a trial for futuristic medical education. Journal of Research in Medical and Dental Science, 8(5), 77-78.

Foong, C. C., Hassan, H., Lee, S. S., & Vadivelu, J. (2015). Using students’ formative feedback to advocate reflective teaching. Medical Education, 49(5), 535. https://doi.org/10.1111/medu.12697

Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., & Dath, D. (2010). Competency-based medical education: Theory to practice. Medical Teacher, 32(8), 638-645. https://doi.org/10.3109/0142159x.2010.501190

Gormley, G. J., Collins, K., Boohan, M., Bickle, I. C., & Stevenson, M. (2009). Is there a place for e-learning in clinical skills? A survey of undergraduate medical students’ experiences and attitudes. Medical Teacher, 31(1), e6-e12. https://doi.org/10.1080/01421590802334317

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysis (7th ed.). Pearson.

Hew, K. F., Jia, C., Gonda, D. E., & Bai, S. (2020). Transitioning to the “new normal” of learning in unpredictable times: Pedagogical practices and learning performance in fully online flipped classrooms. International Journal of Educational Technology in Higher Education, 17(1), 1-22. https://doi.org/10.1186/s41239-020-00234-x

Hillen, M. A., Gutheil, C. M., Strout, T. D., Smets, E. M., & Han, P. K. (2017). Tolerance of uncertainty: Conceptual analysis, integrative model, and implications for healthcare. Social Science & Medicine, 180, 62-75. https://doi.org/10.1016/j.socscimed.2017.03.024

Ifediora, C. O. (2019). Online medical education for doctors: Identifying potential gaps to the traditional, face-to-face modality. Journal of Medical Education and Curricular Development, 6, 2382120519827912. https://doi.org/10.1177%2F2382120519827912

Joseph, N., Rai, S., Madi, D., Bhat, K., Kotian, S. M., & Kantharaju, S. (2016). Problem-based learning as an effective learning tool in community medicine: Initiative in a private medical college of a developing country. Indian Journal of Community Medicine, 41(2), 133-140. https://doi.org/10.4103/0970-0218.177535

Khalil, R., Mansour, A. E., Fadda, W. A., Almisnid, K., Aldamegh, M., Al-Nafeesah, A., Alkhalifah, A., & Al-Wutayd, O. (2020). The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: A qualitative study exploring medical students’ perspectives. BMC Medical Education, 20, 285. https://doi.org/10.1186/s12909-020-02208-z

Laamanen, M., Ladonlahti, T., Uotinen, S., Okada, A., Bañeres, D., & Koçdar, S. (2021). Acceptability of the e-authentication in higher education studies: Views of students with special educational needs and disabilities. International Journal of Educational Technology in Higher Education, 18(1), 1-17. https://doi.org/10.1186/s41239-020-00236-9

Luciani, E., van Dun, P. L., Esteves, J. E., Lunghi, C., Petracca, M., Papa, L., Merdy, O., Jäkel, A., & Cerritelli, F. (2015). Learning environment, preparedness and satisfaction in osteopathy in Europe: The PreSS study. PloS One, 10(6), e0129904. https://doi.org/10.1371%2Fjournal.pone.0129904

Maskey, R., Fei, J., & Nguyen, H.-O. (2018). Use of exploratory factor analysis in maritime research. The Asian Journal of Shipping and Logistics, 34(2), 91-111. https://doi.org/10.1016/j.ajsl.2018.06.006

Musal, B., Keskin, O., & Tuncel, P. (2017). Application of an electronic problem-based learning system in undergraduate medical education program. Journal of Health and Medical Informatics, 8(1), 1000246. https://doi.org/10.4172/2157-7420.1000246

Newman, N. A., & Lattouf, O. M. (2020). Coalition for medical education—A call to action: A proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during COVID‐19. Journal of Cardiac Surgery, 35(6), 1174-1175. https://doi.org/10.1111%2Fjocs.14590

Ngampornchai, A., & Adams, J. (2016). Students’ acceptance and readiness for e-learning in Northeastern Thailand. International Journal of Educational Technology in Higher Education, 13(1), 34. https://doi.org/10.1186/s41239-016-0034-x

Reyna, J. (2020). Twelve tips for COVID-19 friendly learning design in medical education. MedEdPublish, 9(1), 103. https://doi.org/10.15694/mep.2020.000103.1

Robinson, G. F., Moore, C. G., McTigue, K. M., Rubio, D. M., & Kapoor, W. N. (2015). Assessing competencies in a master of science in clinical research program: The comprehensive competency review. Clinical and Translational Science, 8(6), 770-775. https://doi.org/10.1111%2Fcts.12322

Roff, S. (2005). The Dundee Ready Educational Environment Measure (DREEM)—A generic instrument for measuring students’ perceptions of undergraduate health professions curricula. Medical Teacher, 27(4), 322-325. https://doi.org/10.1080/01421590500151054

Rose, S. (2020). Medical student education in the time of COVID-19. Jama Network, 323(21), 2131-2132. https://doi.org/10.1001/jama.2020.5227

Sandhu, P., & de Wolf, M. (2020). The impact of COVID-19 on the undergraduate medical curriculum. Medical Education Online, 25(1), 1764740. https://doi.org/10.1080/10872981.2020.1764740

Schlenz, M. A., Schmidt, A., Wöstmann, B., Krämer, N., & Schulz-Weidner, N. (2020). Students’ and lecturers’ perspective on the implementation of online learning in dental education due to SARS-CoV-2 (COVID-19): A cross-sectional study. BMC Medical Education, 20, 354. https://doi.org/10.1186/s12909-020-02266-3

Shah, A. U. M., Safri, S. N. A., Thevadas, R., Noordin, N. K., Abd Rahman, A., Sekawi, Z., Ideris, A., & Sultan, M. T. H. (2020). COVID-19 outbreak in Malaysia: Actions taken by the Malaysian government. International Journal of Infectious Diseases, 97, 108-116. https://doi.org/10.1016/j.ijid.2020.05.093

Tabatabai, S. (2020). Simulations and virtual learning supporting clinical education during the COVID 19 pandemic. Advances in Medical Education and Practice, 11, 513-516. https://doi.org/10.2147/AMEP.S257750

Thomson, S. (2018). Achievement at school and socioeconomic background—An educational perspective. NPJ Science of Learning, 3, 5. https://doi.org/10.1038/s41539-018-0022-0

Valaitis, R. K., Sword, W. A., Jones, B., & Hodges, A. (2005). Problem-based learning online: Perceptions of health science students. Advances in Health Sciences Education, 10(3), 231-252. https://doi.org/10.1007/s10459-005-6705-3

Vázquez-Cano, E., Meneses, E. L., & García-Garzón, E. (2017). Differences in basic digital competences between male and female university students of social sciences in Spain. International Journal of Educational Technology in Higher Education, 14, 27. https://doi.org/10.1186/s41239-017-0065-y

Verma, S. K., Vidisha, S., Bhopal, M., & Jaloree, S. (2010). Pattern mining approach to categorization of students’ performance using Apriori Algorithm. International Journal of Computer Applications, 121, 36-39. https://doi.org/10.5120/21540-4550

Wong, R. Y. (2020). Medical education during COVID-19: Lessons from a pandemic. British Columbia Medical Journal, 62(5), 170-171.

World Health Organisation. (2020) WHO/Europe | Coronavirus disease (COVID-19) outbreak – About the virus. https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/novel-coronavirus-2019-ncov

Yeo, S., & Chang, B. H. (2017). Implementation of problem-based learning in medical education in Korea. Korean Journal of Medical Education, 29(4), 271-282. https://doi.org/10.3946/kjme.2017.73

Yusoff, M. S. B., Hadie, S. N. H., Mohamad, I., Draman, N., Al-Aarifin, I. M., Rahman, W. F. W. A., Pa, M. N. M., & Yaacob, N. A. (2020). Sustainable medical teaching and learning during the COVID-19 pandemic: Surviving the new normal. The Malaysian Journal of Medical Sciences, 27(3), 137-142. https://doi.org/10.21315/mjms2020.27.3.14

*Jamuna Vadivelu
Medical Education & Research Development Unit (MERDU),
Faculty of Medicine,
University of Malaya,
Lembah Pantai,
50603 Kuala Lumpur, Malaysia.
Email: jamuna@ummc.edu.my

Announcements