Unanticipated learning effects in videoconference continuous professional development

Number of Citations: 0

Submitted: 21 January 2021
Accepted: 16 April 2021
Published online: 5 October, TAPS 2021, 6(4), 135-141
https://doi.org/10.29060/TAPS.2021-6-4/SC2484

Caroline Choo Phaik Ong1,2, Candy Suet Cheng Choo1, Nigel Choon Kiat Tan2,3 & Lin Yin Ong1,2

1Department of Paediatric Surgery, KK Women’s and Children’s Hospital, SingHealth, Singapore; 2SingHealth Duke-NUS Academic Medical Centre, Singapore; 3Department of Neurology, National Neuroscience Institute, SingHealth, Singapore

Abstract

Introduction: The COVID-19 pandemic accelerated use of technology like videoconferencing (VC) in healthcare settings to maintain clinical teaching and continuous professional development (CPD) activities. Sociomaterial theory highlights the relationship of humans with sociomaterial forces, including technology. We used sociomaterial framing to review effect on CPD learning outcomes of morbidity and mortality meetings (M&M) when changed from face-to-face (FTF) to VC.

Methods: All surgical department staff were invited to participate in a survey about their experience of VC M&M compared to FTF M&M. Survey questions focused on technological impact of the learning environment and CPD outcomes. Respondents used 5-point Likert scale and free text for qualitative responses. De-identified data was analysed using Chi-squared comparative analysis with p<0.05 significance, and qualitative responses categorised.

Results: Of 42 invited, 30 (71.4%) responded. There was no significant difference in self-reported perception of CPD learning outcomes between FTF and VC M&M. Participants reported that VC offered more convenient meeting access, improved ease of presentation and viewing but reduced engagement. VC technology allowed alternative communication channels that improved understanding and increased junior participation. Participants requested more technological support, better connectivity and guidance on VC etiquette.

Conclusion: VC technology had predictable effects of improved access, learning curve problems and reduced interpersonal connection. Sociomaterial perspective revealed additional unexpected VC behaviours of chat box use that augmented CPD learning. Recognising the sociocultural and emotional impact of technology improves planning and learner support when converting FTF to VC M&M.

Keywords:            Teleconferencing, Morbidity and Mortality Meeting, Continuous Professional Development, Sociomaterial Theory

I. INTRODUCTION

The COVID-19 pandemic instigated worldwide social distancing and rapid uptake of technology to replace face to face (FTF) communication. Healthcare professionals at clinical workplaces adopted educational technological tools to maintain teaching for students, trainees and continuous professional development (CPD) activities (Cleland et al., 2020). Likewise, our hospital-based department pivoted from FTF to interactive web-based videoconferencing (VC) (Zoom) to continue patient-care quality audits and CPD learning.

Before the pandemic, there was limited interest in teleconferencing for health professions education apart from remote learning and formal CPD webinars (Chipps et al., 2012). VC for informal CPD like the Morbidity and Mortality meeting (M&M) was mentioned only to boost attendance of faculty based at distant campuses. The M&M is a regular audit practice of surgical departments that constitutes an important type of informal CPD for individual and organisational learning (de Feijter et al., 2013). Many guidelines exist for FTF M&M but there are none for VC M&M.

Sociomaterial theory examines the mutual relationship of humans with sociomaterial forces and the resultant changes i.e., humans acting on and influenced by objects, nature, culture and/or technology. It provides a useful perspective to evaluate the effect of VC CPD learning and practice by highlighting the importance of materiality – in this case, technology – that is overlooked by other human-centric sociocultural educational theories (Fenwick, 2014). Using sociomaterial framing, we aimed to review the impact of changing from FTF to VC M&M in terms of CPD learning outcomes and user experience.

II. METHODS

A. Description of Context

On 7 Feb 2020, Singapore declared Orange Alert (severity level 3 out of 4) on the national Disease Outbreak Response System in response to the COVID-19 pandemic. Nationwide infection control measures required staff social distancing in public hospitals. Our department (Appendix A: department context and demographics) organises weekly Journal club and M&M as regular CPD; these were converted from FTF to VC meetings from 25 March 2020 till present. Singapore has widespread digital literacy and familiarity with computer usage; our hospital has used electronic health records since 2018. These factors facilitated our rapid pivot to VC meetings.

B. Description of Study

With institutional research board ethics waiver (CIRB Ref: 2020/2697), we sent an email inviting all department staff to participate in a survey about their experience of VC M&M compared to FTF M&M. The sampling frame comprised 18 permanent staff and 24 temporary staff on rotation in the department, from 1 April to 30 June 2020.

The primary outcomes of the survey were self-reported perceptions comparing FTF and VC M&M, addressing categories of CPD learning relevant to M&M: knowledge, practice change, attitude, user outcomes and intention to change (Table 1: Q1-Q3). We asked additional questions (Q4-14) about the FTF/ VC learning environments to elicit possible technological effects on primary outcomes. Face validity of the questionnaire was assessed by authors CCPOng, NCKTan and LYOng who are physicians familiar with M&M.

Recruitment, data collection, data entry and de-identification was performed by author CSChoo (clinical research coordinator) who is outside the department clinical hierarchy. Survey non-responders were given two reminders by CSChoo before the final 3-week deadline. Consent was implied if participants returned the completed survey.  Authors CCPOng and CSChoo analysed the de-identified data. Participants responded whether they agreed with the statement, using a 5-point Likert scale. We carried out Chi-squared comparative analysis on 3 grouped categories: (strongly agree+ agree); (neutral) and (disagree+ strongly disagree).

III. RESULTS

A. Descriptive Demographics 

We received responses from 30 people out of 42 invited (71.4%) with similar response rates for permanent staff 13/18 (72.2%) and temporary staff 17/24 (70.8%). Appendix A provides details on age, gender, job grade of respondents and prior familiarity with VC.

B. Survey Findings

The participants had attended on average 18.7 (SD 13.4) FTF M&M and 15.1(SD 8.3) VC M&M in the preceding 12 months. Apart from VC M&M, all had attended some other VC event such as administrative meetings, tutorials, webinars and non-work-related workshops or dinners.

Q

Perception

Analysis* group

FTF M&M

VC M&M

p-value

Strongly disagree & Disagree

Neutral

Strongly Agree & Agree

Strongly disagree & Disagree

Neutral

Strongly Agree & Agree

Q1

I learnt new medical knowledge

whole

0

5(16.7)

25 (83.3)

1 (3.3)

0

29 (96.7)

0.043

sub

0

1 (4.2)

23 (95.8)

1 (4.2)

0

23 (95.8)

0.368

Q2

I learnt new skills (e.g. clinical, teaching, communication, research, team, practical)

whole

0

7 (23.3)

23 (76.7)

1 (3.3)

5 (16.7)

24 (80.0)

0.508

sub

0

3 (12.5)

21 (87.5)

1 (4.2)

3 (12.5)

20 (83.3)

0.599

Q3

I would change my practice based on what I learnt

whole**

0

7 (24.1)

22 (75.9)

1 (3.3)

3 (10.0)

26 (86.7)

0.233

sub**

0

3 (13)

20 (87.0)

1 (4.2)

2 (8.3)

21(87.5)

0.548

Q4

Junior staff are comfortable presenting

whole

2 (6.7)

8 (26.7)

20 (66.7)

1 (3.3)

3 (10.0)

26 (86.7)

0.184

sub

2 (8.3)

3 (12.5)

19 (79.2)

1 (4.2)

2 (8.3)

21 (87.5)

0.729

Q5

Participants are comfortable to ask questions to clarify

whole

4 (13.3)

9 (30.0)

17 (56.7)

3 (10.0)

7 (23.3)

20 (66.7)

0.728

sub

4 (17.7)

5 (20.8)

15 (62.5)

3 (12.5)

6 (25)

15 (62.5)

0.890

Q6

Participants are comfortable to raise concerns or disagree with management

whole

3 (10.0)

10 (33.3)

17 (56.7)

4 (13.3)

5(16.7)

21 (70.0)

0.328

sub

3 (12.5)

6 (25.0)

15 (62.5)

4 (16.7)

4 (16.7)

16 (66.7)

0.750

Q7

Tone of discussion is respectful

whole

4 (13.3)

10 (33.3)

16 (53.3)

1 (3.3)

6 (20.0)

23 (76.7)

0.132

sub

3 (12.5)

6 (25.0)

15 (62.5)

1 (4.2)

5 (20.8)

18 (75.0)

0.506

Q8

Participants are engaged during the meeting

whole

2 (6.7)

9 (30.0)

19 (63.3)

6 (20.0)

8 (26.7)

16 (53.3)

0.314

sub

2 (8.3)

4 (16.7)

18 (75.0)

6 (25.0)

7 (29.2)

11(45.8)

0.105

Q9

I can see the slides clearly

whole

0

9 (30.0)

21 (70.0)

2 (6.7)

1 (3.3)

27 (90.0)

0.01

sub

0

4 (16.7)

20 (83.3)

2 (8.3)

1 (4.2)

21 (87.5)

0.148

Q10

I can follow the discussion well

whole

0

5 (16.7)

25 (83.3)

3 (10.0)

3 (10.0

24(80.0)

0.172

sub

0

1 (4.2)

23 (95.8)

3 (12.5)

3 (12.5)

18 (75.0)

0.100

Q11

It is easy to provide comments during the meeting

whole

3 (10.0)

8 (26.7)

19 (63.3)

6 (20.0)

6 (20.0)

18 (60.0)

0.519

sub

3 (12.5)

3 (12.5)

18 (75.0)

6 (25.0)

6 (25.0)

12 (50.0)

0.202

Questions about VC M&M only

Q12

I find it easy to navigate the buttons/ commands

Strongly disagree & Disagree

Neutral

Strongly Agree & Agree

3 (10%)

3 (10%)

24 (80%)

Q13

I prefer to ask questions / comment by

Typing

No preference

Audio

15 (50%)

12 (40%)

3 (10%)

Q14

I prefer to have the video on/ off for

Myself

Host

Presenter

Participant

On

4 (13.3%)

12 (40%)

21 (70%)

2 (6.7%)

Off

22 (73.3%)

3 (10%)

1 (3.3%)

8 (26.7%)

No preference

4 (13.3%)

15 (50%)

8 (26.7%)

20 (66.7%)

Table 1. Results of the survey

Table 1 shows the collated responses to survey questions comparing experience of FTF and VC M&M (Q1-11) and questions specific to VC technology (Q12-14). There were six participants who either had zero experience of FTF M&M or had experienced FTF M&M only in other departments, not ours. We carried out subgroup analysis excluding these 6 persons to remove possible influence of other M&M styles, since the study focus was on impact of VC technology. 

In general, self-reported perceptions of CPD outcomes were similar for both FTF and VC M&M. Participants appreciated that VC allowed us to continue M&M practice during the pandemic while acknowledging both positive and negative technological influences on process. Two questions (Q1 and Q9) had minor differences that were significant on whole group analysis but not significant on subgroup analysis. There was a trend towards decreased engagement for VC M&M compared to FTF M&M (Q8) that was not statistically significant.

When using VC (Table 1: Q12-14; Appendix B qualitative responses), more participants preferred to ask questions or comment by typing in the chat box than speaking on microphone. The most common reason given was to avoid interrupting meeting flow; some highlighted that the chat box facilitated junior staff participation. A few felt that keeping ‘video-on’ for all participants improved engagement but the rest preferred to have own ‘video-off’ with presenter ‘video-on’ to reduce distraction. Participants felt that while technology offered easier meeting access and simplified scheduling, it sometimes reduced engagement and interfered with community-building. Participants preferred more technological support, clearer guidance on expected VC behaviours, better infrastructure and connectivity.

A copy of the informed consent, survey questions and anonymised database are available at https://doi.org/10.6084/m9.figshare.13611611.v1.

IV. DISCUSSION

Sociomaterial perspectives offer new ways to conceptualise health professions education beyond individual cognitive and sociocultural educational lenses (Fenwick, 2014). Underpinned by diverse theories like cultural-historical activity theory, actor-network theory, and complexity theory, it recognises that “objects and humans act upon one another in ways that mutually transform their characteristics and activity” (Fenwick, 2014). Therefore, sociomaterial perspectives illuminate how technology (VC) and related infrastructure (devices and internet connectivity) interact with humans to modify the VC CPD learning environment.

In our context, widespread device penetration and free hospital Wi-Fi access aided rapid adoption of technology. Institution policy mandates internet separation from patient electronic health records, so staff use personal devices instead of hospital computers for meeting access, but it was otherwise straightforward to convert to VC M&M. Nevertheless, some unanticipated issues and VC behaviours manifested.

Introducing new technology is commonly associated with distress with learning how to use it. We chose Zoom as the most user-friendly VC platform because majority had no prior experience with VC. Unfortunately, early issues like ‘Zoom-bombing’ induced the company to make frequent user-interface changes that confused some users. A few participants (both younger and older) felt inadequately supported during their learning curve. We had provided a simple guidance document with link to online Zoom technical support but most preferred trial and error and asking for help during meetings.

Technical support alone is insufficient to address discomfort caused by social aspects of changed processes. We anticipated that uncertainty about protocols or inappropriate participant behaviours could lead to disengagement with poor CPD outcomes. We preempted these risks by following the same CPD framework as FTF M&M (e.g. moderator controls discussion, presentation template, focus on peer review learning without blame) and instituted additional VC safeguards for patient confidentiality by limiting patient identifiers, preventing recording and confirmation of attendee identity for meeting admission. We naturally evolved VC etiquette of queueing using the ‘raise-hand’ button while the moderator invites discussants by name and manages their order.

An ethnographic study of distributed VC in undergraduate medical education found that unintended ‘technologies of exposure’ – visual, curricular and auditory, discomforted the faculty and students (MacLeod et al., 2019). Similarly, many in our study disliked having their ‘video-on’. Although ‘video-on’ could improve interpersonal trust, visual exposure discomfort may interfere with aims of improved engagement and relationship-building. Originally, our department encouraged but did not mandate universal ‘video-on’. Gradually, it became the norm for all to have ‘video-off’ except the host and presenter. Despite ‘video-off’, we can maintain honest conversations necessary for M&M because of trust built through years of training and working together. Prolonged loss of FTF contact may erode trust, hence we created a departmental WhatsApp chat group to enhance social connection.

VC technology afforded unexpected learning contributions. The chat box promotes participation of reticent staff, both senior and junior, especially those preferring written expression; it augments understanding of audio discussion and allows sharing of links to supporting literature. The ease of participation empowers juniors and shifts focus from the vocal few who dominated FTF M&M. While the VC constraint of turn-taking for speakers slows down discussions, it improves interprofessional respect and meeting discipline when host can ‘mute’ the recalcitrant interrupter.

V. CONCLUSION

Sociomaterial perspectives highlight how VC technology changes the CPD learning environment of the M&M. VC provides improved access for participation and alternative communication channels but potentially reduces engagement. Recognising constraints and trade-offs of technology-driven enhancements allows better planning and learner support in VC CPD.

Note on Contributor

Caroline Choo Phaik Ong reviewed the literature, designed the study, analysed de-identified data and wrote the manuscript. Candy Suet Chong Choo performed data collection and de-identification, analysed the data and gave critical feedback to the writing of the manuscript. Nigel Choon Kiat Tan reviewed the literature, advised the design of the study and gave critical feedback to the writing of the manuscript. Lin Yin Ong advised design of the study and gave critical feedback to the writing of the manuscript. All the authors have read and approve the final manuscript.

Ethical Approval

This study received institutional research board ethics waiver (CIRB Ref: 2020/2697).

Acknowledgement

The authors would like to acknowledge the participants of the survey for sharing their responses freely.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Interest

All the authors have no declarations of conflicts of interest.

Data availability

A copy of the informed consent, survey questions and anonymised database are available at http://doi.org/10.6084/m9.figshare.13611611.v1 under CC0 licence.

References

Chipps, J., Brysiewicz, P., & Mars, M. (2012). A systematic review of the effectiveness of videoconference-based tele-education for medical and nursing education. Worldviews on Evidence-Based Nursing, 9(2), 78-87. https://doi.org/10.1111/j.1741-6787.2012.00241.x 

Cleland, J., Tan, E. C. P., Tham, K. Y., & Low-Beer, N. (2020). How COVID-19 opened up questions of sociomateriality in healthcare education. Advances in Health Sciences Education, 25(2), 479-482. https://doi.org/10.1007/s10459-020-09968-9 

de Feijter, J. M., de Grave, W. S., Koopmans, R. P., & Scherpbier, A. J. J. A. (2013). Informal learning from error in hospitals: what do we learn, how do we learn and how can informal learning be enhanced? A narrative review. Advances in Health Sciences Education, 18(4), 787-805. https://doi.org/10.1007/s10459-012-9400-1     

Fenwick, T. (2014). Sociomateriality in medical practice and learning: Attuning to what matters. Medical Education, 48(1), 44-52. https://doi.org/10.1111/medu.12295

MacLeod, A., Cameron, P., Kits, O., & Tummons, J. (2019). Technologies of exposure: Videoconferenced distributed medical education as a sociomaterial practice. Academic Medicine, 94(3), 412-418. https://doi.org/10.1097/ACM.0000000000002536 

*Caroline CP Ong
KK Women’s and Children’s Hospital,
100 Bukit Timah Road,

Singapore 229899
Tel: +65 63941113
Fax: +65 62910161
Email: Caroline.ong.c.p@singhealth.com.sg

Announcements