Entrustable Professional Activities implementation in undergraduate allied health therapy programs

  • Share
Print Article   Download PDF

Number of Citations:

Submitted: 24 January 2023
Accepted: 2 August 2023
Published online: 2 January, TAPS 2024, 9(1), 42-48
https://doi.org/10.29060/TAPS.2024-9-1/SC2997

Rahizan Zainuldin1 & Heidi Siew Khoon Tan1,2

1Health and Social Sciences Cluster, Singapore Institute of Technology, Singapore; 2Pre-Professional Education Office, Tan Tock Seng Hospital, Singapore

Abstract

Introduction: Singapore Institute of Technology’s undergraduate (UG) occupational therapy (OT) and physiotherapy (PT) programs are one of the first implementors of Entrustable Professional Activities (EPAs) in the respective allied health professions training. The aim of the paper is to report the outcomes of the first year of EPAs implementation in clinical practice education (CPE) and share next steps refining implementation.

Methods: A quality improvement (QI) study using the Plan-Do-Check-Act (PDCA) cycle was conducted. UG OT Year 2 and Year 3 students, UG PT Year 3 students and their clinical educators (CEs) who experienced the use of EPAs for the first time were surveyed at the end of the clinical block.

Results: There was generally high agreement (>70% agreed or strongly agreed) among all groups in using EPAs to better understand the learning objectives of CPE and practice expectations as future entry-level practitioners at conditional-registration. More than 70% of OT respondents but less than 50% PT respondents found the EPA assessment forms easy to use. Less than 60% of both program CEs did not include colleagues for EPA assessments. 55% of both OT and PT CEs found the EPA training and resources adequate. Overall, PT respondents showed lower agreement than OT respondents in five survey items.

Conclusion: The first implementation cycle of EPA in the undergraduate OT and PT CPE had mixed acceptability to the EPA assessment tools. Three strategic changes were made for the second implementation cycle., i.e., redesign of EPA-based assessment forms, training focus and ‘just-in-time’ training with streamlined resources.

Keywords:           Clinical Training, Entrustable Professional Activities, Occupational Therapy, Physiotherapy, Undergraduate, Workplace-based Assessment

I. INTRODUCTION

In 2021, the occupational therapy (OT) and physiotherapy (PT) undergraduate programs at Singapore Institute of Technology (SIT) added a novel assessment, Entrustable Professional Activities (EPAs), to the extant competency-based assessment tools in clinical practice education (CPE). EPAs are units of professional activities entrusted to a learner determined by five levels of supervision, once the learner has demonstrated the required competence (ten Cate & Taylor, 2020). OT EPAs and PT EPAs (Zainuldin & Tan, 2021) were developed and introduced to SIT CPE as part of the Ministry of Health’s review of healthcare professions’ training standards. EPA-based assessments are relevant in CPE where students perform professional activities at workplace, supervised by onsite clinical educators (CEs). Previous CPEs assessed only OT and PT student competencies using the validated Student Practice Evaluation Form-Revised Edition (SPEF-R) (Turpin et al., 2011) and the Clinical Competency and Reasoning Assessment (CCRA), respectively. Conceptually, the pairing of EPAs with SPEF-R or CCRA potentially offer CEs an opportunity to empower students through graduated levels of entrustment supported by appropriate proficiency levels. Operationally, EPA assessment does not add new activities. OT and PT CEs can utilise routine observations of students’ tasks, case discussions and case-notes documentation as sources to inform entrustment levels in EPAs.

No EPA implementation in any OT and PT curricula has been documented. At SIT, EPA implementation in CPE needs evaluation. Recognising that implementing process changes requires an iterative approach, SIT embarked on a quality improvement study using the Plan-Do-Check-Act (PDCA) cycle. This paper reports the results of operationalising EPAs for the first time in OT and PT CPE, including the use of EPA-based assessment forms. The Methods section describes the Plan and Do, followed by the Results section reporting outcomes of the Check and the Discussion section highlighting the Act to improve implementation.

II. METHODS

A. CPE Structure

OT CPE consists of four blocks of seven weeks each and interspersed between academic modules in Years 2, 3 and 4. PT CPE consists of five consequent blocks (four core and one elective) of six weeks each, begins only after all academic modules are completed in Year 3 and continues to Year 4. OT and PT students complete different clinical settings for each CPE block.

B. Participants and Study Design

OT Year 2 and Year 3 students, PT Year 3 students and their CEs who experienced EPA use for the first time were surveyed at the end of a clinical block. An online EPA survey is incorporated with routine post placement feedback for both students and CEs, therefore no consent was required. The QI study was exempt from ethics review (SIT Institutional Review Board, No. 2022122). Survey results were extracted from February to December 2021.

C. PDCA Cycle: Plan-Do

OT and PT have five core EPAs each. EPA-based assessment activities are short practice observations, entrustment-based discussions and case-notes evaluations. These activities serve as sources of information (SOIs), or workplace-based assessments (WBAs) in OT CPE, to inform entrustment decision-making. OT CE assesses EPAs by documenting in a single patient case form with all three WBAs per EPA. PT CE assesses EPAs for every patient case anchored by three different SOI forms with written justifications. OT and PT CEs and students were trained on nuts and bolts of EPAs and on using WBA/SOI forms in CE training workshops and student pre-CPE briefing, respectively.

Each OT EPA requires a total of six patient cases entrusted to students at Level 3 entrustment (indirect supervision) across four CPE blocks. Each PT EPA requires six cases at Level 3 entrustment at each core clinical block, which totals 24 cases by end of the program. Appendix 1 and 2 provides visualisation of EPA implementation across multiple CPE blocks.

The EPA survey has ten items. The first eight items are scored on a 4-point Likert-scale (strongly disagree-strongly agree). The final two questions seek qualitative feedback on benefits and challenges and suggestions for improvements. Unless indicated, items are phrased in the same manner in both student and CE surveys.

D. Data Collection and Analysis

Data were counted as proportions of respondents who agreed (pooled response from ‘agree’ and ‘strongly agree’) and proportions disagreed (pooled from ‘disagree’ and ‘strongly disagree’). The authors grouped the qualitative narrative into benefits and challenges.

III. RESULTS

A. PDCA Cycle: Check

There were 99.0% response rate from OT Year 2 students (105/106), 97.7% from Year 3 OT students (85/87), 93.2% from PT Year 3 students (137/147), 98.5% from OT CEs (199/202) and 92.5% from PT CEs (247/267). Proportion of respondents who agreed with each item statement is shown in Table 1. Data on item scores for each student and CE are available at online repository, http://doi.org/10.6084/m9.figshare.21941288

Survey Items

OT Year 3 students

OT Year 2 students

PT Year 3 students

OT CEs

 

PT CEs

 

(n = 85)

(n = 105)

(n = 137)

(n = 199)

(n = 247)

Q1 – Using EPAs in CPE helps me better understand and meet future conditional-registration requirements.

 

90.6

98.1

75.2

89.4

71.7

Q2 – The EPA documents help me to better understand the learning objectives in CPE.

 

84.7

98.1

72.3

75.9

71.7

Q3 – The WBA/SOI forms are easy to use.

 

76.5

85.7

38.7

73.4

47.8

Q4 – CE: The WBA/SOI forms are adequate for me to determine students’ competence and entrustment level. /

Student: The WBAs/SOIs help me to better gauge my progress and level of competence.

 

91.8

97.1

70.1

72.9

64.6

Q5 – I understand the connection between OT EPAs and SPEF-R2 competencies or PT EPAs and CCRA.  

 

84.7

97.1

72.3

69.8

83.0

Q6 – CE: I use the EPA documents explicitly with students during clinical teaching and assessment. / Student: I use the EPA documents to guide my learning goals during CPE.   

 

62.4

92.4

51.1

87.9

59.1

Q7 – CE: I involve other colleagues in doing WBAs/SOIs to calibrate students’ entrustment level. / Student: Besides my CE, I also received feedback from other OTs or other PTs who were involved in my WBAs/SOIs.

 

44.7

66.7

54.0

58.8

45.3

Q8 – I feel the current briefing/training/resources are adequate for me to incorporate the use of EPAs in CPE.   

70.6

89.5

56.2

55.3

55.9

Table 1. Proportion of OT and PT students and CEs who agree with the EPA survey items

PT CEs and students were almost unanimous that SOI forms were difficult to use (Q3). Common to OT and PT CEs, many did not involve colleagues in EPAs (Q7) and felt that training to understand EPAs was inadequate (Q8).

Qualitatively, both disciplines benefitted from the use of WBA/SOI forms to scaffold learning through structured feedback and action plans when addressing identified competency gaps. Feedback from OT and PT students below closely exemplified the appreciation:

“EPAs allow me to track my progress over the weeks and transfer my reflections into action when given the opportunity to receive objective and qualitative feedback from the EPA form.”

OT Student#45

“The discussions with the CE on what to do if the situation was different made me realise the importance of planning even for the worst-case scenario…enabled me to identify the gaps in knowledge and skills that had to be worked on.”

PT Student#67

However, PT groups cited complicated forms design and copious paperwork from numerous SOIs time-consuming and stressful. Ambivalence on its practicality was best summed by PT CE#31, “As a first-time user of the SOIs, I found it quite difficult to navigate the forms, took me some tries to understand how I can determine the students’ competence and entrustment level. As there were many forms, it was quite confusing, and hence stressful and time-consuming. Otherwise, they are useful tools.”

The most common challenge among OT CEs was assessing certain EPAs, such as planning care transition, in some settings. “Some EPAs are harder to do in some settings, for example, in the hands therapy setting; it is harder to do the handover/discharge EPA as there are less of these patients.” (OT CE#32). Calling for more support, one CE suggested “SIT go through a round of training on the different EPAs and give relevant case examples to help us better understand them.” (OT CE#4).

IV. DISCUSSION

Response rates were excellent. The convergence of high agreement rates with narrative feedback on using EPAs and WBA/SOIs for teaching/learning, understanding the CPE learning objectives and meeting practice expectations as future entry-level practitioners suggest early indication that EPAs may facilitate SIT OT and PT students transit to new practitioners. The positive experience in this regard resonated with other EPA survey on final-year dietetics students and their clinical supervisors in Australia (Bramley et al., 2021). Practical challenges with the SOI forms, resulting in onerous and time-consuming evidence collection; low levels in involving colleagues in EPA assessments; and inadequate EPA training/resources for CEs were identified as key areas for change in both disciplines.

A. PDCA Cycle: Act

First, to reduce assessment burden, WBA and SOI forms were redesigned and harmonised in preparation towards a standardised EPA online assessment system currently developed in-house. Multiple WBA/SOI forms were combined into a single-page checklist form with a small open-ended section. A checklist was similarly suggested for nursing EPAs assessment, citing convenience as a reason (Lau et al., 2020). On the single-page form, CEs tick entrustment levels for each WBA/SOI associated to each EPA with all EPAs on the same page. The only narrative section is where CEs describe key justifications supporting their entrustment decisions, followed by students’ reflections. Second, to bridge assessment expectations among clinicians and increase propensity to share EPA assessments with colleagues, EPA training was refined to emphasise balance of supervision control with autonomy and clearer definitions between entrustment levels 2 (direct supervision) and 3 (indirect supervision) through case examples. Third, ‘just-in-time’ refresher training was added to activate volition in assessing EPAs. Toolkits containing briefing videos and streamlined resources in short bites, such as 3-minute videos, powtoons, form samplers and frequently-asked-questions, were released for OT and PT CEs closer to placement block. PT CEs also received a refresher at early weeks of every placement block.

V. CONCLUSION

The PDCA cycle is used to inform and make iterative adaptations to each cycle of EPA implementation. The Plan-Do stage completed the first implementation cycle of EPA in the undergraduate OT and PT CPE in 2021. The Check stage revealed mixed experiences to EPA use. The lowest agreement was the ease of using SOI forms among PT students and CEs. While EPAs were accepted as teaching and learning tools, CEs did not involve colleagues in EPA assessments. Training on EPA assessment for CEs was inadequate. Consequently, the Act stage yielded changes in form design, training focus and streamlined resources for the next implementation cycle.

Notes on Contributors

Rahizan Zainuldin (RZ) led the design of the quality improvement and implementation of EPAs, submitted the study to SIT IRB, analysed and interpreted both quantitative and qualitative data for the PT CPE, prepared the manuscript, wrote the initial draft and finalised for submission.

Heidi Siew Khoon Tan (HSKT) led the design of the EPA survey, analysed and interpreted both quantitative and qualitative data for the OT CPE, provided a critical review of the manuscript, and concurred on the final version.

Ethical Approval

The QI study was exempt from ethics review (SIT Institutional Review Board, No. 2022122).

Data Availability

Data on item scores for each student and CE are available at online repository, publicly accessed at http://doi.org/ 10.6084/m9.figshare.21941288. While the data is available for readers’ perusal and no permission from the authors is needed, please write an email of intention to use the data for any purposes to the corresponding author.

Acknowledgement

We would like to acknowledge the OT and PT CPE committee members who have contributed in the planning and implementation of the EPA. We would also like to thank Ms Annie Wang Haiyan, Manager, Academic Programmes Administration, SIT, for uploading the survey on and extracting the survey results from the online assessment portal of our Clinical Practice Education portal. 

Funding

No funding source is provided.

Declaration of Interest

Rahizan Zainuldin and Heidi Siew Khoon Tan disclose there is no conflict of interest of any form.

References

Bramley, A. L., Thomas, C. J., McKenna, L., & Itsiopoulos, C. (2021). E-portfolios and entrustable professional activities to support competency-based education in dietetics. Nursing and Health Sciences, 23(1), 148-156. https://doi.org/10.1111/nhs.12774

Lau, S. T., Ang, E., Samarasekera, D. D., & Shorey, S. (2020). Evaluation of an undergraduate nursing entrustable professional activities framework: An exploratory qualitative research. Nurse Education Today, 87, Article 104343. https://doi.org/10.1016/j.nedt.2020.104343

ten Cate, O., & Taylor, D. R. (2020). The recommended description of an entrustable professional activity: AMEE Guide No. 140. Medical Teacher 43(10), 1106-1114. https://doi.org/10.1080/0142159X.2020.1838465

Turpin, M., Fitzgerald, C., & Rodger, S. (2011). Development of the Student Practice Evaluation Form Revised Edition Package. Australian Occupational Therapy Journal, 58(2), 67-73. https://doi.org/10.1111/j.1440-1630.2010.00890.x

Zainuldin, R., & Tan, H. Y. (2021). Development of entrustable professional activities for a physiotherapy undergraduate programme in Singapore. Physiotherapy, 112, 64-71. https://doi.org/10.1016/j.physio.2021.03.017

*Rahizan Zainuldin
10 Dover Road,
Singapore 138683
+6596522418
Email: Rahizan.Zainuldin@singaporetech.edu.sg

Announcements