{"id":71006,"date":"2021-09-13T14:36:20","date_gmt":"2021-09-13T06:36:20","guid":{"rendered":"https:\/\/medicine.nus.edu.sg\/taps\/?post_type=issues&#038;p=71006"},"modified":"2022-05-12T08:15:46","modified_gmt":"2022-05-12T00:15:46","slug":"development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students","status":"publish","type":"issues","link":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/","title":{"rendered":"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students"},"content":{"rendered":"<p>Submitted: 31<span lang=\"EN-GB\"> August 2020<\/span><br \/>\r\nAccepted: 8 February 2021<br \/>\r\nPublished online: <span lang=\"EN-GB\">5 October, TAPS 2021, <\/span><span lang=\"EN-GB\">6(4), 118-130<\/span><br \/>\r\n<a href=\"https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/OA2481\">https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/OA2481<\/a><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Isabella E. Supnet, Jose Alvin P. Mojica, Sharon D. Ignacio &amp; Carl Froilan D. Leochico<\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><i><span lang=\"EN-GB\">Department of Rehabilitation Medicine, College of Medicine and Philippine General Hospital, University of the Philippines Manila, Philippines<\/span><\/i><\/p>\r\n<p style=\"text-align: center\"><b><span lang=\"EN-GB\">Abstract<\/span><\/b><\/p>\r\n<p style=\"text-align: justify\"><b><span lang=\"EN-GB\">Introduction:<\/span><\/b><i><span lang=\"EN-GB\"> <\/span><\/i><span lang=\"EN-GB\">In the Philippines, telerehabilitation has been at the forefront of integrating telemedicine into the medical curriculum. However, the course evaluation tool used for traditional classroom-based courses is not appropriate in evaluating the unique teaching-learning tool that is telerehabilitation. This study aimed to develop a questionnaire that will aid in addressing this gap.<\/span><\/p>\r\n<p style=\"text-align: justify\"><b><span lang=\"EN-GB\">Method:<\/span><\/b><i><span lang=\"EN-GB\"> <\/span><\/i><span lang=\"EN-GB\">A mixed methods study was devised to gather information from medical students exposed to telerehabilitation from the College of Medicine, University of the Philippines Manila as well as the residents from the Department of Rehabilitation Medicine, Philippine General Hospital from October to November 2019. The investigators obtained informed consent from all participants as well as their demographics before undergoing interviews. Themes were identified to create questions under the previously identified constructs, along with items derived from other course evaluation tools and opinions gathered from experts in telerehabilitation.<\/span><\/p>\r\n<p style=\"text-align: justify\"><b><span lang=\"EN-GB\">Results:<\/span><\/b><span lang=\"EN-GB\"> In total, 26 individuals participated in the study. Most of the respondents had experience or ownership of various communication technologies and were well-versed in communication strategies through these technologies. There were 52 questions formulated from the interviews and review of previous tools. <\/span><\/p>\r\n<p style=\"text-align: justify\"><b><span lang=\"EN-GB\">Conclusion:<\/span><\/b><span lang=\"EN-GB\"> This study is the first step in providing more research in the student evaluation of telerehabilitation and telemedicine. <\/span><span lang=\"EN-GB\">To match the needs of changing times evaluation of new standards and methods should follow. More research must be done to standardise teaching evaluation tools to validate the data gathered, and allow courses, such as telerehabilitation, an opportunity to adapt and promote further learning.<\/span><\/p>\r\n<p style=\"text-align: justify\"><b><span lang=\"EN-GB\">Keywords<\/span><\/b><b><span lang=\"EN-GB\">:<\/span><\/b><span lang=\"EN-GB\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span><i><span lang=\"EN-GB\">Assessment, Educational, Education, Medical, <\/span><\/i><i><span lang=\"EN-GB\">Telemedicine, Medical Informatics, Physical and Rehabilitation Medicine<\/span><\/i><span lang=\"EN-GB\"><\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">Practice Highlights<\/span><\/strong><\/p>\r\n<ul>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">Telemedicine has been used as a tool to educate students on rehabilitation medicine.<\/span><\/li>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">Almost all students at this time have had experience in the usage of information and communication technologies.<\/span><\/li>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">Evaluation tools must be more specific to the method of instruction used.<\/span><\/li>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">Clarity, congruence, and relevance are the most sought-after characteristics in telerehab courses.<\/span><\/li>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">The recipient of the evaluation must account for the effect of the teacher or logistical concerns.<\/span><span lang=\"EN-GB\"><\/span><\/li>\r\n<\/ul>\r\n<p style=\"text-align: center\"><strong><span lang=\"EN-GB\">I. INTRODUCTION<\/span><\/strong><\/p>\r\n<p><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">A. Background<\/span><\/i><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Telemedicine is defined as the use of advanced telecommunications technologies to exchange health information and provide health care services across geographic, temporal, social and cultural barriers <\/span><span lang=\"EN-GB\">(Myers, 2003)<\/span><span lang=\"EN-GB\">. It has been widely used in the treatment and care of patients as we go through the COVID-19 pandemic and has also been vital in improving medical education in the age of online classes and social distancing. Through telemedicine, students are exposed to a variety of specialties, and gain experience in their observation of the management of diseases, such as in the case of telesurgery, or hone their skills in communication and counselling, such as through telerehabilitation <\/span><span lang=\"EN-GB\">(Jumreornvong et al., 2020)<\/span><span lang=\"EN-GB\">.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Now more than ever, due to the capacity of students to interact and adapt with digital infrastructure <\/span><span lang=\"EN-GB\">(Pathipati et al., 2016)<\/span><span lang=\"EN-GB\">, integration of telemedicine into the medical curriculum is increasingly important and evaluations of telemedicine curricula are necessary to ensure quality and to detect areas for growth and improvement. Previous studies have used general evaluation forms to evaluate their telemedicine programmes or created general surveys from their course objectives, which have yielded some valuable insights <\/span><span lang=\"EN-GB\">(Brockes et al., 2017; Bulik &amp; Shokar, 2010)<\/span><span lang=\"EN-GB\">. However, due to the nature of instruction of telemedicine, specialised and validated tools are necessary to provide a comprehensive assessment.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">In the Philippines, telerehabilitation has been at the forefront of integrating telemedicine into the curriculum, particularly in the University of the Philippines. Because it has been four years since the start of the initiative, and one year since its implementation, it is important to evaluate the previous courses in order to improve them for the next generation.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Through this research, a preliminary evaluation questionnaire for telerehabilitation as a teaching-learning tool was developed. From this questionnaire, perceptions of students regarding telerehabilitation courses will be more efficiently gathered and evaluated, which will serve to further improve the telerehabilitation curriculum and possibly bring forth interventions to improve medical education in general.<\/span><\/p>\r\n<p><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">B. Literature Review<\/span><\/i><\/p>\r\n<p style=\"text-align: justify\"><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">1) Teaching programmes in telemedicine: <\/span><\/i><span lang=\"EN-GB\">In the process of conducting telemedical evaluations in order to aid patients, clinical training may also be received. Telemedicine has had good acceptance in training institutions abroad, with Neurology trainees agreeing that it should be part of their curriculum and supporting a formalised telemedicine rotation within their residency. Dermatology programmes in particular observed that telemedicine supported rather than detracted from the core competencies required from them <\/span><span lang=\"EN-GB\">(Lee &amp; Nambudiri, 2019)<\/span><span lang=\"EN-GB\">. No significant differences were seen between clinical outcomes of patients who underwent surgery through telementoring versus the traditional method, and case supervision using e-mails and voice-over applications afforded similar psychiatry education as compared to rotations in mental health clinics <\/span><span lang=\"EN-GB\">(O\u2032Shea et al., 2015)<\/span><span lang=\"EN-GB\">. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The methods through which telemedicine education is given also vary considerably per institution <\/span><span lang=\"EN-GB\">(Waseh &amp; Dicker, 2019)<\/span><span lang=\"EN-GB\">. Some schools have telemedicine included primarily as didactic sessions. Others allow their students to take part in patient encounters and interprofessional training. Some institutions also allow scholarly projects to be done in telemedicine.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Because telemedicine in medical education has yet to be explored formally, no evaluation tools have been developed to assess its application. Other institutions used generalised forms and made their own questionnaires <\/span><span lang=\"EN-GB\">(Brockes et al., 2017; Bulik &amp; Shokar, 2010)<\/span><span lang=\"EN-GB\">. Literature review for formally developed evaluation materials showed the presence of a Telehealth Usability Questionnaire, with items and domains encompassing many telehealth assessment tools <\/span><span lang=\"EN-GB\">(Parmanto et al., 2016)<\/span><span lang=\"EN-GB\">. It was made for use with various types of telehealth systems, including computer-based systems, videoconferencing programmes, and adaptable for progressive innovations, particularly for mobile telemedicine applications. However, this was primarily utilised as an assessment tool between clinicians and patients and does not assess instruction on the usage of the programme or the organisation of the implementation.<\/span><\/p>\r\n<p style=\"text-align: justify\"><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">2) Teaching programmes in telerehabilitation:<\/span><\/i><span lang=\"EN-GB\"> In 2015, the Department of Rehabilitation Medicine, College of Medicine and Philippine General Hospital, University of the Philippines Manila (PGH DRM) has initiated education on this aspect of telemedicine, with telerehabilitation used as a teaching-learning tool for medical students in full implementation in the 2018 curriculum. The programme has expanded from its origins since then; from students engaging in telerehabilitation to actual telerehabilitation consultations and teletherapy services with the rural health unit of Alfonso, Cavite, as part of the University of the Philippines Community Health and Development Programme (UP CHDP). <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Telerehabilitation as a teaching-learning tool was formally included as part of the curriculum for rehabilitation medicine in 2018 <\/span><span lang=\"EN-GB\">(Philippine General Hospital Department of Rehabilitation Medicine, 2018)<\/span><span lang=\"EN-GB\">. The students start their engagement during their third year of medical school (known as Learning Unit 5), their first year of clinical exposure, with the concepts and theories behind telerehabilitation, and do observations of an actual telerehabilitation session. A year later, during Learning Unit 6, they then get to do a simulated telerehabilitation encounter, with guidance and techniques on how to present a rehabilitation case through telemedicine. During their final year of medical education (Learning Unit 7), the student is then evaluated on the presentation, evaluation and management given during an actual patient encounter. All students from the UP College of Medicine receive two sessions of telerehabilitation instruction per year.\u00a0 On the other hand, Post-Graduate Interns (PGIs), who are students who have received their first four years of medical education in other medical schools and have chosen to spend their last year of medical school in the Philippine General Hospital, receive only one compressed session. Overall, the participants reported the experience to be excellent, and should be explored further <\/span><span lang=\"EN-GB\">(Leochico &amp; Mojica, 2017)<\/span><span lang=\"EN-GB\">. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Telerehabilitation was also included in January 2019 as a formal service for training residents in the Department of Rehabilitation Medicine, featuring the same concepts in a more compressed manner, and highlighting the hybridisation of standard rehabilitation practice with telemedicine.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The initiative is currently being evaluated by the students in part through the Course Evaluation by Students (CEBS) given by the University of the Philippines College of Medicine (UPCM) <\/span><span lang=\"EN-GB\">(University of the Philippines Manila College of Medicine, 2005)<\/span><span lang=\"EN-GB\">, and through a Devised Telerehabilitation Feedback Form for Students <\/span><span lang=\"EN-GB\">(Philippine General Hospital Department of Rehabilitation Medicine, n.d.)<\/span><span lang=\"EN-GB\">, which does not distinguish usability attributes of the telerehabilitation system from the telerehabilitation curriculum itself.<\/span><\/p>\r\n<p style=\"text-align: justify\"><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">3) The usage of student evaluations:<\/span><\/i><span lang=\"EN-GB\"> Student evaluation forms are a commonly used tool in determining teacher and course effectiveness in many areas of higher education. These evaluations commonly serve three purposes: to improve teaching quality, to support faculty decisions, and to provide evidence for institutional accountability. Many evaluation measures have been constructed and performed with students as the main and sometimes sole indicator of education quality; however, the implementation, reliability and validity of these methods and instruments have been a source of concern within academic circles <\/span><span lang=\"EN-GB\">(Soto-Estrada et al., 2018)<\/span><span lang=\"EN-GB\">. Due to this, student evaluation remains a field of active study.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Although helpful, some caution must be undertaken in interpreting results of student evaluations. A recent meta-analysis by Uttl et al. (2017) argues that studies looking into student evaluations of teaching (SETs) were scant in terms of data to support the equivalence of high student evaluation ratings to student performance, and positive correlations between the two were primarily due to small study size effects. However, it is unfair to assume that SETs have no value whatsoever. Responses to the meta-analysis have been published as well, and aside from critiquing the method Uttl used to form his study, they support the view that, while it is poor practice to use student ratings of instruction alone in evaluating teaching, it remains to be necessary <\/span><span lang=\"EN-GB\">(Ryalls et al., n.d.)<\/span><span lang=\"EN-GB\">.<\/span><\/p>\r\n<p><i><span lang=\"EN-GB\">C. Objectives<\/span><\/i><\/p>\r\n<p><i><span lang=\"EN-GB\"><\/span><\/i><i><span lang=\"EN-GB\">1) General objective:<\/span><\/i><\/p>\r\n<ul>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">To develop a questionnaire that will evaluate telerehabilitation as a teaching-learning tool for medical students.<\/span><i><span lang=\"EN-GB\"><\/span><\/i><\/li>\r\n<\/ul>\r\n<p><i><span lang=\"EN-GB\">2) Specific objectives:<\/span><\/i><\/p>\r\n<ul>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">To determine themes regarding the evaluation of telerehabilitation as a teaching-learning tool from residents of the PGH Department of Rehabilitation Medicine, as well from the medical students who have undergone the telemedicine courses.<\/span><\/li>\r\n\t<li style=\"text-align: justify\"><span lang=\"EN-GB\"> <\/span><span lang=\"EN-GB\">To formulate questionnaire items for the telerehabilitation course evaluation tool.<\/span><span lang=\"EN-GB\"><\/span><\/li>\r\n<\/ul>\r\n<p style=\"text-align: center\"><strong><span lang=\"EN-GB\">II. METHODS<\/span><\/strong><\/p>\r\n<ol><\/ol>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">A mixed methods study was devised to gather information from medical students exposed to telerehabilitation from the Philippine General Hospital and the College of Medicine, University of the Philippines Manila (with exposures corresponding to Learning Units 5, 6, and 7) as well as the residents from the PGH DRM from October to November 2019 (Figure 1). All participants were greater than 18 years old.<\/span><\/p>\r\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png\" alt=\"\" class=\"wp-image-71009 aligncenter\" width=\"739\" height=\"341\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-1024x474.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-768x355.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-1536x710.png 1536w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1.png 1695w\" sizes=\"auto, (max-width: 739px) 100vw, 739px\" \/><\/p>\r\n<p style=\"text-align: center\"><span lang=\"EN-GB\">Figure 1. Flowchart of study methodology<\/span><\/p>\r\n<div>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Individuals with no telerehabilitation programme exposure were excluded from the study, and participants were given the option to withdraw their participation at any time. Convenience sampling was employed, and the duration of involvement of each participant with the study was limited to one day.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The students and residents were personally approached at the Department of Rehabilitation Medicine by the principal investigator to inform them of the ongoing study. The rationale of the study and the process the prospective participant will undergo were also explained, as well as any benefits of the research, such as the capacity of the participants to be involved in the improvement of telerehabilitation programmes. It was also explained that refusal to participate will not affect their relationship with the department, the Philippine General Hospital, or the College of Medicine. Informed consent forms were then obtained from each individual prior to their participation by the principal investigator. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">All participants of the focus group discussions were also requested to accomplish a Data Collection Form asking for their age, sex and identified subgroup of exposure to the telerehabilitation programme. This form included scales rating the individual\u2019s exposure to information and communication technologies and social media, and their prior experience with telemedicine\/telehealth. Notes and voice recordings were taken of the group discussions and key informant interview to facilitate transcription and analysis. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">An interview guide was constructed through the guidance of the expert faculty undertaking the study, keeping in mind Tyler\u2019s model of curriculum development and Kirkpatrick\u2019s Four Level Evaluation Model. This initial list of questions was intended to be able to differentiate opinions on telemedicine courses versus regular courses and be open-ended to facilitate discussion. The cognitive interview was then performed with three individuals representing telerehabilitation programme exposure from Learning Units 5, 6, and 7. The initial interview guide was presented to them, and edits were made to improve clarity of the questions.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">All discussions and interviews were held at the PGH DRM offices and outpatient department. Groups of students consisting of seven to eight participants each were scheduled for their interviews during their available time. Before the start of the group discussions, the rationale and the procedure of the discussion was again explained to the students, and time was allotted for them to prepare and ask any questions. Informed consent forms were then distributed, and data collection forms given to the students, which they were given ample time to fill out. The revised interview guide was then asked sequentially, with additional questions added to further probe for the participants\u2019 thoughts and opinions. After the group discussion, important points were summarised and clarified with the participants. The one-on-one key informant interviews were also performed in a similar manner.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">All discussions were then analysed using Microsoft Excel for repeating themes. These were then presented to the expert faculty of the department \u2013 those who are well versed in both medical education and evaluation, as well as telerehabilitation \u2013 for their opinion and review. After this, the final set of questions was collated and formatted. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Facilitation of the focus group discussions and key informant interviews, data collection, and data analysis was performed by the principal investigator, Dr. Supnet. As a graduating resident at the time of data collection, she is deemed a neutral party to the students and other residents. Processing of the interview recordings and transcription was done through the aid of the research assistant.<\/span><\/p>\r\n<p align=\"center\"><strong><span lang=\"EN-GB\">III. <\/span><span lang=\"EN-GB\">RESULTS<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Data gathered for this research was uploaded to an online repository for archiving and validation purposes. It may be accessed through the following Digital Object Identifier <\/span><span lang=\"EN-GB\">(DOI): <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.6084\/m9.figshare.13040786.v1\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.6084\/m9.figshare.13040786.v1<\/span><\/a><\/span><span lang=\"EN-GB\">\u00a0 <\/span><span lang=\"EN-GB\">(Supnet et al., 2020)<\/span><span lang=\"EN-GB\">.<\/span><\/p>\r\n<p><i><span lang=\"EN-GB\">A. Participant Demographics<\/span><\/i><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">In total, 32 individuals were part of the study, with representatives from all Learning Units (LUs) including the Post-Graduate Interns (PGIs) of the Philippine General Hospital, as well as three residents from the Department of Rehabilitation Medicine. For the individuals who took part in the focus group discussions, the average age was 25.31 (2.28) years, ranging from 21 to 31 years old. Furthermore, most of the participants were female. Most respondents came from the post-graduate interns and the LU5 students, with 7 participants each (Table 1).<\/span><\/p>\r\n<\/div>\r\n<div align=\"center\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1-300x119.png\" alt=\"\" class=\"alignnone  wp-image-71010\" width=\"650\" height=\"258\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1-300x119.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1-1024x405.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1-768x304.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1-1536x607.png 1536w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-1.png 1730w\" sizes=\"auto, (max-width: 650px) 100vw, 650px\" \/><br \/>\r\n<p align=\"center\"><span lang=\"EN-GB\">Table 1. Participant Demographics<\/span><\/p>\r\n<div>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">All of the focus group respondents owned a computer and a cell phone, while none owned a virtual reality system. Most of the respondents also had experience using telephones, cell phones, computers, tablets, and video gaming systems, and were well-versed in messaging systems and social media. When it comes to telemedicine, however, exposure was drastically lower &#8211; less than half of the participants had heard of telemedicine prior to the telerehabilitation programme in PGH, while only two participants had used telemedicine applications before. Further, only two of the participants had heard of telerehabilitation before the Department of Rehabilitation Medicine\u2019s curriculum and were able to gain this experience through electives and courses abroad (Table 2).<\/span><\/p>\r\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1-300x144.png\" alt=\"\" class=\" wp-image-71011 aligncenter\" width=\"675\" height=\"324\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1-300x144.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1-1024x491.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1-768x368.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1-1536x737.png 1536w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-1.png 1637w\" sizes=\"auto, (max-width: 675px) 100vw, 675px\" \/><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2-300x92.png\" alt=\"\" class=\"wp-image-71013 aligncenter\" width=\"661\" height=\"202\" srcset=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2-300x92.png 300w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2-1024x315.png 1024w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2-768x236.png 768w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2-1536x472.png 1536w, https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Table-2-Part-2.png 1632w\" sizes=\"auto, (max-width: 661px) 100vw, 661px\" \/><\/p>\r\n<p align=\"center\"><span lang=\"EN-GB\">Table 2. Exposure of participants to Information and Communication Technologies (ICT) and Telemedicine or Telerehabilitation.<\/span><\/p>\r\n<p><i><span lang=\"EN-GB\">B. Focus Group Discussions and Key Informant Interview<\/span><\/i><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">A cognitive interview was performed with three students, one from each of the Learning Units with clinical exposure, to represent various stages of telerehabilitation curriculum. Aside from participating in the analysis of each question, they also improved on the clarity of the wording: for example, \u201cDoes the course being focused on telemedicine\u2026\u201d in their original iteration, the questions have been amended to start with \u201cIn a telemedicine course\u2026\u201d to preface the rest of the question. The participants of the cognitive interview also gave comments and suggestions to the author regarding follow up questions that may be used to further probe into the answers of the respondents. This final interview guide may be found in Appendix A.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Once the interview guide had been finalised, focus group discussions were then performed with students from Learning Units 5, 6, and 7, in three groups consisting of seven to eight participants. The same interview guide was also used for the key informant interviews of the residents. After the sessions were completed, the notes and recordings from the sessions were reviewed and transcribed. Identification of repeating themes and ideas was then performed (Appendix B), and eventually these were transformed into 52 questions that were presented to the three expert faculty members of the Department of Rehabilitation Medicine who gave their opinions on the subject matter. Questions were made through expanding the statements of the minor categories and adding nuances for clarification: distinguishing between the timeliness of giving the results of the evaluation and the conduct of feedback sessions, for example. After applying the suggestions from two rounds of discussion primarily relating to the wording and understandability of the questions, all members of the expert faculty group signified their agreement regarding the content of the final evaluation tool.\u00a0 This final evaluation tool may be found formatted into Likert scales in Appendix C.<\/span><\/p>\r\n<p style=\"text-align: center\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">IV. DISCUSSION<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">In the focus group discussions, the researchers found that many aspects of a good non-telemedicine course were also important to the respondents to find in telemedicine courses. Differences were found, however, in the increased emphasis for sustainability of the method, a greater focus on skills and communication training, the maximisation of multimedia and other methods to impart learning, and the usage of appropriate evaluation tools in telemedicine education.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">On assessing objectives, the students and residents found it very helpful to follow the SMART Framework (Doran, 1981) to ensure clarity of the goal in itself as well as the method expected to reach it. This desire for clarity may also be seen in the other statements in the same section, in terms of objectives being made known to the students as well as communicated in an understandable manner. Congruence of the telerehabilitation curriculum, in terms of how it plays with the medical curriculum in general as well as international standards, was also important for the students.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Clarity and congruence were also major themes for the rest of the domains, with statements for ease of understanding and congruence with objectives in the domain of course content, consistency across batches and defined rules, expectations, and schedules in the domain of organisation. The respondents also supported clear and standardised methods of evaluation in line with the objectives of the course, as well as the delivery of good feedback.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">In the domain of course content, the students also valued applicability of the course in terms of the common diseases they would see as well as its value to the community setting. Regarding organisation, the responders valued ease of communication with the organisers, breaking this theme down further as being composed of approachability and availability.\u00a0 Meanwhile, in results and feedback, timeliness was seen as an important concern.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">For the evaluation of the telerehabilitation curriculum as a whole, the students still valued congruence in terms of adherence to objectives. However, this general theme has been eclipsed by the desire to learn and to find relevance and utility in the learning they had received. This serves as stark contrast to a common misconception about student evaluations, which is that ratings are influenced by how \u201ceasy\u201d the course is; that is, teachers who give out higher grades get higher ratings. In a study involving eight academic disciplines and 50,000 classes, it has been found that the grade students expected to get was only weakly related to student ratings of instruction (Centra, 2003). The assumption therefore that teachers should succumb to student standards to get better evaluations is the inverse of the truth, and educators would be better served by challenging students, stimulating their interests, and making appropriate changes to their course or method of instruction based on feedback (Ryalls et al., n.d.).<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The premise of evaluation is to gather data to determine changes that improve methods to deliver education and increase student learning. The teacher is undoubtedly part of this process, as are technical aspects for a telemedicine course. However, it is important to distinguish the effects of these from the course itself on the learning experience of the student in order to find where exactly to improve. Recent efforts have also been done in order to shift phrasing of evaluation questions to focus more on the student and the course rather than the teacher, as the latter promotes the view of the student being a passive learner, expected to accept the expert-teacher\u2019s methods, instead of a responsible, proactive individual (Louie et al., 1996). The final questionnaire has been phrased as such to avoid the complications of a teacher-based approach as well as distinguish logistical concerns, with one teacher-based question included in the course content (i.e., \u201cThe lectures and facilitation of the faculty members enhanced my learning.\u201d) and two logistical questions, pertaining to concerns on internet and hardware as well as concerns on the telemedicine platform, added to course organisation instead (i.e., \u201cTechnical concerns (hardware and internet issues) are well addressed\u201d and \u201cUsage of telemedicine applications is simple and intuitive.\u201d). These are designed as screening markers instead of the focus of the entire evaluation and are intended to work with other evaluation forms of the College of Medicine of the University of the Philippines, which focus on teacher instruction, as well as evaluation forms of the PGH Department of Rehabilitation Medicine, which focus on service delivery.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">As much as the authors would have liked to have had more participants and an equal set of them in each group during the focused discussions, problems in student exposure as well as scheduling changes prevented them from attending the sessions. The abstract nature of the questions also made answering them less intuitive to the participants, needing guidance and prodding even after multiple revisions during the cognitive interview. Some participants have also commented that, due to being primed that the study would be more related to telemedicine, they found it difficult to separate their responses between a regular medicine course and a telemedicine one. All of this may be taken into consideration for others who would wish to expand on the processes and findings of this research.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">This study is only the first step in creating an evaluation tool for telemedicine courses \u2013 validity and reliability studies should follow to further refine the questions made and strengthen their research value. A pilot test should then follow, to further gather feedback on the evaluation tool and test its capabilities on a larger scale.<\/span><\/p>\r\n<p style=\"text-align: center\"><span lang=\"EN-GB\"><\/span><strong><span lang=\"EN-GB\">V. CONCLUSION<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Medical education, as with medicine, is an ever-advancing field. To match the needs of changing times, we expect that standards set and methods applied in teaching will change too. If this is the case, evaluation of these standards and methods should follow. This study is the first step in providing more research in the student evaluation of telerehabilitation in particular and telemedicine in general, especially since there is an expected increase in demand for quality education in these subjects in the future. More research must be done in order to standardise teaching evaluation tools in this aspect of medicine to validate the data gathered, and allow courses, such as telerehabilitation, an opportunity to adapt.<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Notes on Contributors<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Isabella Supnet, MD served as the primary investigator of this study, and reviewed the literature, designed the protocol, performed data collection, analysed the transcripts, and wrote the manuscript. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Jose Alvin Mojica, MD, MHPEd directed the team on important points in the literature review as well as the design of the protocol; he also gave feedback on the final manuscript. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Sharon Ignacio, MD advised the design of the study and gave critical feedback on the final paper. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Carl Froilan Leochico, MD assisted in the design of the protocol as well as the writing of the manuscript. <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">All authors have read and approved the final manuscript.<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Ethical Approval<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">This research was deemed exempt from ethical approval by the University of the Philippines Manila Research Ethics Board (UPMREB). UPMREB Code: 2019-427-01, 18-Sep-2019).<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Data Availability<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">In line with the aim of making research available to the scientific community for data replication and scientific progress, the dataset of this research may be found for free through the Figshare website: <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.6084\/m9.figshare.13040786.v1\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.6084\/m9.figshare.13040786.v1<\/span><\/a><\/span><span lang=\"EN-GB\"> <\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Acknowledgement<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The authors wish to thank the students who served as participants in the study. With their time and effort, knowledge about telemedicine education has advanced considerably.<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Funding<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The study did not receive any external funding.<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">Declaration of Interest<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">The authors have no conflict of interest to declare.<\/span><\/p>\r\n<p align=\"center\" style=\"text-align: center\"><strong><span lang=\"EN-GB\">References<\/span><\/strong><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Brockes, C., Grischott, T., Dutkiewicz, M., &amp; Schmidt-Weitmann, S. (2017). Evaluation of the education \u201cclinical telemedicine\/e-health\u201d in the curriculum of medical students at the University of Zurich. <i>Telemedicine and E-Health<\/i>, <i>23<\/i>(11), 899\u2013904. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1089\/tmj.2017.0011\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1089\/tmj.2017.0011<\/span><\/a><\/span><span lang=\"EN-GB\"> <\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Bulik, R. J., &amp; Shokar, G. S. (2010). Integrating telemedicine instruction into the curriculum: Expanding student perspectives of the scope of clinical practice. <i>Journal of Telemedicine and Telecare<\/i>, <i>16<\/i>(7), 355\u2013358. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1258\/jtt.2010.090910\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1258\/jtt.2010.090910<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Centra, J. A. (2003). Will Teachers receive higher student evaluations by giving higher grades and less course work? <i>Research in Higher Education<\/i>, <i>44<\/i>(5), 24. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1023\/A:1025492407752\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1023\/A:1025492407752<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Doran, G. (1981). There\u2019s a S.M.A.R.T. way to write management\u2019s goals and objectives. <i>Managment Review<\/i>, <i>70<\/i>(11), 35\u201336.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Jumreornvong, O., Yang, E., Race, J., &amp; Appel, J. (2020). Telemedicine and medical education in the age of COVID-19. <i>Academic Medicine<\/i>, <i>95<\/i>(12), 1838\u20131843. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1097\/ACM.0000000000003711\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1097\/ACM.0000000000003711<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Lee, M. S., &amp; Nambudiri, V. (2019). Integrating telemedicine into training: Adding value to graduate medical education through electronic consultations. <i>Journal of Graduate Medical Education<\/i>, <i>11<\/i>(3), 251\u2013254. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.4300\/JGME-D-18-00754.1\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.4300\/JGME-D-18-00754.1<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Leochico, C. F., &amp; Mojica, J. A. (2017). Telerehabilitation as a teaching-learning tool for medical interns. <i>PARM Proceedings<\/i>, <i>9<\/i>(1), 39\u201343.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Louie, B., Byrne, N., &amp; Wasylenki, D. (1996). From feedback to reciprocity. Developing a student-centered approach to course evaluation. <i>Evaluation and the Health Professions<\/i>, <i>19<\/i>(2), 231\u2013242. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1177\/016327879601900206\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1177\/016327879601900206<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Myers, M. B. (2003). Telemedicine: An emerging health care technology. <i>The Health Care Manager<\/i>, <i>22<\/i>(3), 219\u2013223.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">O\u2032Shea, J., Berger, R., Samra, C., &amp; Van Durme, D. (2015). Telemedicine in education: Bridging the gap. <i>Education for Health<\/i>, <i>28<\/i>(1), 64. <a href=\"https:\/\/doi.org\/10.4103\/1357-6283.161897\">https:\/\/doi.org\/10.4103\/1357-6283.161897<\/a><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Parmanto, B., Lewis, A. N., Jr., Graham, K. M., &amp; Bertolet, M. H. (2016). Development of the Telehealth Usability Questionnaire (TUQ). <i>International Journal of Telerehabilitation<\/i>, <i>8<\/i>(1), 3\u201310. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.5195\/IJT.2016.6196\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.5195\/IJT.2016.6196<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Pathipati, A. S., Azad, T. D., &amp; Jethwani, K. (2016). Telemedical education: Training digital natives in telemedicine. <i>Journal of Medical Internet Research<\/i>, <i>18<\/i>(7), e193. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.2196\/jmir.5534\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.2196\/jmir.5534<\/span><\/a><\/span><span><span lang=\"EN-GB\"><\/span><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Philippine General Hospital Department of Rehabilitation Medicine. (2018). <i>Instructional design for telerehabilitation<\/i>. Philippine General Hospital Department of Rehabilitation Medicine.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Philippine General Hospital Department of Rehabilitation Medicine. (n.d.). <i>Telerehabilitation feedback form for students<\/i>. Philippine General Hospital Department of Rehabilitation Medicine.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Ryalls, K., Benton, S., &amp; Li, D. (n.d.). <i>IDEA editorial note #3 \u2022 response to \u201cZero correlation between evaluations and learning.\u201d<\/i> IDEA.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Soto-Estrada, E., Wellens, A., &amp; G\u00f3mez-Lizarazo, J. (2018). Student course evaluation: A process-based approach. <i>Australasian Journal of Engineering Education<\/i>, <i>23<\/i>(2), 83\u201394. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1080\/22054952.2018.1557099\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1080\/22054952.2018.1557099<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Supnet, I., Leochico, C. F., Mojica, J. A., &amp; Ignacio, S. (2020). <i>Development of a telerehab evaluation tool\u2014Repository data<\/i> (p. 18014 Bytes) [Data set]. Figshare. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.6084\/M9.FIGSHARE.13040786.V1\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.6084\/M9.FIGSHARE.13040786.V1<\/span><\/a><\/span><span><span lang=\"EN-GB\"><\/span><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">University of the Philippines Manila College of Medicine. (2005). <i>Course evaluation by students (CEBS)<\/i>. University of the Philippines Manila College of Medicine.<\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Uttl, B., White, C. A., &amp; Gonzalez, D. W. (2017). Meta-analysis of faculty\u2019s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. <i>Studies in Educational Evaluation<\/i>, <i>54<\/i>, 22\u201342. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.1016\/j.stueduc.2016.08.007\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.1016\/j.stueduc.2016.08.007<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-GB\">Waseh, S., &amp; Dicker, A. P. (2019). Telemedicine training in undergraduate medical education: Mixed-methods review. <i>Journal of Medical Internet Research Medical Education<\/i>, <i>5<\/i>(1), e12515. <\/span><span lang=\"EN-ID\"><a href=\"https:\/\/doi.org\/10.2196\/12515\"><span lang=\"EN-GB\">https:\/\/doi.org\/10.2196\/12515<\/span><\/a><\/span><span lang=\"EN-GB\"><\/span><\/p>\r\n<p style=\"text-align: justify\"><span lang=\"EN-NZ\"><span lang=\"EN-AU\">*<span lang=\"EN-GB\">Isabella E. Supnet<\/span><br \/>\r\n<span lang=\"EN-GB\">Department of Rehabilitation Medicine,<\/span><br \/>\r\n<span lang=\"EN-GB\">Philippine General Hospital,<\/span><\/span><span lang=\"EN-AU\"><br \/>\r\n<span lang=\"EN-GB\">Taft Avenue, Manila<\/span><br \/>\r\n<span lang=\"EN-GB\">E-mail: isabella.supnet@gmail.com<\/span><br \/>\r\n<\/span><\/span><\/p>\r\n<\/div>\r\n<\/div>","protected":false},"featured_media":0,"parent":0,"menu_order":11,"template":"","issues_category":[12],"archive_category":[],"issue_type":[25],"volume_category":[61868],"class_list":["post-71006","issues","type-issues","status-publish","hentry","issues_category-original-articles","issue_type-past-issue","volume_category-volume-6-number-4-october-2021"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar\" \/>\n<meta property=\"og:description\" content=\"Submitted: 31 August 2020 Accepted: 8 February 2021 Published online: 5 October, TAPS 2021, 6(4), 118-130 https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/OA2481 Isabella E. Supnet, Jose Alvin P. Mojica, Sharon D. Ignacio &amp; Carl Froilan D. Leochico Department of Rehabilitation Medicine, College of Medicine and Philippine General Hospital, University of the Philippines Manila, Philippines Abstract Introduction: In the Philippines, telerehabilitation [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/\" \/>\n<meta property=\"og:site_name\" content=\"The Asia Pacific Scholar\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-12T00:15:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"22 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/\",\"name\":\"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar\",\"isPartOf\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png\",\"datePublished\":\"2021-09-13T06:36:20+00:00\",\"dateModified\":\"2022-05-12T00:15:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1.png\",\"contentUrl\":\"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1.png\",\"width\":1695,\"height\":784},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/medicine.nus.edu.sg\/taps\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/\",\"name\":\"The Asia Pacific Scholar\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/","og_locale":"en_US","og_type":"article","og_title":"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar","og_description":"Submitted: 31 August 2020 Accepted: 8 February 2021 Published online: 5 October, TAPS 2021, 6(4), 118-130 https:\/\/doi.org\/10.29060\/TAPS.2021-6-4\/OA2481 Isabella E. Supnet, Jose Alvin P. Mojica, Sharon D. Ignacio &amp; Carl Froilan D. Leochico Department of Rehabilitation Medicine, College of Medicine and Philippine General Hospital, University of the Philippines Manila, Philippines Abstract Introduction: In the Philippines, telerehabilitation [&hellip;]","og_url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/","og_site_name":"The Asia Pacific Scholar","article_modified_time":"2022-05-12T00:15:46+00:00","og_image":[{"url":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"22 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/","url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/","name":"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students - The Asia Pacific Scholar","isPartOf":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website"},"primaryImageOfPage":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage"},"image":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage"},"thumbnailUrl":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1-300x139.png","datePublished":"2021-09-13T06:36:20+00:00","dateModified":"2022-05-12T00:15:46+00:00","breadcrumb":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#primaryimage","url":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1.png","contentUrl":"https:\/\/medicine.nus.edu.sg\/taps\/wp-content\/uploads\/sites\/10\/2021\/09\/OA2481-Figure-1.png","width":1695,"height":784},{"@type":"BreadcrumbList","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/development-of-a-feedback-evaluation-tool-for-the-assessment-of-telerehabilitation-as-a-teaching-learning-tool-for-medical-students\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/medicine.nus.edu.sg\/taps\/"},{"@type":"ListItem","position":2,"name":"Development of a feedback evaluation tool for the assessment of telerehabilitation as a teaching-learning tool for medical students"}]},{"@type":"WebSite","@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website","url":"https:\/\/medicine.nus.edu.sg\/taps\/","name":"The Asia Pacific Scholar","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues\/71006","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues"}],"about":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/types\/issues"}],"wp:attachment":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/media?parent=71006"}],"wp:term":[{"taxonomy":"issues_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues_category?post=71006"},{"taxonomy":"archive_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/archive_category?post=71006"},{"taxonomy":"issue_type","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issue_type?post=71006"},{"taxonomy":"volume_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/volume_category?post=71006"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}