{"id":72815,"date":"2025-03-18T09:45:36","date_gmt":"2025-03-18T01:45:36","guid":{"rendered":"https:\/\/medicine.nus.edu.sg\/taps\/?post_type=issues&p=72815"},"modified":"2025-07-01T08:56:28","modified_gmt":"2025-07-01T00:56:28","slug":"is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students","status":"publish","type":"issues","link":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/","title":{"rendered":"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?"},"content":{"rendered":"

Submitted: 30 April 2024
\r\nAccepted: 25 September 2024
\r\nPublished online: 1 April, TAPS 2025, 10(2), 82-85
\r\nhttps:\/\/doi.org\/10.29060\/TAPS.2025-10-2\/SC3345<\/a><\/p>\r\n

Sethapong Lertsakulbunlue & Anupong Kantiwong<\/span><\/span><\/p>\r\n

Department of Pharmacology, Phramongkutklao College of Medicine, Thailand<\/span><\/i><\/span><\/i><\/p>\r\n

Abstract<\/span><\/b><\/p>\r\n

Introduction:<\/span><\/b> <\/span>Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately\u00a0than Single Best Answer Questions, as multiple-choice options might not be realistic<\/span>. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative assessments. This study determines the validity and reliability of the SM-VSAQs.<\/span><\/p>\r\n

Methods:<\/span><\/b> Ninety-four third-year pre-clinical students took two occasions of 10-question <\/span>SM-<\/span>VSAQ exams regarding cardiovascular drugs. Each question consisted of two steps: (1) clinical vignettes with questions and (2) expected answers with scores, self-marking, and feedback comprehension. <\/span>Scores ranged from 0.00 to 1.00 in 0.25 increments, though not every increment was applied to all questions. A distribution of the rating agreement between students’ and teacher\u2019s ratings was presented to determine criterion-related validity and inter-rater reliability.<\/span><\/span><\/p>\r\n

Results:<\/span><\/b> <\/span>C<\/span>riterion-related validity revealed <\/span>90.64% and 93.19% of the ratings demonstrated exact agreement between students’ and teachers’ ratings, with an inter-rater reliability of 0.972 and 0.977 for the first and second occasions, respectively (p<\/i>=0.001). The exact agreement was relatively lower on the first occasion for questions with more diverse expected answers (85.11%, r=0.867, p<\/i>=0.001) and drugs requiring their specific full names for a perfect mark (74.47%, r=0.849, p<\/i>=0.001). While questions with specific guides<\/span> <\/span>do not require complex answers, they received a higher exact agreement.<\/span><\/span><\/p>\r\n

Conclusion: <\/span><\/b>The SM-VSAQ format effectively combines guided answers with the VSAQ model. The agreement with teacher-rated is excellent. Marking discrepancies rooted in misconceptions underscores the importance of teacher feedback in improving self-grading in formative assessments. Regular self-assessment practice is recommended to enhance grading accuracy.<\/b><\/span><\/p>\r\n

Keywords<\/span><\/b>:<\/span><\/b>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span>Very Short Answer Question, Self-assessment, Medical Education, Undergraduate, Pharmacology<\/span><\/i><\/span><\/b><\/p>\r\n

<\/span>I. INTRODUCTION<\/span><\/strong><\/p>\r\n

Very Short Answer Questions (VSAQs) emerge as a relatively novel assessment format, addressing the constraints of traditional examination methods like Single Best Answer Questions (SBAQs), Constructed Response Questions (CRQs), and Modified Essay Questions (MEQs) (Sam et al., 2018). Although SBAQs are widely adopted in medical education globally, they are prone to cueing effects, leading examinees to depend on contextual clues, promoting a recognition-based learning approach (Sam et al., 2018). Moreover, the absence of multiple-choice options in real-life scenarios diminishes the relevance of SBAQs to medical practice.<\/span><\/p>\r\n

Conversely, while CRQs and MEQs better mimic real-life situations, they suffer from rater dependency and significant evaluation time. Whereas VSAQs, free-response questions with 1\u20135 word answers, lessen rater dependency and evaluation time. Evidence indicates that VSAQs outperform SBAQs in discrimination, validity, and reliability in undergraduate assessments<\/span>. Their open-ended nature prevents recognition-based learning and cueing. Additionally, VSAQs adeptly pinpoint common errors, often missed by SBAQs, and offer valuable feedback opportunities for educators (van Wijk et al., 2023).<\/span><\/p>\r\n

Feedback is crucial for supporting and enhancing learning. Despite its longstanding importance in medical education, effective feedback is frequently deemed insufficient <\/span>(Kuhlmann L\u00fcdeke & Guill\u00e9n Olaya, 2020)<\/span>. Self-assessment, enabled by formative exams, allows learners to identify their learning needs (Gedye, 2010). To improve feedback in formative assessments, Phramongkutklao College of Medicine (PCM) developed the Self-marked VSAQ (SM-VSAQ) format, which pairs a VSAQ with possible answers and a marking guide. Students may assess their understanding and pinpoint study areas through SM-VSAQ, enhancing feedback. Although VSAQs offer several benefits, challenges remain in grading the tests, as they may require a\u00a0longer time. The self-graded format could address this issue in low-stakes examinations. This study assesses whether the SM-VSAQ with partial credit format, utilizing the marking guide, would achieve valid and reliable ratings compared with the teachers.<\/span><\/p>\r\n

<\/span>II. METHODS<\/span><\/strong><\/p>\r\n

Ninety-four third-year pre-clinical students participated in two 10-item SM-VSAQ during a\u00a0cardiovascular pharmacology course. The exams covered antihypertensive, antiarrhythmic, antianginal, antithrombotic drugs, heart failure<\/span> <\/span>drugs,<\/span> rational drug use, dyslipidaemia treatments, and drugs for atherosclerotic cardiovascular disease (ASCVD). <\/span>The second SM-VSAQ sessions vary by changing the clinical vignette, the question, or both while maintaining the same underlying blueprint as the first session<\/span>. <\/span>Difficulty levels align with the Thai Medical Competency Assessment Criteria. <\/span>Students had attended lectures on these drug groups before the exams. <\/span>The VSAQ was content-validated by three professors for relevance, difficulty, feasibility, and simplicity using the Item Objective Congruence method with all over 0.67 of 1.00, indicating acceptable content validity. <\/span>This approach ensured comparable difficulty.<\/span><\/span><\/p>\r\n

The formative test was administered through Google Forms under examination conditions within a one-hour timeframe. Ethical approval was obtained from the Institutional Review Board, Royal Thai Army, and the waiver of the\u00a0requirement for participant consent was deemed unnecessary following national regulations. An information sheet was provided on the first page of the Google Form. This initial test was conducted a day after they completed all lectures. After receiving teacher-led feedback and having time to review, students took a second parallel formative test ten days before the summative exam. <\/span><\/p>\r\n

The SM-VSAQs featured four components for each question: clinical vignettes and questions on the first page, answers with scoring guidelines on the next page after they\u2019ve answered, and a self-scoring option with feedback on answer comprehension. Scores ranged from 0.00 to 1.00 in 0.25 increments, though not every increment was\u00a0applied to all questions. After the students completed the exam, they provided open-ended feedback on the pros and cons of the format. Examples of the format are shown in supplementary figures 1 and 2.<\/span><\/p>\r\n

The self-rated, according to the marking guide, <\/span>were exported into a Microsoft Excel spreadsheet to facilitate teacher ratings of the VSAQ answers. Using the ‘filter’ function in Microsoft Excel, the range of answers for each question was examined, and marks were awarded (Sam et al., 2018). Minor misspellings or alternative correct spellings were considered correct. Three pharmacology professors, who assigned scores, reviewed student answers that fell outside the guide. Consensus-determined scores require agreement from at least two of the three professors.<\/span><\/p>\r\n

The data analyses were performed using StataCorp, 2021, Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC.<\/i> Consistency reliability was analysed using Cronbach\u2019s alpha. Criterion-related validity was demonstrated by the distribution of the rating agreement between student and teacher ratings, presented as frequency and percentages. Inter-rater reliability was calculated using Pearson\u2019s correlation.<\/span><\/p>\r\n

<\/span>III. RESULTS<\/span><\/strong><\/p>\r\n

Cronbach\u2019s alpha for the SM-VSAQ was 0.741 and 0.721 on the first and second occasions, respectively. The teacher-rated alpha was 0.766 initially and 0.735 on the second<\/span>. Criterion-related validity was assessed through agreement analysis (Supplementary Tables 1 and 2). Table 1 summarises the results of the agreement analysis. 90.6% and 93.19% of the ratings showed exact agreement between the students’ and teachers’ ratings, with an inter-rater reliability of 0.972 and 0.977 for the first and second occasions, respectively. The exact agreement is relatively low on the first occasion of Drugs used in heart failure (85.11%) and Anti-angina drugs (74.47%). Conversely, antithrombotics and drugs used in ASCVD receive<\/span>d<\/span> <\/span>a high exact agreement of 96.81%. Example of questions with high and low agreement is demonstrated in supplementary figures 1 and 2. <\/span>Additionally, content analysis of student\u2019s feedback revealed that they perceived that the format helps identify knowledge gaps, encourages review of missed topics, and aids in recognizing their current knowledge level (Supplementary Table 3).<\/span><\/p>\r\n

\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n

Item<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

First Occasion<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

Second Occasion<\/span><\/b><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Exact agreement<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.25 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.50 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.75 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

1.00 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

r*<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

Exact agreement<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.25 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.50 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

0.75 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

1.00 difference<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

r*<\/span><\/b><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

<\/td>\r\n\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

n (%)<\/span><\/b><\/p>\r\n<\/td>\r\n

<\/td>\r\n<\/tr>\r\n
\r\n

Q1. Antihypertensive drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

86 (91.49)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

8 (8.51)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.943<\/span><\/p>\r\n<\/td>\r\n

\r\n

90 (95.74)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

4 (4.26)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.969<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q2. Antihypertensive drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

87 (92.55)<\/span><\/p>\r\n<\/td>\r\n

\r\n

4 (4.26)<\/span><\/p>\r\n<\/td>\r\n

\r\n

3 (3.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.964<\/span><\/p>\r\n<\/td>\r\n

\r\n

91 (96.81)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

3 (3.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.965<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q3. Antihypertensive drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

91 (96.81)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.981<\/span><\/p>\r\n<\/td>\r\n

\r\n

90 (95.74)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.960<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q4. Antiarrhythmic drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

90 (95.74)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.961<\/span><\/p>\r\n<\/td>\r\n

\r\n

91 (96.81)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.980<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q5. Drugs used in heart failure<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

80 (85.11)<\/span><\/p>\r\n<\/td>\r\n

\r\n

7 (7.45)<\/span><\/p>\r\n<\/td>\r\n

\r\n

5 (5.32)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.867<\/span><\/p>\r\n<\/td>\r\n

\r\n

88 (93.62)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

4 (4.26)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.922<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q6. Anti-angina drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

70 (74.47)<\/span><\/p>\r\n<\/td>\r\n

\r\n

9 (9.57)<\/span><\/p>\r\n<\/td>\r\n

\r\n

14 (14.89)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.849<\/span><\/p>\r\n<\/td>\r\n

\r\n

79 (84.04)<\/span><\/p>\r\n<\/td>\r\n

\r\n

5 (5.32)<\/span><\/p>\r\n<\/td>\r\n

\r\n

10 (10.64)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.918<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q7. Antithrombotic drugs<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

91 (96.81)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.983<\/span><\/p>\r\n<\/td>\r\n

\r\n

83 (88.30)<\/span><\/p>\r\n<\/td>\r\n

\r\n

6 (6.38)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.880<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q8. Drugs used in dyslipidemia<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

84 (89.36)<\/span><\/p>\r\n<\/td>\r\n

\r\n

3 (3.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

6 (6.38)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.915<\/span><\/p>\r\n<\/td>\r\n

\r\n

89 (94.68)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.936<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q9. CVS rational drug used<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

82 (87.23)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

10 (10.64)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.907<\/span><\/p>\r\n<\/td>\r\n

\r\n

82 (87.23)<\/span><\/p>\r\n<\/td>\r\n

\r\n

3 (3.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

6 (6.38)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

3 (3.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.851<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Q10. Drugs used in ASCVD<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

91 (96.81)<\/span><\/p>\r\n<\/td>\r\n

\r\n

2 (2.13)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.978<\/span><\/p>\r\n<\/td>\r\n

\r\n

93 (98.94)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

1 (1.06)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.973<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n

\r\n

Total<\/span><\/b><\/p>\r\n<\/td>\r\n

\r\n

852 (90.64)<\/span><\/p>\r\n<\/td>\r\n

\r\n

33 (3.51)<\/span><\/p>\r\n<\/td>\r\n

\r\n

50 (5.32)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0 (0.00)<\/span><\/p>\r\n<\/td>\r\n

\r\n

5 (0.53)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.972<\/span><\/p>\r\n<\/td>\r\n

\r\n

876 (93.19)<\/span><\/p>\r\n<\/td>\r\n

\r\n

18 (1.91)<\/span><\/p>\r\n<\/td>\r\n

\r\n

32 (3.40)<\/span><\/p>\r\n<\/td>\r\n

\r\n

6 (0.64)<\/span><\/p>\r\n<\/td>\r\n

\r\n

8 (0.85)<\/span><\/p>\r\n<\/td>\r\n

\r\n

0.977<\/span><\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

*p<\/i>=0.001 for all items, CVS: Cardiovascular system ASCVD: Atherosclerotic cardiovascular disease<\/span><\/p>\r\n

Table 1. Comparison of rater agreement between the teacher and the self-rating on the VSAQ assessment<\/span><\/span><\/p>\r\n

<\/span>IV. DISCUSSION<\/span><\/strong><\/p>\r\n

VSAQs have demonstrated their discrimination, validity, and reliability among undergraduate assessments and their capacity to identify errors not detectable by SBAQs. However, the marking process poses challenges, potentially requiring more time than SBAQs, even with computerised marking systems <\/span>(Bala et al., 2023)<\/span>. Delayed marking results in slower feedback delivery to students regarding their examination performance. Therefore, to our knowledge, the study is the first to demonstrate the reliability of using self-guided marking to provide students with immediate feedback after a\u00a0formative VSAQ examination.<\/span>\u00a0<\/span><\/p>\r\n

The inter-rater reliability exceeded 0.90 for nearly every question, suggesting the validity of self-grading compared with teacher grading. Moreover, by furnishing students with a partial credit guide, they were encouraged to analyse their answers to each guided answer, fostering a more profound understanding than the singular correct answer required in SBAQs, and encouraging engagement in higher-order thinking. The content analysis of student comments supports this. They found the partial credit guide helpful in identifying key knowledge areas, analyzing expected answers, and engaging in self-directed learning. Additionally, path analysis showed that the first VSAQ attempt score positively influenced the second VSAQ understanding levels, primarily through the second attempt score, highlighting the benefits of multiple attempts for gaining insights (Supplementary Figure 3).<\/span>\u00a0<\/span><\/p>\r\n

Discrepancies in ratings with the teacher likely stem from misconceptions. For example, while the correct response involved furosemide acting as a Na+<\/sup>\/K+<\/sup>\/2Cl–<\/sup> channel inhibitor, some students mistakenly identified it as a “Na+<\/sup>-K+<\/sup>-ATPase” and awarded themselves full marks. Some students gave full marks for partially correct and imprecise responses. For instance, concerning the drug interaction between clarithromycin and warfarin, the answer involves enzyme inhibition by clarithromycin, yet some students merely stated, “Drug interaction between drugs.” Similarly, in the anti-angina question, the correct answer is “sublingual nitroglycerin or sublingual isosorbide dinitrate.” However, those who answered partially correctly still awarded themselves full marks.<\/span> <\/span>Additionally, d<\/span>isagreement may also be related to student ability, as those less familiar with the content, which leads to misconceptions, might not rate as well as those who are. <\/span>\u00a0To address discrepancies in the ratings, reviewing students’ divergent responses could help refine the marking guide. Furthermore, repeated practice in self-assessment will enhance students’ ability to grade their answers accurately.<\/span>\u00a0<\/span><\/p>\r\n

Conversely, questions with a high level of agreement provided detailed answers consisting solely of the drug name without asking for additional components such as the route of administration or mechanism of action. However, asking for multiple components helped enrich the knowledge and feedback that students could gain.<\/span>\u00a0<\/span><\/p>\r\n

The present SM-VSAQ format has several strengths. First, it presents a realistic examination, as multiple-choices might not be available in real life. Second, it is simple, feasible, and adaptable, as perceived by the students. Third, it can be administered as an online formative examination, reducing the burden on teachers and providing immediate feedback to students, which has proven reliable and in high agreement with teachers. Nonetheless, this study has certain limitations. It only included a third-year pre-clinical student from a specific educational context, necessitating further research to assess the external validity of the findings.<\/span>\u00a0<\/span><\/p>\r\n

<\/span>V. CONCLUSION<\/span><\/strong><\/p>\r\n

SM-VSAQ approach facilitates engagement in higher-order thinking more effectively than the traditional single-best answer method. The format is also simple, adaptable to other subjects, and can be easily reviewed. <\/span>The agreement between self-graded and teacher-provided ratings is outstanding. Discrepancies between student and teacher evaluations primarily stem from misconceptions in guided answers, highlighting the crucial need for teacher-led feedback to resolve these misunderstandings. This step is essential before implementing self-grading as an alternative in formative evaluations. Regular practice in self-assessment is advised to refine precision in self-grading. The SM-VSAQ format merges the VSAQ model with guided answers and may be further developed to improve feedback timeliness.<\/span><\/p>\r\n

Notes on Contributors<\/span><\/strong><\/p>\r\n

SL reviewed the literature, designed the study, collected the data, conducted data analysis and wrote the manuscript. AK reviewed the literature, supervised, designed the study, performed the data analysis.<\/span>\u00a0<\/span><\/p>\r\n

Ethical Approval<\/span><\/strong><\/p>\r\n

Ethical approval was obtained from the Medical Department Ethics Review Committee for Research in Human Subjects, Institutional Review Board, Royal Thai Army (IRBRTA) (Approval no. S079q\/66_Xmp).<\/span><\/p>\r\n

The IRBRTA waived the requirement for participant consent, deeming it unnecessary in accordance with national regulations.<\/span>\u00a0<\/span><\/p>\r\n

Data Availability<\/span><\/strong><\/p>\r\n

Data sets analysed during the current study would be available from the corresponding author upon reasonable request. <\/span>The Supplementary file for the current study is available from: <\/span>https:\/\/doi.org\/10.6084\/m9.figshare.26507170<\/span><\/a><\/span> <\/span><\/span>\u00a0<\/span><\/p>\r\n

Acknowledgement<\/span><\/strong><\/p>\r\n

This work would not have been possible without the active support of Phramongkutklao College of Medicine faculty members and its academic leaders, who are too numerous to name individually.<\/span>\u00a0<\/span><\/p>\r\n

Funding<\/span><\/strong><\/p>\r\n

The authors reported no funding associated with the work featured in this article.<\/span>\u00a0<\/span><\/p>\r\n

Declaration of Interest<\/span><\/strong><\/p>\r\n

The authors declare no competing interests.<\/span>\u00a0<\/span><\/p>\r\n

References<\/span><\/strong><\/p>\r\n

Bala, L., Westacott, R. J., Brown, C., & Sam, A. H. (2023). Twelve tips for introducing very short answer questions (VSAQs) into your medical curriculum. Medical Teacher<\/i>, 45<\/i>(4), 360\u2013367. <\/span>https:\/\/doi.org\/10.1080\/0142159X.2022.2093706<\/a><\/span><\/span><\/p>\r\n

Gedye, S. (2010). Formative assessment and feedback: A review. Planet<\/i>, 23<\/i>(1), 40\u201345. <\/span>https:\/\/doi.org\/10.11120\/plan.2010.002300 40<\/a><\/span><\/span><\/p>\r\n

Kuhlmann L\u00fcdeke, A. B. E., & Guill\u00e9n Olaya, J. F. (2020). Effective feedback, an essential component of all stages in medical education. Universitas M\u00e9dica<\/i>, 61<\/i>(3). <\/span>https:\/\/doi.org\/10.11144\/ Javeriana.umed61-3.feed<\/a><\/span><\/span><\/p>\r\n

Sam, A. H., Field, S. M., Collares, C. F., van der Vleuten, C. P. M., Wass, V. J., Melville, C., Harris, J., & Meeran, K. (2018). Very-short-answer questions: Reliability, discrimination and acceptability. Medical Education<\/i>, 52<\/i>(4), 447\u2013455. <\/span>https:\/\/doi.org\/10.1111\/medu.13504<\/a><\/span><\/span><\/p>\r\n

van Wijk, E. V., Janse, R. J., Ruijter, B. N., Rohling, J. H. T., van der Kraan, J., Crobach, S., de Jonge, M., de Beaufort, A. J., Dekker, F. W., & Langers, A. M. J. (2023). Use of very short answer questions compared to multiple choice questions in undergraduate medical students: An external validation study. PLOS ONE, 18<\/i>(7), e0288558. <\/span>https:\/\/doi.org\/10.1371\/journal.pone.0288558<\/a><\/span> <\/span><\/p>\r\n

*Anupong Kantiwong<\/span>
\r\nDepartment of Pharmacology<\/span>
\r\nPhramongkutklao College of Medicine, Bangkok, 10400<\/span>
\r\nEmail: anupongpcm31@gmail.com<\/span><\/p>\r\n<\/div>","protected":false},"featured_media":0,"parent":0,"menu_order":10,"template":"","issues_category":[17],"archive_category":[],"issue_type":[25],"volume_category":[61894],"class_list":["post-72815","issues","type-issues","status-publish","hentry","issues_category-short-communications","issue_type-past-issue","volume_category-volume-10-number-2-april-2025"],"acf":[],"yoast_head":"\nIs Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar\" \/>\n<meta property=\"og:description\" content=\"Submitted: 30 April 2024 Accepted: 25 September 2024 Published online: 1 April, TAPS 2025, 10(2), 82-85 https:\/\/doi.org\/10.29060\/TAPS.2025-10-2\/SC3345 Sethapong Lertsakulbunlue & Anupong Kantiwong Department of Pharmacology, Phramongkutklao College of Medicine, Thailand Abstract Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately\u00a0than Single Best Answer Questions, as multiple-choice options might not […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/\" \/>\n<meta property=\"og:site_name\" content=\"The Asia Pacific Scholar\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-01T00:56:28+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/\",\"name\":\"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar\",\"isPartOf\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\"},\"datePublished\":\"2025-03-18T01:45:36+00:00\",\"dateModified\":\"2025-07-01T00:56:28+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/medicine.nus.edu.sg\/taps\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/medicine.nus.edu.sg\/taps\/#website\",\"url\":\"https:\/\/medicine.nus.edu.sg\/taps\/\",\"name\":\"The Asia Pacific Scholar\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/","og_locale":"en_US","og_type":"article","og_title":"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar","og_description":"Submitted: 30 April 2024 Accepted: 25 September 2024 Published online: 1 April, TAPS 2025, 10(2), 82-85 https:\/\/doi.org\/10.29060\/TAPS.2025-10-2\/SC3345 Sethapong Lertsakulbunlue & Anupong Kantiwong Department of Pharmacology, Phramongkutklao College of Medicine, Thailand Abstract Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately\u00a0than Single Best Answer Questions, as multiple-choice options might not […]","og_url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/","og_site_name":"The Asia Pacific Scholar","article_modified_time":"2025-07-01T00:56:28+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/","url":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/","name":"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? - The Asia Pacific Scholar","isPartOf":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website"},"datePublished":"2025-03-18T01:45:36+00:00","dateModified":"2025-07-01T00:56:28+00:00","breadcrumb":{"@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/medicine.nus.edu.sg\/taps\/issues\/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/medicine.nus.edu.sg\/taps\/"},{"@type":"ListItem","position":2,"name":"Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?"}]},{"@type":"WebSite","@id":"https:\/\/medicine.nus.edu.sg\/taps\/#website","url":"https:\/\/medicine.nus.edu.sg\/taps\/","name":"The Asia Pacific Scholar","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/medicine.nus.edu.sg\/taps\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues\/72815","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues"}],"about":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/types\/issues"}],"wp:attachment":[{"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/media?parent=72815"}],"wp:term":[{"taxonomy":"issues_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issues_category?post=72815"},{"taxonomy":"archive_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/archive_category?post=72815"},{"taxonomy":"issue_type","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/issue_type?post=72815"},{"taxonomy":"volume_category","embeddable":true,"href":"https:\/\/medicine.nus.edu.sg\/taps\/wp-json\/wp\/v2\/volume_category?post=72815"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}