Number of Citations: 0

Published online: 7 May, TAPS 2019, 4(2), 7-13
DOI: https://doi.org/10.29060/TAPS.2019-4-2/OA2079

Yuki Kataoka^1,2, Hiraku Tsujimoto², Yasushi Tsujimoto³, Yuka Urushibara-Miyachi⁴, Yuda Miyamoto⁵, Hideto Yasuda⁶, Hiromitsu Yamashita⁵, Ryohei Yamamoto⁶ & Hiroshi Nishigori⁷

¹Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Japan; ²Hospital Care Research Unit, Hyogo Prefectural Amagasaki General Medical Center, Japan; ³Department of Nephrology and Dialysis, Kyoritsu Hospital, Japan; ⁴Science and Religion, School of Divinity, University of Edinburgh, Scotland; ⁵Outpatient Facility Kameda Clinic, Japan; ⁶Department of Intensive Care Unit, Kameda General Hospital, Japan; ⁷Center for Medical Education, Kyoto University, Japan

Abstract

Practice Highlights

• We described the current status of medical education research in Japan.
• We found a low subsequent publication proportion for abstracts and low representativeness in journals.
• The characteristics of conference abstracts were not related to subsequent publication.

I. INTRODUCTION

The need for the best evidence has been indicated in medical education (Harden, Grant, Buckley, & Hart, 1999). Every medical educator is expected to engage in scholarship by contributing new, peer-reviewed resources that advance the field (Norman, 2002; Simpson et al., 2007). The impact factors of medical educational journals, which reflect some aspects of scholarship activity, have been increasing in recent years (Azer, Holen, Wilson, & Skokauskas, 2016). Most medical education studies are conducted in the US or European countries. Studies from Asia are relatively less (Doja, Horsley, & Sampson, 2014).

The Japan Society for Medical Education (JSME) was founded in 1969. JSME is the largest medical education organisation in Japan. More than 2000 members are engaged in educational and research activities (Suzuki, Gibbs, & Fujisaki, 2008). However, no data is currently available on the current status of medical education research in Japan.

II. METHODS

A. Aims and Objectives

Our research objectives are as follows:

• To describe the characteristics of the conference abstracts of JSME, the Association for Medical Education in Europe (AMEE) Conference, and Asia Pacific Medical Education Conference (APMEC) from Japan.
• To describe the characteristics of articles published in medical education journals from Japan.
• To evaluate the relationship between the characteristics of conference abstracts and subsequent publication in peer-review journals.

B. Design

We conducted a meta-epidemiological investigation, which is a variant method of systematic review and usually used to describe the distribution of research evidence in a specific setting (Murad & Wang, 2017). We have developed a protocol a priori.

C. Types of Study

We included research abstracts presented in JSME from April 2013 to March 2017. We also included research abstracts presented by researchers affiliated with Japanese institutions at the AMEE and APMEC conferences from April 2012 to January 2017. We excluded abstracts of invited presentations, symposium, panel discussions, or educational workshops. We included full articles published by researchers affiliated with Japanese institutions in Medical Education, Academic Medicine, Medical Teacher, BMC Medical Education, Advances in Health Science and Education, Perspectives on Medical Education, or Teaching and Learning in Medicine from April 2012 to March 2017. We searched PubMed using the following search formula: “Name of Journal” [journal] AND “Japan” [affiliation] AND “2012/04/01” [PDAT]: “2017/03/31” [PDAT] NOT “Letter” [pt] in September 2017.

D. Definition of Study Characteristics

We evaluated abstracts without presentation data. We classified abstracts as quantitative, qualitative, or mixed method study. We defined text mining, which is an automatic assessment technique for qualitative text data as a qualitative study (Zhang et al., 2012). We defined quantitative evaluation without data, which were abstracts with the interpretation of numerical results without data as a quantitative study. We defined abstracts without evaluation as “without data”. We classified abstracts as “undergraduate”, “postgraduate”, “continuous medical education (CME)”, “interprofessional education”, and “other” according to the study participants.

We classified topics as follows: knowledge/attitude, clinical diagnosis, clinical interventions, teaching and leadership, assessment, other following the previous study (Cook, Bordage, & Schmidt, 2008). We classified purposes of research as follows: description, justification, or clarification following the previous study (Cook et al., 2008). Description means the simple observation undertaken to answer the question: What was done? Justification means comparing one educational intervention with another to answer the question: Did it work? Clarification means updating models or theories to answer the questions: How does it work? Why does it work? (Cook et al., 2008).

E. Data Extraction

We judged the affiliations of the first author and presentation styles using the Microsoft Excel® 2016 search function. The other characteristics were extracted by some of the authors (YK, HT, YT, YU-M, YM, HY, and HY) and confirmed by others (YK and YU-M). We resolved any disagreements through discussions.

F. Outcome Measures

The primary outcome was the publication proportion. The denominator was the number of abstracts included. The numerator was the number of abstracts published in peer-review journals. One author (YK) searched Google Scholar and PubMed using the first authors’ names in both Japanese and English. The search was conducted in September 2017. Published manuscripts were matched with other authors’ names and titles. We included JSME, AMEE Conference, and APMEC in 2014 because one previous study reported that the median publication duration from the conference was 20 months (Walsh, Fung, & Ginsburg, 2013).

G. Statistical Analysis

We summarised data using descriptive statistics. We calculated risk ratios and 95% confidence interval using general linear models. We used STATA® ver. 14.2 (Stata Corp., College Station, TX, USA). P < 0.05 was considered statistically significant.

H. Ethical Considerations

Because all the data were retrieved from public databases, this study did not require institutional review board approval.

III. RESULTS

A total of 1399 JSME conference abstracts, 193 AMEE conference and APMEC abstracts, and 31 original articles were included (Figure 1). The characteristics of the studies are presented in Table 1. During the study period, 6727 original articles were published in seven journals. Original articles from Japan were found to constitute 0.46% (31/6727).

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference; CME – Continuing Medical Education; IPE – Interprofessional education.
*AMEE Conference from 2012 to 2016 and APMEC from 2012 to 2015.
**NA, not available.
***The number of authors in JSME 2013 is not included.
Table 1. Characteristics of the included abstracts and full articles

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference.
Figure 1. Flow diagram

Note: JSME – Japan Society for Medical Education; AMEE – The Association for Medical Education in Europe;
APMEC – Asia Pacific Medical Education Conference; CME – Continuing Medical Education; IPE – Interprofessional education.
Table 2. The relationship between publication and characteristics in JSME 2014 abstracts

From a total of 382 abstracts, 37 abstracts (10%) presented at the JSME 2014 conference were published. From a total of 39 abstracts, four (10%) abstracts presented at AMEE Conference 2014 and APMEC 2014 from Japan were published, respectively. A total of 11 (30%) articles were published in English journal from JSME. Other 26 abstracts (70%) published in 18 Japanese journals with peer review. All four articles were published in English journal from the AMEE and APMEC conferences. Exploratory analysis showed that the number of authors, affiliation of the first author, presentation style, method, topics, and purposes of the research were not statistically significant associated with subsequent publication rate (Table 2).

IV. DISCUSSION

We evaluated the characteristics of abstracts pertaining to medical education from Japan. Walsh et al. (2013) reported that the proportion of quantitative/mixed/qualitative study in medical education conference in America and Canada were 55%, 8% and 15%, respectively. Cook et al. (2008) reported the proportion of published medical education articles categorised as clarification to be 6 to 26%. Our study revealed almost the same proportion (17% to 35%). The characteristics of our study were not significantly differed from previous reports.

The representativeness of Japan in medical education journals was 0.46%. Fukui, Takahashi, and Rahman (2013) reported that the Japanese contribution was 3.6% in basic science journals and 0.74% in general medicine journals during 2001-2010. The current status of medical education research in Japan is almost equivalent to that of general medicine.

The proportion of subsequent publication (10%) was very low. Scherer, Langenberg and von Elm (2007) reported that the full publication rate was 45% in biomedical science conference abstracts. Other previous studies have reported publication rate for medical education conference abstracts to be were 35% to 44% (Sawatsky et al., 2015; Smith et al., 2014; Walsh et al., 2013). Characteristics of abstracts we have investigated were not related to publication (Table 2).

A possible explanation for the lower subsequent publication proportion may be the worse average quality of each abstract (Sawatsky et al., 2015), or inadequate time of researchers (Smith et al., 2014). The acceptance rates for the conference abstracts in JSME is nearly 100% (personal communication). The number is higher than other medical education conferences. For example, the acceptance rate of AMEE 2018 is nearly 45%, which is estimated from the 3766 submissions (Harden, 2018) and the 1658 occurrences of “background” in the abstract book (The Association for Medical Education in Europe, 2018). This difference would be evidence of poor abstract quality. Love et al. (2016) revealed that education for clinicians to gain skills including quantitative and qualitative methods improved the medical education research output. A lack of quantitative research education programs for post-graduates in Japan was identified (Arimura et al., 2010). To provide education programs for medical teachers in Japan is one of the options to promote educational researches. Considering the high burnout rate of doctors including residents (Tateno et al., 2018), the Japan Ministry of Health, Labour and Welfare (2018) is now attempting to reform the work lives of doctors to reduce their burden. It is important to use any gained surplus time for research productivity improvement.

By offering educational programs on medical education research for several physicians who lack knowledge of research (Kurita et al., 2016; Suzuki & Fukushima, 2014), consequently, the number of studies on medical education would increase. The total quality and publication rate may improve (Huang, 2016). In April 2018, the board certification system changed to standardise the certification of fellows in Japan (Hirokuni, 2017). To improve the system, the post-graduate medicine curriculum should be evaluated (Iwata, Mosby, & Sakane, 2017). There are several opportunities to conduct medical educational research.

Our study has several limitations. First, seven individual authors judged the characteristics of abstracts to reduce burden, which may cause misclassifications. To reduce the information bias, another author confirmed the decision, and we followed the pre-defined protocol. Second, we did not search for medical education abstracts in other conferences, such as the Association of American of Medical Colleges (AAMC) or Ottawa Conference. Furthermore, some medical education abstracts have been presented in clinical conferences, such as general internal medicine or paediatrics (Sawatsky et al., 2015; Smith et al., 2014). In general, medical education research is not often pre-registered, so we believe that to focus on medical education conferences was the best way to know the current status but further evaluation is warranted.

V. CONCLUSION

We described the current status of medical education research in Japan. We identified a low subsequent publication rate for conference abstracts and low representativeness in medical education journals. Further investigation is required to improve the number of publications.

Notes on Contributors

Yuki Kataoka, MD, MPH, is a Medical Head at the Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Japan. He contributed to the design of the research, retrieving data, analysing, prepared the draft of the article, and approved the final version of the article.

Hiraku Tsujimoto, MD, is a research fellow at the Hospital Care Research Unit, Hyogo Prefectural Amagasaki General Medical Center, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Yasushi Tsujimoto, MD, MPH, is an attending staff member at the Department of Nephrology and Dialysis, Kyoritsu Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Yuka Urushibara-Miyachi, MD, MHPE, is an MSc Science and Religion student at the School of Divinity, the University of Edinburgh. She contributed to the design of the research, retrieving data, and approved the final version of the article.

Yuda Miyamoto, MD, is a resident at the Outpatient Facility Kameda Clinic, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hideto Yasuda, MD, is an attending staff member at the Department of Intensive Care Unit, Kameda General Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hiromitsu Yamashita, MD, is a fellow at the Outpatient Facility Kameda Clinic, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Ryohei Yamamoto, MD, is a fellow at the Department of Intensive Care Unit, Kameda General Hospital, Japan. He contributed to the design of the research, retrieving data, and approved the final version of the article.

Hiroshi Nishigori, MD, MMEd, PhD, is an Associate Professor at the Center for Medical Education, Kyoto University, Japan. He contributed to the design of the research and approved the final version of the article.

Ethical Approval

As all the data were retrieved from public databases, this study did not require institutional review board approval.

Acknowledgements

This study was conducted as a part of a project by the Japan Society for Medical Education. The Society supported data retrieval but has no role in the conceptualisation, analysis, or preparation of this manuscript.

Funding

This work was supported in part by a grant from the Hyogo Prefectural Amagasaki General Medical Center fiduciary funds (for English editing).

Declaration of Interest

The authors report no conflicts of interest. All the authors are responsible for the content and writing of the article.

References

Arimura, Y., Nishida, T., Minami, M., Yokoyama, Y., Mishina, H., Yamazaki, S., … Fukuhara, S. (2010). Challenges in the conduct of clinical research: View of the graduates of a clinical research training program in Japan. Igaku Kyoiku [Medical Education (Japan)], 41(4), 259–265.
https://doi.org/10.11307/mededjapan.41.259

Association for Medical Education in Europe. (2018, August 25). ABSTRACT BOOK AMEE 2018 [PDF file]. Retrieved from https://amee.org/getattachment/Conferences/AMEE-2018/Abstracts/AMEE-2018-Abstract-Book.pdf

Azer, S. A., Holen, A., Wilson, I., & Skokauskas, N. (2016). Impact factor of medical education journals and recently developed indices: Can any of them support academic promotion criteria? Journal of Postgraduate Medicine, 62(1), 32-39. https://doi.org/10.4103/0022-3859.173202

Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128–133.
https://doi.org/10.1111/j.1365-2923.2007.02974.x

Doja, A., Horsley, T., & Sampson, M. (2014). Productivity in medical education research: An examination of countries of origin. BioMed Central Medical Education, 14, 243.
https://doi.org/10.1186/s12909-014-0243-8

Fukui, T., Takahashi, O., & Rahman, M. (2013). Japanese representation in leading general medicine and basic science journals: A comparison of two decades. The Tohoku Journal of Experimental Medicine, 231(3), 187–191.
https://doi.org/10.1620/tjem.231.187

Harden, R. M. (2018, March 19). Harden’s Blog: APMEC 2018 Singapore, measuring what matters, Bawa-Garba and fake news, Ottawa conference in Abu Dhabi, and AMEE 2018 in Basel [Blog post]. Retrieved from https://www.mededworld.org/hardens-blog/reflection-items/March-2018/HARDEN-S-BLOG-APMEC-2018-Singapore–Measuring-what.aspx

Harden, R. M., Grant, J., Buckley, G., & Hart, I. R. (1999). BEME guide no. 1: Best evidence medical education. Medical Teacher, 21(6), 553–562. https://doi.org/10.1080/01421599978960

Hirokuni, Y. (2017). 「新たな専門医制度の開始に向けた声明」を公表いたしました [“Statement for the start of a new  specialist system” was announced] [PDF file]. Retrieved from http://www.japan-senmon-i.jp/news/doc/statement_170804_.pdf

Huang, D. (2016). Positive correlation between quality and quantity in academic journals. Journal of Informetrics, 10(2), 329–335. https://doi.org/10.1016/j.joi.2016.02.002

Iwata, K., Mosby, D. J., & Sakane, M. (2017). Board certification in Japan: Corruption and near-collapse of reform. Postgraduate Medical Journal, 93(1101), 436.
https://doi.org/10.1136/postgradmedj-2017-134994

Kurita, N., Murakami, M., Shimizu, S., Kumasawa, J., Azuma, T., Kataoka, Y., … Fukuhara, S. (2016). Preferences of young physicians at community hospitals regarding academic research training through graduate school: A cross-sectional research. BioMed Central Research Notes, 9(1), 227. https://doi.org/10.1186/s13104-016-2036-0

Love, J. N., Yarris, L. M., Santen, S. A., Kuhn, G. J., Gruppen, L. D., Coates, W. C., … Farrell, S. E. (2016). A novel specialty-specific, collaborative faculty development opportunity in education research: Program evaluation at five years. Academic Medicine, 91(4), 548–555.
https://doi.org/10.1097/ACM.0000000000001070

Ministry of Health, Labor and Welfare. (2018). 「医師の働き方改革に関する検討会」が医師の労働時間短縮に向けた取組を緊急アピール [The meeting group to discuss work-style reform for doctors urges medical institutions to step up emergency measures to reduce physicians’ working hours] [PDF file]. Retrieved from
https://www.mhlw.go.jp/stf/newpage_02920.html

Murad, M. H., & Wang, Z. (2017). Guidelines for reporting meta-epidemiological methodology research. Evidence Based Medicine, 22(4), 139–142. https://doi.org/10.1136/ebmed-2017-110713

Norman, G. (2002). Research in medical education: Three decades of progress. British Medical Journal, 324, 1560–1562. https://doi.org/10.1136/bmj.324.7353.1560

Sawatsky, A. P., Beckman, T. J., Edakkanambeth Varayil, J., Mandrekar, J. N., Reed, D. A., & Wang, A. T. (2015). Association between study quality and publication rates of medical education abstracts presented at the society of general internal medicine annual meeting. Journal of General Internal Medicine, 30(8), 1172–1177. https://doi.org/10.1007/s11606-015-3269-7

Scherer, R. W., Langenberg, P., & von Elm, E. (2007). Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews, 2007(2), MR000005. https://doi.org/10.1002/14651858.MR000005.pub3

*Yuki Kataoka
Department of Respiratory Medicine
Hyogo Prefectural Amagasaki General Medical Center
Higashi-Naniwa-Cho 2-17-77,
Amagasaki, Hyogo 660-8550, Japan
Email: youkiti@gmail.com