ChatGPT and Medical Education: A Double-Edged Sword
Keywords:
ChatGPT, medical education, exam, research, abstractAbstract
ChatGPT has gained attention worldwide. In the medical education field, ChatGPT, or any similar large language model, provides a convenient way for students to access information and practice their skills. ChatGPT can simulate patient interactions, allowing students to practice their diagnostic and communication skills in a safe and controlled environment. ChatGPT can also answer questions and provide explanations for complex medical concepts. There have been attempts involving ChatGPT to assist researchers in writing journal articles. Due to its capabilities, ChatGPT has the potential to be abused and sparking ethical concerns. Unwise researchers can now employ ChatGPT to write academic articles. Similarly, irresponsible students, might utilize ChatGPT to cheat during exams. We investigated whether ChatGPT, in its current state, can answer Indonesian medical doctor examination problems. Among the 3 problems that we have picked, ChatGPT can only correctly answer 1 question. We also examine whether ChatGPT-generated abstracts can fool professionals and educators. We have brought 6 abstracts, 3 of which are taken from actual published papers, while the other 3 were generated by ChatGPT. We recruited 12 participants with either a medical doctor (M.D.), a doctorate (Ph.D.), or an M.D., Ph.D. degree from various institutions in Indonesia. Surprisingly, 4 of the participants couldn't guess a single abstract correctly, 6 could only identify 1 abstract accurately, one correctly guessed 2 abstracts, and one correctly identified 3 abstracts. Therefore, it is safe to say that ChatGPT, in its current state, has been able to fool professionals and educators.
Downloads
References
King, M. R. (2022). The future of AI in medicine: a perspective from a Chatbot. Annals of Biomedical Engineering, 1-5.
Kirmani, A. R. (2022). Artificial Intelligence-Enabled Science Poetry. ACS Energy Letters, 8, 574-576.
Zhai, X. (2022). ChatGPT user experience: Implications for education. Available at SSRN 4312418.
Aydın, Ö., & Karaarslan, E. (2022). OpenAI ChatGPT generated literature review: Digital twin in healthcare. Available at SSRN 4308687.
Cahan, P., & Treutlein, B. (2023). A conversation with ChatGPT on the role of computational systems biology in stem cell research. Stem Cell Reports, 18(1), 1-2.
Graham, F. (2022). Daily briefing: Will ChatGPT kill the essay assignment?. Nature.
Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. arXiv preprint arXiv:2212.09292.
Zhang, B., Ding, D., & Jing, L. (2022). How would Stance Detection Techniques Evolve after the Launch of ChatGPT?. arXiv preprint arXiv:2212.14548.
Frye, B. L. (2022). ChatGPT Talks About Animal Rights. Available at SSRN.
Zaremba, A., & Demir, E. (2023). ChatGPT: Unlocking the Future of NLP in Finance. Available at SSRN 4323643.
Oxford Analytica. (2023). Commercial use of ChatGPT requires caution. Emerald Expert Briefings, (oxan-es).
Castelvecchi, D. (2022). Are ChatGPT and AlphaCode going to replace programmers?. Nature.
Professor catches student cheating with ChatGPT: 'I feel abject terror'. (2022). NY Post. https://nypost.com/2022/12/26/students-using-ChatGPT-to-cheat-professor-warns (accessed on 2 January 2023).
Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays-should academics worry?. Nature.
de Winter, J. C. F. (2023). Can ChatGPT Pass High School Exams on English Language Comprehension?.
AI-generated answers temporarily banned on coding Q&A site Stack Overflow. (2022). The Verge. https://www.theverge.com/2022/12/5/23493932/ChatGPT-ai-generated-answers-temporarilybanned-stack-overflow-llms-dangers (accessed on 2 January 2023).
Kim, N., Htut, P. M., Bowman, S. R., & Petty, J. (2022). $^ 2$: Question Answering with Questionable Assumptions. arXiv preprint arXiv:2212.10003.
ChatGPT produces made-up nonexistent references. (2022). Brian Lovin. https://brianlovin.com/hn/33841672 (accessed on 3 January 2023).
A computer can now write your college essay — Maybe better than you can. (2022). Forbes. https://www.forbes.com/sites/emmawhitford/2022/12/09/a-computer-can-now-write-your-collegeessay---maybe-better-than-you-can/?sh=35deca9ddd39 (accessed on 4 January 2023).
Kung, T. H., Cheatham, M., Medinilla, A., ChatGPT, Sillos, C., De Leon, L., ... & Tseng, V. (2022). Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. medRxiv, 2022-12.
O'Connor, S., & ChatGPT. (2023). Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66, 103537. https://doi.org/10.1016/j.nepr.2022.103537
Zhavoronkov, A. (2022). Rapamycin in the context of Pascal's Wager: generative pre-trained transformer perspective. Oncoscience, 9, 82.
GPT Generative Pretrained Transformer, Almira Osmanovic Thunström, Steinn Steingrimsson. Can GPT-3 write an academic paper on itself, with minimal human input?. 2022. ⟨hal-03701250⟩
Stokel-Walker C. (2023). ChatGPT listed as author on research papers: many scientists disapprove. Nature, 10.1038/d41586-023-00107-z. Advance online publication. https://doi.org/10.1038/d41586-023-00107-z
Final Rule Human Subjects Research Exemptions- NIH Infographic. (2019). National Institutes of Health (NIH). https://grants.nih.gov/sites/default/files/exemption_infographic_v8_508c_1-15-2020.pdf (accessed on 4 February 2023).
Subpart A. Basic HHS Policy for Protection of Human Research Subjects. (2018). Office for Human Research Protections. https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/common-rule-subpart-a-46104/index.html (accessed on 4 February 2023).
Walch-Patterson, A. (2020). Exemptions and limited institutional review board review: A practical look at the 2018 common rule requirements for exempt research. Ochsner Journal, 20(1), 87-94.
Hegde, S., & Ellajosyula, R. (2016). Capacity issues and decision-making in dementia. Annals of Indian Academy of Neurology, 19(Suppl 1), S34.
Langsted, A., & Nordestgaard, B. G. (2019). Nonfasting versus fasting lipid profile for cardiovascular risk prediction. Pathology, 51(2), 131-141.
Syabariyah, S., Nurachmah, E., Widjojo, B. D., Prasetyo, S., Sanada, H., Nakagami, G., ... & Hisan, U. K. (2023, January). The Effect of Vibration on the Acceleration of Wound Healing of Diabetic Neuropathic Foot Ulcer: A Prospective Experimental Study on Human Patients. In Healthcare (Vol. 11, No. 2, p. 191). Multidisciplinary Digital Publishing Institute.
Santana, A., Lessa, B., Galrão, L., Lima, I., & Santiago, M. (2005). Kikuchi-Fujimoto's disease associated with systemic lupus erythematosus: case report and review of the literature. Clinical Rheumatology, 24(1), 60-63.
Hisan, U. K., & Amri, M. M. (2022). Artificial Intelligence for Human Life: A Critical Opinion from Medical Bioethics Perspective–Part I. Journal of Public Health Sciences, 1(02), 100-111.
Hisan, U. K., & Amri, M. M. (2022). Artificial Intelligence for Human Life: A Critical Opinion from Medical Bioethics Perspective–Part II. Journal of Public Health Sciences, 1(02), 112-130.
Exclusive: Hindawi and Wiley to retract over 500 papers linked to peer review rings. (2022). Retraction Watch. https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/ (accessed on 17 December 2022).
Exclusive: Elsevier retracting 500 papers for shoddy peer review. (2022). Retraction Watch. https://retractionwatch.com/2022/10/28/exclusive-elsevier-retracting-500-papers-for-shoddy-peer-review/ (accessed on 17 December 2022).
A tale of 32 retraction notices: On publishers, paper mill products, and the sleuths that find them. (2022). Retraction Watch. https://retractionwatch.com/2022/08/15/a-tale-of-32-retraction-notices-on-publishers-paper-mill-products-and-the-sleuths-that-find-them/ (accessed on 17 December 2022).
Physics publisher retracting nearly 500 likely paper mill papers. (2022). Retraction Watch. https://retractionwatch.com/2022/09/09/physics-publisher-retracting-nearly-500-likely-paper-mill-papers/ (accessed on 17 December 2022).
Exclusive: Hindawi and Wiley to retract over 500 papers linked to peer review rings. (2022). Retraction Watch. https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/ (accessed on 17 December 2022).
More than 300 at once: Publisher retracts entire conference proceedings. (2022). Retraction Watch. https://retractionwatch.com/2022/04/20/more-than-300-at-once-publisher-retracts-entire-conference-proceedings/ (accessed on 17 December 2022).
'Our deepest apology': Journal retracts 30 likely paper mill articles after investigation published by Retraction Watch. (2022). Retraction Watch. https://retractionwatch.com/2022/07/05/our-deepest-apology-journal-retracts-30-likely-paper-mill-articles-after-investigation-published-by-retraction-watch/ (accessed on 17 December 2022).
Australian universities to return to 'pen and paper' exams after students caught using AI to write essays. (2023). The Guardian. https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to-pen-and-paper-exams-after-students-caught-using-ai-to-write-essays (accessed on 20 January 2023).
Published
How to Cite
Issue
Section
Copyright (c) 2023 Urfa Khairatun Hisan, Muhammad Miftahul Amri
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
