ANKMJ
Ankyra Medical Journal

Ankyra Medical Journal (AnkMJ), formerly known as the Journal of Translational and Practical Medicine, regularly publishes international quality issues in the field of Medicine in the light of current information.

eISSN: 3023-655X
Creative Commons CC BY-NC-ND 4.0 International License.
EndNote Style

Online Manuscript System

Index
Original Article
Exploring the competence of artificial intelligence programs in the field of oculofacial plastic and orbital surgery
Abstract
Aims: It aims to evaluate the knowledge level of ChatGPT, Bing, and Bard artificial intelligence chatbots developed based on Large Language Models (LLM) about oculofacial plastic surgery and to investigate the presence of superiority over each other.
Methods: Twenty-nine questions that tested knowledge about oculofacial plastic and orbital surgery were taken from the study questions section of the American Academy and Ophthalmology 2022-2023 Basic and Clinical Science Course Oculofacial Plastic and Orbital Surgery. The questions were asked to ChatGPT, Bing, and Bard programs, which are current artificial intelligence chatbots. The questions were classified as either correct or incorrect.
Results: ChatGPT gave 44.8% correct answers, Bing 48.3% correct answers, and Bard 58.6% correct answers to 29 questions about artificial intelligence chatbots. No statistical difference was observed between the rates of correct and incorrect answers given by 3 the intelligence programs (p=0.609, Pearson’s chi-squared test).
Conclusion: The use of artificial intelligence to access information regarding oculofacial plastic and orbital surgery may provide limited benefits. Care should be taken in terms of accuracy and timeliness when evaluating the results of artificial intelligence programs.

Keywords : Bard, Bing, ChatGPT, oculofacial plastic surgery, orbital surgery

Download Article


1. Rahimy E. Deep learning applications in ophthalmology. Curr Opin Ophthalmol. 2018;29(3):254-260.
2. Patel VL, Shortliffe EH, Stefanelli M, et al. The coming of age of artificial intelligence in medicine. Artif Intell Med. 2009;46(1):5-17.
3. Mikolov T, Deoras A, Povey D, Burget L, Cernocký J. Strategies for training large scale neural network language models. 2011 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2011, Proceedings. Published online 2011:196-201.
4. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Ali MJ. ChatGPT and lacrimal drainage disorders: performance and scope of improvement. Ophthalmic Plast Reconstr Surg. 2023;39(3):221-225.
5. Alqahtani H, Kavakli-Thorne M, Kumar G. Applications of generative adversarial networks (GANs): an updated review. Arch Comput Meth Engineering. 2021;28(2):525-552.
6. ChatGPT Generative Pre-trained Transformer, Zhavoronkov A. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience. 2022;9:82-84.
7. O’Connor S. Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Educ Pract. 2023;66:103537.
8. Korn BS, Burkat CN, Carter KD, et al, eds. Oculofacial Plastic andOrbital Surgery. Vol 7. American Academy of Ophthalmology: 2022.
9. Wen J, Wang W. The future of ChatGPT in academic research and publishing: a commentary for clinical and translational medicine. Clin Transl Med. 2023;13(3):e1207.
10. Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT - reshaping medical education and clinical management. Pak J Med Sci. 2023;39(2):605.
11. Gao CA, Howard FM, Markov NS, et al. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. BioRxiv. 2022:2022.12.23.521610.
12. Jeblick K, Schachtner B, Dexl J, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol. Published online December 30, 2022. doi: 10.1007/s00330-023-10213-1
13. Cai LZ, Shaheen A, Jin A, et al. Performance of generative large language models on ophthalmology board style questions. Am J Ophthalmol. 2023;254:141-149.
14. Korngiebel DM, Mooney SD. Considering the possibilities and pitfalls of generative pre-trained transformer 3 (GPT-3) in healthcare delivery. NPJ Digit Med. 2021;4(1):93.
15. Nath S, Marie A, Ellershaw S, Korot E, Keane PA. New meaning for NLP: the trials and tribulations of natural language processing with GPT-3 in ophthalmology. Br J Ophthalmol. 2022;106(7):889-892.
16. Jin Q, Dhingra B, Liu Z, Cohen WW, Lu X. Pubmedqa: a dataset for biomedical research question answering. arXiv preprint arXiv: 1909.06146.
17. Jin D, Pan E, Oufattole N, Weng WH, Fang H, Szolovits P. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Appl Sci. 2021;11(14):6421.
18. Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023;2(2):e0000198.
Article available in :
Volume 3, Issue 3, 2024
Page : 63-65
https://doi.org/10.51271/ANKMJ-0014
Article Information
Received : 15 Nis 2024
Accepted : 20 May 2024
Published : 28 May 2024
Corresponding Author :

Eyüpcan Şensoy: Department of Ophthalmology, Ankara Etlik City Hospital Ankara, Turkiye

[email protected]
Citation : Şensoy E, Çıtırık M. Exploring the competence of artificial intelligence programs in the field of oculofacial plastic and orbital surgery. Ank Med J. 2024;3(3):63-65.

Read: 250
Cited: 0
Download : 143
_Footer