Paging Dr. ChatGPT: safety, accuracy and readability of ChatGPT in ENT emergencies

Introduction

The widespread availability of health information on the internet has led to an age of “Doctor Google”. A 2018 Australian study by Cocco et al. found that 49% of patients regularly searched for health information online, with 34.8% of patients searching their symptoms prior to presenting to the emergency department (1). While easy access to information allows patients gain better understanding of their medical conditions, this can also lead to information overload, health anxiety, and the risk of misinterpretation ​(2).

Effective patient education should be safe, accurate, and easy to understand. In Australia, 44% of adults read at, or below, the Year 11 level (3). Readability is affected by various factors: including word count, sentence length, and syllables per word. Readability metrics assess the reading level required for comprehension of any body of text. Examples of these include the Flesch Readability Ease Score, Flesch Kincaid Grade Level and Gunning Fog Index (4).

ChatGPT is an artificial intelligence (AI) model launched by OpenAI in November 2022 (5). This chatbot is trained on publicly available information and uses machine learning algorithms to produce answers to user’s questions (5). Since its launch, ChatGPT has attracted considerable curiosity and attention, reaching 100 million users within 2 months of launching (6).

A 2023 study by Nov et al. found that 38.8% of patients trusted ChatGPT’s diagnostic advice (7). ChatGPT’s role in patient education has been explored in various medical specialties such as obstetrics, internal medicine, and dermatology (8-10). Its value lies in providing easy access to a broad spectrum of information. Within Ear, Nose and Throat (ENT) Surgery, there is limited literature assessing the readability of ChatGPT responses specifically related to ENT queries. Existing studies primarily focus on readability in relation to patient education for ENT procedures (11,12). Importantly, there is a lack of research addressing the readability of ChatGPT information tailored to the Australian population.

ENT symptoms represent a significant portion of emergency department presentations (13). Enhanced patient education in ENT emergencies is essential for fostering better patient understanding, promoting adherence to treatment plans and improving clinical outcomes (14).

Our study reviewed the accuracy, safety and readability of ChatGPT in providing patient information on common ENT emergency conditions. To our knowledge, this is the first study to review the readability of ChatGPT responses in relation to advice on ENT emergencies.

Methods

This study received an ethics exemption from the Metro North Human Research Ethics Committee given it did not involve human participation, and the data collected was freely available from a public domain.

10 scenarios, consisting of 17 questions of common ENT emergency conditions were input into the free version of ChatGPT-version 3.5. These scenarios can be seen in Table 1. Scenarios were selected based on common ENT emergency presentations encountered in practice. Questions were worded as first-hand accounts without medical jargon to simulate questions that may be asked by patients from a nonmedical background. Each scenario was asked three times over three days (June 17–19, 2024) using three separate accounts to assess the reproducibility and variability of ChatGPT responses.

ChatGPT answers were rated by three Royal Australasian College of Surgeons (RACS) ENT consultants on a Likert scale judging accuracy and safety from a scale of one to five. The consultants subspecialised in otology, head and neck surgery and rhinology respectively. The scale for accuracy ranged from 1—completely incorrect to 5—entirely correct. The scale for safety ranged from 1—extremely unsafe advice to 5—extremely safe advice.

Linguistic features such as word count, syllables per word and words per sentence were calculated to provide a comprehensive breakdown of text complexity. Spearman correlation and statistical significance using a t-test was calculated for the three RACS ENT Scores and for the ChatGPT answers over three days. Statistical significance was set to P<0.05. This examined if ChatGPT answers were correlated over three days and if the results from the three ENTs were similarly correlated.

ChatGPT answers were analysed for their readability using the readability metrics of the Flesch Kincaid Grade Level, Flesch Readability Ease Score, Coleman-Liau Index, Gunning Fog Index, and Simple Measure of Gobbledygook (SMOG) Index. The Flesch Readability Ease Score provides a score from 0–100 that correlates to the reading difficulty level of the document, with 100 being the highest readability score appropriate for a primary school reading age. All other readability metrics used in this study estimate a reading grade that correspond to the years of formal education required to comprehend the text (15,16).

Microsoft Word version 16.86, Microsoft Excel version 16.86 and IBM SPSS Version 30.0.0.0 were used for data analysis.

Results

The mean accuracy of ChatGPT answers to prompts was rated more correct than incorrect on a Likert Scale (3.8±0.3). The mean safety of ChatGPT answers to prompts was rated somewhat safe (4.3±0.4). These can be seen in Table 2. The range in safety ratings was 1.1. and the range in accuracy ratings was 1. Most accuracy ratings clustered around 3.7 and 3.9, with four scenarios falling within this range. Most safety ratings clustered around 4.1, 4.3 and 4.7, with six scenarios falling in this range.

Accuracy and safety were rated lowest on the prompt on acute mastoiditis with scores of 3.3 and 3.6 respectively. Safety was rated extremely safe (5) on questions pertaining to post tonsillectomy bleed, periorbital cellulitis and epiglottitis. Accuracy and safety were rated highest on the prompt on epiglottitis with mean ratings of 4.3 and 4.7 respectively. Safety was also rated highest on the prompt on post tonsillectomy bleed with a score of 4.7.

The correlation coefficient and its statistical significance by the three RACS ENTs can be seen in Tables 3,4. All RACS ENT scores on the safety of ChatGPT responses demonstrated strong and statistically significant correlations (r=0.761, 0.834, 0.810; P<0.05). Additionally, the accuracy scores from our three RACS ENT showed positive correlations. However, the relationship between the accuracy ratings of RACS ENT 1 and RACS ENT 3 while positively correlated, did not reach statistical significance (P=0.20).

The correlation coefficient and its statistical significance for the ChatGPT answers over three days can be seen in Tables 5,6. Our results show a strong and statistically significant positive correlation between the accuracy and safety of day 1 and day 3 ChatGPT answers. This indicates that the safety and accuracy rating on day 1 and day 3 were relatively correlated. However, correlation between ChatGPT answers on day 1 and day 2, and day 2 and day 3 was weak and not statistically significant. This suggests that no reliable relationship existed between the accuracy and safety of ChatGPT answers on these days.

The average number of words per sentence was 13.9, with a maximum of 19.1 words and a minimum of 8.6 words per sentence. This can be seen in Table 7.

From the five-readability metrics used, ChatGPT had a mean readability score of an undergraduate student with 14.1±1.5 years of formal education. This can be seen in Table 8 and Figure 1. Scenario 2—post-tonsillectomy bleed—had the lowest mean readability level scoring 12.2. Scenario 9—malignant otitis externa had the highest mean readability level scoring 16.5. All mean readability scores were higher or equivalent to the reading level of a Year 12 student.

Figure 1 Readability levels of ChatGPT responses across ENT emergency scenarios using multiple readability metrics. ENT, Ear, Nose and Throat.

The years of education required to understand the text varied by readability metric, generally falling within the range suitable for high school to university graduates. The Flesch Kincaid Grade level suggested ChatGPT answers required the lowest education requirement with a mean of 12.4±1.4, suggesting that the text is appropriate for students in year 12 and above. The Flesch Reading Ease score suggested that readers required the highest education requirement with a mean of 28.9±10.9. This score indicates that ChatGPT answers were very difficult to read and at the reading level of a university graduate. The SMOG, Coleman-Liau and Gunning Fog Index all suggested the readability level of ChatGPT corresponds to an undergraduate student. The mean SMOG and Coleman-Liau Index were 15.1±1.4 and 15±1.6 respectively. This indicates that an individual would require approximately 15 years of education to understand the text, roughly corresponding to an undergraduate student. Similarly, the mean Gunning Fog Index was 13.5±1.8, indicating an individual would require approximately 13 years of formal education to understand ChatGPT answers to the scenarios.

Discussion

Accuracy and safety

The findings of this study suggest that ChatGPT responses across the scenarios were reasonably safe and accurate, as judged by ENT surgeons assessing responses. All RACS ENT scores concerning safety showed strong and statistically significant correlations (P<0.05). This indicates that as one consultant’s rating increased, ratings from other consultants tended to rise as well. However, the correlation between the accuracy ratings of RACS ENT 1 and RACS ENT 3, while moderate, did not reach statistical significance (P=0.20).

The mean accuracy and safety ratings of ChatGPT answers were consistently above 3 out of 5. Accuracy ratings ranged from 3 (a balanced mix of correct and incorrect responses) to a perfect score of 5 (completely correct responses). Safety ratings similarly ranged from 4 (answers somewhat safe) to 5 (answers extremely safe). The safety and accuracy of ChatGPT answers were strongly correlated and statistically significant on day 1 and day 3. However, there was a weak correlation was noted in ChatGPT answers on day 2. This suggests some variation in ChatGPT responses. While there was a meaningful relationship on specific days, this could not be consistently applied across all the days analysed. Hence, users should note that the safety and accuracy of ChatGPT responses may vary depending on the day.

The findings of this study on the accuracy of information provided by ChatGPT are corroborated by literature. For example, a 2024 Australian study by Chen et al. conducted on common ENT emergency presentations found that ChatGPT had an appropriate triage for 75.6% of scenarios (17). Kuscu et al. found that information provided by ChatGPT was reproducible in 94.1% of answers (18), with an 11% rate of incorrect answers.

However, there are concerns regarding the wider adoption of ChatGPT in clinical medicine. Firstly, ChatGPT tends to produce “hallucinations”. This is a phenomenon where ChatGPT has been observed to fabricate information. In ENT research, this was first identified by Texeira Marques et al., who found that ChatGPT fabricated findings on an audiogram (19). The results of our study did not identify any confabulation or hallucinations. However, we acknowledge that this may present with more complex questions. To date, there are no studies examining the frequency of ChatGPT hallucinations in response to ENT questions. Hillmann et al. (20) and Hong et al. (21) both found that ChatGPT confabulated <5% of answers. Ali et al. (22) found that ChatGPT version 3.5 had a 20% higher rate of confabulation than ChatGPT version 4 on neurosurgical board examination questions. This poses a major drawback and risk of adopting ChatGPT more broadly. Furthermore, although ChatGPT seemed to provide reasonably safe answers for ENT emergencies with well-established safe treatment algorithms, this was less so for less common or more nuanced scenarios (23,24). This may mean that ChatGPT is less suited for more complex situations with increased difficulty obtaining safe and accurate answers. These pose major drawbacks and risks of adopting ChatGPT more broadly. Thus, while our findings suggest ChatGPT’s advice across our scenarios was reasonably safe and accurate, care needs to be taken given its current limitations.

Readability

Making informed decisions about healthcare relies not only on the accuracy of information presented, but also the reader’s ability to comprehend and apply that information effectively. The Australian Style Manual is a guideline provided by the Australian government for written information. This recommends that information intended for the Australian public be written to a Year 7 reading level as based on Australian literacy levels over 75% of the population would be likely to understand the content (25). It also recommends keeping sentence lengths to an average of 15 words, with a maximum of 25 words (3).

This study found that ChatGPT had a mean readability score 7 years greater than the recommended Australian reading level (14.1±1.5). However, the average number of words per sentence was in keeping with recommendations at a mean of 13.9±4.2.

About 44% of Australian adults have a reading level below Year 11, with literacy levels increasing with age until the late 20s and declining in 40s (3,25). The disparity between ChatGPT’s reading grade and the reading level of the average Australian limits the utility of ChatGPT in providing effective health information. The complexity of patient information on ChatGPT may exclude a large proportion of the Australian public. Of the readability metrics, the SMOG index is thought to be better suited for healthcare applications due to its use of more recent validation criteria to determine reading grade (16). The mean and median SMOG index was higher at 15.1 and 15±1.4 respectively, placing the readability at least 8 years higher than that of the recommended Australian government guidelines.

Interestingly, the lowest mean readability grade was found in scenario 2—post tonsillectomy bleed. This indicates a mean reading level of 12 years of formal education. This scenario had the highest safety rating as judged by ENT surgeons who are well trained in this field. However, with poor readability, how this information is interpreted by the target audience, which are patients, is difficult to know. Mean readability grade levels for all ENT emergency scenarios were greater than or equal to 12 years of formal education—equivalent to the reading level of Year 12 or greater. As of 2023, 63% of Australians aged 15–74 had a non-school qualification (certificate, diploma or degree) and 31.9% had a Bachelors degree or higher (26). While the level of education does not guarantee a reading level, a 2019 Australian study found a correlation between years of formal education and reading level (27).

It is however important to note that health-related written materials typically utilize polysyllabic technical jargon, which can inflate readability formula scores. The highest mean readability grade was found in scenario 6—malignant otitis externa with a mean grade of 16.5. This indicates that 16.5 years of formal education—the approximate reading age of a university graduate would be required to comprehend ChatGPT answers to that scenario. This scenario also had the highest mean syllables per word at 2.2. This may be attributed to the use of polysyllable words such as Pseudomonas aeruginosa used in the context of describing causes of otitis externa. This may reflect the complexity of the issue, which is less easy to articulate in simple language with better readability.

The findings of this study correlate with studies in various other medical fields that found ChatGPT had a readability level higher than the average reader (28,29). For example, Sahin et al. found that ChatGPT answers to questions on otosclerosis surgery had a median SMOG level of 12.3 (12). Polat et al. found that ChatGPT answers to prompts on common ENT surgeries such as tonsillectomy had a Flesch Grade Level of 9.95 (11).

When compared to Google search on various orthopaedic terms, ChatGPT answers were found to be significantly harder to read (30). Various studies have reviewed if variations to ChatGPT prompts could affect the readability of patient information. Ayre et al. examined the ability of ChatGPT to simplify existing health information and found that revisions to the readability level by ChatGPT were marginal and still unable to meet health literacy targets (31). Abou-Abdallah et al. observed that ChatGPT could simplify information on simple ENT operations such as tonsillectomy when prompted but this compromised quality and led to answers with significant omissions (32). Lee et al. found that when the question prompt included information on the identity of the user e.g., “I am a patient” versus “I am a physician”, there was a statistically significant difference in readability (28).

This study suggests that ChatGPT provides medical information that is generally safe and accurate when asked questions on ENT emergencies. However, there is a risk of incorrect or unsafe advice that can be difficult to assess for accuracy. Furthermore, information is often presented at a literacy level that may not be suitable for the broader Australian public. This limits its use as a medium for patient education as patients may not fully comprehend the information provided. It is crucial to acknowledge that AI technology is constantly advancing, and as of now, ChatGPT’s updates are current only until September 2021. As ChatGPT evolves further with new updates, its role in ENT practice will also evolve.

Limitations

This study used the free version of ChatGPT - version 3.5 to simulate the average user experience. A 2024 study by Lee et al. found that ChatGPT version 4.0, which currently needs to be paid for, provided significantly improved word count and readability levels relative to version 3.5 (28). Thus, this study may not accurately reflect the full capabilities of all ChatGPT iterations. This raises an ethical question as individuals with poorer literacy may be less likely to pay for a service that provides more readable information. Furthermore, this study only examines ENT emergency scenarios with specific wording of prompts entered. ChatGPT answers were also assessed by surgeons, who already had a thorough grounding in the content discussed, leading to interpretations that differ from those of patients without ENT training. Patients may also enter less specific queries when seeking medical information from ChatGPT. This may change the nature and reliability of the information that ChatGPT provides, and would need further study to be able to identify the implications of various means of asking medical questions. Additionally, large language models can adjust responses to a lower reading level when prompted. This study did not explore this and thus was unable to elicit if lowering the reading level of responses would affect the safety and accuracy of responses.

There is an ever-growing number of large language models, including BERT, Claude, Mistral, and even medical specific models such as Consensus and OpenEvidence. Future studies should examine the readability of ChatGPT and other AI models and explore how these can assist or be altered to provide safe and reliable information that patients can easily understand. Additionally, future research should compare ChatGPT to other information sources, such as internet search engines and readily available patient information resources.

Conclusions

ChatGPT appears to have reasonable safety and accuracy; however, potential issues with the accuracy and safety of information provided pose significant medical, ethical and legal implications. These implications are evolving and may change over time. The readability of answers provided was two times more than the recommended Australian guidelines on readability. This poses a barrier as it excludes individuals with lower literacy levels. Despite this, ChatGPT shows promise as a valuable tool in patient information. It is important to note that a limitation of this study is that it only evaluated prompts related to ENT emergencies. As newer iterations of ChatGPT are developed, the utility of ChatGPT in clinical medicine will continue to evolve.

Acknowledgments

None.

Footnote

Data Sharing Statement: Available at https://www.theajo.com/article/view/10.21037/ajo-24-56/dss

Peer Review File: Available at https://www.theajo.com/article/view/10.21037/ajo-24-56/prf

Funding: None.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://www.theajo.com/article/view/10.21037/ajo-24-56/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study received an ethics exemption from the Metro North Human Research Ethics Committee given it did not involve human participation, and the data collected was freely available from a public domain.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Cocco AM, Zordan R, Taylor DM, et al. Dr Google in the ED: searching for online health information by adult emergency department patients. Med J Aust 2018;209:342-7. [Crossref] [PubMed]
2. Bujnowska-Fedak MM, Węgierek P. The Impact of Online Health Information on Patient Health Behaviours and Making Decisions Concerning Health. Int J Environ Res Public Health 2020;17:880. [Crossref] [PubMed]
3. Literacy and access | Style Manual. [cited 2024 Jul 8]. Available online: https://www.stylemanual.gov.au/accessible-and-inclusive-content/literacy-and-access
4. Nash E, Bickerstaff M, Chetwynd AJ, et al. The readability of parent information leaflets in paediatric studies. Pediatr Res 2023;94:1166-71. [Crossref] [PubMed]
5. Egli A. ChatGPT, GPT-4, and Other Large Language Models: The Next Revolution for Clinical Microbiology? Clin Infect Dis 2023;77:1322-8. [Crossref] [PubMed]
6. Eysenbach G. The Role of ChatGPT, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation With ChatGPT and a Call for Papers. JMIR Med Educ 2023;9:e46885. [Crossref] [PubMed]
7. Nov O, Singh N, Mann D. Putting ChatGPT's Medical Advice to the (Turing) Test: Survey Study. JMIR Med Educ 2023;9:e46939. [Crossref] [PubMed]
8. Rames MM, O'Hern K, Rames JD, et al. ChatGPT as a resource for patient education in cosmetic dermatological procedures: A boon or a bane? J Cosmet Dermatol 2024;23:1085-6. [Crossref] [PubMed]
9. Almagazzachi A, Mustafa A, Eighaei Sedeh A, et al. Generative Artificial Intelligence in Patient Education: ChatGPT Takes on Hypertension Questions. Cureus 2024;16:e53441. [Crossref] [PubMed]
10. Monje S, Ulene S, Gimovsky AC. Identifying ChatGPT-Written Patient Education Materials Using Text Analysis and Readability. Am J Perinatol 2024;41:2229-31. [Crossref] [PubMed]
11. Polat E, Polat YB, Senturk E, et al. Evaluating the accuracy and readability of ChatGPT in providing parental guidance for adenoidectomy, tonsillectomy, and ventilation tube insertion surgery. Int J Pediatr Otorhinolaryngol 2024;181:111998. [Crossref] [PubMed]
12. Sahin S, Erkmen B, Duymaz YK, et al. Evaluating ChatGPT-4's performance as a digital health advisor for otosclerosis surgery. Front Surg. 2024;11:1373843. [Crossref] [PubMed]
13. Merino-Galvez E, Gomez-Hervas J, Perez-Mestre D, et al. Epidemiology of otorhinolaryngologic emergencies in a secondary hospital: analysis of 64,054 cases. Eur Arch Otorhinolaryngol 2019;276:911-7. [Crossref] [PubMed]
14. Paterick TE, Patel N, Tajik AJ, et al. Improving health outcomes through patient education and partnerships with patients. Proc (Bayl Univ Med Cent) 2017;30:112-3. [Crossref] [PubMed]
15. Kher A, Johnson S, Griffith R. Readability Assessment of Online Patient Education Material on Congestive Heart Failure. Adv Prev Med 2017;2017:9780317. [Crossref] [PubMed]
16. Wang LW, Miller MJ, Schmitt MR, et al. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm 2013;9:503-16. [Crossref] [PubMed]
17. Chen FJ, Nightingale J, You WS, et al. Assessment of ChatGPT vs. Bard vs. guidelines in the artificial intelligence (AI) preclinical management of otorhinolaryngological (ENT) emergencies. Aust J Otolaryngol 2024;7:19. [Crossref]
18. Kuşcu O, Pamuk AE, Sütay Süslü N, et al. Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer? Front Oncol 2023;13:1256459. [Crossref] [PubMed]
19. Teixeira-Marques F, Medeiros N, Nazaré F, et al. Exploring the role of ChatGPT in clinical decision-making in otorhinolaryngology: a ChatGPT designed study. Eur Arch Otorhinolaryngol 2024;281:2023-30. [Crossref] [PubMed]
20. Hillmann HAK, Angelini E, Karfoul N, et al. Accuracy and comprehensibility of chat-based artificial intelligence for patient information on atrial fibrillation and cardiac implantable electronic devices. Europace 2023;26:euad369. [Crossref] [PubMed]
21. Hong J, Calais J, Benz M, et al. Comparative analysis of large language models: ChatGPT and Google Bard answer non-expert questions related to the diagnostic and therapeutic applications of prostate-specific membrane antigen (PSMA) in patients with prostate cancer. Journal of Nuclear Medicine 2024;65:242416.
22. AliRTangOYConnollyIDPerformance of ChatGPT, GPT-4, and Google Bard on a Neurosurgery Oral Boards Preparation Question Bank.medRxiv. 2023;2023.04.06.23288265. Available online: https://www.medrxiv.org/content/10.1101/2023.04.06.23288265v1 10.1101/2023.04.06.23288265
23. Nielsen JPS, von Buchwald C, Grønhøj C. Validity of the large language model ChatGPT (GPT4) as a patient information source in otolaryngology by a variety of doctors in a tertiary otorhinolaryngology department. Acta Otolaryngol 2023;143:779-82. [Crossref] [PubMed]
24. Vaira LA, Lechien JR, Abbate V, et al. Accuracy of ChatGPT-Generated Information on Head and Neck and Oromaxillofacial Surgery: A Multicenter Collaborative Analysis. Otolaryngol Head Neck Surg 2024;170:1492-503. [Crossref] [PubMed]
25. Programme for the International Assessment of Adult Competencies, Australia, 2011 - 2012 | Australian Bureau of Statistics. [cited 2024 Jul 8]. Available online: https://www.abs.gov.au/statistics/people/education/programme-international-assessment-adult-competencies-australia/latest-release
26. Education and Work, Australia, May 2023 | Australian Bureau of Statistics. [cited 2024 Jul 8]. Available online: https://www.abs.gov.au/statistics/people/education/education-and-work-australia/latest-release
27. Oliffe M, Thompson E, Johnston J, et al. Assessing the readability and patient comprehension of rheumatology medicine information sheets: a cross-sectional Health Literacy Study. BMJ Open 2019;9:e024582. [Crossref] [PubMed]
28. Lee TJ, Rao AK, Campbell DJ, et al. Evaluating ChatGPT-3.5 and ChatGPT-4.0 Responses on Hyperlipidemia for Patient Education. Cureus 2024;16:e61067. [Crossref] [PubMed]
29. Onder CE, Koc G, Gokbulut P, et al. Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy. Sci Rep 2024;14:243. [Crossref] [PubMed]
30. Ulusoy I, Yılmaz M, Kıvrak A. How Efficient Is ChatGPT in Accessing Accurate and Quality Health-Related Information? Cureus 2023;15:e46662. [Crossref] [PubMed]
31. Ayre J, Mac O, McCaffery K, et al. New Frontiers in Health Literacy: Using ChatGPT to Simplify Health Information for People in the Community. J Gen Intern Med 2024;39:573-7. [Crossref] [PubMed]
32. Abou-Abdallah M, Dar T, Mahmudzade Y, et al. The quality and readability of patient information provided by ChatGPT: can AI reliably explain common ENT operations? Eur Arch Otorhinolaryngol 2024;281:6147-53. [Crossref] [PubMed]