Assessment of a surgical simulation tool for airway graft carving using 3D-printing technology

Introduction

Practical experience is an important component of surgical training. However, acquiring certain procedural skills can be challenging. This is true for laryngeal and airway reconstruction (1). In addition to complex techniques and unique instrumentation required, the high acuity and low frequency of these surgeries means gaining adequate exposure can be difficult (2). Understandably, the use of simulation models is emerging as an effective tool in surgical education. Simulation allows for trainees to develop skills outside of the operating theatre—providing a safe, low-risk environment (1,3). There have been multiple studies on airway surgery simulations using animal models (4-7). Although some animal models may ensure an aspect of realism, they may be limited by cost, access, and ethical considerations (8).

Three-dimensional (3D) printing, also referred to as ‘additive manufacturing’, is an attractive alternative approach. 3D models produced from computer-aided design (CAD) have been utilized in many sub-specialty areas within otorhinolaryngology training (9-13). This technology allows printing of a structure that can reproduce visuospatial and tactile features of a surgical technique (14). In particular, relating to airway reconstruction, a study by Ha et al. (2017) demonstrated the value and low-cost of a simulation tool using 3D-printed assisted synthetic cartilage for airway graft carving, a technique used in laryngotracheal reconstruction (LTR) (15). The aim of this study is to assess the usefulness of a 3D printing and CAD as a high-fidelity simulator for developing skills in airway reconstruction in Australian Otolaryngology training.

We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/ajo-20-48).

Methods

Simulation was conducted over one year during the Ear, Nose and Throat (ENT) Sinus Surgery Workshop in Perth, Western Australia (WA), the Australian Society of Otolaryngology Head and Neck Surgery (ASOHNS) annual meeting in Perth, and the WA ENT trainees breakfast education session.

Ethical statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Exemption granted under the ASOHNS ethics review committee for research within the medical profession, and consent was obtained from participants at time of study.

Production of simulation tool

As per previously published methods (9,15), a negative mold of a standardized 3D-printed representation of a harvested costal cartilage graft of a normal 8-year-old pediatric rib was created via segmentation of a computed tomography (CT) scan. This was manufactured out of an acrylonitrile butadiene styrene polymer using a fused-deposition modeling 3D printer. The simulation tool was then fabricated by filling the mold with a 2:1 ratio of pure cornstarch and silicone (GE Silicone II White). The composition of the material has been validated by a prior study (15).

Recruitment

The participants included consultant otolaryngologists, surgical education and training (SET) and non-SET registrars, fellows, and medical students. Participants had pre-simulation instructions via viewing a short instructional video used in our previous pilot study (15).

Simulation and assessment

All participants completed sequential simulations of carving an anterior and a posterior graft for airway reconstruction. After completion, all participants completed a previously-validated Likert scale survey which was used in our earlier study, which had been modified to question the simulator characteristics (15,16). Figure 1 illustrates the scales used in the different domains assessed.

Figure 1 Representation of the scales used in the participant survey for the domains assessed.

Statistical analysis

Data was analyzed with SPSS Statistics (SPSS Inc.) software. Chi square test, independent sample student’s t-test and paired sample student’s t-test were used as appropriate. Statistical significance was defined as 5% (α=0.05).

Results

Demographics

There were 46 participants. Approximately half of participants (n=24, 52.2%) were more than seven years from medical school graduation (post-graduate year; PGY). Of all participants, 12 were qualified Otolaryngologists, two were fellows, 14 were SET trainee registrars, 16 were non-SET registrars, and two were medical students (Figure 2). Majority of participants (n=27, 58.7%) had never participated in laryngotracheal reconstruction (LTR) procedures or cartilage grafting. Of the participants who had performed these procedures, the number of LTR procedures previously performed ranged widely from 1 to >250 procedures.

Figure 2 Demographic of participants by training (n=46).

Realism of experience

Eighteen of the 46 participants selected ‘do not know’ when asked to assess the realism of the 3D synthetic costal cartilage, of which all but one did not have any LTR experience. To better evaluate the realism of the 3D synthetic costal cartilage, only the responses of participants with prior LTR experience (n=17) were considered. The majority (n=12, 70.6%) felt that the realism of the cartilage was adequate, four participants (25%) felt that the cartilage grafts were highly realistic, and one participant did not respond. There was a significant association between SET level and the rating of realism of the anterior graft experience (P=0.01) with those not in SET training ranking it less favorably.

The previous LTR experience and the number of cartilage grafts performed was also significantly associated with the favourable ratings of the realism of the cartilage (P<0.01; P<0.01), and both the anterior (P=0.02; P<0.01) and posterior grafts (P<0.01; P=0.01).

Ability and confidence

Participant confidence after performing one anterior rib grafting procedure was rated on a scale (Figure 1). Of the participants who responded, more than half (n=22/40, 55%) reported confidence after the simulations. Similar results of confidence were evident with the posterior graft (n=18/34, 53%). There was a significant association with training level and the ability to perform anterior (P=0.04) and posterior (P=0.012) grafts, with the Otolaryngologists significantly more confident at performing the anterior (P=0.01) and posterior (P=0.025) grafts. Those not in SET training were more likely to score themselves lower for their ability at posterior grafts (P<0.01). Previous LTR and cartilage grafting experience was also strongly associated with the confidence to perform both the anterior (P=0.02; P=0.02) and posterior (P=0.02; P<0.01) grafts.

Value as a training tool and relevance to practice

Despite missing data points for participants, 95% (n=38/40) rated the simulation as having value and 84.6% (n=33/39) rate the simulation as having relevance for training purposes. In total, of the 17 participants with prior LTR experience, only three thought the model required moderate improvements, and the remaining participants believed that this tool could be used in training with no further improvements required (n=6, 35.3%) or only slight improvements required (n=6, 35.3%).

There was a strong association between PGY level and rating of this as a valuable training tool (P=0.014), as well as the level of SET training and the global score (P=0.006), with those not in SET training rating this as a highly valuable tool in training. In addition, those who had performed less LTR in the past felt that this was a relevant training tool in their practice (P=0.02). The participants who were not qualified Otolaryngologists were more significantly likely to rate this as a valuable tool for training (P=0.013).

Discussion

Specialized skills are required in the management of paediatric airway conditions (8). Surgical simulation allows a means to practise and master these advanced skills within a safe environment (1,3). In our experience, the 3D-printed synthetic costal cartilage grafting tool allowed participants to practise a skill-set used in LTR and to assess the value of such a model for surgical education. A recent systematic review by Alwani et al. (2019) has shown that not only is there undoubtedly benefit in surgical simulation models within otolaryngological training, but that a large proportion of these models were actually formulated using 3D printing—thus highlighting the growing interest and accessibility within this field (3).

Three major aspects of the model used in this study were assessed by participants: (I) realism of experience, (II) participant confidence and ability, and (III) value as a tool and relevance to practice. Almost 40% of participants reported that they were unsure whether they could comment on the realism of the cartilage grafts, whilst the remaining participants predominantly felt the models were either adequately or highly realistic. This is not a surprising result given that in order to comment on the accuracy of a model, one would likely need to be exposed to the genuine version. In other words, experience would predetermine a participant’s ability to compare the synthetic costal cartilage graft to a real human costal cartilage graft harvested in LTR. This finding is supported by our participant demographics (ranging from medical students to novice registrars to practiced Otolaryngologist)—58.7% had never participated in or observed LTR procedures. As a result, there was a significant association between rating the realism of the cartilage grafts (both anterior and posterior) with prior experience, such that the SET-registrars and Otolaryngologists rated it higher than the other participants. Similarly, in a LTR simulation study by Kavanagh and Murray (2019), although using a low-fidelity technique, they too found that participant inexperience affected their ability to assess the likeness of the simulator model (8).

In order to investigate the model’s effect on a participant’s confidence and self-reported ability, the questionnaire asked for scoring after performing the grafts. More than half of all participants felt relatively confident or very confident after performing the simulation. This highlights that even with brief exposure to a procedure, a simulation can provide not only familiarity with a complex surgical skill, but may also improve confidence to perform it. However, it is important to note, just like the assessment of graft realism, again there was a significant association between the training level and the ability to perform the grafts, with those with greater experience more likely to rate their themselves higher in ability and confidence. Schwartz et al. (2018) assessed participants who performed simulated paediatric airway procedures using self-reported confidence scores both pre- and post- simulation (17). In hindsight, this analysis (comparing confidence both before and after) would have been useful in our study to better delineate participant confidence and individual improvement, regardless of prior training.

Overall, the simulation used in this study was rated highly by participants, with majority describing it as both valuable and relevant to training (95% and 84.6%, respectively). In fact, most participants who had prior LTR exposure did not think the simulator model required moderate or large improvements. Similarly, with the other domains evaluated, the participants’ prior experiences were significantly associated with their assessment of relevance and value. Those who were not qualified or in SET training were significantly more likely to rate the tool higher as a valuable and relevant tool for training. This is likely explained by the fact that these participants had little or no experience with LTR, and therefore this simulation provided them with a rare opportunity to practice this skill, thus reporting a greater appreciation for it.

Compared to the United States pilot study by Ha et al. (2017) on the use of CAD and 3D printing for costal cartilage simulation, we too demonstrate similar results with regards to the findings of the simulation model as both relevant and valuable. However, in contrast to this study, Ha et al. (2017) reported there was no differentiation made between participants based on their PGY level. This difference is likely a consequence of participant heterogeneity and demographics.

Limitations of this study include the small sample size. This was predicted and somewhat unavoidable, given that it was performed in Western Australia, and the relative number of ENT service and trainee registrars as well as consultants is less when compared to most other states (18). This small number was further affected by missing data and incomplete questionnaires. Additionally, this study bears the potential for response bias, given Likert questionnaires may prompt participants to overvalue the simulator. More objective measures, including assessments both pre- and post- simulation to assess participant progress, may rectify this. A weakness of this study is that there was no control arm. Consequently, as mentioned above, the variables of participant experience could not be eliminated from the effect on their responses on the assessment of the simulation tool.

In conclusion, this simulation allowed participants of all stages in their surgical career to both learn and practise specific LTR skills within the context of a safe environment, and with low-cost materials which provide realism and relevancy to training (15). With emerging technology used in the field of CAD and 3D printing, future investigations should focus on validation studies of these simulators for the purpose of surgical education. In particular, it would be useful to assess not only the simulator itself, but also the participant’s performance and retention of skills learnt.

Acknowledgments

The authors would like to acknowledge Hannah Burns, ASOHNS, and CTEC for allowing us to recruit participants for this study at their conferences.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at http://dx.doi.org/10.21037/ajo-20-48

Data Sharing Statement: Available at http://dx.doi.org/10.21037/ajo-20-48

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/ajo-20-48). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Exemption granted under the ASOHNS ethics review committee for research within the medical profession, and consent was obtained from participants at time of study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Burns JA, Adkins LK, Dailey S, et al. Simulators for laryngeal and airway surgery. Otolaryngol Clin North Am 2017;50:903-22. [Crossref] [PubMed]
2. Myer CM, Jabbour N. Advanced pediatric airway simulation. Otolaryngol Clin North Am 2017;50:923-31. [Crossref] [PubMed]
3. Alwani M, Bandali E, Larsen M, et al. Current state of surgical simulation training in otolaryngology: systematic review of simulation training models. Archives of Otorhinolaryngology-Head & Neck Surgery 2019;3:5. [Crossref]
4. Milner TD, Okhovat S, Clement WA, et al. A systematic review of simulated laryngotracheal reconstruction animal models. Laryngoscope 2019;129:235-43. [Crossref] [PubMed]
5. Deonarain AR, Harrison RV, Gordon KA, et al. Live porcine model for surgical training in tracheostomy and open-airway surgery. Laryngoscope 2020;130:2063-8. [Crossref] [PubMed]
6. Ranguis S, Ryan P, Cheng A. Ovine model as a surgical simulator for pediatric laryngotracheal reconstruction. Int J Pediatr Otorhinolaryngol 2019;118:181-4. [Crossref] [PubMed]
7. Soliman AM, Ianacone DC, Isaacson GC. Ex vivo ovine model for teaching open laryngotracheal surgery. World J Otorhinolaryngol Head Neck Surg 2018;4:140-4. [Crossref] [PubMed]
8. Kavanagh KR, Murray N. A novel, low fidelity simulator for laryngotracheal reconstruction. Int J Pediatr Otorhinolaryngol 2019;125:212-5. [Crossref] [PubMed]
9. Berens AM, Newman S, Bhrany AD, et al. Computer-aided design and 3D printing to produce a costal cartilage model for simulation of auricular reconstruction. Otolaryngol Head Neck Surg 2016;155:356-9. [Crossref] [PubMed]
10. Barber SR, Kozin ED, Dedmon M, et al. 3D-printed pediatric endoscopic ear surgery simulator for surgical training. Int J Pediatr Otorhinolaryngol 2016;90:113-8. [Crossref] [PubMed]
11. Cote V, Schwartz M, Vargas JFA, et al. 3-Dimensional printed haptic simulation model to teach incomplete cleft palate surgery in an international setting. Int J Pediatr Otorhinolaryngol 2018;113:292-7. [Crossref] [PubMed]
12. Gadaleta DJ, Huang D, Rankin N, et al. 3D printed temporal bone as a tool for otologic surgery simulation. Am J Otolaryngol 2020;41:102273 [Crossref] [PubMed]
13. Sparks D, Kavanagh KR, Vargas JA, et al. 3D printed myringotomy and tube simulation as an introduction to otolaryngology for medical students. Int J Pediatr Otorhinolaryngol 2020;128:109730 [Crossref] [PubMed]
14. Rose AS, Kimbell JS, Webster CE, Harrysson OL, Formeister EJ, Buchman CA. Multi-material 3D Models for Temporal Bone Surgical Simulation. Ann Otol Rhinol Laryngol 2015;124:528-36. [Crossref] [PubMed]
15. Ha JF, Morrison RJ, Green GE, et al. Computer-aided design and 3-dimensional printing for costal cartilage simulation of airway graft carving. Otolaryngol Head Neck Surg 2017;156:1044-7. [Crossref] [PubMed]
16. Rooney DM, Tai BL, Sagher O, et al. Simulator and 2 tools: validation of performance measures from a novel neurosurgery simulation model using the current Standards framework. Surgery 2016;160:571-9. [Crossref] [PubMed]
17. Schwartz MA, Kavanagh KR, Frampton SJ, et al. Using simulators to teach pediatric airway procedures in an international setting. Int J Pediatr Otorhinolaryngol 2018;104:178-81. [Crossref] [PubMed]
18. Health Do. Otolaryngology Workforce 2016 Factsheet. 2016. Available online: https://hwd.health.gov.au/webapi/customer/documents/factsheets/2016/Otolaryngology.pdf