The July 2021 OrthoJOE podcasts from JBJS and OrthoEvidence, featuring Mohit Bhandari, MD and Marc Swiontkowski, MD, covered 2 topics of noted interest to orthopaedic surgeons: machine learning and the fragility index (FI).
Machine learning, a subset of artificial intelligence (AI), is the study of computer algorithms that can improve automatically through experience and the use of data. In OrthoJOE episode 15, “Machine-Learning Algorithms in Orthopaedics,” Drs. Bhandari and Swiontkowski discuss the opportunities and challenges of machine learning in our field. They note a recent study in which the authors aimed to develop machine-learning algorithms that could successfully predict which athletes will achieve clinically meaningful improvement after undergoing primary hip arthroscopy for femoroacetabular impingement syndrome1. Nearly 77% of the athletes achieved the minimally clinically important difference (MCID) for the Hip Outcome Score-Sports Subscale (HOS-SS) at a minimum of 2 years. Six patient covariates were responsible for algorithm performance optimization; there was a consistently decreased likelihood of achieving the MCID if a patient had a:
- Preoperative HOS-SS score ≥ 58.3
- Alpha angle of ≥ 67.1°
- BMI of >26.6 kg/m2
- Tönnis angle >9.7°
- Tönnis grade of 1
- Age of >40 years
The best-performing algorithm was the elastic-net penalized logistic regression (ENPLR) model. More on this study can be found in this previous OrthoBuzz post.
Among the take-home points outlined in the podcast:
- Widespread clinical adoption of this particular machine-learning algorithm will not be possible until it is externally validated, but machine learning nonetheless will help us move the orthopaedic surgery field forward once we take time to understand the principles and learn the nomenclature
- At its core, this is a study of prognosis using regression techniques
- It is unlikely that AI will replace what we do daily
- We need to create datasets that are of high quality, specific to AI and machine-learning algorithms
- We must continue to educate one another
In OrthoJOE episode 16, “The Fragility Index: Why Is It Important to the Practicing Surgeon?,” Drs. Swiontkowski and Bhandari discuss how the FI is a sobering reminder that evidence-based medicine in our surgical field needs more large multicenter clinical trials to answer fundamental questions on improving and optimizing orthopaedic care. Fundamentally, the FI is a statistical measure for evaluating the robustness of the results of a clinical trial with dichotomous outcomes. Or simply put, the FI is a number indicating how many patients would be needed to convert the findings of a trial from statistically significant to nonsignificant. Authors from McMaster University conducted a systematic review to determine the FI in randomized controlled trials related to primary total joint arthroplasty2. A total of 34 RCTs met the inclusion criteria, with a median sample size of 103 patients (range, 24 to 791). Using a Fisher exact test, the median FI was determined to be 1 (range, 0 to 45), indicating that reversing the outcome of only one patient in either treatment group of each study would lead to a change from a significant to nonsignificant result. Compared to previously published studies across numerous orthopaedic subspecialties, the median FI for primary total hip and knee arthroplasty is the lowest2.
Among the take-home points:
- The fragility of RCTs for primary total hip and knee arthroplasty is startling
- We may be misleading ourselves if we rely too heavily on small clinical trials to guide our clinical decision-making. Striving toward large multicenter trials may better serve us in answering important questions in orthopaedic surgery
- Small trials (i.e., single-center trials with 100 patients) may not provide definitive evidence when fragility of the findings is high
- Meta-analysis does not eliminate this issue because of heterogeneity in study design and methodology as well as bias
- Evidence-based medicine, from its onset, principally begins and ends with the patient, with the goal of utilizing the best available evidence to inform the patient and the clinician while discussing the risk-to-benefit ratio of a particular treatment strategy
David Kovacevic, MD, FAAOS, is an orthopaedic surgeon who specializes in shoulder, elbow, and sports medicine surgery. He is also a member of the JBJS Social Media Advisory Board.
To access other OrthoJOE episodes or to subscribe to the podcast, click here.
References
- Kunze KN, Polce EM, Clapp I, Nwachukwu BU, Chahla J, Nho SJ. Machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes. J Bone Joint Surg Am. 2021 Jun; 103(12): 1055-62. doi: 10.2106/JBJS.20.01640.
- Ekhtiari S, Gazendam AM, Nucci NW, Kruse CC, Bhandari M. The fragility of statistically significant findings from randomized controlled trials in hip and knee arthroplasty. J Arthroplasty. 2021 Jun; 36(6): 2211-8. doi: 10.1016/j.arth.2020.12.015.