Clinical Relevance vs Statistical Significance: The Good, The Bad, and The Future of Sexual Harassment in Orthopaedic Surgery 

This guest post comes from Jennifer Beck, MD. 

As a female, mid-career orthopaedic surgeon, I believe I bring a unique perspective to reporting on the changes that I have witnessed over the past 15 years in academic orthopaedic surgery. It is well documented that ours is an overly White, male-dominated field, often associated with the stereotype of “dumb jock” doctors bringing locker-room humor to the operating room. Stereotypes and parodies abound on various social media platforms. Additionally, it is widely known that cultural and demographic change in orthopaedics has occurred at a pace slower than in other fields in medicine.  

I must clarify that the news isn’t all bad. Traiblazing individuals including female orthopaedic department chairs (Drs. Lisa Lattanza, Susan Bukata, Michelle Caird, April Armstrong, Valerae Lewis, Leesa Galatz, and Evalina Burger) and society presidents (Drs. Kristy Weber, Lori Karol, Serena Hu, Mary O’Connor, Heather Vallier, Bess Brackett, Marybeth Ezaki, Jo Hannafin, Judy Baumhauer, Lisa Cannada) and groups like the Ruth Jackson Orthopaedic Society, SpeakUp Ortho, the International Orthopaedic Diversity Alliance, and The FORUM have created pathways for positive change in support of diversity. At a time when health care is being attacked from all sides, we must band together for change and cannot lose momentum they have created. 

When reviewing statistical analysis of my clinical research, the perfectionist in me wants to achieve that elusive “statistical significance” so I can say, “Yes, there was a difference.” However, when finding nonsignificant results, I am often left wondering if they are clinically relevant or useful. At the recent 2021 Annual Meeting of the American Academy of Orthopaedic Surgeons, Dr. Emily Whickers and colleagues presented a poster on sexual harassment in the field of orthopaedics. Based on a survey of members of the Ruth Jackson Orthopaedic Society, her group initiated their study “to better understand harassment in orthopedic medicine in light of ‘the stories that we had all heard,’” Whickers told MedPage Today.  

They did not find a statistically significant decline in harassment during orthopaedic training, with 59% of the current residents reporting harassment vs 72% of the past residents; p = 0.10. How can orthopaedic surgery continue to make progress toward “significance”?   

Encouragingly, the authors did find a trend that current residents felt more comfortable reporting harassment, a step toward defining and acknowledging the problems through a supportive culture. Now is the time to thoughtfully reflect on and openly discuss successful programs that have been instituted and their effect. We can learn from other’s successes, and failures.  

 As we do so, let’s discuss options to the 4 characteristics that contribute to this problem as reported by the National Academies of Sciences, Engineering, and Medicine: 

  1. Male-dominated field 
  • Improve the pipeline of female/nonbinary, LGBTQ, and underrepresented minority members of our field through programs and organizations such as the J. Robert Gladden Society and The Perry Initiative 
  • Create and adequately resource diversity, equity, and inclusion committees and programs 
  • Appropriately promote and actively retain women (who are leaving medicine at record rates due to the COVID-19 pandemic)  
  1. Organizational tolerance 
  • Decrease the prevalence of men who “fail up” in medicine 
  • Create pathways for reporting and evaluating sexual harassment in a safe and nonjudgmental fashion 
  • Create and support resources for victims of sexual harassment 
  • Create and enforce repercussions for offenders through education, behavior modification, and situational modification 
  • Absolve any sense of retaliation through the medical hierarchy 
  1. Hierarchical and dependent relationships during surgical training 
  • Create an educational environment that is supportive and engaging of open conversations on critical issues and topics 
  • Identify and resource faculty mentors who can work with victims and perpetrators of sexual harassment 
  1. Isolationist feelings of female residents 
  • See recommendations from Point 1 
  • Create programs focusing on the needs of female surgical trainees in all subspecialities 
  • Hire, promote, and retain female faculty and mentors 
  • Hire, promote, and retain male faculty and mentors who encourage and engage in open conversations 

It is easy to become discouraged when efforts may not be producing the results as quickly as we want. But as numerous Peloton instructors say, “It’s progress, not perfection” that matters. Sexual harassment is not a one-time, one-solution fix. It’s not a checkbox on a to-do list. It’s the daily grind and grassroots cultural change, the need for program and policy creation and implementation, the constant evaluation of program efficacy, and the continued awareness of the struggle that will lead to the progress we desire for an inclusive and safe field of orthopaedic surgery.  

Jennifer Beck, MD is a pediatric sports medicine surgeon at UCLA and a member of the JBJS Social Media Advisory Board. She thanks Jennifer Weiss, MD and Selina Poon, MD, both pediatric orthopaedic surgeons, for their help with initial editing of this post. 

Elite Reviewer Spotlight: Edward Joseph Harvey

JBJS is pleased once again to highlight our Elite Reviewers. The Elite Reviewers Program recognizes our best reviewers for their outstanding efforts. All JBJS reviewers help us maintain the highest standards for quality orthopaedic publishing.

Name:

Edward J. Harvey, MD, MSc, FRCSC

Affiliation:

McGill University, Montreal, QC Canada

Years in practice: 24

How did you begin reviewing for other journals and for JBJS in particular?

I was offered the opportunity to review after several submissions – most of them unsuccessful LOL – and was happy to contribute to the community.

How do you maintain a healthy work/life balance?

I have always balanced my clinical work and research. But I prioritize my family and recreation as an equal time partner in order to prevent work overload. This means saying no sometimes.

Aside from orthopaedic manuscripts, what have you been reading lately?

Tons of business books – reading Built to Last right now and just finished Barking up the Wrong Tree.

Learn more about the JBJS Elite Reviewers program.

Elite Reviewer Spotlight: Douglas R. Dirschl

JBJS is pleased once again to highlight our Elite Reviewers. The Elite Reviewers Program recognizes our best reviewers for their outstanding efforts. All JBJS reviewers help us maintain the highest standards for quality orthopaedic publishing.

Name:

Douglas R. Dirschl, MD

Affiliation:

Department of Orthopaedic Surgery and Rehabilitation Medicine, University of Chicago

Chicago, IL

Years in practice: 28

How did you begin reviewing for other journals and for JBJS in particular?

I have always greatly enjoyed writing, reviewing and editing manuscripts.  I sought out the opportunity early in my career to begin to review for JOT and JBJS, and I have focused most of my reviewing time on those two journals.  I look forward to the invitations to review manuscripts and probably get 3-5 invitations per week. I feel that providing thoughtful and timely reviews is what has led me to be invited to provide a few commentary pieces for JBJS.

What is your top piece of advice for those reviewers who aspire to reach Elite status?

  • Be timely in getting reviews done (we love to get prompt feedback when we submit manuscripts as authors, so as reviewers we should endeavor to do the same).
  • Base your critiques/comments on items that will improve the internal and external validity of the manuscript. Do the study design, sample size, results, and analysis appropriately answer the study question? Is the result relevant to patient care?
  • Be specific in your critiques: the authors need to understand clearly what action to take to remediate your concern.

Aside from orthopaedic manuscripts, what have you been reading lately?

Backpacking and mountain climbing books, Andy Weir novels, The Boys of Summer, civil war history

Learn more about the JBJS Elite Reviewers program.

Cost, Value, and Maintenance of Certification: One Surgeon’s Thoughts  

This guest post comes from David Vizurraga, MD in response to a recent JBJS Orthopaedic Forum article. Additional perspective on this topic is provided by JBJS Editor-in-Chief Dr. Marc Swiontkowski in a related editorial and by The American Board of Orthopaedic Surgery’s Board of Directors in a commentary. 

As Sun Tzu taught, “victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win.” As orthopaedic surgeons, we are challenged with foes in the form of musculoskeletal pathology that demand our knowledge and treatment. To be victorious, we must empower ourselves with the ever-expanding knowledge of these conditions and the ever-evolving techniques used to treat them. Failure is not an option, as defeat means not only a loss for us but for our patients as well. To that end, we must prepare and assess ourselves through continuous learning and feedback.  

Since 1986, the American Board of Orthopaedic Surgery (ABOS), our specialty’s branch of the American Board of Medical Specialties (ABMS), has overseen the Maintenance of Certification (MOC) program. In its current form, the ABOS MOC allows us to individually tailor our development to our own needs while simultaneously providing feedback through evaluation over a 10-year period. Recently the cost of MOC across medical specialties has drawn increasing attention.   

In a recent report in JBJSLaVigne et al. estimate the costs of each of the MOC’s potential pathways on the basis of their time costs and fees. To account for time, the authors distributed a survey asking respondents to select which pathway they chose and to provide estimates on the amount of time spent performing various components. This survey was only distributed to a single state’s orthopaedic society and yielded only 33 responses for inclusion. Leveraging previous reports, the authors then determined the average hourly rate of compensation for orthopaedic surgeons. Merging this hourly compensation and the obtained time estimates, they were able to estimate the time cost of each pathway. Although most fees were established, the Continuous Medical Education (CME) fee was calculated from the Accreditation Council for Continuing Medical Education’s (ACCME’s) total income from CME in 2010 divided by the approximately 850,000 licensed physicians in 2010.  

The authors calculated the average orthopaedic surgeon’s total 10-year costs of MOC to be $71,440.61 for the oral examination pathway, $80,392.78 for the written examination pathway (with dedicated study and review course), $68,871.78 for the written examination pathway (without dedicated study and review course), and $69,721.04 for the ABOS Web-Based Longitudinal Assessment (WLA) pathway.    

While cost represents one side of the coin and is that which must be paid to obtain something, worth represents the other side and is that which is obtained in return. The value of continuing education, skill development, and even camaraderie or professional networking gained through CME and professional meetings must also be considered. So too the value of board certification as determined by an assessment of potential costs imposed on surgeon, patient, and society by negative outcomes at the hands of a surgeon who is not board certified.  

At the same time, while orthopaedic surgeons are among the best-compensated specialists, we must also acknowledge that some of us, military surgeons for example, are not compensated with competitivecontracts, rewarded with robust surgical volumes, or reimbursed for regular fees. Given this and basic principles of finance, this study and others like it will continue to push the ABOS to evolve and develop new strategies that maintain our professional standard, minimize cost, and provide considerable value to us, our patients, and society at large. 

David Vizurraga, MD is a San Antonio-based orthopaedic surgeon specializing in adult hip and knee reconstruction and a member of the JBJS Social Media Advisory Board.

JBJS Case Connector: Celebrating 10 Years  

JBJS is pleased to note that this year marks the 10th anniversary of JBJS Case Connector. Over the past decade, Case Connector has seen rapid growth toward its goal of establishing a sizable inventory of high-quality, peer-reviewed case reports—enriched by a collection of keywords and search functionality that will allow healthcare providers to recognize commonalities between cases, benefit from the experience of their peers, identify trends, and distinguish between truly rare cases and repeated single instances of a larger problem.  

Insight into Case Connector’s journey and mission can be found in a recent editorial by Co-Editors Dr. Tom Bauer and Dr. Ron Lindsey and Editor-in-Chief Dr. Marc Swiontkowski. 

From just over 200 manuscripts submitted from 28 countries in 2011, with 13 articles published that first year, Case Connector received more than 1,000 case report submissions from 54 countries, with 320 articles published, in 2020. Reports have been authored by academic scientists, private practitioners, residents, physical therapists, medical students, and other healthcare professionals.  

JBJS extends its thanks to all who have contributed to Case Connector’s success to date. Authors interested in submitting an article to Case Connector are encouraged to visit the Author Resource Center for additional information and guidance.  

Time Is Running Out…

…to apply for the JBJS Robert Bucholz Journal Club Support Program for the 2021-2022 academic year. Applications are due by September 30, 2021.

Please click here for application information.

Today we heard from Dr. Paula Ramirez, chief resident at the University of Chile’s Department of Orthopaedics and Traumatology. Along with sharing these photos of their first journal club meeting, Dr. Ramirez said, “Thanks again for your support. This pandemic has not been easy for us, but we managed to keep motivated as residents to learn new evidence-based medicine and stay updated. We are definitely applying again this year.”

In Pursuit of Alternative Antibiotics for Use in PMMA Bone Cement

The incorporation of antibiotics within polymethylmethacrylate (PMMA) has been widely used over recent decades for managing infection following skeletal trauma. Early research helped to clarify which antibiotics in which formulations were potentially clinically effective, with a common application of managing “dead space” following debridement of bone and soft tissue, addressing established infection as well as preventing deep infection. As the microbiology involved in these infections evolves, along with the antibiotics available, we have need for continued research into this important area of orthopaedics.

In the September 15, 2021 issue of JBJS, Levack et al. report on their investigation into the suitability of alternative antibiotics (amikacin, meropenem, minocycline, and fosfomycin) for use in PMMA beads,  with a particular focus on thermal stability and in vitro elution characteristics. Tobramycin was also used to validate the study methodology. Minimum inhibitory concentrations of the antibiotics were tested against S. aureus, E. coli, and Acinetobacter baumannii. Antibiotic-laden PMMA beads of different sizes were tested, with antibiotic elution determined using high-performance liquid chromatography with mass spectrometry.

The authors found that amikacin was comparable to tobramycin with respect to heat stability and elution. Meropenem showed favorable elution kinetics and thermal stability in the initial 7 days.

The investigators emphasize that “The data presented are intended to generate further study of these antibiotics to better identify potential areas of clinical utility,” and they rightly point out that their data are not intended for clinical decision-making, “as antibiotic dosages and in vivo applications, specifically with biofilms, have not been evaluated.”  Nonetheless, these new data involving the characteristics of amikacin and meropenem are intriguing. Moreover, this study serves as a great reminder of the need to regularly reevaluate established therapies as research techniques, pharmacology, and clinical conditions (such as evolving microbial pathogens) continue to change.

Marc Swiontkowski, MD
JBJS Editor-in-Chief

Spotlight on Recent OrthoJOE Episodes

This guest post comes from David Kovacevic, MD, FAAOS, who provides a summary overview of recent episodes of the OrthoJOE podcast 

The July 2021 OrthoJOE podcasts from JBJS and OrthoEvidence, featuring Mohit Bhandari, MD and Marc Swiontkowski, MD, covered 2 topics of noted interest to orthopaedic surgeons: machine learning and the fragility index (FI).  

Machine learning, a subset of artificial intelligence (AI), is the study of computer algorithms that can improve automatically through experience and the use of data. In OrthoJOE episode 15, “Machine-Learning Algorithms in Orthopaedics,” Drs. Bhandari and Swiontkowski discuss the opportunities and challenges of machine learning in our field. They note a recent study in which the authors aimed to develop machine-learning algorithms that could successfully predict which athletes will achieve clinically meaningful improvement after undergoing primary hip arthroscopy for femoroacetabular impingement syndrome1. Nearly 77% of the athletes achieved the minimally clinically important difference (MCID) for the Hip Outcome Score-Sports Subscale (HOS-SS) at a minimum of 2 years. Six patient covariates were responsible for algorithm performance optimization; there was a consistently decreased likelihood of achieving the MCID if a patient had a:  

  • Preoperative HOS-SS score ≥ 58.3 
  • Alpha angle of ≥ 67.1° 
  • BMI of >26.6 kg/m2 
  • Tönnis angle >9.7° 
  • Tönnis grade of 1
  • Age of >40 years 

The best-performing algorithm was the elastic-net penalized logistic regression (ENPLR) model. More on this study can be found in this previous OrthoBuzz post.   

Among the take-home points outlined in the podcast:  

  • Widespread clinical adoption of this particular machine-learning algorithm will not be possible until it is externally validated, but machine learning nonetheless will help us move the orthopaedic surgery field forward once we take time to understand the principles and learn the nomenclature  
  • At its core, this is a study of prognosis using regression techniques 
  • It is unlikely that AI will replace what we do daily  
  • We need to create datasets that are of high quality, specific to AI and machine-learning algorithms 
  • We must continue to educate one another 

In OrthoJOE episode 16, “The Fragility Index: Why Is It Important to the Practicing Surgeon?,” Drs. Swiontkowski and Bhandari discuss how the FI is a sobering reminder that evidence-based medicine in our surgical field needs more large multicenter clinical trials to answer fundamental questions on improving and optimizing orthopaedic care. Fundamentally, the FI is a statistical measure for evaluating the robustness of the results of a clinical trial with dichotomous outcomes. Or simply put, the FI is a number indicating how many patients would be needed to convert the findings of a trial from statistically significant to nonsignificant. Authors from McMaster University conducted a systematic review to determine the FI in randomized controlled trials related to primary total joint arthroplasty2. A total of 34 RCTs met the inclusion criteria, with a median sample size of 103 patients (range, 24 to 791). Using a Fisher exact test, the median FI was determined to be 1 (range, 0 to 45), indicating that reversing the outcome of only one patient in either treatment group of each study would lead to a change from a significant to nonsignificant result. Compared to previously published studies across numerous orthopaedic subspecialties, the median FI for primary total hip and knee arthroplasty is the lowest2.   

Among the take-home points:  

  • The fragility of RCTs for primary total hip and knee arthroplasty is startling  
  • We may be misleading ourselves if we rely too heavily on small clinical trials to guide our clinical decision-making. Striving toward large multicenter trials may better serve us in answering important questions in orthopaedic surgery  
  • Small trials (i.e., single-center trials with 100 patients) may not provide definitive evidence when fragility of the findings is high 
  • Meta-analysis does not eliminate this issue because of heterogeneity in study design and methodology as well as bias  
  • Evidence-based medicine, from its onset, principally begins and ends with the patient, with the goal of utilizing the best available evidence to inform the patient and the clinician while discussing the risk-to-benefit ratio of a particular treatment strategy 

David Kovacevic, MD, FAAOS, is an orthopaedic surgeon who specializes in shoulder, elbow, and sports medicine surgery. He is also a member of the JBJS Social Media Advisory Board. 

To access other OrthoJOE episodes or to subscribe to the podcast, click here. 

References 

  1. Kunze KN, Polce EM, Clapp I, Nwachukwu BU, Chahla J, Nho SJ. Machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes. J Bone Joint Surg Am. 2021 Jun; 103(12): 1055-62. doi: 10.2106/JBJS.20.01640.  
  2. Ekhtiari S, Gazendam AM, Nucci NW, Kruse CC, Bhandari M. The fragility of statistically significant findings from randomized controlled trials in hip and knee arthroplasty. J Arthroplasty. 2021 Jun; 36(6): 2211-8. doi: 10.1016/j.arth.2020.12.015. 

Video Summary: Association Between Knee Alignment and Meniscal Tear in Pediatric Patients with Anterior Cruciate Ligament Injury

Delayed ACL reconstruction in patients ≤16 years old with varus-aligned knees might be associated with an increased incidence of secondary medial meniscal tears.

Read the full article here.