Hip and knee arthroplasty are common procedures worldwide and are increasing annually as demographics change and the technical aspects of these surgeries become more accessible to a broader swath of surgeons. The sheer number of these procedures makes them an appropriate focus for randomized controlled trials (RCTs). The aggregation of RCT data into more powerful statistical frameworks is the job of a meta-analysis.
Not surprisingly, we have seen an increasing number of meta-analyses related to hip and knee replacement published across all major orthopaedic journals during the last two decades. Authors have two common motivations for conducting meta-analyses. The first, to summarize data from carefully conducted RCTs into clinically relevant and important recommendations, is hopefully the most common motivation—and certainly the most justifiable. The second, to merely use previously published data as an analytic exercise to advance one’s academic career without investing the time and effort to do prospective research, is not justifiable, in my estimation.
In the December 4, 2019 issue of The Journal, Park et al. conduct quality and usefulness assessments of 114 published meta-analyses about hip and knee arthroplasty that appeared in 3 major orthopaedic journals (one of which was JBJS) from 2000 to 2017. They document a nearly 4-fold increase in the number of meta-analyses published on these topics when comparing 2000 to 2009 with 2010 to 2017. Based on Oxman-Guyatt Index scores of overall study quality, only 12 of the 114 studies were assessed as high quality, 87 as moderate quality, and 15 as low quality.
Here are some additional findings:
- The majority of these meta-analyses were not performed in accordance with established PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.
- Only 39% of these articles showed the risk of bias.
- Many of these meta-analyses covered redundant topics within the same year or within a few years of each other.
- A review by expert attending surgeons of the 24 studies determined to be high quality per PRISMA found that 71% were either clinically unimportant or inconclusive.
It is a positive step to highlight for our readers the important quality issues surrounding meta-analyses, and I agree with James Stoney, who commented on these findings: “The onus is on surgeons to carefully scrutinize meta-analyses…and come to individual conclusions about the quality of the research rather than accept the conclusions at face value.”
But I am discouraged to see the number of problematic meta-analyses that have appeared in our literature, and I suspect most of these quality problems arise from the second, unjustifiable motivation noted above. We need to do better as a research community, as peer reviewers, and as journal editors to improve the quality of published meta-analyses so that we can favorably impact patient care and advance the clinical practice of hip and knee arthroplasty.
Marc Swiontkowski, MD
JBJS Editor-in-Chief
Excellent commentary! Hit the nail right on the head!
Jim ________________________________
Thanks for this Commentary Editor. With the challenges to find funding for original research, the Meta-analysis publications have become the “low hanging fruit”. Highlighting the limitations of this research (primarily poor methodology) was timely and needed!