Meta-Analysis Quality Improving, But Issues Remain
Hip and knee arthroplasty are common procedures worldwide and are increasing annually as demographics change and the technical aspects of these surgeries become more accessible to a broader swath of surgeons. The sheer number of these procedures makes them an appropriate focus for randomized controlled trials (RCTs). The aggregation of RCT data into more powerful statistical frameworks is the job of a meta-analysis.
Not surprisingly, we have seen an increasing number of meta-analyses related to hip and knee replacement published across all major orthopaedic journals during the last two decades. Authors have two common motivations for conducting meta-analyses. The first, to summarize data from carefully conducted RCTs into clinically relevant and important recommendations, is hopefully the most common motivation—and certainly the most justifiable. The second, to merely use previously published data as an analytic exercise to advance one’s academic career without investing the time and effort to do prospective research, is not justifiable, in my estimation.
In the December 4, 2019 issue of The Journal, Park et al. conduct quality and usefulness assessments of 114 published meta-analyses about hip and knee arthroplasty that appeared in 3 major orthopaedic journals (one of which was JBJS) from 2000 to 2017. They document a nearly 4-fold increase in the number of meta-analyses published on these topics when comparing 2000 to 2009 with 2010 to 2017. Based on Oxman-Guyatt Index scores of overall study quality, only 12 of the 114 studies were assessed as high quality, 87 as moderate quality, and 15 as low quality.
Here are some additional findings:
- The majority of these meta-analyses were not performed in accordance with established PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.
- Only 39% of these articles showed the risk of bias.
- Many of these meta-analyses covered redundant topics within the same year or within a few years of each other.
- A review by expert attending surgeons of the 24 studies determined to be high quality per PRISMA found that 71% were either clinically unimportant or inconclusive.
It is a positive step to highlight for our readers the important quality issues surrounding meta-analyses, and I agree with James Stoney, who commented on these findings: “The onus is on surgeons to carefully scrutinize meta-analyses…and come to individual conclusions about the quality of the research rather than accept the conclusions at face value.”
But I am discouraged to see the number of problematic meta-analyses that have appeared in our literature, and I suspect most of these quality problems arise from the second, unjustifiable motivation noted above. We need to do better as a research community, as peer reviewers, and as journal editors to improve the quality of published meta-analyses so that we can favorably impact patient care and advance the clinical practice of hip and knee arthroplasty.
Marc Swiontkowski, MD
JBJS Editor-in-Chief
Opioid-Tapering Plan May Help Prevent Prolonged Use after Trauma/Surgery
Addressing the opioid epidemic requires a concerted effort from all sectors of society, but the role of surgeons (orthopaedic and otherwise) cannot be ignored because they determine how best to manage postoperative pain for millions of patients. OrthoBuzz recently commented on two opioid-related studies from the July 18, 2018 issue of JBJS. In the August 1, 2018 edition of The Journal, Mohamadi et al. explain findings from a meta-analysis of 37 studies involving nearly 2 million patients that pinpoint several patient-related risk factors associated with opioid use beyond 2 months following surgery or trauma.
Using careful meta-analysis methods, the authors determined that about 4% of patients continued to use prescription opioids beyond 2 months after surgery or trauma. They also identified the following risk factors as being “among the most important predictors of prolonged opioid use” in these patients:
- Prior use of opioids or benzodiazepines
- Depression
- Long-duration hospital stay
- History of back pain
Mohamadi et al. also calculated a “number needed to harm” (NNH) from their data. NNH indicates the number of patients with a certain risk factor that is necessary to result in 1 person with prolonged opioid use beyond that of a patient population without that risk factor. They found that for every 3 patients with a history of opioid use, every 23 patients with a history of back pain, every 40 with depression, or every 62 with a history of benzodiazepine use, 1 patient will continue to use prescribed opioids for an extended time period.
Because this meta-analysis was derived from observational studies, the authors caution that “causal inferences could not be drawn for the proposed risk factors.” But they do offer a practical piece of advice gleaned from prior research: Provide patients with an opioid-tapering plan at the time of discharge to significantly reduce the likelihood of prolonged opioid use.
Soccer Players Benefit from Ankle-Injury Prevention Programs
A Level-I meta-analysis by Grimm et al. in the September 7, 2016 issue of The Journal of Bone & Joint Surgery found a significant reduction in the risk of ankle injury among soccer athletes who participated in ankle-injury prevention programs. Researchers reviewed data from 10 randomized controlled trials of such prevention programs involving more than 4,000 female and male soccer players, applying random-effects statistical models to determine pooled risk differences. Not surprisingly, the authors found substantial heterogeneity among the included studies, but there was no evidence of publication bias.
Despite the overall finding of a protective effect from prevention programs, the authors were “unable to comment on the role of individual elements of injury prevention programs,” saying that further research is needed to evaluate the effectiveness of specific exercises and the optimal timing and age for implementing these programs.
Recent Comments