Tag Archive | bias

Big Data Needs Big Progress

This post comes from Fred Nelson, MD, an orthopaedic surgeon in the Department of Orthopedics at Henry Ford Hospital and a clinical associate professor at Wayne State Medical School. Some of Dr. Nelson’s tips go out weekly to more than 3,000 members of the Orthopaedic Research Society (ORS), and all are distributed to more than 30 orthopaedic residency programs. Those not sent to the ORS are periodically reposted in OrthoBuzz with the permission of Dr. Nelson.

The broad term “big data,” when applied to health care, refers to the mining of large databases to find information that might predict and improve clinical outcomes on a national scale. A recent article in AAOSNow cites one example of the merging of big data with artificial intelligence (AI) as the 10-year partnership between the Mayo Clinic and Google to leverage cloud technologies, machine learning, and AI to accelerate change in healthcare delivery. The American Joint Replacement Registry (AJRR) is also using big data to help hospitals make more efficient supply-chain decisions and lower costs.

However, there are limitations and potential flaws in the use of big data. One is the high cost, making it unaffordable for some institutions. In addition, variations in how the data is collected and reported may lead to flawed analyses. Also, data collection may vary in completeness by region, which makes nationwide registries with consistent data collection so important. Big data can also be contaminated by bias. Even large datasets may over- or underrepresent certain groups of people, thereby skewing any analysis made with those data.

There are several solutions to improve the use of big data in orthopaedics. One is the development of registries with uniform and consistent data collection methods to ensure equity. Participation in registries by orthopaedic surgeons is critical. The authors of the AAOSNow article also emphasize that if patient data were linked longitudinally, researchers would have a powerful tool with which to study health outcomes and monitor public health trends. However, current HIPAA rules prevent clearinghouses from linking data that way. To update the law to match our data-driven reality, in 2017 US Rep. Cathy McMorris Rodgers (D-Ore.) introduced the Ensuring Patient Access to Health Records Act (H.R. 4613), which would allow greater access to big data for the purpose of research, public health, and personal patient use. The bill has been tied up in committee since December 15, 2017. The ultimate objective of using big data in medicine is to provide health care that is “predictive, preventive, personalized, and participatory,” conclude the authors.

Meta-Analysis Quality Improving, But Issues Remain

Hip and knee arthroplasty are common procedures worldwide and are increasing annually as demographics change and the technical aspects of these surgeries become more accessible to a broader swath of surgeons. The sheer number of these procedures makes them an appropriate focus for randomized controlled trials (RCTs). The aggregation of RCT data into more powerful statistical frameworks is the job of a meta-analysis.

Not surprisingly, we have seen an increasing number of meta-analyses related to hip and knee replacement published across all major orthopaedic journals during the last two decades. Authors have two common motivations for conducting meta-analyses. The first, to summarize data from carefully conducted RCTs into clinically relevant and important recommendations, is hopefully the most common motivation—and certainly the most justifiable. The second, to merely use previously published data as an analytic exercise to advance one’s academic career without investing the time and effort to do prospective research, is not justifiable, in my estimation.

In the December 4, 2019 issue of The Journal, Park et al. conduct quality and usefulness assessments of 114 published meta-analyses about hip and knee arthroplasty that appeared in 3 major orthopaedic journals (one of which was JBJS) from 2000 to 2017. They document a nearly 4-fold increase in the number of meta-analyses published on these topics when comparing 2000 to 2009 with 2010 to 2017. Based on Oxman-Guyatt Index scores of overall study quality, only 12 of the 114 studies were assessed as high quality, 87 as moderate quality, and 15 as low quality.

Here are some additional findings:

  • The majority of these meta-analyses were not performed in accordance with established PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.
  • Only 39% of these articles showed the risk of bias.
  • Many of these meta-analyses covered redundant topics within the same year or within a few years of each other.
  • A review by expert attending surgeons of the 24 studies determined to be high quality per PRISMA found that 71% were either clinically unimportant or inconclusive.

It is a positive step to highlight for our readers the important quality issues surrounding meta-analyses, and I agree with James Stoney, who commented on these findings: “The onus is on surgeons to carefully scrutinize meta-analyses…and come to individual conclusions about the quality of the research rather than accept the conclusions at face value.”

But I am discouraged to see the number of problematic meta-analyses that have appeared in our literature, and I suspect most of these quality problems arise from the second, unjustifiable motivation noted above. We need to do better as a research community, as peer reviewers, and as journal editors to improve the quality of published meta-analyses so that we can favorably impact patient care and advance the clinical practice of hip and knee arthroplasty.

Marc Swiontkowski, MD
JBJS Editor-in-Chief

JBJS Editor’s Choice—Achieving Incremental Progress in Spinal Deformity Correction

swiontkowski marc color

Orthopaedic surgical procedures to correct axial and appendicular skeletal deformities are usually dependent upon fixation devices, either external or internal or both. These devices are often developed through close collaboration with engineers who are generally employed by major manufacturing companies. After the devices successfully clear rigorous bench, in-vitro, and in-vivo testing, the standard initial presentation of clinical results is a case series.

All too often the initial report of results comes from a co-developer of the device, with inherent selection and detection bias that constitute what most readers would consider a conflict of interest. McCarthy and McCullough’s case series on five-year results with Shilla growth guidance in 33 children with early-onset scoliosis in the October 7, 2015 JBJS is an exception to that rule. The authors report every conceivable major and minor adverse event without holding back any negative information. They categorize complications as infection secondary to wound breakdown, spinal alignment issues, and implant issues. The overall complication rate was 73%, a rate that is not surprising given the fact that the device under study is designed to maintain correction of spinal deformity in growing children.

Thankfully, the authors reported no neurologic complications. Also on the positive side, they found that spinal curves averaging 69° preoperatively averaged 38.4° at the most recent follow-up or prior to definitive spinal instrumentation. McCarthy and McCullough also calculated a 73% reduction in the number of surgical procedures among their cohort, relative to what would be necessary to treat the same population with distraction methods every six months.

I applaud the authors for comprehensively reporting the results of correction of spinal deformity in this difficult clinical situation with high accuracy and strict definitions of major and minor events. This is how we will make advances in correcting deformity for skeletally mature and immature patients—with innovation, incremental improvement, and the widespread sharing of adverse events with the orthopaedic community. Armed with the information from this study, we must now see what the number and severity of complications look like when the broader community of orthopaedic surgeons applies these devices.

Marc Swiontkowski, MD

JBJS Editor-in-Chief