ACL Trial Quality Has Improved Over Time
The randomized controlled trial (RCT) may be the gold standard of clinical research, but not all RCTs are created equal. In the December 20, 2017 issue of The Journal, Kay et al. set out to determine the historical quality of more than 400 RCTs relating to anterior cruciate ligament (ACL) reconstruction published between 1985 and 2016.
The authors evaluated the studies using the Detsky quality index (on which a score of ≥75% is considered “high” methodological quality) and the CONSORT checklist. Fifty-two percent of the studies analyzed were surgical trials (graft choice, femoral fixation method, and single- vs double-bundle techniques), while 48% were nonsurgical trials (analgesic and rehabilitation regimens).
The mean Detsky score across all studies was 68.9% ± 13.2%. Forty-two percent of the studies were considered high-quality according to Detsky scores. There was no significant difference in mean Detsky scores between surgical and nonsurgical trials.
The authors noted a substantial increase in the number of ACL RCTs over time, although that number has not changed significantly since 2009, which is coincidentally the year that JBJS and many other journals required authors to use the CONSORT flow diagram. Notably, the reporting of methodologically sound randomization processes and the prospective calculation of sample sizes have also improved significantly in recent years.
Both the inclusion of a CONSORT flow diagram and the year published were significant predictors of the quality score. A mere 5% of trials published in 2008 or prior included the CONSORT flow diagram, while >40% of trials published since 2009 had a flow diagram.
While the authors are encouraged that “close to 50% of trials in this study reported blinding of outcome assessors,” they say areas for improvement include better “descriptions of allocation concealment and intent-to-treat analyses.”
JBJS Editor’s Choice—Let’s Improve RCT Registration and Reporting
In the March 2, 2016 edition of JBJS, Rongen et al. air some dirty laundry regarding the orthopaedic community’s registering and reporting on randomized controlled trials (RCTs). According to the authors, only 25% of 362 RCTs published in the top-ten orthopaedic journals between January 2010 and December 2014 were reported as having been registered. Furthermore, of those 25%, only 47% were registered before the end of the trial, and only 38% of those 25% were registered before the enrollment of the first patient, as specified by the International Committee of Medical Journal Editors (ICMJE).
Additionally disheartening is the finding that among the 26 trial reports that the authors deemed eligible for evaluation of consistency between the registered outcome measure(s) and outcomes reported in the published article, 14 (54%) were found to have one or more outcome-measure discrepancies.
Let us re-commit collectively to meeting the timely registration standards required by federal payors such as the NIH and encouraged by the ICMJE. Doing so will ultimately improve the care of patients who have the conditions we study. In general, orthopaedic surgeons are leaders among the surgical specialties when it comes to initiatives that improve patient care. But adequate trial registration and prevention of selective outcome reporting are areas where we are behind the curve, and we need to fix that ASAP. As Rongen et al. emphasize, improvement will require the “full involvement of authors, editors, and reviewers.”
Marc Swiontkowski, MD
JBJS Editor-in-Chief
Editor’s Choice—JBJS Reviews, September 2014
It would be an understatement to suggest that the practice of medicine has changed during the past ten years. Indeed, every physician can think of a number of things that have impacted his or her practice. However, among the positive changes that have affected how we treat patients, evidence-based medicine ranks high on the list.
Evidence-based medicine has been defined as “the integration of best research evidence with clinical expertise and patient values.” Those who support evidence-based medicine note that it will prevent the bias that exists among health-care professionals who frequently base clinical decisions on custom and practice. Hence, the growth of evidence-based medicine along with the desire among clinicians to reduce variations in health-care delivery has had an important and positive impact on health-care practice and policy. Simply stated, the principles of evidence-based medicine serve as a means of decreasing variation in health-care delivery and improving patient outcomes.
The history of evidence-based medicine is interesting and is well covered in the article by David Jevsevar in the September 2014 issue of JBJS Reviews. Concepts and terms are defined, and the findings of research on health-care disparity are discussed. Clearly, the randomized controlled trial (RCT) has become the so-called gold standard in research methodology because of its ability to minimize confounding between patient groups. However, Dr. Jevsevar notes that there are concerns regarding the use of RCTs in the practice of medicine, including their expense as well as the time required for patient recruitment, data analysis, and study completion. As a result of these costs and challenges, most RCTs are now funded by industry, raising concerns about the potential external sources of bias.
This article also touches on other important concepts related to evidence-based medicine in clinical practice policy, such as the propagation and control of conflicts of interest, shared decision-making between physician and patient, and the development of best-practice applications to address the individual needs of and risks to each patient. Finally, it is apparent that the Patient Protection and Affordable Care Act (PPACA) that was signed into law on March 23, 2010 introduces important and vast changes in access to the U.S. health-care system. Designed to address the unsustainable growth in federal spending and the depletion of the Medicare trust fund that is predicted to occur by 2026, this legislation represents an attempt to “bend the cost curve” by showing the increase in annual health-care expenditures. It further makes the point that the absence of an essentially controlled U.S. health-care system creates a potentially large research laboratory promoting study opportunities to investigate the delivery of high-quality, evidence-based care. Thus, the opportunity for orthopaedic surgeons to become advocates for their patients, to take a leading role in shaping the future of evidence-based medicine, and to do so in a way that generates costs that our nation can afford presents a real opportunity to positively shape the future of orthopaedic practice.
Thomas A. Einhorn, MD, Editor, JBJS Reviews
Recent Comments