Medical Journals and the Future of Comparative Effectiveness Research
Keynote Address - IM 2009
Harold C. Sox
Good morning and welcome to Philadelphia.
When, I learned that I would sound the keynote for Internal Medicine 2009, decided to speak on the future of medical journals. However, I soon found myself chairing a committee charged with setting national priorities for comparative effectiveness research (CER for short). CER is an enormously exciting development for those who love the conceptual elegance of internal medicine. It’s a perfect keynote for this meeting because its focus is generating the information to make better decisions.
As I immersed myself in CER, I began to see the challenges it faces in becoming self-sustaining. Journals like Annals can play a key role building public confidence in CER. Today I’ll talk mostly about CER, some about journals, and some about they relate to each other.
Let’s back into a definition of CER by asking why current clinical research doesn’t give us what need.
First, most industry-sponsored research does not fully inform clinical practice because it compares a new drug to a placebo, as required by the Food and Drug Administration. Doctors and patients need studies that directly compare real-life choices—so-called head-to-head trials. The federal government sponsors some head-to-head trials, but far fewer than we need. The word “comparative” in CER means head-to-head comparisons.
The second point also reflects the trade-off between cost-effective trial planning and satisfying clinical needs. Most qualifying drug trials do not enroll patients with many chronic diseases. The reason is efficiency. When sick participants die of something other than the disease targeted by the trial, they contribute little or nothing to learning if the new drug is effective. Therefore, trial planners typically exclude them. Excluding patients with other chronic diseases means that we never learn if the trial results apply to them. So-called effectiveness research excludes very few patients. Therefore, the results are applicable to a much wider spectrum of patients. The word “effectiveness” in CER means studies of patients like the ones you encounter in your office.
The final point also relates to clinical decision making. A positive drug trial might show us that 60% of patients respond to a new drug, whereas only 50% of patients responded to standard treatment. Based on that information alone, the patient should prefer the new drug. “Wait a minute,” the doctor says, “50% of the patients responded to the old drug. Is my patient one of those who is 90% likely to respond to the old drug and only 30% likely to respond to the new drug?” Placebo-controlled trials can’t ask that question. Head-to-head trials could tell us what patient characteristics increase the odds of responding to a drug, new or old.
In summary, the public isn’t getting its money’s worth from our system of industry-sponsored clinical research. The public pays the costs of drug trials through higher drug prices drugs but gets research with tells us everything we need to know to make good decisions. We get more for our money with the NIH-sponsored trials that we support with our taxes. However, the NIH funds far fewer trials than industry.
Let’s turn to the recent history of CER. About 18 months ago, leaders of health care began to call for research designed specifically to inform medical decisions. The IOM launched a study of the way that the US develops recommendations for practice. The committee’s report called for a national program to promote better standard-setting for medical practice. The national program would decide which topics were most important to study and would pay for systematic reviews of existing research. Guideline developers like the ACP would do the rest. The IOM report was silent on whether the proposed national program should also sponsor new studies on comparative effectiveness. However, other leaders did call for a national program that would do both. Both presidential candidates endorsed the concept of a national program, and several Congressional committees took ownership of the idea. These efforts came together in the American Recovery and Reinvestment Act of 2009, the Administration’s stimulus package. The legislation recommended 1.2B to support CER and 1.5M to support an IOM study to recommend national priorities for spending the 1.2B. This committee is rushing to meet a June 30th deadline to report to Congress.
CER is important because it is specifically designed to inform patient care decisions. The IOM committee definition tells us why:
“The generation and synthesis of evidence that compares the effectiveness of alternative methods to prevent, diagnose, treat, monitor, and improve delivery of care for a clinical condition. The purpose of CER is to assist patients, clinicians, purchasers, and policy makers in making informed health decisions.”
Let’s focus on these key features of CER.
Direct, head-to-head comparisons to inform the decisions of daily practice.
Applies to a broad spectrum of interventions: tests, treatments, and strategies for disease prevention and care delivery.
Patient-centered, so that it tailors the test or treatment to the specific characteristics of the patient.
Targets decisions by patients, clinicians, purchasers, and policymakers.
Effectiveness research: Testing in representative populations. Application to real-life practice.
These attributes of CER sound almost too good to be true. And, in fact, CER is a dream that we can help to transform into reality. In the words of the Congressional staff who wrote the law, the1.2B in the stimulus package is a down payment on a self-sustaining national program of CER. However, the program faces several obstacles.
One is implementation. If the agency that implements CER doesn’t buy into its key attributes, the research won’t reflect them, and CER will not achieve its objectives. Implementation could be a serious roadblock.
A second obstacle is opposition from industry. Head-to-head trials can create losers as well as winners. Political opposition has been the death or near-death of previous federal programs to study medical tests and treatments. Fortunately, a recent national poll indicates overwhelming public support for research to help doctors make better decisions.
A third potential roadblock is loss of public confidence. If the public acts on the results of CER, and the research turns out to be wrong, the American people will lose confidence in CER, and eventually Congress will cut off its funding. What will protect the public—and CER—from this outcome? The public can’t decide that a research result is true. The public relies on medical journals to evaluate research.
So, the future of CER depends in part on medical journals to do a good job of judging research results. At its best, this societal function of journals works well for society. But, good medical journalism is expensive, and times are tough. My second message today is that medical journals need your support to sustain their unique public function.
Annals is among the very best at evaluating research, and we are expensive. We have 4.5 full-time physician editors, each with strong research skills. We have 5 part-time statistical editors. They work hard to help authors improve their work and to report it clearly. A typical research article undergoes 3 revisions, each in response to detailed letters from the senior editor and the statistician. Our annual editorial expense is 2.2M, which figures out to about $13,000 per published article. A few journals have similar resources but not many. These select journals have several things in common with great newspapers. Most importantly, both institutions have investigative reporters—journals call them editors—who go beyond the headlines to probe for the absolute truth. News reporters publish investigative articles and win Pulitzer prizes. Journal editors reject most articles and lavish their skills on articles that do measure up to scientific standards. Both protect society.
Newspapers are in big financial trouble. Medical journals are mostly healthy but are probably vulnerable to the same forces that endanger the newspaper industry.
Medical journals are at risk because they in the midst of a protracted but inevitable transition from all-print to all-web publishing. Print advertising has been lucrative for journals. If the experience of newspapers is any guide, web-based advertising won’t be nearly as profitable. Journals like Annals could go from being a profit center to needing a subsidy to sustain high standards and their key role in society. They will need your active support.
In closing, let’s circle back to CER. CER has unique features that can improve medical decisions. It emphasizes real-life study populations, head-to-head comparisons of realistic treatment choices, and targeting treatments to those who will benefit most. To realize its promise, the agencies that make grants for CER must care deeply about these key attributes and the mission of CER. To sustain public confidence in CER, authors must use the right study methods, report their findings clearly, and acknowledge their limitations. Journals like Annals can help to sustain confidence in CER by holding authors accountable to the highest standards. Good medical journalism is expensive, but society depends on it and must find a way to sustain it.
Before we close this session, please hear my words of advice about the next 3 days. Think of the goals of CER as you listen to the presentations. If the speakers don’t fully address your decision-making needs, ask a question after the session. Be an active learner. And do enjoy our great city.