Effective Clinical Practice
How Can We Help People Make Sense of Medical Data?
Effective Clinical Practice, July/August 1999.
Steven Woloshin, Lisa M. Schwartz
For author affiliations, current addresses, and contributions, see end of text.
Context. Information is a basic prerequisite to informed medical decision making.
General Question. How can we help people interpret the quantitative data they need to make informed decisions?
Specific Research Challenge. To develop and evaluate interventions that will help people make sense of the quantitative data relevant to their health care decisions.
Standard Approach. Traditional patient education interventions focus on providing disease-specific information (e.g., educational brochures about a single disease).
Potential Difficulties. Interventions that focus on content—the provision of facts—may not be sufficient help for people facing medical decisions. Training that prepares people to make sense of the facts that they are given may be necessary.
Alternate Approach. We propose developing a generic (i.e., not disease-specific) tutorial to prepare people to better understand and more critically evaluate data on disease risk and the benefits and harms of treatment. This tutorial aims to improve critical reading skills by teaching people about risk (e.g., probability and rates) and showing them what to look for in statements about risk (e.g., time frame), how to put disease risk and treatment benefit in context (e.g., evaluating competing risks), how to interpret changes in risk, and whether to believe the statements about changes in risk.
Throughout their lives, people face a bewildering array of medical decisions. Such decisions include: Should I have mammography? Should I have coronary artery bypass graft surgery or angioplasty? Do I need a check-up? To make informed decisions, people need answers to many questions, including: What are the possible outcomes of this test? What are my chances of staying healthy if I forgo this intervention? What are the likely benefits and potential harms of treatment?
Providing the data needed to inform medical decisions is an extremely challenging task. Patients vary widely in their informational needs, decision-making styles, and ability to use data. Information relevant to medical decisions may be particularly difficult to transmit because it is inherently quantitative, probabilistic, and often involves small numbers--all of these characteristics may be important barriers to patient comprehension. Moreover, although "judgment under uncertainty" is not easy in the best of circumstances, it may be extraordinarily difficult when the stakes are high (e.g., when a patient first faces a frightening disease, such as cancer, or a potentially dangerous intervention, such as major surgery).
Despite the contemporary focus on informed decision making, little is known about how best to inform people who need to make medical decisions. Our paper looks at the design and evaluation of an intervention intended to help people make sense of quantitative data. We begin by critiquing the standard approach to patient education, an approach exemplified by disease-specific brochures that are meant to provide patients with facts about a single health issue (e.g., risk for breast cancer). We then propose an alternate general education approach intended to help people become better and more critical consumers of medical data. Finally, we describe our plans to operationalize this approach in the form of a tutorial: "Understanding Numbers in Health."
In general, efforts to enhance patient decision making have largely focused on providing disease-specific facts. The reason for this common-sense approach is straightforward: To make an informed decision, one must have information about the problem. If people lack key facts, their decisions cannot be informed. The solution, then, is to provide the facts. The abundance of disease-specific brochures, videotapes, and other materials reflects the popularity of this approach. These materials vary in quality, but almost all of them provide information about a single disease. To determine how well such materials convey quantitative data to patients, we performed a structured literature review to identify randomized trials of interventions designed to communicate quantitative data about disease risk or the benefits or harms of treatment.
From a MEDLINE search, we identified 129 articles reporting on medical decision-making interventions (see the figure that accompanies the abstract of this article at ecponline.org). With one exception, all of the articles described disease-specific interventions; the exception reported on an educational intervention to promote effective communication among physicians, children, and parents during pediatric office visits. (1) Sixty-nine of the articles reported on randomized clinical trials. Of these, 3 studied interventions designed to expand patient involvement in the care of a specific disease (2-4) and 66 sought to convey facts about a specific disease. Of the 66, 47 presented quantitative data.
Only seven of the randomized trials assessed patient comprehension of the quantitative information provided; the rest assessed general knowledge (e.g., "drug X can affect your liver") or focused primarily on changes in behavior. (5-11) Two of the seven trials compared the comprehension of specific data presented in different formats (10, 11); the other five evaluated patient education tools (a summary of these five studies is available at ecoponline.org). The tools studied range from simple facts to decision aids designed to put information into a decision framework. Comparison across studies is difficult because the interventions and metrics of efficacy are diverse. In general, the interventions had variable effects and left substantial room for improvement.
Problems with the Standard Approach
There are many reasons to question whether it is sufficient simply to give patients disease-specific data (Table 1). First, low levels of numeracy (quantitative literacy) are common and are an important barrier to the communication of even basic medical data--precisely the kinds of facts that disease-specific interventions hope to transmit. (11, 12)
Second, even if people understand the actual numbers, it is unclear whether they understand what the numbers are meant to communicate. The need for guidance in interpreting quantitative data is strongly supported by research in cognitive psychology. In one study, participants were told that there was a 30% chance of rain. (13) Although most understood what "30%" meant, participants were confused about whether the statement meant that there was a 30% chance of rain in a particular spot, that it would rain in 30% of the area, or that it would rain for 30% of the day. Furthermore, the cognitive psychology literature is replete with demonstrations of how people (both physicians and patients) are susceptible to simple changes in the format of identical numeric information (e.g., they respond differently to absolute and relative risk reductions), a concept referred to as framing. (14-25) Problems with numeracy, lack of familiarity with how statistical language is used, and the existence of framing effects suggest that receiving quantitative information without being guided in its interpretation may be meaningless or confusing for a substantial proportion of the population. These issues might explain why a high-intensity educational intervention failed to improve the accuracy of women's perceptions of their risk for breast cancer. (5)
Unfortunately, only a small proportion of the public knows enough about the basic principles of probability and epidemiology to critically evaluate health messages. Rather than informing people and enhancing decision making on health issues, these messages may cause people to feel anxious and even manipulated (i.e., people may be led to request unneeded, unproved, or ineffective treatments). We hypothesize that giving people quantitative information and attending to the basic principles of probability and epidemiology will result in more meaningful communication.
Alternate Approach: A Generic Guide to Understanding Numbers in Health
We propose to develop a generic tutorial to prepare people to better understand and more critically evaluate data on disease risk and the benefits and harms of treatment. This basic education would help establish a framework for understanding and interpreting numbers in health. Rather than focusing on one disease at a time, our tutorial would provide context by comparing across diseases and across treatments. It would be relevant to anyone who is facing or will face a health-related decision--everyone from children to patients with newly diagnosed cancer. It could be used in clinical settings and in school curricula (modeled on "Chance," an Internet-based quantitative literacy course based on current events in the news, available at http://www.dartmouth.edu/%7Echance/Chance.html).
Educators have long understood that presenting facts without first preparing an audience to receive them (i.e., without integrating them into an organizing structure) is counterproductive. The audience will absorb little data, will quickly forget it, will not understand how the information relates to their own experience, and may misinterpret the meaning of the information. The fundamental importance of an organizing structure in effectively teaching clinical medicine has been well articulated. Samuel Their and Harry Kimball (presidents of the Institute of Medicine and the American Board of Internal Medicine, respectively) compared the acquisition of cognitive knowledge (e.g., signs and symptoms) in the absence of clinical reasoning to "driving across the country without a road map." (26)
Knowing how to put facts together is crucial for clinicians. It is no less important for members of the general public who are struggling to make sense of medical facts. We are not arguing that patients need to become physicians but rather for the development of a basic general education intervention that can familiarize the public with how physicians think, know, and communicate. If successful, this intervention would help people make sense of the vast and growing amount of health data to which they are exposed, not only in the clinic but also through the Internet, in the mass media, and in direct-to-patient pharmaceutical advertising.
First Draft of Content
Table 2 presents our tutorial's table of contents. The tutorial focuses on the following broad issues: what risk is, what to look for in statements about risk or benefit, putting risk in context, changing risk, and evidence.
What Is Risk?
Attempts to discuss medical risk are easily undermined by the confusing and imprecise use of language. Our tutorial begins by addressing common sources of confusion (e.g., multiple meanings of the word "risk"), how to use words (and the limits of words) in describing risk, and ways to quantify risk (such as probability, percentages, proportions, and rates). It also introduces the reader to a scale designed to facilitate quantification and communication of risk, particularly risks of less than 1% (Figure 1). This scale, which has been shown to perform well across a wide sociodemographic range, will be used to help teach people about small risks. (27)
What To Look for in a Statement about Risk
This section of the tutorial teaches the reader to look for several essential elements in statements about risk. We present actual statements taken from various sources, including public health campaigns, direct-to-patient advertisements, and reports in the mass media. For example, we consider the familiar "1 in 9" breast cancer statistic promulgated by the American Cancer Society. Readers will be sensitized to ask such questions as, What is the risk under discussion? (Is it the risk for being diagnosed or for dying?), What is the time frame under consideration? (Is it the next 5 years or a lifetime, and what does "lifetime" mean?), and Who is at risk? (All women? Women of a certain age? Women with specific characteristics, such as a family history of breast cancer?).
Putting Risk in Context
A salient but unusual event—such as a celebrity dying of a rare type of cancer—may give undue weight to certain health risks. The tutorial emphasizes the need to put risks into perspective. Readers are encouraged to ask such questions as, How does the chance of getting this disease compare with the chance of getting other diseases or encountering other, familiar events? How dangerous is the disease? (What is the difference between developing the condition and dying of it?), and How do my chances of dying of this disease compare with other competing risks? To illustrate competing risks, we will use disease risk charts that we are now developing. The chart in Table 3 shows 10-year disease-specific mortality data for five diseases for women in 5-year age categories. Moving across the table lets the user compare the magnitude of each disease risk. Because many people may be most concerned about their overall chance of dying, the final column displays all-cause mortality to provide context about how much each disease contributes to the overall chance of dying.
This section of the tutorial focuses on how to interpret statements that measure changes in risk given a particular exposure or intervention. Briefly, we review the typical ways in which such information is presented (e.g., relative and absolute risk reduction and number needed to treat). We then introduce the concept of framing. To highlight the fact that not all risk factors and interventions are equally important, we introduce benefit charts. These charts help patients compare the relative effects of a change in behavior or a specific intervention on all-cause mortality. Table 4 presents a hypothetical benefit chart that displays age- and sex-specific 10-year all-cause mortality rates with or without a given intervention. The numbers in the chart, although hypothetical, are accurate in terms of order of magnitude. This example shows that for a 55-year-old female smoker, giving up cigarettes has a substantially greater effect on all-cause mortality than annual mammography or colon cancer screening does.
The tutorial's final section points out that there is uncertainty in what we know and introduces the idea of grading evidence by highlighting basic types of study design (e.g., differentiating between an observational study and a randomized trial). We introduce the concept of a CI as a way to quantify uncertainty, and we encourage readers to ask themselves, Can I believe the data presented? Could they be wrong? Readers are encouraged to be skeptical and to be alert to the possible overinterpretation of single studies, extrapolation from findings in animals to humans, and extrapolation from intermediate end points (e.g., tumor shrinkage) to ultimate end points (e.g., death).
Figure 2 describes how we plan to develop the tutorial. The first step is an iterative review process, involving content experts in education, mathematics, patient counseling, and epidemiology, to ensure that the tutorial material is accurate and clearly articulated. Once this review is complete, we will recruit persons with diverse educational backgrounds to take the tutorial. After they have completed the tutorial, we will conduct "debriefing" interviews to assess what people learned, or did not learn, to determine which presentations were unclear or confusing and to obtain suggestions for improvement. After this second iterative review process, we will have a final version of the tutorial on which to conduct formal testing.
Formal testing will be done with patients who represent our target audience (patients confronting important medical decisions). We will ask patients to rate the tutorial in terms of usability (e.g., the ease with which it can be understood) and satisfaction (e.g., whether they found it helpful, whether they think they will use the information in making other decisions, and whether they would recommend the tutorial to others). To see whether learning has occurred, we will administer a test of critical reading skills before and after exposure to the tutorial. This test will involve critiquing and interpreting statements about risk (e.g., it will use questions such as, What does the following mean: "A woman's chance of breast cancer is 1 in 9"?) and will ask patients to interpret a newspaper report on a scientific study and a direct-to-patient advertisement.
Our approach has several potential problems. First, some patients say that they do not want information, and many of these patients would therefore have no interest in our tutorial. However, it is possible that for some people, an expressed lack of interest in information may really be a fear that they would be unable to understand what they are told. Our tutorial may make quantitative information accessible to people who might otherwise give up. Next, the tutorial may be too ambitious: Many of the concepts introduced are new, and problems with numeracy may be hard to overcome. Third, the effectiveness of our approach may be limited in certain circumstances. For example, patients who face new and serious decisions (e.g., those who must make decisions about treatment at the time of a new cancer diagnosis) may feel too emotionally overwhelmed to make use of the materials. On the other hand, patients in less-intense circumstances (e.g., an older man considering prostate-specific antigen screening) may find the materials especially useful. Finally, whether the tutorial will actually help patients make better decisions is open to question. Understanding how patients use the materials and evaluating how the materials affect decision making will require careful study.
The feasibility and positive effect of a general education approach has been demonstrated in patient coaching interventions. (2-4) In these studies, patients who were told about "the logic of the medical process" in addition to being given specific facts were more likely to participate in their own care and had improved functional outcomes compared with patients who received disease-specific information in isolation. Although our approach builds on patient coaching, it is fundamentally different. Whereas coaching prepares patients to interact with clinicians, our intervention prepares people to interpret quantitative health information, regardless of its source.
We expect that, like patient coaching, general education will promote patient participation in care. Many patients want information. Our tutorial may help patients understand what sorts of questions they can reasonably ask, believe that they will be able to make sense of the data, and expect disclosure of uncertainty about what they are told. Thus, the tutorial will encourage patients to learn about and compare their options. Finally, it may help patients put disease-specific information into the broad perspective of their overall health.
|Take Home Points
1. Lewis CC, Pantell RH, Sharp L. Increasing patient knowledge, satisfaction, and involvement: randomized trial of a communication intervention. Pediatrics. 1991;88:351-8.
2. Greenfield S, Kaplan S, Ware JE Jr. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med. 1985;102:520-8.
3. Kaplan SH, Greenfield S, Ware JE Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care. 1989;27:S110-27.
4. Greenfield S, Kaplan SH, Ware JE Jr, Yano EM, Frank HJ. Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med. 1988;3:448-57.
5. Lerman C, Lustbader E, Rimer B, Daly M, Miller S, Sands C. Effects of individualized breast cancer risk counseling: a randomized trial. J Natl Cancer Inst. 1995;87:286-92.
6. O'Connor AM, Tugwell P, Wells GA, et al. Randomized trial of a portable, self-administered decision aid for postmenopausal women considering long-term preventive hormone therapy. Med Decis Making. 1998;18:295-303.
7. Inglis S, Farnill D. The effects of providing preoperative statistical anaesthetic-risk information. Anaesth Intens Care. 1993;
8. Flood AB, Wennberg JE, Nease RF Jr, Fowler FJ Jr, Ding J, Hynes LM. The importance of patient preference in the decision to screen for prostate cancer. Prostate Patient Outcomes Research Team. J Gen Intern Med. 1996;11:342-9.
9. Krueter MW, Strecher VJ. Changing inaccurate perceptions of health risk: results from a randomized trial. Health Psychol. 1995;14:56-63.
10. Halpern DF, Blackman S, Salzman B. Using statistical risk information to assess oral contraceptive safety. Applied Cognitive Psychology. 1989;3:251-60.
11. Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997;127:966-72.
12. Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult Literacy in America. Washington, DC: National Center for Education Statistics; 1993.
13. Murphy A, Lichtenstein S, Fischhoff B, Winkler R. Misinterpretations of precipitation probability forecasts. Bulletin of the American Meteorological Society. 1980;61:695-701.
14. Naylor CD, Chen E, Strauss B. Measured enthusiasm: does the method of reporting trial results alter perceptions
of therapeutic effectiveness? Ann Intern Med. 1992;
15. Forrow L, Taylor WC, Arnold RM. Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med. 1992;92:121-4.
16. Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk. J Gen Intern Med. 1993;8:543-8.
17. Hux JE, Naylor CD. Communicating the benefits of chronic preventive therapy: does the format of efficacy data determine patient's acceptance of treatment? Med Decis Making. 1995;
18. O'Connor AM, Boyd NF, Tritchler DL, Kriukov Y, Sutherland H, Till JE. Eliciting preferences for alternative cancer drug treatments. The influence of framing, medium, and rater variables. Med Decis Making. 1985;5:453-63.
19. O'Connor AM. Effects of framing and level of probability on patients' preferences for cancer chemotherapy. J Clin Epidemiol. 1989;42:119-26.
20. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211:453-8.
21. Meyerowitz BE, Chaiken S. The effect of message framing on breast self-examination attitudes, intentions, and behavior.
J Pers Soc Psychol 1987;52:500-10.
22. McNeil BJ, Pauker SG, Sox HC Jr, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med. 1982;306:1259-62.
23. Llewellyn-Thomas HA, McGreal MJ, Thiel EC. Cancer patients' decision making and trial-entry preferences: the effects of "framing" information about short-term toxicity and long-term survival. Med Decis Making. 1995;15:4-12.
24. Kahneman D, Tversky A. The psychology of preferences. Sci Am. 1982;246:160-73.
25. Eraker SA, Sox HC Jr. Assessment of patients' preferences for therapeutic outcomes. Med Decis Making. 1981;1:29-39.
26. Their S, Kimball H. Foreword. In: Kassirer JP, Kopelman RI, eds. Learning Clinical Reasoning. Baltimore: Williams & Wilkins; 1991:vii.
27. Woloshin S, Schwartz LM, Byram S, Fischhoff B, Welch H. Scales for assessing perceptions of event probability: a validation study. Med Decis Making. 1998;14:490.
28. Kahneman D, Slovic P, Tversky A. Judgement under Uncertainty: Heuristics and Biases. Cambridge: Cambridge Univ Pr; 1982.
The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.
Drs. Woloshin and Schwartz are supported by Veterans Affairs Career Development Awards in Health Services Research and Development and by a U.S. Army Medical Research and Material Command Breast Cancer Research Program New Investigator Award (DAMD17-96-MM-6712).
Steven Woloshin, MD, MS, and Lisa M. Schwartz, MD, MS, Veterans Affairs Outcomes Group (111B), Veterans Affairs Medical & Regional Office Center, 215 North Main Street, White River Junction, VT 05009; e-mail: firstname.lastname@example.org; email@example.com.
We searched the MEDLINE database (1966present) using the following MeSH terms: (decision making, choice behavior, informed consent, decision support techniques, communication, or risk assessment) and (patient advocacy, patient education, counseling, health education, treatment refusal, patient participation, patient dropouts, patient compliance, consumer satisfaction, physician-patient relations, attitude to health, health promotion methods, knowledge-attitudes-practice, or patient acceptance of health care.)
Because the search did not pick up some studies known to the authors, we also performed a keyword search using the terms risk, risk communication, or counseling. We then limited the search results to randomized, controlled trials done in human adults and published in English. This strategy yielded 329 articles. We found and reviewed 129 articles that were relevant to the construct of medical decision making.
Have questions about the new ABIM MOC Program?
One Click to Confidence - Free to members
ACP Smart Medicine is a new, online clinical decision support tool specifically for internal medicine. Get rapid point-of-care access to evidence-based clinical recommendations and guidelines. Plus, users can easily earn CME credit. Learn more