Effective Clinical Practice
PATTERNS OF PRACTICE
Variation in Clinical Preventive Services
Effective Clinical Practice, May/June 2001
Leif I. Solberg, Thomas E. Kottke, Milo L. Brekke
For author affiliations, current addresses, and contributions, see end of text.
See related Primer.
Context. Preventive services are not delivered at optimal rates in primary care settings, and the literature suggests that a systems approach is key to improvement. Studying variation among clinics could help us to understand the extent of system use in practice.
Practice Pattern Examined. The proportion of patients who are up-to-date for preventive services in 44 primary care practices in the Midwest.
Preventive Services Examined. Papanicolaou (Pap) smear, cholesterol testing, mammography, clinical breast examination, blood pressure measurement, influenza and pneumococcal vaccinations, and advice on tobacco use.
Data Source. 6830 patients surveyed after their clinic visit (response rate, 85%).
Results. The proportion of patients up-to-date for preventive services varied widely among clinics. For example, up-to-date rates for Pap smear testing ranged from 70% to 93% and 45% to 88% for cholesterol screening. There was little correlation between a clinic's performance on one preventive service (relative to the other 43 clinics) and its performance on others. When correlations between pairs of up-to-date rates within clinics were examined, only 4 of 28 service pairs were positive and statistically significant and only 1 had a correlation coefficient that exceeded 0.5 (for mammography and clinical breast examination).
Conclusion. There is wide variation in the rates at which various preventive services are performed, both between and within clinics. This variation, which is probably due to a lack of organized prevention systems that cover multiple services, provides a clear target for improvement efforts.
There is widespread agreement among purchasers, payers, providers, and policymakers that delivery of clinical preventive services is important. Despite this agreement, most preventive services with the best scientific evidence for efficacy and with broad support in the medical community are not currently being provided at optimal rates, even to patients visiting medical practices. In fact, most comparisons of the attitudes and self-reported behaviors of physicians with their patients' reported recollection and documentation in the medical chart have shown a major discrepancy between the desires and perceptions of physicians and what actually happens. (1-4)
Many researchers have studied this to identify the barriers and underlying causes. Most commonly, these studies have used physician surveys or interviews in which everything from forgetfulness and lack of time to reimbursement and patient attitudes have been blamed. (5-9) However, few studies on preventive medical services have used measures of clinical behavior as a guide to understanding and improving this problem of low rates of desirable services.
Since much has been learned about improving many other types of work processes by studying variation, we chose this line of inquiry to explore the persistence of undesirable rates of clinical preventive services. We obtained information on preventive services delivery during a clinic-based improvement trial (10) to study the following questions: 1) Is the rate at which a clinic performs one service relative to other clinics similar to the relative performance of other services? 2) Is being up-to-date for one service correlated with being up-to-date for others? The second question addresses whether a clinic that performs better than other clinics for some preventive services also performs better in other preventive service categories.
We designed the Improving Prevention through Organization, Vision, and Empowerment (IMPROVE) clinical trial to provide patient-reported data about delivery rates for a diverse group of preventive services. Among 178 eligible clinics, 44 were recruited. Inclusion criteria included a current contract with at least one of the trial's HMO sponsors, location within 50 miles of the center of the Twin Cities of Minneapolis and St. Paul, Minnesota, and the desire and ability to participate. The participating clinics, which are described in Table 1, represented 47% of the 71 medical groups that fit these criteria.
We studied 8 preventive services (Table 2), which were selected on the basis of widespread agreement on their efficacy and importance and because they covered a wide range of problems and delivery needs. To provide a common evaluation across clinics with somewhat varied internal guidelines, the age groups and frequencies for these services were chosen as a compromise between recommendations of the U.S. Preventive Services Task Force (11) and the research need for similar evaluation groupings across services.
Measurement of Up-to-Date
To measure delivery of preventive services, we surveyed by mail consecutive adult patients (ages 20 years or older) 1 to 2 weeks after their visit with a clinician at one of the study clinics. Letters were mailed on randomly selected days during a 6-week period in August and September of 1994. Clinic visits included all visits with a clinician for any reason. The survey queried patients about when they had last received each of the eight preventive services before their most recent visit (up to date at clinic visit). By using repeated mail and telephone follow-ups, we obtained an 85.4% response rate. Further details about this survey, as well as the sample selection and analytic methods, have been published previously. (5)
The rates reported are for the patient-reported delivery of services at each clinic in respondents who met the specific age and sex requirements for each of the eight preventive services. Data on tobacco-use inquiry are for current smokers only. The clinic rates reported are for the percentage of patients who reported being up-to-date for each service at the end of their visit (including having it performed or recommended if needed at that visit). Spearman correlation coefficients were calculated to determine if a clinic's performance on one preventive service (relative to the other 43 clinics) was correlated with its performance on others.
We also conducted chart audits on the 70% of total respondents who consented and whose charts could be located. However, we do not report those rates here because they represent a smaller proportion of participants at each clinic and because they are less sensitive for some services, (12-14) particularly for health habits counseling. (15) Nevertheless, the magnitude of variation observed in the analysis of the chart-audit data was similar to that for the self-reported data. (A similar lack of correlation was seen for within-clinic delivery of services.)
Figure 1 shows the wide variation among clinics in the proportion of patients who were up to date. For example, the proportion who were up to date for cholesterol and mammography screening ranged from 45% to 88% and 49% to 82%, respectively. The least variation was observed for Pap-smear screening (range, 70% to 93%); the most variation occurred in advice about tobacco use (range, 20% to 77%).
Variation in rates among clinics could have been caused by some clinics having higher rates for all services and other clinics having uniformly lower rates. Clinics with preventive-services systems in place would be expected to have uniformly high rates of delivery; conversely, low rates of delivery would be expected for clinics without such systems. To evaluate this possibility, we examined the correlation between rankings for pairs of services. For example, Figure 2 shows the weak relationship between the proportion of patients who were up to date for cholesterol screening and for mammography screening (two preventive services with similar distributions in Figure 1).
Table 3 shows the correlation matrix for all pairs of services. In most cases, the correlation was not significant. Only 4 of the 28 pairs of services had a statistically significant correlation for the rates of being up to date, and only 1 among these 28 pairs had a correlation coefficient greater than 0.5 (for breast examination and mammography).
Our results demonstrate large variation in the patient-reported rates of delivery of clinical preventive services. It is probably not surprising that such variation exists among clinics in the relative rates at which they provide these preventive services to their patients--it seems obvious that some clinics are more efficient at providing these services than others. However, there is also great variation in the provision of clinical preventive services within individual clinics. In other words, good performance on one service (relative to the other 43 clinics) does not imply good performance on others. Because the clinicians and nurses in these clinics have very strongly supported providing each of these services, (16) we suggest that within-clinic variation is a marker of haphazard provision of clinical preventive services. In other words, few clinics have a systematic approach to ensure delivery of valued preventive health services.
Investigators of clinical preventive services strongly agree that the key to improvement lies in implementing well-organized, office-wide systems. (17-24) The case has perhaps been summarized best by McPhee and Detmer (25) and the American Cancer Society Advisory Group on Preventive Health Care Reminder Systems (26) after extensive review of the literature. The principal issue remaining is how to stimulate the real-world development and use of these systems.
We need to view undesirable variation as "an opportunity to improve the process of delivering services." (27) This could involve developing systems that do not rely on physicians for successful implementation. Should this be a cause for concern among physicians? Probably not. Berwick, who has addressed physician concerns about quality management, describes quality management as consisting of three essential elements: 1) understanding/meeting customer needs, 2) leadership formation of an appropriate organizational culture, and 3) "efforts to continuously increase knowledge of and control over variation in the processes of work through the widespread use of the scientific methods of collection, analysis, and action upon data." (28)
Our study is subject to several limitations. The participating clinics were volunteers for a clinical intervention and therefore may not be representative of all primary care clinics. In addition, patients tend to overreport both their degree of being up to date and medical actions, while record documentation often underestimates them. (15, 29) However, each of these limitations should not have a large effect on our principal finding of the extensive intraclinic variability in delivery rates. Finally, despite considerable variation in the size of the participating clinics and the insurance status among their patients, the clinics had very few mid-level practitioners and few differences across services provided on site. Moreover, there is no evidence to suggest that staff attitudes and insurance status of the patients have an effect on clinic rates of providing preventive services. (15, 30)
We believe that the variation described in this study is unintended, undesirable, and avoidable. This variation may explain the unacceptably low rates of preventive service delivery. The solution is to develop and implement consistent, well-organized, and integrated systems across all preventive services to ensure the implementation of our intended actions.
|Take Home Points
1. Woo B, Woo B, Cook EF, Weisberg M, Goldman L. Screening procedures in the asymptomatic adult. Comparison of physicians' recommendations, patients' desires, published guidelines, and actual practice. JAMA. 1985;254:1480-4.
2. Lurie N, Manning WG, Peterson C, Goldberg GA, Phelps CA, Lillard L. Preventive care: do we practice what we preach? Am J Public Health. 1987;77:801-4.
3. Weingarten S, Stone E, Hayward R, et al. The adoption of preventive care practice guidelines by primary care physicians: do actions match intentions? J Gen Intern Med. 1995;10:138-44.
4. Osborn EH, Bird JA, McPhee SJ, Rodnick JE, Fordham D. Cancer screening by primary care physicians. Can we explain the differences? J Fam Pract. 1991;32:465-71.
5. Kottke TE, Solberg LI, Brekke ML, Cabrera A, Marquez MA. Delivery rates for preventive services in 44 midwestern clinics. Mayo Clinic Proc. 1997;72:515-23.
6. Kottke TE, Blackburn H, Brekke ML, Solberg LI. The systematic practice of preventive cardiology. Am J Cardiol. 1987;59:690-4.
7. McPhee SJ, Richard RJ, Solkowitz SN. Performance of cancer screening in a university general internal medicine practice: comparison with the 1980 American Cancer Society Guidelines. J Gen Intern Med. 1986;1:275-81.
8. McPhee SJ, Bird JA. Implementation of cancer prevention guidelines in clinical practice. J Gen Intern Med. 1990;5(Suppl): S116-22.
9. Henry RC, Ogle KS, Snellman LA. Preventive medicine: physician practices, beliefs, and perceived barriers for implementation. Fam Med. 1987;19:110-3.
10. Solberg LI, Kottke TE, Brekke ML., et al. Failure of a continuous quality improvement intervention to increase the delivery of preventive services. A randomized trial. Eff Clin Pract. 2000; 3:105-15.
11. U.S. Preventive Services Task Force. Guide to clinical preventive services: report of the U.S. Preventive Services Task Force. 2nd ed. Baltimore, MD: Williams & Wilkins; 1996.
12. Boyer GS, Templin DW, Goring WP, et al. Discrepancies between patient recall and the medical record. Potential impact on diagnosis and clinical assessment of chronic disease. Arch Intern Med. 1995;155:1868-72.
13. Hiatt RA, Perez-Stable EJ, Quesenberry C Jr, Sabogal F, Otero-Sabogal R, McPhee SJ. Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California. Prev Med. 1995;24:278-85.
14. Montano DE, Phillips WR. Cancer screening by primary care physicians: a comparison of rates obtained from physician self-report, patient survey, and chart audit. Am J Public Health. 1995;85:795-800.
15. Stange KC, Zyzanski SJ, SmithTF, et al. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36:851-67.
16. Solberg LI, Brekke ML, Kottke TE. How important are clinician and nurse attitudes to the delivery of clinical preventive services? J Fam Pract. 1997;44:451-61.
17. Yano EM, Fink A, Hirsch SH, Robbins AS, Rubenstein LV. Helping practices reach primary care goals. Lessons from the literature. Arch Intern Med. 1995;155:1146-56.
18. Cohen SJ, Halvorson HW, Gosselink CA. Changing physician behavior to improve disease prevention. Prev Med. 1994;23: 284-91.
19. Thompson RS, Taplin SH, McAfee TA, Mandelson MT, Smith AE. Primary and secondary prevention services in clinical practice. Twenty years' experience in development, implementation, and evaluation. JAMA. 1995;273:1130-5.
20. Carney PA, Dietrich AJ, Keller A, Landgraf J, O'Connor GT. Tools, teamwork, and tenacity: an office system for cancer prevention. J Fam Pract. 1992;35:388-94.
21. Dietrich AJ, O'Connor GT, Keller A, Carney PA, Levy D, Whaley FS. Cancer: improving early detection and prevention. A community practice randomised trial. BMJ. 1992;304: 687-91.
22. McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians. Results of a randomized, controlled trial. JAMA. 1991;266:538-44.
23. Kottke TE, Solberg LI, Brekke ML, Conn SA, Maxwell PL, Brekke MJ. Doctors helping smokers: development of a clinic-based smoking intervention system. In: Shopland DR, Burns DM, Cohen SI, Kottke TE, Gritz ER, eds. Tobacco and the Clinician: Interventions for Medical and Dental Practice. Bethesda, MD: U.S. Department of Health; 1994:69-91.
24. Davis JE, McBride PE, Bobula JA. Improving prevention in primary care: physicians, patients, and process. J Fam Pract. 1992;35:385-7.
25. McPhee SJ, Detmer WM. Office-based interventions to improve delivery of cancer prevention services by primary care physicians. Cancer. 1993;72:1100-12.
26. Leininger LS, Finn L, Dickey L, et al. An office system for organizing preventive services: a report by the American Cancer Society Advisory Group on Preventive Health Care Reminder Systems. Arch Fam Med. 1996;5:108-15.
27. Blumenthal D. The variation phenomenon in 1994 [Editorial]. New Engl J Med. 1994;331:1017-8.
28. Berwick DM. Controlling variation in health care: a consultation from Walter Shewhart. Med Care. 1991;29:1212-25.
29. Solberg LI. Practical implications of recall bias [Editorial]. Tob Control. 1996;5:95-6.
30. Solberg LI, Brekke ML, Kottke TE. Are physicians less likely to recommend preventive services to low-SES patients? Prev Med. 1997;26:350-7.
Supported by grant RO1 HS08091 from the Agency for Health Care Policy and Research.
Leif I. Solberg, MD, Associate Medical Director for Care Improvement Research, HealthPartners Research Foundation, 8100 34th Avenue South, P.O. Box 1524, Minneapolis, MN 55440-1524; telephone: 952-967-5017; fax: 952-967-5022; e-mail: email@example.com.
Earn MOC Points for Medical Knowledge
ACP offers its members many ways to earn ABIM MOC points for Medical Knowledge and to make the process easier. See our MOC Timeline Page for details.
Ceramic Bistro-Style ACP Mug
Enjoy your morning brew and show your ACP spirit with our 15-ounce dishwasher- and microwave-safe mug. Enjoy free shipping within the continental U.S.