Effective Clinical Practice
Judging the Effectiveness of Clinical Pathways for Pneumonia: The Role of Risk Adjustment
Effective Clinical Practice, September/October 2000.
Carlos A. Estrada, John N. Unterborn, Jammie Price, Debra Thompson, Laurin Gibson
For author affiliations, current addresses, and contributions, see end of text.
Context. Although observational studies suggest that clinical pathways may decrease costs and improve quality in hospitalized patients with community-acquired pneumonia, inferences from these studies are limited by potential selection bias and inadequate case-mix adjustment.
Objective. To compare the assessment of a clinical pathway for community-acquired pneumonia with and without adjusting for patient characteristics and disease
Design. Retrospective cohort study.
Patients and Setting. Consecutive series of adult patients admitted with clinical diagnosis of community-acquired pneumonia, treated with either a clinical pathway (which included guidelines for antibiotics, tests, and ancillary care) or usual care.
Main Outcome Measures. Total hospital charges, length of stay, clinical deterioration (requiring mechanical ventilation or intensive care unit transfer), and in-hospital mortality. We used multiple linear and logistic regression to adjust for patient case mix.
Results. Compared with patients receiving usual care (n=275), patients in the pathway group (n=97) were more likely to be treated by family physicians than specialists and had lower pneumonia severity scores. In the unadjusted analysis, total hospital charges were lower among pathway patients ($2456; 95% CI, $175 to $4737; P =0.04); in the adjusted analysis, the difference in total charges was smaller (average reduction $1807; CI, $4164 lower to $549 higher; P =0.13). In the unadjusted analysis, length of stay was lower among pathway patients (1.8 days lower; CI, 3.9 lower to 0.4 higher; P =0.12); in the adjusted analysis, the difference in length of stay was smaller (0.9 days lower; CI , 3.2 lower to 1.3 higher; P =0.4). Although unadjusted analysis showed significantly lower in-hospital mortality in pathway patients, this difference was not confirmed in the adjusted analysis.
Conclusions. Clinical pathways may reduce costs and improve quality of care in community-acquired pneumonia. In nonrandomized studies, however, selection bias and case-mix differences may explain some of the apparent effectiveness.
Community-acquired pneumonia is a leading cause of morbidity and mortality in the United States. Three million patients each year develop this condition, which has an overall mortality rate of approximately 10%. (1-4) The total cost of caring for patients with community-acquired pneumonia in the United States is approximately $4 billion per year. (1) One way to reduce costs and improve quality of care is practice guidelines and clinical pathways, (5-10) many of which have been developed for treating patients with pneumonia. (11-13) These efforts are intended to reduce unwanted variation in treatment and outcomes, (2, 14-22) identify best practices, and promote efficient use of resources.
Unfortunately, the effectiveness of practice guidelines and clinical pathways in community-acquired pneumonia is uncertain. Most studies evaluating their effectiveness have been observational in design and are limited by potential selection bias. (6-10) Inconsistent findings across these studies may reflect inadequate adjustment for patient age, comorbid conditions, or severity of illness. To test the importance of risk adjustment in this setting, we compared the assessment of a clinical pathway for community-acquired pneumonia with and without adjusting for patient characteristics and disease severity.
We conducted a retrospective cohort study of 372 patients hospitalized with community-acquired pneumonia in a 715-bed primary and tertiary care hospital between January 1996 and March 1997. The retrospective cohort design is useful in the initial evaluation of clinical pathways. (23) Less than 10% of patients had managed care health insurance.
A pulmonary clinical nurse specialist and nursing staff screened patients in the emergency department or within 24 hours after admission and reminded the physician about the existence of the pathway. Physicians then elected to use the pathway or not. Patients on the clinical pathway served as the intervention group (n=97). Patients not on the clinical pathway served as the control group (n=275). This study was approved by our institutional review board.
A clinical pathway is a multidisciplinary tool that outlines the process of care and expected outcomes. (5, 24) It was designed with the input of local infectious disease specialists, pulmonologists, general internists, respiratory therapists, and nursing personnel. The pathway provides guidelines for the use of antibiotics, tests and procedures, oxygen and bronchodilators, nutritional evaluation, early mobilization, and patient assessment and education (Table 1). The guidelines of empirical initial antibiotic therapy were based on local epidemiology, and consideration was given to current guidelines (12); however, antibiotics were not restricted. Nursing personnel documented specific goals accomplished (e.g., antibiotics begun within 4 hours, oxygenation, mobility, and discharge planning). Failure to accomplish a goal would trigger an action to resolve it. For example, if saturation of oxygen was less than 90%, then the oxygen flow would be increased; conversely, if saturation was over 93%, then the action would be to decrease or discontinue oxygen supplementation. A copy of the clinical pathway is available from the authors.
Inclusion and Exclusion Criteria
Hospital discharges with selected diagnosis-related groups or primary or secondary diagnosis codes from the International Classification of Diseases diagnosis codes, Ninth Revision, Clinical Modification (ICD-9-CM) were selected. Patients were included if they were diagnosed as having pneumonia, received treatment, and did not meet any of the exclusion criteria (Figure 1).
The primary economic outcomes were total hospital charges and length of hospital stay. The clinical outcomes were clinical deterioration requiring mechanical ventilation or intensive care unit transfer, and mortality during the index hospitalization. A secondary clinical outcome was 30-day readmission.
We collected the following data: age, sex, race, year, season of year, insurance status, source of admission, severity, comorbid conditions, (25) processes of care (use of Gram stain and time to first dose of antibiotics), and physician's specialty and teaching appointment. We used the Fine severity score--a validated measure of severity of pneumonia that uses age and physical examination, laboratory, and radiographic findings to rate disease severity. (1) The index sums the weights assigned to each characteristic and then groups them into five categories of severity (Table 2). The index predicts 30-day mortality, readmission, intensive care unit transfer, and length of stay. (1) We computed the Charlson Index, (26-29) which sums weights for each chronic comorbid condition that the patient has had according to current and previous hospital admission records. (30,31) The index has been shown to predict the risk for death within 1 year of medical hospitalization. (26,27,32)
We obtained data from a combination of administrative databases and chart reviews. Severity data were only available through chart reviews. We validated administrative data with chart reviews. We restricted the chart abstractor's access to areas of the chart that revealed whether the patient was on the pathway, thereby blinding the abstractor to study group assignment. Audits were done throughout the data collection process.
All analyses were planned a priori. We used the Student t and chi-square tests for the unadjusted, bivariate analyses. We then used linear and logistic regression to test the significance of the pathway for the adjusted analyses. (33,34) We adjusted for sex, race, pneumonia- specific severity of illness, season, physician specialty, whether the physician was an academic, admission source (emergency department or other), and insurance type. The reference categories were spring (for season), internist (for specialty), commercial (for insurance type), and class I (for severity of illness).
The adjusted analyses are multivariate techniques that provide statistical control for covariates, also called confounding variables. The effect of the explanatory variables are isolated, which removes the effect of the confounding variables. For example, when looking at the pathway's effect on length of stay, we want to eliminate the possible impact of severity, sex, race, insurance type, and so on. All hypotheses were tested at a = 0.05.
To illustrate the effect of risk adjustment, we calculated the adjusted predicted values for total charges and length of stay as follows: We used the coefficients from the linear regression model to solve the regression equation for the clinical pathway and usual care groups, and used the mean or overall proportion for the entire population for each independent variable. We recognized that total charges and length of stay had skewed distributions and repeated the analyses by using log transformation, with similar results (data not presented). We analyzed the data by using the SPSS 8.0 software (Chicago, Illinois).
During the study period, 1878 patients were discharged with a diagnosis of pneumonia. Of these, 19.8% were included (n=372) and 80.2% (n=1506) were excluded (Figure 1). Of the 372 study patients, 97 were on the pathway and 275 were not. The baseline characteristics, comorbid conditions, and severity for the two groups are shown in Table 2. Pathway patients were treated more frequently by family physicians (P =0.002), had lower pneumonia-specific severity of illness (P = 0.07), and had fewer comorbid conditions (P =0.005).
In the unadjusted analysis, the total charges were $2456 lower among patients in the pathway group (CI, $175 lower to $4737 lower; P =0.04). In the adjusted analysis, however, the average difference in total charges was smaller and not statistically significant ($1807 reduction; CI, $4164 lower charges to $549 higher charges; P =0.13) (Figure 2).
Similarly, there was a trend toward shorter length of stay in pathway patients (1.8 days lower; CI, 3.9 lower to 0.4 higher; P =0.12). Differences in length of stay were smaller in the adjusted analysis (0.9 days lower; CI, 3.2 lower to 1.3 higher; P =0.4) (Figure 2).
In the unadjusted analysis, mechanical ventilation or transfer to the intensive care unit was similar between patients in the pathway group (4.1%) and those in the usual care group (2.9%) (P =0.5).
In the unadjusted analysis, hospital mortality was lower among patients in the pathway group (0% vs. 5.1% in usual care; CI, 2.5% to 7.7%; P =0.03). However, the association between mortality and use of the pathway disappeared after adjustment for baseline characteristics and pneumonia-specific severity of illness (P =0.8).
There were small differences in the use of Gram stains and time to first use of antibiotics between the pathway and usual care groups (Table 3). Pathway patients had generally lower charges for each type of resource use. However, only differences in laboratory and pharmacy costs were statistically significant.
We examined the importance of risk adjustment in evaluating the effectiveness of a clinical pathway in patients with community-acquired pneumonia. Without adjusting for baseline differences in risk between pathway and usual care patients, patients on the pathway tended to have lower mortality, lower health care costs, and shorter lengths of stay in the hospital. With risk adjustment, however, differences between the two groups narrowed and, in each case, became statistically nonsignificant. Higher costs and higher mortality in usual care patients were at least partly attributable to higher severity of illness in this group.
Although our findings underscore the importance of risk adjustment in evaluating clinical pathways, they do not imply that pathways and practice guidelines for community-acquired pneumonia are not effective. Although the decrease in charges for pathway patients was not statistically significant, these findings can be interpreted in two ways: 1) No real difference between groups exists (true negative), or 2) a difference exists, but the study failed to detect it (false negative). Because most of the confidence interval falls within the savings range, the latter explanation is more likely. At the very least, the potential $1807 decrease in charges in patients treated under the clinical pathway deserves further study. We also hypothesized that use of a pneumonia pathway would maintain or improve medical outcomes. The low frequency of mortality and intensive care unit admission did not permit firm comparisons between the groups. However, mortality or readmisson rates did not seem to be increased among patients treated with the pathway.
Other efforts to evaluate practice guidelines in patients with community-acquired pneumonia at low risk for complications have been published. (35,36) A practice guideline of switching patients from parenteral to oral antibiotics and early hospital discharge did not improve medical or economic outcomes. (35,36) Health-related quality of life, patient satisfaction, (35) 3- to 5-month outcomes (readmissions, health status, and patient satisfaction), (36) and length of stay (35,36) were similar for intervention and control groups. In a recent clinical trial, Marrie and colleagues (37) found that implementation of a clinical pathway reduced the number of bed-days per patient managed by 1.7 and decreased admission of low-risk patients by 18%, without causing harm to patients. The potential savings per patient treated was estimated at $1700. Unlike these studies, our study was not restricted to low-risk patients.
Our study has several limitations. First, like any observational study, it is subject to both bias and confounding. (38) We hoped to minimize these problems by blinding the chart abstractors to the intervention and adjusting carefully for differences in patient characteristics and disease severity. Second, we supplemented our analysis dataset with administrative data. Although the limitations of such data are well-known, the combination of administrative data and chart review has been shown to be reliable for predicting death in patients with acute myocardial infarction, pneumonia, heart failure, and cerebrovascular accident. (39,40) Third, we did not examine compliance with the pathway. Low compliance may explain the reported lack of effectiveness.
Because development and implementation of clinical pathways demand enormous resources, pathways should be subjected to rigorous testing to determine their effectiveness. Our study underscores the importance of accounting for patient case mix in the evaluation process.
|Take Home Points
1. Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243-50.
2. Lave JR, Fine MJ, Sankey SS, Hanusa BH, Weissfeld LA, Kapoor WN. Hospitalized pneumonia. Outcomes, treatment patterns, and costs in urban and rural areas. J Gen Intern Med. 1996;11:415-21.
3. Fine MJ, Smith MA, Carson CA, et al. Prognosis and outcomes of patients with community-acquired pneumonia.
A meta-analysis. JAMA. 1996;275:134-41.
4. Fine MJ, Orloff JJ, Arisumi D, et al. Prognosis of patients hospitalized with community-acquired pneumonia. Am J Med. 1990;88:1N-8N.
5. Pearson SD, Goulart-Fisher D, Lee TH. Critical pathways as a strategy for improving care: problems and potential. Ann Intern Med. 1995;123:941-8.
6. Fortune G, Elder S, Jaco D, Bentivegna P, Luebbering T, Boechler M. Opportunities for improving the care of patients with community-acquired pneumonia. Clin Perform Qual Health Care. 1996;4:41-3.
7. Anonymous. Study prompts overhaul of COPD/pneumonia care. Hosp Peer Rev. 1996;21(Suppl 2):121-2.
8. Anonymous. Indiana hospital revamps path for continuum care. Hosp Case Manag. 1997;5:62-4.
9. Sperry S, Birdsall C. Outcomes of a pneumonia critical path. Nurs Econ. 1994;12:332-9.
10. Snowden K, Larrabee JH. Reducing pneumonia incidence through team effort. Nurs Qual Connect. 1994;4:3.
11. American Thoracic Society. Hospital-acquired pneumonia in adults: diagnosis, assessment of severity, initial antimicrobial therapy, and preventive strategies. A consensus statement, American Thoracic Society, November 1995. Am J Respir Crit Care Med. 1996;153:1711-25.
12. Niederman MS, Bass JB Jr, Campbell GD, et al. Guidelines for the initial management of adults with community-acquired pneumonia: diagnosis, assessment of severity, and initial antimicrobial therapy. American Thoracic Society. Medical Section of the American Lung Association. Am Rev Respir Dis. 1993;148:1418-26.
13. Bartlett JG, Breiman RF, Mandell LA, File TM Jr. Community-acquired pneumonia in adults: guidelines for management. The Infectious Diseases Society of America. Clin Infect Dis. 1998;26:811-38.
14. Fine MJ, Hough LJ, Medsger AR, et al. The hospital admission decision for patients with community-acquired pneumonia. Results from the pneumonia Patient Outcomes Research Team cohort study. Arch Intern Med. 1997;157:36-44.
15. Hanson S, Most A, Tammaro D, Hanna L, Stein MD. Community vs. hospital-based pneumonia care: economic effects [Abstract]. J Gen Intern Med. 1997;12:83.
16. Gilbert K, Gleason PP, Singer DE, et al. Variations in antimicrobial use and cost in more than 2,000 patients with community-acquired pneumonia. Am J Med. 1998;104:17-27.
17. Bratzler DW, Murray CK, Bumpus LJ, Moore LL. Community-acquired pneumonia in Oklahoma: characteristics and management of hospitalized Medicare beneficiaries. J Okla State Med Assoc. 1996;89:87-92.
18. Fine MJ, Singer DE, Phelps AL, Hanusa BH, Kapoor WN. Differences in length of hospital stay in patients with community-acquired pneumonia: a prospective four-hospital study. Med Care. 1993;31:371-80.
19. Fine MJ, Smith DN, Singer DE. Hospitalization decision in patients with community-acquired pneumonia: a prospective cohort study. Am J Med. 1990;89:713-21.
20. Gleason PP, Kapoor WN, Stone RA, et al. Medical outcomes and antimicrobial costs with the use of the American Thoracic Society guidelines for outpatients with community-acquired pneumonia. JAMA. 1997;278:32-9.
21. Marsh JA, Gottlieb MS, Casanova JE. Impact of community-acquired pneumonia practice guidelines on total hospital charges and length of stay in Wisconsin hospitals [Abstract].
J Gen Intern Med. 1997;12:83.
22. Weingarten S, Riedinger MS, Sandhu M, et al. Can practice guidelines safely reduce hospital length of stay? Results from a multicenter interventional study. Am J Med. 1998;105:33-40.
23. Nelson EC, Splaine ME, Batalden PB, Plume SK. Building measurement and data collection into medical practice. Ann Intern Med. 1998;128:460-6.
24. Spath P. Clinical Paths: Tools for Outcomes Management. Chicago: American Hospital Publications; 1994.
25. Craven DE, Steger KA. Epidemiology of nosocomial pneumonia. New perspectives on an old disease. Chest. 1995;108:
26. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40:373-83.
27. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613-9.
28. D'Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49:1429-33.
29. D'Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med. 1993;32:382-7.
30. Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;
31. Melfi C, Holleman E, Arthur D, Katz B. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol. 1995;
32. Rochon PA, Katz JN, Morrow LA, et al. Comorbid illness is associated with survival and length of hospital stay in patients with chronic disability. A prospective comparison of three comorbidity indices. Med Care. 1996;34:1093-101.
33. Kleinbaum DG, Kupper LL, Muller KE. Applied regression analysis and other multivariable methods. 2nd ed. Boston: PWS-Kent; 1988.
34. Hosmer DW, Lemeshow S. Applied logistic regression. New York: J Wiley; 1989.
35. Weingarten SR, Riedinger MS, Hobson P, et al. Evaluation of a pneumonia practice guideline in an interventional trial. Am J Respir Crit Care Med. 1996;153:1110-5.
36. Rhew DC, Riedinger MS, Sandhu M, Bowers C, Greengold N, Weingarten SR. A prospective, multicenter study of a pneumonia practice guideline. Chest. 1998;114:115-9.
37. Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. JAMA. 2000;
38. Wen SW, Hernandez R, Naylor CD. Pitfalls in nonrandomized outcomes studies. The case of incidental appendectomy with open cholecystectomy. JAMA. 1995;274:1687-91.
39. Pine M, Norusis M, Jones B, Rosenthal GE. Predictions of hospital mortality rates: a comparison of data sources. Ann Intern Med. 1997;126:347-54.
40. Hannan EL, Kilburn H Jr, Lindsey ML, Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care. 1992;30:892-907.
The authors thank Ms. Amy Jackson for assisting in the preparation of the manuscript, the Clinical Support and Support Office and Information Systems of University Health Systems for providing the data, and Drs. Ralph Whatley and James Byrd for assisting in the design of the study and revision of the manuscript.
Support for this work was provided through the East Carolina University School of Medicine and the Clinical Information and Support Office, Pitt County Memorial Hospital, University Health Systems of East Carolina, Greenville, NC.
Carlos Estrada, MD, MS, Clinical Information and Support Office, Pitt County Memorial Hospital, 2100 Stantonsburg Road, Greenville, NC 27835; telephone: 252-816-7596; fax: 252-816-6719; e-mail: email@example.com.
Earn MOC Points for Medical Knowledge
ACP offers its members many ways to earn ABIM MOC points for Medical Knowledge and to make the process easier. See details.
Ceramic Bistro-Style ACP Mug
Enjoy your morning brew and show your ACP spirit with our 15-ounce dishwasher- and microwave-safe mug. Enjoy free shipping within the continental U.S.