NUR 653 Module 7 Milestone Three Example

Milestone Three: Evaluating Population Programs Under a Shared Savings Contract With a Fair Comparison

[Student Name]

Southern New Hampshire University

NUR 653: Population Care Management

Module Seven Milestone Three

[Instructor Name]

[Date]

Milestone Three: Evaluating Population Programs Under a Shared Savings Contract With a Fair Comparison

Accountable care organizations are paid according to whether their patients' total spending comes in under a benchmark, provided quality thresholds are also reached. When Mesa Valley receives its next shared savings reconciliation, leaders will be tempted to attribute any savings to the new diabetes program and embedded care management. That attribution may be wrong in either direction. Savings could reflect changes in coding, patient mix or regional trends; a program could succeed even in a year without shared savings. This milestone plans an evaluation that isolates the programs' effects. It argues that the randomized enrollment waves already built into the programs, measures organized around the Triple Aim, rules for interpreting results set in advance and reporting tailored to each audience will give Mesa Valley an honest answer about whether its programs work.

What Shared Savings Can and Cannot Show

McWilliams et al. (2016) judged how the first Medicare ACOs fared by comparing spending for their patients with spending for patients of other providers in the same areas, adjusting for trends. The organizations that entered the program first achieved modest spending reductions that grew over time, while those entering later showed smaller or no significant savings in the period studied, and quality measures were stable or improved. The authors noted that savings were small relative to total spending and that organizations varied widely. Those findings mean that an organization's shared savings result reflects many influences, including its benchmark, how patients are attributed and regional trends, and that a single program's contribution cannot be read from it. Mesa Valley's evaluation must therefore compare patients in its programs with similar patients who are not yet in them.

The Trap of Before-and-After Comparisons

Programs that enroll patients because their costs or use are high face a particular risk of false success. Finkelstein et al. (2020) randomized hospitalized patients with complex needs to a well-known care management program or usual care. Hospital use fell dramatically in both groups after enrollment, and readmission rates at 180 days were nearly the same, about 62% in each. Earlier reports based on before-and-after comparisons had credited the program with large reductions, but the trial showed that patients selected at a peak in their use would have improved without it. The same trap applies to Mesa Valley's patients with an A1c above 9%, some of whom were enrolled after an unusually high reading and would improve somewhat without intervention.

Design: Randomized Enrollment Waves

Both programs have more eligible patients than they can serve at once, so enrollment was designed in three waves, four months apart, with eligible patients assigned to waves at random. During the first four months, wave one patients are enrolled while waves two and three receive usual care; during the next four months, waves one and two are enrolled while wave three waits. Comparing enrolled patients with those still waiting, at each point, gives an estimate of the program's effect that is free of regression to the mean, because both groups were selected in the same way at the same time. Everyone eligible is eventually offered the program, so no patient is denied it for the sake of research. The analysis will be done on the basis of assignment, whether or not patients actively participated, to avoid favoring the program by including only engaged patients.

Measures Organized Around the Triple Aim

Berwick et al. (2008) argued that population health efforts should pursue better experience of care, healthier populations and lower spending per person all together, and should measure all three so that gains in one are not achieved at the expense of another. The evaluation follows this structure. For population health, primary measures are the share of registry patients whose most recent A1c exceeds 9% or who went untested, the share with pressure under 140/90 and, for care management, admissions and emergency department visits per thousand enrolled patients. For experience, patients will complete a short survey on whether they feel their care is coordinated and whether staff understand their goals. For cost, total cost of care per patient per month will be calculated from claims. Process measures, such as case manager contacts, in-person visits and medication reconciliation after discharge, will show whether the programs were delivered as designed, including fidelity to the six features of effective care management.

Data will come from three sources already in place: the registry for clinical measures, claims for utilization and cost and a brief survey administered by text or phone at enrollment and six months later. Because claims arrive with a lag of about three months, cost results will trail clinical results, and early reports will say so plainly.

Equity and Subgroups

Because the assessment found worse control for patients who prefer Spanish, for Medicaid enrollees and in the two rural sites, every primary measure will be reported for these groups. The programs will be judged successful only if these groups improve at least as much as others; a program that raises average control while widening gaps will be revised. Sample sizes within some groups will be small, so results will be presented with confidence intervals and interpreted cautiously.

Decision Rules and Reporting

Before the first results, the population health committee agreed on how they will be read. A program will be judged effective if enrolled patients show a meaningful improvement over waiting patients on its primary population health measure, experience does not worsen, cost does not rise beyond the program's direct cost and equity gaps do not widen. If process measures show poor delivery, the first response will be to fix delivery, not to abandon the model. Results will be reported quarterly to the committee, with a one-page summary for practice teams showing their own patients' progress, an annual report to the board and a plain-language summary for patients and the community advisory group.

Conclusion

A shared savings check, or its absence, will not tell Mesa Valley whether its programs work. Randomized enrollment waves that compare enrolled and waiting patients, measures that cover health, experience and cost together, equity as a condition of success and decision rules written in advance will. The design protects the organization from crediting its programs for regression to the mean or regional trends and from abandoning a good program in a bad year.

References

Berwick, D. M., Nolan, T. W., & Whittington, J. (2008). The Triple Aim: Care, health, and cost. Health Affairs, 27(3), 759-769. https://doi.org/10.1377/hlthaff.27.3.759

Finkelstein, A., Zhou, A., Taubman, S., & Doyle, J. (2020). Health care hotspotting: A randomized, controlled trial. New England Journal of Medicine, 382(2), 152-162. https://doi.org/10.1056/NEJMsa1906848

McWilliams, J. M., Hatfield, L. A., Chernew, M. E., Landon, B. E., & Schwartz, A. L. (2016). Early performance of accountable care organizations in Medicare. New England Journal of Medicine, 374(24), 2357-2366. https://doi.org/10.1056/NEJMsa1600142