PHE 423 Module 2 Frameworks Short Paper Example

Two Maps for One Evaluation: Applying the CDC Framework and RE-AIM to a Smoke Alarm Installation Program

[Student Name]

Southern New Hampshire University

PHE 423: Evaluation Methods in Public Health

Module Two Short Paper

[Instructor Name]

[Date]

The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.

Two Maps for One Evaluation: Applying the CDC Framework and RE-AIM to a Smoke Alarm Installation Program

Before choosing measures or designs, an evaluator needs a structure for thinking about what the evaluation is for and what it should cover. Two frameworks are widely used in public health. The CDC framework describes the process of evaluation in six steps, while RE-AIM describes the dimensions on which a program's public health impact should be judged. This paper applies both to a program in a composite river city in southeast Missouri, where the fire department and county health department install free sealed ten-year smoke alarms in older rental homes in two neighborhoods, with a third neighborhood scheduled next year. Each framework is applied in turn, then the two are combined.

The CDC Framework

The CDC's framework for program evaluation sets out six linked steps, from engaging stakeholders to ensuring use, and holds every evaluation to standards of usefulness, practicality, ethical conduct and accuracy (Centers for Disease Control and Prevention, 1999). The first step is engaging stakeholders. For this program, they include the fire chief and installation crews, the health department, landlords, tenants and the neighborhood associations, the state fire marshal's grant program that funds the alarms and the hospital emergency department. The second step is describing the program: who it serves, what it does, what resources it uses and what it expects to change. The third is focusing the design, choosing the questions that matter most to these stakeholders. The fourth is gathering credible evidence, the fifth justifying conclusions by comparing findings with standards and the sixth ensuring use and sharing lessons. Each step builds on the one before it.

What Stakeholders Want to Know

Conversations with stakeholders showed different priorities. The fire chief wants to know whether the program reaches the homes where fires are most likely. Crews want to know whether tenants keep alarms working or take them down after false alarms from cooking. Landlords want to know whether the alarms meet code and whether installation causes problems with tenants. The grant program wants evidence of reduced injuries and deaths. Tenants, in a small listening session, mostly wanted to know whether the program was free and whether anyone would enter their homes for any other purpose. The CDC framework treats these differences as information: the evaluation must answer questions that matter to people who will use the results, and must be designed so that tenants trust it.

RE-AIM

Glasgow et al. (1999) proposed RE-AIM to judge the public health impact of programs on five dimensions. Reach is the share and representativeness of the target population that participates. Effectiveness is the impact on outcomes, including unintended effects. Adoption asks how many of the agencies, sites and staff who could offer the program choose to offer it, and how typical they are. Implementation asks how faithfully the program is carried out in the field. Maintenance is whether effects and delivery last over time. The authors argued that programs evaluated only on effectiveness can look promising while reaching few people or failing to last, which wastes resources.

RE-AIM Applied

Applied to the smoke alarm program, each dimension points to a measure. Reach is the share of targeted rental homes in which alarms were installed, and whether those homes include the highest-risk households, such as older adults living alone, homes with space heaters and homes with past fires. Effectiveness is the share of homes with working alarms at follow-up and, in the long run, fire injuries and deaths. Adoption is the share of fire stations whose crews take part and the share of landlords who allow installation. Implementation is whether crews install the specified number of alarms in the right locations, test them and explain them to tenants. Maintenance is whether alarms are still working after a year and whether the program continues after the grant ends.

Lessons From Earlier Programs

The London trial reported by DiGuiseppi et al. (2002) shows why implementation and maintenance matter. Alarms were provided, but few households installed or kept them working, so the program could not reduce injuries. In RE-AIM terms, the program had reach on paper but failed on implementation and maintenance. An evaluation of the river city program that measured only alarms installed would miss exactly the failure that sank the London effort.

How the Frameworks Fit Together

The two frameworks answer different questions. The CDC framework tells the evaluator how to proceed: whom to involve, in what order to make decisions and how to judge whether the evaluation itself is useful, feasible, ethical and accurate. RE-AIM tells the evaluator what to measure to judge impact. Used together, the CDC steps provide the process and RE-AIM supplies much of the content for step three, focusing the design. The logic model in Module Three will connect the program's activities to these measures.

Applying the Standards

The CDC framework's four standards also shape choices. Utility means the evaluation must answer questions stakeholders will act on, which is why working alarms at one year, the measure crews and the fire chief care about, comes before injury outcomes that will take years to show. Feasibility means the evaluation must fit the program's budget and staff; follow-up visits can be added to crews' routine work rather than requiring new staff. Propriety means the evaluation must respect the people involved; tenants will be asked for consent before any follow-up visit, told exactly what will be checked and assured that no information goes to landlords or code enforcement. Accuracy means measures must be valid and reliable; crews will use a short checklist and test each alarm rather than relying on tenants' reports. Applying the standards early prevents an evaluation that is rigorous on paper but impossible or harmful in practice.

Conclusion

Applying the CDC framework revealed stakeholders with different questions and a need for tenant trust. Applying RE-AIM showed that the program's success depends as much on reach, implementation and maintenance as on whether alarms work when installed. The evaluation will therefore measure all five dimensions, focusing first on working alarms a year after installation. Injury outcomes will follow as data accumulate.

References

Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR Recommendations and Reports, 48(RR-11), 1-40.

DiGuiseppi, C., Roberts, I., Wade, A., Sculpher, M., Edwards, P., Godward, C., Pan, H., & Slater, S. (2002). Incidence of fires and related injuries after giving out free smoke alarms: Cluster randomised controlled trial. BMJ, 325(7371), 995-998. https://doi.org/10.1136/bmj.325.7371.995

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322-1327. https://doi.org/10.2105/AJPH.89.9.1322