NUR 640 Module 7 Milestone Three Example

Milestone Three: Evaluating a School and Hospital Nurse Residency Partnership at All Four Kirkpatrick Levels

[Student Name]

Southern New Hampshire University

NUR 640: Assessment and Evaluation in Nursing Education

Module Seven Milestone Three

[Instructor Name]

[Date]

Milestone Three: Evaluating a School and Hospital Nurse Residency Partnership at All Four Kirkpatrick Levels

New graduate nurses leave their first jobs at high rates, and every departure costs a hospital recruitment, orientation and overtime while it disrupts care. Residency programs are one response: structured, year-long support that combines precepted practice, classroom sessions and peer groups. Lakemont College of Nursing and Riverbend Regional Medical Center, both composites, launched a twelve-month residency three years ago for Lakemont graduates hired by Riverbend. Leaders of both organizations believe it works, but the only data collected is a satisfaction survey at the end of the year, which nearly every resident rates highly. With funding up for renewal, the partnership needs stronger evidence. This milestone designs an evaluation plan using Kirkpatrick's four levels. It argues that measuring at every level, with a named measure and data source for each, is necessary because satisfaction alone says little about whether residents practice more safely or stay longer.

The Framework

Kirkpatrick and Kirkpatrick (2006) describe four levels for evaluating training. Level one, reaction, asks how participants felt about the training. Level two, learning, asks what knowledge, skills or attitudes changed. Level three, behavior, asks whether participants apply what they learned on the job. Level four, results, asks whether the organization's outcomes changed because of the training. Each level is harder and more costly to measure than the one before, and success at a lower level does not guarantee success at a higher one; residents can enjoy sessions and learn content without changing how they practice. The framework fits this partnership because its stated goals span all four levels: residents who feel supported, gain confidence and competence, practice safely and stay.

What the Evidence Suggests to Measure

Goode et al. (2013) summarized ten years of research on a national post-baccalaureate residency program used in academic medical centers. Their residents' one-year retention was consistently higher than commonly reported rates for new graduates, and residents' scores on the Casey-Fink Graduate Nurse Experience Survey showed growth in confidence, organization and communication over the year, with the lowest point often occurring around six months. They also reported that support from managers and preceptors mattered to outcomes. Rush et al. (2013) reviewed studies of formal transition programs and identified features associated with better outcomes, including programs lasting longer than six months, preceptor preparation and structured peer support, while noting that many studies relied on weak designs without comparison groups. Both sources point to retention, confidence and competence as key outcomes and to the need for comparison data.

Measures at Each Level

Table 1 assigns one primary measure to each level. At level two, the Casey-Fink survey was chosen because it is widely used in residency programs, which allows comparison with published results. At level three, the observation checklist focuses on two behaviors the residency teaches explicitly and that matter for safety: structured handoffs and timely escalation when a patient deteriorates. Preceptors will be trained on the checklist using recorded examples. At level four, turnover is the outcome leaders care most about, and delayed escalation before rapid response calls is a safety outcome linked to the behavior measured at level three.

Table 1. Evaluation Measures by Kirkpatrick Level

Note. Each level has one primary measure with a named source and schedule, which keeps data collection realistic for a small partnership.

Attribution and Design

The hardest problem at level four is attribution. If turnover falls, the residency may be the cause, or the change may reflect a new pay scale, a slower job market or a new unit manager. A randomized design is not possible, since every new graduate enters the residency. The plan uses the strongest practical alternative: comparing first-year turnover for the three residency cohorts with the three cohorts hired before the residency began, and with new graduates hired by Riverbend from other schools who completed standard orientation only. Other changes during the period, such as wage increases, will be documented and reported alongside the results. The evaluation will describe its findings as consistent with an effect of the residency rather than as proof.

Using the Findings

Evaluation is useful only if it changes decisions. Results will be reviewed twice a year by a joint committee of Lakemont faculty, Riverbend educators and two residents. Specific triggers are set in advance: if Casey-Fink scores show the expected six-month dip, additional peer support sessions will be scheduled in months five through seven; if the level three checklist shows escalation problems, the related simulation will be repeated; and if session ratings for a topic fall below an average of 3.5 on a five-point scale for two consecutive months, the topic will be redesigned. An annual report will go to both organizations' leaders and to the funding committee, presenting findings at all four levels, including any that are disappointing.

Feasibility and Ethics

The plan was kept small enough to sustain. Most data already exists or can be collected in a few minutes: turnover and rapid response data are routine hospital records, and the survey takes about fifteen minutes. The largest new burden is preceptor observation, which is limited to three points in the year. Residents will be told how their data will be used, survey responses will be reported in aggregate, and results will never be used in individual performance reviews, so residents can answer honestly.

Conclusion

A satisfaction survey told the partnership that residents liked the program. An evaluation at all four of Kirkpatrick's levels, with a named measure at each, a comparison design that addresses attribution honestly, triggers that link findings to changes and transparent reporting, will tell the partnership whether residents learn, practice more safely and stay, which is the evidence the renewal decision requires.

References

Goode, C. J., Lynn, M. R., McElroy, D., Bednash, G. D., & Murray, B. (2013). Lessons learned from 10 years of research on a post-baccalaureate nurse residency program. Journal of Nursing Administration, 43(2), 73-79. https://doi.org/10.1097/NNA.0b013e31827f205c

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler.

Rush, K. L., Adamack, M., Gordon, J., Lilly, M., & Janke, R. (2013). Best practices of formal new graduate nurse transition programs: An integrative review. International Journal of Nursing Studies, 50(3), 345-356. https://doi.org/10.1016/j.ijnurstu.2012.06.009