NUR 685 Module 7 Milestone Three Example

Milestone Three: Evaluating a Sepsis Simulation Course from Knowledge to Practice

[Student Name]

Southern New Hampshire University

NUR 685: Nurse Educator Capstone

Module Seven Milestone Three

[Instructor Name]

[Date]

Milestone Three: Evaluating a Sepsis Simulation Course from Knowledge to Practice

The residency's old sepsis lecture was evaluated by a satisfaction form and a recall post-test, and both looked excellent while practice did not change. This milestone plans an evaluation for the new simulation course that avoids that trap. It explains the thinking behind the plan, matches each objective to a measure at the right level of competence, describes instruments, design, data collection and analysis and includes a check on whether the course was delivered as intended.

Thinking About Evaluation

Kirkpatrick's four-level scheme, running from how learners felt about training to what changed for the organization, is the most widely used way of organizing training evaluation. Yardley and Dornan (2012) examined how the model is applied in medical education and argued that treating the levels as a hierarchy, in which higher levels are always more valuable and lower levels naturally lead upward, fits poorly with learning in complex clinical settings, where many factors besides teaching shape behavior and results. They recommended that evaluation be designed around the questions stakeholders actually need answered and that it attend to how and why an education program works, not just to outcomes.

The plan takes that advice in two ways. It measures several outcomes because each answers a different question: whether residents can reason through borderline cases, whether they can act in simulation and whether they act on the floor. And it includes process measures, such as debriefing fidelity and residents' own accounts, so that results can be explained rather than just reported.

Matching Measures to Competence

Miller (1990) described clinical competence as a pyramid rising from knowing, to knowing how, to showing how and finally to doing in real practice, and he argued that assessment must reach the level at which competence is claimed. The course objectives sit mainly at showing how and doing, so the evaluation must reach those levels. Table 1 matches each level to a measure.

Table 1. Evaluation Measures by Level

Note. Satisfaction is collected briefly but not used as an outcome.

Instruments

The case-based test replaces the old recall post-test with ten short cases, six of them borderline, each asking for the next action and a reason. Two experienced sepsis nurses and a hospitalist reviewed the items for accuracy and relevance.

Simulated performance will be scored with the Lasater Clinical Judgment Rubric. Lasater (2007) developed the rubric through observation of nursing students in high-fidelity simulation, grounding it in Tanner's model of clinical judgment. Its four sections follow the four moves of that model, from taking in the situation through acting and looking back, each broken into several dimensions, and four levels of development from beginning to exemplary. Lasater designed it to give students and faculty a shared language for judgment and to support feedback, and she noted that further testing of its measurement properties was needed. The next module examines the argument for using it here. In addition, the time from the patient meeting criteria to the resident's call will be recorded from the simulation log, a simple and objective measure closely tied to the practice gap.

Design and Data Collection

The design combines a within-group comparison with a historical comparison. Within the cohort, case-based test scores are compared before and after the course, and rubric scores and time to call are compared between the first and third scenarios and again at a six-week follow-up scenario, which tests whether performance holds. On the floor, the quality team's chart review of sepsis cases involving current residents in the eight weeks after the course will be compared with the same measure for the previous cohort at the same point in their residency. Two raters, the student and a simulation educator, will score all rubric performances from video after a calibration session, and their agreement will be calculated.

Analysis

With 38 residents, analysis will be modest and descriptive. Paired test and rubric scores will be summarized with medians and compared using a nonparametric paired test. Median time to escalation on the floor will be compared between cohorts, with the number of cases reported, since small numbers limit certainty. Interrater agreement will be reported for rubric scores. Residents' written reflections and debriefing notes will be read for recurring themes that help explain the numbers, such as whether fear of calling persists.

Fidelity and Satisfaction

A short fidelity checklist, completed by the co-facilitator after each debriefing, will record whether each phase occurred, whether advocacy and inquiry were used and whether every resident led a scenario. If fidelity is low, weak results may reflect delivery rather than design. A three-item satisfaction question will still be collected because stakeholders expect it, but the plan states clearly that satisfaction is not evidence of learning.

Timeline and Reporting

Baseline case-based testing occurs on the morning of each course session, followed by the scenarios, with the post-test at the end of the day. Follow-up scenarios run six weeks later during a regular residency day. The quality team pulls floor chart data eight weeks after the last session, and analysis is complete by week twelve. Results will be reported in three forms: a one-page summary for residents, a short presentation to the residency committee with a recommendation to adopt, revise or drop the course and the written capstone. Residents will see their own rubric scores privately, with facilitator comments, since feedback is part of the learning.

Limitations

A historical comparison cannot rule out other changes between cohorts, such as a new screening alert. The follow-up scenario may partly measure familiarity with simulation. Floor data depend on how many septic patients residents encounter in eight weeks, which may be few. The plan accepts these limits and reports them openly.

Conclusion

The evaluation plan reaches from reasoning to simulated action to floor practice, explains results with process measures and treats satisfaction as context. It gives the course a fair test of whether it closes the gap the needs assessment found.

References

Lasater, K. (2007). Clinical judgment development: Using simulation to create an assessment rubric. Journal of Nursing Education, 46(11), 496-503. https://doi.org/10.3928/01484834-20071101-04

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63-S67. https://doi.org/10.1097/00001888-199009000-00045

Yardley, S., & Dornan, T. (2012). Kirkpatrick's levels and education 'evidence'. Medical Education, 46(1), 97-106. https://doi.org/10.1111/j.1365-2923.2011.04076.x