HCM 440 Module 7 Project Two Example

Project Two: A Proposal to Evaluate Two-Way Text Reminders Across Fairhaven Medical Group

[Student Name]

Southern New Hampshire University

HCM 440: Healthcare Research and Evaluation Methodologies

Project Two

[Instructor Name]

[Date]

The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.

Project Two: A Proposal to Evaluate Two-Way Text Reminders Across Fairhaven Medical Group

Abstract

Missed appointments cost Fairhaven Medical Group more than 54,000 visits a year. Six of its 14 clinics replaced automated phone reminders with two-way texts in March 2026, and their no-show rate fell, but other changes occurred at the same time. This proposal describes a fourteen-month evaluation combining a controlled interrupted time series for the pilot clinics with a stepped wedge rollout, in randomized order, for the remaining eight. Weekday no-shows, counted from scheduling timestamps, serve as the main outcome. A patient survey and staff interviews will explain how the change works. Results will guide a $140,000-a-year expansion decision.

Why the Study Matters

Roughly one appointment in six at Fairhaven ends with nobody in the exam room and no warning. Behavioral health loses about a quarter of its visits this way, while new patients there wait more than three weeks for an opening. The finance office puts forgone revenue at about $8.1 million a year. Leadership must now decide whether to buy the texting service for every clinic. A decision based on the pilot's raw before-and-after figure could waste money or discard something useful, so a stronger answer is worth a modest investment.

Aims and Hypotheses

Aim 1 is to estimate the effect of two-way texts, relative to automated calls, on the weekday no-show rate; the hypothesis is a drop of two points or more. Aim 2 is to estimate the effect on the share of canceled slots rebooked before the visit; the hypothesis is an increase. Aim 3, exploratory, is to learn whether effects vary with age, preferred language or specialty. Aim 4 is to capture what the switch feels like from behind the reception desk and from the patient side.

Program Theory

Texts are expected to work in two ways. Messages sent three days and one day ahead jog the memory of patients who would otherwise forget. And because patients can cancel with a one-word reply at any hour, some visits that would have been silent no-shows become early cancellations that schedulers can offer to someone on the waiting list. The second path matters for access even if the no-show rate changes little, which is why the refill rate is a formal outcome.

What the Evidence Suggests

Pooled randomized trials reviewed by Guy et al. (2012) showed a consistent attendance gain with texts, with an odds ratio of 1.48, across primary care and hospital outpatient settings. The Cochrane review by Gurol-Urganci et al. (2013) found texts better than no reminder and roughly equal to phone calls, though with low to moderate certainty. Earlier misses, youth and bookings made far ahead all raise risk according to Dantas et al. (2018), predictors which the analysis will adjust for. Few studies compare two-way texts with automated calls in a US multispecialty group, the gap this study addresses.

Design

Part one compares the six pilot clinics with the eight phone clinics using monthly data from March 2024 through the month before the first phone clinic switches. Part two moves the eight phone clinics to texting two at a time, every second month, with the order set by a lottery drawn at a leadership meeting. Hemming et al. (2015) note that this stepped wedge approach produces randomized comparisons while every site eventually adopts the program, matching Fairhaven's intended rollout. The randomized part protects against the selection problem in part one, where clinics volunteered.

Setting, Participants and Exclusions

All 14 clinics in the group take part, from family medicine and children's care to heart, bone and joint, mental health and obstetric clinics. Eligible appointments are scheduled in-person weekday visits for patients aged 18 or older. Same-day bookings, telehealth, hospital-scheduled procedures and patients who opted out of reminders are excluded. Pediatric appointments are excluded because parents receive the reminders and would need a separate analysis.

Measures and Data Sources

A no-show is counted when the scheduling system shows no check-in and no cancellation logged two or more hours before the start time; this timestamp rule replaced status codes after an audit found miscoding, and it will be validated against record review of 300 visits. The refill outcome uses rebooking timestamps. Reminder type comes from clinic assignment and platform logs. Covariates include age, language, payer, lead time, prior no-shows, specialty and month. An eight-item satisfaction scale, tested in a 40-patient pilot, measures patient experience.

Sample and Power

Appointment analyses include every eligible visit, about 240,000 a year, so no sampling is needed, though power depends on the 14 clinics as clusters. With 24 months of prior data and clinic-level variation estimated from past records, a statistician's simulation suggests at least 80% power to detect a 2-point reduction in part two. For the survey, 356 responses per group are required to detect a 10-point difference in satisfaction, and with about 45% of those asked expected to reply, invitations go to 790 people in each arm.

Analysis

Part one uses a controlled segmented regression on monthly counts with appointment volume as an offset, month terms for seasonality and a negative binomial model if counts are overdispersed, following Lopez Bernal et al. (2017). Part two uses mixed-effects logistic regression with clinic as a random effect and period as a fixed effect. Subgroups are tested with interaction terms and labeled exploratory. Survey results are compared with chi-square and t-tests. Effects are reported with 95% confidence intervals and compared with the 2-point threshold leadership agreed on in advance.

Table 1. Aims, Outcomes and Analysis

Note. Prespecified before data extraction.

Qualitative Strand

Twelve to sixteen front-desk staff, chosen to represent every specialty and both reminder types, will be interviewed for about 30 minutes each about how patients respond, what replies require and what slows rebooking. Braun and Clarke (2006) provide the approach: two coders will build a shared codebook from early transcripts, then develop and test themes across all interviews. Findings will explain the numbers, for example why refill rates may lag in clinics without a waiting list.

Ethics and Data Protection

Lynn et al. (2007) argued that improvement projects deserve ethical oversight proportionate to their risk, whatever label they carry. The proposal will go to an external review board, requesting expedited review and a waiver of consent for appointment data. The Belmont principles guide specific safeguards (National Commission, 1979): opt-outs and voluntary surveys respect patients' choices, texts omit specialty and clinician names to protect privacy on shared phones, the lottery prevents favoritism and Spanish messaging and continued phone calls protect groups at higher risk of missing visits. Staff interviews are confidential from managers.

Budget

The evaluation costs about $47,500, roughly a third of one year's texting contract, excluding the texting service itself, which operations has already funded. Table 2 lists the items.

Table 2. Evaluation Budget

Note. Estimates from the finance office and vendor quotes.

Timeline

Months one and two cover ethics review, the data dictionary and the lottery. Part one analysis runs in months three and four, as the first pair of phone clinics switches. The remaining pairs switch in months five, seven and nine, while interviews and the survey run between months five and ten. Final extraction and analysis occupy months eleven and twelve, and reporting months thirteen and fourteen.

Dissemination

Leadership will receive a written report and a 20-minute briefing with a clear recommendation. Front-desk teams will see results in their weekly huddles, including what their own interviews contributed. The patient advisory council will review findings on satisfaction and privacy. Beyond Fairhaven, the analyst plans a conference abstract aimed at medical group managers and, if results are informative, a short article to a health services journal.

Limitations

Part one may still be influenced by the Saturday hours and other pilot-clinic changes, although weekday-only analysis and comparison clinics reduce this. If a pair of clinics switches late, the stepped wedge comparison loses some of its balance. With only 14 clusters, small effects may go undetected. And results from one metropolitan group may not transfer to rural practices or populations with different phone access.

Conclusion

This proposal gives Fairhaven a way to answer an expensive question with evidence it can trust. It uses data the group already collects and a rollout it already plans, adds randomization where it matters most, protects patients and staff and costs about a third of one year's contract. When the evaluation ends, leadership will know not only whether texts reduce no-shows but for whom and why.

References

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Dantas, L. F., Fleck, J. L., Cyrino Oliveira, F. L., & Hamacher, S. (2018). No-shows in appointment scheduling: A systematic literature review. Health Policy, 122(4), 412-421. https://doi.org/10.1016/j.healthpol.2018.02.002

Gurol-Urganci, I., de Jongh, T., Vodopivec-Jamsek, V., Atun, R., & Car, J. (2013). Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database of Systematic Reviews, 2013(12), Article CD007458. https://doi.org/10.1002/14651858.CD007458.pub3

Guy, R., Hocking, J., Wand, H., Stott, S., Ali, H., & Kaldor, J. (2012). How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Services Research, 47(2), 614-632. https://doi.org/10.1111/j.1475-6773.2011.01342.x

Hemming, K., Haines, T. P., Chilton, P. J., Girling, A. J., & Lilford, R. J. (2015). The stepped wedge cluster randomised trial: Rationale, design, analysis, and reporting. BMJ, 350, Article h391. https://doi.org/10.1136/bmj.h391

Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348-355. https://doi.org/10.1093/ije/dyw098

Lynn, J., Baily, M. A., Bottrell, M., Jennings, B., Levine, R. J., Davidoff, F., Casarett, D., Corrigan, J., Fox, E., Wynia, M. K., Agich, G. J., O'Kane, M., Speroff, T., Schyve, P., Batalden, P., Tunis, S., Berlinger, N., Cronenwett, L., Fitzmaurice, J. M., . . . James, B. (2007). The ethics of using quality improvement methods in health care. Annals of Internal Medicine, 146(9), 666-673. https://doi.org/10.7326/0003-4819-146-9-200705010-00155

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979). The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. U.S. Department of Health, Education, and Welfare.