HCM 440 Module 5 Measurement Short Paper Example

Counting What Matters: Measurement and Data Collection for the Fairhaven Reminder Study

[Student Name]

Southern New Hampshire University

HCM 440: Healthcare Research and Evaluation Methodologies

Module Five Short Paper

[Instructor Name]

[Date]

The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.

Counting What Matters: Measurement and Data Collection for the Fairhaven Reminder Study

The proposal for Fairhaven Medical Group's reminder study defined its outcomes in a sentence each. A sentence is not yet a measure. This paper explains exactly how each concept will be turned into data, where the data come from, who will be sampled and how the team will know that the measures are reliable and valid. Several decisions below changed after a small audit showed that the obvious data were less accurate than everyone assumed.

Concepts and Operational Definitions

A conceptual definition says what an idea means; an operational definition says how it will be observed and counted. "Missed appointment" is a concept. The operational version must say which appointments count, what counts as missing and over what period. Shadish et al. (2002) call the fit between a concept and its measure construct validity, and they warn that a measure capturing only part of a concept, or something else entirely, can make a real effect look absent. Each variable below therefore has both definitions written out.

Rebuilding the No-Show Measure

The scheduling system assigns each appointment a status: arrived, no-show, canceled by patient or canceled by clinic. The first plan was to count no-show codes. A chart audit of 400 randomly selected appointments showed that about 4% of visits coded as patient cancellations were actually cases in which the patient never came, recoded later to keep rates down or to reuse the slot. Because front-desk habits may differ between clinics, this error could bias comparisons. The revised measure uses timestamps instead: an appointment is a no-show if no check-in was recorded and no cancellation was logged at least two hours before the start time.

Checking the New Measure

To confirm the timestamp rule, a second audit of 300 appointments will compare it against a reviewer's reading of the full record, including phone notes. Agreement will be reported as the percentage of appointments classified the same way and as sensitivity and specificity for detecting true no-shows. If agreement falls below 95%, the rule will be adjusted before analysis. This comparison against a more thorough standard is a check of criterion validity.

Secondary Outcomes

The refill rate is defined as patient-canceled slots in which another appointment was booked before the original start time, divided by all patient-canceled slots. The late cancellation rate counts cancellations logged less than 24 hours ahead. Both come from timestamps. Patient satisfaction with reminders will be measured through a short survey described below. Clinician idle time is excluded because clinics record it inconsistently, a limit noted in the proposal.

Measuring Exposure

The study compares reminder types, so it must know which reminder each patient actually received. The texting platform logs each message's delivery status and any reply, and the phone system logs completed calls. About 7% of patients in pilot clinics have no working mobile number and receive a phone call instead. The main analysis will classify appointments by the clinic's assigned reminder type, an approach similar to intention to treat, while a secondary analysis will use the reminder actually delivered.

Covariates

Dantas et al. (2018) identified prior no-shows, younger age and longer time between booking and visit among the strongest predictors of missed appointments, alongside insurance type and distance. These will be extracted for every appointment so the analysis can adjust for differences between clinics and over time. Additional covariates include preferred language, specialty, new versus returning patient, day of week and month. Weather is excluded because the clinics share one metropolitan area and would experience the same conditions.

Levels of Measurement

Each variable's level of measurement determines which statistics can be used later. Table 1 lists the main variables, their level and source. Appointment outcomes are nominal at the visit level but become ratio-level rates when aggregated by clinic and month, which matters for the time series analysis.

Table 1. Main Variables, Levels and Sources

Note. Prepared by the author; all extracts use de-identified study numbers.

Sampling for Appointment Data

No sampling is needed for appointment outcomes. The study includes every eligible appointment, about 240,000 weekday adult visits a year across the 14 clinics. A full census avoids sampling error and removes the temptation to choose convenient months. The unit that matters statistically, however, is the clinic-month, and with 14 clinics the study has limited power to detect small differences between individual clinics even though the appointment count is large.

Sample Size for the Patient Survey

The survey compares the share of patients satisfied with their reminders in text and phone clinics. Following the approach in Hulley et al. (2013), detecting a difference between 60% and 70% satisfied when the study wants an 80% chance of spotting it and accepts a 5% false-alarm risk in either direction requires about 356 completed surveys per group. Past Fairhaven surveys achieved about a 45% response rate, so 790 patients per group will be invited, randomly selected from recent appointments and stratified so Spanish-speaking patients receive the survey in Spanish in proportion to their share.

Reliability of the Survey Scale

The satisfaction scale has eight items, such as whether the reminder arrived at a useful time and whether canceling was easy. Tavakol and Dennick (2011) explain that Cronbach's alpha estimates internal consistency, how closely items measuring the same idea agree, and that values from about 0.70 to 0.95 are usually acceptable, with very high values suggesting redundant items. They also note that alpha assumes the items measure one underlying construct. The scale will be piloted with 40 patients, and items that lower alpha or confuse respondents will be revised.

Validity of the Survey

Content validity will be addressed by asking Fairhaven's patient advisory council and four front-desk leads to review whether the items cover the experiences that matter. Face validity, whether items make sense to respondents, will be checked in the pilot through brief follow-up questions. Spanish items will be translated, back-translated and reviewed by bilingual staff so that both versions measure the same thing.

Qualitative Sampling

Front-desk staff see how patients respond to reminders every day. Twelve to sixteen staff will be interviewed, chosen purposively to include every specialty, both text and phone clinics and both new and experienced employees. Interviews will continue until new interviews stop adding new ideas. Tong et al. (2007) developed the COREQ checklist, 32 items covering the research team and reflexivity, study design and analysis and findings, and the qualitative report will address each item, including the analyst's own role as a colleague of the staff interviewed.

Data Quality Plan

A data dictionary will fix every definition before extraction. Monthly extracts will be checked for missing timestamps, duplicate records and impossible values, such as check-ins before booking. Any clinic with more than 2% missing timestamps in a month will be reviewed with its manager. All data will be stored on the group's secure research drive using study numbers rather than names.

Conclusion

Measurement decisions will decide whether this study's answer can be believed. Replacing status codes with timestamps, checking the rule against record review, measuring actual exposure, adjusting for known predictors, sizing and testing the survey and sampling staff purposively give Fairhaven data that fit its design. Module Six will describe how these data will be analyzed and how patients and staff will be protected.

References

Dantas, L. F., Fleck, J. L., Cyrino Oliveira, F. L., & Hamacher, S. (2018). No-shows in appointment scheduling: A systematic literature review. Health Policy, 122(4), 412-421. https://doi.org/10.1016/j.healthpol.2018.02.002

Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013). Designing clinical research (4th ed.). Lippincott Williams & Wilkins.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53-55. https://doi.org/10.5116/ijme.4dfb.8dfd

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349-357. https://doi.org/10.1093/intqhc/mzm042