IHP 340 Module 8 Final Project Analysis Review example

Did the Training Change What Patients Heard? A Statistical Review of a Physician Communication Survey

[Student Name]

Southern New Hampshire University

IHP 340: Statistics for Healthcare Professionals

Module Eight Final Project

[Instructor Name]

[Date]

The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.

Did the Training Change What Patients Heard? A Statistical Review of a Physician Communication Survey

Context and Research Questions

A composite multispecialty physician group with 22 physicians invested in a half-day communication skills program covering plain-language explanation, teach-back and shared agenda setting. The quality director wants to know whether patients noticed. Good physician communication matters beyond satisfaction: a meta-analysis found that patients of physicians who communicate poorly have a 19 percent higher risk of nonadherence, and that communication training increased the odds of patient adherence by 1.62 times (Zolnierek & DiMatteo, 2009).

The analysis addresses three questions. First, did the share of patients giving the top rating for how clearly the doctor explained things change after the training? Second, did the mean communication composite score change? Third, is the length of the visit related to how patients rate communication?

Data and Variables

The group mailed a short survey based on the communication items of a standard patient experience instrument for clinician visits (Agency for Healthcare Research and Quality [AHRQ], n.d.) to random samples of patients seen in the three months before training and the three months after. Each period yielded 180 completed surveys. The variables are the period (before or after, nominal); the answer to how often the doctor explained things clearly, recoded as top box (always) or not (nominal); a communication composite, the mean of four items scored 1 to 4, treated as interval; and visit length in minutes from the scheduling system (ratio).

Question One: Top-Box Ratings

Because both variables are categorical, a chi-square test of independence was used. Before training, 112 of 180 patients (62.2 percent) said the doctor always explained things clearly; after training, 130 of 180 (72.2 percent) did. The test gave chi-square = 4.08 with 1 degree of freedom and p = .043. The share of top-box ratings is associated with the period, rising by 10 percentage points. The result is statistically significant, but only just, so it should be read as encouraging rather than conclusive.

Question Two: Composite Scores

For the composite score, an independent-samples t test compared the two periods, since different patients responded each time (Daniel & Cross, 2018). The mean rose from 3.41 (SD 0.52) before training to 3.56 (SD 0.48) after. The test gave t = 2.84 with 358 degrees of freedom and p = .005. The 95 percent confidence interval for the increase runs from 0.05 to 0.25 points on the 4-point scale, and Cohen's d is 0.30, a small effect. Patients rated communication somewhat higher after training, but the improvement is modest in size.

Question Three: Visit Length

A Pearson correlation between visit length and composite score across all 360 patients gave r = 0.21, p < .001. Longer visits were associated with slightly higher communication ratings. The coefficient of determination is 0.04, however, meaning visit length accounts for only about 4 percent of the variation in scores. With a large sample, even a weak correlation is statistically significant, so this finding says that time helps a little but is far from the main driver of how patients rate communication.

Summary of Results

The table summarizes the three analyses.

Table 1

Summary of Survey Analyses

Statistical Versus Practical Significance

All three results are statistically significant, but they differ in practical importance. The 10-point rise in top-box ratings would matter to the group if it held, because patient experience scores are publicly reported for many practices and often tied to contracts, yet its P value sits close to .05. The composite score change is more firmly significant but small, about three tenths of a standard deviation. The visit length correlation is highly significant only because the sample is large; its effect is slight. A manager reading only the P values would conclude that all three findings are strong. Reading the effect sizes shows a more accurate picture: a modest improvement in communication ratings, with little support for the idea that longer visits are the answer. This distinction is the main lesson the analysis offers the quality committee, which had planned to present the results to the board as proof that the training worked.

Limitations

The design is before and after without a comparison group, so changes could reflect seasonal differences in patients, new staff or national attention to patient experience rather than the training itself. Surveys were mailed, and patients who return surveys may differ from those who do not. Ratings were analyzed by patient, but patients are clustered within physicians, which the simple tests used here do not account for; a physician with many surveys could influence the results. Finally, ratings measure perceived communication, not adherence or health outcomes.

Recommendations

The evidence supports continuing the communication program, since all three analyses point in a favorable direction and the costs are modest. It does not support claiming a large effect. Three steps would strengthen the next evaluation: stagger training so that some physicians serve as a comparison group for a few months, analyze results by physician to find who improved and who did not, and add an outcome closer to health, such as medication refill rates for patients with chronic conditions. Because longer visits were only weakly related to ratings, the group should focus on communication skills rather than lengthening appointments, which would reduce access without much gain in how patients experience the visit.

Conclusion

After communication training, the proportion of patients giving the top rating for clear explanations rose from 62 to 72 percent, a borderline significant change, and composite communication scores rose by a small but significant amount. Visit length had only a weak association with ratings. Taken together, the results suggest a real but modest improvement, and the next evaluation should use a stronger design to confirm it.

References

Agency for Healthcare Research and Quality. (n.d.). CAHPS clinician & group survey. https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html

Daniel, W. W., & Cross, C. L. (2018). Biostatistics: A foundation for analysis in the health sciences (11th ed.). Wiley.

Zolnierek, K. B. H., & DiMatteo, M. R. (2009). Physician communication and patient adherence to treatment: A meta-analysis. Medical Care, 47(8), 826-834. https://doi.org/10.1097/MLR.0b013e31819a5acc