PSY 270 Module 4 Performance Appraisal and Training Report example

Measuring and Building Grow Technician Performance at Brightleaf Growers

[Student Name]

Southern New Hampshire University

PSY 270: Introduction to Industrial Organizational Psychology

Module Four Assignment

[Instructor Name]

[Date]

The organization, setting and figures below are a composite written as a model document. No real employer, client, colleague or patient is described.

Measuring and Building Grow Technician Performance at Brightleaf Growers

The Current System

Brightleaf Growers appraises its 140 grow technicians once a year. Each supervisor gives an overall rating from 1 to 5 with a short comment, and the rating determines a small raise. New technicians learn the job by shadowing an experienced colleague for two weeks. The job analysis showed that the most important tasks are nutrient monitoring, crop inspection, data recording, food safety, alarm response and shift handoff, yet neither the appraisal nor the training addresses them directly.

What Last Year's Ratings Reveal

Table 1 summarizes last year's ratings by supervisor group.

Table 1

Annual Ratings and Crop Loss Incidents by Supervisor Group

Note. Composite data from last year's appraisal records and incident logs.

Rater Errors

Three errors appear in the data. Leniency is the most obvious: 85 percent of technicians were rated 4 or 5, so the ratings cannot distinguish strong from weak performers, and raises are almost uniform. Differences between raters are the second: day supervisors rate higher than night supervisors, even though crop loss incidents, a rough objective indicator, are no lower on day shifts. The day shift at Farm 2 has the highest ratings and the most crop losses. A technician's rating therefore depends partly on which supervisor they happen to have. Halo is the third: supervisors' comments suggest that a single visible trait, usually friendliness or speed at harvesting, colors the whole rating, while less visible tasks such as data accuracy are rarely mentioned.

These errors are not signs of careless supervisors. A single global rating asks raters to combine many aspects of performance in their heads with no shared standard, and research on appraisal has long found that the structure of the system shapes rating quality as much as rater effort. DeNisi and Murphy (2017), reviewing a century of appraisal research, concluded that the field has shifted from trying to perfect rating formats toward understanding appraisal as part of performance management, in which purpose, context and feedback matter. A rating that no one trusts cannot guide development, pay or promotion, however carefully it is filled in.

A Redesigned Appraisal

The new appraisal rates six dimensions drawn directly from the job analysis: nutrient management, crop inspection and reporting, data accuracy, food safety compliance, alarm response and shift handoff. Each dimension uses a behaviorally anchored scale describing what performance looks like at each level. For shift handoff, for example, a 1 describes a technician who leaves without passing on known problems, a 3 describes one who records open issues in the log, and a 5 describes one who records issues, explains their urgency in person to the incoming technician and follows up on the next shift. Two dimensions also use objective measures: data entry error rates from the farm management system and food safety audit results. Ratings will be discussed in quarterly check-ins focused on development, with the annual rating used for pay.

Training the Raters

Anchored scales help only if supervisors share an understanding of the anchors. All eight supervisors will complete frame-of-reference training: a half-day session in which they review each dimension and its anchors, rate written and video examples of technician performance, compare their ratings with expert ratings and discuss the differences. Roch et al. (2012) updated the meta-analytic evidence on frame-of-reference training and found that it improved rating accuracy, making it one of the better-supported methods for reducing rater error. Supervisors will repeat a shorter calibration session each quarter, rating two common examples and comparing results across shifts.

A Training Program for New Technicians

The two-week shadowing approach will be replaced by a 90-day program. The first two weeks combine classroom instruction on plant nutrition, food safety and the farm management system with supervised practice in a training room. Weeks three to eight pair each new technician with a certified trainer, with a checklist of tasks that must be demonstrated before they are performed alone. Alarm response is practiced monthly in simulated drills. Weeks nine to twelve are independent work with weekly feedback from the supervisor. Arthur et al. (2003) found in a meta-analysis that organizational training produced medium to large effects overall, with effectiveness depending on matching the training method to the skills being taught, which is why the program uses hands-on practice for procedural skills and simulation for rare but critical events.

Evaluating the Training

Kirkpatrick and Kirkpatrick (2006) describe four levels of training evaluation. Table 2 shows how each will be measured.

Table 2

Four-Level Evaluation Plan for the New Technician Program

Note. Composite plan.

Technicians' Side of the Process

An appraisal also has to feel fair to the people being rated. Technicians will receive the six anchored scales during their first week, so the standard is known long before the first rating. Before each quarterly check-in, they will rate themselves on the same scales, and the conversation will start from any gaps between their view and the supervisor's. Night technicians, who rarely see the day supervisors, will be rated only by their own supervisor, with a second supervisor reviewing any rating of 1 or 2 before it is final. These steps give technicians a voice in the process and a route to question a rating, which matters for acceptance as much as the accuracy of the scales themselves.

How the Two Systems Work Together

The appraisal and training systems now share the same six dimensions, so a technician's ratings show which skills need more training, and training records show whether a low rating reflects a skill that was never taught. Over time, comparing the ratings of technicians trained under the new program with those trained by shadowing will show whether the program works at the behavior level. Brightleaf should expect ratings to fall at first, since leniency will decline under the new scales, and should explain this to technicians in advance so that lower numbers are not read as punishment.

References

Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234-245. https://doi.org/10.1037/0021-9010.88.2.234

DeNisi, A. S., & Murphy, K. R. (2017). Performance appraisal and performance management: 100 years of progress? Journal of Applied Psychology, 102(3), 421-433. https://doi.org/10.1037/apl0000085

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler.

Roch, S. G., Woehr, D. J., Mishra, V., & Kieszczynska, U. (2012). Rater training revisited: An updated meta-analytic review of frame-of-reference training. Journal of Occupational and Organizational Psychology, 85(2), 370-395. https://doi.org/10.1111/j.2044-8325.2011.02045.x