Calibration Committees and Rating Distribution Guidance Effects on Leniency Bias in Subjective Performance Evaluations

Abstract

Firms use both calibration committees and rating distribution guidance to reduce leniency bias in subjective performance ratings. Leniency bias is the tendency to provide subordinates with higher ratings than deserved which can weaken the link between incentives and effort, leading to suboptimal and subordinate performance. I employ a 2x2 online experiment to assess how the presence versus absence of peer calibration committees [PCCs] and rating distribution guidance [RDG] affects leniency bias present in supervisors’ ratings of subordinates’ performance. I find support that supervisors may display more leniency in ratings prepared in anticipation of a PCC, especially among low performers. As the increased bias appears to impact low-performers, this may create additional fairness concerns for moderate and high-performers, which could demotivate these subordinates. Next, I find support that rating distribution guidance does have a main effect of reducing the leniency bias displayed among low and high performers. Further, using planned contrast testing, I find support for my predicted pattern of results for low performers. That is, the presence of a PCC has a main effect of increasing leniency bias, the presence of RDG has the main effect of reducing leniency bias, and the interactive effect such that when a PCC is present, the presence of RDG weakens the effect of PCCs on leniency bias. This finding indicates that rating distribution guidance may be helpful in settings with a PCC.