Courses / Hiring Operations (processes & tooling)
30. Inter-Rater Reliability Check
Compute or qualitatively assess consistency across raters for a single rubric item to detect drift.
Hiring operations (processes & tooling)
Inter-Rater Reliability Check
A way to measure whether multiple interviewers score the same rubric item consistently.
"I worry that different interviewers mean different things by 'culture fit'—are we even scoring the same thing?"
Inter-rater reliability (IRR) checks quantify or qualitatively assess agreement among raters on a single rubric item to detect drift, bias, or unclear criteria so you can fix the rubric or retrain raters.
"I worry that different interviewers mean different things by 'culture fit'—are we even scoring the same thing?"
Inter-rater reliability (IRR) checks quantify or qualitatively assess agreement among raters on a single rubric item to detect drift, bias, or unclear criteria so you can fix the rubric or retrain raters.