When a panel scores without a shared understanding of what the criteria mean, the variation that shows up later isn’t really about the applications — it’s about the reviewers. One reads “demonstrated need” generously, another strictly. Both are reasonable. Together they produce results that are hard to defend.
What a norming session is
Before scoring opens, gather the reviewers for half an hour. Score a couple of sample applications together. Talk about where people landed and why. That’s the whole exercise. The goal isn’t consensus on a number — it’s a shared yardstick.
Why it works
Reviewers leave with the same picture of what a 2 looks like versus a 4. The scores that follow mean the same thing across the panel, and the disagreements that remain are real disagreements about merit — not artifacts of different interpretations.
The payoff reaches the board
Calibrated scores produce recommendations that hold up under questioning. When a trustee asks why one application beat another, the answer is grounded in a standard everyone applied the same way.
Calibration is one of the least visible parts of a fair review and one of the highest-return uses of thirty minutes in the entire cycle.
Harborline helps organizations run review panels that are consistent and defensible. If a review round is ahead, we can help.
← Back to Insights