How to Score One-Way Interview Answers

Decide what a good answer looks like before you open a single recording. If you build the standard while watching, early candidates set the bar by accident and your scores drift.

Pick 3 to 5 criteria tied to the role, not to personality. For each question, know which criterion it is meant to test. Then choose one rating scale and use it for everyone.

• Keep criteria to 3 to 5 so reviewers can hold them in mind while watching.
• Define each point on the scale in plain words, so a 3 means the same thing to every reviewer.
• Write down what evidence would earn a top score versus a low score before you start.
• Weight criteria if some matter more, and agree on the weights up front.

Run every candidate through the same steps in the same order. Consistency is what makes async scores comparable.

1. 1

   Define the rubric

   Confirm the criteria, the scale, and any weights. Share them with every reviewer so all of you score against the same standard.

2. 2

   Watch or read each answer

   Go through one candidate's answers fully before scoring. Rewatch a section if you need to. Focus on what they say and how they reason, not the room they filmed in.

3. 3

   Score each criterion

   Rate every criterion on the scale independently. Do not roll everything into one overall gut number, and do not let one criterion pull the others.

4. 4

   Note the evidence

   Write one short line per rating pointing to what in the answer justified it. Evidence forces honesty and makes disagreements between reviewers resolvable.

5. 5

   Compare and shortlist

   Sort candidates by criterion, not by first impression. Compare all candidates on criterion one, then criterion two, and shortlist from the totals.

Here is a simple four-criterion rubric on a 1 to 4 scale. Adapt the criteria to your role, but keep each scale point defined so reviewers agree on what a rating means.

Add the four scores for a total out of 16, or apply weights if some criteria matter more. Use the same math for every candidate.

Because the recording is fixed, more than one person can score the same answer. Have two reviewers score independently, then compare. Where scores diverge by more than one point, reread the evidence notes and reconcile.

AI scoring can transcribe answers, flag rubric criteria, and give a first-pass score. It speeds up review and adds consistency across a large batch. It does not replace judgment.

• Audit AI scores against your own on a sample before trusting them at scale.
• Read the transcript and evidence, not just the AI's number, before you accept or override it.
• Watch for AI penalizing accents, non-native phrasing, or nonstandard delivery that has nothing to do with the answer.
• Keep the final decision with a human, and record who made it.

Recorded answers introduce their own biases. Naming them makes them easier to catch in yourself and in a review panel.

• Judging setup, background, lighting, or accent instead of the content of the answer.
• Halo effect: one strong answer inflating every other score for that candidate.
• Order effects: rating the tenth candidate against the ninth instead of against the rubric.
• Leniency or severity drift, where your scale quietly loosens or tightens as the batch goes on.
• Similarity bias, favoring candidates who remind you of yourself or the team.

To counter order effects, score by criterion across candidates rather than finishing one candidate before starting the next. To counter drift, recheck a few early scores against your rubric halfway through the batch.