Learning how to score video interviews is mostly learning to decide in advance what a good answer contains, then holding every candidate to that same description. In a one-way interview the recordings arrive all at once, and without a written standard the reviewer's mood, the order of viewing and the candidate's lighting quietly become criteria. This guide covers the rubric, anchored scales, scoring from transcripts, keeping reviewers consistent, and the realistic role of AI scoring, with a free scorecard at the end.
Key takeaways
- Write the rubric before you open a single recording. Criteria decided afterwards drift toward whoever you saw first.
- Use a small number of criteria, each with written anchors that describe what a strong, acceptable and weak answer contains.
- Score from the transcript, then watch video only for close calls. Text is faster and hides what you should not be scoring.
- AI scoring orders the queue and points to evidence. The decision, and any rejection, belongs to a person.
Write the rubric before you review
A rubric is a short list of what you are judging and what each level looks like. Its job is to move the judgement from after the recording to before it. If you watch five answers and then decide what mattered, those five have set the bar, and everyone after is scored against them, not against the job.
Start from the questions. Each was written to reveal something, so each criterion should name that something. A question about an unhappy customer is scored on whether a specific situation was described, what the candidate did, and how it ended, not on confidence or vocabulary unless the job requires them. Four to six criteria is enough; beyond that reviewers stop telling them apart and the scores collapse into a general impression.
- Relevance: did the answer address the question that was asked, or a nearby easier one?
- Specificity: is there a real example with detail, or a general statement of approach?
- Clarity: could a colleague understand the answer on first reading?
- Role fit: does the content show the skill or judgement this job actually needs?
- Must-haves: any fact the role requires, kept as a yes or no flag rather than a score.
Anchor the scale in words
A number on its own means different things to different people: one reviewer's three is another's four. Anchoring fixes this with a sentence per criterion describing what each score contains, in terms of what is in the answer rather than how the reviewer felt about it.
| Score | Specificity anchor | Clarity anchor |
|---|---|---|
| 1 | No example. General statements about what one should do. | Hard to follow. Point unclear even after re-reading. |
| 2 | An example is mentioned but with no detail of what the candidate did. | Understandable with effort. Wanders or repeats. |
| 3 | A specific situation, the candidate's own actions, and an outcome. | Clear on first reading. Structured with a beginning and end. |
| 4 | As 3, plus what they learned or would change, tied to this role. | Concise and precise. Nothing to cut, nothing missing. |
A four-point scale is easier to anchor than a five-point one because it has no middle to hide in. Write the anchors so a 3 describes the answer you would be happy to advance. If most candidates land at 1 and 2, the question may be badly worded rather than the pool weak.
How to score video interviews from the transcript
The transcript should be your primary document, and the video a reference. Two reasons. Speed: text is read far faster than a recording is watched. Fairness: text removes background, lighting, clothing and accent, none of which are criteria, all of which shape a first impression on video.
- 1
Score one question across all candidates
Score every candidate's answer to question one before moving to question two. Comparing like with like is more consistent than forming a view of each person in turn.
- 2
Find the evidence first, then the score
Highlight the line that earns the score. If you cannot point to one, the score is an impression, not an assessment.
- 3
Write one phrase per score
'Named the complaint and the refund decision' is evidence. 'Good answer' is not.
- 4
Open video for the close calls
When a transcript reads oddly, or two candidates sit on the same total, watch the recording. Tone can settle a tie; it should not create one.
- 5
Total last
Add the criteria only after each has been scored on its own, so a memorable answer does not pull the rest up with it.
Calibrate between reviewers
When two people score the same pool, the risk is not disagreement but undetected disagreement. If one reviewer is a point harsher and you split the candidates alphabetically, half the pool competes under a stricter standard and nobody knows.
- Both reviewers score the same three candidates independently before anyone else.
- Compare criterion by criterion, not total by total. A matching total can hide two opposite disagreements.
- Where scores differ by more than one point, read the anchor together and ask what each person was pointing at. Usually the anchor is ambiguous, so rewrite it.
- Agree what a 3 looks like for each question, using a real example, and keep it visible while reviewing.
- Re-check on a shared candidate halfway through the pool, since reviewers drift as they tire.
If disagreement persists, the criterion is often really two things. 'Communication' hides both structure and vocabulary; one reviewer weights the first and one the second. Split it, and the scores usually converge.
What AI scoring does and does not do
AI scoring reads each transcript against your rubric and gives a provisional score with the supporting lines before a reviewer opens anything. It changes where the reviewer starts: at the top of a ranked list rather than an alphabetical one. On springhire every AI score links to the transcript passage it drew on, so checking a score means reading a sentence rather than watching a recording.
| AI scoring does | AI scoring does not |
|---|---|
| Apply the same rubric to every answer, at any hour, in the same way. | Know what you meant if the rubric is vague. A weak rubric produces consistent, wrong scores. |
| Order the queue so the strongest transcripts are read first. | Decide who advances. The cut line and the decision are a person's. |
| Point to the evidence for each score. | Reject anyone. Only a knock-out question about a fact should do that, never a score. |
Treat the AI score as a first reviewer whose reasoning is always on the page. Read the evidence for anyone within a point of your cut line, and anyone whose ranking surprises you. Where you disagree, note why: a pattern of disagreements means the rubric wording needs changing.
Bias traps specific to recorded answers
Much of how to score video interviews fairly is knowing the biases the format adds. Recorded interviews remove some sources of bias, since everyone gets the same questions, and introduce others.
- Production quality. A quiet room and a good camera signal circumstances, not competence. Scoring from the transcript neutralises this.
- Order effects. The first candidates set the bar and the last meet a tired reviewer. Score by question, not by candidate.
- Fluency over content. A smooth answer with no example often outscores a halting one with a precise example. The specificity anchor exists for this.
- Confirmation from the resume. Reading the CV first colours every transcript. Score the interview first and read the resume afterwards.
A scorecard to start from
If you do not have a scoring sheet, this one is the structure described above: criteria in rows, an anchored one-to-four scale, a must-have flag, an evidence column, a decision line and a short bias check. Rename the criteria to match your questions.
Interview scorecard template (CSV)Six criteria, must-have flags, evidence column, decision line and a three-question bias check. Opens in Excel, Numbers or Sheets. No email required.The same criteria can be entered as the rubric for a one-way interview on springhire, where the AI applies them to every transcript and the sheet becomes the record of the human decision. When the interview sits beside a skills test from the test library, keep the two scores side by side rather than blending them; the one-way interview page shows where the rubric is set.
