Technical interview questions that actually discriminate

A useful technical interview question separates candidates who can do the job from candidates who cannot. That sounds obvious and it rules out most published questions, because a question everyone has seen sorts by preparation rather than by ability.

The test is discrimination, in the statistical sense: does the answer distribution actually spread? A question every strong candidate answers well and every weak one answers badly is worth an hour of design. A question everyone answers identically tells you nothing, however technical it sounds.

Two consequences follow immediately. Questions with a single known correct answer decay as they circulate. And the most informative question is usually the second one — the follow-up you could not have planned, because it depends on what they just said.

The standard technical question set was designed when the constraint was that good answers were expensive to produce. Preparation took months, the questions circulated slowly, and a strong answer was reasonable evidence of a strong candidate.

Both halves of that have gone. Question banks are indexed and searchable, preparation is a well-understood genre with its own industry, and any question with a canonical answer now has that answer available instantly. A polished response to a famous question is evidence that the candidate prepared, which is worth something, and it is not what you were trying to measure.

This is the same structural problem as the unproctored take-home, arriving through a different door — when candidates use AI on your assessment covers why detection is the wrong response to it and what still holds.

Most technical questions are one of four kinds. They are not interchangeable, and three of the four are routinely used for the thing the fourth does better.

The bottom two rows are where the signal is, and they are the two that need the most preparation from the interviewer. That asymmetry is why the top two remain popular: recall and puzzle questions score themselves.

A prepared answer survives one question. It rarely survives the obvious next one, because preparation covers the answer and not the reasoning underneath it.

The follow-ups that do the most work are unglamorous and they are the same four almost every time. What did you consider and reject? What would have changed your mind? What broke? What would you do differently now? None can be pre-baked, because each depends on the specific thing the candidate just claimed.

This is also the argument against every asynchronous format for the deciding round. A recorded answer to a fixed prompt is a rehearsed answer by construction — the candidate can re-record until it is smooth, which measures editing. A live conversation is the format that can ask the second question at all.

What discriminates differs sharply by role, and in each of these the thing worth asking about is not the thing most interviews cover:

• DevOps and SRE — ask about an incident they owned end to end. The clause that matters is what they changed so it could not recur.
• Data engineering — ask about the conversation with the people who wanted the data, not the pipeline.
• Engineering management — ask for the hard conversation they got wrong, and what no interview will tell you.

Interview scoring is where structure is most often abandoned, because the format feels conversational and writing things down feels rude. The result is a decision made on recall of a feeling, hours later.

• Decide the competencies before the first interview. Three or four, with a sentence describing weak and strong for each.
• Ask the same opening questions of everyone. Follow-ups vary by necessity — that is the point of them — but the starting question should not.
• Write evidence, not a verdict. One line per competency saying what they said that produced the score.
• Score immediately, before discussing with anyone. The first opinion voiced in a debrief reshapes every opinion after it, and the effect is strong enough to make a panel less accurate than its members.

For the wider framework this sits inside — which method suits which role, and where interviews are the wrong instrument entirely — see what candidate assessment is and how to test technical skill.

An interview measures how someone performs in a conversation with a stranger who is judging them. That is a real skill for some roles and unlike most of the actual work for all of them. Quiet under that pressure is not the same as quiet in a team they know.

It also cannot verify that anything described actually happened, and it rewards articulacy in a way that penalises people who do excellent work and describe it plainly. That is why an interview belongs next to a work sample rather than instead of one: the sample establishes what they can do, and the conversation establishes whether they understand why they did it.