Before sending the coding test, write the scoring rule

A TypeScript invitation with four answers per question prompted my original note about hiring tests. The format was easy to recognize. The connection between the score and the work needed more explanation.

A hiring team should be able to describe the decision a test will support before sending it to a candidate. For a developer role, that might be whether someone can diagnose a failure in unfamiliar code. A score becomes useful when the team knows what the observed behavior says about that requirement.

That leaves room for several assessment formats. It also gives the candidate a reasonable explanation for the time being requested.

Begin with the work expected on arrival

The US Office of Personnel Management describes work samples as exercises that resemble tasks employees perform. Its guidance includes a condition that is easy to overlook: the assessed competence should be expected when someone enters the position. A task that the employer intends to teach after hiring may be a poor choice for that assessment. OPM's work-sample guidance

For a hypothetical TypeScript maintenance role, a small repository could contain a reproducible bug. The exercise could ask the applicant to explain the failure before proposing a change. A reviewer would then have something concrete to examine: whether the explanation accounts for the reported behavior.

The repository should contain enough context to make the task solvable within its stated scope. Missing setup instructions would introduce another demand. If diagnosing a broken development environment belongs to the role, that could be intentional. Otherwise, the setup failure would distract from the competence the team meant to observe.

A multiple-choice question can also have a defined purpose. Recognizing a language rule may be relevant to a particular task. The employer still needs to explain how performance on that question contributes to the hiring decision. Replacing every quiz with a repository would leave the same problem unresolved if nobody had designed the scoring.

Write the scoring rule before seeing the answer

In that maintenance exercise, a reviewer could distinguish identifying the cause from merely changing the visible symptom. A patch that makes one example pass might leave the underlying behavior unchanged. The scoring instructions need to tell reviewers how to handle that difference.

The team should also decide how much weight to give the explanation. If the role requires discussing a change with colleagues, an unexplained patch leaves part of the requirement unobserved. If the exercise only measures a narrow technical operation, a long presentation may add an unrelated demand.

This is where an attractive assignment can fail. A realistic-looking task still leaves room for arbitrary judgment when one reviewer rewards a familiar coding style while another rewards the smallest diff. Writing the criteria first gives the team a way to resolve that disagreement before a candidate's answer is involved.

The instructions can describe acceptable alternatives without publishing a single model solution. Two implementations may satisfy the same requirement. Scoring should follow the stated requirement closely enough that either implementation can earn the appropriate result.

Give the conversation some structure

OPM's structured-interview guidance describes using predefined questions with proficiency benchmarks. It connects the questions to competencies identified through job analysis. That provides a basis for comparing responses to the same requirement. OPM's structured-interview guidance

A follow-up on the hypothetical bug could ask the candidate what evidence would make them reject their first diagnosis. The hiring team would need an agreed way to judge that response. Asking the question consistently gives each applicant a comparable opportunity to explain their reasoning.

The interviewer can still listen closely. Structure sets expectations for what will be assessed; it leaves room to understand the answer being given. Follow-up questions need boundaries so one applicant doesn't receive substantially more help with the assessed task.

Keep the research attached to its assumptions

The familiar rankings of hiring methods have been revised. Sackett and colleagues' 2023 paper explains how their earlier reanalysis changed validity estimates after revisiting statistical corrections. Structured interviews had the highest mean validity in that comparison. The authors also discuss variation between settings, which a single average can hide. The authors' explanation of the revised evidence

That finding supports taking interview design seriously. It leaves the performance of a particular employer's interview to be evaluated. Giving an informal conversation a structured label supplies no evidence about its questions or scoring.

Current employer instructions also show why broad claims about one format replacing another are risky. NIST's hiring guidance, updated in January 2026, describes general assessments followed by additional methods for some positions. Those can include structured interviews or job-related exercises. That is one institution's process, with a defined scope. NIST's January 2026 guidance

Make the candidate's commitment explicit

The invitation should state the expected time before the applicant accepts. If the assignment grows beyond that scope, the team needs to reduce it or agree on the additional work. Paying for a substantial assignment is a reasonable hiring policy; the appropriate limit depends on what is being requested.

The same invitation should explain the permitted use of AI tools. A candidate who is allowed to use an assistant needs to know which parts of their own work will be assessed. A later conversation can examine how they checked the generated change, using criteria disclosed in advance.

At the end, the reviewer should be able to connect the decision to the requirement introduced at the beginning. A candidate who receives a rejection may disagree with the assessment. They should still be able to understand what was assessed and how their submitted work was judged.