Across 30,384 résumé evaluations, a peer-reviewed bias audit put the gender effect at d=−0.00 (p=0.97) and the Black–White contrast at d=+0.03 (p=0.29). Not "not significant" — formally equivalent to zero, with the gender null holding down to a bound of |d|=0.035 (Pavlopoulos, Analytics, 2026). Then the same study replaced the scoring rubric with a casual, gut-feel instruction, held candidate experience exactly constant, and a d=0.41 preference for older graduation cohorts walked back in.
The model was clean. The prompt was not. And your bias audit tested the model.
That gap is the whole operational story, because the artefact your vendor hands you certifies an object that is not the one making your screening decisions. The decision is made by a model plus an instruction plus a context window, and only one of those three gets audited.
The Audit Was Built to Break Its Own Null
Most LLM fairness studies test one condition and declare a verdict. This one treats robustness as the object of study, which is why its null is worth reading rather than filing.
Three instruction-tuned models were scored through their APIs: Gemini 2.5 Flash, Gemini 2.5 Flash-Lite, and the open-weight Llama 3.1 8B Instruct — a deliberate spread across hosted commercial tiers and the self-hosted path. Scoring was reference-anchored rather than absolute, because absolute 0–100 scoring collapsed the variance needed to detect anything at all: in pilot runs nearly every résumé landed near 95. Each candidate was instead rated against a fixed neutral reference for the same occupation, which restored a 12-to-18-point spread. Sampling ran at temperature 0.7 with replicate evaluations per cell, so effects were measured against real within-cell variability rather than one deterministic output.
Three design choices make the result hard to wave away.
A positive control that worked. All three models separated mid-level from junior candidates at d=3.40, 3.21 and 1.93. The instrument demonstrably reads quality, so a demographic null cannot be dismissed as a dead measure. The seniority effects exceeded the demographic ones by roughly two orders of magnitude.
Equivalence testing, not absence of significance. Every core contrast rejected the TOST null at the |d|=0.2 bound at p<0.001, with achieved margins of 0.035 for gender, 0.056 for Asian–White, and 0.080 and 0.085 for Black–White and Hispanic–White. The seniority control returned TOST p=1.000, confirming the procedure does not certify effects that are genuinely large.
Deliberate provocation. Study 3 weakened the candidates, relaxed the prompt, and prepended culture-fit language about a close-knit, high-energy team. The manipulation bit: borderline résumés scored 32.5 against 67.6 for clean ones, a 35-point drop. That is exactly the regime where a decision-maker is theoretically most likely to substitute a heuristic for an assessment. Race and gender effects stayed negligible there too.
One small real effect surfaced: candidates from non-elite institutions scored slightly lower, d=−0.05 (p=0.026), left marginal by false-discovery control at q=0.052. Worth noting, not worth a policy.
What Came Back Was the Shortcut, Not the Prejudice
The headline finding in the main design was an apparent age effect of d=0.28. The author did the thing most papers skip and re-ran it with the confound removed — because the older-graduate résumés had also carried extra experience, conflating age with tenure.
Under the structured rubric, with experience held identical, the effect collapsed to d=0.02 (p=0.90). Under the casual instruction, it survived at d=0.41 (p=0.03).
Read that pair slowly. A rubric forced the model to ground its judgement in stated qualifications that were identical across age levels, so the heuristic had nowhere to act. A casual instruction invited a holistic impression, and length of career stood in for quality. Same model, same résumés, same candidates — opposite behaviour, decided by the wording of the request. And notably, the casual prompt did not revive race or gender effects, so this was a seniority shortcut rather than a general loosening of restraint.
The open-weight model carried its own quieter pattern. Llama's largest shifts came from explicit minority signals: foreign credentials from the Global South at d=−0.16, a Muslim religious affiliation at d=−0.13, a disclosed disability at d=−0.12. None survived false-discovery correction, so they are suggestive rather than confirmed. Their consistent negative direction and concentration in the smallest, self-hosted model is still the kind of thing you would want to know before standardising on it.
A Second Study Found 12% Gaps. Both Results Can Be True.
If this study were the only evidence, the conclusion would be comfortable. It is not.
An interpretability-focused audit tested leading commercial and open models — GPT-4o, Claude 4 Sonnet, Gemini 2.5 Flash, Gemma-2 27B, Gemma-3, Mistral-24B — and found that adding realistic context induced significant racial and gender bias, up to 12 percentage-point differences in interview rates (Karvonen and Marks, 2025). The context was mundane: company names, culture descriptions lifted from public careers pages, and selective hiring constraints such as accepting only the top 10% of candidates. Two further findings should stop a compliance conversation cold. Simple anti-bias prompt instructions, which eliminate bias in controlled evaluations, failed once that realistic context was present. And models inferred demographics from subtle cues like college affiliation, with the resulting bias invisible even on inspection of the chain-of-thought.
So one careful audit finds a defended null under provocation, and another finds double-digit interview-rate gaps under deployment-realistic framing. The reconciliation is not that one is wrong. It is that both located the variance in the same place — the surrounding instruction, not the model's demographic priors. One study's framing suppressed the shortcut; the other's admitted it. Neither result transfers to your pipeline, because your pipeline has its own prompt.
That is the uncomfortable part. A clean audit result is not portable to a different instruction.
Human Screening Obeys the Same Law
This is not a machine pathology. It is what discretion does, and the human evidence points the same way.
A résumé audit sent 36,880 applications to 9,220 US job advertisements for new college graduates. In management occupations, callbacks ran 28 to 43 percent lower for Black men, Black women, White women and Hispanic men than for otherwise identical White men (Braun et al., 2026). The structural result matters more than the headline: broad occupation categories concealed the variation, and gaps concentrated in jobs combining high analytical and interpersonal demands with low routine content. Decomposing task content, subjective evaluation widened callback gaps while objective precision compressed them. Most pointedly, randomly assigned credentials that lifted callbacks overall closed gaps in low-discretion jobs but not in high-discretion ones.
Candidate-side signalling cannot repair an employer-side discretion problem. Only constraining the evaluation does that. It is a working paper under review and it measures the first gate rather than hires — but the mechanism is the same one the LLM audit isolated. Discretion carries bias. Structure removes its room to act. The prompt is simply where discretion lives when the screener is a model.
Your Bias Audit Is Certifying the Wrong Object
Now put the compliance artefact next to that mechanism.
New York City's Local Law 144 prohibits using an automated employment decision tool unless "the tool has been subject to a bias audit within one year of the use of the tool, information about the bias audit is publicly available, and certain notices have been provided" (NYC Department of Consumer and Worker Protection, 2026). The audited object is the tool.
European obligations fall on the other party. Deployer duties for Annex III high-risk systems — hiring and candidate evaluation among them — require use in accordance with the provider's instructions for use, and under the amended timeline they apply from 2 December 2027 (EU AI Act Explorer, Article 113). The provider certifies the system. You own how it is actually used.
Between those two sits the finding of this research: the thing that decides whether bias appears is neither the model's weights nor your intentions, but the instruction wording, the framing text and the temperature at which it all runs. Nobody is auditing that. It usually is not even versioned.
What to Change This Quarter
Four moves. None requires a procurement cycle.
- Version the scoring prompt as an instrument, not configuration. Put it under change control with an owner and a date. Any edit — including a "harmless" rewording — is a change to the measuring device and invalidates the prior audit result for that device.
- Delete ambient culture-fit language from scoring instructions. It functions as an unmeasured criterion. If cultural context genuinely matters for a role, make it an explicit, separately scored dimension with a written reason, not a paragraph of atmosphere prepended to the request.
- Re-test on prompt change, not on calendar. An annual audit against a prompt you have since edited eleven times certifies nothing. Re-run your fairness check as a regression test whenever the instrument changes, and pin the sampling temperature while you are at it.
- Ask two questions at renewal. Which model generation and weights were audited, and against which prompt? The open-weight model in this study behaved differently from the commercial ones on exactly the signals a casual prompt surfaces, and no audit report names your prompt.
Structurally, all four moves point the same way: shrink the share of the decision that an unversioned instruction controls. Scovai's scoring architecture is one worked example of that separation — a fixed weight budget split 50/50, half computed arithmetically from the role's stated requirements under criticality weights, half returned as four labelled model-judgement subscores at low temperature, each with its own written reason rather than buried in a total. The design point worth borrowing is not the split itself but what it does: it names which part of the score is judgement, so that part can be audited as the instrument it is.
Honest limits before anyone builds policy on the null. The résumés were synthetic, the design was US-centred in names, institutions and occupations, six occupations were covered and legal, public-sector and service roles were not. The comparison design measures relative judgements, so it may miss bias acting only at an accept-or-reject threshold. The suggestive open-weight effects need larger samples. Two of the three models share the Gemini family. And it is a snapshot of three models at one moment — the author's own closing position is that auditing has to continue as systems are updated. The study received no external funding and declares no conflicts.
The conclusion that survives all of it: fairness measured on a clean benchmark does not guarantee fairness in deployment, because how a model is prompted can decide whether bias appears.
So pull your live screening prompt this week, read it as the measuring instrument it is, and check one thing — whether the bias audit in your compliance folder was run against this wording or an earlier one. If you cannot answer that, you do not have a bias audit. You have a certificate for a device you have since modified.
