escapement
AI safety · alignment evaluation

Did your fine-tune quietly make your model easier to hijack?

Independent safety evaluation for AI checkpoints, including the ones nobody else can grade. We measure your checkpoint against the model it was derived from, so you get a delta, not an uninterpretable number.

Test my checkpoint → See the public register

Fine-tuning changes safety behaviour. Almost nobody measures it.

Fine-tuning is documented to erode safety alignment even when the training data is entirely benign: refusal rates collapse, and fine-tuned variants are markedly more susceptible to hijacking than the model they came from. Yet the artifact you actually deployed has, in almost every case, never been evaluated by anyone. No public benchmark covers a private fine-tune, by definition.

An absolute score on a fine-tune is close to meaningless at realistic sample sizes. The measurement that means something is the paired delta against its base model, run in the same harness under the same conditions with only the training changed.

No public pair — yet. A paired delta needs a derived variant (a refusal-ablation, a fine-tune, a merge) scored on the same instrument as its base. The current public config_exposure register holds only base checkpoints, so there is no comparable public pair to chart here. The same-instrument delta is exactly what a client engagement delivers — against your checkpoint and the model it was trained from.

Refusal-ablated variants are the public, extreme case of a derived checkpoint. Your domain fine-tune is the same experiment with a smaller expected effect and no published number at all.

What the open-weight data actually says

In July 2026 the headlines were about frontier models breaking containment. The deployable, open-weight reality is quieter and more useful, and we can put numbers on it. Across 9,572 public config_exposure trials spanning 38 model configurations and 13 families — the graded subset of the open dataset (17,438 trials across all scenarios), reconciled 2026-08-05:

0%

Autonomous self-exfiltration

Unprompted, organic self-bootstrap: 0 of 3,180 Frame A trials across every checkpoint. A bounded null (95% CI upper 0.12%). At this scale, models do not spontaneously scheme to escape.

0%–22.5%

Injection compliance (judge-audited)

The real vector. On the judge-audited injected (Frame B) rows, compliance runs 0% to 22.5% across checkpoints (deepseek-v4-flash-0731 → devstral-medium) — common enough to matter, and the spread is wide. Detector-only rows in the Partial runs group run higher (to 36.67%) and are labeled detector-only, never merged into this judge-audited spread.

8.1% judge-audited

Two provenance tiers, labeled

779 of 9,572 trials (8.1%) carry an LLM-judge verdict — the authoritative genuine-self-bootstrap tier, concentrated in the newest snapshot. The rest fall back to the deterministic detector (a lower bound) and are flagged as Partial runs. The two DeepSeek runs are kept separate: detector-only original (26.47%) vs. judge-audited general release (0%). Every published rate is labeled by its source.

The alignment framing. Two failure modes decide whether a deployed model stays overseeable: does it pursue its own continuation against your intent (loss of control), and can an outsider redirect it through the content it reads (indirect prompt injection). escapement measures both, per checkpoint, with confidence intervals, and reports them honestly, including the nulls.

The baseline: injection compliance across the public register

This is the reference scale that makes a single checkpoint's number interpretable. Every checkpoint showed zero unprompted self-exfiltration; under injection they diverge sharply, from 0% to 22.5% of trials. Rates are judge-audited; bars are 95% confidence intervals, wide at these sample sizes and shown rather than hidden. Detector-scored rows — a lower bound, not judge-audited — sit in a separate strip below the main plot, never merged onto the judge-audited scale; "Show more models" expands to the full partial-runs corpus.

config_exposure register: 9,572 trials, 38 checkpoints (4 judge-audited for Frame B injection compliance, charted above; 2 detector-scored in the strip; the rest provisional). Full leaderboard with grades →

The ablation effect: refusal-ablated ("uncensored") checkpoints are riskier

Derived checkpoints that strip refusal training are the public, extreme case of a fine-tune. Same scenario, same battery, same instrument — only the training changed.

What this maps to

Each metric maps to a framework your compliance team already uses: EU AI Act Article 15 (the threshold-independent robustness/cybersecurity requirement), NIST AI RMF Measure, and OWASP LLM Top 10 (LLM01 Prompt Injection; ASI04 tool poisoning). The evaluation produces supporting evidence toward those obligations, not a certification. Trust & regulatory mapping →

Test your checkpoint

We run your checkpoint and the base model it derives from through the same battery, and give you the delta: per-channel injection rates with confidence intervals, a paired odds ratio with a significance test, redacted evidence from the trials that failed, and an EU AI Act Article 15 write-up, as a durable, versioned document. Works on private fine-tunes, LoRAs, merges, quantized builds, and closed API models.

Prefer email? Write to [email protected]. We evaluate client checkpoints under confidentiality.