Beyond undocumented thresholds: a six-layer justification stack

Executive overview

Problem: Clinical AI systems depend on numeric thresholds (performance targets, alert cut-offs, deployment gates) whose evidentiary justification is often thin—“magic numbers” embedded in models without traceable documentation.

Why it matters: Undocumented thresholds are hard to audit, hard to defend after incidents, and easy to game. A structured documentation stack makes expectations explicit for safety leaders and committee review.

What was done

The repository renders manuscript Tables 1–3 into structured CSV from JSON sources: six failure mechanisms, the six-layer Threshold Justification Stack with tiered requirements (primary safety vs secondary operational), and an interpretive comparison to other governance instruments. Supplementary pathways cover gaming resistance sketches and NHS governance artefact mapping tables; illustrative TJS records demonstrate schema completion for both tiers.

What was found

The artefacts make the epistemic status of each table explicit—what is empirical versus interpretive—and separate in-repo reproduction scope from companion empirical papers (see boundary statement in the repository). Deterministic notebook outputs are hash-validated against expected manifests in QA.

Why this matters for regulation, safety, and deployment

• Committees: receive a checklist-like stack for what must be documented before thresholds are accepted.
• Integration: maps TJS layers to existing hospital risk artefacts to reduce duplicate paperwork while preserving safety intent.
• Integrity: gaming pathways are named so assessments are proactive rather than purely reactive.

Limitations and ethics

Framework comparisons are author interpretive codings, not regulators’ official positions. Feasibility pilots, inter-rater reliability, and some empirical validations are explicitly required before scalability claims (P3-C14–C15) and sit outside this repository’s numerical scope per docs/claim_boundary_statement.md.