S04.3 — Select the validation threshold and finalize the immutable artifact

Planning / historicaltaskdelivery

Historical

Planning material. It summarizes and links to the authoritative specification and the current measured state — it does not replace either, and it is not current engineering truth.

Migration identity#

Canonical identifier: S04.3 (GitHub issue type: subtask).
Legacy identifier: C04.3 (retained as old_identifier).
Parent epic: E04.
Milestone: M1 — IntentGuard Weekend MVP.

Objective#

Choose confidence threshold from validation predictions only, persist it with complete transformer assets/provenance, and prove reload parity.

Rationale#

One immutable deployable unit prevents evaluation/serving drift and makes leakage auditable.

Parent identifier#

E04.

Source task#

T-004.

Traceability#

Primary: FR-004, FR-008, AC-003, AC-005, AC-010. Secondary: NFR-008.

Prerequisites#

C04.2 validation predictions complete; threshold objective and tie-breaking rule frozen.

Likely files#

src/intentguard/threshold.py, src/intentguard/artifacts.py, scripts/train_transformer.py, tests/unit/test_threshold.py, tests/unit/test_artifacts.py.

Implementation boundary#

Threshold selection belongs only to training and validation; final evaluation can load but never modify it.

MUST scope#

Threshold algorithm, deterministic tie break, validation provenance, tokenizer/weights/config/labels, checksums, completion marker, immutability, reload parity.

Explicit non-goals#

Test-label selection, per-class thresholds, temperature scaling, mutable artifact repair, or serving behavior.

Acceptance criteria#

Artifact is complete and uniquely identified; threshold is reproducible from stored validation evidence; no test input is accepted; reload matches labels/probabilities/decision within declared tolerance.

Validation commands#

make train && uv run pytest tests/unit/test_threshold.py tests/unit/test_artifacts.py -q.

Expected evidence#

Threshold value/objective/tie rule, validation fingerprint, immutable manifest/checksums, and reload-parity output.

Fallback and status consequence#

Hard-coded or test-selected threshold is Partial/Mocked and fails strict MVP; incomplete artifact is Blocked.

Stop condition#

Stop for leakage, ambiguous ties, missing assets/provenance, post-completion mutation, or reload divergence.

Definition of ready#

Validation predictions and selection rule are approved; artifact schema and tolerances are resolved.

Definition of done#

AC-003, AC-005, and AC-010 pass for the same immutable artifact.

Labels#

type:subtask, priority:MUST, area:model

Estimate#

1.25 hours.