S04.2 — Fine-tune DistilBERT and produce validation predictions

Planning / historicaltaskdelivery

Historical

Planning material. It summarizes and links to the authoritative specification and the current measured state — it does not replace either, and it is not current engineering truth.

Migration identity#

Canonical identifier: S04.2 (GitHub issue type: subtask).
Legacy identifier: C04.2 (retained as old_identifier).
Parent epic: E04.
Milestone: M1 — IntentGuard Weekend MVP.

Objective#

Run deterministic fine-tuning on training data and produce persisted validation probabilities without accessing test labels.

Rationale#

The improved model and threshold input must be produced by a leak-free, reproducible lifecycle.

Parent identifier#

E04.

Source task#

T-004.

Traceability#

Primary: FR-003, NFR-001. Secondary: FR-004, AC-003.

Prerequisites#

C04.1 passes; training configuration, seed behavior, and checkpoint rule are frozen.

Likely files#

src/intentguard/training.py, scripts/train_transformer.py, configs/default.toml, tests/integration/test_training_smoke.py.

Implementation boundary#

Train and validation inference only. The test split and test labels are inaccessible in this command path.

MUST scope#

Tokenization, seeded training, validation checkpointing, 77-class probabilities, run provenance, and failure cleanup.

Explicit non-goals#

Hyperparameter sweep, other models, test evaluation, serving, or calibration extras.

Acceptance criteria#

Training produces finite loss, complete validation predictions in canonical order, exact run configuration, and no test-split dependency.

Validation commands#

make train && uv run pytest tests/integration/test_training_smoke.py -q.

Expected evidence#

Training log, validation prediction artifact, seed/environment/revision metadata, and leakage boundary test.

Fallback and status consequence#

One epoch may be Implemented if genuine fine-tuning completes and measured claims are honest. Frozen embeddings is Partial and fails strict MVP.

Stop condition#

Stop for test access, non-finite training, nondeterministic labels/order, incomplete predictions, or unexplained interruption.

Definition of ready#

Smoke path passes and all training choices are recorded before execution.

Definition of done#

Fine-tuned weights and complete validation predictions exist with reproducible provenance.

Labels#

type:subtask, priority:MUST, area:model

Estimate#

1.25 hours excluding training wall time.