A workshop receives free-text requests. We want a JSON record that copies the information present, chooses a category from defined categories and leaves absent information as null. There is no access to a document system and no intervention is executed.
From the same ticket, show the model output with a prompt alone, after full SFT and after adaptation limited to its final two blocks. Parse the JSON and check the fields without constrained decoding to isolate the effect of training.
Download the complete project, revision r002
A good output defined by the data
SFT maximizes the likelihood of an expected response conditioned on a prompt. It is not about having the model reread raw tickets: ticket/record pairs teach a precise transformation here. The Qwen2.5-0.5B base model already has pretrained weights; this TP does not claim to learn a language from scratch.
InterventionPairs contains 400 families of six phrasings, or 2,400 pairs. Families 0–279 provide 1,680 training examples; 280–339 provide 360 validation examples; 340–399 provide 360 test examples. Similar phrasings from one family remain together. Opening/closing combinations and unit identities distinguish families; symptom, location and sentence-form vocabulary remains shared. Normalized hashes check duplicates after normalizing spaces and case, not a universal absence of semantic overlap.
The fields are exactly unit, symptom, location and scheduled_date. Their values are strings or null; a non-null symptom must be one of noise, leak, heat or vibration. The schema checks types and keys, not whether a string is true or fits a calendar. An incorrect fictional date can therefore satisfy the schema and fail field evaluation. Business equality concerns exact values, not JSON key order.
Targets recover no information absent from the ticket: an absence clause replaces the relevant fact. Dates are absent in two of six phrasings; other cases remove a location, unit or symptom. The actual proportion in every split is recorded and must exceed one quarter. Eight demonstrations are selected by training identifiers fixed in the generator before observing any test result.
Response boundary and loss
The local format is an instruction, Ticket: ... then JSON:. We use no implicit conversation template. The fast tokenizer comes from the frozen base. The contract encodes the prompt and JSON response separately, concatenates their token lists and then adds EOS=151643. This construction is not presented as identical to retokenizing a concatenated string: the boundary is explicit and tested with accents and empty JSON.
Prompt and padding labels are −100. Every response token and EOS has a target. To predict the next token, logits[:, :-1] are aligned with labels[:, 1:]. The sequence's first token has no predecessor; it is never a target in this calculation, including in the ablation that supervises the prompt.
PAD and EOS use the same identifier. The mask must therefore depend on real positions rather than equality with that identifier: ignoring every 151643 would also remove the stopping signal. No sequence is packed with another. SFT/zero-shot prompts are limited to 384 tokens; responses to 96, including EOS. An empty response or overly long input raises an error with the case identifier; nothing is silently truncated.
Cross-entropy is coded as a sum over active positions divided by their exact number. For eight accumulated examples, every microbatch uses the total denominator of those eight examples even when their responses have different lengths. An average of averages would change each token's weight. The decoder and projection to the vocabulary compute the full sequence, then the mask selects supervised positions for cross-entropy. Keeping the same projection shape as Transformers avoids an FP32 rounding difference from a matrix product on a subset of rows. A test compares selected logits and loss with Transformers at the initial FP32 tolerances; prompt and padding remain excluded from supervision.
Which parameters can change?
The base is Qwen/Qwen2.5-0.5B, revision 060db6499f32faf8b98477b0a26969ef7d8b9987. The acquired architecture is checked: width 896, 24 blocks, 14 query heads, two K/V heads, MLP 4864, vocabulary 151936 and native context 32768. Input embeddings and output projection share the same Parameter. The program computes the exact number of unique parameters.
Full makes every parameter trainable. Partial makes only blocks 22 and 23 trainable; embeddings, tied output, final normalization and other blocks remain frozen. The selection walks the architecture's real objects and checks names, identities and no duplicate in the optimizer. Before/after hashes must preserve every frozen parameter while permitted parameters must actually change. Making a parameter trainable does not guarantee that every tensor changes numerically at every step.
Both branches start from exactly the same base, in FP32 with eager attention and TF32 disabled. They use the same data, draw order, two epochs and quantities of supervised response tokens. AdamW, weight_decay 0.01, microbatch 1, accumulation of eight examples, clipping at 1, warm-up of 42 updates then linear decay to 10% of the initial rate complete the fixed configuration.
Response-only validation is measured after every epoch, with no best-checkpoint selection. Resumption happens at an epoch boundary after one interrupted epoch. It restores weights, AdamW, scheduler state, Python/NumPy/PyTorch CPU/CUDA RNG, sampler and history. A continuous two-epoch run and a resumed run must give exactly the same states.
The .pt states are created locally, written to a temporary file then atomically renamed; they must not be replaced by untrusted files. The final model is also saved as Safetensors with handling for shared embedding/output, then reloaded into an isolated instance. Files and parameters are checked with SHA-256.
Baselines and unconstrained generation
The base model is evaluated zero-shot and with eight fixed demonstrations. These eight examples require a separate context budget of 2,048 tokens: they are not artificially forced into the other prompts' 384 tokens. Their actual cost is recorded. Generation reserves 96 new tokens in the native context, without prompt truncation.
Every branch uses greedy generation, one beam and EOS=151643 stopping, with a maximum of 96 new tokens. No schema is enforced during decoding. Input/output IDs, raw text, stopping reason, token counts and duration are retained for each of the 360 validation and 360 test cases. This step does not adjust weights.
The parser distinguishes three properties: parsable JSON text, valid schema, then accepted complete response. A response stopped at the limit can contain well-formed JSON; it remains rejected as a complete response. No Markdown-tag removal, field addition, repair or enrichment is applied. Duplicate keys and non-finite constants are rejected.
Global exact match and accuracy of every field use all 360 test cases. An invalid or truncated output fails every field. For absent information, the report distinguishes correct null, an invented non-null value in a valid complete response, and an invalid output. The invented-value rate uses every occasion where the field is missing; the failure rate for null also includes invalid outputs. A model that produces no JSON cannot therefore look reliable through an artificially zero invention rate.
Test response-only NLL is recalculated with the same zero-shot prompt for the base, full, partial and ablation. It includes EOS and the same targets. Few-shot generation is a separate comparison with more context; its quality and tokens are reported without confusing that cost with a learning effect.
Exercise: include the prompt in the loss
A third full branch genuinely starts again from the base for two epochs, with the same document order and full configuration, but also supervises prompt tokens. It therefore counts more active tokens. A drop in its training loss cannot be compared directly with the response-only average.
Its NLL is recalculated on the same test responses and its 720 validation/test generations, then exact match and absent fields are compared. No gain or regression is required. Main models and results are protected by SHA before the exercise. The response mask remains the reference configuration; the ablation has a separate name.
Limits
These tickets are synthetic and regular. Their labels reflect a generator, not the variety of real requests. Conforming JSON can contain false information. A single training seed cannot estimate training variability. The rules that produce targets are a teaching oracle, not a model discovery. No document repository or intervention act is created or executed.
Define the task, schema and families
Supervised demonstration
Common error: the generator writes a target date absent from the ticket; test every field against the rendered facts.
Invariant: 1,680/360/360 pairs and absence proportions audited before any exclusions.
File src/sft_complet/data.py
import hashlib, json, re
from pathlib import Path
from .config import save_json, sha
FIELDS = ["unit", "symptom", "location", "scheduled_date"]
SYMPTOMS = ["noise", "leak", "heat", "vibration"]
LOCATIONS = [
"inlet",
"outlet",
"motor bay",
"upper housing",
"control panel",
"rear cover",
"lower bracket",
"supply line",
"return line",
"service hatch",
]
OPENINGS = [
"A technician submitted this observation.",
"The operator left a short report.",
"The shift log contains this request.",
"A maintenance note describes the following.",
"The service team received this message.",
"An internal ticket records these details.",
"The workshop inbox received a report.",
"A routine handover included this note.",
"The operator asks for a review.",
"The inspection desk received this request.",
"The shift coordinator recorded a concern.",
"A service request includes these facts.",
"The team copied this observation into a ticket.",
"The workshop clerk transcribed this report.",
"A technician added the following account.",
"The afternoon handover contains this message.",
"The maintenance inbox lists this observation.",
"The service desk forwarded this written request.",
"An operator documented the following concern.",
"A new workshop note contains these details.",
]
CLOSINGS = [
"Please retain the original observations.",
"The note contains no other confirmed details.",
"This is a request for review only.",
"No repair has been performed by this note.",
"The text is an observation, not an instruction to run equipment.",
"The report remains available for the next shift.",
"Keep unreported details unspecified.",
"The service desk can review the record.",
"This message does not confirm a diagnosis.",
"The observation is awaiting a technical review.",
"Only explicitly reported facts should be copied.",
"The ticket has not authorized any intervention.",
"Retain uncertainty when details are missing.",
"No additional findings are supplied here.",
"The report is ready for manual review.",
"This entry is a written observation only.",
"The workshop has supplied no supplementary facts.",
"The request can be filed for later inspection.",
"The maintenance team should read the original note.",
"The text does not provide a cause for the concern.",
]
ORDERS = [
["unit", "symptom", "location", "scheduled_date"],
["location", "unit", "scheduled_date", "symptom"],
["scheduled_date", "symptom", "unit", "location"],
["symptom", "location", "unit", "scheduled_date"],
["unit", "scheduled_date", "location", "symptom"],
["location", "symptom", "scheduled_date", "unit"],
]
MISSING = {
"unit": "The unit identifier was not provided.",
"symptom": "The symptom was not specified.",
"location": "The location was not stated.",
"scheduled_date": "No inspection date has been scheduled.",
}
def split_scenarios(family):
return "train" if family < 280 else "validation" if family < 340 else "test"
def normalized(text):
return " ".join(text.casefold().split())
def render_target(values):
return json.dumps(
{key: values[key] for key in FIELDS}, ensure_ascii=False, separators=(",", ":")
)
def render_fact(key, value, j):
if value is None:
return MISSING[key]
alternatives = {
"unit": [
f"The unit is {value}.",
f"Unit {value} is named in the report.",
f"The reported unit identifier is {value}.",
f"This observation concerns unit {value}.",
f"The unit was identified as {value}.",
f"The record names unit {value}.",
],
"symptom": [
f"The reported symptom is {value}.",
f"The operator reports {value}.",
f"The symptom is described as {value}.",
f"The note explicitly reports {value}.",
f"The observed symptom is {value}.",
f"The report describes {value}.",
],
"location": [
f"The location is {value}.",
f"The issue was reported near the {value}.",
f"The stated location is the {value}.",
f"The observation was made at the {value}.",
f"The report points to the {value}.",
f"The location named is the {value}.",
],
"scheduled_date": [
f"The inspection is scheduled for {value}.",
f"The scheduled inspection date is {value}.",
f"An inspection has been scheduled on {value}.",
f"The planned inspection date is {value}.",
f"The inspection appointment is set for {value}.",
f"The record gives {value} as the inspection date.",
],
}
return alternatives[key][j]
def generate_tickets():
pages = []
gold = []
templates = []
for family in range(400):
opening = OPENINGS[family // 20]
closing = CLOSINGS[family % 20]
templates.append(
{
"family": family,
"split": split_scenarios(family),
"opening": opening,
"closing": closing,
}
)
for j in range(6):
values = {
"unit": f"U{family+1000}",
"symptom": SYMPTOMS[family % 4],
"location": LOCATIONS[(family // 4) % 10],
"scheduled_date": f"2026-{family%12+1:02d}-{family//12%27+1:02d}",
}
if j in [0, 3]:
values["scheduled_date"] = None
if j == 4 and family % 3 == 0:
values["location"] = None
if j == 5 and family % 4 == 0:
values["unit"] = None
if j == 2 and family % 7 == 0:
values["symptom"] = None
clauses = [render_fact(key, values[key], j) for key in ORDERS[j]]
ticket = " ".join([opening, *clauses, closing])
identifier = f"I{family:03d}-{j}"
row = {
"document_id": identifier,
"family": family,
"formulation": j,
"split": split_scenarios(family),
"ticket": ticket,
"content_sha256": hashlib.sha256(ticket.encode()).hexdigest(),
"normalized_sha256": hashlib.sha256(
normalized(ticket).encode()
).hexdigest(),
}
pages.append(row)
gold.append(
{
"document_id": identifier,
"target": values,
"target_json": render_target(values),
"absent_fields": [key for key in FIELDS if values[key] is None],
}
)
# Every non-null label appears in a rendered clause; absent facts are
# represented only by their missingness statement, never hidden metadata.
for key in FIELDS:
if values[key] is not None:
assert str(values[key]) in clauses[ORDERS[j].index(key)]
assert len({r["normalized_sha256"] for r in pages}) == 2400
return pages, gold, templates
def prepare(config):
root = Path("data/generated")
root.mkdir(parents=True, exist_ok=True)
if (root / "manifest.json").exists():
raise ValueError("Existing frozen corpus; extract into a new directory")
pages, gold, templates = generate_tickets()
byid = {r["document_id"]: r for r in gold}
save_json(root / "input/pages.json", pages)
save_json(root / "input/templates.json", templates)
save_json(root / "labels/gold.json", gold)
audit = {}
for split in ["train", "validation", "test"]:
selected = [r for r in pages if r["split"] == split]
labels = [byid[r["document_id"]] for r in selected]
save_json(root / f"{split}/pages.json", selected)
if split != "test":
save_json(root / f"labels/{split}.json", labels)
absent = sum(bool(r["absent_fields"]) for r in labels)
assert absent / len(labels) >= 0.25
audit[split] = {
"documents": len(selected),
"families": len({r["family"] for r in selected}),
"with_absent_field": absent,
"with_absent_field_rate": absent / len(labels),
}
demo_ids = [
f"I{family:03d}-{j}"
for family, j in [
(0, 0),
(3, 1),
(8, 2),
(13, 3),
(21, 4),
(34, 5),
(55, 0),
(89, 1),
]
]
lookup = {r["document_id"]: r for r in pages}
demonstrations = [
{
"document_id": identifier,
"ticket": lookup[identifier]["ticket"],
"target_json": byid[identifier]["target_json"],
}
for identifier in demo_ids
]
assert all(lookup[i]["split"] == "train" for i in demo_ids)
save_json(root / "input/demonstrations.json", demonstrations)
save_json(
root / "manifest.json",
{
"dataset": "InterventionPairs-v1",
"rights": "CC0-1.0",
"seed": config["seed"],
"split_audit": audit,
"normalized_duplicates": 0,
"demonstration_ids": demo_ids,
"files": {str(f.relative_to(root)): sha(f) for f in root.rglob("*.json")},
},
)
save_json(
"outputs/data-audit.json",
{"splits": audit, "normalized_duplicates": 0, "demonstration_ids": demo_ids},
)
return audit
Code the response collator
Token boundary and EOS
Common error: right truncation removes EOS and the response; detect it before training.
Invariant: every expected response token has a target and the prompt is ignored.
File src/sft_complet/collator.py
import torch
import torch.nn.functional as F
INSTRUCTION = "Extract exactly unit, symptom, location, scheduled_date as a JSON object. Missing information must be null. A reported symptom is noise, leak, heat, or vibration. Return only the JSON object."
def prompt_text(ticket):
return INSTRUCTION + "\nTicket: " + ticket + "\nJSON:"
def encode_prompt_response(
tokenizer, ticket, response, config, identifier, supervise_prompt=False
):
# Explicit concatenation of two separately encoded token streams. We do not
# pretend it equals re-tokenizing the concatenated character string.
prompt = tokenizer.encode(prompt_text(ticket), add_special_tokens=False)
completion = tokenizer.encode(response, add_special_tokens=False) + [151643]
if not prompt or len(prompt) > config["max_prompt_tokens"]:
raise ValueError(
identifier + ": prompt exceeds the declared token budget; no truncation"
)
if (
len(completion) <= 1
or len(completion) > config["max_response_tokens_including_eos"]
):
raise ValueError(
identifier + ": missing or overlength response including EOS; no truncation"
)
ids = prompt + completion
labels = list(ids) if supervise_prompt else [-100] * len(prompt) + completion
return {
"document_id": identifier,
"ids": ids,
"labels": labels,
"prompt_tokens": len(prompt),
"response_tokens": len(completion),
"supervise_prompt": supervise_prompt,
}
def completion_labels(row):
return row["labels"]
def collate(rows, device="cuda", pad_to=None):
if not rows:
raise ValueError("Empty batch")
length = max(len(r["ids"]) for r in rows) if pad_to is None else pad_to
if length < max(len(r["ids"]) for r in rows):
raise ValueError("Silent truncation prohibited")
ids = torch.full((len(rows), length), 151643, dtype=torch.long, device=device)
labels = torch.full_like(ids, -100)
mask = torch.zeros_like(ids, dtype=torch.bool)
for i, row in enumerate(rows):
n = len(row["ids"])
ids[i, :n] = torch.tensor(row["ids"], device=device)
labels[i, :n] = torch.tensor(row["labels"], device=device)
mask[i, :n] = True
return ids, mask, labels
def completion_nll(logits, labels):
selected = labels[:, 1:] != -100
if not selected.any():
raise ValueError("No supervised next token")
values = F.cross_entropy(
logits[:, :-1][selected].float(), labels[:, 1:][selected], reduction="none"
)
return values.sum(), int(selected.sum()), values
Measure the prompt alone and select trainable weights
Initialization and adaptation space
Common error: embeddings tied to lm_head are unfrozen by mistake; check storage identities and the optimizer list.
Invariant: the final two blocks are identified and no other partial parameter is in the optimizer.
File src/sft_complet/baseline.py
from .model import load_base, select_trainables, parameter_manifest, manifest_digest
from .config import save_json
def report_trainable_parameters():
model = load_base()
result = {}
for branch in ["full", "partial"]:
parameters = select_trainables(model, branch)
rows = parameter_manifest(model)
result[branch] = {
"parameters": rows,
"total_parameters": sum(r["numel"] for r in rows),
"trainable_parameters": sum(r["numel"] for r in rows if r["requires_grad"]),
"optimizer_parameter_objects": len(parameters),
"unique_optimizer_parameter_objects": len({id(p) for p in parameters}),
"model_digest": manifest_digest(rows),
"tied_embedding_output": model.lm_head.weight
is model.model.embed_tokens.weight,
}
if result["full"]["model_digest"] != result["partial"]["model_digest"]:
raise ValueError("Selecting trainable parameters changed weights")
save_json("outputs/parameter-audit.json", result)
import shutil
shutil.copyfile("assets/model/tokenizer.json", "outputs/tokenizer-reference.json")
def select_last_blocks(model):
return select_trainables(model, "partial")
def evaluate_prompt_only(config, variant):
from .evaluate import predict
if variant not in ["zeroshot", "fewshot"]:
raise ValueError("Expected frozen-base baseline")
predict(config, variant)
Perform SFT and resume it
Conditional cross-entropy and optimization state
Common error: accumulating the average of short responses gives the wrong gradient; sum NLL and counters.
Invariant: expected full and partial weights changed; separate checkpoints, sampler and RNG states restored.
File src/sft_complet/train.py
import copy, hashlib, math, os, time
from pathlib import Path
import numpy as np
import torch
from .config import read_json, save_json, sha, seed_all, rng_state, restore_rng
from .model import (
load_base,
load_tokenizer,
select_trainables,
parameter_manifest,
manifest_digest,
save_learned,
active_response_logits,
)
from .collator import encode_prompt_response, collate
def equal(a, b):
if isinstance(a, torch.Tensor):
return isinstance(b, torch.Tensor) and torch.equal(a, b)
if isinstance(a, np.ndarray):
return isinstance(b, np.ndarray) and np.array_equal(a, b)
if isinstance(a, dict):
return (
isinstance(b, dict)
and a.keys() == b.keys()
and all(equal(a[k], b[k]) for k in a)
)
if isinstance(a, (list, tuple)):
return (
type(a) == type(b)
and len(a) == len(b)
and all(equal(x, y) for x, y in zip(a, b))
)
return a == b
def save_resume_state(path, state):
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists():
raise ValueError("Existing training state is preserved")
temporary = path.with_suffix(".pending")
torch.save(state, temporary)
os.replace(temporary, path)
def encode_split(config, tokenizer, split, supervise_prompt=False):
pages = read_json(f"data/generated/{split}/pages.json")
labels = {
r["document_id"]: r for r in read_json(f"data/generated/labels/{split}.json")
}
if any(p["split"] != split for p in pages):
raise ValueError("Wrong training/validation partition")
return [
encode_prompt_response(
tokenizer,
p["ticket"],
labels[p["document_id"]]["target_json"],
config,
p["document_id"],
supervise_prompt,
)
for p in pages
]
def learning_rate(config, branch, step, total):
peak = (
config["learning_rate_partial"]
if branch == "partial"
else config["learning_rate_full"]
)
warmup = config["warmup_updates"]
if step < warmup:
return peak * (step + 1) / warmup
return peak * (
1
- min(1.0, (step - warmup) / max(1, total - warmup - 1))
* (1 - config["minimum_lr_ratio"])
)
@torch.no_grad()
def validate_response(model, rows):
model.eval()
total = 0.0
count = 0
for row in rows:
ids, mask, labels = collate([row])
_, n, values, _, _ = active_response_logits(model, ids, mask, labels)
total += float(values.double().sum())
count += n
model.train()
return {
"response_nll_nats_per_token": total / count,
"response_tokens": count,
"documents": len(rows),
}
def train_sft(config, branch="full", mode="continuous"):
if (
Path("data/generated/test").exists()
or Path("data/generated/labels/gold.json").exists()
):
raise ValueError("Learning service must not mount test inputs or gold")
if mode not in ["interrupt", "continuous", "resume"]:
raise ValueError("Unknown training mode")
seed_all(config["seed"])
tokenizer = load_tokenizer()
rows = encode_split(config, tokenizer, "train", branch == "prompt_loss")
validation = encode_split(config, tokenizer, "validation", False)
if len(rows) != 1680 or len(validation) != 360:
raise ValueError("Expected frozen1680/360 population")
model = load_base()
parameters = select_trainables(model, branch)
initial = parameter_manifest(model)
initial_digest = manifest_digest(initial)
model = model.cuda()
parameters = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.AdamW(
parameters, lr=config["learning_rate_full"], weight_decay=config["weight_decay"]
)
sampler = torch.Generator().manual_seed(config["seed"] + 100)
updates_per_epoch = math.ceil(len(rows) / config["accumulation_examples"])
total_updates = updates_per_epoch * config["epochs"]
start = updates = 0
history = []
validation_history = []
identity = {
"branch": branch,
"config": config,
"base_sha256": sha("assets/model/model.safetensors"),
"tokenizer_sha256": sha("assets/model/tokenizer.json"),
"model_config_sha256": sha("assets/model/config.json"),
"model_lock_sha256": sha("locks/models.lock.json"),
"implementation_sha256": {
name: sha("src/sft_complet/" + name) for name in ["model.py", "train.py"]
},
"train_input_sha256": sha("data/generated/train/pages.json"),
"train_labels_sha256": sha("data/generated/labels/train.json"),
"validation_input_sha256": sha("data/generated/validation/pages.json"),
"validation_labels_sha256": sha("data/generated/labels/validation.json"),
"collator_sha256": sha("src/sft_complet/collator.py"),
"initial_model_digest": initial_digest,
"trainable_names": [r["name"] for r in initial if r["requires_grad"]],
}
root = Path("assets/checkpoints")
destination = root / f"{branch}-{mode}.pt"
if destination.exists():
raise ValueError("Preserve existing checkpoint")
if mode == "resume":
state = torch.load(
root / f"{branch}-interrupt.pt", map_location="cpu", weights_only=False
)
if state["identity"] != identity:
raise ValueError("Resume identity changed")
model.load_state_dict(state["model"], strict=True)
optimizer.load_state_dict(state["optimizer"])
sampler.set_state(state["sampler"])
restore_rng(state["rng"])
start = state["epoch"]
updates = state["updates"]
history = state["history"]
validation_history = state["validation_history"]
if state["scheduler"] != {
"next_update": updates,
"next_lr": learning_rate(config, branch, updates, total_updates),
}:
raise ValueError("Resume scheduler changed")
del state
stop = config["interrupt_epoch"] if mode == "interrupt" else config["epochs"]
begun = time.perf_counter()
torch.cuda.reset_peak_memory_stats()
model.train()
for epoch in range(start, stop):
permutation = torch.randperm(len(rows), generator=sampler).tolist()
epoch_loss = 0.0
epoch_tokens = 0
for offset in range(0, len(rows), config["accumulation_examples"]):
selected = permutation[offset : offset + config["accumulation_examples"]]
denominator = sum(
sum(v != -100 for v in rows[i]["labels"][1:]) for i in selected
)
lr = learning_rate(config, branch, updates, total_updates)
for group in optimizer.param_groups:
group["lr"] = lr
optimizer.zero_grad(set_to_none=True)
summed = 0.0
for i in selected:
ids, mask, labels = collate([rows[i]])
loss_sum, n, values, _, _ = active_response_logits(
model, ids, mask, labels
)
if not torch.isfinite(loss_sum) or n != sum(
v != -100 for v in rows[i]["labels"][1:]
):
raise ValueError("Nonfinite or miscounted active-token loss")
(loss_sum / denominator).backward()
summed += float(values.detach().double().sum())
norm = torch.nn.utils.clip_grad_norm_(
parameters, config["gradient_clip"], error_if_nonfinite=True
)
optimizer.step()
updates += 1
epoch_loss += summed
epoch_tokens += denominator
history.append(
{
"update": updates,
"epoch": epoch + 1,
"document_ids": [rows[i]["document_id"] for i in selected],
"active_tokens": denominator,
"nll_nats_per_active_token": summed / denominator,
"learning_rate": lr,
"gradient_norm_before_clip": float(norm),
}
)
if updates % 42 == 0:
print(history[-1], flush=True)
observed = validate_response(model, validation)
validation_history.append({"epoch": epoch + 1, **observed})
print(
{
"epoch": epoch + 1,
"train_nll_nats_per_active_token": epoch_loss / epoch_tokens,
"train_active_tokens": epoch_tokens,
"validation": observed,
},
flush=True,
)
torch.cuda.synchronize()
final = parameter_manifest(model)
before = {r["name"]: r for r in initial}
changed = [r["name"] for r in final if r["sha256"] != before[r["name"]]["sha256"]]
frozen_changed = [
r["name"]
for r in final
if not before[r["name"]]["requires_grad"]
and r["sha256"] != before[r["name"]]["sha256"]
]
if not changed or frozen_changed:
raise ValueError("No learning or frozen parameters changed")
state = {
"identity": identity,
"epoch": stop,
"updates": updates,
"model": model.state_dict(),
"optimizer": optimizer.state_dict(),
"sampler": sampler.get_state(),
"scheduler": {
"next_update": updates,
"next_lr": learning_rate(config, branch, updates, total_updates),
},
"rng": rng_state(),
"history": history,
"validation_history": validation_history,
}
save_resume_state(destination, state)
if mode == "resume" or (branch == "prompt_loss" and mode == "continuous"):
save_learned(model, root / f"{branch}-{mode}.safetensors", identity)
save_json(
f"outputs/train-{branch}-{mode}.json",
{
"branch": branch,
"mode": mode,
"identity": identity,
"epochs": stop,
"updates": updates,
"history": history,
"validation_history": validation_history,
"tokenization": rows,
"initial_parameters": initial,
"final_parameters": final,
"changed_parameters": changed,
"frozen_changed": frozen_changed,
"trainable_parameters": sum(
r["numel"] for r in initial if r["requires_grad"]
),
"total_parameters": sum(r["numel"] for r in initial),
"initial_model_digest": initial_digest,
"final_model_digest": manifest_digest(final),
"checkpoint_sha256": sha(destination),
"checkpoint_bytes": destination.stat().st_size,
"max_torch_allocated_bytes": torch.cuda.max_memory_allocated(),
"seconds": time.perf_counter() - begun,
"test_visible": False,
},
)
def verify_resume(branch):
root = Path("assets/checkpoints")
a = torch.load(
root / f"{branch}-continuous.pt", map_location="cpu", weights_only=False
)
b = torch.load(root / f"{branch}-resume.pt", map_location="cpu", weights_only=False)
checks = {k: equal(a[k], b[k]) for k in a}
if not all(checks.values()):
raise ValueError("Full resume differs: " + str(checks))
save_json(
f"outputs/resume-{branch}.json",
{
"status": "PASS",
"all_state_exact": True,
"checks": checks,
"boundary": "complete epoch; not a mid-microbatch recovery claim",
},
)
Evaluate without decoding constraints
Format and semantics kept separate
Common error: a tolerant parser fills absent fields; forbid all silent enrichment.
Invariant: 360 IDs per branch, without discarding invalid JSON.
File src/sft_complet/evaluate.py
import json, math, time
from pathlib import Path
import torch
from transformers import GenerationConfig
from .config import read_json, save_json, sha, seed_all
from .model import (
load_base,
load_tokenizer,
reload_learned,
active_response_logits,
parameter_manifest,
manifest_digest,
)
from .collator import prompt_text, encode_prompt_response, collate
from .data import FIELDS, SYMPTOMS
def parse_generated_record(text, finish_reason):
def pairs(items):
result = {}
for key, value in items:
if key in result:
raise ValueError("Duplicate key")
result[key] = value
return result
def constant(value):
raise ValueError("Nonfinite JSON constant")
try:
record = json.loads(text, object_pairs_hook=pairs, parse_constant=constant)
except (ValueError, TypeError):
return {
"json_parsable": False,
"schema_valid": False,
"completion_valid": False,
"record": None,
"reason": "TRUNCATED" if finish_reason != "eos" else "INVALID_JSON",
}
valid = (
isinstance(record, dict)
and set(record) == set(FIELDS)
and all(v is None or type(v) == str for v in record.values())
)
if valid:
valid = record["symptom"] is None or record["symptom"] in SYMPTOMS
return {
"json_parsable": True,
"schema_valid": valid,
"completion_valid": valid and finish_reason == "eos",
"record": record if valid else None,
"reason": (
"TRUNCATED"
if finish_reason != "eos"
else "VALID_SCHEMA" if valid else "INVALID_SCHEMA"
),
}
def generation_prompt(tokenizer, ticket, demonstrations, config):
text = "\n\n".join(
prompt_text(d["ticket"]) + d["target_json"] for d in demonstrations
)
if demonstrations:
text += "\n\n"
text += prompt_text(ticket)
ids = tokenizer.encode(text, add_special_tokens=False)
limit = (
config["max_fewshot_prompt_tokens"]
if demonstrations
else config["max_prompt_tokens"]
)
if len(ids) > limit:
raise ValueError("Generation prompt exceeds declared token cap; no truncation")
return text, ids
@torch.no_grad()
def predict(config, variant):
if Path("data/generated/labels").exists():
raise ValueError("Free generation must not mount gold labels")
seed_all(config["seed"])
tokenizer = load_tokenizer()
learned = variant in ["full", "partial", "prompt_loss"]
path = Path("assets/checkpoints") / (
variant
+ (
"-continuous.safetensors"
if variant == "prompt_loss"
else "-resume.safetensors"
)
)
if learned:
model, metadata = reload_learned(path)
identity = metadata["model_digest"]
else:
model = load_base()
identity = manifest_digest(parameter_manifest(model))
model = model.cuda().eval()
pages = read_json("data/generated/input/pages.json")
selected = [r for r in pages if r["split"] in ["validation", "test"]]
demonstrations = (
read_json("data/generated/input/demonstrations.json")
if variant == "fewshot"
else []
)
if len(selected) != 720 or (variant == "fewshot" and len(demonstrations) != 8):
raise ValueError("Expected fixed720heldout cases and8train demos")
generation = GenerationConfig(
do_sample=False,
num_beams=1,
max_new_tokens=config["max_new_tokens"],
use_cache=True,
eos_token_id=151643,
pad_token_id=151643,
bos_token_id=151643,
)
rows = []
begun = time.perf_counter()
torch.cuda.reset_peak_memory_stats()
for index, page in enumerate(selected):
text, ids = generation_prompt(tokenizer, page["ticket"], demonstrations, config)
input_ids = torch.tensor([ids], device="cuda")
mask = torch.ones_like(input_ids)
started = time.perf_counter()
output = (
model.generate(
input_ids=input_ids, attention_mask=mask, generation_config=generation
)[0, len(ids) :]
.cpu()
.tolist()
)
torch.cuda.synchronize()
finish = "eos" if output and output[-1] == 151643 else "length"
content = output[:-1] if finish == "eos" else output
decoded = tokenizer.decode(
content, skip_special_tokens=False, clean_up_tokenization_spaces=False
)
parsed = parse_generated_record(decoded, finish)
rows.append(
{
"document_id": page["document_id"],
"split": page["split"],
"variant": variant,
"input_ids": ids,
"input_text": text,
"output_ids": output,
"text": decoded,
"finish_reason": finish,
"parsed": parsed,
"input_tokens": len(ids),
"output_tokens": len(output),
"seconds": time.perf_counter() - started,
}
)
if (index + 1) % 60 == 0:
print(
{"variant": variant, "generated": index + 1, "total": len(selected)},
flush=True,
)
save_json(
f"outputs/predictions-{variant}.json",
{
"variant": variant,
"model_digest": identity,
"model_file_sha256": (
sha(path) if learned else sha("assets/model/model.safetensors")
),
"tokenizer_sha256": sha("assets/model/tokenizer.json"),
"input_sha256": sha("data/generated/input/pages.json"),
"demonstration_ids": [d["document_id"] for d in demonstrations],
"generation": generation.to_dict(),
"rows": rows,
"seconds": time.perf_counter() - begun,
"max_torch_allocated_bytes": torch.cuda.max_memory_allocated(),
"gold_visible": False,
},
)
def score_missing_fields(rows, gold):
n = len(rows)
correct = {k: 0 for k in FIELDS}
absent = {k: 0 for k in FIELDS}
invented = {k: 0 for k in FIELDS}
null_correct = {k: 0 for k in FIELDS}
invalid_absent = {k: 0 for k in FIELDS}
exact = parsable = schema = truncated = 0
details = []
for row in rows:
expected = gold[row["document_id"]]["target"]
parsed = parse_generated_record(row["text"], row["finish_reason"])
assert parsed == row["parsed"]
valid = parsed["completion_valid"]
record = parsed["record"]
fields = {key: valid and record[key] == expected[key] for key in FIELDS}
row_exact = all(fields.values())
exact += row_exact
parsable += parsed["json_parsable"]
schema += parsed["schema_valid"]
truncated += row["finish_reason"] == "length"
for key in FIELDS:
correct[key] += fields[key]
if expected[key] is None:
absent[key] += 1
null_correct[key] += valid and record[key] is None
invented[key] += valid and record[key] is not None
invalid_absent[key] += not valid
details.append(
{
"document_id": row["document_id"],
"fields_correct": fields,
"exact_match": row_exact,
"absent_fields": [k for k in FIELDS if expected[k] is None],
}
)
return {
"documents": n,
"json_parsable": parsable,
"json_parsable_rate": parsable / n,
"schema_valid": schema,
"schema_valid_rate": schema / n,
"exact_match": exact,
"exact_match_rate": exact / n,
"field_correct": correct,
"field_accuracy": {k: v / n for k, v in correct.items()},
"absent_opportunities": absent,
"correct_null": null_correct,
"invented_nonnull_in_valid_record": invented,
"invalid_record_with_absent_field": invalid_absent,
"invented_rate_over_all_absent_opportunities": {
k: invented[k] / absent[k] if absent[k] else None for k in FIELDS
},
"null_failure_rate_over_all_absent_opportunities": {
k: (absent[k] - null_correct[k]) / absent[k] if absent[k] else None
for k in FIELDS
},
"truncated": truncated,
"rows": details,
}
def evaluate_branches(variants):
gold = {r["document_id"]: r for r in read_json("data/generated/labels/gold.json")}
result = {}
for variant in variants:
predictions = read_json(f"outputs/predictions-{variant}.json")
result[variant] = {}
for split in ["validation", "test"]:
rows = [r for r in predictions["rows"] if r["split"] == split]
if len(rows) != 360 or len({r["document_id"] for r in rows}) != 360:
raise ValueError("Do not omit invalid heldout outputs")
result[variant][split] = score_missing_fields(rows, gold)
name = (
"exercise-evaluation.json" if variants == ["prompt_loss"] else "evaluation.json"
)
save_json(
"outputs/" + name,
{
"results": result,
"quality_gain_required": False,
"decoding_constraints": False,
"invalid_outputs_discarded": False,
},
)
print(
{
v: {k: value for k, value in splits["test"].items() if k != "rows"}
for v, splits in result.items()
},
flush=True,
)
@torch.no_grad()
def response_nll(config, variant):
tokenizer = load_tokenizer()
learned = variant in ["full", "partial", "prompt_loss"]
path = Path("assets/checkpoints") / (
variant
+ (
"-continuous.safetensors"
if variant == "prompt_loss"
else "-resume.safetensors"
)
)
model = reload_learned(path)[0] if learned else load_base()
model = model.cuda().eval()
pages = [
r for r in read_json("data/generated/input/pages.json") if r["split"] == "test"
]
gold = {r["document_id"]: r for r in read_json("data/generated/labels/gold.json")}
records = []
probes = []
# Always the same zero-shot response-only conditioning contract, including
# the prompt-loss ablation. Few-shot cost/quality is evaluated separately.
for index, page in enumerate(pages):
row = encode_prompt_response(
tokenizer,
page["ticket"],
gold[page["document_id"]]["target_json"],
config,
page["document_id"],
False,
)
ids, mask, labels = collate([row])
_, n, losses, logits, targets = active_response_logits(model, ids, mask, labels)
records.append(
{
"document_id": page["document_id"],
"input_ids": row["ids"],
"prompt_tokens": row["prompt_tokens"],
"active_response_tokens": n,
"response_targets": targets.cpu().tolist(),
"response_nll": losses.cpu().tolist(),
}
)
if index == 0:
for pos in [0, n - 1]:
probes.append(
{
"document_id": page["document_id"],
"response_position": pos,
"target": int(targets[pos]),
"logits": logits[pos].cpu().tolist(),
"nll": float(losses[pos]),
}
)
total = math.fsum(v for row in records for v in row["response_nll"])
count = sum(r["active_response_tokens"] for r in records)
save_json(
f"outputs/response-nll-{variant}.json",
{
"variant": variant,
"conditioning": "zero-shot completion-only for every compared weight state",
"documents": len(records),
"response_tokens": count,
"response_nll_nats_per_token": total / count,
"rows": records,
"probes": probes,
},
)
Change the mask to understand the signal
Supervision ablation
Common error: reuse full results as ablation results; train a new branch and retain its identity.
Invariant: a table of active tokens, cost and absent fields for every mask.
File src/sft_complet/exercise.py
from .config import read_json, save_json, sha
def preserve_main():
paths = [
"outputs/evaluation.json",
"outputs/response-nll-full.json",
"outputs/response-nll-partial.json",
"outputs/resume-full.json",
"outputs/resume-partial.json",
"assets/checkpoints/full-resume.safetensors",
"assets/checkpoints/partial-resume.safetensors",
]
save_json("outputs/exercise-before.json", {name: sha(name) for name in paths})
def compare_response_losses():
before = read_json("outputs/exercise-before.json")
if any(sha(name) != digest for name, digest in before.items()):
raise ValueError("Exercise changed main artifacts")
main = read_json("outputs/evaluation.json")
exercise = read_json("outputs/exercise-evaluation.json")
rows = {}
for branch in ["full", "partial", "prompt_loss"]:
training = read_json(
f"outputs/train-{branch}-"
+ ("continuous" if branch == "prompt_loss" else "resume")
+ ".json"
)
nll = read_json(f"outputs/response-nll-{branch}.json")
metrics = (exercise if branch == "prompt_loss" else main)["results"][branch][
"test"
]
rows[branch] = {
"training_active_tokens": sum(
r["active_tokens"] for r in training["history"]
),
"updates": training["updates"],
"trainable_parameters": training["trainable_parameters"],
"test_response_tokens": nll["response_tokens"],
"test_response_nll_nats_per_token": nll["response_nll_nats_per_token"],
"test_metrics": {k: v for k, v in metrics.items() if k != "rows"},
"model_digest": training["final_model_digest"],
"initial_model_digest": training["initial_model_digest"],
}
if (
len({r["test_response_tokens"] for r in rows.values()}) != 1
or len({r["initial_model_digest"] for r in rows.values()}) != 1
):
raise ValueError("Ablation initial state or evaluation population changed")
if (
rows["prompt_loss"]["training_active_tokens"]
<= rows["full"]["training_active_tokens"]
):
raise ValueError("Prompt-loss branch must actually supervise additional tokens")
save_json(
"outputs/exercise/loss-masks.json",
{
"rows": rows,
"protected_sha256": before,
"main_artifacts_unchanged": True,
"fresh_prompt_loss_training": True,
"quality_gain_required": False,
"training_denominators_differ": True,
},
)
Sources
Qwen2.5-0.5B base model, Qwen2 implementation in Transformers 4.56.2 and shared-tensor handling in Safetensors 0.6.2 were consulted to check the mechanisms used. The model is a pretrained base, not the Instruct variant. Executable versions and hashes are in locks and requirements.lock.txt.
Interpret the results actually obtained
During development, the full and partial branches each reach 360 exactly correct responses out of 360 test tickets. The base without examples reaches 145; the context with eight demonstrations reaches none under the complete-response contract. Its 360 test outputs reach the generation limit. A raw example starts with a JSON object then continues with a new instruction and another ticket: the base continues the demonstration sequence. We do not repair this output by retaining only its first object. This observation belongs to this base model and this serialization; it is not a general rule against few-shot prompting.
The regular corpus makes both adaptations sufficient here. The full/partial comparison includes two distinct learning rates; it does not isolate only the effect of parameter count. Observed continuous training+validation times are372.91s(full)and152.13s(partial), with PyTorch allocation peaks of9,964,563,968and2,611,971,584 bytes. These measurements depend on the machine and its load; they include neither acquisition, generation, resume repetitions nor final serialization. They are not all VRAMreserved byDocker.
The exercise that also supervises the prompt also reaches360/360. It activates430238 tokens versus105052in response-only loss for the same420updates. Its test response NLL is1.06265e−5, versus6.89464e−7forfull. No degradation in final responses is observed in this exercise; that does not remove the difference in signal and denominator. Detailed values and limits are provided below, and the reader path recalculates them fully.
Development observations
Values recalculated from this run's raw outputs. They are not performance thresholds; the path from the ZIP recalculates them on the same populations.
{
"status": "PASS",
"documents": 2400,
"test_documents": 360,
"parameters": 494032768,
"partial_trainable_parameters": 29824768,
"metrics": {
"zeroshot": {
"documents": 360,
"json_parsable": 329,
"json_parsable_rate": 0.9138888888888889,
"schema_valid": 167,
"schema_valid_rate": 0.4638888888888889,
"exact_match": 145,
"exact_match_rate": 0.4027777777777778,
"field_correct": {
"unit": 167,
"symptom": 161,
"location": 160,
"scheduled_date": 158
},
"field_accuracy": {
"unit": 0.4638888888888889,
"symptom": 0.44722222222222224,
"location": 0.4444444444444444,
"scheduled_date": 0.4388888888888889
},
"absent_opportunities": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"correct_null": {
"unit": 0,
"symptom": 0,
"location": 6,
"scheduled_date": 71
},
"invented_nonnull_in_valid_record": {
"unit": 0,
"symptom": 6,
"location": 7,
"scheduled_date": 9
},
"invalid_record_with_absent_field": {
"unit": 15,
"symptom": 3,
"location": 7,
"scheduled_date": 40
},
"invented_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.6666666666666666,
"location": 0.35,
"scheduled_date": 0.075
},
"null_failure_rate_over_all_absent_opportunities": {
"unit": 1.0,
"symptom": 1.0,
"location": 0.7,
"scheduled_date": 0.4083333333333333
},
"truncated": 31
},
"fewshot": {
"documents": 360,
"json_parsable": 0,
"json_parsable_rate": 0.0,
"schema_valid": 0,
"schema_valid_rate": 0.0,
"exact_match": 0,
"exact_match_rate": 0.0,
"field_correct": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"field_accuracy": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"absent_opportunities": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"correct_null": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invented_nonnull_in_valid_record": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invalid_record_with_absent_field": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"invented_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"null_failure_rate_over_all_absent_opportunities": {
"unit": 1.0,
"symptom": 1.0,
"location": 1.0,
"scheduled_date": 1.0
},
"truncated": 360
},
"full": {
"documents": 360,
"json_parsable": 360,
"json_parsable_rate": 1.0,
"schema_valid": 360,
"schema_valid_rate": 1.0,
"exact_match": 360,
"exact_match_rate": 1.0,
"field_correct": {
"unit": 360,
"symptom": 360,
"location": 360,
"scheduled_date": 360
},
"field_accuracy": {
"unit": 1.0,
"symptom": 1.0,
"location": 1.0,
"scheduled_date": 1.0
},
"absent_opportunities": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"correct_null": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"invented_nonnull_in_valid_record": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invalid_record_with_absent_field": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invented_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"null_failure_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"truncated": 0
},
"partial": {
"documents": 360,
"json_parsable": 360,
"json_parsable_rate": 1.0,
"schema_valid": 360,
"schema_valid_rate": 1.0,
"exact_match": 360,
"exact_match_rate": 1.0,
"field_correct": {
"unit": 360,
"symptom": 360,
"location": 360,
"scheduled_date": 360
},
"field_accuracy": {
"unit": 1.0,
"symptom": 1.0,
"location": 1.0,
"scheduled_date": 1.0
},
"absent_opportunities": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"correct_null": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"invented_nonnull_in_valid_record": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invalid_record_with_absent_field": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invented_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"null_failure_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"truncated": 0
},
"prompt_loss": {
"documents": 360,
"json_parsable": 360,
"json_parsable_rate": 1.0,
"schema_valid": 360,
"schema_valid_rate": 1.0,
"exact_match": 360,
"exact_match_rate": 1.0,
"field_correct": {
"unit": 360,
"symptom": 360,
"location": 360,
"scheduled_date": 360
},
"field_accuracy": {
"unit": 1.0,
"symptom": 1.0,
"location": 1.0,
"scheduled_date": 1.0
},
"absent_opportunities": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"correct_null": {
"unit": 15,
"symptom": 9,
"location": 20,
"scheduled_date": 120
},
"invented_nonnull_in_valid_record": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invalid_record_with_absent_field": {
"unit": 0,
"symptom": 0,
"location": 0,
"scheduled_date": 0
},
"invented_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"null_failure_rate_over_all_absent_opportunities": {
"unit": 0.0,
"symptom": 0.0,
"location": 0.0,
"scheduled_date": 0.0
},
"truncated": 0
}
},
"response_nll": {
"zeroshot": 0.3579458403824175,
"full": 6.894639002962341e-07,
"partial": 1.5182218123592618e-05,
"prompt_loss": 1.0626541892029729e-05
},
"quality_gain_required": false,
"scope": "Data-only tokenizer decoding, label formulas, per-tensor hashes, exact active-token budgets, resume evidence, full-vocabulary CE probes and all-case metrics; not BPE re-encoding or scalar reimplementation of the 0.5B Transformer"
}
