Five sites use the same routing scheme but have different vocabularies and class frequencies. A local model may lack data, while uniform aggregation can overweight a small site. Sharing weights alone is not a privacy guarantee.
A local simulator compares per-site models, FedAvg, and FedProx using global and per-site metrics. Exchanged messages can be inspected; no real document is used and no secure federated deployment is claimed.
Download the complete project, revision r002
The populations
Sites A/B/C/D/E have 200/300/400/500/600 documents, giving 2,000. Each family contains five documents. The 400 families remain within one site and split: 70% train, 15% validation, and 15% test. This gives 1,400/300/300 documents, including 140/210/280/350/420 training items per site. Class counts are recorded and every class is represented in every test.
Four synthetic classes describe maintenance, quality, purchasing, and engineering. Styles and proportions vary by site. Template-fragment pairs and family identities are disjoint, but the semantic clauses of classes repeat across sites and splits. Results do not measure generalisation to a real document-management system or unknown tasks.
E5-small, pinned at 614241f622f53c4eeff9890bdc4f31cfecc418b3, encodes each text with query:, masked mean, and L2 normalisation. The 384-dimensional vectors remain in client directories. Inputs beyond 512 tokens are rejected. No gold data is used for encoding; final inference has no label.
The local model
The FP32 CPU head is 384→128, ReLU, then 4 logits. It has neither BatchNorm nor integer buffer. Clients start a round with the same global parameters. Each client performs three SGD epochs, learning_rate 0.1, momentum 0.9, batch 32; the optimiser is reset every round. Each client retains its permutation generator across rounds.
FedAvg uses local cross-entropy. FedProx adds mu/2 sum(||w−wt||²), with mu 0.1 and detached wt. The penalty retains its reference point from the beginning of the round throughout local updates. With mu 0, the same data, batch, and RNG give exactly the FedAvg update; the test really trains the small head to verify that identity.
The server's limited role
A client sends parameters, identifier, round_id, starting-model hash, and number of unique training examples. Bytes are FP32 little-endian, encoded as base64 JSON with dtype, shape, length, and SHA-256. The decoder rejects non-finite data, incorrect keys, dimensions, dtype, and fingerprints. It loads no pickle.
The server weights by nk/sum(nj) across present participants. Weights do not depend on the number of batches. Accumulation is FP64 in a fixed order, then returns to FP32; the exchanged format remains FP32. The scalar reference uses 1 and 3 with counts 1 and 3, giving 2.5. If B is absent, A receives all the weight; no null update replaces an absent participant.
The server service mounts exchange only. It reads neither client texts, features, nor labels. The host coordinator remains trusted; virtual clients share an administrable machine. This functional separation does not protect against its administrator and is neither secure aggregation nor differential privacy. Updates may contain information about data; no attack or privacy guarantee is measured here.
Protocol and resumption
Twenty rounds with five sites and three local epochs per round. FedAvg and FedProx have the same initial parameters, permutation RNGs, and schedule. Final parameters are fixed at round 20; validation/test selects no checkpoint. Stale, duplicate, non-finite updates or updates with an incorrect count are rejected. A server response already written is not overwritten.
The workflow interrupts after eight rounds, separately runs twenty continuous rounds, then resumes the remaining twelve rounds from retained state 8. The comparison covers server models, complete requests, client RNG states, schedule, and local losses. Resumption is at a round boundary; it does not promise restoration in the middle of a write or mini-batch.
The five local references train for sixty epochs on their own site. The centralised reference trains for sixty epochs on the 1,400 fictional documents, outside the federated scenario. Their momentum is retained during those sixty epochs; federated clients reset it per round. The example-pass budget matches for the full schedule, but trajectories and global-update counts differ. The primary controlled comparison is FedAvg/FedProx.
Reading metrics and costs
For each site, macro-F1 averages the four class F1 scores. We then display the unweighted mean of five sites and the mean of those macro-F1 scores weighted by their counts. These are two different aggregations; neither replaces reading the worst site. Best-minus-worst spread and confusion matrices keep counts visible. The local reference is evaluated with each site's own model.
Loss per round is a weighted average of cross-entropy observed along local training. It is not the loss of a global model evaluated on a control set. Upstream/downstream parameter bytes are calculated from transported tensors; actual JSON request sizes are separate because they include metadata, base64, and the initial model. This simulator does not measure network throughput. E5 encoding time is distinct from head and container-launch time.
Exercise: five absences of E
Remove E in rounds 3, 6, 9, 12, and 15, before any result, for both FedAvg and FedProx. Start again with the same initial parameters and RNGs. Weights of present clients are renormalised; the absent client's RNG does not consume a permutation. Its data is not recreated by the proximal term. Compare E's scores with those of the other four sites, without requiring either method to win.
The exercise reduces passes over E's data for both methods, so its total budget differs from the full schedule. The FedAvg/FedProx comparison remains on an equal schedule. Main results are retained by SHA-256. No original is sent to a document-management system and no document decision is applied.
Step 01 — Create five local distributions without family leakage
Non-IID and grouped partitions
Error: The same business template appears across several sites and splits
Correction: Partition families/templates before producing texts; check hashes.
File src/ged_apprentissage_federe/data.py
import hashlib
from .config import save_json
SIZES = [200, 300, 400, 500, 600]
STYLES = [
"Workshop note:",
"Laboratory file:",
"Procurement-facing register:",
"Field operations memo:",
"Engineering service report:",
]
CONTENTS = [
[
"Replace the damaged seal and restore pump operation.",
"Lubricate the bearings and repair the actuator.",
"Inspect the vibration source then restore the mechanical assembly.",
],
[
"Measure the sample against the tolerance and record conformity.",
"Verify calibration evidence and audit the inspection results.",
"Review the nonconformity record and confirm the measurement protocol.",
],
[
"Request a supplier quotation and place an equipment purchase order.",
"Compare offers and record supplier delivery conditions.",
"Prepare the replacement parts order and procurement approval.",
],
[
"Design a revised assembly and validate the prototype geometry.",
"Document a new control algorithm and its engineering test plan.",
"Revise the technical drawing and assess the design alternatives.",
],
]
OPEN = [
"A recorded item",
"The submitted note",
"A service record",
"The current memorandum",
"This technical entry",
"A pending file",
"An operational message",
"The reviewed page",
"A local dispatch",
"The archived request",
"An incoming record",
"A reference note",
"A newly received item",
"The working copy",
"A documented request",
"An indexed message",
"The consolidated entry",
"A dated report",
"The shared note",
"A formal record",
]
CLOSE = [
"has supporting context",
"retains the initial details",
"keeps a local reference",
"contains the required description",
"preserves the source wording",
"has a recorded owner",
"includes the attached context",
"keeps its revision history",
"carries a local annotation",
"remains a proposed assignment",
"needs a routing suggestion",
"is available for review",
"has a traceable origin",
"keeps the technical wording",
"contains no routing label",
"records the site terminology",
"uses the local writing style",
"preserves the original entry",
"remains a fictitious example",
"belongs to the demo inventory",
]
def prepare(config):
rows = []
gold = []
templates = {}
counts = {}
family_global = 0
for site_index, (site, size) in enumerate(zip(config["sites"], SIZES)):
families = size // 5
train = int(0.7 * families)
validation = int(0.15 * families)
site_rows = []
site_gold = []
for family in range(families):
split = (
"train"
if family < train
else "validation" if family < train + validation else "test"
)
template = OPEN[family_global // 20] + " " + CLOSE[family_global % 20] + "."
template_id = f"F{family_global:03d}"
templates[template_id] = {"site": site, "split": split, "text": template}
for j in range(5):
label = site_index % 4 if j < 3 else (family + site_index + j - 3) % 4
identifier = f"{site}-{family:03d}-{j}"
content = f"{STYLES[site_index]} {template} {CONTENTS[label][(family+j)%3]} Equipment ticket {site}{family:03d}{j}. Please preserve the original source."
row = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"text": content,
"content_sha256": hashlib.sha256(content.encode()).hexdigest(),
}
g = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"label": label,
}
rows.append(row)
gold.append(g)
site_rows.append(row)
site_gold.append(g)
family_global += 1
save_json(f"data/generated/clients/{site}/input.json", site_rows)
save_json(
f"data/generated/clients/{site}/train-labels.json",
[g for g in site_gold if g["split"] == "train"],
)
counts[site] = {
split: {
"documents": sum(g["split"] == split for g in site_gold),
"classes": [
sum(g["split"] == split and g["label"] == c for g in site_gold)
for c in range(4)
],
}
for split in ["train", "validation", "test"]
}
save_json("data/generated/input/pages.json", rows)
save_json("data/generated/input/templates.json", templates)
save_json("data/generated/labels/gold.json", gold)
save_json(
"data/generated/labels/train.json", [g for g in gold if g["split"] == "train"]
)
save_json(
"data/generated/manifest.json",
{
"documents": 2000,
"families": 400,
"sites": counts,
"train": 1400,
"validation": 300,
"test": 300,
"simulation_only": True,
"privacy_guarantee": False,
"limits": "Class-specific semantic clauses repeat across families/sites; template pairs and document identities are disjoint. Site style and imbalance are deliberate.",
},
)
Step 02 — Measure local models and common features
Pinned encoder and identical features
Error: The server receives features together with weights
Correction: Restrict the message protocol to parameters, counts, and specified metrics.
File src/ged_apprentissage_federe/baseline.py
import torch
from torch.nn import functional as F
from .client import load_train
from .model import initial_model
from .weights import pack, fingerprint
from .config import save_json
def fit(x, y, ids, config):
model = initial_model(config["seed"])
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
g = torch.Generator().manual_seed(config["seed"])
history = []
steps = 0
for epoch in range(config["baseline_epochs"]):
order = torch.randperm(len(x), generator=g)
loss_sum = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
loss = F.cross_entropy(model(x[selected]), y[selected])
loss.backward()
optimizer.step()
steps += 1
loss_sum += float(loss.detach()) * len(selected)
history.append(
{"epoch": epoch + 1, "CE": loss_sum / len(x), "examples": len(x)}
)
weights = pack(model.state_dict())
return {
"parameters": weights,
"sha256": fingerprint(weights),
"train_ids": ids,
"train_examples": len(x),
"epochs": config["baseline_epochs"],
"steps": steps,
"history": history,
"optimizer_reset": "once at baseline start; not federation rounds",
}
def baseline(config):
all_x = []
all_y = []
all_ids = []
for site in config["sites"]:
x, y, ids = load_train(site)
save_json(f"outputs/baselines/local-{site}.json", fit(x, y, ids, config))
all_x.append(x)
all_y.append(y)
all_ids.extend(ids)
save_json(
"outputs/baselines/central.json",
fit(torch.cat(all_x), torch.cat(all_y), all_ids, config),
)
save_json(
"outputs/baselines/NOTICE.json",
{
"centralization": "Separate reference on original fictitious data only, not a federated deployment",
"privacy_guarantee": False,
"baseline_budget": "60epochs, continuous SGD momentum; federated clients reset momentum per round",
},
)
Step 03 — Write an aggregation whose weights have meaning
FedAvg and named parameters
Error: Uniform average despite very different counts
Correction: Use unique n_k values; publish the site mean separately from training weights.
File src/ged_apprentissage_federe/server.py
"""Server reads public parameter states and updates only, never client features."""
import argparse, json
from pathlib import Path
import torch
from .model import initial_model
from .weights import pack, unpack, fingerprint
def aggregate(base, updates, round_id, participants, train_counts):
if not participants or len(set(participants)) != len(participants):
raise ValueError("Empty or duplicate participation")
if len(updates) != len(participants) or {u["client"] for u in updates} != set(
participants
):
raise ValueError("Missing or duplicate client update")
expected = initial_model().state_dict()
old = unpack(base, expected)
digest = fingerprint(base)
decoded = {}
for update in updates:
client = update["client"]
if update["round_id"] != round_id or update["base_sha256"] != digest:
raise ValueError("Stale round or initial model")
if (
type(update["train_examples"]) is not int
or update["train_examples"] != train_counts[client]
or update["train_examples"] <= 0
):
raise ValueError("Wrong unique training count")
decoded[client] = unpack(update["parameters"], expected)
total = sum(train_counts[c] for c in participants)
coefficients = {c: train_counts[c] / total for c in sorted(participants)}
# FP64 accumulation then one FP32 cast, order fixed; wire tensors stay FP32.
state = {
name: sum(
decoded[c][name].double() * coefficients[c] for c in sorted(participants)
).float()
for name in old
}
result = pack(state)
return {
"round_id": round_id,
"parameters": result,
"sha256": fingerprint(result),
"participants": sorted(participants),
"coefficients": coefficients,
"unique_train_examples": total,
"parameter_bytes": sum(v["bytes"] for v in result.values()),
"server_reads_client_data": False,
}
def main():
p = argparse.ArgumentParser()
p.add_argument("--request", required=True)
p.add_argument("--output", required=True)
args = p.parse_args()
request = json.loads(Path(args.request).read_text())
output = Path(args.output)
if output.exists():
raise ValueError("Refuse duplicate committed server round")
result = aggregate(
request["base"],
request["updates"],
request["round_id"],
request["participants"],
request["train_counts"],
)
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(result, sort_keys=True, allow_nan=False) + "\n")
if __name__ == "__main__":
main()
Step 04 — Run rounds with and without a proximal term
Local optimisation and FedProx
Error: w_t retains a gradient and moves in the penalty
Correction: Clone/detach the reference at the start of the round and check that it remains constant.
File src/ged_apprentissage_federe/client.py
import base64, math
from pathlib import Path
import torch
from torch.nn import functional as F
from .config import read_json, save_json
from .model import initial_model, proximal_loss
from .weights import pack, unpack, fingerprint
TRAIN_COUNTS = {"A": 140, "B": 210, "C": 280, "D": 350, "E": 420}
def generator_state(g):
return base64.b64encode(g.get_state().numpy().tobytes()).decode()
def generator_from_state(state):
g = torch.Generator(device="cpu")
g.set_state(
torch.tensor(list(base64.b64decode(state, validate=True)), dtype=torch.uint8)
)
return g
def load_train(site):
rows = read_json(f"outputs/clients/{site}/features.json")
labels = {
r["document_id"]: r["label"]
for r in read_json(f"data/generated/clients/{site}/train-labels.json")
}
selected = [r for r in rows if r["split"] == "train"]
if (
len(selected) != TRAIN_COUNTS[site]
or {r["document_id"] for r in selected} != labels.keys()
):
raise ValueError("Client training identity/count mismatch")
return (
torch.tensor([r["vector"] for r in selected], dtype=torch.float32),
torch.tensor([labels[r["document_id"]] for r in selected]),
[r["document_id"] for r in selected],
)
def train_update(site, base, round_id, mu, state, config):
x, y, ids = load_train(site)
model = initial_model()
anchor = unpack(base, model.state_dict())
model.load_state_dict(anchor, strict=True)
g = generator_from_state(state)
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
epochs = []
steps = 0
for epoch in range(config["local_epochs"]):
order = torch.randperm(len(x), generator=g)
ce_total = prox_total = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
ce = F.cross_entropy(model(x[selected]), y[selected])
prox = proximal_loss(model, anchor, mu)
loss = ce + prox
if not torch.isfinite(loss):
raise ValueError("Non-finite local loss")
loss.backward()
optimizer.step()
steps += 1
ce_total += float(ce.detach()) * len(selected)
prox_total += float(prox.detach()) * len(selected)
epochs.append(
{
"epoch": epoch + 1,
"CE": ce_total / len(x),
"proximal": prox_total / len(x),
"objective": (ce_total + prox_total) / len(x),
"examples": len(x),
}
)
parameters = pack(model.state_dict())
return {
"client": site,
"round_id": round_id,
"base_sha256": fingerprint(base),
"train_examples": len(ids),
"train_ids": ids,
"mu": mu,
"parameters": parameters,
"parameter_bytes": sum(v["bytes"] for v in parameters.values()),
"steps": steps,
"epochs": epochs,
"sampler_before": state,
"sampler_after": generator_state(g),
"optimizer_reset": True,
"device": "cpu",
}
def initialize(config):
base = pack(initial_model(config["seed"]).state_dict())
states = {
site: generator_state(torch.Generator().manual_seed(config["seed"] + 100 * i))
for i, site in enumerate(config["sites"])
}
save_json(
"outputs/shared/initial.json",
{
"parameters": base,
"sha256": fingerprint(base),
"sampler_states": states,
"round": 0,
"train_counts": TRAIN_COUNTS,
"model": "384->128ReLU->4 FP32 no buffers",
},
)
def request_round(config, variant, mode, round_id, exercise=False):
if variant not in ["fedavg", "fedprox"] or mode not in [
"interrupt",
"continuous",
"resume",
"exercise",
]:
raise ValueError("Unknown run")
initial = read_json("outputs/shared/initial.json")
root = Path(f"outputs/exchange/{variant}/{mode}")
root.mkdir(parents=True, exist_ok=True)
target = root / f"round_{round_id:03d}.request.json"
if target.exists():
raise ValueError("Refuse duplicate client round")
if round_id == 1:
base = initial["parameters"]
states = initial["sampler_states"]
else:
previous_mode = (
"interrupt"
if mode == "resume" and round_id == config["interrupt_round"] + 1
else mode
)
previous = Path(f"outputs/exchange/{variant}/{previous_mode}")
server = read_json(previous / f"round_{round_id-1:03d}.server.json")
request = read_json(previous / f"round_{round_id-1:03d}.request.json")
base = server["parameters"]
states = request["sampler_states_after"]
if server["round_id"] != round_id - 1:
raise ValueError("Previous round mismatch")
participants = list(config["sites"])
if exercise and round_id in config["exercise_absent_rounds"]:
participants.remove(config["exercise_absent_site"])
mu = 0.0 if variant == "fedavg" else config["mu"]
updates = [
train_update(site, base, round_id, mu, states[site], config)
for site in participants
]
after = dict(states)
for update in updates:
after[update["client"]] = update["sampler_after"]
save_json(
target,
{
"base": base,
"updates": updates,
"round_id": round_id,
"participants": participants,
"train_counts": TRAIN_COUNTS,
"sampler_states_before": states,
"sampler_states_after": after,
"variant": variant,
"mode": mode,
"exercise": exercise,
},
)
def verify_resume(config, variant):
full = Path(f"outputs/exchange/{variant}/continuous")
resume = Path(f"outputs/exchange/{variant}/resume")
checks = []
for i in range(config["interrupt_round"] + 1, config["rounds"] + 1):
a = read_json(full / f"round_{i:03d}.server.json")
b = read_json(resume / f"round_{i:03d}.server.json")
ra = read_json(full / f"round_{i:03d}.request.json")
rb = read_json(resume / f"round_{i:03d}.request.json")
ra.pop("mode")
rb.pop("mode")
checks.append(a == b and ra == rb)
if not all(checks):
raise AssertionError("Federated full state resume mismatch")
save_json(
f"outputs/resume-{variant}.json",
{
"status": "EXACT",
"rounds_checked": list(
range(config["interrupt_round"] + 1, config["rounds"] + 1)
),
"server_parameters": True,
"client_sampler_states": True,
"updates_losses_calendar": True,
"optimizer_policy": "reset per client round, boundary resume only",
},
)
Step 05 — Read winners and losers by site
Heterogeneity and disaggregated evaluation
Error: The largest site's score hides a decline at the smallest
Correction: Publish both aggregations and supports without forcing a favourable conclusion.
File src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Step 06 — Simulate irregular participation
Client schedule and resumption
Error: The simulation is described as a secure distributed document-management system
Correction: Restate the local scope, clear-text messages, and absence of a privacy audit.
File src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Sources
McMahan et al., 2017 presents FedAvg. Li et al., FedProx studies heterogeneous federated optimisation; this practical exercise does not reproduce its benchmarks. Deep Leakage from Gradients motivates caution about update contents; no attack is run here. Pinned E5.
Precision of exported probabilities
The head, its parameters, training, and logits remain FP32. The softmax used to export probabilities is calculated in FP64 from these logits: its independent comparison uses the same precision. r001 observed a maximum difference of 1.80 × 10⁻⁷ between CPU FP32 softmax and the FP64 reference, above the fixed threshold of 10⁻⁷. That threshold remains unchanged. This numerical-representation correction is neither a classification gain nor a change to FedAvg/FedProx.
Technical reference: CPU softmax implementation in PyTorch 2.8.0. The FP32 path uses its own rounding, vectorised reduction, and multiplication by the inverse sum; it is not an evaluation in exact arithmetic.
Observed results and limits of this experiment
The four methods—local models, centralised, FedAvg, and FedProx—all reach a test macro-F1 of 1 on each of the five sites. The 300 test documents are distributed as 30/45/60/75/90 documents. E's absence in rounds 3, 6, 9, 12, and 15 produces a macro-F1 variation of zero for both federated methods across all five sites. The differences chart is therefore zero: it is not missing a result. Shared semantic templates make this corpus easy; these scores demonstrate no gain from federation, FedProx, or resilience to a real failure.
Resumption of both methods exactly recovers the parameters and client states of continuous branches. Each head has 49,796 FP32 parameters, or 199,184 logical bytes. Twenty rounds with five clients represent 19,918,400 upstream bytes per method; these counts are neither JSON sizes nor network throughput. The practical exercise simulates sites sequentially on one machine: it provides neither secure aggregation, differential privacy, nor a privacy guarantee.
Local and centralised models retain their momentum during their 60 epochs; federated optimisers are reset every round of three epochs. The comparisons therefore do not change only the placement of data. One seed and this fictional corpus cannot support a business conclusion.
r002 retains exactly r001's models, FP32 logits, and classes. Only export softmax is calculated in FP64; the independent audit threshold of 10⁻⁷ remains unchanged. This numerical correction does not improve classification.
Development observations
Values recalculated from the raw outputs of this run. They are not improvement thresholds; the workflow from the ZIP recalculates them over the same populations.
{
"status": "PASS",
"documents": 2000,
"train": 1400,
"validation": 300,
"test": 300,
"sites": 5,
"parameters": 49796,
"rounds_per_method": 20,
"resume": "EXACT",
"privacy_guarantee": false,
"test_metrics": {
"central/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedavg/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedprox/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"local/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
}
},
"exercise_deltas": {
"fedavg": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
},
"fedprox": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
}
}
}
