Cinq sites utilisent un même schéma de routage mais des vocabulaire et fréquences de classes différents. Un modèle local peut manquer de données et une agrégation uniforme peut surpondérer un petit site. Partager des poids ne constitue pas, à lui seul, une garantie de confidentialité.
Un simulateur local compare modèles par site, FedAvg et FedProx, avec métriques globales et par site. Les messages échangés sont inspectables ; aucun document réel n'est utilisé et aucun déploiement fédéré sécurisé n'est prétendu.
Télécharger le projet complet, révision r002
Les populations
Les sites A/B/C/D/E possèdent 200/300/400/500/600 pièces, soit 2000. Chaque famille contient cinq pièces. Les 400 familles restent dans un site et un split :70%train,15%validation,15%test. Cela donne 1400/300/300 pièces, dont140/210/280/350/420train par site. Les comptes de classes sont enregistrés et toutes les classes sont représentées dans chaque test.
Quatre classes synthétiques décrivent maintenance, qualité, achats et ingénierie. Les styles et proportions varient par site. Les paires de fragments de gabarits et identités de famille sont disjointes, mais les clauses sémantiques des classes se répètent entre sites/splits. Les résultats ne mesurent pas une généralisation à une vraie GED ni à des tâches inconnues.
E5-small gelé à614241f622f53c4eeff9890bdc4f31cfecc418b3 encode chaque texte avec query:, moyenne masquée et normalisationL2. Les vecteurs384 restent dans les répertoires clients. Les entrées dépassant512 tokens sont refusées. Aucun gold n’est utilisé pour l’encodage ; l’inférence finale n’a aucun label.
Le modèle local
La tête CPU FP32 est384→128, ReLU, puis4 logits. Elle n’a ni BatchNorm ni buffer entier. Les clients commencent un tour avec les mêmes paramètres globaux. Chaque client fait trois époques SGD, learning_rate0,1, momentum0,9, batch 32 ; l’optimiseur est réinitialisé à chaque tour. Chaque client conserve son générateur de permutations entre les tours.
FedAvg utilise la cross-entropy locale. FedProx ajoute mu/2 somme(||w−wt||²), avec mu 0,1 et wt détaché. La pénalité garde son point de référence du début du tour pendant toutes les mises à jour locales. Avec mu 0, mêmes données, batch et RNG, on retrouve exactement l’actualisation FedAvg ; le test entraîne réellement la petite tête pour vérifier cette identité.
Le rôle limité du serveur
Un client envoie paramètres, identifiant, round_id, hash du modèle de départ et nombre d’exemples train uniques. Les octets sont FP32 little-endian, encodés en base64 JSON avec dtype, shape, longueur et SHA-256. Le décodeur refuse données non finies, clés/dimensions/dtype incorrects et empreintes fausses. Il ne charge aucun pickle.
Le serveur pondère par nk/somme(nj) sur les participants présents. Les poids ne dépendent pas du nombre de batches. L’accumulation se fait en FP64 dans un ordre fixé puis revient en FP32 ; le format échangé reste FP32. La référence scalaire utilise1et3 avec effectifs1et3, donnant2,5. Si B est absent, A reçoit tout le poids ; aucune mise à jour nulle ne remplace un absent.
Le service server monte seulement exchange. Il ne lit ni textes, ni features, ni labels clients. Le coordinateur hôte reste de confiance ; les clients virtuels partagent une machine administrable. Cette séparation fonctionnelle ne protège pas contre son administrateur et ne constitue ni agrégation sécurisée, ni confidentialité différentielle. Les mises à jour peuvent contenir de l’information sur les données ; aucune attaque ni garantie de confidentialité n’est mesurée ici.
Protocole et reprise
Vingt tours avec cinq sites, trois époques locales par tour. FedAvg et FedProx ont mêmes paramètres initiaux, mêmes RNG de permutation et même calendrier. Les paramètres finaux sont fixés au tour20 ; validation/test ne choisissent aucun checkpoint. Les mises à jour stales, dupliquées, non finies ou portant un mauvais effectif sont refusées. Une réponse serveur déjà écrite n’est pas écrasée.
Le parcours interrompt après huit tours, exécute séparément vingt tours continus puis reprend les douze tours restants à partir de l’état8conservé. La comparaison porte sur les modèles serveur, les demandes complètes, les états des RNG clients, le calendrier et les pertes locales. La reprise est à une frontière de tour ; elle ne promet pas une restauration au milieu d’une écriture ou d’un mini-batch.
Les cinq références locales apprennent soixante époques sur leur propre site. La référence centralisée apprend soixante époques sur les 1400 pièces fictives, en dehors du scénario fédéré. Leur momentum est conservé pendant ces soixante époques ; les clients fédérés le réinitialisent par tour. Le budget de passages sur les exemples coïncide pour le calendrier complet, mais les trajectoires et nombres de mises à jour globales ne sont pas identiques. La comparaison contrôlée principale est FedAvg/FedProx.
Lire les métriques et coûts
Pour chaque site, la macro-F1 moyenne les quatre F1 de classe. On affiche ensuite moyenne non pondérée des cinq sites et moyenne de ces macro-F1 pondérée par leurs effectifs. Ce sont deux agrégations différentes ; aucune ne remplace la lecture du pire site. L’écart meilleur−pire et les matrices de confusion gardent les effectifs visibles. La référence locale est évaluée avec le modèle propre à chaque site.
La loss par tour est une moyenne pondérée des CE observées le long des entraînements locaux. Elle n’est pas la loss d’un modèle global évalué sur un jeu de contrôle. Les octets de paramètres montants/descendants sont calculés à partir des tenseurs transportés ; les tailles réelles des demandes JSON sont séparées, car elles incluent métadonnées, base64 et modèle initial. Ce simulateur ne mesure pas un débit réseau. Le temps d’encodage E5 est distinct de celui des têtes et des lancements de conteneurs.
Exercice : cinq absences d’E
Retirer E aux tours 3,6,9,12et15, avant tout résultat, pour FedAvg comme FedProx. Repartir des mêmes paramètres et RNG initiaux. Les poids des présents sont renormalisés ; le RNG du client absent ne consomme pas de permutation. Ses données ne sont pas reconstituées par le terme proximal. Comparer les scores d’E et des quatre autres sites, sans exiger qu’une méthode gagne.
L’exercice réduit les passages sur les données d’E pour les deux méthodes ; son budget total diffère donc du calendrier complet. La comparaison FedAvg/FedProx reste à calendrier égal. Les résultats principaux sont conservés par SHA-256. Aucun original n’est transmis à une GED, aucune décision documentaire appliquée.
Étape 01 — Créer cinq distributions locales sans fuite de familles
Non-IID et partitions groupées
Erreur : Le même patron d'affaire apparaît dans plusieurs sites et splits
Correction : Partitionner familles/patrons avant de produire les textes ; vérifier les hashes.
Fichier src/ged_apprentissage_federe/data.py
import hashlib
from .config import save_json
SIZES = [200, 300, 400, 500, 600]
STYLES = [
"Workshop note:",
"Laboratory file:",
"Procurement-facing register:",
"Field operations memo:",
"Engineering service report:",
]
CONTENTS = [
[
"Replace the damaged seal and restore pump operation.",
"Lubricate the bearings and repair the actuator.",
"Inspect the vibration source then restore the mechanical assembly.",
],
[
"Measure the sample against the tolerance and record conformity.",
"Verify calibration evidence and audit the inspection results.",
"Review the nonconformity record and confirm the measurement protocol.",
],
[
"Request a supplier quotation and place an equipment purchase order.",
"Compare offers and record supplier delivery conditions.",
"Prepare the replacement parts order and procurement approval.",
],
[
"Design a revised assembly and validate the prototype geometry.",
"Document a new control algorithm and its engineering test plan.",
"Revise the technical drawing and assess the design alternatives.",
],
]
OPEN = [
"A recorded item",
"The submitted note",
"A service record",
"The current memorandum",
"This technical entry",
"A pending file",
"An operational message",
"The reviewed page",
"A local dispatch",
"The archived request",
"An incoming record",
"A reference note",
"A newly received item",
"The working copy",
"A documented request",
"An indexed message",
"The consolidated entry",
"A dated report",
"The shared note",
"A formal record",
]
CLOSE = [
"has supporting context",
"retains the initial details",
"keeps a local reference",
"contains the required description",
"preserves the source wording",
"has a recorded owner",
"includes the attached context",
"keeps its revision history",
"carries a local annotation",
"remains a proposed assignment",
"needs a routing suggestion",
"is available for review",
"has a traceable origin",
"keeps the technical wording",
"contains no routing label",
"records the site terminology",
"uses the local writing style",
"preserves the original entry",
"remains a fictitious example",
"belongs to the demo inventory",
]
def prepare(config):
rows = []
gold = []
templates = {}
counts = {}
family_global = 0
for site_index, (site, size) in enumerate(zip(config["sites"], SIZES)):
families = size // 5
train = int(0.7 * families)
validation = int(0.15 * families)
site_rows = []
site_gold = []
for family in range(families):
split = (
"train"
if family < train
else "validation" if family < train + validation else "test"
)
template = OPEN[family_global // 20] + " " + CLOSE[family_global % 20] + "."
template_id = f"F{family_global:03d}"
templates[template_id] = {"site": site, "split": split, "text": template}
for j in range(5):
label = site_index % 4 if j < 3 else (family + site_index + j - 3) % 4
identifier = f"{site}-{family:03d}-{j}"
content = f"{STYLES[site_index]} {template} {CONTENTS[label][(family+j)%3]} Equipment ticket {site}{family:03d}{j}. Please preserve the original source."
row = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"text": content,
"content_sha256": hashlib.sha256(content.encode()).hexdigest(),
}
g = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"label": label,
}
rows.append(row)
gold.append(g)
site_rows.append(row)
site_gold.append(g)
family_global += 1
save_json(f"data/generated/clients/{site}/input.json", site_rows)
save_json(
f"data/generated/clients/{site}/train-labels.json",
[g for g in site_gold if g["split"] == "train"],
)
counts[site] = {
split: {
"documents": sum(g["split"] == split for g in site_gold),
"classes": [
sum(g["split"] == split and g["label"] == c for g in site_gold)
for c in range(4)
],
}
for split in ["train", "validation", "test"]
}
save_json("data/generated/input/pages.json", rows)
save_json("data/generated/input/templates.json", templates)
save_json("data/generated/labels/gold.json", gold)
save_json(
"data/generated/labels/train.json", [g for g in gold if g["split"] == "train"]
)
save_json(
"data/generated/manifest.json",
{
"documents": 2000,
"families": 400,
"sites": counts,
"train": 1400,
"validation": 300,
"test": 300,
"simulation_only": True,
"privacy_guarantee": False,
"limits": "Class-specific semantic clauses repeat across families/sites; template pairs and document identities are disjoint. Site style and imbalance are deliberate.",
},
)
Étape 02 — Mesurer modèles locaux et caractéristiques communes
Encodeur gelé et mêmes features
Erreur : Le serveur reçoit les features avec les poids
Correction : Restreindre le protocole de message aux paramètres, effectifs et métriques prévues.
Fichier src/ged_apprentissage_federe/baseline.py
import torch
from torch.nn import functional as F
from .client import load_train
from .model import initial_model
from .weights import pack, fingerprint
from .config import save_json
def fit(x, y, ids, config):
model = initial_model(config["seed"])
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
g = torch.Generator().manual_seed(config["seed"])
history = []
steps = 0
for epoch in range(config["baseline_epochs"]):
order = torch.randperm(len(x), generator=g)
loss_sum = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
loss = F.cross_entropy(model(x[selected]), y[selected])
loss.backward()
optimizer.step()
steps += 1
loss_sum += float(loss.detach()) * len(selected)
history.append(
{"epoch": epoch + 1, "CE": loss_sum / len(x), "examples": len(x)}
)
weights = pack(model.state_dict())
return {
"parameters": weights,
"sha256": fingerprint(weights),
"train_ids": ids,
"train_examples": len(x),
"epochs": config["baseline_epochs"],
"steps": steps,
"history": history,
"optimizer_reset": "once at baseline start; not federation rounds",
}
def baseline(config):
all_x = []
all_y = []
all_ids = []
for site in config["sites"]:
x, y, ids = load_train(site)
save_json(f"outputs/baselines/local-{site}.json", fit(x, y, ids, config))
all_x.append(x)
all_y.append(y)
all_ids.extend(ids)
save_json(
"outputs/baselines/central.json",
fit(torch.cat(all_x), torch.cat(all_y), all_ids, config),
)
save_json(
"outputs/baselines/NOTICE.json",
{
"centralization": "Separate reference on original fictitious data only, not a federated deployment",
"privacy_guarantee": False,
"baseline_budget": "60epochs, continuous SGD momentum; federated clients reset momentum per round",
},
)
Étape 03 — Écrire une agrégation dont les poids ont un sens
FedAvg et paramètres nommés
Erreur : Moyenne uniforme malgré des effectifs très différents
Correction : Utiliser n_k uniques ; publier moyenne sites séparément des poids d'entraînement.
Fichier src/ged_apprentissage_federe/server.py
"""Server reads public parameter states and updates only, never client features."""
import argparse, json
from pathlib import Path
import torch
from .model import initial_model
from .weights import pack, unpack, fingerprint
def aggregate(base, updates, round_id, participants, train_counts):
if not participants or len(set(participants)) != len(participants):
raise ValueError("Empty or duplicate participation")
if len(updates) != len(participants) or {u["client"] for u in updates} != set(
participants
):
raise ValueError("Missing or duplicate client update")
expected = initial_model().state_dict()
old = unpack(base, expected)
digest = fingerprint(base)
decoded = {}
for update in updates:
client = update["client"]
if update["round_id"] != round_id or update["base_sha256"] != digest:
raise ValueError("Stale round or initial model")
if (
type(update["train_examples"]) is not int
or update["train_examples"] != train_counts[client]
or update["train_examples"] <= 0
):
raise ValueError("Wrong unique training count")
decoded[client] = unpack(update["parameters"], expected)
total = sum(train_counts[c] for c in participants)
coefficients = {c: train_counts[c] / total for c in sorted(participants)}
# FP64 accumulation then one FP32 cast, order fixed; wire tensors stay FP32.
state = {
name: sum(
decoded[c][name].double() * coefficients[c] for c in sorted(participants)
).float()
for name in old
}
result = pack(state)
return {
"round_id": round_id,
"parameters": result,
"sha256": fingerprint(result),
"participants": sorted(participants),
"coefficients": coefficients,
"unique_train_examples": total,
"parameter_bytes": sum(v["bytes"] for v in result.values()),
"server_reads_client_data": False,
}
def main():
p = argparse.ArgumentParser()
p.add_argument("--request", required=True)
p.add_argument("--output", required=True)
args = p.parse_args()
request = json.loads(Path(args.request).read_text())
output = Path(args.output)
if output.exists():
raise ValueError("Refuse duplicate committed server round")
result = aggregate(
request["base"],
request["updates"],
request["round_id"],
request["participants"],
request["train_counts"],
)
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(result, sort_keys=True, allow_nan=False) + "\n")
if __name__ == "__main__":
main()
Étape 04 — Faire des tours avec et sans terme proximal
Optimisation locale et FedProx
Erreur : w_t conserve un gradient et bouge dans la pénalité
Correction : Cloner/détacher la référence de début de tour et vérifier sa constance.
Fichier src/ged_apprentissage_federe/client.py
import base64, math
from pathlib import Path
import torch
from torch.nn import functional as F
from .config import read_json, save_json
from .model import initial_model, proximal_loss
from .weights import pack, unpack, fingerprint
TRAIN_COUNTS = {"A": 140, "B": 210, "C": 280, "D": 350, "E": 420}
def generator_state(g):
return base64.b64encode(g.get_state().numpy().tobytes()).decode()
def generator_from_state(state):
g = torch.Generator(device="cpu")
g.set_state(
torch.tensor(list(base64.b64decode(state, validate=True)), dtype=torch.uint8)
)
return g
def load_train(site):
rows = read_json(f"outputs/clients/{site}/features.json")
labels = {
r["document_id"]: r["label"]
for r in read_json(f"data/generated/clients/{site}/train-labels.json")
}
selected = [r for r in rows if r["split"] == "train"]
if (
len(selected) != TRAIN_COUNTS[site]
or {r["document_id"] for r in selected} != labels.keys()
):
raise ValueError("Client training identity/count mismatch")
return (
torch.tensor([r["vector"] for r in selected], dtype=torch.float32),
torch.tensor([labels[r["document_id"]] for r in selected]),
[r["document_id"] for r in selected],
)
def train_update(site, base, round_id, mu, state, config):
x, y, ids = load_train(site)
model = initial_model()
anchor = unpack(base, model.state_dict())
model.load_state_dict(anchor, strict=True)
g = generator_from_state(state)
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
epochs = []
steps = 0
for epoch in range(config["local_epochs"]):
order = torch.randperm(len(x), generator=g)
ce_total = prox_total = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
ce = F.cross_entropy(model(x[selected]), y[selected])
prox = proximal_loss(model, anchor, mu)
loss = ce + prox
if not torch.isfinite(loss):
raise ValueError("Non-finite local loss")
loss.backward()
optimizer.step()
steps += 1
ce_total += float(ce.detach()) * len(selected)
prox_total += float(prox.detach()) * len(selected)
epochs.append(
{
"epoch": epoch + 1,
"CE": ce_total / len(x),
"proximal": prox_total / len(x),
"objective": (ce_total + prox_total) / len(x),
"examples": len(x),
}
)
parameters = pack(model.state_dict())
return {
"client": site,
"round_id": round_id,
"base_sha256": fingerprint(base),
"train_examples": len(ids),
"train_ids": ids,
"mu": mu,
"parameters": parameters,
"parameter_bytes": sum(v["bytes"] for v in parameters.values()),
"steps": steps,
"epochs": epochs,
"sampler_before": state,
"sampler_after": generator_state(g),
"optimizer_reset": True,
"device": "cpu",
}
def initialize(config):
base = pack(initial_model(config["seed"]).state_dict())
states = {
site: generator_state(torch.Generator().manual_seed(config["seed"] + 100 * i))
for i, site in enumerate(config["sites"])
}
save_json(
"outputs/shared/initial.json",
{
"parameters": base,
"sha256": fingerprint(base),
"sampler_states": states,
"round": 0,
"train_counts": TRAIN_COUNTS,
"model": "384->128ReLU->4 FP32 no buffers",
},
)
def request_round(config, variant, mode, round_id, exercise=False):
if variant not in ["fedavg", "fedprox"] or mode not in [
"interrupt",
"continuous",
"resume",
"exercise",
]:
raise ValueError("Unknown run")
initial = read_json("outputs/shared/initial.json")
root = Path(f"outputs/exchange/{variant}/{mode}")
root.mkdir(parents=True, exist_ok=True)
target = root / f"round_{round_id:03d}.request.json"
if target.exists():
raise ValueError("Refuse duplicate client round")
if round_id == 1:
base = initial["parameters"]
states = initial["sampler_states"]
else:
previous_mode = (
"interrupt"
if mode == "resume" and round_id == config["interrupt_round"] + 1
else mode
)
previous = Path(f"outputs/exchange/{variant}/{previous_mode}")
server = read_json(previous / f"round_{round_id-1:03d}.server.json")
request = read_json(previous / f"round_{round_id-1:03d}.request.json")
base = server["parameters"]
states = request["sampler_states_after"]
if server["round_id"] != round_id - 1:
raise ValueError("Previous round mismatch")
participants = list(config["sites"])
if exercise and round_id in config["exercise_absent_rounds"]:
participants.remove(config["exercise_absent_site"])
mu = 0.0 if variant == "fedavg" else config["mu"]
updates = [
train_update(site, base, round_id, mu, states[site], config)
for site in participants
]
after = dict(states)
for update in updates:
after[update["client"]] = update["sampler_after"]
save_json(
target,
{
"base": base,
"updates": updates,
"round_id": round_id,
"participants": participants,
"train_counts": TRAIN_COUNTS,
"sampler_states_before": states,
"sampler_states_after": after,
"variant": variant,
"mode": mode,
"exercise": exercise,
},
)
def verify_resume(config, variant):
full = Path(f"outputs/exchange/{variant}/continuous")
resume = Path(f"outputs/exchange/{variant}/resume")
checks = []
for i in range(config["interrupt_round"] + 1, config["rounds"] + 1):
a = read_json(full / f"round_{i:03d}.server.json")
b = read_json(resume / f"round_{i:03d}.server.json")
ra = read_json(full / f"round_{i:03d}.request.json")
rb = read_json(resume / f"round_{i:03d}.request.json")
ra.pop("mode")
rb.pop("mode")
checks.append(a == b and ra == rb)
if not all(checks):
raise AssertionError("Federated full state resume mismatch")
save_json(
f"outputs/resume-{variant}.json",
{
"status": "EXACT",
"rounds_checked": list(
range(config["interrupt_round"] + 1, config["rounds"] + 1)
),
"server_parameters": True,
"client_sampler_states": True,
"updates_losses_calendar": True,
"optimizer_policy": "reset per client round, boundary resume only",
},
)
Étape 05 — Lire les gagnants et perdants par site
Hétérogénéité et évaluation désagrégée
Erreur : Le score du plus grand site cache une dégradation du plus petit
Correction : Publier les deux agrégations et les supports sans forcer une conclusion favorable.
Fichier src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Étape 06 — Simuler la participation irrégulière
Calendrier de clients et reprise
Erreur : La simulation est décrite comme GED sécurisée distribuée
Correction : Rappeler le périmètre local, les messages en clair et l'absence d'audit de confidentialité.
Fichier src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Sources
McMahan et al.,2017 présente FedAvg. Li et al.,FedProx étudie l’optimisation fédérée hétérogène ; ce TP n’en reproduit pas les benchmarks. Deep Leakage from Gradients motive la prudence sur le contenu des mises à jour ; aucune attaque n’est exécutée ici. E5 figé.
Précision des probabilités exportées
La tête, ses paramètres, son apprentissage et ses logits restent en FP32. La softmax utilisée pour exporter les probabilités est calculée en FP64 à partir de ces logits : sa comparaison indépendante emploie la même précision. La r001 a observé un écart maximal de 1,80 × 10⁻⁷ entre la softmax CPU FP32 et la référence FP64, au-dessus du seuil fixé de 10⁻⁷. Ce seuil reste inchangé. Cette correction de représentation numérique ne constitue pas un gain de classification ou une modification de FedAvg/FedProx.
Référence technique : implémentation CPU de softmax dans PyTorch 2.8.0. Le chemin FP32 utilise ses propres arrondis, une réduction vectorisée et une multiplication par la somme inversée ; il n’est pas une évaluation en arithmétique exacte.
Résultats observés et limites de cette expérience
Les quatre méthodes — modèles locaux, centralisé, FedAvg et FedProx — atteignent toutes un macro-F1 test de 1 sur chacun des cinq sites. Les 300 pièces test sont réparties en 30/45/60/75/90 pièces. L’absence du site E aux tours 3, 6, 9, 12 et 15 donne une variation de macro-F1 de zéro pour les deux méthodes fédérées sur les cinq sites. Le graphique des différences reste donc à zéro : il ne manque pas de résultat. Les patrons sémantiques partagés rendent ce corpus facile ; ces scores ne démontrent aucun gain du fédéré, de FedProx ou de robustesse à une panne réelle.
Les reprises des deux méthodes retrouvent exactement les paramètres et états de clients des branches continues. Chaque tête contient 49 796 paramètres FP32, soit 199 184 octets logiques. Vingt tours à cinq clients représentent 19 918 400 octets montants par méthode ; ces comptes ne sont ni les tailles JSON ni un débit réseau. Le TP simule des sites séquentiellement sur une machine : il ne fournit ni agrégation sécurisée, ni confidentialité différentielle, ni garantie de confidentialité.
Les modèles locaux et centralisé conservent leur momentum pendant leurs 60 époques ; les optimiseurs fédérés sont réinitialisés à chaque tour de trois époques. Les comparaisons ne changent donc pas uniquement le placement des données. Une seule graine et ce corpus fictif ne permettent pas une conclusion métier.
La r002 conserve exactement les modèles, logits FP32 et classes de la r001. Seule la softmax d’export est calculée en FP64 ; le seuil d’audit indépendant de 10⁻⁷ reste inchangé. Cette correction numérique n’améliore pas la classification.
Observations de développement
Valeurs recalculées depuis les sorties brutes de ce parcours. Elles ne sont pas des seuils de gain ; le parcours depuis le ZIP les recalcule sur les mêmes populations.
{
"status": "PASS",
"documents": 2000,
"train": 1400,
"validation": 300,
"test": 300,
"sites": 5,
"parameters": 49796,
"rounds_per_method": 20,
"resume": "EXACT",
"privacy_guarantee": false,
"test_metrics": {
"central/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedavg/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedprox/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"local/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
}
},
"exercise_deltas": {
"fedavg": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
},
"fedprox": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
}
}
}
