Cinco sitios usan el mismo esquema de clasificación, pero con vocabularios y frecuencias de clases diferentes. Un modelo local puede carecer de datos y una agregación uniforme puede dar demasiado peso a un sitio pequeño. Compartir pesos no constituye por sí solo una garantía de confidencialidad.
Un simulador local compara modelos por sitio, FedAvg y FedProx, con métricas globales y por sitio. Los mensajes intercambiados se pueden inspeccionar; no se usa ningún documento real ni se pretende realizar un despliegue federado seguro.
Descargar el proyecto completo, revisión r002
Las poblaciones
Los sitios A/B/C/D/E tienen 200/300/400/500/600 elementos, es decir, 2000. Cada familia contiene cinco elementos. Las 400 familias permanecen en un sitio y una partición: 70 % train, 15 % validation y 15 % test. Esto da 1400/300/300 elementos, con 140/210/280/350/420 de train por sitio. Se registran los recuentos de clases y todas las clases están representadas en cada test.
Cuatro clases sintéticas describen mantenimiento, calidad, compras e ingeniería. Los estilos y las proporciones varían según el sitio. Los pares de fragmentos de plantillas y las identidades de familia son disjuntos, pero las cláusulas semánticas de las clases se repiten entre sitios y particiones. Los resultados no miden la generalización a una GED real ni a tareas desconocidas.
E5-small congelado en 614241f622f53c4eeff9890bdc4f31cfecc418b3 codifica cada texto con query:, media enmascarada y normalización L2. Los vectores 384 permanecen en los directorios de los clientes. Se rechazan las entradas que superan 512 tokens. No se usa ningún gold para la codificación; la inferencia final no tiene ninguna etiqueta.
El modelo local
La cabeza CPU FP32 es 384→128, ReLU y después 4 logits. No tiene BatchNorm ni búfer entero. Los clientes comienzan una ronda con los mismos parámetros globales. Cada cliente realiza tres épocas SGD, learning_rate 0,1, momentum 0,9 y batch 32; el optimizador se reinicializa en cada ronda. Cada cliente conserva su generador de permutaciones entre rondas.
FedAvg utiliza la entropía cruzada local. FedProx añade mu/2 suma(||w−wt||²), con mu 0,1 y wt desconectado. La penalización conserva su punto de referencia del inicio de la ronda durante todas las actualizaciones locales. Con mu 0, los mismos datos, batch y RNG, se obtiene exactamente la actualización de FedAvg; la prueba entrena de verdad la pequeña cabeza para comprobar esta identidad.
El papel limitado del servidor
Un cliente envía parámetros, identificador, round_id, hash del modelo inicial y número de ejemplos únicos de train. Los bytes son FP32 little-endian, codificados en JSON base64 con dtype, shape, longitud y SHA-256. El decodificador rechaza datos no finitos, claves/dimensiones/dtype incorrectos y huellas falsas. No carga ningún pickle.
El servidor pondera mediante nk/suma(nj) sobre los participantes presentes. Los pesos no dependen del número de batches. La acumulación se realiza en FP64 en un orden fijo y después vuelve a FP32; el formato intercambiado sigue siendo FP32. La referencia escalar usa 1 y 3 con efectivos 1 y 3, dando 2,5. Si B está ausente, A recibe todo el peso; ninguna actualización nula sustituye a un ausente.
El servicio server solo monta exchange. No lee textos, features ni etiquetas de clientes. El coordinador anfitrión sigue siendo de confianza; los clientes virtuales comparten una máquina administrable. Esta separación funcional no protege frente a su administrador y no constituye ni agregación segura ni privacidad diferencial. Las actualizaciones pueden contener información sobre los datos; aquí no se mide ningún ataque ni garantía de confidencialidad.
Protocolo y reanudación
Veinte rondas con cinco sitios y tres épocas locales por ronda. FedAvg y FedProx tienen los mismos parámetros iniciales, los mismos RNG de permutación y el mismo calendario. Los parámetros finales se fijan en la ronda 20; validation/test no eligen ningún checkpoint. Se rechazan actualizaciones stale, duplicadas, no finitas o con un efectivo incorrecto. Una respuesta del servidor ya escrita no se sobrescribe.
El recorrido se interrumpe tras ocho rondas, ejecuta por separado veinte rondas continuas y después reanuda las doce rondas restantes desde el estado 8 conservado. La comparación abarca los modelos del servidor, las solicitudes completas, los estados de RNG de los clientes, el calendario y las pérdidas locales. La reanudación se hace en el límite de una ronda; no promete una restauración en medio de una escritura o de un mini-batch.
Las cinco referencias locales aprenden durante sesenta épocas en su propio sitio. La referencia centralizada aprende durante sesenta épocas sobre los 1400 elementos ficticios, fuera del escenario federado. Su momentum se conserva durante esas sesenta épocas; los clientes federados lo reinicializan por ronda. El presupuesto de pasadas sobre los ejemplos coincide para el calendario completo, pero las trayectorias y los números de actualizaciones globales no son idénticos. La comparación controlada principal es FedAvg/FedProx.
Leer las métricas y los costes
Para cada sitio, la macro-F1 promedia las cuatro F1 de clase. A continuación se muestran la media no ponderada de los cinco sitios y la media de estas macro-F1 ponderada por sus efectivos. Son dos agregaciones diferentes; ninguna sustituye la lectura del peor sitio. La diferencia mejor−peor y las matrices de confusión mantienen visibles los efectivos. La referencia local se evalúa con el modelo propio de cada sitio.
La loss por ronda es una media ponderada de las CE observadas durante los entrenamientos locales. No es la loss de un modelo global evaluado en un conjunto de control. Los bytes de parámetros ascendentes/descendentes se calculan a partir de los tensores transportados; los tamaños reales de las solicitudes JSON se separan porque incluyen metadatos, base64 y el modelo inicial. Este simulador no mide un caudal de red. El tiempo de codificación de E5 es distinto del de las cabezas y de los arranques de contenedores.
Ejercicio: cinco ausencias de E
Retire E en las rondas 3, 6, 9, 12 y 15, antes de cualquier resultado, tanto para FedAvg como para FedProx. Parta de los mismos parámetros y RNG iniciales. Los pesos de los presentes se renormalizan; el RNG del cliente ausente no consume ninguna permutación. Sus datos no se reconstruyen mediante el término proximal. Compare las puntuaciones de E y de los otros cuatro sitios, sin exigir que gane un método.
El ejercicio reduce las pasadas sobre los datos de E para ambos métodos; por tanto, su presupuesto total difiere del calendario completo. La comparación FedAvg/FedProx sigue teniendo el mismo calendario. Los resultados principales se conservan mediante SHA-256. No se transmite ningún original a una GED ni se aplica ninguna decisión documental.
Paso 01 — Crear cinco distribuciones locales sin fuga de familias
No IID y particiones agrupadas
Error: El mismo patrón de negocio aparece en varios sitios y particiones
Corrección: Particionar familias/patrones antes de producir los textos; verificar los hashes.
Archivo src/ged_apprentissage_federe/data.py
import hashlib
from .config import save_json
SIZES = [200, 300, 400, 500, 600]
STYLES = [
"Workshop note:",
"Laboratory file:",
"Procurement-facing register:",
"Field operations memo:",
"Engineering service report:",
]
CONTENTS = [
[
"Replace the damaged seal and restore pump operation.",
"Lubricate the bearings and repair the actuator.",
"Inspect the vibration source then restore the mechanical assembly.",
],
[
"Measure the sample against the tolerance and record conformity.",
"Verify calibration evidence and audit the inspection results.",
"Review the nonconformity record and confirm the measurement protocol.",
],
[
"Request a supplier quotation and place an equipment purchase order.",
"Compare offers and record supplier delivery conditions.",
"Prepare the replacement parts order and procurement approval.",
],
[
"Design a revised assembly and validate the prototype geometry.",
"Document a new control algorithm and its engineering test plan.",
"Revise the technical drawing and assess the design alternatives.",
],
]
OPEN = [
"A recorded item",
"The submitted note",
"A service record",
"The current memorandum",
"This technical entry",
"A pending file",
"An operational message",
"The reviewed page",
"A local dispatch",
"The archived request",
"An incoming record",
"A reference note",
"A newly received item",
"The working copy",
"A documented request",
"An indexed message",
"The consolidated entry",
"A dated report",
"The shared note",
"A formal record",
]
CLOSE = [
"has supporting context",
"retains the initial details",
"keeps a local reference",
"contains the required description",
"preserves the source wording",
"has a recorded owner",
"includes the attached context",
"keeps its revision history",
"carries a local annotation",
"remains a proposed assignment",
"needs a routing suggestion",
"is available for review",
"has a traceable origin",
"keeps the technical wording",
"contains no routing label",
"records the site terminology",
"uses the local writing style",
"preserves the original entry",
"remains a fictitious example",
"belongs to the demo inventory",
]
def prepare(config):
rows = []
gold = []
templates = {}
counts = {}
family_global = 0
for site_index, (site, size) in enumerate(zip(config["sites"], SIZES)):
families = size // 5
train = int(0.7 * families)
validation = int(0.15 * families)
site_rows = []
site_gold = []
for family in range(families):
split = (
"train"
if family < train
else "validation" if family < train + validation else "test"
)
template = OPEN[family_global // 20] + " " + CLOSE[family_global % 20] + "."
template_id = f"F{family_global:03d}"
templates[template_id] = {"site": site, "split": split, "text": template}
for j in range(5):
label = site_index % 4 if j < 3 else (family + site_index + j - 3) % 4
identifier = f"{site}-{family:03d}-{j}"
content = f"{STYLES[site_index]} {template} {CONTENTS[label][(family+j)%3]} Equipment ticket {site}{family:03d}{j}. Please preserve the original source."
row = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"text": content,
"content_sha256": hashlib.sha256(content.encode()).hexdigest(),
}
g = {
"document_id": identifier,
"site": site,
"family": template_id,
"split": split,
"label": label,
}
rows.append(row)
gold.append(g)
site_rows.append(row)
site_gold.append(g)
family_global += 1
save_json(f"data/generated/clients/{site}/input.json", site_rows)
save_json(
f"data/generated/clients/{site}/train-labels.json",
[g for g in site_gold if g["split"] == "train"],
)
counts[site] = {
split: {
"documents": sum(g["split"] == split for g in site_gold),
"classes": [
sum(g["split"] == split and g["label"] == c for g in site_gold)
for c in range(4)
],
}
for split in ["train", "validation", "test"]
}
save_json("data/generated/input/pages.json", rows)
save_json("data/generated/input/templates.json", templates)
save_json("data/generated/labels/gold.json", gold)
save_json(
"data/generated/labels/train.json", [g for g in gold if g["split"] == "train"]
)
save_json(
"data/generated/manifest.json",
{
"documents": 2000,
"families": 400,
"sites": counts,
"train": 1400,
"validation": 300,
"test": 300,
"simulation_only": True,
"privacy_guarantee": False,
"limits": "Class-specific semantic clauses repeat across families/sites; template pairs and document identities are disjoint. Site style and imbalance are deliberate.",
},
)
Paso 02 — Medir modelos locales y características comunes
Codificador congelado y mismas features
Error: El servidor recibe las features junto con los pesos
Corrección: Restringir el protocolo de mensajes a los parámetros, efectivos y métricas previstos.
Archivo src/ged_apprentissage_federe/baseline.py
import torch
from torch.nn import functional as F
from .client import load_train
from .model import initial_model
from .weights import pack, fingerprint
from .config import save_json
def fit(x, y, ids, config):
model = initial_model(config["seed"])
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
g = torch.Generator().manual_seed(config["seed"])
history = []
steps = 0
for epoch in range(config["baseline_epochs"]):
order = torch.randperm(len(x), generator=g)
loss_sum = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
loss = F.cross_entropy(model(x[selected]), y[selected])
loss.backward()
optimizer.step()
steps += 1
loss_sum += float(loss.detach()) * len(selected)
history.append(
{"epoch": epoch + 1, "CE": loss_sum / len(x), "examples": len(x)}
)
weights = pack(model.state_dict())
return {
"parameters": weights,
"sha256": fingerprint(weights),
"train_ids": ids,
"train_examples": len(x),
"epochs": config["baseline_epochs"],
"steps": steps,
"history": history,
"optimizer_reset": "once at baseline start; not federation rounds",
}
def baseline(config):
all_x = []
all_y = []
all_ids = []
for site in config["sites"]:
x, y, ids = load_train(site)
save_json(f"outputs/baselines/local-{site}.json", fit(x, y, ids, config))
all_x.append(x)
all_y.append(y)
all_ids.extend(ids)
save_json(
"outputs/baselines/central.json",
fit(torch.cat(all_x), torch.cat(all_y), all_ids, config),
)
save_json(
"outputs/baselines/NOTICE.json",
{
"centralization": "Separate reference on original fictitious data only, not a federated deployment",
"privacy_guarantee": False,
"baseline_budget": "60epochs, continuous SGD momentum; federated clients reset momentum per round",
},
)
Paso 03 — Escribir una agregación cuyos pesos tengan sentido
FedAvg y parámetros con nombre
Error: Media uniforme pese a efectivos muy diferentes
Corrección: Usar n_k únicos; publicar la media de sitios por separado de los pesos de entrenamiento.
Archivo src/ged_apprentissage_federe/server.py
"""Server reads public parameter states and updates only, never client features."""
import argparse, json
from pathlib import Path
import torch
from .model import initial_model
from .weights import pack, unpack, fingerprint
def aggregate(base, updates, round_id, participants, train_counts):
if not participants or len(set(participants)) != len(participants):
raise ValueError("Empty or duplicate participation")
if len(updates) != len(participants) or {u["client"] for u in updates} != set(
participants
):
raise ValueError("Missing or duplicate client update")
expected = initial_model().state_dict()
old = unpack(base, expected)
digest = fingerprint(base)
decoded = {}
for update in updates:
client = update["client"]
if update["round_id"] != round_id or update["base_sha256"] != digest:
raise ValueError("Stale round or initial model")
if (
type(update["train_examples"]) is not int
or update["train_examples"] != train_counts[client]
or update["train_examples"] <= 0
):
raise ValueError("Wrong unique training count")
decoded[client] = unpack(update["parameters"], expected)
total = sum(train_counts[c] for c in participants)
coefficients = {c: train_counts[c] / total for c in sorted(participants)}
# FP64 accumulation then one FP32 cast, order fixed; wire tensors stay FP32.
state = {
name: sum(
decoded[c][name].double() * coefficients[c] for c in sorted(participants)
).float()
for name in old
}
result = pack(state)
return {
"round_id": round_id,
"parameters": result,
"sha256": fingerprint(result),
"participants": sorted(participants),
"coefficients": coefficients,
"unique_train_examples": total,
"parameter_bytes": sum(v["bytes"] for v in result.values()),
"server_reads_client_data": False,
}
def main():
p = argparse.ArgumentParser()
p.add_argument("--request", required=True)
p.add_argument("--output", required=True)
args = p.parse_args()
request = json.loads(Path(args.request).read_text())
output = Path(args.output)
if output.exists():
raise ValueError("Refuse duplicate committed server round")
result = aggregate(
request["base"],
request["updates"],
request["round_id"],
request["participants"],
request["train_counts"],
)
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(result, sort_keys=True, allow_nan=False) + "\n")
if __name__ == "__main__":
main()
Paso 04 — Realizar rondas con y sin término proximal
Optimización local y FedProx
Error: w_t conserva un gradiente y se mueve en la penalización
Corrección: Clonar/desconectar la referencia del inicio de la ronda y comprobar que permanece constante.
Archivo src/ged_apprentissage_federe/client.py
import base64, math
from pathlib import Path
import torch
from torch.nn import functional as F
from .config import read_json, save_json
from .model import initial_model, proximal_loss
from .weights import pack, unpack, fingerprint
TRAIN_COUNTS = {"A": 140, "B": 210, "C": 280, "D": 350, "E": 420}
def generator_state(g):
return base64.b64encode(g.get_state().numpy().tobytes()).decode()
def generator_from_state(state):
g = torch.Generator(device="cpu")
g.set_state(
torch.tensor(list(base64.b64decode(state, validate=True)), dtype=torch.uint8)
)
return g
def load_train(site):
rows = read_json(f"outputs/clients/{site}/features.json")
labels = {
r["document_id"]: r["label"]
for r in read_json(f"data/generated/clients/{site}/train-labels.json")
}
selected = [r for r in rows if r["split"] == "train"]
if (
len(selected) != TRAIN_COUNTS[site]
or {r["document_id"] for r in selected} != labels.keys()
):
raise ValueError("Client training identity/count mismatch")
return (
torch.tensor([r["vector"] for r in selected], dtype=torch.float32),
torch.tensor([labels[r["document_id"]] for r in selected]),
[r["document_id"] for r in selected],
)
def train_update(site, base, round_id, mu, state, config):
x, y, ids = load_train(site)
model = initial_model()
anchor = unpack(base, model.state_dict())
model.load_state_dict(anchor, strict=True)
g = generator_from_state(state)
optimizer = torch.optim.SGD(
model.parameters(), lr=config["learning_rate"], momentum=config["momentum"]
)
epochs = []
steps = 0
for epoch in range(config["local_epochs"]):
order = torch.randperm(len(x), generator=g)
ce_total = prox_total = 0.0
for start in range(0, len(x), config["batch_size"]):
selected = order[start : start + config["batch_size"]]
optimizer.zero_grad(set_to_none=True)
ce = F.cross_entropy(model(x[selected]), y[selected])
prox = proximal_loss(model, anchor, mu)
loss = ce + prox
if not torch.isfinite(loss):
raise ValueError("Non-finite local loss")
loss.backward()
optimizer.step()
steps += 1
ce_total += float(ce.detach()) * len(selected)
prox_total += float(prox.detach()) * len(selected)
epochs.append(
{
"epoch": epoch + 1,
"CE": ce_total / len(x),
"proximal": prox_total / len(x),
"objective": (ce_total + prox_total) / len(x),
"examples": len(x),
}
)
parameters = pack(model.state_dict())
return {
"client": site,
"round_id": round_id,
"base_sha256": fingerprint(base),
"train_examples": len(ids),
"train_ids": ids,
"mu": mu,
"parameters": parameters,
"parameter_bytes": sum(v["bytes"] for v in parameters.values()),
"steps": steps,
"epochs": epochs,
"sampler_before": state,
"sampler_after": generator_state(g),
"optimizer_reset": True,
"device": "cpu",
}
def initialize(config):
base = pack(initial_model(config["seed"]).state_dict())
states = {
site: generator_state(torch.Generator().manual_seed(config["seed"] + 100 * i))
for i, site in enumerate(config["sites"])
}
save_json(
"outputs/shared/initial.json",
{
"parameters": base,
"sha256": fingerprint(base),
"sampler_states": states,
"round": 0,
"train_counts": TRAIN_COUNTS,
"model": "384->128ReLU->4 FP32 no buffers",
},
)
def request_round(config, variant, mode, round_id, exercise=False):
if variant not in ["fedavg", "fedprox"] or mode not in [
"interrupt",
"continuous",
"resume",
"exercise",
]:
raise ValueError("Unknown run")
initial = read_json("outputs/shared/initial.json")
root = Path(f"outputs/exchange/{variant}/{mode}")
root.mkdir(parents=True, exist_ok=True)
target = root / f"round_{round_id:03d}.request.json"
if target.exists():
raise ValueError("Refuse duplicate client round")
if round_id == 1:
base = initial["parameters"]
states = initial["sampler_states"]
else:
previous_mode = (
"interrupt"
if mode == "resume" and round_id == config["interrupt_round"] + 1
else mode
)
previous = Path(f"outputs/exchange/{variant}/{previous_mode}")
server = read_json(previous / f"round_{round_id-1:03d}.server.json")
request = read_json(previous / f"round_{round_id-1:03d}.request.json")
base = server["parameters"]
states = request["sampler_states_after"]
if server["round_id"] != round_id - 1:
raise ValueError("Previous round mismatch")
participants = list(config["sites"])
if exercise and round_id in config["exercise_absent_rounds"]:
participants.remove(config["exercise_absent_site"])
mu = 0.0 if variant == "fedavg" else config["mu"]
updates = [
train_update(site, base, round_id, mu, states[site], config)
for site in participants
]
after = dict(states)
for update in updates:
after[update["client"]] = update["sampler_after"]
save_json(
target,
{
"base": base,
"updates": updates,
"round_id": round_id,
"participants": participants,
"train_counts": TRAIN_COUNTS,
"sampler_states_before": states,
"sampler_states_after": after,
"variant": variant,
"mode": mode,
"exercise": exercise,
},
)
def verify_resume(config, variant):
full = Path(f"outputs/exchange/{variant}/continuous")
resume = Path(f"outputs/exchange/{variant}/resume")
checks = []
for i in range(config["interrupt_round"] + 1, config["rounds"] + 1):
a = read_json(full / f"round_{i:03d}.server.json")
b = read_json(resume / f"round_{i:03d}.server.json")
ra = read_json(full / f"round_{i:03d}.request.json")
rb = read_json(resume / f"round_{i:03d}.request.json")
ra.pop("mode")
rb.pop("mode")
checks.append(a == b and ra == rb)
if not all(checks):
raise AssertionError("Federated full state resume mismatch")
save_json(
f"outputs/resume-{variant}.json",
{
"status": "EXACT",
"rounds_checked": list(
range(config["interrupt_round"] + 1, config["rounds"] + 1)
),
"server_parameters": True,
"client_sampler_states": True,
"updates_losses_calendar": True,
"optimizer_policy": "reset per client round, boundary resume only",
},
)
Paso 05 — Leer ganadores y perdedores por sitio
Heterogeneidad y evaluación desagregada
Error: La puntuación del sitio más grande oculta una degradación del más pequeño
Corrección: Publicar las dos agregaciones y los soportes sin forzar una conclusión favorable.
Archivo src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Paso 06 — Simular la participación irregular
Calendario de clientes y reanudación
Error: La simulación se describe como una GED distribuida segura
Corrección: Recordar el ámbito local, los mensajes en claro y la ausencia de auditoría de confidencialidad.
Archivo src/ged_apprentissage_federe/evaluate.py
from pathlib import Path
import torch
from .config import read_json, save_json, sha
from .weights import unpack, fingerprint
from .model import initial_model
def probabilities_from_logits(logits):
"""Export FP64 probabilities while keeping learned head/logits FP32."""
if logits.dtype != torch.float32 or logits.ndim != 2 or logits.shape[1] != 4:
raise ValueError("Expected a batch of four FP32 class logits")
if not torch.isfinite(logits).all():
raise ValueError("Nonfinite classification logits")
return logits.to(torch.float64).softmax(-1)
def classification_metrics(rows, gold):
confusion = [[0] * 4 for _ in range(4)]
for row in rows:
confusion[gold[row["document_id"]]][row["predicted"]] += 1
classes = []
for c in range(4):
tp = confusion[c][c]
fp = sum(confusion[r][c] for r in range(4) if r != c)
fn = sum(confusion[c][j] for j in range(4) if j != c)
classes.append(
{
"class": c,
"support": sum(confusion[c]),
"tp": tp,
"fp": fp,
"fn": fn,
"F1": 2 * tp / (2 * tp + fp + fn) if 2 * tp + fp + fn else 0.0,
}
)
return {
"documents": len(rows),
"macro_F1": sum(c["F1"] for c in classes) / 4,
"accuracy": sum(confusion[i][i] for i in range(4)) / len(rows),
"confusion": confusion,
"per_class": classes,
}
def predict(config, exercise=False):
rows = []
models = {}
variants = (
["exercise-fedavg", "exercise-fedprox"]
if exercise
else ["local", "central", "fedavg", "fedprox"]
)
for variant in variants:
for site in config["sites"]:
if variant == "local":
path = f"outputs/baselines/local-{site}.json"
elif variant == "central":
path = "outputs/baselines/central.json"
else:
method = variant.removeprefix("exercise-")
mode = "exercise" if exercise else "resume"
path = f'outputs/exchange/{method}/{mode}/round_{config["rounds"]:03d}.server.json'
state = read_json(path)
model = initial_model()
model.load_state_dict(unpack(state["parameters"], model.state_dict()))
model.eval()
models[path] = state["sha256"]
features = [
f
for f in read_json(f"outputs/clients/{site}/features.json")
if f["split"] != "train"
]
x = torch.tensor([f["vector"] for f in features], dtype=torch.float32)
with torch.inference_mode():
logits = model(x)
probabilities = probabilities_from_logits(logits)
for f, z, prob in zip(features, logits.tolist(), probabilities.tolist()):
rows.append(
{
"document_id": f["document_id"],
"site": site,
"split": f["split"],
"variant": variant,
"logits": z,
"probabilities": prob,
"predicted": max(range(4), key=lambda c: prob[c]),
"model_path": path,
"model_sha256": state["sha256"],
}
)
save_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json",
{
"rows": rows,
"models": models,
"labels_visible": False,
"device": "cpu",
"logits_dtype": "float32",
"probability_dtype": "float64",
},
)
def evaluate(config, exercise=False):
rows = read_json(
"outputs/predictions-exercise.json" if exercise else "outputs/predictions.json"
)["rows"]
gold = {
g["document_id"]: g["label"]
for g in read_json("data/generated/labels/gold.json")
}
result = {}
for variant in sorted({r["variant"] for r in rows}):
for split in ["validation", "test"]:
per_site = {
site: classification_metrics(
[
r
for r in rows
if r["variant"] == variant
and r["site"] == site
and r["split"] == split
],
gold,
)
for site in config["sites"]
}
values = [m["macro_F1"] for m in per_site.values()]
total = sum(m["documents"] for m in per_site.values())
result[variant + "/" + split] = {
"sites": per_site,
"unweighted_site_macro_F1": sum(values) / 5,
"size_weighted_site_macro_F1": sum(
m["documents"] * m["macro_F1"] for m in per_site.values()
)
/ total,
"best_minus_worst_site": max(values) - min(values),
"documents": total,
}
communication = {}
for variant in ["fedavg", "fedprox"]:
mode = "exercise" if exercise else "continuous"
rounds = []
for i in range(1, config["rounds"] + 1):
path = Path(f"outputs/exchange/{variant}/{mode}/round_{i:03d}.request.json")
request = read_json(path)
server = read_json(path.with_name(f"round_{i:03d}.server.json"))
total = server["unique_train_examples"]
weighted = (
sum(
u["train_examples"]
* sum(e["CE"] for e in u["epochs"])
/ len(u["epochs"])
for u in request["updates"]
)
/ total
)
rounds.append(
{
"round": i,
"participants": server["participants"],
"coefficients": server["coefficients"],
"local_steps": sum(u["steps"] for u in request["updates"]),
"uploaded_parameter_bytes": sum(
u["parameter_bytes"] for u in request["updates"]
),
"downloaded_parameter_bytes": server["parameter_bytes"]
* len(request["updates"]),
"request_json_file_bytes": path.stat().st_size,
"mean_local_training_CE": weighted,
"CE_interpretation": "sample-weighted mean along local training trajectories, not held-out/global-model loss",
}
)
communication[variant] = rounds
save_json(
"outputs/evaluation-exercise.json" if exercise else "outputs/evaluation.json",
{
"metrics": result,
"communication": communication,
"privacy_guarantee": False,
"one_host_sequential_simulation": True,
"seed": config["seed"],
},
)
def protect_exercise(config):
save_json(
"outputs/exercise-protected.json",
{
p: sha(p)
for p in [
"outputs/evaluation.json",
"outputs/predictions.json",
"outputs/resume-fedavg.json",
"outputs/resume-fedprox.json",
]
},
)
def finish_exercise(config):
protected = read_json("outputs/exercise-protected.json")
if not all(sha(p) == v for p, v in protected.items()):
raise AssertionError("Main outputs changed")
main = read_json("outputs/evaluation.json")["metrics"]
other = read_json("outputs/evaluation-exercise.json")["metrics"]
deltas = {
variant: {
site: other["exercise-" + variant + "/test"]["sites"][site]["macro_F1"]
- main[variant + "/test"]["sites"][site]["macro_F1"]
for site in config["sites"]
}
for variant in ["fedavg", "fedprox"]
}
save_json(
"outputs/exercise.json",
{
"absent_site": config["exercise_absent_site"],
"absent_rounds": config["exercise_absent_rounds"],
"site_macro_F1_deltas": deltas,
"protected_sha256": protected,
"main_unchanged": True,
"privacy_guarantee": False,
},
)
Fuentes
McMahan et al., 2017 presenta FedAvg. Li et al., FedProx estudia la optimización federada heterogénea; este TP no reproduce sus benchmarks. Deep Leakage from Gradients motiva la prudencia sobre el contenido de las actualizaciones; aquí no se ejecuta ningún ataque. E5 congelado.
Precisión de las probabilidades exportadas
La cabeza, sus parámetros, su aprendizaje y sus logits permanecen en FP32. La softmax utilizada para exportar las probabilidades se calcula en FP64 a partir de estos logits: su comparación independiente emplea la misma precisión. La r001 observó una diferencia máxima de 1,80 × 10⁻⁷ entre la softmax CPU FP32 y la referencia FP64, por encima del umbral fijado de 10⁻⁷. Este umbral permanece sin cambios. Esta corrección de representación numérica no constituye una mejora de clasificación ni una modificación de FedAvg/FedProx.
Referencia técnica: implementación CPU de softmax en PyTorch 2.8.0. La ruta FP32 usa sus propios redondeos, una reducción vectorizada y una multiplicación por la suma inversa; no es una evaluación en aritmética exacta.
Resultados observados y límites de este experimento
Los cuatro métodos —modelos locales, centralizado, FedAvg y FedProx— alcanzan todos una macro-F1 de test de 1 en cada uno de los cinco sitios. Los 300 elementos de test se reparten en 30/45/60/75/90 elementos. La ausencia del sitio E en las rondas 3, 6, 9, 12 y 15 da una variación de macro-F1 de cero para los dos métodos federados en los cinco sitios. Por tanto, el gráfico de diferencias permanece en cero: no falta ningún resultado. Los patrones semánticos compartidos facilitan este corpus; estas puntuaciones no demuestran ninguna ganancia del enfoque federado, de FedProx ni de robustez ante un fallo real.
Las reanudaciones de ambos métodos recuperan exactamente los parámetros y los estados de los clientes de las ramas continuas. Cada cabeza contiene 49 796 parámetros FP32, es decir, 199 184 bytes lógicos. Veinte rondas con cinco clientes representan 19 918 400 bytes ascendentes por método; estos recuentos no son ni los tamaños JSON ni un caudal de red. El TP simula los sitios secuencialmente en una máquina: no ofrece ni agregación segura, ni privacidad diferencial, ni garantía de confidencialidad.
Los modelos locales y centralizado conservan su momentum durante sus 60 épocas; los optimizadores federados se reinicializan en cada ronda de tres épocas. Por tanto, las comparaciones no cambian únicamente la ubicación de los datos. Una sola semilla y este corpus ficticio no permiten una conclusión de negocio.
La r002 conserva exactamente los modelos, logits FP32 y clases de la r001. Solo la softmax de exportación se calcula en FP64; el umbral de auditoría independiente de 10⁻⁷ permanece sin cambios. Esta corrección numérica no mejora la clasificación.
Observaciones de desarrollo
Valores recalculados a partir de las salidas brutas de este recorrido. No son umbrales de ganancia; el recorrido desde el ZIP los recalcula sobre las mismas poblaciones.
{
"status": "PASS",
"documents": 2000,
"train": 1400,
"validation": 300,
"test": 300,
"sites": 5,
"parameters": 49796,
"rounds_per_method": 20,
"resume": "EXACT",
"privacy_guarantee": false,
"test_metrics": {
"central/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedavg/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"fedprox/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
},
"local/test": {
"sites": {
"A": {
"documents": 30,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
21,
0,
0,
0
],
[
0,
2,
0,
0
],
[
0,
0,
3,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 21,
"tp": 21,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 2,
"tp": 2,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 3,
"tp": 3,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"B": {
"documents": 45,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
5,
0,
0,
0
],
[
0,
32,
0,
0
],
[
0,
0,
4,
0
],
[
0,
0,
0,
4
]
],
"per_class": [
{
"class": 0,
"support": 5,
"tp": 5,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 32,
"tp": 32,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 4,
"tp": 4,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"C": {
"documents": 60,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
6,
0,
0,
0
],
[
0,
6,
0,
0
],
[
0,
0,
42,
0
],
[
0,
0,
0,
6
]
],
"per_class": [
{
"class": 0,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 42,
"tp": 42,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 6,
"tp": 6,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"D": {
"documents": 75,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
7,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
8,
0
],
[
0,
0,
0,
52
]
],
"per_class": [
{
"class": 0,
"support": 7,
"tp": 7,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 52,
"tp": 52,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
},
"E": {
"documents": 90,
"macro_F1": 1.0,
"accuracy": 1.0,
"confusion": [
[
63,
0,
0,
0
],
[
0,
8,
0,
0
],
[
0,
0,
9,
0
],
[
0,
0,
0,
10
]
],
"per_class": [
{
"class": 0,
"support": 63,
"tp": 63,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 1,
"support": 8,
"tp": 8,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 2,
"support": 9,
"tp": 9,
"fp": 0,
"fn": 0,
"F1": 1.0
},
{
"class": 3,
"support": 10,
"tp": 10,
"fp": 0,
"fn": 0,
"F1": 1.0
}
]
}
},
"unweighted_site_macro_F1": 1.0,
"size_weighted_site_macro_F1": 1.0,
"best_minus_worst_site": 0.0,
"documents": 300
}
},
"exercise_deltas": {
"fedavg": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
},
"fedprox": {
"A": 0.0,
"B": 0.0,
"C": 0.0,
"D": 0.0,
"E": 0.0
}
}
}
