During a document-management migration, some categories are renamed, split, merged, or have no equivalent. A forced one-to-one match creates misleading correspondences and can move documents into an unsuitable category.
An explainable matrix proposes multiple links with costs, masses, and warnings, then exports a review file. No skos:exactMatch or document move is created automatically.
Download the complete project, revision r001
A corpus of multiple correspondences
Twenty-four taxonomy pairs each have 8 source concepts and 10 targets, giving 192 and 240 concepts. Twelve domains are used for calibration and twelve others for testing. Domains and text wrappers are distinct, but semantic relation prototypes and synonyms are shared: this is not a fully out-of-distribution experiment. This choice makes the mechanisms understandable; performance does not demonstrate business-domain generalisation.
Each pair has 64 original workload notes. Their two classification assignments are simulated independently and produce a and b from counts/64; they are not correspondence annotations. Every concept in the main workflow has positive mass. Gold data contains one-to-many, many-to-one, and one category with no equivalent. It remains in a separate file, never in the costs.
broader relations remain internal to a taxonomy; the code rejects duplicates, missing parents, and cycles. This practical exercise adopts this local acyclic convention; it does not claim to define every general SKOS constraint. No export declares skos:exactMatch and no migration is applied.
E5: meaning and context
E5-small, pinned at 614241f622f53c4eeff9890bdc4f31cfecc418b3, encodes label plus definition and a second ancestor-context string. The query: prefix is symmetric for source and target. Masked pooling, real special tokens included, then L2 normalisation in 384 dimensions; the code rejects more than 512 tokens and checks every loaded file and tensor.
A root uses the same explicit root-context sentence, so root/root has identical context. The cost is Cij=1−cos(ei,ej)+0.25(1−cos(ai,aj)). The 0.25 factor is fixed. Cosines calculated from FP32 vectors are accumulated in FP64, then bounded to [-1,1] for rounding. Costs do not know the gold data.
Three methods, three interpretations
Top-1 cosine selects the textually closest target, with a threshold chosen during calibration. It ignores masses, so it remains unchanged in the mass exercise. Balanced transport enforces its a and b marginals: it can move a workload toward a semantically poor target to satisfy those constraints.
Unbalanced transport minimises sum(ΠC)+epsilon sum(Π(logΠ−1))+rho_a KL(Π1||a)+rho_b KL(Πᵀ1||b). Generalised KL is sum(p log(p/q)−p+q), with zero handled explicitly. Marginal penalties allow deficit and excess; a marginal may exceed its initial workload. None of these quantities is a correspondence probability.
The logarithmic kernel is −C/epsilon. We alternate logu=fa(loga−logsumexp(logK+logv)) and logv=fb(logb−logsumexp(logKᵀ+logu)), where fa=rho_a/(rho_a+epsilon). Infinite rho gives fa=1 for the balanced case. Stabilisation subtracts the maximum in logsumexp. Zero masses are removed from the subproblem and then restored as zero rows or columns. All-zero totals, non-finite data, or invalid parameters are rejected; failure to converge is a readable error.
Convergence requires a maximum change in logarithmic potentials of at most 1e-11, under 20,000 iterations, and balanced marginals at 1e-8. The small 2×2 case is compared with POT 0.9.7 in FP64, with explicit reg_type='entropy' and plan tolerance 1e-8. The kl default uses another regularisation reference. Both plans are evaluated using our same objective formula; a total_cost reported by POT is not blindly treated as this constant convention.
Calibrate before testing
Epsilon ranges over 0.03 and 0.1; rho over 0.1, 1, and 10. Link thresholds apply to Πij/ai, and a review threshold to sum_jΠij/ai. These ratios can exceed 1. They are neither normalised as probabilities nor called confidence. business_scope compatibility filters incompatible financial proposals as EXCLUDED; the solver itself uses finite costs, and this rule is not a hidden transport constraint.
The grids and their order are fixed. Across the 12 calibration pairs, each method selects multiple-link micro-F1, then the first grid element in a tie. The 12 test pairs select no threshold, epsilon, or rho. Test plans can be calculated without gold data; their metrics are calculated only after calibration.
Proposals are compared with the gold set of links: TP/FP/FN, precision, recall, F1, source-concept coverage, false links on the category without an equivalent, REVIEW, and EXCLUDED. Outputs show marginals, deficits, excesses, objective, and solver residual. CPU solver time is separate from GPU encoding. A better score on these small taxonomies does not justify an automatic migration.
Exercise with fixed embeddings
Test pair 12 is selected before the results. Multiply the mass of Workshop visits by ten, then renormalise a without changing b, embeddings, or selected parameters. Recalculate balanced and unbalanced transport. Top-1 remains textual; the plans may change. Read the observed values without requiring an improvement, then compare the definitions before any human migration decision. The main outputs remain protected by hash.
Step 01 — Load taxonomies as identified concepts
SKOS, labels, and hierarchy
Error: Concepts are merged based on their label alone
Correction: Keep the URI and source taxonomy; reject cycles in the minimum model.
File src/ged_migration_taxonomies/data.py
import hashlib, json, random
from .config import save_json
DOMAINS = [
"water pumping",
"railway switches",
"solar panels",
"industrial cooling",
"harbour cranes",
"food conveyors",
"hospital ventilation",
"urban elevators",
"hydraulic presses",
"optical sensors",
"textile looms",
"district heating",
"wind turbines",
"laboratory freezers",
"mining drills",
"airport lighting",
"marine generators",
"paper rollers",
"waste sorting",
"telecom antennas",
"irrigation gates",
"battery storage",
"ceramic furnaces",
"robotic welders",
]
SOURCE = [
(
"Visites atelier",
"Scheduled routine and exceptional site visits for equipment maintenance.",
),
("Inspections", "Safety inspections and conformity observations."),
(
"Étalonnage",
"Calibration of measurement instruments against reference standards.",
),
("Remises en état", "Repair of defective components and restoration of service."),
("Commandes neuves", "Purchase orders for new equipment from external suppliers."),
("Renouvellements", "Replacement equipment procurement from external suppliers."),
(
"Ancien registre papier",
"Retired paper visitor sign-in book; no maintenance content.",
),
(
"Comptes rendus",
"Technical reports covering completed jobs and incident investigations.",
),
]
TARGET = [
("Tournées programmées", "Routine scheduled equipment maintenance visits."),
("Déplacements urgents", "Exceptional unscheduled equipment maintenance visits."),
("Contrôles sécurité", "Safety inspection records and conformity observations."),
("Réglage métrologique", "Instrument calibration against reference standards."),
("Réparations", "Restoration of defective components to working service."),
(
"Approvisionnements",
"Purchase and replacement equipment procurement from suppliers.",
),
("Rapports de travaux", "Technical reports about completed maintenance work."),
("Analyses incidents", "Technical investigation reports concerning incidents."),
("Archives financières", "Historical financial ledger and payment records."),
("Innovation", "Proposals for new engineering research projects."),
]
LINKS = [[0, 1], [2], [3], [4], [5], [5], [], [6, 7]]
def validate_broader(concepts):
ids = {c["id"] for c in concepts}
if len(ids) != len(concepts):
raise ValueError("Duplicate concept ID")
byid = {c["id"]: c for c in concepts}
for c in concepts:
seen = {c["id"]}
parent = c["broader"]
while parent is not None:
if parent not in byid or parent in seen:
raise ValueError("Invalid or cyclic broader relation")
seen.add(parent)
parent = byid[parent]["broader"]
def ancestors(concept, concepts):
validate_broader(concepts)
byid = {c["id"]: c for c in concepts}
values = []
parent = concept["broader"]
while parent is not None:
c = byid[parent]
values.append(c["label"] + ". " + c["definition"])
parent = c["broader"]
return (
" Ancestor: ".join(values)
if values
else "Root concept of this independent classification scheme."
)
def prepare(config):
pairs = []
gold = []
pieces = []
templates = {}
for family, domain in enumerate(DOMAINS):
split = "calibration" if family < 12 else "test"
template = (
f"Independent {domain} vocabulary revision {family+1}."
if split == "calibration"
else f"The {domain} documentation office defines scope {family+1} below."
)
templates[str(family)] = {
"id": f"template-{family:02d}",
"split": split,
"text": template,
}
source = []
target = []
for side, specs, parents in [
("s", SOURCE, [None, 0, 0, 0, None, 4, None, 0]),
("t", TARGET, [None, 0, 0, 2, 0, None, 0, 6, None, None]),
]:
values = []
for i, (label, definition) in enumerate(specs):
values.append(
{
"id": f"P{family:02d}-{side}{i}",
"label": label,
"definition": f"{template} {definition} Domain: {domain}.",
"broader": (
None
if parents[i] is None
else f"P{family:02d}-{side}{parents[i]}"
),
"business_scope": (
"financial" if side == "t" and i == 8 else "technical"
),
"active": True,
}
)
validate_broader(values)
if side == "s":
source = values
else:
target = values
rng = random.Random(17000 + family)
src = list(range(8)) + [rng.randrange(8) for _ in range(56)]
tgt = list(range(10)) + [rng.randrange(10) for _ in range(54)]
rng.shuffle(src)
rng.shuffle(tgt)
for j, (a, b) in enumerate(zip(src, tgt)):
body = f"Inventory note {family:02d}-{j:02d} about {domain}. Independently recorded source and destination filing loads, not an alignment annotation."
pieces.append(
{
"piece_id": f"piece-{family:02d}-{j:02d}",
"pair": family,
"text": body,
"text_sha256": hashlib.sha256(body.encode()).hexdigest(),
"source_bucket": source[a]["id"],
"target_bucket": target[b]["id"],
}
)
a = [src.count(i) / 64 for i in range(8)]
b = [tgt.count(i) / 64 for i in range(10)]
pair = {
"pair": family,
"split": split,
"domain": domain,
"template_id": f"template-{family:02d}",
"source": source,
"target": target,
"a": a,
"b": b,
"mass_source": "64 independently simulated filing-load notes; not gold link counts",
}
pairs.append(pair)
gold.append(
{
"pair": family,
"split": split,
"links": [
[source[i]["id"], target[j]["id"]]
for i, links in enumerate(LINKS)
for j in links
],
"without_equivalent": [source[6]["id"]],
}
)
save_json("data/generated/input/pages.json", pairs)
save_json("data/generated/input/pieces.json", pieces)
save_json("data/generated/input/templates.json", templates)
save_json("data/generated/labels/gold.json", gold)
save_json(
"data/generated/labels/validation.json",
[g for g in gold if g["split"] == "calibration"],
)
save_json(
"data/generated/manifest.json",
{
"pairs": 24,
"source_concepts": 192,
"target_concepts": 240,
"pieces": 1536,
"calibration_pairs": 12,
"test_pairs": 12,
"model_features_have_gold": False,
"shared_semantic_relation_pattern": True,
"domain_and_template_disjoint": True,
"limits": "Concept relation pattern and synonym prototypes shared across pairs; distinct domains/templates do not make the task fully out-of-distribution.",
},
)
Step 02 — Build an initial textual correspondence
Embeddings and symmetric cost
Error: Gold correspondences are used for ancestor context
Correction: Build context from broader only, with no access to gold mapping.
File src/ged_migration_taxonomies/features.py
import numpy as np
from .encoder import Encoder
from .data import ancestors
from .config import read_json, save_json
def baseline(config):
pairs = read_json("data/generated/input/pages.json")
encoder = Encoder(config)
rows = []
for pair in pairs:
concepts = pair["source"] + pair["target"]
texts = [c["label"] + ". " + c["definition"] for c in concepts]
contexts = [
ancestors(c, pair["source"] if i < 8 else pair["target"])
for i, c in enumerate(concepts)
]
values = encoder.encode_queries(texts).astype(np.float64)
context = encoder.encode_queries(contexts).astype(np.float64)
cosine = np.clip(values[:8] @ values[8:].T, -1, 1)
ancestor_cosine = np.clip(context[:8] @ context[8:].T, -1, 1)
cost = 1 - cosine + config["ancestor_weight"] * (1 - ancestor_cosine)
rows.append(
{
"pair": pair["pair"],
"concept_ids": [c["id"] for c in concepts],
"texts": texts,
"contexts": contexts,
"embeddings": values.tolist(),
"ancestor_embeddings": context.tolist(),
"cosine": cosine.tolist(),
"ancestor_cosine": ancestor_cosine.tolist(),
"cost": cost.tolist(),
"top1": [int(np.argmax(row)) for row in cosine],
}
)
save_json(
"outputs/features.json",
{
"rows": rows,
"encoder": encoder.identity,
"encoding_calls": encoder.calls,
"labels_visible": False,
"ancestor_weight": config["ancestor_weight"],
},
)
Step 03 — Derive transport scalings
Entropy, marginal KL, and log space
Error: Compare reg_type='kl' with the practical exercise's entropic calculation
Correction: Pass reg_type='entropy' to the reference and explain this convention difference.
File src/ged_migration_taxonomies/transport.py
"""Entropy-regularized transport in float64 log space; explicit mass semantics."""
import math, time, warnings
import numpy as np
def logsumexp(values, axis):
maximum = np.max(values, axis=axis, keepdims=True)
return (
maximum + np.log(np.exp(values - maximum).sum(axis=axis, keepdims=True))
).squeeze(axis)
def generalized_kl(p, q):
p = np.asarray(p, dtype=np.float64)
q = np.asarray(q, dtype=np.float64)
if (
np.any(p < 0)
or np.any(q < 0)
or not np.isfinite(p).all()
or not np.isfinite(q).all()
):
raise ValueError("Invalid KL masses")
if np.any((q == 0) & (p > 0)):
return math.inf
active = p > 0
return float(
np.sum(p[active] * np.log(p[active] / q[active])) - np.sum(p) + np.sum(q)
)
def objective(plan, cost, a, b, epsilon, rho_a, rho_b):
positive = plan > 0
entropy = float(np.sum(plan[positive] * (np.log(plan[positive]) - 1)))
linear = float(np.sum(plan * cost))
left = generalized_kl(plan.sum(1), a)
right = generalized_kl(plan.sum(0), b)
value = linear + epsilon * entropy
if math.isfinite(rho_a):
value += rho_a * left
if math.isfinite(rho_b):
value += rho_b * right
return {
"linear": linear,
"negative_entropy": entropy,
"row_KL": left,
"column_KL": right,
"regularized": value,
"convention": "P*(logP-1); generalized KL includes -p+q",
}
def sinkhorn_unbalanced(
a, b, cost, epsilon, rho_a, rho_b=None, tolerance=1e-11, max_iterations=20000
):
started = time.perf_counter()
a = np.asarray(a, dtype=np.float64)
b = np.asarray(b, dtype=np.float64)
cost = np.asarray(cost, dtype=np.float64)
rho_b = rho_a if rho_b is None else rho_b
if a.ndim != 1 or b.ndim != 1 or cost.shape != (len(a), len(b)):
raise ValueError("Mass/cost dimensions")
if (
not np.isfinite(a).all()
or not np.isfinite(b).all()
or not np.isfinite(cost).all()
or np.any(a < 0)
or np.any(b < 0)
or a.sum() <= 0
or b.sum() <= 0
):
raise ValueError("Finite nonnegative masses with positive total required")
if (
not math.isfinite(epsilon)
or epsilon <= 0
or any(math.isnan(r) or r <= 0 for r in [rho_a, rho_b])
or tolerance <= 0
or max_iterations < 1
):
raise ValueError("Invalid regularization/convergence configuration")
balanced = math.isinf(rho_a) and math.isinf(rho_b)
if balanced and abs(a.sum() - b.sum()) > 1e-12:
raise ValueError("Balanced totals differ")
active_a = a > 0
active_b = b > 0
aa = a[active_a]
bb = b[active_b]
C = cost[np.ix_(active_a, active_b)]
kernel = -C / epsilon
lu = np.zeros(len(aa))
lv = np.zeros(len(bb))
fa = 1.0 if math.isinf(rho_a) else rho_a / (rho_a + epsilon)
fb = 1.0 if math.isinf(rho_b) else rho_b / (rho_b + epsilon)
residual = math.inf
for iteration in range(1, max_iterations + 1):
new_u = fa * (np.log(aa) - logsumexp(kernel + lv[None, :], axis=1))
new_v = fb * (np.log(bb) - logsumexp(kernel + new_u[:, None], axis=0))
residual = max(
float(np.max(np.abs(new_u - lu))), float(np.max(np.abs(new_v - lv)))
)
lu, lv = new_u, new_v
if not np.isfinite(lu).all() or not np.isfinite(lv).all():
raise FloatingPointError("Non-finite Sinkhorn dual iterate")
if residual <= tolerance:
break
else:
raise RuntimeError(
f"Sinkhorn did not converge after {max_iterations} iterations; residual={residual}"
)
small = np.exp(lu[:, None] + kernel + lv[None, :])
plan = np.zeros_like(cost)
plan[np.ix_(active_a, active_b)] = small
if not np.isfinite(plan).all():
raise FloatingPointError("Non-finite transport plan")
row = plan.sum(1)
col = plan.sum(0)
if balanced and max(np.max(np.abs(row - a)), np.max(np.abs(col - b))) > 1e-8:
raise RuntimeError("Balanced marginal residual too large")
return {
"plan": plan.tolist(),
"row_mass": row.tolist(),
"column_mass": col.tolist(),
"row_deficit": np.maximum(a - row, 0).tolist(),
"row_excess": np.maximum(row - a, 0).tolist(),
"column_deficit": np.maximum(b - col, 0).tolist(),
"column_excess": np.maximum(col - b, 0).tolist(),
"objective": objective(plan, cost, a, b, epsilon, rho_a, rho_b),
"iterations": iteration,
"residual": residual,
"seconds": time.perf_counter() - started,
"active_rows": np.flatnonzero(active_a).tolist(),
"active_columns": np.flatnonzero(active_b).tolist(),
"balanced": balanced,
"epsilon": epsilon,
"rho_a": None if math.isinf(rho_a) else rho_a,
"rho_b": None if math.isinf(rho_b) else rho_b,
}
def compare_pot(a, b, cost, epsilon, rho_a, rho_b=None, tolerance=1e-8):
import ot
rho_b = rho_a if rho_b is None else rho_b
a = np.asarray(a, dtype=np.float64)
b = np.asarray(b, dtype=np.float64)
cost = np.asarray(cost, dtype=np.float64)
ours = sinkhorn_unbalanced(a, b, cost, epsilon, rho_a, rho_b)
ia = a > 0
ib = b > 0
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
small, info = ot.unbalanced.sinkhorn_unbalanced(
a[ia],
b[ib],
cost[np.ix_(ia, ib)],
epsilon,
(rho_a, rho_b),
reg_type="entropy",
numItermax=20000,
stopThr=1e-13,
log=True,
)
reference = np.zeros_like(cost)
reference[np.ix_(ia, ib)] = small
error = float(np.max(np.abs(reference - np.asarray(ours["plan"]))))
if error > tolerance:
raise AssertionError(f"POT plan discrepancy {error} > {tolerance}")
return {
"ours": ours,
"reference_plan": reference.tolist(),
"max_plan_error": error,
"tolerance": tolerance,
"reg_type": "entropy",
"pot_version": ot.__version__,
"pot_reported_total_cost": float(info["total_cost"]),
"reference_objective_recomputed": objective(
reference, cost, a, b, epsilon, rho_a, rho_b
),
"warnings": [str(w.message) for w in caught],
}
Step 04 — Solve plans with merges and splits
Marginal relaxation and review
Error: Every row is normalised to 1 and then called confidence
Correction: Retain raw masses, marginals, costs, and calibrated review rules.
File src/ged_migration_taxonomies/run.py
import itertools, math
import numpy as np
from .config import read_json, save_json, sha
from .transport import sinkhorn_unbalanced, compare_pot
from .proposals import propose, metrics
def inspect_mechanism(config):
a = [0.7, 0.3]
b = [0.4, 0.6]
cost = [[0.1, 0.9], [0.8, 0.2]]
reference = compare_pot(a, b, cost, 0.1, 1.0, 0.7)
permutation = sinkhorn_unbalanced(
a[::-1], b[::-1], np.asarray(cost)[::-1, ::-1], 0.1, 1.0, 0.7
)
if (
np.max(
np.abs(
np.asarray(reference["ours"]["plan"])
- np.asarray(permutation["plan"])[::-1, ::-1]
)
)
> 1e-10
):
raise AssertionError("Permutation failure")
zero = sinkhorn_unbalanced([1.0, 0.0], [0.4, 0.6], cost, 0.1, 1.0)
save_json(
"outputs/mechanism.json",
{
"a": a,
"b": b,
"cost": cost,
"reference": reference,
"permutation": permutation,
"zero_mass": zero,
"status": "PASS",
},
)
def compute_all(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
solutions = []
for pair in pairs:
for eps in config["epsilon_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "balanced",
"epsilon": eps,
"rho": None,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
math.inf,
),
}
)
for rho in config["rho_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "unbalanced",
"epsilon": eps,
"rho": rho,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
rho,
tolerance=config["solver_tolerance"],
max_iterations=config["max_iterations"],
),
}
)
save_json("outputs/plans.json", solutions)
def calibrate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/validation.json")}
if set(gold) != set(range(12)):
raise ValueError("Calibration gold only required")
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
choices = {}
tables = {}
for variant in ["top1", "balanced", "unbalanced"]:
options = (
[(None, None, None, None, t) for t in config["cosine_threshold_grid"]]
if variant == "top1"
else [
(e, r, l, m, None)
for e in config["epsilon_grid"]
for r in ([None] if variant == "balanced" else config["rho_grid"])
for l in config["link_mass_ratio_grid"]
for m in config["minimum_row_mass_ratio_grid"]
]
)
rows = []
for eps, rho, link, row, cos in options:
total = {"tp": 0, "fp": 0, "fn": 0}
for pair in pairs[:12]:
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, eps, rho]
)
result = metrics(
propose(
pair, features[pair["pair"]], variant, solution, link, row, cos
),
gold[pair["pair"]],
)
for k in total:
total[k] += result[k]
score = (
2 * total["tp"] / (2 * total["tp"] + total["fp"] + total["fn"])
if 2 * total["tp"] + total["fp"] + total["fn"]
else 0.0
)
rows.append(
{
"epsilon": eps,
"rho": rho,
"link_ratio": link,
"row_ratio": row,
"cosine_threshold": cos,
"counts": total,
"micro_F1": score,
}
)
# Stable predefined grid order breaks ties, independent of test labels.
winner = max(range(len(rows)), key=lambda i: (rows[i]["micro_F1"], -i))
choices[variant] = dict(rows[winner], grid_index=winner)
tables[variant] = rows
save_json(
"outputs/calibration.json",
{
"pairs": list(range(12)),
"choices": choices,
"grid": tables,
"tie": "first in predefined grid",
"test_gold_visible": False,
},
)
def evaluate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/gold.json")}
selected = read_json("outputs/calibration.json")["choices"]
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
rows = []
for pair in pairs:
for variant, choice in selected.items():
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, choice["epsilon"], choice["rho"]]
)
proposals = propose(
pair,
features[pair["pair"]],
variant,
solution,
choice["link_ratio"],
choice["row_ratio"],
choice["cosine_threshold"],
)
rows.append(
{
"pair": pair["pair"],
"split": pair["split"],
"variant": variant,
"proposals": proposals,
"metrics": metrics(proposals, gold[pair["pair"]]),
"choice": choice,
}
)
save_json(
"outputs/evaluation.json",
{
"rows": rows,
"populations": {"calibration_pairs": 12, "test_pairs": 12},
"applied": False,
"semantics": "proposals only; transported mass is not probability",
},
)
def exercise(config):
protected = {
p: sha(p)
for p in [
"outputs/features.json",
"outputs/calibration.json",
"outputs/evaluation.json",
]
}
pair = read_json("data/generated/input/pages.json")[12]
original = list(pair["a"])
pair["a"][0] *= 10
total = sum(pair["a"])
pair["a"] = [v / total for v in pair["a"]]
feature = read_json("outputs/features.json")["rows"][12]
gold = read_json("data/generated/labels/gold.json")[12]
choices = read_json("outputs/calibration.json")["choices"]
rows = []
for variant, c in choices.items():
solution = (
None
if variant == "top1"
else sinkhorn_unbalanced(
pair["a"],
pair["b"],
feature["cost"],
c["epsilon"],
math.inf if variant == "balanced" else c["rho"],
)
)
proposals = propose(
pair,
feature,
variant,
solution,
c["link_ratio"],
c["row_ratio"],
c["cosine_threshold"],
)
rows.append(
{
"variant": variant,
"solution": solution,
"proposals": proposals,
"metrics": metrics(proposals, gold),
}
)
save_json(
"outputs/exercise.json",
{
"pair": 12,
"source_label": pair["source"][0]["label"],
"original_a": original,
"changed_a": pair["a"],
"b": pair["b"],
"multiplier": 10,
"rows": rows,
"protected_sha256": protected,
"main_unchanged": all(sha(p) == v for p, v in protected.items()),
"applied": False,
},
)
Step 05 — Evaluate mapping, not only its cost
Many-to-many gold data
Error: A low objective is interpreted as a good ontology
Correction: Compare with gold data and analyse definition errors together with their texts.
File src/ged_migration_taxonomies/run.py
import itertools, math
import numpy as np
from .config import read_json, save_json, sha
from .transport import sinkhorn_unbalanced, compare_pot
from .proposals import propose, metrics
def inspect_mechanism(config):
a = [0.7, 0.3]
b = [0.4, 0.6]
cost = [[0.1, 0.9], [0.8, 0.2]]
reference = compare_pot(a, b, cost, 0.1, 1.0, 0.7)
permutation = sinkhorn_unbalanced(
a[::-1], b[::-1], np.asarray(cost)[::-1, ::-1], 0.1, 1.0, 0.7
)
if (
np.max(
np.abs(
np.asarray(reference["ours"]["plan"])
- np.asarray(permutation["plan"])[::-1, ::-1]
)
)
> 1e-10
):
raise AssertionError("Permutation failure")
zero = sinkhorn_unbalanced([1.0, 0.0], [0.4, 0.6], cost, 0.1, 1.0)
save_json(
"outputs/mechanism.json",
{
"a": a,
"b": b,
"cost": cost,
"reference": reference,
"permutation": permutation,
"zero_mass": zero,
"status": "PASS",
},
)
def compute_all(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
solutions = []
for pair in pairs:
for eps in config["epsilon_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "balanced",
"epsilon": eps,
"rho": None,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
math.inf,
),
}
)
for rho in config["rho_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "unbalanced",
"epsilon": eps,
"rho": rho,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
rho,
tolerance=config["solver_tolerance"],
max_iterations=config["max_iterations"],
),
}
)
save_json("outputs/plans.json", solutions)
def calibrate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/validation.json")}
if set(gold) != set(range(12)):
raise ValueError("Calibration gold only required")
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
choices = {}
tables = {}
for variant in ["top1", "balanced", "unbalanced"]:
options = (
[(None, None, None, None, t) for t in config["cosine_threshold_grid"]]
if variant == "top1"
else [
(e, r, l, m, None)
for e in config["epsilon_grid"]
for r in ([None] if variant == "balanced" else config["rho_grid"])
for l in config["link_mass_ratio_grid"]
for m in config["minimum_row_mass_ratio_grid"]
]
)
rows = []
for eps, rho, link, row, cos in options:
total = {"tp": 0, "fp": 0, "fn": 0}
for pair in pairs[:12]:
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, eps, rho]
)
result = metrics(
propose(
pair, features[pair["pair"]], variant, solution, link, row, cos
),
gold[pair["pair"]],
)
for k in total:
total[k] += result[k]
score = (
2 * total["tp"] / (2 * total["tp"] + total["fp"] + total["fn"])
if 2 * total["tp"] + total["fp"] + total["fn"]
else 0.0
)
rows.append(
{
"epsilon": eps,
"rho": rho,
"link_ratio": link,
"row_ratio": row,
"cosine_threshold": cos,
"counts": total,
"micro_F1": score,
}
)
# Stable predefined grid order breaks ties, independent of test labels.
winner = max(range(len(rows)), key=lambda i: (rows[i]["micro_F1"], -i))
choices[variant] = dict(rows[winner], grid_index=winner)
tables[variant] = rows
save_json(
"outputs/calibration.json",
{
"pairs": list(range(12)),
"choices": choices,
"grid": tables,
"tie": "first in predefined grid",
"test_gold_visible": False,
},
)
def evaluate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/gold.json")}
selected = read_json("outputs/calibration.json")["choices"]
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
rows = []
for pair in pairs:
for variant, choice in selected.items():
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, choice["epsilon"], choice["rho"]]
)
proposals = propose(
pair,
features[pair["pair"]],
variant,
solution,
choice["link_ratio"],
choice["row_ratio"],
choice["cosine_threshold"],
)
rows.append(
{
"pair": pair["pair"],
"split": pair["split"],
"variant": variant,
"proposals": proposals,
"metrics": metrics(proposals, gold[pair["pair"]]),
"choice": choice,
}
)
save_json(
"outputs/evaluation.json",
{
"rows": rows,
"populations": {"calibration_pairs": 12, "test_pairs": 12},
"applied": False,
"semantics": "proposals only; transported mass is not probability",
},
)
def exercise(config):
protected = {
p: sha(p)
for p in [
"outputs/features.json",
"outputs/calibration.json",
"outputs/evaluation.json",
]
}
pair = read_json("data/generated/input/pages.json")[12]
original = list(pair["a"])
pair["a"][0] *= 10
total = sum(pair["a"])
pair["a"] = [v / total for v in pair["a"]]
feature = read_json("outputs/features.json")["rows"][12]
gold = read_json("data/generated/labels/gold.json")[12]
choices = read_json("outputs/calibration.json")["choices"]
rows = []
for variant, c in choices.items():
solution = (
None
if variant == "top1"
else sinkhorn_unbalanced(
pair["a"],
pair["b"],
feature["cost"],
c["epsilon"],
math.inf if variant == "balanced" else c["rho"],
)
)
proposals = propose(
pair,
feature,
variant,
solution,
c["link_ratio"],
c["row_ratio"],
c["cosine_threshold"],
)
rows.append(
{
"variant": variant,
"solution": solution,
"proposals": proposals,
"metrics": metrics(proposals, gold),
}
)
save_json(
"outputs/exercise.json",
{
"pair": 12,
"source_label": pair["source"][0]["label"],
"original_a": original,
"changed_a": pair["a"],
"b": pair["b"],
"multiplier": 10,
"rows": rows,
"protected_sha256": protected,
"main_unchanged": all(sha(p) == v for p, v in protected.items()),
"applied": False,
},
)
Step 06 — Change the weight of a dominant category
Sensitivity to distributions
Error: The solver is presented as reading business intent
Correction: Show the workload assumptions and submit changes to the review file.
File src/ged_migration_taxonomies/run.py
import itertools, math
import numpy as np
from .config import read_json, save_json, sha
from .transport import sinkhorn_unbalanced, compare_pot
from .proposals import propose, metrics
def inspect_mechanism(config):
a = [0.7, 0.3]
b = [0.4, 0.6]
cost = [[0.1, 0.9], [0.8, 0.2]]
reference = compare_pot(a, b, cost, 0.1, 1.0, 0.7)
permutation = sinkhorn_unbalanced(
a[::-1], b[::-1], np.asarray(cost)[::-1, ::-1], 0.1, 1.0, 0.7
)
if (
np.max(
np.abs(
np.asarray(reference["ours"]["plan"])
- np.asarray(permutation["plan"])[::-1, ::-1]
)
)
> 1e-10
):
raise AssertionError("Permutation failure")
zero = sinkhorn_unbalanced([1.0, 0.0], [0.4, 0.6], cost, 0.1, 1.0)
save_json(
"outputs/mechanism.json",
{
"a": a,
"b": b,
"cost": cost,
"reference": reference,
"permutation": permutation,
"zero_mass": zero,
"status": "PASS",
},
)
def compute_all(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
solutions = []
for pair in pairs:
for eps in config["epsilon_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "balanced",
"epsilon": eps,
"rho": None,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
math.inf,
),
}
)
for rho in config["rho_grid"]:
solutions.append(
{
"pair": pair["pair"],
"variant": "unbalanced",
"epsilon": eps,
"rho": rho,
"solution": sinkhorn_unbalanced(
pair["a"],
pair["b"],
features[pair["pair"]]["cost"],
eps,
rho,
tolerance=config["solver_tolerance"],
max_iterations=config["max_iterations"],
),
}
)
save_json("outputs/plans.json", solutions)
def calibrate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/validation.json")}
if set(gold) != set(range(12)):
raise ValueError("Calibration gold only required")
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
choices = {}
tables = {}
for variant in ["top1", "balanced", "unbalanced"]:
options = (
[(None, None, None, None, t) for t in config["cosine_threshold_grid"]]
if variant == "top1"
else [
(e, r, l, m, None)
for e in config["epsilon_grid"]
for r in ([None] if variant == "balanced" else config["rho_grid"])
for l in config["link_mass_ratio_grid"]
for m in config["minimum_row_mass_ratio_grid"]
]
)
rows = []
for eps, rho, link, row, cos in options:
total = {"tp": 0, "fp": 0, "fn": 0}
for pair in pairs[:12]:
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, eps, rho]
)
result = metrics(
propose(
pair, features[pair["pair"]], variant, solution, link, row, cos
),
gold[pair["pair"]],
)
for k in total:
total[k] += result[k]
score = (
2 * total["tp"] / (2 * total["tp"] + total["fp"] + total["fn"])
if 2 * total["tp"] + total["fp"] + total["fn"]
else 0.0
)
rows.append(
{
"epsilon": eps,
"rho": rho,
"link_ratio": link,
"row_ratio": row,
"cosine_threshold": cos,
"counts": total,
"micro_F1": score,
}
)
# Stable predefined grid order breaks ties, independent of test labels.
winner = max(range(len(rows)), key=lambda i: (rows[i]["micro_F1"], -i))
choices[variant] = dict(rows[winner], grid_index=winner)
tables[variant] = rows
save_json(
"outputs/calibration.json",
{
"pairs": list(range(12)),
"choices": choices,
"grid": tables,
"tie": "first in predefined grid",
"test_gold_visible": False,
},
)
def evaluate(config):
pairs = read_json("data/generated/input/pages.json")
features = {r["pair"]: r for r in read_json("outputs/features.json")["rows"]}
plans = read_json("outputs/plans.json")
gold = {g["pair"]: g for g in read_json("data/generated/labels/gold.json")}
selected = read_json("outputs/calibration.json")["choices"]
lookup = {
(r["pair"], r["variant"], r["epsilon"], r["rho"]): r["solution"] for r in plans
}
rows = []
for pair in pairs:
for variant, choice in selected.items():
solution = (
None
if variant == "top1"
else lookup[pair["pair"], variant, choice["epsilon"], choice["rho"]]
)
proposals = propose(
pair,
features[pair["pair"]],
variant,
solution,
choice["link_ratio"],
choice["row_ratio"],
choice["cosine_threshold"],
)
rows.append(
{
"pair": pair["pair"],
"split": pair["split"],
"variant": variant,
"proposals": proposals,
"metrics": metrics(proposals, gold[pair["pair"]]),
"choice": choice,
}
)
save_json(
"outputs/evaluation.json",
{
"rows": rows,
"populations": {"calibration_pairs": 12, "test_pairs": 12},
"applied": False,
"semantics": "proposals only; transported mass is not probability",
},
)
def exercise(config):
protected = {
p: sha(p)
for p in [
"outputs/features.json",
"outputs/calibration.json",
"outputs/evaluation.json",
]
}
pair = read_json("data/generated/input/pages.json")[12]
original = list(pair["a"])
pair["a"][0] *= 10
total = sum(pair["a"])
pair["a"] = [v / total for v in pair["a"]]
feature = read_json("outputs/features.json")["rows"][12]
gold = read_json("data/generated/labels/gold.json")[12]
choices = read_json("outputs/calibration.json")["choices"]
rows = []
for variant, c in choices.items():
solution = (
None
if variant == "top1"
else sinkhorn_unbalanced(
pair["a"],
pair["b"],
feature["cost"],
c["epsilon"],
math.inf if variant == "balanced" else c["rho"],
)
)
proposals = propose(
pair,
feature,
variant,
solution,
c["link_ratio"],
c["row_ratio"],
c["cosine_threshold"],
)
rows.append(
{
"variant": variant,
"solution": solution,
"proposals": proposals,
"metrics": metrics(proposals, gold),
}
)
save_json(
"outputs/exercise.json",
{
"pair": 12,
"source_label": pair["source"][0]["label"],
"original_a": original,
"changed_a": pair["a"],
"b": pair["b"],
"multiplier": 10,
"rows": rows,
"protected_sha256": protected,
"main_unchanged": all(sha(p) == v for p, v in protected.items()),
"applied": False,
},
)
Sources
Chizat et al., Scaling Algorithms for Unbalanced Transport Problems describes the scaling family. POT 0.9.7 documents the reference solver and its regularisation choice. SKOS Reference distinguishes organisational and mapping relations; the practical exercise's proposals do not assert them. Pinned E5.
Interpretation of the development run
Across the 12 test pairs, mean F1 per pair is 0.875 for Top-1, 0.591636 for balanced transport, and 0.742633 for unbalanced transport. Total TP/FP/FN values are 84/0/24, 49/9/59, and 74/18/34 respectively. The two transport methods create 2 and 1 false links respectively on concepts without an equivalent; Top-1 creates 0. All 192 plans converge and the POT 2×2 check has a maximum error of 2.4871e-12. The solver is verified, with no relevance gain over the baseline. For this corpus, Top-1 is the choice supported by measured quality; a need to distribute masses globally would require a new business evaluation.
The exercise multiplies the workload of source 0 by ten and then renormalises a; costs, target marginal, and thresholds remain fixed. A change in transport masses does not represent a change in link probability. The two heatmaps have their own colour scales: compare the bar values, not colours alone.
Development observations
Values recalculated from the raw outputs of this run. They are not improvement thresholds; the workflow from the ZIP recalculates them over the same populations.
{
"status": "PASS",
"pairs": 24,
"calibration_pairs": 12,
"test_pairs": 12,
"source_concepts": 192,
"target_concepts": 240,
"pieces": 1536,
"plans": 192,
"calibration_choices": {
"top1": {
"epsilon": null,
"rho": null,
"link_ratio": null,
"row_ratio": null,
"cosine_threshold": 0.95,
"counts": {
"tp": 84,
"fp": 0,
"fn": 24
},
"micro_F1": 0.875,
"grid_index": 19
},
"balanced": {
"epsilon": 0.03,
"rho": null,
"link_ratio": 0.3,
"row_ratio": 0.1,
"cosine_threshold": null,
"counts": {
"tp": 66,
"fp": 3,
"fn": 42
},
"micro_F1": 0.7457627118644068,
"grid_index": 4
},
"unbalanced": {
"epsilon": 0.03,
"rho": 0.1,
"link_ratio": 0.3,
"row_ratio": 0.1,
"cosine_threshold": null,
"counts": {
"tp": 83,
"fp": 4,
"fn": 25
},
"micro_F1": 0.8512820512820513,
"grid_index": 4
}
},
"test_metrics": [
{
"pair": 12,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 12,
"variant": "balanced",
"true_links": 9,
"proposed_links": 4,
"tp": 4,
"fp": 0,
"fn": 5,
"precision": 1.0,
"recall": 0.4444444444444444,
"F1": 0.6153846153846154,
"covered_source_concepts": 4,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 12,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 6,
"tp": 6,
"fp": 0,
"fn": 3,
"precision": 1.0,
"recall": 0.6666666666666666,
"F1": 0.8,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 13,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 13,
"variant": "balanced",
"true_links": 9,
"proposed_links": 4,
"tp": 4,
"fp": 0,
"fn": 5,
"precision": 1.0,
"recall": 0.4444444444444444,
"F1": 0.6153846153846154,
"covered_source_concepts": 4,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 13,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 8,
"tp": 7,
"fp": 1,
"fn": 2,
"precision": 0.875,
"recall": 0.7777777777777778,
"F1": 0.8235294117647058,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 14,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 14,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 5,
"fp": 0,
"fn": 4,
"precision": 1.0,
"recall": 0.5555555555555556,
"F1": 0.7142857142857143,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 14,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 9,
"tp": 7,
"fp": 2,
"fn": 2,
"precision": 0.7777777777777778,
"recall": 0.7777777777777778,
"F1": 0.7777777777777778,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 15,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 15,
"variant": "balanced",
"true_links": 9,
"proposed_links": 6,
"tp": 3,
"fp": 3,
"fn": 6,
"precision": 0.5,
"recall": 0.3333333333333333,
"F1": 0.4,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 1,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 15,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 11,
"tp": 6,
"fp": 5,
"fn": 3,
"precision": 0.5454545454545454,
"recall": 0.6666666666666666,
"F1": 0.6,
"covered_source_concepts": 8,
"source_concepts": 8,
"false_links_without_equivalent": 1,
"without_equivalent_concepts": 1,
"review_rows": 0,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 16,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 16,
"variant": "balanced",
"true_links": 9,
"proposed_links": 4,
"tp": 4,
"fp": 0,
"fn": 5,
"precision": 1.0,
"recall": 0.4444444444444444,
"F1": 0.6153846153846154,
"covered_source_concepts": 4,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 4,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 16,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 6,
"tp": 6,
"fp": 0,
"fn": 3,
"precision": 1.0,
"recall": 0.6666666666666666,
"F1": 0.8,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 17,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 17,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 5,
"fp": 0,
"fn": 4,
"precision": 1.0,
"recall": 0.5555555555555556,
"F1": 0.7142857142857143,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 17,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 18,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 18,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 4,
"fp": 1,
"fn": 5,
"precision": 0.8,
"recall": 0.4444444444444444,
"F1": 0.5714285714285714,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 18,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 8,
"tp": 6,
"fp": 2,
"fn": 3,
"precision": 0.75,
"recall": 0.6666666666666666,
"F1": 0.7058823529411765,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 19,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 19,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 5,
"fp": 0,
"fn": 4,
"precision": 1.0,
"recall": 0.5555555555555556,
"F1": 0.7142857142857143,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 19,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 8,
"tp": 7,
"fp": 1,
"fn": 2,
"precision": 0.875,
"recall": 0.7777777777777778,
"F1": 0.8235294117647058,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 20,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 20,
"variant": "balanced",
"true_links": 9,
"proposed_links": 4,
"tp": 4,
"fp": 0,
"fn": 5,
"precision": 1.0,
"recall": 0.4444444444444444,
"F1": 0.6153846153846154,
"covered_source_concepts": 4,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 20,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 6,
"tp": 6,
"fp": 0,
"fn": 3,
"precision": 1.0,
"recall": 0.6666666666666666,
"F1": 0.8,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 21,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 21,
"variant": "balanced",
"true_links": 9,
"proposed_links": 6,
"tp": 5,
"fp": 1,
"fn": 4,
"precision": 0.8333333333333334,
"recall": 0.5555555555555556,
"F1": 0.6666666666666666,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 21,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 8,
"tp": 6,
"fp": 2,
"fn": 3,
"precision": 0.75,
"recall": 0.6666666666666666,
"F1": 0.7058823529411765,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 22,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 22,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 3,
"fp": 2,
"fn": 6,
"precision": 0.6,
"recall": 0.3333333333333333,
"F1": 0.42857142857142855,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 1,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 22,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 9,
"tp": 6,
"fp": 3,
"fn": 3,
"precision": 0.6666666666666666,
"recall": 0.6666666666666666,
"F1": 0.6666666666666666,
"covered_source_concepts": 6,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 23,
"variant": "top1",
"true_links": 9,
"proposed_links": 7,
"tp": 7,
"fp": 0,
"fn": 2,
"precision": 1.0,
"recall": 0.7777777777777778,
"F1": 0.875,
"covered_source_concepts": 7,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 1,
"excluded_rows": 0,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 23,
"variant": "balanced",
"true_links": 9,
"proposed_links": 5,
"tp": 3,
"fp": 2,
"fn": 6,
"precision": 0.6,
"recall": 0.3333333333333333,
"F1": 0.42857142857142855,
"covered_source_concepts": 4,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 3,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
},
{
"pair": 23,
"variant": "unbalanced",
"true_links": 9,
"proposed_links": 6,
"tp": 4,
"fp": 2,
"fn": 5,
"precision": 0.6666666666666666,
"recall": 0.4444444444444444,
"F1": 0.5333333333333333,
"covered_source_concepts": 5,
"source_concepts": 8,
"false_links_without_equivalent": 0,
"without_equivalent_concepts": 1,
"review_rows": 2,
"excluded_rows": 1,
"applied": false,
"skos_exactMatch_assertions": 0
}
],
"POT_max_error": 2.4871216197652757e-12,
"proposals_applied": false
}
