Language models produce variable outputs, and their quality depends on the context, the prompt, the tools, and the data. A manual test on ten pleasant examples does not allow one to choose an architecture or to secure an update. A discipline of evaluation comparable to software testing is needed, adapted to probabilistic responses.
The goal is not to reduce every quality to a score. It is to make errors visible, to compare versions on the same cases, and to define what blocks a production release.
Distinguish between benchmark and product evaluation
A public benchmark measures general capabilities on a given corpus. It helps to understand a model, but does not necessarily represent the company's vocabulary, formats, languages, and risks.
A product evaluation uses:
- real tasks;
- representative data;
- format constraints;
- tools and RAG;
- refusal policy;
- business thresholds;
- cost and latency.
The best public model may be less suited than a smaller model for a bounded task.
Break down the system
A final response depends on several components: intent classification, retrieval, prompt, model, tools, and post-processing. Evaluating them separately helps to locate a regression.
For a RAG, measure retrieval and generation. For an agent, measure tool selection, parameters, compliance with permissions, and result. For an extraction, distinguish document recognition and field value.
Build a representative case set
The game must cover:
- frequent cases;
- high-value case;
- varied formulations;
- languages;
- incomplete data;
- ambiguities;
- expected refusals;
- attacks;
- historically faulty cases.
Each case has an identifier, an intention, inputs, an expected result or a section, a risk level, and tags. Production examples are anonymized and selected according to a policy.
Pyramid combining deterministic controls, component evaluations, complete scenarios, human review, and production monitoring.
Define the quality dimensions
According to the task, evaluate:
- accuracy of facts or values;
- completeness of the necessary elements;
- faithfulness to the sources;
- format and structure;
- pertinence ;
- usefulness for the action;
- style and tone ;
- security ;
- quality of the refusal ;
- explainability or citations.
Each dimension receives a definition and examples of grades. A vague rubric produces inconsistent judges.
Deterministic controls first
Before a semantic judgment, check what can be checked by code: valid JSON, present fields, allowed values, existing references, calculations, length, language, quotations, and tool call.
These tests are fast, reproducible, and easy to integrate into CI. They eliminate errors without requiring a model to judge a format.
Build a ground truth
For extraction or classification, experts can annotate the correct result. For generation, it is often better to define required elements, forbidden elements, and a category rather than a single sentence.
Disagreements among experts are measured. If they do not agree, asking the model for a single answer may be unrealistic.
The ground truth is versioned and reviewed when the business changes.
Use human judges
Humans remain necessary for usefulness, nuances, and risks. Evaluation can be done blindly, with random order, in order to limit brand or position bias.
The evaluators receive instructions, examples, and a disagreement mechanism. a subset is double-scored to measure consistency.
Use a model as a judge with caution
An LLM judge allows more cases to be evaluated, but has its biases. It may prefer long answers, its own style, or the first option. It can be influenced by the content to be judged.
The safeguards include:
- specific criteria;
- structured output;
- anonymization of models;
- random order;
- comparison with human grades;
- several repetitions;
- monitoring of disagreements.
The judge speeds up the sorting; he does not turn an opinion into absolute truth.
Compare two versions
The paired comparison runs A and B on the same cases. The report shows gains, losses, and critical cases, not just an overall average.
The results are segmented by intent, language, and risk. An improvement on summaries can hide a regression on refusals. The sample size and uncertainty are visible.
Measuring variability
The same input can produce multiple outputs. For sensitive tasks, perform multiple repetitions and measure the stability of the format, tool choice, or response.
The temperature, the seed when available, the model version, and the parameters are recorded. A system may be generally correct but too unstable for production.
Latency and availability
Measure total time and time for each stage: recovery, model, tools, post-processing. Track median and percentiles, not just average.
Errors, timeouts, quotas, and retries are part of the evaluation. A slightly better model but often unavailable may be less useful.
Cost per successful task
The cost includes tokens, embeddings, reranking, tools, storage, and human validation. It must be reported per successful case, not just per request.
An incorrect output can trigger an expensive recovery. Volume and peak scenarios help compare routing and caching.
Assess safety
The game includes prompt injection, exfiltration, prohibited data, policy circumvention, unauthorized tools, and toxic content depending on use. The tests are carried out in a safe environment.
A critical security error blocks the launch regardless of the average score.
Set the thresholds
Each dimension has:
- objective;
- minimum acceptable;
- critical threshold;
- owner of the exception.
The thresholds come from the business risk. A financial extract may require very high accuracy and control; a creative draft allows for more variation.
Integrate evaluations into delivery
A small quick test runs with each modification. A full suite runs before release or periodically. Costs are controlled through sampling and caching of unchanged components.
The report compares to a baseline and blocks critical regressions. Changes to the prompt, model, RAG, tool, or policy are all versioned.
Monitor in production
Offline evaluations do not cover all queries. Production tracks errors, rejections, feedback, escalations, latency, and cost. Samples are reviewed according to a privacy policy.
Actual incidents become regression cases. Tracking data should not collect more information than necessary.
Avoid contamination of the game
If the game is used to constantly adjust the prompt, it becomes an implicit training game. Keeping a hidden set and renewing the cases protects the ability to generalize.
Suppliers and models may also have seen certain public benchmarks, another reason to favor clean cases.
An evaluation as a produced asset
The set of cases, the categories, the thresholds, and the reports become a strategic asset. They make it possible to change the model, optimize the cost, and prove that one version is better.
Partitech can build evaluation tests, automate checks, compare models, and integrate regressions into the delivery chain. The goal is to manage quality based on evidence, not on the impression of a demonstration.
Let's talk about your project
Set up an AI evaluation platform with Partitech. Contact Partitech.