A traditional application changes when its code or configuration evolves. A generative application also changes when the provider updates a model, a prompt is modified, a document enters the index, or a tool exposes a new schema. Without traceability, two different responses can seem inexplicable.
LLMOps brings together the practices for evaluating, deploying, monitoring, and evolving these systems. It extends DevOps and MLOps with the specificities of generative models, RAG, agents, and qualitative judgment.
Define the production unit
The product is not just a model. It includes:
- application;
- instructions ;
- model and parameters;
- sources;
- ingestion pipeline;
- index ;
- tools;
- policies;
- output format;
- assessments;
- infrastructure.
A delivered version must identify this set. Changing a single element can alter quality, cost, or safety.
The nine objects to version
1. Code
Orchestration, API, interface, and post-processing follow the usual software cycle.
2. Prompts and instructions
They are stored like code, reviewed, tested, and linked to evaluation cases.
3. Models
Provider, exact identifier, date, parameters, and options are recorded. An alias 'latest' is not enough to reproduce.
4. Source data
The documents and records have an identifier, version, and status.
5. Ingestion pipeline
Extractor, segmentation, embeddings, metadata, and filters are versioned.
6. Index
An index version corresponds to sources and a pipeline. It can coexist with the old one during validation.
7. Tools
Schema, behavior, permissions, and API version are known.
8. Policies
Routing, rejections, limits, approvals, and data rules are versioned.
9. Evaluations
Case studies, sections, judges, and thresholds evolve with the product.
Traceability of an AI response to the version of the code, the prompt, the model, the index, the sources, the tools, and the policies.
Environments
Development, testing, pre-production, and production must isolate data, keys, quotas, and tools. Non-production environments use authorized datasets and simulated actions.
External models may differ depending on the environment for cost, but the final validation must use the target configuration. Deviations are documented.
A delivery chain
A change triggers:
- static controls and diagrams;
- deterministic tests;
- quick assessments;
- security tests;
- comparison to the baseline;
- review;
- gradual deployment;
- observation;
- promotion or return.
The critical thresholds block. An exception has an owner and a duration.
Deploy gradually
A new configuration can be activated for a team, a portion of the traffic, or a type of task. Routing keeps the version used in order to compare.
The canary tracks proxy quality, errors, latency, cost, and feedback. For high-risk uses, an additional human review is temporarily enabled.
The rollback must include the model, prompt, index, and compatible tools, not just the code.
Draw without saving everything
A useful note contains:
- request identifier;
- user or pseudonym as needed;
- versions ;
- steps and durations;
- tools;
- identified documents;
- tokens and cost;
- errors;
- validation result;
- feedback.
It does not automatically retain full prompts, sensitive documents, or private reasoning. Minimization, masking, rights, and retention are defined.
Technical observability
Monitor availability, errors, timeouts, quotas, queues, time of each step, context size, cache, and resources. Distributed traces link API, search, model, and tools.
Service objectives focus on the journey: time to result, task success rate, and degraded mode.
Quality observability
Quality is not measured solely by technical logs. Signals include:
- refusal;
- lack of source;
- format failure;
- corrections;
- climbs;
- rejected tools;
- abandoned answers;
- sentinel cases;
- Reviewed samples.
The feedback is categorized and added to the regression set after validation.
Cost
The tracking assigns cost and tokens by team, use case, version, and stage. Budgets trigger alert, routing, or limit.
The cost per request is complemented by the cost per successful task and the cost of human validation. A price drop does not justify a drop in quality.
Lineage
For a given answer, the team must find:
- code ;
- prompt ;
- model;
- index ;
- sources;
- tools;
- policy;
- result;
- associated evaluations.
This traceability allows for an investigation, an approximate reproduction, and targeted notification if a source was incorrect. It does not require storing a chain of thought.
Supplier change management
A provider can modify a model, limits, or a rate. Pinned versions, periodic tests, and contract alerts reduce surprises.
A routing layer allows testing another model. The differences in formats and capabilities remain explicit rather than hidden.
Index management
A new embedding or segmentation often requires a reconstruction. The index is created alongside the old one, fed, evaluated, and then activated. Reversion remains possible.
Ingestion errors, delays, and deletions are monitored. An unindexed source must not go unnoticed.
Prompt management
Prompts have an owner, an intention, tests, and documentation of variables. They avoid secrets and hard-coded data.
A prompt studio can facilitate editing, but the source of truth remains versioned and reviewed. Changes in production without history are prohibited.
AI Incident Management
An incident can be:
- leak;
- unauthorized action;
- dangerous response;
- model unavailable;
- abnormal cost;
- quality drift;
- expired source;
- incomplete index.
The plan includes shutdown, degraded mode, revocation, analysis, correction, evaluation, and communication. Product, security, data, and technical managers collaborate.
RACI and ownership
Each system has:
- product owner;
- technical manager;
- data owners;
- security;
- compliance;
- support;
- budget;
- launch decision-maker.
A central platform can provide the foundations, while each use case handles its data and quality.
Start small
A minimal foundation includes versioning, evaluation set, correlated traces, cost tracking, gradual deployment, and rollback. A complex platform is only necessary as teams and uses multiply.
Tools must serve the process, not replace it. A simple, automated convention is better than an unmaintained sophisticated catalog.
Exploiting AI as a product
LLMOps transforms a moving configuration into an observable product. It allows you to know what has changed, what has improved, and how to revert.
Partitech can design the pipeline, evaluations, logs, version management, and the operation of RAGs and agents. The goal is to deliver frequently without losing control over quality, cost, and risk.
Let's talk about your project
Industrialize your AI applications and their observability with Partitech. Contact Partitech.