A RAG prototype can be assembled quickly: a few documents, a vector index, a language model, and a conversational interface. The first demonstrations are impressive when the system retrieves precise information. They say little about its behavior in the face of contradictory documents, rights, updates, ambiguous questions, and thousands of users.
Going into production involves transforming this chain into a governed service. The system must know which sources it uses, explain its answers, measure its errors, protect data, and operate even when one of its components slows down.
Define the promise and the limits
A documentation assistant should not be described as capable of 'answering everything.' The promise must specify:
- covered bodies;
- authorized populations;
- types of questions;
- level of freshness;
- language;
- decisions that remain human;
- out-of-scope behavior;
- expected evidence.
A limited promise, such as retrieving and synthesizing internal procedures with citations, is more testable than a generalist expert assistant.
Build an inventory of sources
Each source has an owner, a sensitivity level, a date, a version, a language, and a retention rule. Obsolete, draft, or duplicate documents must be identified before indexing.
The system must retain the provenance up to the passage used: document identifier, version, section, rights, and timestamp. Without this traceability, a visual citation does not guarantee that the answer is based on the correct source.
Design ingestion as a data pipeline
Ingestion is not limited to extracting text. It must:
- retrieve the content in an authenticated way;
- detect the format and errors;
- extract structure, tables, and metadata;
- normalize without erasing the meaning;
- to cut up;
- enrichment;
- indexer ;
- validate;
- manage update and deletion.
Each step is idempotent and observable. An error on one file does not block the whole batch, but it is visible and attributed.
The cutting influences the response
A fragment that is too short loses definitions and exceptions. A fragment that is too long dilutes useful terms and consumes the context. The division must follow the structure: headings, paragraphs, articles, procedures, tables, and appendices.
The parent's metadata is preserved. For certain questions, a retrieved passage must be enriched with its title, section, or neighboring paragraphs. Several strategies can coexist depending on the type of document.
Combine research methods
Vector search finds similar formulations. Full text remains superior for references, names, acronyms, and exact quotations. Structured filters apply language, date, type, entity, and rights.
A robust chain can combine:
- lexical candidates;
- vector candidates;
- merging of ranks;
- reranking ;
- diversification;
- relevance threshold;
- retrieval of the surrounding context.
Sophistication is only valuable if it enhances a real game of questions.
Rights must intervene before generation
A user must not learn of the existence of a prohibited document through an excerpt, a quote, or an answer. Permissions must be reproduced in the index or enforced during retrieval, with a reliable update strategy.
Post-retrieval filtering can fail when the initial results are forbidden. The engine must search for enough allowed candidates. Caches are segmented by policy, not just by query text.
Build an assessment game before optimizing
A useful game contains frequent, rare, ambiguous, out-of-scope, contradictory, and sensitive questions. For each case, the reference sources and the elements that a good answer must include are kept.
It is necessary to evaluate separately:
- recovery: have the correct passages been found?
- generation: does the answer adhere to the passages?
- citation: do the references really support the claim?
- usefulness: does the answer help accomplish the task?
- refusal: does the system know not to respond?
Improvement loop of a RAG based on a set of questions, error analysis, and regression tests.
Design a response based on evidence
The generation prompt must recall the scope, enforce the use of context, and request a refusal when elements are missing. It must distinguish between quotation, reasoning, and suggestion.
An answer may indicate:
- synthesis;
- sources used;
- date or version;
- uncertain points;
- next action;
- system limit.
Citations must refer to the passage that can be consulted by the user, subject to their rights. A link to a one-hundred-page document without localization is not sufficient.
Deal with conflicting sources
Two procedures may contradict each other because one version is outdated, they concern different scopes, or an error exists. The RAG must not arbitrate silently.
The strategy can favor the current version, indicate the divergence, cite both sources, and invite validation. The priority rules are documented and tested.
Knowing how to say no
The system refuses when the corpus does not contain sufficient evidence, when the question is prohibited, when it asks for a decision outside responsibility, or when the rights do not allow responding.
A useful refusal explains the limit and suggests a safe way: rephrase, choose a scope, consult a source, or contact a person in charge. It does not invent a generic answer to fill the gap.
Protect against instructions contained in documents
A document may contain a sentence asking the model to ignore rules or to exfiltrate information. The retrieved content must be treated as unreliable data, never as system instruction.
The tools and actions are separated from the documentary RAG. The formats are cleaned, links and scripts neutralized, and abnormal behaviors tested. Public and internal documents can be isolated in separate indexes or policies.
Choose the model by role
The same model is not necessary for embeddings, reranking, and generation. The choice depends on the language, the domain, latency, cost, and deployment constraints.
An architecture can route simple questions to a lighter model and complex summaries to a more capable model. Changing the model requires a regression campaign, as answers and refusals may evolve.
Manage the context and cost
Adding more passages does not always improve the response. The context must be relevant, deduplicated, ordered, and limited. Long documents can be processed in steps or summarized with traceability.
The cost is tracked per request and per successful task: embedding, search, reranking, tokens, storage, and computation. A cache can reuse stable results provided that rights and freshness are respected.
Observability
Each request generates a correlation ID. Logs may contain intent, retrieved documents, scores, versions of prompts and models, latency, errors, and feedback, with a strict minimization policy.
The dashboards track:
- success rate;
- requests without proof;
- open quotes;
- refusal;
- latency;
- cost;
- ingestion errors;
- update delay;
- critical cases from feedback.
User feedback and human validation
A useful button is not limited to thumbs up or down. It allows you to indicate incorrect source, incomplete answer, outdated information, or access problem. Feedback feeds a triage queue and the evaluation set.
For high-risk uses, the response is a draft. A person validates, modifies, and takes responsibility for the decision. The system maintains the distinction between suggestion and approved action.
Deploy gradually
A pilot must cover a limited population and corpus, with identified support. The criteria for scaling up are defined: quality, safety, cost, adoption, escalation rate, and operational capacity.
The public launch comes after authorization, load, prompt injection, document deletion, and index recovery tests.
Govern the lifecycle
Sources change, models evolve, and usages shift. Each modification of pipeline, model, prompt, or policy receives a version and an evaluation. Indexes can be rebuilt and the old version kept for the time of a rollback.
A product committee arbitrates new corpora, the level of risk, and requests for action. The RAG thus becomes a controlled capability, not an isolated demonstration.
From the impressive response to the reliable service
Production requires less magic and more evidence: governed corpus, evaluated research, rights, refusals, observability, and exploitation. It is this discipline that turns an LLM into a working tool.
Partitech develops RAG chains, model integrations, and open source components around PHP, Mistral, and PostgreSQL/pgvector. Support can cover scoping, prototyping, evaluation, security, and production deployment.
Let's talk about your project
Frame or industrialize your document assistant with Partitech. Contact Partitech.