RAG, fine-tuning or prompt engineering: choosing the right method of specializing a model

A company wants to adapt a model to its business. Three solutions are often proposed: improve the prompt, connect a documentary database using RAG or train the model on examples. They don't solve the same problem.

The prompt gives instructions at the time of the call. The RAG takes external information and adds it to the context. Fine-tuning adjusts model parameters to reinforce a behavior. A mature architecture can combine all three, but only after identifying the cause of the errors.

Start with a baseline

Before adding a technology, build a minimal version with:

• a reference model;
• a clear prompt;
• a structured exit if necessary;
• twenty to one hundred representative test cases;
• metrics;
• error analysis.

Without a baseline, it is impossible to know whether RAG or fine-tuning brings an improvement. The team risks measuring an impression on a few favorable demonstrations.

Prompt engineering: framing the task

A good prompt specifies:

• the expected role;
• the task;
• entries;
• constraints;
• the output format;
• refusal criteria;
• such useful examples;
• the tools available.

The prompt is suitable when the necessary knowledge is already in the model or provided in the input. It is suitable for transforming text, extracting fields, classifying, summarizing or applying a short procedure.

Its advantages are speed, low implementation cost and ease of iteration. Its limits appear when the instructions become very long, contradictory or difficult to maintain. A prompt does not update the internal knowledge of the model and does not guarantee accurate business information.

RAG: bringing knowledge at the moment of the answer

Retrieval-Augmented Generation searches for relevant passages in sources and then transmits them to the model. It is suitable when the information:

• change regularly;
• belong to the company;
• must be cited;
• are too numerous for a fixed prompt;
• have access rights;
• must be removed without retraining.

A RAG is not a simple vector base. You have to manage ingestion, division, metadata, search, reranking, rights, citations, evaluation and updating.

Its faults often come from the search, not the model: bad document, incomplete chunk, access filter, different vocabulary or missing information.

Fine-tuning: learning a behavior

Fine-tuning trains the model on pairs of examples to increase the probability of a behavior. It may be relevant for:

• respect a precise format;
• adopt a consistent style;
• classify according to a stable taxonomy;
• improve a language or jargon;
• reduce the length of the prompt;
• learn repetitive decisions from quality examples.

It is generally not the best solution for storing changing documentation. The facts become difficult to update and cite. A fine-tuned model can also reproduce errors, biases or sensitive corpus data.

The main effort lies in the selection, annotation, rights, training/test separation and evaluation, not in the initiation of the training command.

The central question: knowledge or behavior?

When the model does not know the day's procedure, it lacks knowledge: the RAG often takes priority. When it knows the elements but doesn't follow a format or confuses a taxonomy, the behavior is at fault: prompt or fine-tuning can help.

Some problems are mixed. A support assistant must find the right procedure and then write a response in line with the tone of the company. The RAG provides the background; the prompt or fine-tuning frames the shape.

Compare the approaches

This table gives trends. The actual benchmark on the task remains decisive.

The long context does not always replace the RAG

The models accept increasingly long contexts. It is tempting to send all the documents. This approach can work for a one-off file, but it has limitations: cost, latency, noise, access control, repetition of data and difficulty in guaranteeing that the correct passage will be used.

RAG reduces the context to relevant elements and allows for a shared index. For small sets, a hybrid strategy can first filter by metadata and then send multiple complete documents.

RAG does not correct all behaviors

Adding more documents doesn't fix a template that refuses poorly, doesn't respect JSON, or has an incorrect tone. We must separate the metrics:

• recall and precision of the search;
• fidelity to sources;
• accuracy of the response;
• format;
• style ;
• refusal;
• latency;
• cost.

This breakdown shows which lever to modify.

Fine-tuning requires a governed corpus

Examples should be:

• representative of production;
• correct;
• consistent between annotators;
• authorized;
• free from unnecessary data;
• separated from the test game;
• versioned.

A corpus of historical responses can contain the bad habits that we want to eliminate. It needs to be curated, not just exported.

Provide an internal card model: basic version, data, parameters, limits, results, risks and withdrawal procedure.

A common hybrid strategy

A robust application can follow this chain:

1. a short system prompt defines the rules;
2. a classifier or router chooses the task;
3. the RAG retrieves the authorized sources;
4. a possibly fine-tuned model produces the expected format;
5. a validator checks schema and citations;
6. a human approves sensitive cases.

Each brick must justify its complexity. Adding a fine-tuned model before having stabilized the errors makes the architecture more difficult to maintain.

Build a comparative experiment

The decision can be made in four iterations.

Iteration 1: reference prompt

Measure quality without retrieval or training. Identify knowledge, format, reasoning, and safety errors.

Iteration 2: Minimal RAG

Index a limited corpus, define reference questions, and measure retrieval then response. Do not start with all the documentation.

Iteration 3: targeted fine-tuning

Only if a repetitive behavior persists despite the prompts and if examples are available. Compare to the baseline on a never-before-seen game.

Iteration 4: combination and exploitation

Measure latency, cost, drift, reversibility, and update capacity. The best out-of-production score is not always the best operable solution.

When not to use fine-tuning

Avoid fine-tuning when:

• the facts change every week;
• the source must be cited;
• the corpus is small or contradictory;
• the rights are uncertain;
• the error comes from the research;
• the need can be resolved by an exit plan;
• no reliable evaluation exists.

When not to build a RAG

A RAG is excessive for a transformation task without external knowledge, a few stable rules, or a single document provided at each call. It adds ingestion, indexing, security, and monitoring.

Driven by cost per successful task

Compare the full cost: data preparation, infrastructure, calls, annotations, reindexing, training, testing, incidents, and maintenance. A cheaper solution per token can cost more if it fails more often or requires heavy human review.

The useful metric is the cost per accepted task, with the expected level of quality.

Choose a reversible method

Keep prompts, corpus, evaluations, and schemas independently of the provider. Version the configurations and provide for an adaptation layer. A model or an API can evolve, be withdrawn, or change in price.

Partitech can build the baseline, the RAG pipeline, the fine-tuning corpus, and the benchmark evaluation. The decision then relies neither on a trend nor on an isolated demonstration, but on measured errors, controlled data, and a realistic operating cost.

Let's talk about your project

Build a benchmark comparing prompt, RAG, and fine-tuning on your data with Partitech. Contact Partitech.