Discuss your project
AI Governance

Human in the loop: designing human validation of an AI system without creating a false sense of control

An 'approve' button is not a control if the person has neither the time, nor the evidence, nor the power to refuse.

Human in the loop: designing human validation of an AI system without creating a false sense of control

Human validation is often added as a universal response to risk: 'AI proposes and the human validates.' This phrase seems reassuring, but it can become a formality. If the person receives too many requests, does not see the sources, lacks the expertise, or faces time pressure, they approve mechanically.

Effective human oversight must be implemented at the right time, provide useful information, and genuinely allow for refusal, correction, or escalation. It must also remain compatible with the volume and the goal of automation.

Define what the control protects

Before choosing a screen, specify the risk:

  • factual error;
  • unfavorable decision;
  • data breach;
  • expense;
  • contractual commitment;
  • external communication;
  • security;
  • reputation;
  • unavailability.

The control must be linked to a consequence. A stylistic review does not protect against an authorization error.

Place the validation before the irreversible point

A check after sending, payment, or deletion becomes a detection, not a prevention. The workflow must identify the last moment when the action can be changed without consequence.

For a long task, several steps can exist: validation of the plan, the data used, the draft, and then the final action. Unnecessary checks are removed in order to focus attention.

Five forms of human control

1. Systematic review

Each release is checked. Suitable for beginners, for rare tasks, or for high-impact ones.

2. User confirmation

The person requesting confirms the settings and consequences. Useful for an action that concerns them and that they understand.

3. Independent approval

Another person approves according to a separation of duties, for example for a payment or a sensitive decision.

4. Sampling

A portion of the outputs is reviewed, with an increase in the rate if the quality decreases. Suitable for stable and reversible tasks.

5. Escalation by exception

The system executes the simple cases and forwards those that exceed a threshold, contain a contradiction, or lack evidence.

Supervision scale ranging from assisted information to independent approval and forbidden autonomy.

Give the validator the right evidence

The interface must display:

  • original request;
  • proposed output;
  • sources and versions;
  • modified data;
  • rules or calculations;
  • uncertainties;
  • differences with the current state;
  • consequences;
  • timeframe and possibility of rollback.

Displaying only the result promotes automation bias. The validator must be able to open the evidence without reconstructing the entire file.

Design the review actions

The choices must be explicit:

  • approve;
  • modify then approve;
  • reject;
  • request a new proposal;
  • escalate;
  • report a source or policy issue.

An approval is tied to the exact version. If the content changes, it becomes invalid.

Avoid automation bias

A smooth answer inspires confidence even when it is false. The interface can reduce this bias by:

  • showing the evidence before the recommendation for certain uses;
  • indicating unsupported areas;
  • comparing with the source value;
  • avoiding an uncalibrated confidence score;
  • varying the test cases;
  • training on typical errors.

The control must measure the actual disagreement, not just the approval rate.

Calibrate the uncertainty

An LLM does not spontaneously provide a reliable probability. Useful signals can come from source coverage, consistency between methods, validation rules, or a calibrated classifier.

When uncertainty is high, the system asks for clarification, refuses, or escalates. It does not hide the problem behind a cautious phrase.

Size the load

Calculate:

volume × temps moyen de revue × taux de contrôle

Add peaks, absences, escalations, and training. If the load exceeds the capacity, approvals become superficial or the process gets blocked.

The solutions are: reduce the scope, improve quality, automate deterministic checks, sample weak cases, or add resources. Silently removing the review is not a solution.

Choose the right validator

The person must have:

  • skill;
  • access to data;
  • independence;
  • time;
  • authority to refuse;
  • understanding of responsibility.

A business expert is not necessary for a format check, but is required for complex interpretation. Roles are documented and backups are planned.

Shape and size

Validators use examples of good and bad outputs, a rubric, and an escalation procedure. Calibration sessions compare decisions and clarify the criteria.

A frequent disagreement can reveal an ambiguous business rule, not a problem with the evaluator.

Record a proportionate proof

The log retains identity, version, decision, modification, justification when necessary, and timestamp. It avoids storing unnecessary sensitive data.

For an important action, the evidence includes the approved parameters and the result. A simple 'validated' line without an object does not allow for an audit.

Learning loop

The corrections are categorized: fact, source, format, tone, law, calculation, or policy. They feed the evaluation set and pipeline improvements.

One should avoid automatically retraining on all corrections without validation. Human errors and sensitive data can be propagated.

Dynamic sampling

When quality is demonstrated, the review rate can decrease for weak and known cases. It increases after a change in model, source, prompt, or when an alert appears.

New cases, rare or high-impact ones are still reviewed. Sampling is documented and reproducible.

Measuring the effectiveness of control

Follow:

  • approval rate;
  • modification rate;
  • rejection rate;
  • disagreement among validators;
  • review time;
  • errors discovered after approval;
  • escalations;
  • workload;
  • fatigue or workarounds.

An approval rate close to 100% can indicate excellent quality or unnecessary control. It is necessary to check through sentinel cases and audits.

Human control and user experience

Validation must be integrated into the business tool, maintain context, and work on real devices. Keyboard shortcuts, comparisons, filters, and batch actions are useful, but a mass approval of sensitive actions is avoided.

Deadlines are visible and users know who is making the decision.

Failure scenarios

Plan:

  • no approver available;
  • deadline exceeded;
  • conflict between approvers;
  • data modified during review;
  • model unavailable;
  • action already executed;
  • approval withdrawn;
  • security incident.

The process chooses wait, transfer, cancellation, or manual mode.

A control that evolves with the evidence

The human in the loop is not a permanent, identical state. It often starts with a strong review, then adapts according to risk and quality. Some decisions always remain human; others can become supervised by sampling.

Partitech can map risks, design approval interfaces, integrate auditing, and measure the workload. The goal is real human control that protects the decision without turning automation into double work.

Let's talk about your project

Design a human validation chain tailored to your risks with Partitech. Contact Partitech.

Share this article