Human-in-the-loop

Definition
A person reviewing or approving an AI agent's work at the points that matter, before it goes live or takes effect.

Why it matters

Agents make mistakes that look convincing. A hallucination reads just as fluently as a correct answer. Without a check, a wrong price, a wrong claim or a wrong deletion goes straight to a customer or into the books.

It also lets trust grow in steps. A team watches where the agent proves reliable, relaxes the check there, and keeps it where a mistake would cost real money or reputation.

How to apply it

  • List the steps in a workflow and mark the few that cannot be undone or carry real cost: sending a message, paying an invoice, deleting data, quoting a price.
  • Let the agent run the rest at full speed.
  • Put the review just before the consequential action, not at the start, where it adds delay without adding safety.
  • Show the reviewer what they need to decide: the proposed action, the source it used and a one-click approve or edit.
  • Track how often the reviewer changes something. A check that has not changed anything in months is a candidate to loosen.

What it is

Human-in-the-loop is a design choice for automation. An AI agent does the preparation, such as drafting a reply, proposing a refund or filling in a record, and a person checks it before anything irreversible happens.

It sits between two extremes. At one end a person does everything by hand. At the other, an autonomous agent acts alone. Human-in-the-loop keeps the speed of the agent and the judgement of the person at the points where judgement counts.

Common mistakes

  • Asking a person to approve everything. They become the bottleneck the agent was meant to remove, and they start approving without reading.
  • Giving the reviewer no context, so the check is a formality.
Worked example

Suppose a ten-person online shop lets an AI agent draft refund offers for complaints. The agent reads the order and the message, then proposes an amount. Make writes each proposal as a row in Airtable, where a person reviews the queue each morning. Refunds under 25 euros are sent straight away, because a mistake there costs little. Anything larger waits for a click on approve or edit. Say the reviewer sees about forty rows a day and changes one in twenty during the first month. After two months, with the rate still low, the team raises the automatic limit to 50 euros. Large refunds keep the check, because a wrong one costs real money and damages trust.

Tools in the example

Some links are affiliate links: we may earn a commission at no cost to you. It never decides a ranking. How we work with partners

  1. Article

    Guardrails

    The wider set of limits, of which a human check is one.

  2. Article

    Handoff

    The moment work passes from agent to person.

Where it shows up

  • Writing copy that feels like conversation instead of marketing. How to sound like yourself.
    12 chapters