Human-in-the-loop is a design decision, not a disclaimer

Ask a team how their AI feature avoids doing damage and you will usually hear "there's a human in the loop." Ask where, and the answer gets vague. Sometimes it means a person reads every output. Sometimes it means a person could, in principle, look at a log. Sometimes it means nothing at all; it is a phrase that made the launch review go smoothly.

I want to argue for treating it as what it is: an architectural decision, with options, trade-offs, and a place in the diagram. Once you do, the vagueness goes away and the system gets both safer and more useful — because you stop putting humans where they add nothing and start putting them where they matter.

Three patterns, not one

Every human-in-the-loop design I have seen reduces to one of three patterns, or a mix of them.

1. Approve before act

The system proposes; a person confirms; then the action happens. The refund is drafted but not issued. The email is composed but not sent. The database migration is written but not run. This is the strongest control and the most expensive, because a person is on the critical path of every action.

2. Review after act

The system acts; the action is logged with enough context that a person can audit it; problems are caught and reversed afterward. This is how most ordinary software works already — nobody approves each row a cron job writes — and it is the right default for actions that are cheap to undo.

3. Sample and correct

The system acts at volume; a person reviews a random or risk-weighted sample; findings feed back into prompts, tools, and the eval set. This is quality control rather than control. It does not prevent any individual error, but it is the only pattern that scales to thousands of actions a day, and over time it is the one that most improves the system.

The mistake is treating pattern one as the only "real" human in the loop. A design that approves everything is a design that gets rubber-stamped by week three, which is pattern two with worse ergonomics and a false sense of security.

Choosing: reversibility × blast radius

Two questions decide which pattern an action gets.

How hard is this to undo, and how many people does it touch if it is wrong?

	Small blast radius	Large blast radius
Easy to reverse	Act, log, sample. (Draft a reply; tag a ticket; update an internal note.)	Act, log, review promptly. (Bulk re-categorize; send an internal digest to a team.)
Hard to reverse	Approve before act. (Issue a refund; send an external email; delete a record.)	Approve before act, with a second reviewer or a hard block. (Mass customer email; production deploy; payment run.)

Written down like that it looks obvious, and it is. The value is in doing it per action rather than per system. An agent that handles support tickets might take forty distinct kinds of action; most are "update a field" and belong in the top-left box, two or three are "refund" or "reply to the customer" and belong in the bottom row. Giving the whole agent one review policy means either forty approval prompts a ticket or none.

Write the policy down, next to the prompt

The output of that exercise is a permission policy: a list of the actions the agent can take, grouped into allowed, needs approval, and forbidden, with the conditions that move an action between groups. It belongs in the repository, versioned, reviewed in the same pull request as the prompt and the tool schemas, because it is the same kind of artifact. A policy nobody can find is a disclaimer.

The major agent platforms have been moving the same direction this year — approval steps before sensitive actions, policies you can set ahead of time rather than prompts you answer in the moment. That is the right shape. The policy should be a declaration the system enforces, not a habit the model is asked to remember.

actions:
  update_ticket_field:    allow
  add_internal_note:      allow
  draft_customer_reply:   allow
  send_customer_reply:    approve         # external, hard to recall
  issue_refund:           approve
  issue_refund_over_500:  approve + second_reviewer
  delete_customer_record: forbid
  change_billing_plan:    forbid          # route to a human, never automate

Plain and boring is the goal. Anyone on the team should be able to read it and say whether it is right.

Make review cheap, or it will not happen

Approval gates fail in one predictable way: the reviewer stops reading. Not out of laziness — because the interface made reading expensive and approving free. Everything about the review step should be designed to reverse that.

• Show the diff, not the document. If the agent changed three fields, show three fields. If it drafted a reply, show the reply and the message it is replying to, not the whole thread.
• Show the evidence. The retrieved passage, the tool result, the policy line the agent relied on. A reviewer who can see why can judge in seconds; one who cannot has to redo the work.
• Show the alternative. "Approve", "edit", "reject with reason". The edit path matters: a reviewer who can fix a small mistake inline will keep reviewing; one who has to reject and wait will start approving.
• Batch what can be batched. Ten low-risk approvals in one screen, one high-risk approval on its own.
• Measure the reviewers. Time per review and edit rate. If time per review collapses toward zero, the gate has become a rubber stamp and should be redesigned or demoted to sampling.

"I don't know" is a valid output

One more design decision that is usually left to chance: what the system does when it is not confident. The default — produce the best-looking answer anyway — is the worst option in every row of the table above. Give the agent an explicit way to stop and hand off, make that path easy to take, and count how often it is taken.

A system that escalates 15% of cases and gets the other 85% right is a good product. A system that escalates nothing and gets 95% right is a liability, because the 5% are invisible until a customer finds them. The escalation rate is not a failure metric; it is the system being honest about its edges.