Human-reviewed AI requires a boundary between proposal, approval, and execution

Why "human in the loop" is too vague

The phrase does not tell a buyer or operator:

• Which human role is responsible
• Whether the person has relevant domain knowledge
• What information the person sees
• Which decisions the person may make
• Whether the person can change the proposal
• Whether approval and execution are performed by the same account
• What happens when evidence conflicts
• What happens when the reviewer is unavailable
• Whether the system can proceed after a timeout
• How the decision is recorded

A workflow can technically contain a human step and still create little meaningful control. A reviewer may receive a polished summary with no source evidence. A queue may contain too many items to examine. A default-approve timer may turn absence into consent. A reviewer may click approval while the system later executes a broader action than the interface displayed.

The design problem is therefore not whether a human is present. It is whether the system preserves informed, scoped, accountable authority.

Map consequence before automation

Begin with the consequence of a wrong action. A drafting aid and an account-closure system should not use the same review model.

For each candidate action, document:

• The person, organization, asset, or process affected
• Financial, legal, safety, privacy, service, or reputational consequences
• Reversibility and time to detect an error
• Whether a later correction can fully repair the harm
• Required expertise and organizational authority
• Existing policy or legal constraints
• Required evidence and retention

This consequence map determines whether the system may provide a suggestion, require review, require dual approval, or refuse automation entirely.

NIST's AI RMF emphasizes governance, contextual mapping, measurement, and management rather than a one-size-fits-all control. [S1] The Generative AI Profile adds risks and suggested actions specific to generative systems. [S2] A human-review design should therefore be tied to the actual use case and risk tolerance, not copied from a generic chatbot pattern.

Proposal is not permission

An AI output should have an explicit status. Useful statuses include:

• Draft
• Proposed classification
• Proposed match
• Proposed decision
• Needs evidence
• Needs domain review
• Rejected
• Approved for execution
• Executed
• Failed
• Reversed

The status should travel with the work product. A draft email should not be represented as sent. A proposed eligibility classification should not be written to the system of record. A model-generated remediation plan should not authorize a production change.

The interface and API should both enforce this distinction. Visual labels alone are insufficient if the backend accepts an unapproved request.

Name the authority

Authority is a role and a scope, not merely an authenticated user.

A practical authority matrix identifies:

• Proposer: the model or service that creates the recommendation
• Reviewer: the person qualified to inspect the evidence
• Approver: the role authorized to accept the consequence
• Executor: the service or person allowed to perform the action
• Escalation owner: the role responsible for disputed or high-risk cases
• Policy owner: the person who defines the governing rule
• System owner: the person accountable for the workflow's operation

One person may occupy more than one role in a low-risk workflow. Higher-consequence work may require separation. The important point is that the combination is deliberate and visible.

Least-authority principles support this design. NIST's Zero Trust guidance rejects implicit trust based solely on location and emphasizes explicit authentication and authorization for resources. [S3] Applied to AI workflows, the execution mechanism should receive only the permission needed for the approved action, not broad standing authority because an agent might need it later.

Approval needs more than a button

A reviewer interface should support actual judgment. It should expose:

• The proposal in plain language
• The underlying source evidence
• Relevant missing or conflicting evidence
• The model or rule version
• Confidence or score only when it is meaningful and calibrated
• Known limitations
• Comparable prior cases when their use is authorized
• The consequence of approval
• The exact downstream action that will occur
• Available alternatives

The reviewer should be able to approve, edit, reject, request evidence, refuse, or escalate. The system should not force all uncertainty into a binary yes/no decision.

For complex cases, require a reason code or short explanation. Do not require unnecessary narrative for every low-risk action; review design should avoid turning evidence into administrative noise.

Separate approval from execution

The execution layer should verify a signed or durable approval record before acting. That record should identify:

• Proposal identifier and version
• Approved action and allowed parameters
• Approver identity and role
• Approval time and expiry
• Evidence set or snapshot
• Policy or rule version
• Any conditions or edits
• Execution status

The executor should reject requests that exceed the approved scope. If the reviewer approved a refund up to a certain amount, the execution call should not accept a larger value. If the reviewer approved a document for one recipient, the system should not send it to an expanded distribution list.

This separation also improves incident response. Investigators can determine whether the model proposed an incorrect action, the reviewer misunderstood the evidence, or the executor performed something different from what was approved.

Support rejection, refusal, and escalation

A mature workflow expects that some work should not proceed.

Rejection means the proposal is wrong or inappropriate.

Refusal means the system should not attempt the task within its current authority or evidence. Examples include missing required source material, a prohibited action, an access conflict, or a request outside the use case.

Escalation means a higher-authority or specialized role must decide.

These are not failure states to hide. They are evidence that the workflow is respecting its boundary.

Blocked-action criteria should be testable. Examples:

• Required evidence is absent or contradictory
• The user lacks access to a cited source
• The action affects a protected or high-consequence decision class
• The proposal exceeds an approved financial or operational threshold
• The model or evaluator version is outside the approved release
• A reviewer conflict of interest is present
• A required second approval is missing
• The downstream system cannot confirm the intended scope

Calibrate reviewers, not just models

Two reviewers may disagree even when they see the same evidence. That is useful information.

A review program should measure:

• Agreement rates by case type
• Common reasons for disagreement
• Cases that require policy clarification
• Reviewer load and time pressure
• Override patterns
• False confidence caused by polished model output
• Cases where the model finds evidence humans missed
• Cases where humans correct unsupported model claims

Calibration sessions should use representative cases and document why the accepted interpretation changed. The goal is not perfect uniformity. It is to make differences visible and improve the policy, evidence, interface, and training behind the decision.

Retain an audit trail without creating a data dump

A useful trail connects:

• Source state
• Proposal
• Review actions
• Approval or refusal
• Execution
• Result
• Exceptions
• Subsequent correction

It should be possible to reconstruct the decision without retaining unnecessary sensitive content indefinitely. Use identifiers, hashes, snapshots, or redacted evidence where appropriate. Retention should match the business and regulatory context rather than defaulting to "keep everything."

Practical workflow example

Consider an AI-assisted contract intake process.

1. A user submits a public-safe or approved contract package.
2. The system extracts clauses and retrieves the approved playbook.
3. AI proposes clause classifications and draft issues.
4. The reviewer sees each proposal beside the cited clause and playbook source.
5. The reviewer approves, edits, rejects, requests evidence, or escalates.
6. The system creates an approved issue list, but does not send negotiation language automatically.
7. A designated legal or commercial owner approves the external response.
8. A separate sending service transmits only that approved version.
9. The system retains the source version, review record, final approval, and delivery result.

The AI accelerates comparison and drafting. It does not become the legal authority, the commercial owner, or the sending account.

Failure modes to test

• A proposal cites a source the reviewer cannot access
• The evidence changes after approval
• A reviewer edits the proposal but the executor uses the original
• The approval expires before execution
• The same approval is executed twice
• A reviewer account is compromised
• A default action proceeds after no response
• The system hides disagreement behind a single score
• A model update changes classifications without a new release decision
• A downstream API accepts broader parameters than the approval allowed

Implementation checklist

• [ ] Map consequence, reversibility, affected parties, and required expertise.
• [ ] Define proposal, review, approval, execution, escalation, and policy roles.
• [ ] Give AI outputs explicit proposed states.
• [ ] Show source evidence, unknowns, and the exact consequence of approval.
• [ ] Support edit, reject, request-evidence, refuse, and escalate outcomes.
• [ ] Separate approval authority from execution authority where risk warrants.
• [ ] Bind execution to the approved proposal version and parameters.
• [ ] Set approval expiry, duplicate protection, and exception handling.
• [ ] Retain disagreement and override reasons.
• [ ] Test blocked-action and access-failure cases.
• [ ] Monitor reviewer load, calibration, and over-reliance.
• [ ] Preserve an inspectable, appropriately minimized decision trail.

Decision connection

A buyer should be able to ask, "Who can make this decision, based on what evidence, and what exactly happens next?" If the answer is unclear, the workflow is not yet governed.

The human-reviewed AI workflow capability defines authority and evidence boundaries. The Trust Center states the public operating position that AI output is proposed work and consequential action requires explicit authority. The Human-Reviewed AI Workflow Checklist provides a practical review aid.