Durable AI systems need memory, state, retries, and evidence

Chat history is not durable operational memory

Chat history is usually a sequence of messages. It may be truncated, summarized, or placed back into a model context window. That can improve conversational coherence, but it does not answer the operational questions that matter when a workflow spans hours, systems, or people:

• Which business object is being changed?
• Which source documents were authorized for this user?
• Which model, prompt, retrieval configuration, and tool versions were used?
• Which external calls completed before the process stopped?
• Which human approved the next action?
• Which results remain proposed rather than executed?
• What should happen after a timeout, process restart, deployment, or partial failure?

Operational memory is therefore not one thing. It normally includes several layers:

1. Workflow state - the current step, pending work, completed work, timers, and approval state.
2. Business state - the records, documents, permissions, and transactions the workflow is allowed to read or change.
3. Episodic evidence - what happened during a particular run, including inputs, outputs, tool calls, decisions, and exceptions.
4. Reusable knowledge - approved policies, patterns, prior cases, and system knowledge that may inform future work.
5. Configuration state - model, prompt, retrieval, evaluator, tool, and policy versions.

Treating these layers as one undifferentiated memory store creates risk. A retrieved note from six months ago may be useful context, stale instruction, confidential data, or an unapproved conclusion. Durable memory needs ownership, time, source, trust state, retention, and access boundaries.

Episodic memory needs provenance

Semantic retrieval and knowledge graphs can help a system find conceptually related history. Time-aware records can help it distinguish current policy from older practice. Those capabilities are useful only when the retrieved material carries enough provenance to be evaluated.

A practical memory record should answer:

• Source: Where did this information come from?
• Authority: Who or what is allowed to establish it?
• Time: When was it observed, approved, or superseded?
• Scope: Which customer, tenant, system, task, or role may use it?
• Trust state: Is it verified, disputed, proposed, inferred, or unknown?
• Retention: How long should it remain active?
• Supersession: What current record replaces it?

Without those fields, better retrieval can produce more confident misuse. A system that remembers everything but cannot distinguish current authority from historical observation is not well governed; it is merely well supplied with text.

This is why memory should not automatically become instruction. Retrieved material should enter a bounded reasoning context, not silently rewrite the system's rules. High-consequence workflows often need a separate policy or authority layer that memory cannot override.

Durable execution is a control boundary

Durable execution platforms preserve progress across failures. Temporal describes a Workflow Execution as a durable, reliable function execution, and its Event History records what has happened so the workflow can recover through replay. [S1] [S3] Pydantic AI's durable-execution integrations similarly distinguish long-running coordination from model and tool calls that can fail or take place outside deterministic workflow code. [S4] [S5]

The central architectural idea is separation:

• Workflow code coordinates the process and must behave deterministically when replayed.
• Activities perform non-deterministic or external work such as model requests, API calls, database writes, file generation, or notifications.
• Signals or human events resume a workflow after an approval, correction, refusal, escalation, or new evidence arrives.

This separation does not eliminate failure. It makes failure behavior explicit and recoverable.

A durable workflow can wait for a reviewer for hours or days without holding an application process open. It can preserve which steps completed and which remain pending. It can resume after deployment. It can record that a model call returned a particular result rather than calling the model again during replay.

That is materially different from a background job with a retry loop and an ad hoc checkpoint table.

Retry safety is not exactly-once magic

Retry behavior deserves careful language. A workflow engine can retry work reliably, but an external side effect may still happen more than once if the caller cannot determine whether a prior attempt completed.

Consider a sequence:

1. The system sends a request to create a payment.
2. The external service creates the payment.
3. The network response is lost.
4. The activity times out and retries.

Without an idempotency key or a reconciliation step, the second call may create a duplicate payment. Temporal's activity guidance explicitly treats retries as part of durable execution behavior. [S2] The engineering obligation is to design side effects so retries are safe.

Useful controls include:

• Stable idempotency keys tied to a workflow and business operation
• Unique constraints or deduplication records
• Read-before-write reconciliation where appropriate
• Explicit activity timeouts and retry policies
• Separation between proposal, approval, and execution
• Compensation or rollback logic for reversible operations
• Manual exception queues for uncertain effects

Claims of "exactly once" should be bounded to the layer that can support them. A workflow history may record one logical activity result, while an external service still needs its own idempotency and reconciliation contract.

Human review belongs in the state machine

A human approval should not be a comment attached after the fact. It should be an explicit workflow state with a named role, allowed decisions, expiry behavior, and evidence requirements.

A useful review state defines:

• What the AI proposed
• What source evidence supports the proposal
• What the reviewer may approve, edit, reject, refuse, or escalate
• Whether the reviewer is also allowed to execute the action
• What happens if the reviewer does not respond
• What additional evidence can be requested
• How overrides and disagreements are retained

Long-running workflows make this practical because the process can stop at the authority boundary and resume only after a valid signal. The system does not need to grant the model standing authority simply because the model completed its part of the task.

Observe the whole workflow, not only the model

Model quality is one component of operational reliability. A production workflow also depends on retrieval, authorization, tools, external services, data freshness, latency, cost, reviewer load, and downstream execution.

Useful telemetry normally includes:

• Model, prompt, retrieval, and evaluator versions
• Token, latency, timeout, and cost information
• Retrieved source identifiers and access decisions
• Tool requests, results, and failures
• Workflow state transitions
• Retry counts and reasons
• Human review outcomes and elapsed time
• Overrides, refusals, escalations, and blocked actions
• Final business outcome when it can be measured responsibly

Do not turn observability into indiscriminate data capture. Logs should minimize sensitive content, use appropriate retention, and separate operational identifiers from raw confidential payloads.

NIST's AI RMF organizes risk work around Govern, Map, Measure, and Manage, and the Generative AI Profile adds considerations specific to generative systems. [S6] [S7] Those frameworks do not prescribe one workflow engine, but they reinforce the need to connect ownership, context, measurement, and response rather than treating model selection as the entire risk program.

Common failure modes

Stale memory becomes current instruction

A retrieved decision or policy is treated as authoritative even after it has been replaced. Mitigation requires effective dates, supersession links, authority labels, and retrieval filters.

Cross-tenant or cross-role leakage

Semantic search retrieves relevant content that the current user is not allowed to see. Access checks must apply before or during retrieval, not only after generation.

Duplicated side effects

A retry repeats an external write, message, or transaction. Idempotency keys, deduplication, reconciliation, and explicit uncertainty handling are required.

Nondeterministic workflow code

A replayed workflow takes a different path because it reads the current time, makes an unrecorded network call, or uses changed code incorrectly. Keep nondeterministic work in activities and use the workflow platform's versioning guidance.

Silent retries hide degraded service

A system eventually succeeds, but only after repeated failures, high latency, or excessive cost. Retry counts and elapsed time should be visible as reliability signals, not hidden as implementation detail.

Unsupported memory becomes a confident fact

The model retrieves a plausible note without source or trust state. Generated claims need source support, and an unknown result must remain available when support is insufficient.

A practical evidence pattern

The LongTermCapabilities proof sequence can be applied to durable AI:

Source

Record the authorized documents, business state, workflow state, model and prompt versions, tool contracts, and access policy used for the run.

Test

Exercise expected, edge, adversarial, refusal, access, timeout, retry, restart, and duplicate-effect cases. Include human-review delays and rejected proposals.

Review

Have named technical and domain reviewers examine results, disputed cases, authority boundaries, and unresolved effects. Calibrate automated evaluators against human judgment where they are used.

Decision

Define whether the workload proceeds, narrows, remediates, remains in pilot, or stops. Name release thresholds, rollback triggers, blocked actions, and evidence-retention expectations.

Implementation checklist

• [ ] Separate conversational context from workflow, business, evidence, and configuration state.
• [ ] Add source, authority, time, scope, trust, retention, and supersession metadata to reusable memory.
• [ ] Keep authorization and policy outside model-controlled memory.
• [ ] Identify every external side effect and define retry safety.
• [ ] Use stable idempotency keys or reconciliation for non-repeatable operations.
• [ ] Model human approval, rejection, edit, escalation, and timeout as explicit states.
• [ ] Record model, prompt, retrieval, evaluator, and tool versions.
• [ ] Test restart, replay, timeout, duplicate, access, refusal, and stale-memory cases.
• [ ] Observe retries, cost, latency, state transitions, human review, and final outcomes.
• [ ] Define release, rollback, blocked-action, and unknown-result criteria.

Decision connection

A team should not move an AI workflow toward production merely because a model can complete the happy path. The production decision depends on whether the surrounding system can preserve state, recover safely, retain authority boundaries, expose failure, and leave an inspectable record.

The AI Production Readiness & Evidence Sprint evaluates that wider system boundary. The AI Evaluation & Release-Gate Implementation turns representative cases, versioned evidence, failure categories, and release decisions into a repeatable operating process.