An AI Workflow Is Not in Production Until It Can Fail Safely

Evidence label: operator viewpoint with a clearly labelled synthetic example

The easiest part of many AI projects is the moment the model produces an impressive answer.

The difficult part starts one second later.

Can the system act on that answer? Is it allowed to change a customer record, create work, send a response or move money? What happens when the context is incomplete, an upstream API is unavailable or the model is confident and wrong?

That is the boundary between a demonstration and an operation.

The write path is the product

An AI workflow that only drafts text lives in a relatively safe world. A person still decides what happens next.

Once the workflow can write into a system of record, it needs an operating envelope:

• explicit permissions for what it may read and change;
• a known source of truth for each decision;
• thresholds that separate automatic action from human review;
• idempotency so a retry does not create the same action twice;
• logs that reconstruct the context, decision and action;
• alerts when a dependency or policy check fails;
• a safe rollback or compensating action; and
• a named owner for the exception queue.

These controls are not an extra governance layer applied after the build. They are part of the implementation.

A synthetic multi-site example

Consider a service group receiving enquiries across 60 locations.

An incoming message may need to be matched to a customer, classified, checked against service availability, routed to a location and written into a CRM. A response may then be sent and a follow-up task created.

The attractive demo is the classification and response.

The real operation has harder questions:

• Which location owns the enquiry when the postcode boundary is ambiguous?
• What happens when two customer records appear to match?
• May the workflow promise an appointment, or only propose one?
• What information must be withheld from the model?
• How is a duplicate message detected?
• Who sees an enquiry that has not progressed within the response clock?
• Can an operator replay the failed step after correcting the source data?

A sensible implementation might allow automatic routing when identity, geography and service availability are unambiguous. It might require approval before promising a constrained appointment. It should place unresolved identity matches into an exception queue with the evidence needed for a human decision.

The example is synthetic. It proves no client outcome. It illustrates the controls a production implementation must make explicit.

Human-in-the-loop is not a control by itself

“A human checks it” sounds reassuring but leaves the important questions unanswered.

Which human? How quickly? With what context? What authority do they have? What happens if they do nothing?

A real intervention path has:

1. a trigger—uncertainty, policy, consequence or technical failure;
2. an owner—a person or team accountable for the response;
3. a decision packet—the relevant source data, proposed action and reason;
4. a clock—the time before escalation or expiry;
5. an action—approve, amend, reject, retry or roll back; and
6. closure evidence—a log entry showing what happened.

Without those elements, the human is merely somewhere near the loop.

Measure the whole operation

A fast model response does not prove a faster workflow.

Before implementation, record the end-to-end baseline: volume, hands-on effort, elapsed time, exception rate, rework, cost and leakage. After release, add intervention rate, failure rate, recovery time and cost to operate.

Then compare like with like over an agreed measurement period.

This often reveals that the model was never the economic constraint. The value came from removing hand-offs, improving source data, resolving identity, changing an approval rule or exposing work that previously disappeared between systems.

The production test

Before calling an AI workflow live, I want to see a deliberate failure.

Break a dependency. Supply ambiguous data. Remove a permission. Retry an action. Confirm that the system stops safely, creates the right exception, alerts the owner and can recover without duplicate or silent work.

Happy-path accuracy matters. Recoverable failure is what makes the workflow operable.

KNIC Ventures works with multi-site service businesses on one expensive workflow at a time. The first engagement is a fixed-fee diagnostic that establishes the baseline, implementation case and operating controls before a production commitment is made.

Review the AI Workflow Diagnostic.