From Chat to Action: Designing AI Workflow Automations That Don't Break

There is a large gap between an AI that drafts an email and an AI that sends it. The first is a suggestion you can ignore; the second changes the world and cannot be un-sent. Once an automation takes real actions — updating a CRM, issuing a refund, posting to a channel — it inherits every hard problem of distributed systems, plus a non-deterministic component making the decisions. Built naively, these are the automations that quietly double-charge a customer at 2am.

Built well, they are transformative. The difference is a handful of engineering disciplines that have nothing to do with the model.

Idempotency: safe to run twice

Anything that can retry will retry — a webhook redelivery, a timeout that actually succeeded, a queue that guarantees at-least-once. If your automation issues a refund on every run, a retry issues two. Make every action idempotent: give each unit of work a stable key and record that it happened, so a repeat is a no-op.

// Same key twice = one effect. The backbone of every safe automation.
export async function issueRefundOnce(orderId: string, amount: number) {
  const key = `refund:${orderId}`
  if (await store.has(key)) return store.get(key)   // already done, no-op

  const receipt = await payments.refund(orderId, amount)
  await store.set(key, receipt)                     // record before returning
  return receipt
}

Retries and failure that does not cascade

External systems fail transiently. Retry with backoff — but only idempotent actions, or you multiply effects. Just as important is what happens when retries are exhausted: the work should land in a dead-letter queue a human can inspect, not vanish. A silent failure in an automation is worse than a loud one, because you find out when a customer complains.

• Retry transient failures (timeouts, 5xx) with exponential backoff.
• Do not retry deterministic failures (validation, 4xx) — they will just fail again.
• After N attempts, dead-letter with full context and alert a human.

Human in the loop, on the actions that warrant it

Not every action needs approval, and gating everything defeats the point of automating. The trick is to classify actions by blast radius. Low-stakes and reversible? Let it run. High-stakes or irreversible — a large refund, an external email, anything touching money or a customer relationship? Route it to a person for one-click approval, with the model's reasoning attached.

type Action = { kind: string; reversible: boolean; value: number }

function needsApproval(a: Action): boolean {
  if (!a.reversible) return true          // can't undo -> ask a human
  if (a.value > APPROVAL_THRESHOLD) return true
  return false                            // low blast radius -> auto-run
}

This keeps humans focused on the decisions that actually need judgement while the routine 90% flows automatically — which is the whole promise of automation, delivered without the recklessness.

Know when not to automate

The most valuable advice we give is sometimes "don't automate this yet". If a process is ambiguous, changes weekly, or has a failure cost far above its frequency, the honest move is to keep a human in the driver's seat and let the AI assist rather than act. Automation earns its place on stable, well-understood, high-volume work. Force it onto the wrong process and you have built a fast way to make the same mistake at scale.

A good automation is not the one that does the most. It is the one you can trust to run unattended — and know when to stop.