What I'm learning putting AI to work inside real organizations.

Which outperforms? Pick the metric first

Daniel Cornelius · November 17, 2026 · 3 min read

Last week I split the things people call "agentic" into four groups. The obvious next question is which one performs best.

Picture a design review. One team built a workflow for an intake job. Another built a scripted agent for the same job. One says theirs is more accurate. The other says theirs is cheaper. Both can be right, because they're measuring different things.

So a fair answer starts with a question back: outperform on what?

New here? Here are the four terms, from last week's post:

  • Automation: a fixed script or formula. No model.
  • Workflow: a fixed route in code, with a model inside the steps that need judgment.
  • Scripted agent: you write the procedure in a prompt, and the model decides how to move through it.
  • Open agent: you give it a goal and tools, and it picks the route itself.

What matters below is who enforces the route: code, or a prompt.

The scorecard

A fixed route wins some metrics and a model-driven route wins others. These are tendencies from my own work, not measurements. Automation is left off: if the logic can be written down, it wins on cost, speed, and repeatability, and the comparison ends there.

WorkflowCode sets the routeScripted agentPrompt sets the routeOpen agentModel picks the routeHandles unanticipated inputsWeakerMediumStrongerTime to first versionGrows with branchesFastFastestUpkeep as process changesEdit branchesEdit promptWatch for driftCost per runLowestMediumHighestSpeed per runFastestMediumSlowestSame route every timeAlwaysUsuallyVariesEasy to auditEasiestNeeds run logsNeeds logs + checksHow it failsthe row to underlineLoudly,unless swallowedQuietly,route can slipQuietlyBrighter cell = the better fit on that metric. Typical tendencies from my own work, not measurements. As of November 2026.

The last row is the one to underline. A broken workflow usually errors, and someone sees it. A drifting agent keeps producing confident, slightly wrong output. That's question three from last week: code stops when it breaks, and a prompt just bends.

Here's one quiet failure from my own work. A pricing agent looked up a rate by location. The lookup failed without an error and returned a default rate, which could be far too high or far too low depending on the location. The agent had no signal anything was wrong and built a confident answer around it. Each answer looked plausible on its own. The team caught it only when many locations came back with the same price. Our automated checks had confirmed the tool call succeeded, and it had: the fallback counted as a success. We fixed it by making the tool return a hard error when every location falls back to the default.

The lesson isn't that agents are worse. A coded workflow with the same silent fallback could fail the same way. The bug was the fallback, and nothing was positioned to see it. A failed lookup should stop the run, and checks should test the answer, not just the tool call.

Weight the metrics, then decide

The scorecard has no winner until you weight it. Two things set the weights:

  • Blast radius. Large, hard-to-reverse mistakes make repeatability and auditability weigh heavily. Small, cheap ones favor coverage and speed to build. That's the blast radius from the first post, applied to the design choice.
  • Volume. At high volume, cost and speed add up. At low volume, they barely matter.

High volume with little judgment points to automation or a workflow. Low volume full of judgment points to an agent. In between, a scripted agent usually earns its keep, as long as its controls live in code and not only in the prompt.

Run the comparison yourself

Don't take anyone's word for this, including mine. Most teams can run this test in a week:

  1. Pick the metrics and weights first, so you can't choose them to fit the result.
  2. Pull 50 real cases, including some ugly ones.
  3. Run them through each design. Run each agent case twice, and note whether the route changed, not just the wording.
  4. Have someone who didn't build either design score them.

The ugly cases are the point. Easy ones make everything look good.

Models keep getting better and cheaper, so the line between "needs a fixed route" and "can choose its own" keeps moving. (Observation, as of November 2026.) Revisit the comparison when your tools or models change.

Have you run a side-by-side like this? I'd like to hear which metric surprised you.

Enjoyed this? Subscribe by email on Substack or via RSS. I post one note a week.