Choosing the Right LLM for Your Use Case: A Model-Agnostic Decision Framework

The question we get most often is "which model should we use?" — and the honest answer is "for which part of which workflow?". Treating model choice as a single verdict is the root of most AI cost and latency problems we're later called in to fix.

A model is a component with a cost, a latency, and a capability profile. You choose it the way you choose a database: against the requirements of the job in front of you, not against a leaderboard.

Four axes that actually decide it

Almost every real decision comes down to trading off four things:

• Capability — can it do the task at an acceptable quality? This is a floor, not a maximum. A model that clears the bar reliably beats a stronger one that clears it 95% of the time if the last 5% costs you a support ticket.
• Latency — a user-facing chat needs a fast first token; an overnight batch job does not care. Match the model tier to the interaction, not the org.
• Cost — per-token price times realistic volume. The gap between a flagship and a small model is often 10-20x, and most tasks do not need the flagship.
• Data residency — can this data leave your environment at all? Sometimes the answer forces an open model you host yourself, and that constraint outranks the other three.

The tiering pattern

The highest-leverage move is refusing to use one model for everything. Route by difficulty. A cheap, fast model handles classification, extraction, routing, and short replies; a flagship handles the genuinely hard reasoning; an open model handles anything that cannot leave your walls.

// Route each task to the cheapest model that clears its quality bar.
type Task = 'classify' | 'extract' | 'reason' | 'draft'

const modelFor: Record<Task, string> = {
  classify: 'small-fast',   // pennies, sub-second
  extract:  'small-fast',
  draft:    'mid',
  reason:   'flagship',     // only where it earns its cost
}

export function pickModel(task: Task, sensitive: boolean) {
  if (sensitive) return 'self-hosted-open'  // data residency wins outright
  return modelFor[task]
}

Keep the choice swappable

The single most valuable architectural decision is to make the model a swappable dependency. Put every call behind one interface. When a new model ships — and one ships every few weeks — you evaluate it against your own tasks and switch by changing a config value, not a codebase.

// One interface, many models. The rest of your app never imports a vendor SDK.
interface ChatModel {
  chat(input: ChatInput): Promise<ChatResult>
}

const registry: Record<string, ChatModel> = {
  'flagship':         claude(),
  'mid':              gpt(),
  'small-fast':       gemini(),
  'self-hosted-open': llama(),
}

export const chat = (name: string, input: ChatInput) =>
  registry[name].chat(input)

This is what "model-agnostic" means in practice: not that every model is equal, but that swapping one costs you an afternoon of evals instead of a rewrite.

A worked example

Take a support workflow. Incoming tickets are classified and routed — a small, fast model, called thousands of times a day, cents per thousand. Draft replies for common issues — a mid model, quality matters but latency is forgiving. A handful of gnarly escalations get flagged for the flagship model with full context. And because some tickets contain regulated data, those route to a self-hosted open model. Four models, one workflow, each chosen against its own axis. The result is a system that is both cheaper and better than any single-model version of the same thing.