--- title: "A three-hour meeting to pick a model. Then they picked the one I suggested at the start" url: "https://www.mikefluff.com/en/blog/model-choice-2026" locale: "en" author: "Mike Fluff" updated: "2026-07-08" description: "About one project that got delayed seven weeks because the team couldn't agree on Claude, ChatGPT, Gemini, or a local. And about how this usually gets cured." tags: - "ai" - "tools" - "strategy" alternates: - locale: "en" url: "https://www.mikefluff.com/en/blog/model-choice-2026" markdown: "https://www.mikefluff.com/en/blog/model-choice-2026.md" - locale: "ru" url: "https://www.mikefluff.com/ru/blog/model-choice-2026" markdown: "https://www.mikefluff.com/ru/blog/model-choice-2026.md" - locale: "pt" url: "https://www.mikefluff.com/pt/blog/model-choice-2026" markdown: "https://www.mikefluff.com/pt/blog/model-choice-2026.md" - locale: "es" url: "https://www.mikefluff.com/es/blog/model-choice-2026" markdown: "https://www.mikefluff.com/es/blog/model-choice-2026.md" --- # A three-hour meeting to pick a model. Then they picked the one I suggested at the start > **In short:** Three questions settle it, in this order. If the data cannot go to any cloud, run a local model; above 100,000 requests a day, price local or the cheapest API on real load; otherwise run twenty real examples through Claude and ChatGPT and take the winner. One team argued for seven weeks, then shipped on Claude Sonnet after a three-day prototype. The meeting was on a Tuesday. On Zoom — six of us: CTO, two backend engineers, product, security, and me. One question on the agenda: "which model do we pick for feature X." Three hours later the meeting ended without a decision. The format chosen was "let's think until Friday." By Friday no one had thought, because there was a prod incident that week and nobody cared about models anymore. Over the next six weeks we convened on this question four more times. Each time a new argument joined in — "new Sonnet just dropped," "Gemini now has a two-million context," "Llama 3.3 is now slightly better on a benchmark that, by the way, isn't relevant to us." Feature X went nowhere for those seven weeks. In week seven I lost patience. I said: "Take Claude Sonnet, build a prototype in three days on your twenty real examples, and let's see." They did. The prototype worked. They took Claude Sonnet. Still running. Feature X shipped. Those seven weeks cost the client roughly a million rubles in wasted time of four people. Plus the demoralisation — something I don't know how to count, but which I think was of similar order. --- This is a typical story that repeats for me two or three times a year. **Picking a model** is a task it's very easy to drown in, because: — new models ship every three to four months, — benchmarks flip, — every engineer has their personal opinion ("I tried Gemini, liked it"), — industry blogs publish "why GPT-4o beat Claude at X" (and, naturally, "why Claude beat GPT-4o at Y"), — nobody in the team wants to be the first to say "whatever, take this one," because it sounds unprofessional. As a result, instead of working, the team **discusses tools**. And that would be fine if the discussion had a point. Usually it doesn't. --- Why it doesn't. The benchmarks you're debating with — **MMLU, GPQA, HumanEval** and the rest of the synthetics — measure how a model handles **standard abstract tasks**. That's probably irrelevant to you. You need to know whether **this specific model handles your specific task at a reasonable cost**. Which gets verified in **half a day** on your data, not seven weeks of argument. > **Don't repeat my clients** > > Don't spend a week comparing models "in general." Take your task, twenty real > examples, run them through two or three. Pick a winner. Everything else is > theory. --- When I do have to ground a decision — because a client wants a "reasoned choice," not "I said so" — I split models roughly like this. **Claude (Sonnet / Opus)** — long contexts, tasks with nuanced instructions, refactors, careful text work. I use it for most of my real scenarios, because it is, first, disciplined (follows instructions), and second, rarely hallucinates on facts it doesn't know (it usually says so honestly). Downside — pricier per token. A mid-size project: $50–200/month at normal load. **ChatGPT (GPT-4o / o-series)** — multimodal, real time, chat interfaces for end users. Massive plugin ecosystem. Downside — hallucinations on niche facts are **more visible** than Claude's. Few write about it, but I convinced myself of it several times this year, including once with reputational cost for a client who nearly sent an email citing invented research. **Gemini (2.x)** — if the whole company lives in Google Workspace, this is the shortest path. Long contexts (in 2026, no longer unique). Strong on multimodal with charts. Downside — **the ecosystem**. Outside Google it feels like someone else's kitchen. **Local (Llama 3.x, Mistral, Qwen)** — privacy, cost at scale, edge scenarios. If you hold data you **cannot send to any cloud** — decision closed, go local. Downside — **you need an engineer**, a server, a GPU. And the quality gap with flagships on hard tasks is still real in 2026. --- I collapse the choice to three questions in this order: **First — privacy.** Can't send? Local. Stop reading. **Second — volume.** Over 100,000 requests per day? Local or cheapest API. Price it on real load. **Third — everything else** — take Claude or ChatGPT, run 20 examples, take the winner. Three hours of work. Not three weeks. > **One trick that saves money** > > If your load is mixed — some tasks complex, some routine — use **two models**. > Flagship (Opus/GPT-4o) for hard. Cheap (Haiku/mini) for routine. Routing is > twenty lines of code. Savings on real projects: 3–5×. --- One thing I say to clients a lot, as the external person: Don't migrate between models every three months "because a newer one dropped." **Stability** costs more than 2% on a benchmark. Every migration is rewriting prompts, re-tuning, retesting. Work that adds zero value for the customer — they don't care whether you're on Sonnet 3.5 or Sonnet 4. One client this year spent two weeks migrating from Sonnet to Opus because "Opus is now smarter." A month later we noticed that for their task Opus wasn't notably smarter, just three times more expensive. Migrated back. Another week. Minus three weeks of real work. (Actually good that they came back. Worse would have been staying.) --- A model is **not a religion**. It's a tool for a task. In 2026 tools are plentiful and roughly comparable; choice takes half a day on your data, not three hours of a six-person Zoom. If your "let's pick a model" meeting lasts more than an hour, you're not debating models. You're debating the **fear** of choosing wrong. That's a different problem, and not one a benchmark comparison can solve. **[Vibe coding course](https://www.mikefluff.com/en/vibe-coding-course)** — Includes how to route requests between models, when flagship, when cheap. 6 modules, $19.