Blog
How to Choose an AI Development Partner: 7 Questions to Ask
AI & automation

How to Choose an AI Development Partner: 7 Questions to Ask

Key takeaways
  • With the model fixed, changing only the harness delivered a 6x gain on SWE-Bench Mobile (State of AI Report 2026).
  • In a controlled test of three models and three harnesses, variance caused by the harness was 7.8 times the variance caused by the model.
  • GPT-6 Astra scored 62.7% on ARC-AGI-3 with the standard harness and 99.9% with a state-persistent adapter — same model, same test.
  • The harness layers that stay valuable as models improve are interfaces to your systems, state management, observability and agent infrastructure.
To choose an AI development partner, ask about the software around the model before you ask about the model: which of your systems the agent will read and write, what it sees and remembers, how each result is checked, where the logs are, how fast the model can be replaced and who owns the code at handover. The State of AI Report 2026 shows why: with the model fixed, changing only that software — the harness — improved results up to 6x.
"Which model do you use?" is the first question most companies ask an AI vendor. GPT, Claude or Gemini, open or closed, this year's release or last year's.
The question is understandable: the model is the part everyone has heard of. But the State of AI Report 2026 collects a series of experiments that point elsewhere. When researchers kept the model the same and changed only the software around it, results moved far more than they did when the model changed. The industry calls that software the harness.
This article explains what a harness is, what the 2026 data shows, which parts of it are worth investing in, and how to use this when choosing an AI development partner.

What an AI agent harness is

A model on its own reads text and writes text. It cannot open a CRM, check a price, remember last week's customer or notice its own mistake. The State of AI Report defines the harness as the software around a model that runs its commands and stores its memory and skills.
In business terms, a harness answers five questions:
  • Tools — what is the agent allowed to do: read the inbox, query the database, create an invoice?
  • Context — what information does it see for each task: the price list, the customer's history, the company's policies?
  • Memory — what does it keep between tasks, and what does it forget on purpose?
  • Checks — how is each result verified before anyone relies on it, and what does "done" mean?
  • Logs and people — who can see what happened, and at which point does a person step in?
A useful comparison is a car. The model is the engine; the harness is everything that lets the car steer, brake and carry a load. A better engine in a badly built car is still a badly built car.
What an AI agent harness is: the model in the centre, surrounded by tools, context, memory, checks, logs and people

Same model, very different results

The report brings together several experiments in which the model stayed fixed and only the harness changed:
  • 6x on mobile software engineering. With each model fixed, changing the harness delivered a 6x gain on SWE-Bench Mobile.
  • 7.8x more variance from the harness than from the model. A controlled test paired three models with three harnesses on 100 SWE-bench Verified tasks. Harness-induced variance in results was 7.8 times model-induced variance. The open model GLM-5.1 rose from 52.5% to 65.5% between the minimal and the full harness.
  • A smaller model, a better result. Meta-Harness raised Anthropic's small model Haiku 4.5 from 27.5% with Claude Code to 37.6% on TerminalBench-2. On another benchmark, a long-horizon harness lifted Alibaba's Qwen 3.7-Plus above the baseline score of Anthropic's larger Opus 4.7.
  • 63% or 99.9%. OpenAI's GPT-6 Astra scored 62.7% on ARC-AGI-3 with the standard harness and 99.9% with a state-persistent adapter, which also ran 3.66 times faster on 49% fewer tokens.
The report's own conclusion is that meaningful model comparisons require disclosing the harness and holding it fixed. Put simply: when a vendor says model A beats model B, the first follow-up question is in which harness?
Same AI model, different harness: GLM-5.1 52.5% to 65.5%, Claude Haiku 4.5 27.5% to 37.6%, GPT-6 Astra 62.7% to 99.9%

Why the effect is stronger in business processes

These experiments use coding and reasoning benchmarks. Business processes run on enquiries, quotes, invoices, contracts and reports — and there the principle usually applies with more force, because raw model intelligence is rarely the bottleneck. Typical failures look like this:
  • The agent did not have the current price list, so it answered confidently and wrongly.
  • It could read the CRM but not write to it, so a person still copied everything by hand.
  • Nobody defined what "done" means, so the agent reported success and moved on.
  • There was no log, so when a customer complained, nobody could see what had happened.
Each of these is a harness problem, and none of them is solved by switching to a stronger model. LexisNexis makes the same point in its 2026 report on AI in intellectual property work: the model should orchestrate and communicate the analysis, not improvise the underlying methodology. The method and the data have to come from the business.
This is also why so many pilots stall between demo and production, a pattern we looked at in 95% Have an AI Strategy. 8% Can Show the Money: a demo needs a model, a production system needs a harness.

Models are becoming commodities; harnesses are not

The frontier is crowded. The report describes it as a three-lab race: on Artificial Analysis's Intelligence Index, Anthropic's Claude Opus 5.5 leads at 58, while OpenAI's GPT-6 Astra and Google's Gemini 4 Argon tie at 53; on Arena's ranking built from people's votes, Gemini 4 Argon is first. There is no single best model, and any lead lasts until the next release.
Benedict Evans, in his May 2026 presentation AI Eats the World, notes that for most general and consumer use the leading models are very similar and have no network effects. One scenario he sets out is that models become infrastructure and innovation moves up the stack.
The harness behaves differently. It contains a company's tools, data connections, rules, tests and history of corrections, and nobody else can download it. A well-built harness also makes the model replaceable: when a better model appears, it can be swapped in days rather than rebuilt around. In short, the model is rented, the harness is owned.

More harness is not always better

The same report contains an important caveat. Many early harnesses were full of detailed rules, reminders and planning steps written to compensate for weak models. As models improve, these workarounds become redundant and can even make stronger models worse. Claude Code removed 80% of its system prompt — the standing instructions given to the model — for advanced models, with no measurable loss on coding evaluations.
The report lists the harness investments that stay valuable as models improve:
  • Interfaces to your systems — tools and MCP servers, a standard way to connect AI to business software.
  • State management — what the agent knows about where each task stands.
  • Observability — logs and dashboards that show what the agent did and why.
  • Agent infrastructure — safe environments where agents run and are tested.
These four layers are unglamorous, and they compound. Prompts written to work around one model's limits usually do not survive the next model.
Two further cautions. The multipliers above come mostly from coding benchmarks, so the effect in a sales or finance process will differ. And the tooling is changing quickly: research papers on agent "skills" — packaged instructions and code an agent can reuse — rose from 152 to 1,486 between the first eight months of the previous year and the same months of 2026. Any number, including the ones in this article, should be checked on your own process before it is trusted.
AI agent harness layers that survive the next model: interfaces, state management, observability and agent infrastructure

Who owns the harness?

If the harness decides most of the result, an uncomfortable question follows: who owns it?
If an AI vendor built it on their platform, in their accounts, with prompts and connectors the client cannot see, the most valuable part of the project belongs to the vendor. The client owns a subscription. That can be a reasonable choice, but it should be a decision rather than something discovered when the company wants to change vendors.

Where the cost sits

A token costs the same in Munich and in Ho Chi Minh City; the model bill does not depend on where the team is. What differs is the cost of the engineering around the model — integrations, tests, monitoring and the months of small fixes after launch.
That work is not a one-off sprint. It continues for as long as the agent runs, which is why it pays to plan who does it. One option is to keep senior AI engineers close to the business in Europe and build the long tail of harness work with an engineering team in a second market such as Vietnam, so that most of the budget does not go to the part that is easiest to copy.

Seven questions to ask an AI development partner

Before asking which model a vendor uses, ask these:
  1. Which of our systems will the agent read from, and which will it write to?
  2. What exactly does the agent see for each task, and where does that data come from?
  3. What does it remember, for how long, and who can delete it?
  4. How is each result checked, and what does "done" mean for this process?
  5. Where are the logs, and can we see every action the agent took?
  6. If a better model ships next quarter, how long does switching take?
  7. At the end of the project, who owns the prompts, the connectors, the tests and the code?
A strong partner answers all seven in detail. If the answers are vague, the model name does not matter.
Seven questions to ask an AI development partner before asking which model they use

How Bonsay approaches it

Bonsay is model-agnostic by design: depending on the task and the client's rules, agents run on GPT, Claude, Gemini or open-weight models on the client's own servers, and the harness is built so the model can be replaced without starting over. In a self-updating knowledge base built for a client, the model is one box on the architecture diagram; the connections, data, checks, logs and the point where a person approves make up the rest — and together they cut manual work on transcription and document updates by 70%.
Projects start with a process audit and a pilot — one process automated end to end in the client's environment, measured against how it ran before — and at handover the client owns everything that was built, harness included. More on this approach: AI & automation. A good first process to start with is described in AI agent for customer enquiries.

Frequently asked questions

How do I choose an AI development partner?

Ask about the harness before the model. Seven questions cover it: which systems the agent reads from and writes to; what it sees for each task and where that data comes from; what it remembers and who can delete it; how each result is checked and what "done" means; where the logs are; how long switching to a better model takes; and who owns the prompts, connectors, tests and code at the end. A strong partner answers all seven in detail.

What is an AI agent harness?

The software around a model that runs its commands and stores its memory and skills. In business terms it covers the agent's tools, the context it sees, its memory, the checks on each result, the logs and the point where a person steps in.

Does the choice of AI model matter?

Less than most buyers assume. In a controlled test of three models and three harnesses, variance caused by the harness was 7.8 times the variance caused by the model, and Anthropic's small Haiku 4.5 rose from 27.5% to 37.6% on TerminalBench-2 with a better harness. Leading models are close, and any lead lasts until the next release.

Who should own the harness?

That should be a deliberate decision. If a vendor builds it on their platform, in their accounts, with prompts and connectors the client cannot see, the most valuable part of the project belongs to the vendor and the client owns a subscription.

Which parts of the harness are worth investing in?

The ones the State of AI Report 2026 lists as staying valuable as models improve: interfaces to your systems, state management, observability and agent infrastructure. Prompts written to work around one model's limits usually do not survive the next model.

Does the 6x gain apply to business processes?

Not directly. The multipliers come mostly from coding benchmarks, so the effect in a sales or finance process will differ. Any number should be checked on your own process before it is trusted.

Sources

  • Nathan Benaich and Air Street Capital, State of AI Report 2026.
  • Benedict Evans, AI Eats the World, May 2026.
  • LexisNexis Intellectual Property Solutions, AI in IP, 2026.