How Decagon Beat Frontier Models with Fine-Tuned Small Models

Source: a16z | Published: 2026-07-31T14:30:14Z

Decagon runs 90% of its workflows on open-source models, with task-specific fine-tuned small models outperforming top Anthropic and OpenAI models on quality, speed, and cost.

That's what Decagon co-founders Jesse and Asha shared on the a16z podcast, in a rare conversation where they laid out the company's technical roadmap and product thinking with unusual candor.

Small Models Can Beat Large Ones — If You Know How to Tune Them

Decagon's pivot to open source started from a practical constraint: voice agents require low latency, and frontier models can't deliver it. They began exploring whether smaller models could do the job instead.

What they found surprised them. Jesse says the prevailing framing — "big models are smart but expensive, small models are cheap but worse" — is fundamentally a false dichotomy.

What we see in practice is that even supposedly dumber models, when fine-tuned for a specific task, actually outperform large frontier models on that task. So we ended up with all three things — better task performance, lower cost, and faster speed.

The logic is straightforward: frontier models are generalists. They can do math, write code, and generate creative content. But Decagon's agents only need to do one thing well within a conversation — identify a topic category, or determine whether a user is being adversarial. Individual subtasks don't require the full intelligence of a generalist. A small model trained specifically for that one thing is enough.

Jesse is quick to add, though: this path isn't easy. Deciding to "use open source" is just the beginning. You need data, and more importantly, you need an evaluation framework tailored to your own use cases. Public benchmarks are useless — you have to generate your own evals and run end-to-end tests yourself. Decagon built a dedicated research team for this, and that doesn't come cheap.

Models Keep Changing, So Training Never Stops

There's a common misconception that fine-tuning is a one-time project: pick a model, train it, done.

Decagon's reality is that they're continuously training new models and retiring old ones. The reason is that the frontier keeps moving. Open-source base models get new releases every few months. When new capabilities emerge, subtasks that previously needed separate handling may get absorbed entirely, making old models obsolete — while also opening up new possibilities that weren't previously viable.

Asha describes Decagon Labs as a "model factory," with the core goal of shortening the time between a new model's release and having a usable fine-tuned version ready. That's their justification for the sustained investment in a research team.

As for whether enterprise customers can take this path — Jesse thinks yes, but much more slowly than most expect. Large companies don't lack the will; they lack the speed. Governance processes, model risk reviews, security approvals, and a shortage of internal owners willing to take accountability all act as brakes.

Application-Layer Companies Aren't "Thin UIs" — But the Lines Are Blurring

Early 2026 saw a popular take that Anthropic and OpenAI are the last real startups, and application-layer companies are nothing but thin wrappers with headcount-heavy implementation bolted on.

Jesse thinks this misses something fundamental. His analogy: humans have always been a kind of AGI, yet humans still need databases, CRMs, and all kinds of software to get work done. Even in a true AGI world, agents will need somewhere to store data, pull information, and run reasoning — software doesn't disappear, and neither does the application layer.

His deeper view is that application-layer companies may evolve over time into vertical "labs" — where your core product is a set of models that are exceptionally good at a specific domain of tasks.

At the same time, frontier labs are moving into the application layer, because that's how enterprise customers actually see ROI. Pressure is building from both directions, and Jesse expects that dynamic to continue.

"Forward-Deployed Engineers" Are There to Learn Workflows, Not Do Custom Consulting

Decagon drew considerable attention early on for championing the concept of "agent PM" and "forward-deployed engineer." But Asha has reservations about how the term is being used across Silicon Valley.

Palantir made forward-deployed engineers famous, she notes, but very few companies can replicate Palantir's playbook — land a massive contract first, then pour in resources. Many companies flying the "forward-deployed" banner are really doing custom consulting, and what they build simply doesn't scale.

Decagon's model is different by design: what the forward-deployed engineer produces isn't a one-off build for a single customer — it goes back into the core product, ready for the next customer with a similar need. Asha frames it bluntly:

The forward-deployed engineer eats the pain and excretes product.

They're specific about how this works in practice. The concept of AOP — Agent Operating Procedures — was itself born when forward-deployed engineers realized that writing code to define workflows was too slow. They turned it into natural-language procedures, which then became a product feature.

Duet: An Agent That Manages Another Agent

Jesse talked about what he considers the most "magical" product moment — the emergence of Duet.

In the early days, onboarding each new customer meant engineers manually writing AOPs, tool-calling logic, and test cases. After launch, someone had to manually read through conversations and surface problems. That was a massive amount of repetitive manual work, and as the customer base grew, the cost scaled linearly.

Duet is a "second agent" — larger and slower than the primary agent — built specifically to handle all of that. You give it a batch of conversation logs and documentation, and it infers the right workflow, writes its own tests, monitors live conversations post-launch, flags anomalies, and proactively drafts improvement proposals.

Jesse says this only became possible because reasoning models got better. The capabilities Duet needs — the ability to take on complex, open-ended exploratory tasks rather than simply executing a fixed set of instructions — are the same capabilities Claude Code needs. When models crossed that threshold, Duet became possible. That was the moment he could concretely feel the models improving.

Competing With Sierra: Glass Box vs. Black Box

Asha mentioned that a recent customer win came from a company that had previously been using Sierra. The reason they gave was direct: Sierra's deployment model relies heavily on FDEs, leaving customers unable to see what's happening inside the system. Changing a single flow means waiting in the FDE queue — over the course of a year, they had shipped just three new journeys.

After moving to Decagon, they shipped seven themselves in under a month.

Decagon calls their approach the "glass box" — customer teams, including non-technical staff, can directly see the agent's logic, make changes themselves, and trace exactly what happened in any conversation. That's a deliberate product philosophy, not a positioning exercise.

Some customers do genuinely prefer the "handle everything for us" black-box model. Asha doesn't deny that demand exists — Decagon just isn't building for it.

From Support to "Front Office": One Agent for All Customer Touchpoints

Decagon's first product was customer support, for a simple reason: it was at the boundary of what models could reliably do at the time.

As model capabilities improved, something shifted. Customers started to realize: Decagon already understands our product, our brand voice, our customer communication style — why can't it also handle inbound sales? When a new prospect comes in asking product questions, why not run a qualification pass and hand them off to enterprise sales?

Another customer started using Decagon for proactive outreach — detecting account anomalies and reaching out to customers directly, rather than waiting for them to file a support ticket.

Asha's take is that this is all the same thing extended: they were never building "an agent that's good at support" — they were building "an agent that's good at executing business processes." Customer support, sales qualification, proactive operations — these are all business processes, just in different contexts.

What enabled this expansion was one specific capability improvement: the ability to understand broad instructions. Early models required extremely precise constraints to behave reliably — even minimal ambiguity caused drift. Current models can operate in a much larger instruction space, filling in reasonable gaps themselves, understanding intent the way a person would.

AI Kills Jobs, Not Careers

Customer support was one of the first domains cited in conversations about AI displacing employment.

Jesse shared a counterintuitive case. A customer was handling roughly 50,000 support tickets a month before Decagon. After deploying, they discovered that customers actually had far more questions — they just hadn't had accessible channels to ask them. So the company embedded support entry points on every page and opened access to free-tier users. Support volume went up significantly. They didn't cut half the team.

The underlying logic: in many contexts, support demand is suppressed — held back by the cost of handling it. When cost drops, demand isn't eliminated; it's released.

Jesse's framing: AI will kill jobs, but not careers. What gets replaced is high-repetition execution — looking things up in documentation, clicking through interfaces, delivering scripted answers. People move toward work that genuinely requires human judgment. He acknowledges the outcomes vary by customer — some have meaningfully reduced their outsourcing footprint, some have held headcount flat while growing, and some have redeployed the freed-up team toward revenue-generating work.

Saying "AI won't affect employment" is dishonest. But "AI will trigger mass layoffs" isn't the whole picture either. The reality is more complex and more granular than either narrative.