“Agent” became one of those words that mean everything and nothing. Vendors stretched it to fit any LLM with a function call. Conference decks promised autonomous teammates. The reality on the ground has been quieter, and more interesting.
We have been deploying agents in production for the last eighteen months. Not as demos, not as wrappers around a chat box, but as components woven into customer-facing software that ships, breaks, gets fixed, and ships again. This piece is about what we learned at the boundary between an agent and a chatbot — and why that boundary changes how we now think about product architecture from day one.
The definition that actually matters
An agent is software that can take a goal, choose a sequence of actions to pursue it, and recover from failure without a human re-prompting it. Three properties have to be present:
- A goal it can hold across multiple turns.
- Tools it can choose between, with judgment about when each one applies.
- The ability to read its own intermediate output and decide what to do next.
The first generation of chatbots had one of those properties, sometimes. Today's frontier models have all three, but only when the surrounding software actually delegates the loop to them. That delegation is the part most teams skip.
Most products that call themselves “agentic” still keep the loop on the server side. The model produces a single response per turn. The architect makes every routing decision in code. The user sees a chat interface and thinks: chatbot. They are right. There is nothing wrong with chatbots — they are useful and well understood — but a chatbot dressed in agent language creates confusion that costs everyone time.
Where the loop lives
The simplest question to ask any system that claims to be agentic: who decides what step happens next? If your application has a switch statement that dispatches based on a classifier output, you have classification, not agency. Agency moves to the model when the model is the one writing the plan, choosing the next tool, and deciding when the work is done.
This sounds like semantics until you try to maintain a complex workflow. With agency on the model side, you spend your engineering time on the surface area the model interacts with: tool contracts, evals, guardrails. With agency on the application side, you spend it writing routing logic that the next model release will make obsolete.
What changed in our product work
The interesting effect was upstream of the model. Once we knew the model could hold the loop reliably, we stopped designing screens. We started designing what the agent should be able to perceive, which tools should be available to it, and which failure modes we needed to surface to the user.
The visible UI became thin. The work moved into tool contracts, observability, and the way state is exposed to both the model and the human. This is not a smaller engineering effort — it is a different shape of effort.
The chat window is the smallest possible expression of the system. Everything important happens around it. — internal note, October 2025
That sounds abstract until you watch a designer who used to draw seventeen screens for a workflow draw three screens, six tool definitions, and a list of states the agent needs to signal. The shift is not from one design discipline to another. It is a recognition that the model is now a participant in the product, not just a feature inside it.
The cost we did not see coming
Production agents bring a new category of failure: silent drift. The model returns a sensible-looking answer that is wrong in a way no schema validator catches. We learned to invest more in evaluations than in features for the first two months of any new deployment. Otherwise the agent shipped a polite hallucination on day one and we discovered it on day forty.
We also learned that the cost of an agent is not the per-token cost. It is the cost of the work surface around it: the eval harness, the guardrails, the human-review queue, the rollback path. Teams that skip those steps ship a demo and call it a product, then spend the next two quarters firefighting.
What a senior engineer actually does on these projects
Less code. More contracts. More attention to what the model sees and what the model is allowed to touch. The work looks closer to platform engineering than to traditional product engineering — you are building an environment that another system operates within, and you are responsible for what that other system can do inside it.
This is also why junior engineers find agent work harder than expected. The leverage point is no longer the syntax you write. It is the judgment about where in the system to put the loop, which tools to expose, and what to instrument.
Where we go from here
We expect the next twelve months to make this work less novel. Patterns will solidify, libraries will mature, and the questions our clients ask us will move from “can we build an agent” to “should this be an agent or a deterministic workflow.” Both answers will keep being correct depending on the problem.
If you are building something whose workflow can be expressed cleanly in a flow chart, ship the flow chart. If you are building something where the next step depends on context that only emerges during the work, that is when an agent earns its place in the architecture.
The thing to avoid is using the word “agent” to mean “we used an LLM somewhere.” That is the chatbot pattern, dressed up. And the cost of that confusion is going to keep being paid by the teams that ship it and the users who try to use it.
