Your AI agent changed. Would you know if it got worse?

Software fails loudly. An agent fails quietly, by being a little less right than it was last month, and by then the only thing measuring it is your customers.

What happened

CIO ran a roundup of thirteen tools for evaluating and benchmarking agentic AI. Braintrust, LangSmith, Langfuse, DeepEval, Promptfoo, RAGAS, MLflow and the rest, sorted into three rough buckets: evaluation and benchmarking, observability, and guardrails.

No single one of those tools is the story. The story is that a roundup was worth writing at all. Two years ago you built an agent, tried it a few times, and shipped it. There is now a funded product category whose entire reason to exist is answering a question nobody used to ask out loud: is this thing still giving the right answer?

That question got asked because enough companies found out the hard way that the answer was no.

Why this matters to your business

Normal software breaks in a way you can see. A page returns an error, a job fails, someone gets paged. An agent does not do that. It keeps replying in the same confident tone it always used, and the replies are slightly worse.

Nothing alerts. The slide shows up sideways, weeks later, as a small rise in escalations, a handle time that crept up, a refund that went out when it should not have. By the time it reaches a dashboard, you have to work backwards to guess when it started.

There are a lot of ways for it to start. Someone edited the prompt to fix one complaint and moved four other things. Your model vendor pushed a new version under the same name. A colleague loaded 400 documents into the knowledge base, half of them outdated. Or a policy changed in January and the document the agent reads still says December.

The cost is not the mistake itself. It is the gap between the day it broke and the day someone noticed. If the only thing detecting that gap is a customer who got a bad answer, the gap is measured in weeks and paid for in trust.

Why this is a CX-Builder use case

Before you shop for a tool, build the asset none of them can give you: a list of real questions your agent gets, each with the answer you would want a good employee to give. Fifty is plenty to start. Two hundred is a serious test set. It comes out of last month's transcripts, and pulling it together is an afternoon of work by someone who knows the business, not an engineering project.

What you need after that is somewhere to run it, and here the shape of your platform matters. The agent under test has to be the agent your customers hit, not an approximation of it. In CX-Builder the flow is one thing you can see on a canvas and call over an API, so a test points straight at it. Same nodes, same retrieval, same model settings. There is no second copy to drift away from the first.

Running it on your own infrastructure matters more than it sounds, because a good test set is made of real customer conversations. Self-hosted means those transcripts never leave your environment to get scored in somebody else's cloud.

The last piece is the gate. A change to a prompt or a knowledge base should not reach customers because one person felt it read better. A human-in-the-loop step turns that into a decision someone owns, with a score attached to it.

What this looks like if you build it

It is a second flow, and it is smaller than you expect. Its input is your list of question and expected answer pairs. It loops through them, sends each one to the production agent, and scores the reply, either by asking a model to judge it against the expected answer or by checking that retrieval pulled the document it should have pulled. The output is a number per run and a list of the cases that failed.

Hang that off whatever already runs your nightly jobs, since the flow is an endpoint like any other. Put an approval step in front of prompt and knowledge base changes so the score gets read before the change ships. Keep the failed cases; they are the next version of the test set.

The tools in the CIO roundup do a version of this with better charts, and some are worth buying once you are at scale. None of them will write your questions for you.