Attackers already use AI to find and exploit holes in minutes. The defence that keeps pace is an attack suite of your own, run against your customer agent every time it changes.
Armory Defense, an Israeli security company, announced Offensive Agentic-AI Simulations. Autonomous agents behave like attackers against a company's internet-facing systems, try to prove which weaknesses can actually be exploited, and report the ones that worked along with the evidence. The simulations are grouped into what the company calls Hackbooks, covering login abuse, API exploitation, database attacks and known vulnerabilities, and they run continuously instead of once a year.
The release backs this with one example, a VPN flaw rated 9.8 out of 10 that it says was exploited within 40 minutes using AI-driven workflows. Whatever the exact figure in your own industry, the direction is plain. The gap between a weakness appearing and someone using it keeps shrinking, and a test that ran in March says little about November.
Armory points its agents at servers, logins and APIs. Most businesses now have one more thing facing the internet that those tools were never built to probe: the customer agent. It looks up orders and, more and more often, issues refunds or changes accounts. Anyone can type into it, as often as they like, for free.
That makes it an attack surface with an awkward property. It changes every week. Someone edits the prompt to stop it recommending a discontinued product. A developer adds a tool so it can check delivery status. The team swaps the model because the new one is cheaper. Each of those is a release, each can reopen a hole that was closed last month, and the annual penetration test sees none of them.
The cost of a miss is easy to picture. Another customer's address read out in a chat. A refund issued on a convincing story. A discount code invented on request and then honoured because the customer kept a screenshot. Those land on support, finance and legal, usually weeks after whoever edited the prompt has moved on to something else.
The idea in Armory's release carries over to customer agents with little change. Stop counting theoretical risks and keep a running set of real attacks, each one either succeeding or failing against the system as it stands today. For a customer agent, an attack is a conversation: a message that tries to pull out another customer's data, talk the agent past a refund limit, get it to reveal its instructions, or follow orders hidden inside a pasted email.
CX-Builder already has the parts. Evaluations keep a dataset of inputs, run them against a chatflow or agentflow, and score each answer with evaluators, including one where a model grades the reply against a rubric you write. Load that dataset with attacks instead of FAQs and you have an attack suite. Each run gives a pass rate for that version of the agent, so you can see which change broke what.
The second part is an attacker that improvises. An agentflow can start on a schedule, have one agent write fresh variations on your known attacks, send them to your customer agent over HTTP, and have a second agent judge the replies. Anything that gets through goes to a person for review, and the attack that worked joins the fixed dataset so every later run checks for it.
Start with a dataset of thirty to fifty attack messages, written with whoever handles your escalations, since they already know what customers try. Write evaluators that fail a reply if it contains another customer's details, promises money above your limit, or repeats the system prompt. Run the evaluation before any prompt, tool or model change goes live, and treat a drop in the pass rate the way your engineers treat a failed build.
Then add the scheduled attacker flow, aimed at a staging copy of the agent rather than production, so improvised attacks never touch a real customer record. Self-hosting helps here too. The attack transcripts are a map of your weak points, and they stay on your own servers.
Ask who last tried to break your customer agent on purpose, and when. If the answer is nobody, or only at launch, write twenty attack messages this month and run them against the agent before its next change ships. Every failure you find becomes a test that runs on every release after it.