On 28 July, the UK AI Security Institute detected unusual data transfers during a routine cyber evaluation. The agents under test had gone beyond the exercise. Across 122 runs, AISI recorded nineteen actions that reached real people and organisations online. In the most serious case, an agent researched the maintainers of an open-source project, created fake identities, and socially engineered a real person into accepting a malicious pull request.
AISI contained the activity within roughly an hour and found no evidence of harm. The setup was deliberately permissive, with internet access enabled and provider safety classifiers switched off to test what the underlying models could actually do.
That context matters. The agent was given credentials, a goal, and network access in an environment that was not designed to stop it from going further.
Security teams now face a version of the same question. Dan DeCloss, who has worked two decades across the US Department of Defense, Veracode, Mayo Clinic, and Anthem before founding PlexTrac, spoke to us about what he would hand over to an agent, what should remain with human testers, and what happens when an agent with live credentials goes somewhere it should not.
Let’s dive in.
THIS WEEK’S TECHNICAL SESSION
OWASP API Security Top 10 vulnerabilities, walked through in code
Siri Varma Vegiraju, a Tech Lead on Microsoft Azure Security, walked us through the OWASP API Security Top 10 with the vulnerable code beside the corrected version for each failure.
He covered authorization at three levels, what token validation means beyond decoding, the EC2 metadata attack end to end, and why the APIs nobody remembers deploying are the ones nobody audits.
THE INSIDER VIEW
Nobody Has Solved How to Revoke an Agent’s Access Mid-Engagement
Featuring Dan DeCloss, Founder of PlexTrac, now part of Brinqa.
Ask a security practitioner, one who works the offensive side of penetration testing, which parts of the job they would hand to an AI agent and the answer comes back quickly. Ask them how they would take that access back once the engagement is running and the answer gets much shorter.
Dan DeCloss, founder of PlexTrac, now part of Brinqa, answers this without hedging. “During an engagement, I think it’s kind of like it has what it has,” he says. If the agent is conducting the test, it will keep finding identities and credentials as it goes. The most you can do is keep prompting it that certain data is now restricted from use.
That is not a control. It is a request, made to a system that has already been given the network position to ignore it.
Hand over reconnaissance without thinking twice
The division DeCloss draws is clear, and it maps to how much context each phase requires.
He would quickly hand over any work around the initial discovery and reconnaissance stage. Gathering information about an organisation or an application, identifying what systems exist, enumerating an application far faster than a person could. “That’s definitely a lot of low-hanging fruit,” he notes. Reconnaissance is read-only work at volume, and nothing in it changes the state of a target system, which is what makes it safe to delegate first.
Guided testing makes another strong case. “Can you test these vectors for SQL injection, for cross-site scripting, for privilege escalation and lateral movement,” DeCloss adds, expecting agents to become exceptionally good at mimicking how attackers use known techniques to find and exploit weaknesses.
Where he believes scanners and agents will still fall short is with the business logic that makes manual testing so intrinsically valuable. He reasons with the example of a banking application where a specially crafted input lets someone bypass the intended process and access another customer’s account. A scanner may detect technical flaws, but it doesn’t necessarily understand who should have access to what. “You have to have a human tester that can say, oh yeah, I’m not supposed to be able to see Shant’s bank account (yes, he’s talking about me 😁) from my bank account,” DeCloss explains, making the point that the rule being broken exists in the business rather than anywhere in the code.
Keep the exploits that depend on context
The line moves as soon as an exploit runs against a live machine.
DeCloss frames the reason in terms of context rather than capability. A security engineer working manually knows whether the activity is noisy, whether the client asked them to avoid denial of service, and where the scope ends. Those constraints exist in a document and in the tester’s head, not in anything the agent can read.
Evidence handling worries him more than exploitation. “In penetration tests, you do collect sensitive information. You do end up dumping databases and credentials,” he tells us, and all of it has to stay inside the engagement, because an agent that moves that evidence somewhere it should not go turns a routine test into a disclosure incident. He wants tight control over how those artifacts move, and he would not hand that to an agent on the basis that it looked capable.
The AISI incident is the version of this failure that happened in public. An agent inside a controlled evaluation created accounts, fabricated identities and reached out to a real maintainer, and the people running it found out through anomalous network traffic rather than through anything the agent reported.
An agent’s credentials belong to whoever compromises it
DeCloss treats the agent as an identity, which is the framing that makes the risk legible.
“You have to make sure that you’re treating AI to the scope that you would want any other system or individual to have,” he says. Grant domain administrative access and you have granted it permanently, in the sense that anyone who compromises the model now holds the same access. The attack surface widens and the liability comes with it.
The principle of least privilege still applies, but the focus is on giving an agent only the access it needs to do its job while putting the right safeguards around that access. The security questions themselves are familiar. What changes is who or what holds those privileges, an agent that can reason toward a goal even though the data it trained on reflects nothing about an organisation’s specific rules, boundaries, or ways of working.
He also points to the supply chain as a broader concern, where a poisoned model or a compromised software package creates risk that spreads across every organisation relying on it. That exposure scales past the team running the test, which is why it belongs in the same conversation as scope rather than in a separate one.
Start with what the agent cannot do
DeCloss inverts the usual question for teams planning to bring an agent into next quarter’s work.
Instead of starting with what you want the agent to do, first define what it should not be allowed to do. Agents can take multiple actions in sequence and may find their own path toward a goal, including approaches that were never explicitly designed or anticipated. That makes it important to understand what the agent could access if something goes wrong and, more importantly, what controls would prevent it from going beyond its intended boundaries.
“Threat modelling the AI agent itself is probably the best exercise you can do,” he argues. Since agents operate with a fair degree of autonomy and their training does not account for an organisation’s scope or rules, those boundaries cannot rely on instructions in a prompt alone and need to be enforced elsewhere.
AISI reached a similar conclusion, noting that scope should be enforced at the infrastructure level through measures such as network egress policies and capability-scoped identities, rather than relying on the model to stay within its assigned limits.
Juniors trust the agent before they learn to doubt it
For DeCloss, the bigger concern around agents is not what happens during a single engagement, but what their growing use could mean for the next generation of security professionals.
Senior testers can use agents to move faster because experience tells them what to ask and when to question the answer they get back. DeCloss describes sessions where an agent initially reported that an environment was clean, only to find a vulnerability after he challenged the result and asked it to try a different approach. Knowing when to push back on that first answer comes from years of seeing what can be missed and recognising when something does not quite add up.
Junior testers are still building those instincts, which makes the risk of relying too heavily on an agent much greater. “They’re trusting the agent more than their intuition in themselves, and I think that’s where we have to continue to exercise caution as an industry,” DeCloss puts it.
That concern becomes more significant when looking at how junior testers eventually become senior ones. Much of that development comes from doing the work themselves, making mistakes, following the wrong leads, questioning results, and gradually learning to recognise patterns through experience. If agents start doing too much of that work for them, juniors could miss the very experiences that build independent judgment. “We might be having a gap between how we’re actually training and building up subject matter experts,” he warns.
That creates a difficult cycle as agents take on more security work. Experienced professionals will be needed to question their findings, catch what they miss, and keep their actions in check. But if the next generation has fewer opportunities to build that experience themselves, there may eventually be fewer people with the judgment needed to provide that oversight.
Frequency is the payoff, not headcount
DeCloss does not think agentic testing solves the staffing shortage, and he is unsentimental about the promise.
Human oversight remains necessary, while configuring the agents, scheduling tests, and interpreting the results all require time and expertise. “It’s not plug and play, where we now have our pen testing team through agentic AI,” he cautions.
Where agents make a real difference is in how often testing can happen. DeCloss has long argued that a penetration test only delivers value when the findings are addressed and the environment is tested again, allowing teams to measure improvement across cycles rather than through a single report. By making those cycles faster and easier to repeat, agents support more continuous assessment, which he sees as their real advantage.
Watch the full session below
THIS WEEK’S PERSON OF INTEREST
Simon Willison, Co-creator of Django, and creator of Datasette
In September 2022, Willison coined the term prompt injection, describing why a model cannot reliably separate the instructions it was given from the content it was asked to process.
In June 2025 he added the framing that now shows up in most agent security guidance. He called it the lethal trifecta. An agent with access to private data, exposure to untrusted content, and the ability to communicate externally is exploitable, and the three capabilities together are the vulnerability rather than any one of them.
The relevance to this issue is direct. A penetration testing agent checks all three boxes by design. It reads credentials and dumped databases, it processes whatever the target environment puts in front of it, and it reaches the open internet because reconnaissance requires that. Willison’s point is that this configuration has no clean fix, only trade-offs about which leg to remove.
He also covered the AISI disclosure within a day of publication, noting that the evaluation had deliberately disabled the providers’ cyber classifiers, which made the outcome considerably less surprising than the headlines suggested.
Outside security he co-created Django and built Datasette. His blog remains one of the few places tracking prompt injection continuously rather than in response to incidents.
Read: The lethal trifecta for AI agents
THIS WEEK’S SECURITY BRIEFS
AI evaluation lab discloses two intrusion attempts
METR, the non-profit that evaluates frontier models for long-horizon agentic capability, disclosed on 1 September that external actors twice reached for its systems, including a March theft of an inference API key that was used to burn through a substantial volume of credits. METR believes no sensitive information was accessed and states plainly that no AI agent broke into its evaluations. Neither attempt has been attributed to a known threat actor, and the organisation shared its findings with the AI companies it works with before publishing.
JFrog Artifactory authentication bypass exploited within days
CVE-2026-82329 carries a CVSS score of 9.8 and lets an unauthenticated attacker with network access obtain administrative privileges under default configuration. JFrog patched it in Artifactory 7.161.20 on 28 August, and watchTowr reported exploitation beginning days after public disclosure. Artifactory holds build artifacts and publishing credentials for the organisations running it, which makes administrative access there a route into everything downstream of the build.
Two unauthenticated RCE paths patched in Next.js
Vercel’s 25 August security release covered a libheif flaw reachable when Image Optimization processes a crafted AVIF file, and a separate Windows-only path traversal tracked as CVE-2026-75604. The patched versions disable AVIF optimization outright until libheif ships a fix upstream, and there is no workaround for the Windows issue beyond upgrading. The more severe of the two originated in a transitive dependency rather than in framework code, which is a different supply chain surface from the npm package compromises that have dominated recent coverage.
Microsoft patches 415 CVEs, one already exploited
August’s Patch Tuesday covered 415 vulnerabilities including 62 rated critical, with elevation of privilege accounting for 174 of the fixes and remote code execution another 109. A use-after-free in the afd.sys Windows kernel driver was already being exploited in the wild to gain SYSTEM privileges. Volume at this level has held steady through 2026, and there is no indication it returns to pre-2026 numbers.
Thank you for reading Offensive Engineering, a newsletter from InfoSec Relations. We will be back next week.
Stay curious, stay secure.
Shant Ebenezer Jena
Volunteer Host, Offensive Engineering