Offensive AI Security: A Technical Guide for Security Leaders
Executive summary
Your team is shipping more code than ever, and attackers are exploiting vulnerabilities faster than ever. Traditional cybersecurity practices aren't keeping up and you can't afford to wake up to a bad breach because of it.
AI is changing both sides of AppSec: attackers can move faster, but defenders can use similar capabilities to fix weaknesses before they're exploited.
Having built software and security at companies like Plaid, Kraken, and 0x, we know how hard it is to find and fix vulnerabilities while also shipping fast. The question is no longer whether AI can help find vulnerabilities but how to incorporate it into a security program in a way that's secure and most effective.
Offensive AI isn't just asking a coding agent to find vulnerabilities in your code. That can be a useful learning step, but there's much more that security leaders can do to implement an offensive AI security program that has dramatically fewer false positives (to protect your team's precious time), fewer false negatives (to avoid missing the critical vulnerability that leads to a breach), and strong guardrails to prevent offensive AI agents themselves from causing havoc.
There's no one right answer. The best approach depends on your organization's maturity, risk profile, and budget. That's what this guide is for: we cover where offensive AI is most useful, the guardrails you need, different levels of adoption, how to evaluate what you're getting, and a pragmatic rollout plan. We hope this helps you secure your application more quickly and easily, so you and your team can sleep easier at night 🙂.
Why offensive AI matters now
In the last 12 months, LLMs have made exploitation dramatically cheaper and faster:
Google's Threat Intelligence Group identified, for the first time, a threat actor using a zero-day exploit it believes was developed with AI.
Anthropic disrupted what it described as the first reported AI-orchestrated cyber espionage campaign, including autonomous exploit code.
Mandiant measured an average time-to-exploit of 32 days in 2022; by 2025 it was an estimated -7 days, meaning exploitation on average begins before a patch is even available.
However, LLMs haven't only benefited attackers. Defenders can now leverage modern AI agents to read an entire codebase, form hypotheses about where it's weak, and validate those hypotheses against a running application. That combination of autonomous code-level depth and dynamic validation simply wasn't possible before, and it addresses structural gaps in the tools most security programs rely on today:
- Traditional SAST: Poor signal-to-noise, misses whole vulnerability classes like business logic flaws, and needs constant tuning of custom rules.
- Traditional DAST: Good for breadth, but can't see the code, so struggles to validate exploitability or reason through multi-step attack paths.
- Bug bounty: Too much noise (especially in the AI era), and no whitebox testing of proprietary code, which is where defenders hold their biggest advantage in finding major vulns before attackers.
- Periodic pentesting: Creative human-led expertise is powerful but expensive, can't run continuously, and has highly variable quality across pentesters.
For security leaders, the reason to act now rather than later is simple: attackers are already using AI against your applications. Defenders should use AI to find and fix serious vulns before they're exploited.
What “offensive AI” means in practice
Offensive AI is the use of AI systems to emulate attacker-style reasoning against software systems: discovering, validating, prioritizing, and remediating vulnerabilities before real attackers exploit them.
For that to be useful in practice, an offensive AI system needs to be able to:
- Understand your codebase and unique business context.
- Identify real vulnerabilities.
- Explain the impact and exploitability of each finding.
- Produce proof-of-concept (PoC) scripts so you can quickly validate a finding.
- Provide remediation guidance so you can quickly fix issues.
- Continuously test as your code and the threat landscape change.
Just as important is what offensive AI is not:
- It is not a replacement for all human security expertise.
- It is not just another SAST or DAST scanner.
- It is not something to point at sensitive systems without appropriate guardrails.
Harmony's AI AppSec Agent was testing whether it could independently find a known Next.js auth bypass. In the process, it also discovered a previously unknown unauthenticated DoS vulnerability affecting self-hosted Next.js servers.
The agent validated the issue by writing and testing a proof-of-concept script against a staging app, after which Harmony's team reviewed the finding, responsibly disclosed it, and the vulnerability was patched.
Where offensive AI is most useful today
Offensive AI isn't equally valuable everywhere. It delivers the most value for:
- Security-sensitive applications: Offensive AI is especially valuable when your application has a meaningful attack surface, particularly in security-sensitive sectors like fintech, Web3, healthtech, govtech, or SaaS handling valuable customer data.
- Teams shipping fast: If your team ships frequently, annual pentests or occasional manual reviews leave vulnerabilities sitting in production for weeks or months before the next review.
- Teams with complex codebases or context: For a small app, pointing a general coding agent at the code may be enough. But as applications grow, purpose-designed offensive AI agents do better because they can break the problem down and better manage context to surface more findings with less noise.
- Teams mature enough to act on validated findings: Offensive AI adds value when your team has owners who can review and prioritize remediations. That load is lighter than it once was: AI can help with both validated findings and remediations, with humans (in-house or via a fully managed platform) serving as final reviewers.
0x secures infrastructure behind Coinbase, Robinhood, MetaMask, and Phantom, moving more than $200B across millions of wallets. Traditional pentesters lacked the application context to give the team confidence everything was covered.
To strengthen security for their users, 0x moved to fully managed AI agents for security testing, with findings validated against staging before reaching the team. After their first app audit with Harmony, they expanded coverage across all their security surface areas.
Guardrails: using offensive AI securely
Offensive AI means giving an AI agent attacker-style capabilities and access to your systems so that you can find vulnerabilities before a malicious actor does. It's powerful and can create new risks, which is why guardrails matter. Some of the guardrails you'd want in place are:
- Untrusted-by-default agents: Treat agent actions as untrusted by default. Agents are probabilistic, can overreach, and can be manipulated through prompt injection.
- Untrusted input handling: Treat external content, application responses, documents, logs, and other AI-consumed inputs as untrusted. Agents should not blindly follow instructions found in retrieved content or application data.
- Secure sandboxing: Run agents in isolated environments to reduce their blast radius.
- Scoped access controls: Put hard controls around the agent, not inside it: approved repos, services, environments, data, permissions, and test types only.
- Network-level egress restrictions: Use network-level egress restrictions so agents cannot reach arbitrary IPs or external services.
- Staging-first dynamic testing: Validate exploitability against staging or test environments rather than production, to reduce operational and customer-data risk.
- Logging and monitoring: Log and monitor agent actions for debugging and auditability.
- Defense in depth: Assume no single safeguard is perfect.
We manage thousands of AI agents that autonomously and continuously test our customers' applications for critical vulnerabilities. We protect our customers by deploying AI agent guardrails including:
- Sub-agent privileges are limited to the absolute minimum set they each need to complete their task.
- Agent outputs are treated as untrusted by default. Agent generated code runs in gVisor sandbox pods, with outbound network access restricted to a strict allowlist, so even a successfully prompt-injected agent has nowhere to exfiltrate data to.
- External inputs into an agent (e.g. resources from the web) are enclosed in delimiters so the agent clearly knows what is an instruction vs. untrusted input. Delimiters include a secret to avoid delimiter injection attacks.
- Every agent action is logged to LangSmith for auditability.
- Exploits are validated against staging, not production, by default to minimize risks to production systems & data.
3 approaches to adopting offensive AI
LLM-assisted security review
Use frontier LLMs directly to review code for potential security issues.
Where it works:
- Quick second opinions on code.
- Generating ideas for threat modeling and security design review.
- Helping developers understand possible attack paths before involving security teams.
Limitations:
- They don't dynamically validate whether a finding is exploitable, leading to noisy outputs.
- They can hallucinate issues or miss vulnerabilities that require broader application context or other agent harness features.
- They require evals, human review, and well designed guardrails to be effective and safe.
Open-source deep security review
Open source AI security agent harnesses that can help inspect broader parts of a codebase, use tools, and attempt deeper reasoning.
Where it works:
- Security teams with bandwidth to set up, implement, and maintain.
- Teams that want to experiment, learn and understand what AI agents can do before investing in a managed solution.
Limitations:
- Requires significant engineering time to evaluate, set up, and maintain.
- Unlike older security tooling, the maintenance burden is not just patching and uptime: teams also need to keep up-to-date with a very fast evolving AI landscape of model capabilities, agent frameworks, eval methods, and new attack/defense techniques.
- Quality and reliability can vary significantly.
- Guardrails, permissions, logging, and validation are often left to the team to design.
Fully managed offensive AI platform
A purpose-built system that combines agentic vulnerability discovery with human security expertise and guardrails.
Where it works:
- Security is mission critical.
- The organization has a complex codebase or high value attack surface.
- The team needs high quality, validated findings, not just AI-generated suggestions.
- The team doesn't have bandwidth to build or maintain the necessary infrastructure, or keep up with a fast-evolving AI offsec landscape of models, agents, etc.
Limitations:
- Costs more than running an LLM or open source harness yourself.
- You're relying on the vendor's methodology and guardrails, so due diligence matters (more on evaluation criteria below).
How to evaluate offensive AI tools or workflows
Here are some dimensions to think about when assessing an offensive AI solution:
- Methodology: soundness of the agent's approach to uncovering findings
- Clarity & accuracy of findings: findings are real and described clearly and accurately; where false positives exist, they are limited in number and are reasonable errors (e.g. missing context) rather than egregiously wrong
- Impact of findings: the more findings that are severe, exploitable, and business-relevant, the better
- Scalability & cost: the approach is scalable to your real world needs, including price and feasibility
- Ease: the workflow is easy to adopt for security and engineering teams
- Agent security guardrails: agent access is scoped, monitored, and contained, with clear controls for data handling, egress restrictions, auditability, and prompt injection risk
A pragmatic rollout plan
- Pick a narrow, high-value scope.
- Choose the right starting level based on your organization's maturity, risk profile, budget, time availability, and ability to operate AI securely:
Level 1 Level 2 Level 3 Plan Run a section of your codebase through a frontier LLM Implement an open source AI security agent harness Trial a fully managed offensive AI platform Ideal use case Fast, low-cost experiment If you have internal engineering bandwidth If your app is highly sensitive and you need validated outcomes - Compare outputs against evaluation criteria: methodology, accuracy, impact, scalability, ease, and guardrails.
- Implement your ongoing offensive AI program: once you've picked an approach, move from a one-off trial to continuous coverage, so new code is tested as it ships rather than waiting for the next review cycle.
Conclusion & key takeaways
Offensive AI is an important part of modern AppSec. The most mature AI offensive security testing programs combine state-of-the-art AI systems, human expertise, strong guardrails, and ongoing coverage.
There's a path for every team. Off-the-shelf LLMs and open source options are useful for quick, low cost experimentation. If your application is security sensitive, your team ships quickly, and you need validated findings without the ongoing AI infrastructure maintenance burden, you should consider a fully managed offensive AI program.
Whichever level you start at, the important thing is to start. Attackers are already using AI against your applications. The defenders who avoid bad outcomes will be those that got ahead of the risk.
About Harmony Intelligence
Harmony Intelligence builds AI agents that continuously battle test your web app or API to find exploitable vulnerabilities. Our fully managed continuous security testing platform empowers engineering teams to ship fast without waking up to a breach. We work incredibly hard as long-term infosec partners to keep you at the bleeding edge of AI and cybersecurity.
If you want to see what offensive AI can find in your app, book a demo with Harmony.