Someone on the platform team says it in a planning meeting, and it lands with the finality of a closed door: "We turned on AWS Security Agent last sprint. We're covered." Everyone nods. Nobody in the room can actually say what "covered" means, and the meeting moves on. That silence is the subject of this post.
Here is the thesis up front. AWS Security Agent is a genuinely good product with a clearly drawn boundary. It is not a toy and it is not marketing vapour. Your job is not to argue with the boundary, it is to know precisely where it sits relative to your actual attack surface. That question arrives with a deadline attached: a SOC 2 window, an ISO 27001 surveillance audit, a PCI cycle, or an enterprise questionnaire with a line on page four asking whether your last penetration test was performed by an independent third party. Meanwhile engineering has found a $50-an-hour answer that runs inside the CI pipeline, and internally that answer is winning.
Getting this wrong usually fails slowly rather than loudly. In week six of an audit, or three days before a deal closes, you find out that the thing you were counting on does not cover the thing you were asked about. Occasionally it fails the other way and faster: the asset nobody declared tends to be the asset nobody patched, and forgotten subdomains, abandoned admin panels, and dangling DNS records are a well-worn initial access route. Both failure modes trace back to the same unexamined word in that meeting.
What AWS Security Agent Actually Does (No Straw Man)
A comparison built on a weak version of the other product is worthless to you, so let's be accurate and generous.
Every factual statement about AWS Security Agent in this post is drawn from AWS's own published documentation and is stated as at 24 July 2026. AWS ships changes quickly, so check the primary sources linked throughout before you rely on any of it.
AWS Security Agent's on-demand penetration testing reached general availability on 31 March 2026 in six regions: US East (N. Virginia), US West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Sydney), and Asia Pacific (Tokyo). The AWS Security Agent user guide describes a multi-agent system rather than a scanner with a chat interface, and that description holds up.
The interesting part is what it does before it attacks anything. The agent builds application context from your source code, documentation, and architecture diagrams, then uses that understanding to plan multi-step attack scenarios against the running application, folding static analysis, dynamic analysis, and penetration testing into one context-aware pipeline. That is a real architectural advantage: a black-box tester has to infer your authorisation model from the outside, while this agent can read it. It then attempts real exploitation with targeted payloads and attack chains and verifies that what it found is genuinely exploitable, which is the step that separates a penetration test from a scan.
The output is a downloadable PDF that will look familiar to anyone who has read a consultancy deliverable: executive summary, scope statement, methodology, and per-finding detail with CVSS v3.1 vectors, steps to reproduce, and remediation guidance. The integration story is equally strong. Source connects from GitHub (including GitHub Enterprise), GitLab, Bitbucket, or S3, with Confluence for documentation context. Kiro and Claude Code plugins plus an open MCP integration mean reviews run from an IDE or CLI, and it supports cross-account testing and pentests against shared VPCs.
Since GA the product has kept moving, and a fair account has to say so. AWS added threat modelling, Kiro support, and a Claude Code plugin in June 2026: the agent will now analyse design documents or source and enumerate threats against the STRIDE framework, in public preview at time of writing. It also posts findings as pull request and merge request comments on connected repositories, and can be configured to open pull requests carrying the fix rather than only describing it. AWS Security Agent has since been folded into AWS Continuum. None of that changes the boundary this post is about, but a comparison that left it out would be selling you a weaker competitor than the one you are actually evaluating.
AWS also publishes a two-month free trial for new customers, with each trial month including up to 200 pentesting task-hours, so up to 400 across the trial. Pricing and trial terms above are as at 24 July 2026; check the AWS Security Agent pricing page for the current terms before budgeting against them. Underneath the pricing, the service runs as three connected stages rather than a single scan pass.
Where it is genuinely excellent
Velocity is the headline. A test that took six weeks to schedule, run, and report now completes in hours, which changes what testing is for. It stops being an annual event you prepare for and becomes something that happens because a pull request merged.
The code context is the real differentiator. Most tools guess at your application's intent from the outside; this one reads the source, the docs, and the architecture, so its attack scenarios follow how the system is supposed to work rather than a signature library. It also fits the CI loop in a way traditional testing never has, and continuous coverage that keeps pace with deploy cadence is worth more week to week than a snapshot that is stale by Tuesday. We made that case in our guide to agentic pentesting; AWS has built one of the better implementations of it.
So the product is good. The question was never whether it is good. The question is what it is pointed at.
The Coverage Map, Part One: What Sits Inside the Boundary
Everything inside the boundary shares one property, and it is worth naming before you read the list: you declared it. Tick these against the environment you actually have, not the one on the architecture diagram. If every line applies to you, AWS Security Agent may genuinely be all you need.
One AWS-native application, source in a supported repo, staging that faithfully mirrors production, nobody outside the company reading the output. That is a real profile, plenty of companies fit it, and if you are one of them you should stop reading and go run a test.
Most companies between 200 and 2,000 people do not fit it. The question is which boxes fail to tick, and how much of your risk lives there.
The Coverage Map, Part Two: What Sits Outside It
Each gap below comes with a test you can apply to your own environment in about a minute. None of these are flaws. They are boundaries, and boundaries are what well-designed products have.
Gap 1: The estate that is not an application endpoint
The real boundary is the shape of the target rather than its address. AWS documents this as an application and API penetration testing agent: it tests web applications and APIs, and does its best work when it can read the source. Your estate contains a great deal that is not an application endpoint. Your Entra ID tenant and its conditional access policies. The on-prem Active Directory domain and the trusts inside it. GCP IAM bindings on the project the data team spun up two years ago. Identity and configuration posture in a non-AWS control plane is not a URL, so on AWS's published description of the service it sits outside what the agent is built to assess, and a meaningful share of real intrusions start exactly there.
Does this apply to you? Think about your last three incidents or near-misses. Were they application vulnerabilities, or identity, configuration, and network path problems? If it is the second, the agent is not looking at the layer where your risk lives. And at 200 to 2,000 people, genuinely single-cloud estates are rare in practice. There is nearly always a residue of acquired, legacy, or departmental systems somewhere else, and that residue tends to be the least maintained part of the estate precisely because nobody owns it.
That gap is about the shape of your estate. The next one is about the shape of your paperwork, and it is the one that costs teams the most.
Gap 2: The attack surface you forgot to declare
This is the most important gap on the list, and it is structural rather than technical.
Scope is configured. You declare the resources, list the endpoints, and prove ownership of every target domain, which AWS requires you to establish before a test can run, via a DNS TXT record, an HTTP route that returns a validation string, or Route 53 for domains you host there. That is exactly right from a safety standpoint, and it has an unavoidable consequence: the agent tests what you told it about. Nothing in that model reaches the staging subdomain a contractor stood up in 2024, the marketing microsite on a domain that never made the asset register, or the bucket serving a legacy admin panel three people still bookmark, because none of them are in the config.
Attackers do not work from your scope configuration. They enumerate. Your real external surface is almost always larger than your declared one, and the delta is where the interesting findings live, because the delta is the part nobody maintains.
Does this apply to you? If that comparison returns hosts you cannot immediately name an owner for, you have just measured the gap. The size of the number matters less than whether it is zero, and it is rarely zero.
Gap 3: Third-party and SaaS surface
Does this apply to you? Count the OAuth-connected applications in your identity provider, then count the ones a human has reviewed in the past twelve months. Our free AI agent security check covers the newest slice of this surface.
Gap 4: Production reality and the 'mirrors production' assumption
The catch is in the phrase "mirrors production." Almost nobody's staging is a faithful mirror. The security group that is tighter in staging. The WAF rule that only exists in prod. The IAM role that accumulated three extra policies during an incident and was never cleaned up. Drift between prod and non-prod is the normal state of a fast-moving engineering organisation, and it is precisely the class of finding an independent test against the real production surface catches and a non-production test cannot.
Does this apply to you? Ask whoever owns your infrastructure-as-code when they last diffed production against staging. The length of the pause is the answer.
Gap 5: Human-creative attack classes
Business logic abuse, multi-step authorisation bypass that depends on knowing what your application is for, and paths that chain a social element to a technical one all still resist automation. This is not specific to AWS Security Agent. It is true of every automated approach including ours, and we would rather say so plainly than pretend otherwise: automation covers the systematic majority of an attack surface properly, and the remainder still rewards human creativity. We argued the full case in when automated pentesting is enough (and when it isn't). If your application moves money or manages multi-tenant permissions, it applies to you.
That is the coverage map. There is a second boundary that has nothing to do with coverage, and for most readers it is the one that decides the question.
See what your external surface exposes, mapped to the controls it touches.
Run a free External Security Check →The Second Boundary: A Test You Run on Yourself
Assume coverage is solved. Assume your whole estate is one AWS-native application, the scope config is perfect, and the agent finds every real issue. You could still be stuck, and it would have nothing to do with the engine. The report is technically strong. The question is who it is addressed to.
Every framework you report against treats a penetration test as an assessment, and assessments derive their weight partly from independence. It is worth being precise about what these frameworks actually say, because your auditor will be, and because this is the point where vendor marketing usually overreaches.
SOC 2's CC4.1 maps to COSO Principle 16 and is built around ongoing and separate evaluations performed objectively, per the AICPA's Trust Services Criteria. It does not name penetration testing and it does not require an external tester. In ISO 27001:2022, Annex A 8.8 covers management of technical vulnerabilities and carries no independence requirement of its own; the independence theme sits in Annex A 5.35, independent review of information security, and clause 9.2 requires internal audits to be planned and conducted in a way that ensures objectivity and impartiality. Clause 9.2 governs ISMS internal audit rather than penetration testing, so treat it as the standard's general posture on self-assessment rather than a rule about who runs your pentest. PCI DSS 11.4.1 is the most explicit of the four and also the most commonly misquoted: it requires organisational independence of the tester and expressly permits a qualified internal resource, noting the tester need not be a QSA or ASV. APRA CPS 234, which binds APRA-regulated entities, requires testing to be conducted by appropriately skilled and functionally independent specialists.
Framework requirements are summarised here for orientation, not as compliance advice. Confirm the current text of any control with your auditor, QSA, or assessor before you rely on it.
Read together, the requirement is not "an outsider must do it". It is that the tester must be independent of the people who build and run the system under test. None of these say a human must type the commands. All of them care who stands behind the result, and our guide to SOC 2 penetration testing requirements walks the control mapping in detail.
That is where a self-run agent gets into difficulty, and it is worth being exact about why, because the sloppy version of this argument does not survive a conversation with a QSA. If the platform team that owns the application also scopes the test, configures it, runs it, and exports the report, the tester is not independent of the system under test, whatever the engine quality. A large organisation with a security function genuinely separated from engineering could argue the qualified-internal-resource path under PCI DSS 11.4.1 and make it stick. Most companies between 200 and 2,000 people cannot, because the same team owns the application, the pipeline, and the scope config. The principle cuts against us too, and it is exactly why we do not sell you a scanner to point at yourself. A CyberOrbit engagement is scoped and run by us, never self-run, and the report is reviewed and signed by a certified security professional who carries accountability for what it says. Independence is not a setting you switch on. It is a property of who did the work.
We have written the long version in what auditors actually accept from AI pentest reports and, for compliance platforms, in Vanta's pentest options and auditor independence. The auditor was never asking whether a human or a machine found the bug. They were asking whether an independent, accountable party proved it and signed for it.
If independence decides the evidence question, the next thing your finance lead will ask is what the alternative costs. That number is smaller than most vendors admit, and larger than the headline rate.
The Cost Question ($50 a Task-Hour Is Not the Whole Number)
At $50 per task-hour, and on AWS's own worked example of a 24 task-hour evaluation, a comprehensive AWS Security Agent run costs up to about $1,200. Against a traditional consultancy engagement, which commonly quotes somewhere in the $15,000 to $30,000 band depending on scope, that is not close, and any vendor telling you otherwise is hoping you cannot divide.
Treat that band as the consultancy tier rather than the going rate. Published 2026 pentest pricing guides put most organisations between $10,000 and $30,000 per engagement, with an all-types average nearer $18,300, and scoped web application testing spanning roughly $5,000 to $30,000 depending on roles, integrations, and multi-tenant complexity. A $15,000 to $30,000 quote is entirely real; it just sits in the upper half of the market rather than defining it.
Separate two things that get conflated at this point in the conversation. That $15,000 to $30,000 band is the price of the traditional delivery model: weeks of scheduling, a statement of work, and a consultant's calendar. It is not the price of independence. Independent testing now spans a wide band, and the six-week consultancy engagement sits at the top of it, so do not read the headline comparison as "cheap agent versus expensive independence."
The number that matters is the total cost of the outcome you need, and it has four components. Task-hour spend across a real estate rather than one application, which multiplies with every distinct application and environment. Engineering time to define and maintain scope, which is salaried hours from people who are not cheap. Triage, because a genuine finding still needs someone to decide whether it matters in context, and a test that produces 34 findings produces 34 decisions. And whatever you buy separately to cover the surface the agent does not reach and to produce a signed, independent deliverable.
Sometimes that arithmetic still favours the agent alone. If you run one AWS-native application, your driver is internal assurance rather than an external audit, and your engineers act on findings directly without needing a formal report, then AWS Security Agent alone is not just cheaper but better: continuous, in the loop, and unencumbered by process you do not need. Buying an independent engagement in that situation is spending money to solve a problem you do not have.
For the wider baseline on what the alternatives cost, our penetration testing cost breakdown sets it out. The point here is narrower: compare total cost of outcome, not headline rate.
How to Decide (and Why It's Usually Not Either/Or)
Three profiles. Find yours.
AWS Security Agent alone is enough when you run a single AWS-native application, have no external compliance driver, need internal assurance only, and your security function is engineering-led. Run it in CI, act on the findings, spend the saved budget elsewhere.
You need both when any one of these is true: your estate is multi-cloud or hybrid, you have a SOC 2, ISO 27001, PCI DSS, or DORA driver, enterprise customers read your pentest report during procurement, or your infrastructure was assembled through acquisitions and nobody has a complete map. This is the majority profile at 200 to 2,000 people, and it is not a compromise. It is two tools doing two different jobs.
You need a full human red team when your application has genuinely novel business logic, when you are subject to regulated threat-led testing (TIBER-EU, CBEST, and similar regimes), or when your threat model includes a determined human adversary probing your people and processes.
Laid side by side, the trade is easy to see, and it is a trade rather than a verdict.
- Hours, not weeks: a test that once took six weeks to schedule now completes inside a sprint, so testing follows deploys instead of the audit calendar
- Reads your source, docs, and architecture, so attack scenarios follow your real authorisation model rather than a signature library
- CI-native via IDE and CLI plugins plus open MCP integration, which makes continuous coverage a pipeline step rather than a project
- Up to about $1,200 on AWS's own 24 task-hour worked example, metered per second with no minimum commitment
- No procurement cycle, no statement of work, no waiting on a consultancy's calendar
- Scope is configured, not discovered, so the forgotten subdomain and the shadow microservice never enter scope at all
- Requires an AWS account to run, which puts it inside your own control plane by design
- Produces no independent attestation, because you scope, run, and export the test yourself
- Non-application surface sits outside AWS's documented scope for the service: Entra ID conditional access, on-prem AD trusts, GCP IAM bindings, third-party SaaS and OAuth grants
- Independence has to come from your org chart rather than the tool: where the team that builds and runs the application also scopes and runs the test, that is a weak position to take to a QSA under PCI DSS 11.4.1 or to an auditor assessing objectivity under SOC 2 CC4.1, and to functional independence under APRA CPS 234, whatever the engine quality. A genuinely separated internal security function is a different case, and PCI DSS 11.4.1 expressly allows for it
For most readers the answer is the layered one. Run the agent in your CI loop for continuous coverage of the application surface, where its code context and speed are best in class. Run an independent scoped test across your externally reachable surface, discovered rather than only declared, on the cadence your compliance driver requires. These are complements: the agent gives you coverage between audits, the independent test gives you evidence at audit.
Be clear-eyed about what that second layer does and does not close. An independent external assessment closes the declaration gap, because the surface is enumerated rather than typed into a config, and it closes the evidence gap, because a third party scopes it, runs it, and signs it. It does not close the identity and configuration posture gap on its own. Entra ID conditional access policies, on-premises Active Directory trusts, and GCP IAM bindings are assessed from inside those control planes with credentials, which is a configuration review rather than an external penetration test, and you should buy it as such. Anyone who tells you one engagement covers every layer of your estate is selling you the word "everything" rather than a scope.
Here is a four-step evaluation you can run this week without buying anything. Steps three and four need no budget and no vendor call: our free subdomain finder enumerates the surface you did not declare, and the free header checker tells you whether the hosts it turns up are maintained by anyone. Run both only against domains you own or are authorised to assess.
Answer the fourth question first and the other three get much easier. If the answer is "an auditor," the layered approach is not optional, and the independent half of it is the part you cannot build yourself. That is the half we do: scoped and run by us against your real external surface rather than only the list you supply, evidenced with actual HTTP captures and reproduction steps, and signed by someone who puts their name to it.
Frequently Asked Questions
Does AWS Security Agent replace a penetration test?
What does AWS Security Agent cover and what does it miss?
Will my auditor accept an AWS Security Agent report for SOC 2?
Can AWS Security Agent test Azure, GCP, or on-premises systems?
How much does AWS Security Agent cost per pentest?
What is the difference between AWS Security Agent and an independent pentest platform?
Do I need an independent pentest if I am fully on AWS?
Sources
- AWS Security Agent on-demand penetration testing is now generally available, AWS, 31 March 2026
- AWS Security Agent adds support for penetration tests on shared VPCs across AWS accounts, AWS, 25 February 2026
- AWS Security Agent pricing and FAQs, AWS
- AWS Security Agent User Guide and security best practices, AWS
- AICPA Trust Services Criteria
- APRA Prudential Standard CPS 234 Information Security
- Claranet: Autonomous Penetration Testing Agents, AWS Security Agent, and the Compliance Question
Amazon Web Services, AWS, and AWS Security Agent are trademarks of Amazon.com, Inc. or its affiliates. Entra ID is a trademark of Microsoft Corporation. Google Cloud and GCP are trademarks of Google LLC. Kali Linux is a trademark of OffSec Services Limited. CREST is a trademark of CREST GB Ltd. Claranet is a trademark of Claranet Group Limited. All other marks are the property of their respective owners. These marks are used here only to identify and comment on the products and organisations referred to. CyberOrbit AI Pty Ltd is not affiliated with, endorsed by, or sponsored by any of them.
This post is general information about security testing and compliance frameworks. It is not legal, audit, or compliance advice, and it is not a substitute for the current text of any standard or the judgement of your own auditor or assessor. Statements about third-party products describe those products as publicly documented as at 24 July 2026 and may change without notice.