AI Pentesting

Inside ARES: Three Systems, Fifteen Clusters, One Autonomous Red Team

We have been shipping product under the "Phalanxia" name for over a year. Today we are naming the engine publicly for the first time: ARES — Autonomous Red-team & Exposure System. This post is the technical introduction we would want to read if we were evaluating an AI pentesting vendor.

We will cover the three core systems that make ARES work, the fifteen specialized agent clusters that execute campaigns, and — because transparency matters in security — honest performance data including where we currently trail competitors and why.

Why Name It Now

The AI security market has a credibility problem: vendors cite detection rates without methodology, publish benchmark numbers that do not survive scrutiny, and avoid naming their underlying systems so they cannot be held accountable. We built ARES to be the exception. Naming it publicly is step one of holding ourselves to that standard.

The name matters for another reason: ARES has fifteen named agent clusters — SCOUT, ORACLE, BREACH, SPIDER, PRISM, FORGE, HAVOC, NIMBUS, SKELETON KEY, PHANTOM, GHOST, NOMAD, LENS, SCRIBE, ANVIL. These are not marketing labels; they are the actual module names in the codebase. When we say "SCOUT identified three undocumented subdomains" in a campaign report, that is the literal agent cluster that ran the recon. Naming creates accountability that generic "AI agents" language does not.

STRATEGOS — The Mission Orchestrator

STRATEGOS is the LLM-based mission planner at the center of every ARES campaign. It takes SCOUT's reconnaissance output and builds a directed acyclic task graph — a structured attack plan that assigns work to the right clusters, tracks dependencies between tasks, and dynamically adjusts priorities as the campaign develops.

The DAG approach matters because real attack campaigns are not sequential. While SPIDER probes web endpoints, BREACH can be testing network services in parallel, and ORACLE can be pulling threat intelligence on discovered infrastructure simultaneously. STRATEGOS manages this concurrency across all 76 sub-agents without letting parallel execution create conflicts or duplicated work.

One capability worth highlighting is game-theory-based target prioritization. STRATEGOS uses a Nash equilibrium model to identify the attack paths most likely to yield high-impact findings given the known constraints of the target environment and the capabilities of each cluster. In practice, this means ARES makes better decisions about where to spend campaign budget than a round-robin or severity-based scheduler would.

STRATEGOS also manages failure recovery. When a cluster stalls — no progress within a configurable window, or a failure loop detected — STRATEGOS reassigns tasks without operator intervention. Cross-session learning means findings and dead ends from prior campaigns against the same target inform the attack graph for future ones, so the system gets better at each environment the longer it runs.

SENTINEL — Safety By Design

The first question every security team asks before authorizing an autonomous agent to attack their infrastructure is: what stops it from going too far?

SENTINEL is the answer, and we designed it before we wrote the first cluster. This sequencing was intentional. Too many AI security products bolt safety on after the core capabilities are built, which means the safety layer never fully understands what the system can actually do. SENTINEL was designed in parallel with the attack clusters, so it has first-class visibility into every action any sub-agent can take.

SENTINEL implements a six-level autonomy spectrum. At L0, every proposed action requires operator approval before execution. At L5, the system runs fully autonomously within the defined Rules of Engagement file. Most enterprise customers start at L2 or L3 — enough autonomy to run campaigns overnight without constant supervision, enough control to require approval for high-impact actions such as exploitation attempts against production databases.

The Rules of Engagement file is a structured configuration that operators define before any campaign begins. It specifies in-scope targets, prohibited targets, allowed attack categories (recon, scanning, exploitation, post-exploitation), timing windows, rate limits, and escalation thresholds. SENTINEL enforces the ROE for every sub-agent action throughout the campaign — not as a post-hoc audit, but as a blocking constraint that fires before the action executes.

Finally: the kill switch. Any ARES campaign — in whole or in part — can be terminated in under 5 seconds. All active sub-agents receive the abort signal simultaneously via the NATS message bus. Tool execution containers (gVisor-sandboxed Docker) are torn down. Network activity stops. Findings are snapshotted in place. We have tested this at full scale and the 5-second bound holds.

ARES also supports air-gap deployment for environments that cannot allow outbound connections. In air-gap mode, local Llama or Mistral models replace the Claude API for execution tasks, and all data remains inside the network boundary. The capability set is reduced — local models are weaker than Claude Sonnet for complex reasoning — but the safety and ROE enforcement layers are identical.

CRUCIBLE — The Validation Engine

The detection-rate problem in AI pentesting is often framed as "finding more vulnerabilities." We think it is more precisely framed as "reporting findings you can actually trust." CRUCIBLE is our answer to the second framing.

Every potential finding from any cluster passes through CRUCIBLE's four-stage validation pipeline before it is accepted and queued for SCRIBE to report.

Stage 1: Schema validation. Does the finding have all required fields, are the types correct, is the evidence record complete? Malformed findings from agents in unusual states are caught here before consuming any additional compute.

Stage 2: Cross-agent deduplication. When SPIDER's web scanning sub-agents and PRISM's vulnerability research sub-agents both identify the same SQL injection endpoint from different angles, CRUCIBLE merges them into a single finding with both evidence chains attached. Without this stage, a campaign against a target with multiple access paths to the same vulnerability would generate redundant findings that waste remediation time.

Stage 3: Evidence grading. CRUCIBLE assigns each piece of evidence to one of four levels: L1 (weak indication — anomalous response, possible vulnerability signature), L2 (moderate confidence — reproducible behavior consistent with the vulnerability class), L3 (strong evidence — successful probe with response confirming exploitability), L4 (confirmed PoC — working exploit with extraction or execution sample). Findings without at least L2 evidence in their chain are rejected back to the originating cluster.

Stage 4: LLM reflector veto. A separate LLM call reviews the complete finding — description, evidence chain, attack steps used, cluster that originated it — and decides whether the evidence supports the stated conclusion. The reflector is prompted to be skeptical: its job is to find reasons the finding might be wrong. If the reflector vetoes, the finding returns to the cluster with the specific objection. The cluster can resolve the objection with additional evidence or abandon the finding. Nothing ships below ≥0.85 confidence.

CRUCIBLE adds latency to campaign completion. A finding that fails Stage 4 and needs additional evidence can add minutes to the campaign runtime. We think this is the right trade-off for enterprise customers who need to justify remediation effort to engineering teams — a list of fifty confirmed findings is more actionable than a list of two hundred probable ones.

The Fifteen Clusters

ARES deploys 76 sub-agents organized into 15 specialized clusters. Each cluster has a focused mission and a specific set of tools configured in its YAML manifest. The clusters are:

  • SCOUT (6 sub-agents) — Reconnaissance: passive OSINT, port scanning, service enumeration, certificate analysis, subdomain discovery. Every other cluster depends on SCOUT's output.
  • ORACLE (4 sub-agents) — Threat intelligence: CVE context enrichment, adversary TTP mapping, infrastructure reputation, feed correlation.
  • BREACH (5 sub-agents) — Network exploitation: service-level vulnerabilities, protocol weaknesses, firewall and ACL testing.
  • SPIDER (8 sub-agents) — Web and API: authentication testing, injection points, business logic flaws, API enumeration and abuse.
  • PRISM (5 sub-agents) — Vulnerability research: CVE matching against discovered services, version correlation, public PoC retrieval.
  • FORGE (6 sub-agents) — Exploit development: turning vulnerability research into working proof-of-concepts, payload generation, bypass techniques.
  • HAVOC (4 sub-agents) — Fuzzing: input mutation, boundary testing, state-space exploration for logic and memory vulnerabilities.
  • NIMBUS (5 sub-agents) — Cloud: IAM misconfiguration, storage bucket exposure, cloud-native service abuse, cross-account paths.
  • SKELETON KEY (6 sub-agents) — Active Directory and identity: Kerberoasting, AS-REP roasting, ACL abuse, privilege escalation paths through AD.
  • PHANTOM (8 sub-agents) — Post-exploitation: C2 channel simulation, lateral movement, persistence mechanisms, data staging.
  • GHOST (4 sub-agents) — Evasion: testing detection coverage by running attack patterns against the target's defensive tooling.
  • NOMAD (5 sub-agents) — Mobile: iOS and Android application testing, mobile API surface assessment.
  • LENS (4 sub-agents) — Code review: static analysis of discovered source code, secret scanning, dependency vulnerability identification.
  • SCRIBE (5 sub-agents) — Reporting: evidence synthesis, finding narratives, remediation template selection, executive summary generation.
  • ANVIL (4 sub-agents) — Tooling: shared infrastructure for tool installation, configuration, output parsing, and environment management.

The clusters with the strongest current coverage are SPIDER (web and API) and BREACH (network). SKELETON KEY is the most complex given Active Directory's surface area. NOMAD and NIMBUS are the most under-developed relative to their full spec — mobile and cloud coverage is where we are investing the most research effort right now.

The Benchmark Paradox

We compared ARES against twelve comparable open-source and published AI security systems in a structured benchmark. By almost every architectural metric — codebase size (88,103 LoC), agent count (15 clusters, 76 sub-agents), test coverage (2,054 passing tests across 140 test files), enterprise features (RBAC, multi-tenancy, Helm charts, audit logging), safety system depth (SENTINEL), validation pipeline rigor (CRUCIBLE) — ARES is larger and more comprehensive than any system we found.

The detection rate tells a different story. On DVWA v10 black-box assessment, ARES achieved approximately 53%. The best-performing comparable systems reach 90–96% on similar benchmarks, with codebases three to six times smaller.

We are publishing this because it is important context for anyone evaluating the platform. The gap is not in infrastructure — it is in agent reasoning quality. The clusters have the right tools. STRATEGOS builds reasonable attack graphs. CRUCIBLE enforces evidence standards. But the individual sub-agents do not yet use the causal reasoning graph and reflector systems as effectively as the architecture enables. The bottleneck is prompt engineering and the quality of reasoning inside the ReAct loop, not the scaffolding around it.

The v12 campaign detected 9 of 11 vulnerability classes on DVWA, which represents a meaningful improvement over v10. We have not yet run a statistically comparable benchmark against the v12 build at the same scale as v10, so we are not citing that as a detection rate. We will publish updated numbers when the benchmark is clean.

The architectural advantages — enterprise RBAC, SENTINEL safety, CRUCIBLE validation, cross-session learning, air-gap support, Helm deployment — are meaningful for regulated enterprise environments where detection rate is not the only variable. But we are not going to hide the gap. The improvement roadmap is specific: better causal graph utilization in the sub-agent ReAct loops, more aggressive scientific method enforcement in FORGE and SPIDER, and prompt engineering improvements to the CRUCIBLE reflector that reduce false negatives in evidence grading.

What This Means For You

If you are evaluating AI pentesting platforms today, ask every vendor three questions: What is your detection rate on a published benchmark, and what is the methodology? What happens when the operator wants to stop the campaign immediately? How does your system prevent a finding from being reported without evidence?

We have answered all three here — including the answer we are least proud of (the detection rate). We think that is more useful than a polished marketing claim, and we think it is the kind of transparency the security industry should hold AI vendors to.

ARES is in active development. If you want to run it against your environment, reach out. If you are running comparable systems and want to compare notes on agent reasoning or benchmark methodology, we are equally interested in that conversation.

Engage

See ARES in action on your own environment.

A 15-minute consultation. 30-day proof of concept available.