Stop Sending Humans to Do a Machine's Job: agent-smith Exposed

What if your next penetration test didn't require a human at the keyboard? What if an AI could think like a hacker, adapt like a predator, and report like a senior consultant—all while you watch from a live dashboard?

Here's the uncomfortable truth: most security teams are burning six figures on manual pentests that miss critical attack paths. Human testers get tired. They follow checklists. They reuse the same payloads they've been carrying since 2019. Meanwhile, attackers are innovating in real-time, chaining zero-days with AI-generated exploits that no signature-based tool can catch.

The gap between attacker innovation and defender methodology has never been wider. But what if the defenders could weaponize that same AI advantage?

Enter agent-smith—the open-source AI-driven offensive security agent that's making human-only pentests look like fax machines in a Slack world. Built by the team at nullpointer.studio, this isn't another scanner with a fancy wrapper. This is a paradigm shift: the LLM becomes the brain, and the tools become its hands.

In this deep dive, I'll expose exactly how agent-smith works, why its "skills-as-pattern-teaching" approach destroys traditional automation, and how you can deploy it today. Whether you're a red teamer, appsec engineer, or CISO desperate for coverage that actually scales—this changes everything.

What is agent-smith?

agent-smith is an open-source, AI-driven offensive security framework that transforms large language models into autonomous penetration testing agents. Created by 0x0pointer and maintained under nullpointer.studio, it represents a fundamental departure from conventional security automation.

The project's tagline—"Never send a human to do a machine's job"—isn't marketing fluff. It's a mission statement backed by architecture.

Unlike traditional tools that ship static payload libraries and execute linearly, agent-smith treats skills as methodological teachings. Each skill is a carefully crafted prompt that teaches the LLM how to think about a vulnerability class—not what specific string to inject. The LLM then invents contextually appropriate attacks based on live reconnaissance data.

This matters because no two scans look identical. Run agent-smith against the same target twice, and you'll get different attack paths, different pivot sequences, and often different findings. The agent adapts to what it discovers in real-time, chaining skills dynamically based on intermediate results.

The project has gained serious traction in the security community for three reasons:

• Bring-your-own-LLM flexibility: Works with Claude Code, OpenCode (supporting OpenAI, Gemini, Ollama, OpenRouter, local models), or any MCP-capable client
• End-to-end automation: From reconnaissance through CVE submission packages—fully autonomous
• Sandboxed execution: Every scanner runs in ephemeral Docker containers with hard cost/time/call-count limits

With 25+ cybersecurity skills spanning penetration testing, cloud security, AI red-teaming, and automated remediation, agent-smith isn't a tool—it's a platform for autonomous offensive security.

Key Features That Destroy Traditional Pentest Automation

🧠 The LLM Is the Brain, Not a Payload Library

Traditional scanners ship 10,000 payloads and pray one hits. agent-smith ships methodology and lets the LLM generate targeted attacks. A SQL injection skill describes the vulnerability pattern, validation logic, and extraction strategies—the LLM crafts the actual union-based, error-based, or blind boolean payload based on the specific database fingerprint it just discovered.

This is the difference between a script kiddy and a senior pentester.

🔗 Skills Chain Themselves Autonomously

The /pentester skill doesn't just run reconnaissance and stop. When it discovers an injection point, it pivots into /web-exploit. When /codebase finds an LLM call site during code review, it pivots into /ai-redteam. The agent maintains state, evaluates findings, and decides what to run next—no human intervention required.

🛠 Bring Your Own LLM

Vendor lock-in is dead. agent-smith speaks MCP (Model Context Protocol), meaning any compatible client can drive it:

📦 End-to-End Deliverables

Stop paying consultants for PDFs nobody reads. agent-smith generates:

• findings.json with machine-readable structured data
• Burp-ready .http PoC files in pocs/
• Live topology diagrams and coverage matrices
• Auto-generated code patches via /remediate
• GitHub issue templates via /gh-export
• Complete CVE submission packages via /request-cves

🐳 Sandboxed by Default

Every scanner executes in an ephemeral Docker container. The Kali Linux container (~3GB) handles deep exploitation tools. The Metasploit container validates exploits. Lightweight tools (nmap, nuclei, httpx, ffuf, semgrep, trufflehog) auto-pull on first use.

Hard limits prevent runaway costs: server-side enforcement of cost, time, and call-count budgets.

📊 Live Dashboard

Watch your attack surface materialize in real-time at localhost:7777. Six tabs track findings, topology, components, coverage matrices, threat models, and execution logs. This isn't post-scan reporting—this is live situational awareness.

Real-World Use Cases Where agent-smith Dominates

1. Full Autonomous Pentest: Zero to Report

The scenario: Your startup just deployed a new staging environment. Compliance needs a pentest report in 48 hours. Budget: $0.

The command:

/pentester scan https://staging.example.com depth=thorough

What happens: OSINT → subdomain enumeration → port scanning → service fingerprinting → web exploitation → post-exploitation → lateral movement assessment → automated reporting. The agent decides every pivot. You get findings.json, PoC files, a topology diagram, coverage matrix, and ready-to-commit patches.

Time saved: 40+ hours of manual work.

2. Pre-Production Secure Code Review

The scenario: Your team ships a new microservice. You need ASVS 5.0 coverage across 16 chapters and 427 requirements before production.

The command:

/codebase path=./src

What happens: White-box analysis maps every route, every sink, every dangerous pattern. The output enriches downstream skills—/pentester now knows exactly where to focus, /web-exploit has precise injection points, and /ai-redteam identifies LLM call sites for adversarial testing.

3. Dependency CVE Triage with Proof-of-Concept

The scenario: SCA flags lodash 4.17.20 with CVE-2021-23337. Is this actually exploitable in your code, or noise?

The command:

/analyze-cve lodash 4.17.20 CVE-2021-23337

What happens: The agent traces the vulnerable function from user input to sink, determines exploitability in your specific context, and—if confirmed—writes a Burp-ready PoC for your security team to validate.

4. AI/LLM Application Red-Teaming

The scenario: Your product has a chatbot interface. You need OWASP LLM Top 10 coverage plus the new MCP runtime attack surface.

The command:

/ai-redteam https://your-app.com/api/chat provider=openai depth=thorough

What happens: Prompt injection, jailbreaks, model extraction, MCP runtime attacks, and post-access infrastructure checks. Payloads generate on-the-fly using FuzzyAI, Garak, PyRIT, and promptfoo. This isn't a static checklist—it's adversarial AI versus your AI.

5. Threat Modeling Without the Whiteboard

The scenario: Architecture review for a new payment flow. You need PASTA + STRIDE + risk prioritization.

The command:

/threat-modeling

What happens: Component map, data flow diagram, attack tree, prioritized risk register, and mitigation plan—all generated from system context the agent gathered from previous scans.

Step-by-Step Installation & Setup Guide

Prerequisites

Install Poetry if missing:

curl -sSL https://install.python-poetry.org | python3 -

Installation Paths

Option A: Claude Code (Recommended for UX)

git clone --recursive https://github.com/0x0pointer/agent-smith
cd agent-smith
./installers/install.sh

Requires Claude Code and an Anthropic API key.

Option B: OpenCode (Bring Your Own LLM)

git clone --recursive https://github.com/0x0pointer/agent-smith
cd agent-smith
./installers/install_opencode.sh

Configure your model in ~/.config/opencode/opencode.json. Supports OpenAI, Anthropic, Google, OpenRouter, Ollama, llama.cpp, vLLM, or custom endpoints.

Option C: Custom MCP Client

poetry install
poetry run python -m mcp_server

Wire the stdio MCP server into your client. Skills are plain markdown in skills/—load them as your client expects.

⚠️ Critical: Fully restart your client after install. The MCP server connects at startup.

Optional Hardened Containers

# Kali Linux container — required for /credential-audit, deep /web-exploit tools
docker build -t pentest-agent/kali-mcp ./tools/kali/      # ~10 min, ~3 GB

# Metasploit container — required for /metasploit
docker build -t pentest-agent/metasploit ./tools/metasploit/   # ~5 min

Lightweight tools auto-pull on first use. No manual management needed.

REAL Code Examples: agent-smith in Action

The following examples are extracted directly from the agent-smith repository, demonstrating actual usage patterns and architecture.

Example 1: Full Autonomous Pentest Command

This is the entry point that starts everything—the slash command that triggers a complete engagement:

/pentester scan https://staging.example.com depth=thorough

What happens under the hood:

You (/pentester scan target.com)
  └── Your LLM (Claude / GPT / Gemini / local …)
        └── MCP server (python -m mcp_server)
              ├── Lightweight scanners — docker run --rm (nmap, nuclei, httpx, …)
              ├── Kali container       — persistent kali-mcp (nikto, sqlmap, ffuf, …)
              ├── Metasploit container — exploit validation
              └── FastAPI dashboard    — live findings at localhost:7777

The LLM interprets each tool's output and decides the next action. No hardcoded flowchart—adaptive decision-making based on live results. The depth=thorough parameter expands the cost/time budget for deeper exploration.

Example 2: White-Box Code Review with ASVS Coverage

For teams doing shift-left security, this command replaces manual code review with AI-driven analysis:

/codebase path=./src

The skill teaches the LLM to:

1. Map all application routes and entry points
2. Identify data sinks (database queries, file operations, subprocess calls)
3. Evaluate against ASVS 5.0's 16 chapters and 427 requirements
4. Enrich downstream skills with precise vulnerability locations

Critical insight: The /codebase output feeds /pentester, /web-exploit, /api-security, and /ai-redteam. White-box context makes black-box testing exponentially more effective.

Example 3: CVE Analysis with Automated PoC Generation

When your dependency scanner fires, this separates real risk from false positives:

/analyze-cve lodash 4.17.20 CVE-2021-23337

The agent:

1. Locates the vulnerable function in your dependency tree
2. Traces data flow from user input to the vulnerable sink
3. Determines if your specific usage pattern is exploitable
4. Generates a Burp-ready .http file in pocs/ if confirmed

This is the difference between "we have a CVE" and "we have a verified exploit path."

Example 4: AI Red-Teaming for LLM Applications

Modern applications with AI components need adversarial testing that evolves as fast as the threats:

/ai-redteam https://your-app.com/api/chat provider=openai depth=thorough

Coverage includes:

• OWASP LLM Top 10 (2025)
• OWASP AI Testing Guide (AITG v1, Nov 2025)
• OWASP MCP Top 10 runtime attacks

Payload generation uses multiple frameworks:

• FuzzyAI: Fuzzing for LLM inputs
• Garak: LLM vulnerability scanner
• PyRIT: Microsoft's Python Risk Identification Toolkit
• promptfoo: Prompt injection and jailbreak testing

Example 5: Automated Remediation with Build Verification

The most underrated feature—fixing what you find:

/remediate

For every confirmed finding, the agent:

1. Generates a code or configuration patch
2. Applies the patch in a test environment
3. Verifies the build still passes
4. Presents the fix for human review

This closes the loop from find → exploit → fix → verify without human intervention.

Advanced Usage & Best Practices

Optimize Your LLM Selection

Bigger models find more interesting attack paths—but cost more. Our testing suggests:

Chain Skills Strategically

Don't just run /pentester. For maximum coverage:

1. Start with /codebase for white-box context
2. Run /threat-modeling to identify high-risk components
3. Execute /pentester with enriched targeting
4. Follow with /ai-redteam if LLM components detected
5. Remediate with /remediate and export with /gh-export

Monitor Cost Limits in Real-Time

The dashboard's coverage tab shows endpoint × technique tracking. If coverage stalls, the agent may have hit a cost limit. Adjust in core/session.py:

# Default limits (customize per engagement)
MAX_COST_USD = 50.0        # Per-scan budget ceiling
MAX_DURATION_MINUTES = 120 # Hard timeout
MAX_TOOL_CALLS = 500       # Prevent infinite loops

Extend with Custom Skills

Skills live in a separate repo (github.com/0x0pointer/skills) as a git submodule. To add your own:

cd skills/
# Create new markdown skill following existing patterns
git add . && git commit -m "Add custom /api-gateway skill"
cd ..
git add skills && git commit -m "Update skills submodule"
./installers/install.sh  # Re-deploy to client

Comparison with Alternatives

The verdict: Traditional tools are detection engines. agent-smith is a reasoning agent. Use scanners for compliance baselines. Use agent-smith when you need adversarial thinking at scale.

FAQ: What Developers Ask About agent-smith

Is agent-smith legal to use?

Only for authorized testing. The repository prominently warns: "Use against systems you own or have explicit written permission to test. Unauthorized access is illegal." The tool includes audit logging (logs/pentest.log) for accountability.

How much does it cost to run?

The software is open source under AGPL-3.0. Costs are LLM token usage only. A thorough /pentester scan with Claude Opus might run $20-50. With local models via Ollama, marginal cost approaches zero.

Can it replace human pentesters?

Not entirely—yet. It replaces repetitive methodology execution and report writing, freeing humans for complex business logic attacks and client relationships. The most effective teams use agent-smith for coverage at scale and humans for creative validation.

What if the LLM generates dangerous commands?

All execution is sandboxed in Docker with resource limits. The Kali and Metasploit containers are isolated. Server-side enforcement prevents runaway execution. However, never run against production without explicit authorization.

How do I contribute new skills?

Skills are markdown files in the skills submodule. Follow existing patterns for vulnerability class description, verification logic, and chaining rules. Submit via PR.

Does it work with my existing CI/CD?

Yes. The MCP server architecture means any orchestration tool that can invoke CLI commands can trigger scans. Output artifacts (findings.json, exit codes) integrate with GitHub Actions, GitLab CI, Jenkins, or custom platforms.

What about false positives?

The LLM's verification logic reduces noise compared to signature-based tools. Each finding includes confidence scoring and reproducibility steps. The /remediate skill's build verification provides additional validation.

Conclusion: The Future of Offensive Security Is Autonomous

agent-smith isn't just another scanner with AI lipstick. It's a fundamental reimagining of how offensive security operates—teaching methodology to machines that can reason, adapt, and deliver complete engagements without human micromanagement.

The "skills as pattern teachings" approach solves the core limitation of security automation: rigidity. Traditional tools fail when targets deviate from expected patterns. agent-smith thrives on novelty, generating context-specific attacks that evolve with the threat landscape.

For red teams, it scales coverage. For appsec engineers, it automates the tedious. For CISOs, it provides defensible, repeatable, and cost-predictable security validation.

The machines are ready. Are you?

👉 Star agent-smith on GitHub and deploy your first autonomous engagement today. The repository includes complete documentation, active development, and a growing skill ecosystem. Don't let your next breach be found by someone else using the same tool you're about to discover.

Built for offensive-security professionals. Use it to make the internet safer.