t3mp3st contributorsandClaude Opus 4.8
06ffe3ef66
Adds three coordinated capabilities and tightens two defaults on purpose — this is safety hardening, NOT a backward-compatible drop-in (see "Tightened defaults"). Fifteen independent adversarial review passes; findings fixed and test-pinned. - Pack-hunt safety: a strict-majority adjudicator with a comment/string/regex/ raw-string/heredoc/POD-aware cite-check (language-gated across C/C++/Py/Rust/Go/ Java/JS/TS/shell/SQL/PHP/Ruby/Perl) so a hallucinated killing-guard can't refute a real finding — the cited comparison's operator must match source, and a string/regex/heredoc/generic-arglist/template-interpolation is blanked before matching. Plus a rate-limit/degraded-health tracker, a global LLM semaphore, and a PackBoard with an atomic claim CAS + snapshot-on-emit/clone-on-ingest. - Opt-in Kali+ arsenal (T3MP3ST_FULL_ARSENAL, OFF by default): 35 default tools (31 built-in + 4 external-CLI wrappers) -> 83 with the catalog armed (verify-claims CLAIM 4). Most adapters need the underlying binary installed. metasploit / hydra / bloodhound are never minted through the generic factory. - Capability-approval + spicy-action gate for the NEW dangerous arsenal only (pre-existing built-ins keep running freely): approve-once, fail-safe deny when unattended, redacted audit trail over SSE. T3MP3ST_GATE_BUILTINS=1 extends it to the spicy built-in probes. Audit redaction is a pure, unit-tested module. Tightened defaults (intentional, ON by default — NOT backward-compatible): - Egress containment: once a mission target is set, built-in networked tools refuse off-scope public hosts (not the target/subdomains, not loopback/private) with SCOPE DENIED — keeps the agent on authorized scope. - Role-specialized operator toolkits: each archetype gets a curated default tool menu (scanner -> scan tools, exploiter -> sqli/xss/etc). Tools stay registered team-wide; the per-role menu is the operator's working set. UX (from live-tester feedback): a one-click "Approve & run" modal when a standard Hunt blocks on an external-target receipt (was a dead-end); bounded-parallel benchmark file ingestion; the deterministic mission-gate chip co-located with the subjective SITREP pace narration (no more "behind schedule" next to a clean gate); and click-through finding detail + evidence in the UI (was export-only). Docs: README + INTEGRITY_LEDGER made re-derivable — honest 35/83 tool count, the egress-containment default documented, and the Cybench number reconciled to the single verify-claims-enforced 23/40 = 58% (dropped the stale/uncited 21 and 26). README voice pass: killed the underselling `Successful Exploits: 0` disqualifier (read as "never popped anything" — false, given CVE-Zero + the live disclosure pipeline); swagger now rests on receipts, not vibes. Also corrected the operator status — the downstream operators run the SAME real tool-backed ReAct loop as recon (not "orchestration stubs"); the honest limit is that the coordinated swarm is unproven (single-agent is the benchmarked path), not that they're scaffolding. Gates: 292 tests, tsc + lint clean, verify-claims 20/20, arsenal:smoke 125/0, disclosure-CLI self-test 11/11. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 line
15 KiB
JSON
1 line
15 KiB
JSON
{"drill": "t3mp3st-full-arsenal-smoke", "baseUrl": "http://127.0.0.1:3333", "startedAt": "2026-06-05T14:32:44.687Z", "finishedAt": "2026-06-05T14:33:05.070Z", "passed": false, "summary": {"checks": 134, "passed": 129, "failed": 5, "byArea": {"core": 4, "arsenal": 7, "knowledge": 13, "contracts": 46, "learning": 10, "pliny": 9, "tools": 11, "missionControl": 21}, "warnings": 0}, "checks": [{"name": "Health endpoint is operational", "passed": true, "detail": "standalone / 45 tools"}, {"name": "Health exposes arsenal summary", "passed": true, "detail": "48 adapters / 45 command-ready"}, {"name": "API health mirror is operational", "passed": true, "detail": "t3mp3st"}, {"name": "Capability preflight runs", "passed": true, "detail": "100/100"}, {"name": "SSE event stream opens", "passed": true, "detail": "text/event-stream"}, {"name": "Arsenal catalog exposes adapter spine", "passed": true, "detail": "48 adapters"}, {"name": "Arsenal safe commands exclude catalog-only heavy tools", "passed": true, "detail": "45 safe commands"}, {"name": "Web/API arsenal includes modern recon chain", "passed": true, "detail": "curl, dig, host, whois, nmap, subfinder, httpx, naabu, katana, nuclei, ffuf, gobuster, feroxbuster, nikto, dalfox, sqlmap, radamsa, metasploit, hydra"}, {"name": "Supply-chain arsenal includes evidence scanners", "passed": true, "detail": "file, nuclei, semgrep, gitleaks, trufflehog, trivy, syft, grype, osv-scanner, slither, mythril, echidna, foundry-forge, solhint, openssl, john, hashcat, radamsa, afl-fuzz, radare2, apktool, jadx, exiftool, yara"}, {"name": "Arsenal status reports installed and missing adapters", "passed": true, "detail": "6 installed / 35%"}, {"name": "Arsenal planner turns adapters into gated evidence steps", "passed": true, "detail": "19 planned steps"}, {"name": "Workflow presets exposed", "passed": true, "detail": "5 presets"}, {"name": "Resource packs exposed", "passed": true, "detail": "14 packs"}, {"name": "Resource pack detail works", "passed": true, "detail": "OWASP Web Security Testing Guide"}, {"name": "Agent prompt packs exposed", "passed": true, "detail": "prompt-ai-boundary-cartographer, prompt-web-api-operator, prompt-repo-supply-chain-sentinel, prompt-vuln-prioritization-analyst, prompt-agent-warfare-sentinel"}, {"name": "Agent prompt pack detail works", "passed": true, "detail": "AI Boundary Cartographer"}, {"name": "Operator runbooks exposed", "passed": true, "detail": "5 runbooks"}, {"name": "Operator runbook detail works", "passed": true, "detail": "AI Agent Boundary Runbook"}, {"name": "Forefront radar exposed", "passed": true, "detail": "frontier-agent-command-injection, frontier-browser-tool-privilege, frontier-ci-release-trust, frontier-identity-handoff, frontier-data-egress-control, frontier-wallet-signing-boundary"}, {"name": "Forefront lane detail works", "passed": true, "detail": "Agent Command-Injection Boundary"}, {"name": "Resource search returns CVE sources", "passed": true, "detail": "cisa-kev, nvd-cve-api, first-epss, cve-list-v5"}, {"name": "Agent context bundles resources, prompts, runbook", "passed": true, "detail": "3 refs / 1 prompts"}, {"name": "Operator doctrine exposed", "passed": true, "detail": "hypothesis -> evidence -> finding -> fix -> retest"}, {"name": "Route scorecard exposed", "passed": true, "detail": "no production writes without explicit approval"}, {"name": "Mission draft can be created", "passed": true, "detail": "draft_0b1bc6ed"}, {"name": "Mission draft list includes draft", "passed": true, "detail": "4 drafts"}, {"name": "Mission draft detail works", "passed": true, "detail": "Arsenal Smoke: Owned Local Web/API"}, {"name": "Mission draft patch works", "passed": true, "detail": "queued / high"}, {"name": "Route preview emits operation contract", "passed": true, "detail": "t3mp3st.operation/v1"}, {"name": "Mission receipt can be requested and approved", "passed": true, "detail": "approval_00c830a5"}, {"name": "Approval rejection path works", "passed": true, "detail": "approval_4e1d6b30"}, {"name": "Approval ledger lists receipts", "passed": true, "detail": "21 approvals"}, {"name": "Evidence ledger accepts entry", "passed": true, "detail": "evidence_361b3683"}, {"name": "Evidence ledger labels weak provenance", "passed": true, "detail": "weak"}, {"name": "Evidence ledger labels tool provenance", "passed": true, "detail": "tool"}, {"name": "Hypothesis ledger accepts claim", "passed": true, "detail": "hypothesis_ae5831db"}, {"name": "Hypothesis ledger can be filtered by mission", "passed": true, "detail": "1 hypotheses"}, {"name": "Evidence graph links hypotheses and proof", "passed": true, "detail": "3 nodes / 2 edges"}, {"name": "Cross-family hypothesis can join same mission", "passed": true, "detail": "hypothesis_37109d2e"}, {"name": "Mission-wide graph preserves specialist lane truth", "passed": true, "detail": "2 hypotheses / 2 lanes"}, {"name": "Evidence graph summarizes strong provenance", "passed": true, "detail": "1 strong"}, {"name": "Mission gate counts cross-family hypotheses", "passed": true, "detail": "2 mission hypotheses / 2 lanes"}, {"name": "Mission gate recognizes strong evidence", "passed": true, "detail": "1 strong"}, {"name": "Hypothesis decomposes into specialist work orders", "passed": true, "detail": "6 work orders"}, {"name": "Work order queue can be filtered by mission", "passed": true, "detail": "6 queued"}, {"name": "Work order completion auto-attaches evidence", "passed": true, "detail": "work_99bc487f"}, {"name": "Evidence graph includes work-order nodes", "passed": true, "detail": "6 work orders"}, {"name": "Watch-loop fixture hypothesis created", "passed": true, "detail": "hypothesis_7dc4f5ce"}, {"name": "Watch loop emits scoped signals", "passed": true, "detail": "6 signals / 4 actions"}, {"name": "Watch loop nudge spawns specialist work orders", "passed": true, "detail": "12 spawned"}, {"name": "Watch loop status recalls latest pulse", "passed": true, "detail": "watch_a4262d34 latest / 2 retained"}, {"name": "Self-heal diagnoses and safely applies watch repair", "passed": true, "detail": "repair / 1 applied"}, {"name": "Hypothesis patch path works", "passed": true, "detail": "supported"}, {"name": "Hypothesis can be promoted to finding", "passed": true, "detail": "finding_f3c15653"}, {"name": "Finding ledger accepts claim", "passed": true, "detail": "finding_4a9c8d94"}, {"name": "Finding patch path works", "passed": true, "detail": "fix_ready"}, {"name": "Retest can be queued", "passed": true, "detail": "retest_8c8d2475"}, {"name": "Mission gate holds unresolved retest", "passed": true, "detail": "hold 80/100"}, {"name": "Retest can be completed", "passed": true, "detail": "passed"}, {"name": "Repro packs summarize replay readiness", "passed": true, "detail": "1/2 ready"}, {"name": "Repro pack includes safe replay contract", "passed": true, "detail": "repro_finding_4a9c8d94"}, {"name": "Pressure paths convert repro into gated offensive chain", "passed": true, "detail": "1/2 armed, max 91"}, {"name": "Pressure path includes simulator and no-go gates", "passed": true, "detail": "pressure_finding_4a9c8d94"}, {"name": "Pressure canary rehearses top path locally", "passed": true, "detail": "Local canary matched expected signal for pressure_finding_4a9c8d94 without leaving simulator posture."}, {"name": "Pressure canary writes replayable evidence and retest receipt", "passed": true, "detail": "evidence_cd3f0ab4 / retest_e748d518"}, {"name": "Pressure duel survives only evidence-backed canary route", "passed": true, "detail": "survived 100/100"}, {"name": "Pressure duel writes skeptic evidence and follow-up task", "passed": true, "detail": "evidence_a411371a / work_fc464530"}, {"name": "Pressure mutation gauntlet forks survived route", "passed": true, "detail": "5 mutations / max 88"}, {"name": "Pressure mutation gauntlet queues specialist work orders", "passed": true, "detail": "evidence_f2f3557d / 3 work orders"}, {"name": "Pressure fang chains compose local weird-machine routes", "passed": true, "detail": "3 chains / max 100"}, {"name": "Pressure fang chains queue staged specialist work orders", "passed": true, "detail": "evidence_1e8ed3df / 2 work orders"}, {"name": "Evidence ledger can be filtered by mission", "passed": true, "detail": "7 evidence"}, {"name": "Evidence ledger includes pressure canary receipt", "passed": true, "detail": "evidence_cd3f0ab4"}, {"name": "Evidence ledger includes pressure duel receipt", "passed": true, "detail": "evidence_a411371a"}, {"name": "Evidence ledger includes pressure mutation receipt", "passed": true, "detail": "evidence_f2f3557d"}, {"name": "Evidence ledger includes pressure fang-chain receipt", "passed": true, "detail": "evidence_1e8ed3df"}, {"name": "Finding ledger can be filtered by mission", "passed": true, "detail": "2 findings"}, {"name": "Retest ledger can be listed", "passed": true, "detail": "7 retests"}, {"name": "Mission bundle exports handoff", "passed": true, "detail": "Mission bundle has 3 evidence/control gaps before hard claims."}, {"name": "Mission bundle carries repro packs", "passed": true, "detail": "1/2 ready"}, {"name": "Mission bundle carries pressure paths", "passed": true, "detail": "1/2 armed"}, {"name": "Mission bundle can be rebuilt by mission id", "passed": true, "detail": "Mission bundle has 3 evidence/control gaps before hard claims."}, {"name": "Mission gate evaluates ledgers and receipts", "passed": true, "detail": "hold 80/100"}, {"name": "Mission draft delete path works", "passed": true, "detail": "draft_60170ce1"}, {"name": "Improvement proposal can be created", "passed": true, "detail": "proposal_8ada5fbf"}, {"name": "Improvement proposals can be listed", "passed": true, "detail": "3 proposals"}, {"name": "Promotion gate approves only evidence-backed proposal", "passed": true, "detail": "eligible_for_promotion"}, {"name": "Learning status exposes explicit proposal policy", "passed": true, "detail": "type + canonical content fingerprint"}, {"name": "Learning review proposes inspectable memory", "passed": true, "detail": "4 proposals"}, {"name": "Manual memory proposal can be created", "passed": true, "detail": "memprop_934ab559"}, {"name": "Memory proposal reject path works", "passed": true, "detail": "rejected"}, {"name": "Memory proposal accept path works", "passed": true, "detail": "mem_5f29e5de"}, {"name": "Repeated learning review reinforces, not duplicates", "passed": true, "detail": "8 proposals / reinforced=true"}, {"name": "Memory capsule exposes accepted entries only", "passed": true, "detail": "2 entries"}, {"name": "LEVIATHAN builds kill-chain plan", "passed": true, "detail": "6 phases"}, {"name": "SPHINX validates synthetic findings", "passed": true, "detail": "4 confirmed"}, {"name": "GORGON returns strike plan without execution", "passed": true, "detail": "8 payload candidates"}, {"name": "CERBERUS ranks synthetic escalation vectors", "passed": true, "detail": "9 vectors"}, {"name": "TYPHON encodes benign canary variants", "passed": true, "detail": "6 variants"}, {"name": "GRIFFIN detects synthetic secrets without real values", "passed": true, "detail": "5 synthetic matches"}, {"name": "SIMURGH produces synthetic research map", "passed": true, "detail": "difficulty=hard"}, {"name": "HYDRA composes multi-vector plan", "passed": true, "detail": "5 vectors"}, {"name": "ARACHNE builds viable exploit chain", "passed": true, "detail": "15 chains"}, {"name": "Tool registry exposes safe command allowlist", "passed": true, "detail": "45 tools"}, {"name": "Tool execution rejects missing command", "passed": true, "detail": "Command required"}, {"name": "Tool execution rejects shell metacharacters", "passed": true, "detail": "Shell control characters are not allowed; use direct argv-style commands only."}, {"name": "Tool execution requires receipt before local command", "passed": true, "detail": "approval_1db668e1"}, {"name": "Tool execution rejects forged client authority", "passed": true, "detail": "approval_e05bdaa3"}, {"name": "Wildcard command approvals are rejected", "passed": true, "detail": "Wildcard approval target is not allowed for active tool, mission, or autonomous actions"}, {"name": "Command receipt cannot satisfy mission gate authority", "passed": true, "detail": "0 mission / 1 command"}, {"name": "Approved local command executes", "passed": true, "detail": "package.json: JSON data"}, {"name": "Recon rejects unsupported target characters", "passed": true, "detail": "Target contains unsupported characters"}, {"name": "Private LAN recon requires receipt", "passed": true, "detail": "approval_d90b35c3"}, {"name": "Loopback recon path is callable without external target", "passed": true, "detail": "127.0.0.1"}, {"name": "LLM status endpoint responds", "passed": true, "detail": "connected=true hasApiKey=true"}, {"name": "LLM chat fails closed when unconfigured", "passed": false, "detail": "This operation was aborted"}, {"name": "Mission status endpoint responds", "passed": true, "detail": "active=false"}, {"name": "Mission start requires configured key or supplied key", "passed": false, "detail": "Approval required before active execution"}, {"name": "Operator list endpoint responds", "passed": true, "detail": "0 operators"}, {"name": "Operator spawn fails closed without active mission", "passed": true, "detail": "No active mission \u2014 start a mission first"}, {"name": "Operator task dispatch fails closed without active mission", "passed": true, "detail": "No active mission"}, {"name": "Operator terminate fails closed without active mission", "passed": true, "detail": "No active mission"}, {"name": "Mission findings endpoint responds", "passed": true, "detail": "0 findings"}, {"name": "Mission pause reports no active mission", "passed": true, "detail": "No active mission"}, {"name": "Mission resume reports no active mission", "passed": true, "detail": "No active mission"}, {"name": "Mission stop reports no active mission", "passed": true, "detail": "No active mission"}, {"name": "General plan endpoint responds without active General", "passed": false, "detail": "plan=null"}, {"name": "General sitreps endpoint responds without active General", "passed": true, "detail": "0 sitreps"}, {"name": "General planner requires objective", "passed": true, "detail": "Objective required \u2014 tell the General what you want to accomplish"}, {"name": "General planner requires key before model call", "passed": false, "detail": "This operation was aborted"}, {"name": "General execute fails closed without plan", "passed": true, "detail": "No plan available \u2014 call /api/general/plan first"}, {"name": "General auto requires key before execution", "passed": false, "detail": "Approval required before active execution"}, {"name": "General sitrep fails closed without General", "passed": true, "detail": "No active mission"}, {"name": "General assess fails closed without General", "passed": true, "detail": "No active mission"}], "warnings": []} |