Addresses review feedback on the Windows detection PR.
1. Command injection (the important one). spawnAgent ran .cmd/.bat shims via a
hand-rolled `spawn('cmd.exe', ['/d','/s','/c', shim, ...args])`. cmd.exe
re-parses its tail with its own rules and ignores `\"`-escaping, so an
adversarial prompt (claude `-p <prompt>`, codex `exec <prompt>`) containing
`"` + `& | > %VAR%` broke out and executed — the BatBadBut / CVE-2024-1874
class. Verified with a `x" & echo PWNED` payload: it ran.
Fix: use Node's own safe `.cmd` handling — `spawn(shim, args, {shell:true})`.
On Node >=18.20.2 / >=20.12.2 / >=21 (CVE-2024-27980) Node escapes each arg
for cmd, so the whole prompt lands as ONE literal argument and cannot break
out (verified on Node 24: the same payload arrived intact in %1). .exe/POSIX
still spawn directly with shell:false.
2. POSIX detection regression. Detection gated on resolveBin() returning a
value; on a bash-less/busybox image where both `command -v` and `which` are
absent, an installed agent flipped to "not installed". Fix: on POSIX fall
back to the bare name (`resolveBin(bin) || (isWindows ? undefined : bin)`) —
the OS resolves PATH at exec time and the ENOENT branch still catches
genuinely-absent binaries. Windows keeps resolveBin authoritative.
Typecheck clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
detectOne() ran the `--version` probe via execFile INSIDE a `new Promise`
executor but only handled the async error path. execFile can throw
SYNCHRONOUSLY (e.g. `spawn EPERM` when child-process creation is restricted),
and a synchronous throw in a Promise executor REJECTS the promise. Because
detectLocalAgents() used Promise.all, one agent's spawn failure rejected the
whole batch → GET /api/agents/local/detect returned 500 → the Settings "Local
Agents" card silently rendered "0/0 installed" even though all CLIs were on PATH.
- detectOne: wrap execFile in try/catch. The binary was already found on PATH
(resolveBin returned a path) so it IS installed; on a sync throw we resolve
installed:true with version '?' and filesystem-based auth, never rejecting.
- detectLocalAgents: Promise.allSettled instead of Promise.all, so one agent's
failure degrades to "not installed" for that agent only, never the batch.
Verified: /detect returns 200 with claude/codex/hermes installed+ready.
Typecheck clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Detection reported installed npm-global agents (Claude Code, Codex) as
'binary not found on PATH' on Windows, and an authed Hermes desktop as
'NOT AUTHED'. Two root causes:
1. resolvePath() used POSIX-only 'command -v'/'which', and
execFile/spawn('claude', ...) cannot resolve npm's .cmd shims via
PATHEXT -> ENOENT -> installed:false. Add resolveBin() (where.exe on
Windows, preferring .exe > .cmd > .bat) and route detection + both
spawn call sites through the resolved path.
2. Hermes desktop on Windows stores its login under %LOCALAPPDATA%\hermes not ~/.hermes/. Add localAppData() and check both (presence-only).
Windows .cmd shims are launched via a new spawnAgent() helper that invokes
cmd.exe with an argv ARRAY (shell:false) instead of shell:true, so the
untrusted prompt is never re-parsed by cmd.exe -> no command-injection surface.
No new deps; typecheck clean.
Fixes all issues raised in PR #47 review:
1. Add .dockerignore to prevent:
- Real .env secrets baked into image layers
- Host node_modules (glibc) overwriting alpine (musl) → ABI mismatch
- .git, reports/, evidence/ bloating the image
2. Fix compose host-binding:
- Remove T3MP3ST_HOST=127.0.0.1 override (was breaking port forwarding)
- Server now binds to 0.0.0.0 in-container (DOCKER=true from Dockerfile)
- Host mapping stays 127.0.0.1:3333:3333 for security
3. Add critical security warning in DOCKER.md:
- Explain exposure risk of 0.0.0.0 binding + disabled Host-header guard
- Document safe vs unsafe deployment patterns
- Warn against network_mode: host and -p 3333:3333 without 127.0.0.1
4. Update base image:
- node:18-alpine → node:22-alpine (Node 18 EOL)
- Pin by digest @sha256:16e22a550... (prevent floating tag drift)
5. Fix healthcheck in compose:
- Was always returning success (process.exit(0))
- Now actually tests /api/health endpoint
- Add start_period: 40s to align with Dockerfile
6. Fix port typos in DOCKER.md:
- 3000 → 3333 in troubleshooting examples
- Update all example snippets to node:22-alpine with digest
The quickstart tells users to run `npx tempest config` to switch to the local
provider, but there is no `config` subcommand — the CLI exposes `interactive`
(default), `setup`, `status`, `test`, and `models`. The "Change default
provider" flow lives in the default `interactive` command, so the correct
invocation is `npx tempest`.
Add a first-class `gemini` provider so operators can drive missions with a
direct GEMINI_API_KEY, instead of only reaching Gemini through OpenRouter.
Gemini is wired exactly like xAI: dispatched through OpenAIAdapter against
Google's OpenAI-compatible surface. The base URL is
https://generativelanguage.googleapis.com/v1beta/openai — the `/openai`
segment is required because OpenAIAdapter posts to `${baseUrl}/chat/completions`;
without it requests hit `/v1beta/chat/completions`, which Google does not serve.
Wires the provider through the type union, adapter factory, config
(interface, defaults, model catalog, key env map, getLLMConfig, fallback
chain, redaction, .env template), and adds regression tests pinning the
`/openai` base-URL segment.
The Kali+ adapter factory only carries bespoke ARG_TEMPLATES for the network
and source/supply-chain scanners. The reverse-engineering, mobile and
smart-contract analysers catalogued by the cloud/mobile/binary-RE loadouts fall
through to DEFAULT_TEMPLATE and spawn a bare `<binary> <file>` — a broken
invocation for a subcommand/flag-driven tool, and for radare2 one that drops
into an INTERACTIVE prompt and hangs the agent loop until the timeout burns out.
Add conservative, read-only ARG_TEMPLATES (mirroring each catalog commandHint)
for objdump, readelf, checksec, r2, exiftool, myth (mythril), apkleaks, slither
and mobsfscan. File-oriented tools resolve their artifact through a new
`artifactPath` helper that refuses an absent path or an inherited http(s) URL by
throwing, which the handler turns into a clean {success:false} result (never a
bare `objdump ''`). No intrusive or code-executing flag is added — risk stays
where the catalog put it, and the existing option-looking-target / scope gates
still apply.
Add an invocation-honesty guard test (sibling of the tool-count honesty test):
every mintable adapter must be either templated or explicitly filed under one
fall-through reason — POSITIONAL_TARGET_OK, OPERATOR_DRIVEN, or KNOWN_DEBT — so a
newly catalogued subcommand tool can never silently ship a broken positional
invocation, and the remaining debt (feroxbuster, osv-scanner, hashcat, apktool,
yara) is tracked in the open rather than hidden.
All offline gate checks pass (lint typecheck test doctor verify-claims
test:no-fitting test:no-self-fitting test:gate prompt:audit smoke
test:arsenal-tools arsenal:smoke).
- Bind to 0.0.0.0 inside containers (DOCKER=true env marker)
- Healthcheck was failing because localhost resolved to IPv6 ::1
but server was bound to IPv4 127.0.0.1
- External exposure still protected via Coolify port mapping to 127.0.0.1
- Add Dockerfile with Node.js 18 alpine build
- Add docker-compose.yml with environment configuration
- Add docs/DOCKER.md deployment guide
- Update README.md with Docker quick start section
- Server binds to 0.0.0.0:3333 for external access
- UI available at http://localhost:3333/ui/
- Volumes mounted for reports and evidence persistence
Two issues blocked OpenAI-compatible servers that aren't Ollama-on-/v1:
1. LocalAdapter.isOpenAIWire matched only a literal /v1 path, so versioned
endpoints like Zhipu/z.ai's /api/paas/v4 silently fell through to Ollama's
native /api/chat wire format and failed. Match any /vN instead of literally /v1.
2. The `local` provider was treated as always-keyless, so servers that require
a bearer (Zhipu/z.ai, Together, ...) could never authenticate. Read an
optional key from TEMPEST_LOCAL_API_KEY (ZAI_API_KEY / ZHIPUAI_API_KEY as
aliases), wire it through getApiKey/hasApiKey so `tempest test` and the
mission engine accept a configured local provider, and document the options
in the env template.
The key is read from env in-memory and never persisted into the store, matching
the existing env-key hardening invariant (api-key-env-static).
Adds src/__tests__/local-openai-wire-static.test.ts pinning both fixes.
Verified end-to-end against z.ai's GLM models (glm-4.5-air / 4.6 / 5.2).
The arsenal advertises "83 tools" and verify-claims defends it by counting
`id:`/`name:` source lines — a source-line count that would keep passing even
if an entry became uncallable. Add a guard that ties the number to the real
registered arrays (TOOL_ADAPTERS + BUILTIN_TOOLS + EXTERNAL_TOOLS === 83) and
pins the invariant that makes the count honest:
an adapter is off the generic callable surface (adapterToCustomTool → null)
IFF it is explicitly gated by execution mode (catalog_only / import_only).
So a tool can be counted-but-not-generically-callable only on purpose —
metasploit / hydra (catalog_only, reachable solely via the approval-gated
post-ex drivers) and bloodhound (import_only) — never through a silent DEFAULT
or mint slip. Also asserts isMintable agrees with actual callability, the gated
set is exactly {metasploit, hydra, bloodhound}, and every minted adapter has a
real handler and a unique name. Sibling of no-phantom-tools / stub-honesty.
The opt-in code/supply-chain scanners (semgrep, gitleaks, trufflehog, trivy,
syft, grype, checkov) are subcommand-driven and non-networked, but had no
bespoke arg template — so adapterToCustomTool fell through to DEFAULT_TEMPLATE
and spawned `<binary> <target>`. For these that is a broken invocation:
`semgrep <host>` never runs a scan, and with no target it is `semgrep ''`.
semgrep in particular is advertised on the arsenal's headline tool list.
Add per-binary templates that mirror each tool's catalog commandHint and
operate on a filesystem PATH (default: the working dir "."):
semgrep → scan --config auto --json <path>
gitleaks → detect --source <path> --report-format json --redact --no-banner
trufflehog → filesystem <path> --json --no-update
trivy → fs --format json <path>
syft → dir:<path> -o cyclonedx-json
grype → dir:<path> -o json
checkov → -d <path> -o json
Subcommand + output flags are hardcoded (never taken from an LLM string),
matching the conservative pattern of the existing templates. The path is the
only tunable and is sanitised by scanPath(): a leading '-' (flag injection) or
an inherited http(s) mission target is rejected in favour of ".". Option-looking
targets are still refused up front by the factory's existing guard (no spawn).
Adds table-driven tests asserting the exact argv for each scanner, the "."
default, http(s)-target rejection, and the option-looking-path refusal.
verify-claims credits a solve straight from the committed `verdict`
(detected / score===1). Nothing asserted that a scored solve actually
agrees with its own committed flag oracle, so a hand-edited `detected:true`
(or a merge slip / fabricated placeholder flag) would pass verification
silently. The case-insensitive oracle comparator `ciMatch` was defined but
never called — this is exactly the check it was meant for.
Add an INTEGRITY GATE that re-derives, for every scored solve across the
pinned golden + white-box XBEN sweeps and the Cybench per-run artifacts,
that it is oracle-grounded: EITHER the self-reported flag matches the
committed `expected` (ciMatch), OR the harness auto-detected `expected`
verbatim in tool output (auto_detected, reported may be "UNKNOWN").
Anything scored that is neither is a phantom pass and fails the build.
Also rejects a fabricated placeholder credited via the self-report path,
and reports how much of the Cybench headline is backed by an oracle-bearing
per-run artifact vs. counted only in the hand-editable aggregate.
Adds src/__tests__/oracle-consistency.test.ts in the same spirit as
no-phantom-tools / stub-honesty: audits the committed artifacts AND proves
the gate has teeth against synthetic tampered/fabricated verdicts.
Current receipts pass clean: 0 phantom / 606 scored solves, 0 fabricated.
エルジー🐧