t3mp3st contributors
537d5345dc
Three new category loadouts, each = arsenal + an offline-scorable benchmark, opt-in and
honestly scoped (roster marks all 🚧; pacu/frida stay catalog_only, unit-tested non-mintable):
- cloud: aws/az/gcloud + scoutsuite/cloudfox/pmapper; cloud:bench IaC-misconfig detection
- mobile: apkleaks/mobsfscan/drozer/objection/class-dump; mobile:bench static ruleset
- binary/RE: ghidra/objdump/checksec/strings/readelf/gdb; binary:bench decompiled-sink detector
Mobile/binary benches are DETECTOR REGRESSION CHECKS (rules+fixtures co-authored — a pass is a
floor, not a detection rate). No new headline numbers; verify-claims 24/24.
tsc clean; 337 tests; lint 0 errors.
Benchmark artifacts
Every headline number in the top-level README re-derives from the committed result
JSONs in this directory. Run npm run verify-claims from the repo root — it recomputes
each claim from these files under a provenance-strict scorer and fails if any number
drifts from its artifact.
Each result file records the structured verdict for a challenge run: the score / detection, canary status, the target id, and (for CVE hunts) the finding metadata. Those are the fields the scorer reads.
A note on transcripts
The raw per-run agent transcripts are intentionally omitted from this public release.
During a run, an operator's recon commands (find, ls, env) get captured verbatim
into the transcript, which means they can photograph the author's local machine. Rather
than hand-scrub an unbounded capture surface, the release ships the structured verdicts
only. This is lossless for reproducibility: verify-claims never reads the transcript
field — it re-derives everything from the verdicts — so every number here remains
independently checkable. To regenerate full transcripts, re-run the harness against the
same challenge suites from scratch.