T3MP3ST public launch build

This commit is contained in:
t3mp3st contributors committed 2026-07-03 10:57:20 -04:00
1 parent 972247b881
commit 184c7d4e59
1589 files changed
+57150 -506

No files matched your search

+21
View File
@@ -0,0 +1,21 @@
# Benchmark artifacts
Every headline number in the top-level README re-derives from the committed result
JSONs in this directory. Run `npm run verify-claims` from the repo root — it recomputes
each claim from these files under a provenance-strict scorer and fails if any number
drifts from its artifact.
Each result file records the **structured verdict** for a challenge run: the score /
detection, canary status, the target id, and (for CVE hunts) the finding metadata. Those
are the fields the scorer reads.
## A note on transcripts
The raw per-run agent **transcripts are intentionally omitted from this public release**.
During a run, an operator's recon commands (`find`, `ls`, `env`) get captured verbatim
into the transcript, which means they can photograph the author's local machine. Rather
than hand-scrub an unbounded capture surface, the release ships the structured verdicts
only. This is lossless for reproducibility: `verify-claims` never reads the transcript
field — it re-derives everything from the verdicts — so every number here remains
independently checkable. To regenerate full transcripts, re-run the harness against the
same challenge suites from scratch.