docs: add contribution receipt template (#29)
Co-authored-by: Claude (agent) <238336761+hummbl-dev@users.noreply.github.com>
This commit is contained in:
1 parent
089591b18f
commit
0f3213b310
3 files changed
+104
No files matched your search
@@ -49,6 +49,10 @@ npm test
|
||||
npm run doctor
|
||||
```
|
||||
|
||||
For changes that affect claims, run modes, agent/tool execution, target scope,
|
||||
egress, redaction, reports, or benchmark artifacts, include a contribution
|
||||
receipt using [`docs/CONTRIBUTION_RECEIPTS.md`](docs/CONTRIBUTION_RECEIPTS.md).
|
||||
|
||||
If the API is running:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -170,6 +170,7 @@ Deeper reading: [WALL_FORENSICS](docs/WALL_FORENSICS.md) (per-challenge misses),
|
||||
| [FEATURES.md](FEATURES.md) | feature-by-feature status (`[x]` shipped / `[~]` partial / `[ ]` planned) |
|
||||
| [SCOPE_AND_AUTHORIZATION](docs/SCOPE_AND_AUTHORIZATION.md) | authority model, scope receipts, evidence and retest rules |
|
||||
| [VERIFIED_PROVENANCE](docs/VERIFIED_PROVENANCE.md) | how findings become tool-proven instead of model-asserted |
|
||||
| [CONTRIBUTION_RECEIPTS](docs/CONTRIBUTION_RECEIPTS.md) | PR receipt template for scope, run mode, model/harness labels, redaction, and verification |
|
||||
| [TEAM_PREVIEW](docs/TEAM_PREVIEW.md) | first-run path and review script |
|
||||
| [INSTALL_MATRIX](docs/INSTALL_MATRIX.md) | macOS / Linux readiness table |
|
||||
| [ARSENAL_ACTIVATION_PLAN](docs/ARSENAL_ACTIVATION_PLAN.md) | optional external-tool setup |
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Contribution Receipts
|
||||
|
||||
T3MP3ST already treats benchmark claims as something reviewers should be able to
|
||||
re-derive. Contribution receipts apply the same discipline to pull requests:
|
||||
make scope, run mode, evidence, redaction, and model/harness labels explicit
|
||||
before a change is merged.
|
||||
|
||||
A receipt can live in the PR description, an issue comment, or a committed
|
||||
artifact when the change itself introduces a durable benchmark or claim.
|
||||
|
||||
## When A Receipt Is Needed
|
||||
|
||||
Include a receipt for changes that affect:
|
||||
|
||||
- benchmark numbers, benchmark artifacts, scoring, or claim verification;
|
||||
- local-agent, API-backed, MCP, or Arsenal execution behavior;
|
||||
- target scope, egress containment, approval gates, or evidence gates;
|
||||
- UI or docs labels for preview, wired, installed, gated, synthetic, live,
|
||||
planning-only, tool-backed, or experimental behavior;
|
||||
- redaction, source ingest, evidence storage, reports, or exported artifacts.
|
||||
|
||||
A receipt is optional for narrow typo fixes, formatting-only docs edits, and
|
||||
mechanical dependency-free cleanup that does not alter behavior or claims.
|
||||
|
||||
## Required Fields
|
||||
|
||||
| Field | What To Record |
|
||||
| --- | --- |
|
||||
| Change | Short summary of what changed and why. |
|
||||
| Scope class | `docs_only`, `static_fixture`, `local_lab`, `ctf_range`, or `authorized_live`. |
|
||||
| Target authority | Who owns or authorized the target. Use `not_applicable` for docs-only/static work. |
|
||||
| Network use | `none`, `loopback`, `private_lab`, or `authorized_external`. |
|
||||
| Run mode labels | For example `planning_only`, `static_test`, `mocked`, `local_agent`, `api_backed`, `tool_backed`, `approval_gated`, `swarm`, `live_authorized`. |
|
||||
| Model/harness labels | Model, provider, agent runtime, harness, tool access, target class, and attempt policy. |
|
||||
| Commands run | Exact commands and pass/fail result. Include skipped commands and why. |
|
||||
| Artifacts | Files, logs, screenshots, reports, or benchmark outputs that support the claim. |
|
||||
| Redaction | What was removed or masked, especially secrets, tokens, private keys, credentials, flags, and private target data. |
|
||||
| Claims changed | State `none` when no README/docs/headline claim changed. Otherwise name the claim and how it is re-derived. |
|
||||
| Abstentions/refusals | Count separately from failed attempts when model or agent behavior is measured. |
|
||||
| Residual risk | Known limits, unverified paths, optional missing tools, or reviewer follow-up. |
|
||||
|
||||
## Model And Harness Matrix
|
||||
|
||||
Benchmark and agent-run claims should not blend model capability, harness
|
||||
capability, tool availability, target choice, and run mode. Use a small matrix
|
||||
when a change introduces or updates measured behavior.
|
||||
|
||||
| Field | Example |
|
||||
| --- | --- |
|
||||
| model | `gpt-5.5`, `local-llama`, `not_applicable` |
|
||||
| provider | `OpenRouter`, `Venice`, `Ollama`, `not_applicable` |
|
||||
| model_version_or_date | Provider version, model date, or retrieval date if known. |
|
||||
| harness | `verify-claims`, `cybench-bench`, `xbow`, `local-agent`, `manual-review`. |
|
||||
| agent_runtime | `Codex`, `Claude Code`, `Hermes`, `none`, or other runtime. |
|
||||
| tool_access | `none`, `read_only`, `mocked`, `local_only`, `approval_gated`, `live_authorized`. |
|
||||
| target_class | `static_fixture`, `local_lab`, `ctf_range`, `authorized_external`. |
|
||||
| run_mode | `single_agent`, `local_agent`, `api_backed`, `swarm`, `planning_only`. |
|
||||
| attempts | Number of runs or tasks attempted. |
|
||||
| successes | Number of evidence-backed successes. |
|
||||
| failures | Number of attempts with evidence-backed failure. |
|
||||
| abstentions | Refusals, skipped tasks, or no-action outcomes. |
|
||||
| artifacts | Paths to committed artifacts or private reviewer-only receipts. |
|
||||
|
||||
## PR Receipt Template
|
||||
|
||||
```markdown
|
||||
## Contribution Receipt
|
||||
|
||||
- Change:
|
||||
- Scope class:
|
||||
- Target authority:
|
||||
- Network use:
|
||||
- Run mode labels:
|
||||
- Model/harness labels:
|
||||
- Commands run:
|
||||
- `npm run typecheck` -> pass/fail/skipped
|
||||
- `npm test` -> pass/fail/skipped
|
||||
- `npm run doctor` -> pass/fail/skipped
|
||||
- `npm run verify-claims` -> pass/fail/skipped
|
||||
- Artifacts:
|
||||
- Redaction:
|
||||
- Claims changed:
|
||||
- Abstentions/refusals:
|
||||
- Residual risk:
|
||||
```
|
||||
|
||||
## Review Rules
|
||||
|
||||
- Do not accept a live or external-target result unless the receipt names the
|
||||
authorization path and target scope.
|
||||
- Do not accept raw secrets, tokens, credentials, private keys, recovered
|
||||
passwords, or private target data in a receipt or artifact.
|
||||
- Do not promote a UI/docs label from preview, planned, synthetic, or
|
||||
planning-only to live/tool-backed unless a reviewer can trace the supporting
|
||||
command or artifact.
|
||||
- Do not blend refusals, abstentions, skipped runs, infrastructure failures, and
|
||||
failed attempts into a single failure bucket when reporting model behavior.
|
||||
- Do not update headline claims unless `npm run verify-claims` or the relevant
|
||||
claim-specific verifier re-derives the number from committed artifacts.
|
||||
Reference in new issue
Block a user