chore(aiwg): adopt shared BT6 maintainer workflow (#148)
* docs(policy): allow low-impact contributor courtesy cleanup * chore(aiwg): migrate to shared BT6 maintainer plugin * chore: reconcile delivery validation
This commit is contained in:
1 parent
8f658891f2
commit
afc9dad1b2
89 files changed
+4683
-1512
No files matched your search
@@ -1,16 +1,126 @@
|
||||
# ADR-001 — Retain a Local-First Modular Monolith
|
||||
|
||||
**Status:** Accepted (retrospective)
|
||||
**Date:** 2026-07-20
|
||||
## Metadata
|
||||
|
||||
- **ADR ID:** ADR-001
|
||||
- **Status:** Accepted (retrospective baseline)
|
||||
- **Decision date:** 2026-07-20
|
||||
- **Authors:** Architecture baseline process
|
||||
- **Reviewers:** Maintainer and safety-review roles; named human approval is not recorded
|
||||
- **Supersedes:** None
|
||||
- **Current level:** L3 — Accepted; current implementation predates the record
|
||||
|
||||
## Context
|
||||
|
||||
T3MP3ST ships CLI, library, localhost HTTP/War Room, MCP, orchestration, tools, evidence, and benchmarks in one Node/TypeScript package. Operators value low-friction self-hosting and local control.
|
||||
### Problem
|
||||
|
||||
T3MP3ST exposes a CLI, typed library, localhost HTTP/War Room, MCP, orchestration, tool adapters, evidence, and benchmarks from one Node/TypeScript package. The baseline needs an honest architectural description and an evolution rule that preserves local control without implying service isolation that does not exist.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Self-hosting and low-friction local installation are product values.
|
||||
- CLI, library, HTTP, and MCP reuse the same domain behavior.
|
||||
- Reports, evidence, configuration, and benchmark artifacts are primarily filesystem-backed.
|
||||
- Real tool execution creates safety boundaries that cannot rely on process naming alone.
|
||||
- No application database, distributed queue, multi-tenant identity service, or fleet control plane exists in the current baseline.
|
||||
|
||||
### Stakeholders
|
||||
|
||||
- Operators need local control and visible artifacts.
|
||||
- Integrators need stable typed and protocol surfaces.
|
||||
- Safety reviewers need clear trust boundaries despite a shared runtime.
|
||||
- Maintainers need coherent module ownership without distributed-system overhead.
|
||||
|
||||
## Source Verification and Claim Tracking
|
||||
|
||||
| Claim | Source | Verification |
|
||||
| --- | --- | --- |
|
||||
| Runtime and delivery surfaces ship in one package | `package.json`, CLI/server/MCP/library entry points | Direct code/package inspection |
|
||||
| HTTP defaults to local exposure | Server and Compose configuration | Static hardening tests and configuration inspection |
|
||||
| No application database or queue is part of the baseline | Source/configuration inventory and SAD | Repository inspection; future addition triggers a new ADR |
|
||||
|
||||
## Decision
|
||||
|
||||
Treat the current architecture as a modular monolith with adapter boundaries. Do not describe it as microservices. Keep the default HTTP surface loopback-only and local artifacts operator-controlled.
|
||||
Treat the current system as a **local-first modular monolith with ports/adapters characteristics**. Delivery adapters and provider/tool adapters translate into shared domain modules in one Node.js runtime. Do not describe the architecture as microservices. Keep default HTTP exposure loopback-only and keep local artifacts under operator control.
|
||||
|
||||
Module boundaries are enforced through typed contracts, narrow adapters, tests, and architecture review. A hosted multi-tenant mode, distributed worker topology, or new durable shared store requires a new architecture baseline rather than being treated as a deployment toggle.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Microservices now
|
||||
|
||||
- **Pros:** Process isolation, independently deployable components, per-service scaling.
|
||||
- **Cons:** Adds network identity, distributed state, deployment, observability, and failure modes without evidenced fleet requirements.
|
||||
- **Why not chosen:** The operational cost conflicts with local-first installation and would not automatically solve tool/target safety.
|
||||
|
||||
### Plug-in processes for every domain
|
||||
|
||||
- **Pros:** Stronger fault and dependency isolation for tools/domains.
|
||||
- **Cons:** IPC contracts, packaging complexity, lifecycle management, and difficult local debugging.
|
||||
- **Why not chosen:** Selective external tool processes already exist; making every module a process is not justified by current evidence.
|
||||
|
||||
### Unstructured single-module application
|
||||
|
||||
- **Pros:** Minimal ceremony.
|
||||
- **Cons:** Unsafe coupling, poor test seams, and high regression risk across trust boundaries.
|
||||
- **Why not chosen:** The codebase already has meaningful domain/adaptor modules whose contracts must remain reviewable.
|
||||
|
||||
## Consequences
|
||||
|
||||
Installation and typed reuse remain simple. Module boundaries require review/tests rather than process isolation. Hosted multi-tenancy or distributed workers require a new decision and architecture baseline.
|
||||
### Positive
|
||||
|
||||
- One install and runtime supports all primary local surfaces.
|
||||
- Shared typed modules reduce duplicated domain and safety logic.
|
||||
- Filesystem artifacts remain inspectable and operator-controlled.
|
||||
|
||||
### Negative
|
||||
|
||||
- Process boundaries do not enforce module isolation.
|
||||
- A crash or resource-exhaustion defect can affect multiple domains.
|
||||
- Scaling and hosted tenancy are not inherent properties of the architecture.
|
||||
|
||||
### Operational Impact
|
||||
|
||||
- Maintainers must review dependency direction and surface parity.
|
||||
- Local process startup, shutdown, filesystem permissions, and loopback exposure are architectural concerns.
|
||||
- External tools remain separate processes with explicit argument, timeout, and output contracts.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood / impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Unsafe cross-module coupling | Medium / High | Typed adapters, negative safety tests, ADR review for trust-boundary changes |
|
||||
| Local process resource exhaustion | Medium / Medium | Finite timeouts, bounded ingest, controlled subprocess execution |
|
||||
| Architecture mislabeled as distributed or production-fleet ready | Medium / High | SAD and maturity matrix; ADR-005 claim discipline |
|
||||
|
||||
## Implementation and Verification
|
||||
|
||||
- **Key integration points:** `src/cli.ts`, `src/index.ts`, `src/server.ts`, and `src/mcp-server.ts` compose shared domain code.
|
||||
- **State model:** In-memory mission state plus local configuration, reports, evidence, and benchmark artifacts; no shared transactional store.
|
||||
- **Concurrency model:** Node event-loop and explicitly managed parallel tasks/processes; shared mutable state remains process-local unless persisted as an artifact.
|
||||
- **Testing:** Build/typecheck, entry-surface tests, local API hardening, agent parallelism, adapter tests, smoke and doctor commands.
|
||||
- **Regression guard:** A change introducing a remotely exposed service, shared durable store, or distributed worker cannot close without an ADR/SAD update.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [x] Context, decision, alternatives, consequences, and risks documented.
|
||||
- [x] Current implementation and tests identified.
|
||||
- [x] SAD describes the same architectural style.
|
||||
- [ ] Named maintainer approval is recorded in a future review-incorporation log.
|
||||
- [ ] Measured startup/shutdown and resource baselines are added before any related performance claim.
|
||||
|
||||
## References
|
||||
|
||||
- @.aiwg/requirements/UC-003.md — Delivery-surface requirements.
|
||||
- @.aiwg/requirements/nfr-register.md — Local security, portability, compatibility, and maintainability constraints.
|
||||
- @.aiwg/architecture/software-architecture-doc.md — Current component, state, and deployment views.
|
||||
- @src/cli.ts — CLI adapter.
|
||||
- @src/index.ts — Library composition surface.
|
||||
- @src/server.ts — Local HTTP/War Room adapter.
|
||||
- @src/mcp-server.ts — MCP adapter.
|
||||
- @docker-compose.yml — Local container deployment and volume/port configuration.
|
||||
- @src/__tests__/local-api-hardening-static.test.ts — Local HTTP boundary evidence.
|
||||
|
||||
## Review and Approval
|
||||
|
||||
This record captures an implemented brownfield decision. It is accepted as the documentation baseline, not as evidence of a historical architecture-board vote. A maintainer may amend or supersede it; that review must name the reviewer, date, conditions, and affected SAD sections.
|
||||
@@ -1,16 +1,124 @@
|
||||
# ADR-002 — Keep the Reasoning Backbone Provider-Neutral
|
||||
|
||||
**Status:** Accepted (retrospective)
|
||||
**Date:** 2026-07-20
|
||||
## Metadata
|
||||
|
||||
- **ADR ID:** ADR-002
|
||||
- **Status:** Accepted (retrospective baseline)
|
||||
- **Decision date:** 2026-07-20
|
||||
- **Authors:** Architecture baseline process
|
||||
- **Reviewers:** Maintainer, integration, and safety-review roles; named human approval is not recorded
|
||||
- **Supersedes:** None
|
||||
- **Current level:** L3 — Accepted; multiple adapter paths are implemented
|
||||
|
||||
## Context
|
||||
|
||||
Users may use hosted APIs, local OpenAI-compatible servers, or already-authenticated coding-agent CLIs. No single provider supplies a stable universal capability contract.
|
||||
### Problem
|
||||
|
||||
Operators may use hosted model APIs, local OpenAI-compatible servers, or already-authenticated coding-agent CLIs. These providers differ in authentication, model names, base URLs, timeouts, tool-calling formats, and failure behavior. Binding mission orchestration to one provider would weaken local/keyless operation and make provider output an accidental trust boundary.
|
||||
|
||||
### Constraints
|
||||
|
||||
- No single provider exposes a stable universal capability contract.
|
||||
- Local agent and local inference paths must remain viable without new cloud credentials.
|
||||
- Provider responses are nondeterministic and may include unsafe tool proposals.
|
||||
- Provider-specific behavior must remain observable rather than erased by a lowest-common-denominator abstraction.
|
||||
- Secrets must not cross provider/base-URL boundaries or appear in diagnostics.
|
||||
|
||||
### Stakeholders
|
||||
|
||||
- Operators need provider choice and fallback clarity.
|
||||
- Integrators need a stable common interface with explicit capability differences.
|
||||
- Safety reviewers need deterministic controls below model output.
|
||||
- Maintainers need provider-specific tests without duplicating mission logic.
|
||||
|
||||
## Source Verification and Claim Tracking
|
||||
|
||||
| Claim | Source | Verification |
|
||||
| --- | --- | --- |
|
||||
| Hosted, local OpenAI-compatible, and connected-agent paths exist | Configuration, LLM, and local-agent modules | Direct code and provider test inspection |
|
||||
| Provider/model compatibility is registry-driven | Provider/model registry | Registry tests |
|
||||
| Local-agent tool proposals use shared runtime controls | Local-agent and arsenal integration | Tool-calling and approval/scope tests |
|
||||
|
||||
## Decision
|
||||
|
||||
Resolve provider/model differences behind shared configuration and LLM/agent adapters. Treat provider responses as untrusted proposals; safety and tool contracts remain in T3MP3ST.
|
||||
Resolve provider/model differences behind shared configuration and LLM/agent adapter contracts. Preserve provider capability metadata and explicit routing/fallback behavior. Treat all provider output as untrusted proposals: target authorization, scope, dangerous-action approval, argument validation, evidence, and reporting policy remain in T3MP3ST.
|
||||
|
||||
Provider-specific adapters may expose additional capability only through declared contract extensions and tests. A fallback must be configured, bounded, and recorded; silent provider switching is not acceptable for evidence-bearing work.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Standardize on one hosted provider
|
||||
|
||||
- **Pros:** Smaller adapter/test matrix and simpler support.
|
||||
- **Cons:** Vendor dependency, credential requirement, weaker local operation, and provider outage concentration.
|
||||
- **Why not chosen:** Conflicts with the existing product promise and implemented local-agent/provider diversity.
|
||||
|
||||
### Let each workflow call providers directly
|
||||
|
||||
- **Pros:** Maximum provider-specific freedom.
|
||||
- **Cons:** Duplicated authentication, retry, timeout, secret, and response logic; safety behavior drifts by workflow.
|
||||
- **Why not chosen:** Makes provider behavior inseparable from domain and safety logic.
|
||||
|
||||
### Normalize every provider to the lowest common feature set
|
||||
|
||||
- **Pros:** Superficially uniform orchestration.
|
||||
- **Cons:** Hides meaningful capability and error differences and can misroute tool behavior.
|
||||
- **Why not chosen:** The common contract must normalize transport, not erase capabilities.
|
||||
|
||||
## Consequences
|
||||
|
||||
Users retain keyless/local options and provider choice. Compatibility and fallback testing expand, and lowest-common-denominator assumptions must not erase provider-specific behavior.
|
||||
### Positive
|
||||
|
||||
- Users retain hosted, local, and connected-agent choices.
|
||||
- Mission/orchestration code depends on stable internal contracts.
|
||||
- Safety controls remain provider-independent.
|
||||
|
||||
### Negative
|
||||
|
||||
- Compatibility, fallback, timeout, and secret-routing test matrices grow with every provider.
|
||||
- Provider-specific capabilities require explicit extension and documentation.
|
||||
- Similar-looking model names or base URLs can still create configuration hazards.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood / impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Credential sent to wrong endpoint | Low / Critical | Explicit provider/base-URL routing, environment/static tests, redaction |
|
||||
| Silent fallback changes evidence identity | Medium / High | Record primary/fallback outcomes and model/provider identity |
|
||||
| Adapter reports malformed output as success | Medium / High | Schema/response validation and explicit error feedback |
|
||||
| Lowest-common-denominator abstraction hides capability | Medium / Medium | Capability registry and provider-specific contract tests |
|
||||
|
||||
## Implementation and Verification
|
||||
|
||||
- **Configuration:** `src/config/` resolves provider, model, base URL, secret source, and defaults.
|
||||
- **Hosted/local interface:** `src/llm/index.ts` normalizes reasoning requests and responses.
|
||||
- **Connected agents:** `src/agent/local-agents.ts` discovers and invokes supported authenticated agent CLIs.
|
||||
- **Concurrency/state:** Requests are bounded tasks; no provider response becomes shared durable state until mission/evidence code records it.
|
||||
- **Testing:** Provider-model registry, provider-specific adapters, base-URL routing, local-agent selection/path/tool-calling, timeout/fallback, and redaction.
|
||||
- **Regression guard:** A new provider is incomplete until configuration, capability metadata, error/timeout, secret, and tool-proposal behavior are tested.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [x] Provider-neutral contract and trust boundary documented.
|
||||
- [x] Hosted, local inference, and connected-agent implementations identified.
|
||||
- [x] Alternatives and security risks recorded.
|
||||
- [ ] Named maintainer/integration approval is recorded.
|
||||
- [ ] Evidence receipts consistently capture provider/model/fallback identity across all benchmark paths.
|
||||
|
||||
## References
|
||||
|
||||
- @.aiwg/requirements/UC-002.md — Provider and local-agent use case.
|
||||
- @.aiwg/requirements/nfr-register.md — Secret, portability, compatibility, testability, and timeout constraints.
|
||||
- @.aiwg/architecture/software-architecture-doc.md — Provider adapter and trust-boundary views.
|
||||
- @src/config/index.ts — Runtime configuration.
|
||||
- @src/config/provider-models.ts — Provider/model capability registry.
|
||||
- @src/llm/index.ts — LLM adapter interface.
|
||||
- @src/agent/local-agents.ts — Connected-agent adapter.
|
||||
- @src/__tests__/provider-models.test.ts — Compatibility registry evidence.
|
||||
- @src/__tests__/provider-base-url-routing-static.test.ts — Endpoint-routing evidence.
|
||||
- @src/__tests__/local-agent-tool-calling.test.ts — Tool-proposal integration evidence.
|
||||
- @scripts/test-model-fallback.mjs — Fallback behavior evidence.
|
||||
|
||||
## Review and Approval
|
||||
|
||||
This is a retrospective description of implemented adapter policy. “Accepted” means it governs the current baseline; it does not fabricate historical approval. Future provider additions or contract changes require maintainer/integration review and an updated evidence matrix.
|
||||
@@ -1,16 +1,124 @@
|
||||
# ADR-003 — Enforce Scope and Approval Below Model Reasoning
|
||||
|
||||
**Status:** Accepted (retrospective)
|
||||
**Date:** 2026-07-20
|
||||
## Metadata
|
||||
|
||||
- **ADR ID:** ADR-003
|
||||
- **Status:** Accepted (retrospective baseline)
|
||||
- **Decision date:** 2026-07-20
|
||||
- **Authors:** Architecture baseline process
|
||||
- **Reviewers:** Maintainer and safety-review roles; named human approval is not recorded
|
||||
- **Supersedes:** None
|
||||
- **Current level:** L4 — Implemented; verification is continuous in CI
|
||||
|
||||
## Context
|
||||
|
||||
Models and target content are nondeterministic and may suggest unsafe or off-scope actions. The arsenal performs real operations.
|
||||
### Problem
|
||||
|
||||
Models, imported repositories, and target responses are nondeterministic untrusted inputs. They can suggest off-scope hosts, dangerous operations, credential forwarding, or malformed arguments. The arsenal performs real security operations, so prompt-only rules cannot be the authorization boundary.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Written authorization and target scope are operator responsibilities but must be represented at runtime.
|
||||
- Built-in network-capable adapters have different argument and destination shapes.
|
||||
- Some operations are materially dangerous even when in scope.
|
||||
- Target-specific credentials must not follow redirects or unrelated origins.
|
||||
- Safety failures must fail closed and remain testable without live targets.
|
||||
|
||||
### Stakeholders
|
||||
|
||||
- Target/system owners require activity to remain authorized.
|
||||
- Operators need explicit control over dangerous actions.
|
||||
- Safety reviewers need deterministic, auditable enforcement.
|
||||
- Tool-adapter contributors need a clear integration contract.
|
||||
|
||||
## Source Verification and Claim Tracking
|
||||
|
||||
| Claim | Source | Verification |
|
||||
| --- | --- | --- |
|
||||
| Scope is enforced in target/arsenal paths | Target and arsenal modules | Scope-gate negative tests |
|
||||
| Dangerous tools require approval | Arsenal approval module/catalog | Approval and approval-gate tests |
|
||||
| Target credentials are origin-bound | Target/server request paths | Target-header static tests |
|
||||
| Prompt/model output is not sufficient authority | Architecture and implementation boundary | Tool proposals still traverse deterministic gates |
|
||||
|
||||
## Decision
|
||||
|
||||
Target scope, egress containment, exact-origin credentials, and dangerous-tool approvals are enforced in deterministic target/arsenal/runtime paths, not solely in prompts.
|
||||
Enforce target authorization, egress containment, exact-origin credential routing, dangerous-tool approval, argument construction, and relevant timeouts in deterministic target/arsenal/runtime code below model reasoning. Prompts may explain policy, but they do not grant execution authority.
|
||||
|
||||
Every new network-capable or privileged adapter must declare its destination derivation, risk class, approval behavior, credential behavior, timeout, and negative tests. A bypass in these controls is a release-blocking defect.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Prompt-only safety policy
|
||||
|
||||
- **Pros:** Easy to change; uniform natural-language guidance.
|
||||
- **Cons:** Nondeterministic, vulnerable to prompt injection, and cannot guarantee tool dispatch behavior.
|
||||
- **Why not chosen:** It does not form an enforceable authorization boundary.
|
||||
|
||||
### Rely solely on operator review of every action
|
||||
|
||||
- **Pros:** Human judgment for all behavior.
|
||||
- **Cons:** High fatigue, inconsistent review, and still vulnerable to hidden destination/argument transformations.
|
||||
- **Why not chosen:** Human approval complements but cannot replace deterministic scope and secret controls.
|
||||
|
||||
### Isolate everything in containers and allow arbitrary execution
|
||||
|
||||
- **Pros:** Limits some host impact.
|
||||
- **Cons:** Containers do not establish target authorization, prevent credential leakage, or guarantee egress scope.
|
||||
- **Why not chosen:** Isolation is defense-in-depth, not an authorization model.
|
||||
|
||||
## Consequences
|
||||
|
||||
Prompt compromise does not automatically authorize execution. Every new network/tool adapter must carry and test these controls; bypasses are release-blocking defects.
|
||||
### Positive
|
||||
|
||||
- Prompt compromise does not automatically authorize real execution.
|
||||
- Safety behavior can be tested deterministically and reviewed per adapter.
|
||||
- Operator approval is attached to the actual classified action.
|
||||
|
||||
### Negative
|
||||
|
||||
- Adapter development requires more metadata, validation, and negative tests.
|
||||
- Novel protocols and redirect behavior need careful destination modeling.
|
||||
- False refusals are possible when scope cannot be derived safely.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood / impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Adapter bypasses central gate | Medium / Critical | Inventory every network-capable adapter; mandatory scope/approval tests |
|
||||
| Argument or shell injection | Medium / Critical | Structured argument construction, validation, no generic arbitrary execution |
|
||||
| Credential follows redirect/origin change | Low / Critical | Exact-origin binding and negative tests |
|
||||
| Approval UI obscures actual impact | Medium / High | Present classified tool/action/target; approval never bypasses scope |
|
||||
|
||||
## Implementation and Verification
|
||||
|
||||
- **Authorization/scope:** `src/target/index.ts`, `src/opsec/`, and arsenal validation/gates.
|
||||
- **Approval:** `src/arsenal/approval.ts` plus catalog risk metadata.
|
||||
- **Secret boundary:** Exact-origin target-header behavior and redaction paths.
|
||||
- **Concurrency/state:** Approval applies to the concrete pending action; async/parallel task scheduling must not reuse approval for a materially different target/action.
|
||||
- **Testing:** Arsenal scope/approval/validation, target-header, parameter splitting/types, tool error, proxy bypass, and redaction suites.
|
||||
- **Regression guard:** Security-control failures block release and cannot be waived by model quality or benchmark success.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [x] Deterministic enforcement boundary documented and implemented.
|
||||
- [x] Scope, approval, secret, and argument risks mapped to tests.
|
||||
- [x] UC-001 and NFR-01/02/05 trace to the decision.
|
||||
- [ ] Complete machine-maintained inventory maps every network-capable adapter to its scope and approval test.
|
||||
- [ ] Named safety-review approval is recorded.
|
||||
|
||||
## References
|
||||
|
||||
- @.aiwg/requirements/UC-001.md — Scoped mission and failure behavior.
|
||||
- @.aiwg/requirements/nfr-register.md — NFR-01, NFR-02, and NFR-05.
|
||||
- @.aiwg/intake/risk-screening.md — Scope, secret, and external-tool risks.
|
||||
- @src/target/index.ts — Authorized-target representation.
|
||||
- @src/arsenal/approval.ts — Dangerous-action approval policy.
|
||||
- @src/arsenal/catalog.ts — Tool metadata and classifications.
|
||||
- @src/__tests__/arsenal-scope-gate.test.ts — Off-scope refusal evidence.
|
||||
- @src/__tests__/arsenal-approval-gate.test.ts — Approval enforcement evidence.
|
||||
- @src/__tests__/target-headers-static.test.ts — Exact-origin credential evidence.
|
||||
- @src/__tests__/param-split.test.ts — Argument construction evidence.
|
||||
|
||||
## Review and Approval
|
||||
|
||||
This retrospective ADR documents a current release-blocking safety invariant. Named historical sign-off is not available. Any amendment weakening default containment, authorization, approval, or exact-origin controls requires explicit maintainer and safety-review approval plus replacement tests.
|
||||
@@ -1,16 +1,123 @@
|
||||
# ADR-004 — Derive Public Claims from Versioned Evidence
|
||||
|
||||
**Status:** Accepted (retrospective)
|
||||
**Date:** 2026-07-20
|
||||
## Metadata
|
||||
|
||||
- **ADR ID:** ADR-004
|
||||
- **Status:** Accepted (retrospective baseline)
|
||||
- **Decision date:** 2026-07-20
|
||||
- **Authors:** Architecture baseline process
|
||||
- **Reviewers:** Maintainer and evaluation-review roles; named human approval is not recorded
|
||||
- **Supersedes:** None
|
||||
- **Current level:** L4 — Implemented in repository verification and CI
|
||||
|
||||
## Context
|
||||
|
||||
Offensive-AI performance claims are easy to overfit or report without reproducible evidence.
|
||||
### Problem
|
||||
|
||||
Offensive-AI findings and benchmark scores are easy to overfit, misgrade, or report without enough context to reproduce them. Model/provider outages can be confused with model failures, and successful corpus-specific results can be overstated as general product capability.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Some third-party corpora or raw sensitive transcripts cannot be redistributed.
|
||||
- Model/provider behavior and live infrastructure are nondeterministic.
|
||||
- Public claims must remain auditable after documentation or implementation changes.
|
||||
- Grading, provenance, and anti-fitting must be enforceable in deterministic CI where possible.
|
||||
- Evidence retention must respect coordinated-disclosure and operator-controlled sensitive paths.
|
||||
|
||||
### Stakeholders
|
||||
|
||||
- Users and researchers need credible, scoped performance claims.
|
||||
- Maintainers need release gates that detect stale documentation.
|
||||
- Vulnerability recipients need evidence-backed findings.
|
||||
- Corpus owners and target operators need licensing/privacy boundaries respected.
|
||||
|
||||
## Source Verification and Claim Tracking
|
||||
|
||||
| Claim | Source | Verification |
|
||||
| --- | --- | --- |
|
||||
| Public claims are recomputed by repository tooling | Claim verifier and package scripts | Execute deterministic claim gate |
|
||||
| Fitting controls are tested | Anti-fitting test scripts | Seeded negative tests |
|
||||
| CI includes claim/provenance gates | CI workflow | Workflow inspection and local command parity |
|
||||
| Some raw material may be held locally | Ignore/disclosure policy and benchmark staging rules | Repository policy inspection |
|
||||
|
||||
## Decision
|
||||
|
||||
Headline claims must be recomputed from committed corpus, ground truth, and receipts. Anti-fitting, provenance, and claim-verification gates are architectural product controls.
|
||||
Headline claims and verified findings must derive from versioned corpus/manifest, ground truth, and retained receipts or evidence. Anti-fitting, provenance, finding verification, and claim verification are product architecture controls and release gates, not optional documentation tasks.
|
||||
|
||||
Every published result must state the evaluated scope: corpus, harness/metric, model/provider identity when relevant, and reproduction boundary. If raw inputs cannot be published, documentation must distinguish a re-derivable graded receipt from full raw reproduction. Missing evidence fails the claim; it is not replaced with an estimate.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Publish manually curated benchmark tables
|
||||
|
||||
- **Pros:** Simple presentation and low runtime cost.
|
||||
- **Cons:** Stale values, transcription errors, and no automated derivation chain.
|
||||
- **Why not chosen:** Cannot enforce public-claim integrity across releases.
|
||||
|
||||
### Retain only aggregate scores
|
||||
|
||||
- **Pros:** Small repository and simpler privacy posture.
|
||||
- **Cons:** Prevents per-case audit, regrading, and error classification.
|
||||
- **Why not chosen:** Aggregate-only evidence is insufficient for disputed findings and fitting analysis.
|
||||
|
||||
### Publish every raw transcript and corpus
|
||||
|
||||
- **Pros:** Maximum reproduction detail.
|
||||
- **Cons:** May violate licensing, privacy, security, or coordinated-disclosure constraints.
|
||||
- **Why not chosen:** Evidence detail must be maximized within lawful/safe retention boundaries, with limitations disclosed.
|
||||
|
||||
## Consequences
|
||||
|
||||
Repository size and CI cost increase, but claims are auditable. Raw sensitive transcripts may be withheld, so documentation must distinguish re-derivable graded verdicts from full raw reproduction.
|
||||
### Positive
|
||||
|
||||
- Claim drift and seeded provenance/fitting defects can block release.
|
||||
- Findings and scores are reviewable at the case level where artifacts permit.
|
||||
- Corpus-scoped language reduces misleading generalization.
|
||||
|
||||
### Negative
|
||||
|
||||
- Repository/storage and CI cost increase.
|
||||
- Receipt schemas and graders become compatibility surfaces.
|
||||
- Full independent reproduction may remain impossible for legally or sensitively withheld raw inputs.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood / impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Receipt exists but does not support claim | Medium / High | Deterministic recomputation, verifier tests, exact claim mapping |
|
||||
| Corpus leakage/self-fitting | Medium / Critical | Split policy and no-fitting/no-self-fitting gates |
|
||||
| Infrastructure failure misclassified | Medium / High | Separate run/infrastructure outcome from model verdict |
|
||||
| Sensitive evidence published | Low / Critical | Hold policies, redaction, coordinated-disclosure paths |
|
||||
|
||||
## Implementation and Verification
|
||||
|
||||
- **Claim derivation:** `scripts/verify-claims.mjs` and package scripts.
|
||||
- **Finding lifecycle:** verification/refutation scripts and evidence modules.
|
||||
- **Anti-fitting:** dedicated no-fitting and no-self-fitting tests.
|
||||
- **Gate enforcement:** required CI jobs for claims, provenance, prompt integrity, tests, and smoke.
|
||||
- **State model:** Versioned manifests/ground truth/receipts; raw sensitive inputs may stay in explicitly local ignored staging.
|
||||
- **Regression guard:** Any change to a claim, receipt, grader, ground truth, or benchmark harness must make derivation pass from the committed state.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [x] Evidence-derived claim policy documented.
|
||||
- [x] Deterministic claim and anti-fitting gates identified.
|
||||
- [x] Withheld-raw-input limitation made explicit.
|
||||
- [ ] Every headline claim has a machine-readable mapping to exact supporting receipts and corpus version.
|
||||
- [ ] Named evaluation-review approval is recorded.
|
||||
|
||||
## References
|
||||
|
||||
- @.aiwg/requirements/UC-005.md — Reproduction and failure flows.
|
||||
- @.aiwg/requirements/nfr-register.md — Provenance, testability, and honesty requirements.
|
||||
- @.aiwg/architecture/vision-alignment.md — Corpus-scoped maturity interpretation.
|
||||
- @scripts/verify-claims.mjs — Claim derivation implementation.
|
||||
- @scripts/verify-finding.mjs — Finding verification.
|
||||
- @scripts/refute-finding.mjs — Candidate refutation.
|
||||
- @scripts/test-no-fitting.mjs — Fitting-prevention evidence.
|
||||
- @scripts/test-no-self-fitting.mjs — Self-fitting-prevention evidence.
|
||||
- @.github/workflows/ci.yml — Automated gate orchestration.
|
||||
|
||||
## Review and Approval
|
||||
|
||||
This ADR records an implemented evaluation/release policy. Its accepted status does not imply that withheld source material is fully independently reproducible. Amendments require maintainer and evaluation review, a stated evidence-retention impact, and passing seeded negative tests.
|
||||
@@ -1,16 +1,124 @@
|
||||
# ADR-005 — Separate Current State from Research Vision
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-07-20
|
||||
## Metadata
|
||||
|
||||
- **ADR ID:** ADR-005
|
||||
- **Status:** Accepted
|
||||
- **Decision date:** 2026-07-20
|
||||
- **Authors:** Architecture baseline process
|
||||
- **Reviewers:** Maintainer, architecture, and evaluation-review roles; named human approval is not recorded
|
||||
- **Supersedes:** None
|
||||
- **Current level:** L3 — Accepted and applied to the SDLC baseline
|
||||
|
||||
## Context
|
||||
|
||||
`VISION.md` deliberately explores autonomous, distributed, cognitive, and evolutionary directions beyond the stable product. Treating it as current architecture would misrepresent capability and risk.
|
||||
### Problem
|
||||
|
||||
`VISION.md` intentionally explores autonomous, distributed, cognitive, adversarial-ML, swarm, knowledge, and evolutionary directions beyond the stable product. Treating that direction as current architecture would create fictitious components, unsafe assumptions, and misleading capability claims. Conversely, ignoring the vision would make design evolution incoherent.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Current architecture must be supported by code, tests, retained evidence, or explicit runtime configuration.
|
||||
- A benchmark proves only its declared corpus, harness, model, and metric.
|
||||
- Stable and experimental features share product surfaces and therefore need conspicuous maturity boundaries.
|
||||
- Persistent autonomy, distributed execution, and shared knowledge services materially change safety and operational architecture.
|
||||
|
||||
### Stakeholders
|
||||
|
||||
- Users need honest expectations about reliability and support.
|
||||
- Researchers need freedom to explore future directions without premature product claims.
|
||||
- Maintainers need explicit promotion gates and architecture triggers.
|
||||
- Safety/evaluation reviewers need evidence that matches claim strength.
|
||||
|
||||
## Source Verification and Claim Tracking
|
||||
|
||||
| Claim | Source | Verification |
|
||||
| --- | --- | --- |
|
||||
| Vision contains broader future directions | `VISION.md` | Direct document inspection |
|
||||
| Current implementation realizes directions unevenly | Source, tests, benchmark receipts, SAD | Vision-alignment evidence matrix |
|
||||
| Product docs label maturity | README/features documentation | Documentation consistency review; automation remains planned |
|
||||
|
||||
## Decision
|
||||
|
||||
Maintain a current-state SAD backed by code paths and a separate vision-alignment matrix. Classify each direction as implemented, partial/experimental, research, or future. Promotion requires evidence and documentation updates.
|
||||
Maintain two linked but distinct views:
|
||||
|
||||
1. A current-state SAD derived from implemented code, tests, configuration, and versioned evidence.
|
||||
2. A vision-alignment matrix that classifies each direction as implemented, partial/experimental, research, or future and states the next proof required.
|
||||
|
||||
Alignment is not implementation. Promotion to stable requires deterministic shared-safety tests, an operational path, current documentation, reproducible evidence appropriate to the claim, and updated UC/US/NFR/SAD traceability. Persistent autonomy, distributed workers, hosted multi-tenancy, or shared knowledge infrastructure triggers new threat modeling and ADR/SAD work.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Treat the vision as the target architecture
|
||||
|
||||
- **Pros:** A single aspirational design narrative.
|
||||
- **Cons:** Documents unimplemented systems as real and obscures immediate trust boundaries.
|
||||
- **Why not chosen:** Violates evidence-based architecture and capability honesty.
|
||||
|
||||
### Remove future vision from architecture governance
|
||||
|
||||
- **Pros:** Purely current-state documentation.
|
||||
- **Cons:** Loses strategic intent and makes research promotion ad hoc.
|
||||
- **Why not chosen:** The vision is useful when explicitly separated and evidence-gated.
|
||||
|
||||
### Use a single stable/unstable label
|
||||
|
||||
- **Pros:** Simple classification.
|
||||
- **Cons:** Conflates scaffold, experiment, research result, and supported capability.
|
||||
- **Why not chosen:** The lifecycle needs more precise promotion and claim language.
|
||||
|
||||
## Consequences
|
||||
|
||||
Architecture remains honest and reviewable. Maintainers must update two views when capabilities evolve, and vision alignment alone cannot justify a stable label.
|
||||
### Positive
|
||||
|
||||
- Architecture remains factual while research direction remains visible.
|
||||
- Each maturity promotion has an evidence and governance trigger.
|
||||
- Benchmark success cannot silently upgrade general product maturity.
|
||||
|
||||
### Negative
|
||||
|
||||
- Maintainers must synchronize product docs, SAD, alignment, requirements, and tests.
|
||||
- Classification disagreements require explicit evidence review.
|
||||
- A manual consistency process can drift until automation is implemented.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood / impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Aspirational language leaks into stable claims | High / High | Alignment matrix, claim verifier, documentation audit |
|
||||
| Mature feature remains mislabeled experimental | Medium / Medium | Promotion checklist and periodic review |
|
||||
| Benchmark is generalized beyond evaluated scope | Medium / High | ADR-004 evidence boundary and corpus-scoped wording |
|
||||
| New distributed/autonomous path bypasses architecture review | Medium / Critical | Explicit new-baseline triggers in SAD/NFR-12 |
|
||||
|
||||
## Implementation and Verification
|
||||
|
||||
- **Current-state source:** SAD, ADRs, requirements, code/tests, and CI evidence.
|
||||
- **Directional source:** `VISION.md`.
|
||||
- **Reconciliation artifact:** `vision-alignment.md` with evidence and next-proof columns.
|
||||
- **Testing/governance:** Claim verification, stub-honesty/arsenal-count tests, documentation review, and planned machine-checkable maturity audit.
|
||||
- **Regression guard:** No roadmap/research item is described as stable without completing the promotion evidence and traceability update.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
- [x] Current-state and directional documents are distinct and cross-referenced.
|
||||
- [x] Each vision vector has a maturity classification, evidence, and next proof.
|
||||
- [x] Promotion triggers are documented.
|
||||
- [ ] Machine-checkable cross-document maturity consistency gate is implemented.
|
||||
- [ ] Named maintainer/evaluation approval is recorded.
|
||||
|
||||
## References
|
||||
|
||||
- @VISION.md — Directional research vision.
|
||||
- @README.md — Primary product capability surface.
|
||||
- @FEATURES.md — Feature and maturity inventory.
|
||||
- @.aiwg/architecture/software-architecture-doc.md — Implemented current-state baseline.
|
||||
- @.aiwg/architecture/vision-alignment.md — Evidence-based maturity reconciliation.
|
||||
- @.aiwg/requirements/UC-005.md — Claim reproduction requirements.
|
||||
- @.aiwg/requirements/nfr-register.md — Honesty and maintainability constraints.
|
||||
- @src/__tests__/stub-honesty.test.ts — Stub/maturity evidence.
|
||||
- @src/__tests__/arsenal-count-honesty.test.ts — Public-count honesty evidence.
|
||||
- @scripts/verify-claims.mjs — Claim derivation gate.
|
||||
|
||||
## Review and Approval
|
||||
|
||||
The decision governs the current baseline. Named historical review evidence is not present; no approval is inferred. Promotion or reclassification requires an evidence review recorded in the SAD review log and alignment matrix.
|
||||
@@ -1,116 +1,394 @@
|
||||
---
|
||||
template_id: software-architecture-doc
|
||||
template_version: 3.0.0
|
||||
status: baselined
|
||||
baseline_revision: 186afe6b50e365371774aa2ed7986d73eb0656db
|
||||
baseline_date: 2026-07-20
|
||||
last_reviewed: 2026-07-21
|
||||
owner: architecture-steward
|
||||
---
|
||||
|
||||
# Software Architecture Document — Current-State Baseline
|
||||
|
||||
**System:** T3MP3ST
|
||||
**Baseline:** revision `186afe6` on 2026-07-20
|
||||
**Status:** BASELINED (brownfield description)
|
||||
**Scope:** Implemented repository state; `VISION.md` is a directional source, not proof of implementation.
|
||||
## Reasoning
|
||||
|
||||
## 1. Purpose and Architectural Drivers
|
||||
1. **Core challenge:** Coordinate tool-backed offensive-security work through several user/provider surfaces while keeping authorization, scope, credentials, evidence, and claims deterministic and reviewable.
|
||||
2. **Constraints:** Local-first Node/TypeScript distribution; one runtime/package; external models and tools; filesystem artifacts; no hosted-service control plane; mixed stable/experimental/research maturity.
|
||||
3. **Alternatives:** A distributed service architecture was rejected for the current baseline because it introduces identity, state, deployment, and failure complexity without evidenced fleet requirements. Prompt-only safety was rejected because it cannot authorize execution.
|
||||
4. **Rationale:** A modular monolith with delivery/provider/tool adapters fits the implemented code and local operator model. Deterministic controls remain below model reasoning.
|
||||
5. **Primary risks:** Scope or credential bypass, unsafe adapter coupling, prompt/content injection, evidence confusion, and over-generalized capability claims. ADR-003 through ADR-005 govern their controls.
|
||||
|
||||
T3MP3ST turns a model or coding agent into the reasoning layer of a local offensive-security platform. The architecture must coordinate long-running, tool-backed security work while preserving target authorization, evidence, and honest claims. The strongest drivers are therefore not raw throughput. They are safe execution, local control, portability across reasoning providers, reproducibility, and the ability to evolve specialized security domains without copying the whole platform.
|
||||
## 1. Executive Summary
|
||||
|
||||
The system supports UC-001 through UC-005. UC-001 drives mission and safety boundaries; UC-002 drives provider abstraction; UC-003 drives multiple delivery adapters; UC-004 drives source-ingest extensibility; UC-005 drives the committed benchmark/provenance subsystem. NFR-01 through NFR-12 constrain all changes, particularly scope, authorization, provenance, secret isolation, and maturity labeling.
|
||||
T3MP3ST is a local-first offensive-security platform for authorized testing, research, and education. It connects a model or authenticated coding agent to mission planning, role-specific operators, real reconnaissance/exploitation tools, evidence handling, finding verification, and reporting. Operators use a CLI, library, localhost War Room/HTTP API, or a deliberately narrower MCP tool.
|
||||
|
||||
## 2. System Context
|
||||
The current architecture is a **modular TypeScript monolith with ports/adapters characteristics**. Delivery adapters translate surface input into shared mission/domain operations; reasoning adapters translate provider/local-agent protocols; arsenal adapters translate approved tool intents into internal functions or bounded subprocesses. Configuration, mission state, reports, evidence, and benchmark artifacts are local/in-process/filesystem-backed; no application database or distributed queue is part of this baseline.
|
||||
|
||||
The principal human is an authorized operator. The operator uses a terminal, the browser War Room, or an integrating client. T3MP3ST communicates with one or more hosted model APIs, local inference servers, or authenticated local coding-agent processes. It invokes internal Node implementations and approved external security tools against targets named in rules of engagement. It writes local reports, evidence, configuration, and benchmark results. MCP clients can request the narrower MCP-exposed capability. GitHub hosts source collaboration and CI, while isolated Docker environments support application and challenge execution.
|
||||
The most consequential decisions are local-first modular deployment (ADR-001), provider-neutral/untrusted reasoning (ADR-002), deterministic scope and approval below model output (ADR-003), evidence-derived public claims (ADR-004), and separation of implemented current state from research vision (ADR-005).
|
||||
|
||||
Trust boundaries exist between: browser and localhost server; client and HTTP/MCP adapters; T3MP3ST and each model provider; the orchestration core and tool processes; tools and target networks; runtime and local artifact storage; repository source and third-party benchmark/tool corpora. No diagram or prose may collapse these into a single trusted process merely because deployment is local.
|
||||
The critical residual risks are an adapter bypassing scope/approval, secrets or sensitive evidence escaping, imported/model content manipulating execution, and experimental results being presented as stable capability. Negative safety tests, redaction, evidence/claim gates, maturity labels, and architecture review reduce but do not eliminate these risks.
|
||||
|
||||
## 3. Architectural Style
|
||||
## 2. Architectural Goals and Constraints
|
||||
|
||||
The implementation is a modular TypeScript monolith with ports/adapters characteristics. Domain modules share one Node.js runtime and package but expose cohesive responsibilities. Delivery adapters translate CLI, HTTP, library, and MCP calls into common modules. Provider adapters translate common model requests to hosted APIs, local OpenAI-compatible endpoints, or connected coding agents. Arsenal adapters translate approved tool intents to internal functions or external processes. Filesystem artifacts form the principal persistence mechanism; there is no application database in the current baseline.
|
||||
| Driver | Source | Architectural impact |
|
||||
| --- | --- | --- |
|
||||
| Conduct a scoped mission | UC-001; NFR-01/02/05 | Target context and deterministic scope/approval/origin gates precede tool dispatch. |
|
||||
| Support diverse reasoning backbones | UC-002; ADR-002 | Configuration and LLM/local-agent adapters isolate provider differences. |
|
||||
| Preserve surface parity without privilege expansion | UC-003; NFR-04/07 | CLI/library/HTTP/MCP are adapters over shared modules; HTTP defaults loopback; MCP is narrow. |
|
||||
| Analyze source with evidence and containment | UC-004; NFR-08/09/11 | Bounded ingest, parser/source locations, explicit experimental maturity. |
|
||||
| Reproduce claims | UC-005; NFR-03/09 | Versioned corpora/receipts/graders and CI gates are product components. |
|
||||
| Local-first portability | ADR-001; NFR-06 | Node.js 18+, one package/runtime, filesystem artifacts, optional Docker. |
|
||||
| Honest evolution | ADR-005; NFR-12 | New persistence, privilege, distributed/autonomous, or maturity boundaries trigger ADR/SAD updates. |
|
||||
|
||||
This style is intentional for a self-hosted tool: installation remains straightforward, domain operations can share typed contracts, and local artifacts remain visible to operators. The cost is that dependency discipline must be maintained in code review because process boundaries do not enforce module boundaries.
|
||||
Unknown active-user counts, hosted-service SLOs, fleet concurrency, maintainer capacity, and regulated deployment obligations are not converted into architecture requirements. The current operator estimate of thousands to tens of thousands of users is planning context, not measured telemetry.
|
||||
|
||||
## 4. Runtime and Deployment Views
|
||||
## 3. System Context (C4 Level 1)
|
||||
|
||||
### 4.1 CLI and Library
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Operator[Authorized operator / researcher]
|
||||
Client[Integrating CLI, library, or MCP client]
|
||||
System[T3MP3ST local platform]
|
||||
Model[Hosted model API / local inference / coding-agent CLI]
|
||||
Target[Authorized target systems]
|
||||
Tools[External security tools]
|
||||
Storage[(Local config, reports, evidence, benchmarks)]
|
||||
GitHub[Source collaboration and CI]
|
||||
Operator -->|CLI / browser| System
|
||||
Client -->|typed API / HTTP / MCP stdio| System
|
||||
System -->|bounded reasoning requests| Model
|
||||
System -->|approved scoped operations| Tools
|
||||
Tools -->|network/file observations| Target
|
||||
System -->|read/write operator-controlled artifacts| Storage
|
||||
GitHub -->|source and deterministic gates| System
|
||||
```
|
||||
|
||||
`src/cli.ts` provides the operator command surface, while `src/index.ts` exports the library-facing composition surface. Commands construct configuration, targets, missions, operators, and tools in-process. This is the lowest-overhead path and the natural entry for scripts and terminal operators.
|
||||
Trust boundaries exist at browser→localhost server, client→HTTP/MCP adapter, runtime→provider/local agent, orchestration→tool process, tool→target network, runtime→local artifacts, and repository→third-party corpora. Local deployment does not make these inputs trusted.
|
||||
|
||||
### 4.2 War Room and HTTP API
|
||||
## 4. Container View (C4 Level 2)
|
||||
|
||||
`src/server.ts` hosts the browser War Room assets and JSON endpoints. It binds to `127.0.0.1:3333` by default. Localhost CORS/origin and Host-header defenses matter because HTTP endpoints can initiate local tool or model activity. Docker Compose preserves this boundary by publishing `127.0.0.1:3333:3333`, mounts `reports/` and `evidence/`, and checks `/api/health`.
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Host[Operator host / optional application container]
|
||||
CLI[CLI + library adapter\nNode.js/TypeScript]
|
||||
HTTP[War Room + HTTP adapter\nExpress :3333]
|
||||
MCP[MCP stdio adapter]
|
||||
Core[Mission / Admiral / Operators / Analysis]
|
||||
Safety[Target / OPSEC / Approval / Arsenal]
|
||||
Providers[LLM + local-agent adapters]
|
||||
Artifact[(Local filesystem + config store)]
|
||||
CLI --> Core
|
||||
HTTP --> Core
|
||||
MCP --> Safety
|
||||
Core --> Safety
|
||||
Core --> Providers
|
||||
Core --> Artifact
|
||||
Safety --> Artifact
|
||||
end
|
||||
Providers --> Model[External/local reasoning providers]
|
||||
Safety --> ToolProc[Internal functions / external tool processes]
|
||||
ToolProc --> Target[Authorized targets]
|
||||
```
|
||||
|
||||
### 4.3 MCP
|
||||
| Container | Deployment unit | Responsibility | Protocol / exposure |
|
||||
| --- | --- | --- | --- |
|
||||
| CLI/library | `t3mp3st` Node package/process | Terminal and typed composition entry points | Local process calls |
|
||||
| War Room/HTTP | Same Node package/process; optional Docker service | Browser UI and JSON API | HTTP/1.1, `127.0.0.1:3333` by default |
|
||||
| MCP | Same package, separate stdio process mode | Narrow `security_recon` tool | MCP over stdio |
|
||||
| Core/runtime | In-process modules | Mission state, planning, operators, analysis | TypeScript calls/events |
|
||||
| External tools | Bounded child processes or internal implementations | Real security operations | Structured argv/stdout/stderr; target protocols |
|
||||
| Local artifacts | Host/container-mounted paths and config store | Config, reports, evidence, receipts | Filesystem; operator permissions |
|
||||
|
||||
`src/mcp-server.ts` exposes a stdio MCP server with a deliberately narrower supported surface than the entire HTTP/CLI application. MCP is an adapter, not a second mission engine; it must reuse shared validation and tool behavior and avoid silently broadening privileges.
|
||||
## 5. Component View and Interface Inventory
|
||||
|
||||
### 4.4 Model and Agent Providers
|
||||
| Component | Responsibility | Primary collaborators | Exposed seam / test mode |
|
||||
| --- | --- | --- | --- |
|
||||
| `MissionControl` / `TaskQueue` | Mission lifecycle and task state | Admiral, operators, arsenal | EventEmitter interfaces; unit/integration tests |
|
||||
| `Admiral` | Convert operator intent into authorized mission brief/directive | LLM backbone, mission | Class and pure conversion helpers; mocked LLM |
|
||||
| `OperatorAgent` / `OperatorCell` | Role-specific task execution and coordination | LLM, arsenal, mission | Event interfaces; mock backbone or controlled tools |
|
||||
| `LLMProviderAdapter` / `LLMBackbone` | Normalize hosted/local reasoning | Config, agent loop | Adapter interface; provider contract/mocks |
|
||||
| Local-agent adapter | Discover and invoke Claude/Codex/Hermes CLIs | Config, mission | Executable/process boundary; fixture binaries/mocks |
|
||||
| `TargetEnvironment` | Authorized target state and normalization | Mission, arsenal | Class/factory functions; direct unit tests |
|
||||
| `Arsenal` / `ToolAdapter` | Catalog, scope check, dispatch, parse | Approval, target, subprocesses | Adapter metadata/interface; fixture outputs |
|
||||
| `ApprovalController` | Risk-based approval state | HTTP/UI, arsenal | Policy/controller interface; deterministic unit tests |
|
||||
| `EvidenceVault` / live gate | Evidence/finding lifecycle, redaction, verification gate | Analysis, reporting | Events and pure gate helpers; integrity/redaction tests |
|
||||
| `DecompositionOrchestrator` | Parallel code/query decomposition and synthesis | LLM, context packs | Event interface and typed results; mock providers |
|
||||
| Delivery adapters | Parse/validate input and shape output | Shared components | CLI/HTTP/MCP contracts; static/integration tests |
|
||||
|
||||
`src/llm/`, `src/config/`, and agent/provider code resolve credentials, model identifiers, capabilities, timeouts, and fallback. Cloud-provider keys are optional when a connected local agent or local OpenAI-compatible server is used. Provider output is untrusted input to orchestration: it may suggest operations, but target, approval, evidence, and tool contracts remain authoritative.
|
||||
Interfaces without an external process boundary still share memory and failure domain. External tool subprocesses are not trusted merely because they are separate; their argument, timeout, exit, and parse contracts require tests.
|
||||
|
||||
## 5. Core Logical Components
|
||||
## 6. Data Model and Classification
|
||||
|
||||
### 5.1 Mission and Orchestration
|
||||
The canonical domain types live in `src/types/index.ts`; component-specific interfaces live beside their implementations. Representative logical entities are:
|
||||
|
||||
`src/mission/` owns mission state and lifecycle. `src/admiral/` and `src/orchestration/` plan and coordinate tasks, context packs, prompts, and adjudication. `src/operators/` supplies role-specific behavior. The current system implements a coordinated operator model, but public evidence is stronger for benchmarked single-agent paths than for reliable end-to-end swarm exploitation. The architecture baseline therefore classifies swarm superiority as unproven rather than an invariant.
|
||||
```typescript
|
||||
type MissionState = {
|
||||
id: string;
|
||||
target: Target;
|
||||
rulesOfEngagement: RulesOfEngagement;
|
||||
tasks: Task[];
|
||||
status: string;
|
||||
};
|
||||
|
||||
### 5.2 Target, Scope, and OPSEC
|
||||
type ToolExecution = {
|
||||
tool: string;
|
||||
context: ToolContext;
|
||||
startedAt: Date;
|
||||
completedAt?: Date;
|
||||
result?: ToolResult;
|
||||
};
|
||||
|
||||
`src/target/` represents the authorized target context. `src/opsec/` and arsenal approval logic apply operational constraints. Once a mission target is set, built-in networked tools reject unrelated public hosts while allowing target/subdomain and loopback/private contexts under documented rules. Exact-origin target headers prevent credential forwarding to a different origin. These controls implement NFR-01, NFR-02, and NFR-05 and are the primary safety boundary.
|
||||
type Finding = {
|
||||
id: string;
|
||||
title: string;
|
||||
severity: Severity;
|
||||
evidence: Evidence[];
|
||||
verificationState: string;
|
||||
};
|
||||
```
|
||||
|
||||
### 5.3 Arsenal
|
||||
| Entity / store | Location | Classification | Durability and constraints |
|
||||
| --- | --- | --- | --- |
|
||||
| Provider settings/API keys | `conf` store / environment | Restricted | Local; values never committed or logged |
|
||||
| Target headers/credentials | Target configuration | Restricted | Exact-origin use; redacted representations only |
|
||||
| Mission/task state | Process memory and selected artifacts | Internal/Confidential | Process-lifetime unless explicitly persisted |
|
||||
| Evidence/findings/reports | Local mounted directories | Confidential/Restricted | Operator-controlled retention and backup |
|
||||
| Benchmark manifests/receipts | `bench/` and retained local staging | Internal/Public or Restricted by corpus | Versioned selectively; sensitive/third-party raw data may remain ignored |
|
||||
| Browser preferences/state | localStorage | Internal | Browser-profile durability; not authoritative evidence |
|
||||
|
||||
`src/arsenal/` provides catalog, parser, adapter, post-exploitation, and approval modules. Some tools are internal Node implementations; others adapt external binaries. Dangerous or catalog-only drivers have narrow approved paths instead of generic arbitrary execution. Tool output is evidence, not automatically a validated finding. Changes here require adversarial tests for argument construction, scope propagation, timeout/exit behavior, and output parsing.
|
||||
There is no relational schema, cache keyspace, or migration framework in the baseline. Configuration migration helpers support legacy settings. Adding a shared database/cache/queue requires schema, migration, backup, tenancy, and data-classification design plus a new ADR.
|
||||
|
||||
### 5.4 Evidence, Findings, and Reporting
|
||||
## 7. Key Sequence Diagrams
|
||||
|
||||
`src/evidence/`, `src/analysis/`, and report/disclosure scripts transform raw observations into retained artifacts and deliverables. Verification scripts and disclosure generation support the transition from candidate to substantiated finding. Reports and evidence are mounted/preserved local paths and may contain credentials or pre-disclosure vulnerabilities; filesystem ownership does not remove the need for retention and access procedures.
|
||||
### 7.1 Scoped Mission Tool Execution
|
||||
|
||||
### 5.5 Reconnaissance and Source Ingest
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
actor O as Operator
|
||||
participant D as Delivery adapter
|
||||
participant M as Mission/Admiral
|
||||
participant T as Target+Scope
|
||||
participant A as Approval+Arsenal
|
||||
participant X as Tool process
|
||||
participant E as Evidence
|
||||
O->>D: target + RoE + objective
|
||||
D->>M: validated mission request
|
||||
M->>T: create authorized target context
|
||||
M->>A: proposed tool action
|
||||
A->>T: validate destination and credentials
|
||||
alt dangerous and not approved
|
||||
A-->>M: refused
|
||||
else allowed and approved
|
||||
A->>X: bounded structured execution
|
||||
X-->>A: output / error / timeout
|
||||
A->>E: observation + execution metadata
|
||||
E-->>M: candidate/verified disposition
|
||||
end
|
||||
```
|
||||
|
||||
Black-box reconnaissance uses real DNS, network, TLS, and HTTP operations. White-box ingestion in `src/recon/code-ingest.ts`, `ts-parse.ts`, `ts-grammars.ts`, and `whitebox.ts` extracts structural units across supported languages using web-tree-sitter, with Python retaining a specialized parser. The current documentation identifies multi-language ingest as experimental. Benchmark results can prove performance on a corpus without promoting the general engine to stable.
|
||||
### 7.2 Provider-Neutral Reasoning
|
||||
|
||||
### 5.6 Benchmarks and Provenance
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant M as Mission/Agent loop
|
||||
participant C as Config/registry
|
||||
participant P as LLM or local-agent adapter
|
||||
participant R as Provider/agent process
|
||||
participant A as Arsenal gates
|
||||
M->>C: resolve provider/model/capabilities
|
||||
M->>P: bounded normalized request
|
||||
P->>R: provider-specific request
|
||||
R-->>P: content/tool proposal/error
|
||||
P-->>M: normalized result + identity
|
||||
M->>A: validate any tool proposal
|
||||
A-->>M: allowed/refused outcome
|
||||
```
|
||||
|
||||
`bench/` stores corpora, manifests, ledgers, ground truth, and receipts. Scripts recompute claims, grade flags/findings, test model matrices, and reject fitting. `.github/workflows/ci.yml` runs lint, typecheck, tests, coverage, doctor, claim verification, anti-fitting checks, provenance gate, prompt audit, and smoke tests. This subsystem is part of product architecture because reproducibility is a product promise, not auxiliary documentation.
|
||||
### 7.3 Claim Reproduction
|
||||
|
||||
## 6. Data and State
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
actor E as Evaluator/CI
|
||||
participant V as Claim verifier
|
||||
participant C as Corpus+manifest
|
||||
participant G as Ground truth+grader
|
||||
participant R as Receipts
|
||||
E->>V: run verification
|
||||
V->>C: load versioned evaluation scope
|
||||
V->>R: validate provenance and outcomes
|
||||
V->>G: recompute per-case and aggregates
|
||||
G-->>V: derived values
|
||||
alt derived claim matches
|
||||
V-->>E: PASS with scope
|
||||
else missing/stale/mismatch
|
||||
V-->>E: FAIL with discrepant artifact
|
||||
end
|
||||
```
|
||||
|
||||
Configuration comes from environment variables, a local configuration store, and browser localStorage. Target header configuration is origin-bound. Mission state lives in memory and local artifacts according to the execution path. Reports and evidence are persisted in repository-local mounted directories by the Docker setup. Benchmark inputs and derived receipts are committed selectively. Update tooling preserves declared sensitive or expensive local paths.
|
||||
## 8. External API Surface
|
||||
|
||||
There is no current transactional application database, distributed queue, centralized cache, or multi-tenant identity store. Any future hosted architecture would introduce fundamentally new trust, tenancy, retention, backup, migration, and compliance requirements and must not be treated as a small deployment variation.
|
||||
The HTTP implementation is large; `docs/API_REFERENCE.md` is the route-level inventory. Architectural groups are summarized here.
|
||||
|
||||
## 7. Security Architecture
|
||||
| Surface / group | Representative entry points | Contract and auth boundary | UC |
|
||||
| --- | --- | --- | --- |
|
||||
| CLI | default `interactive`, `setup`, `status`, `test`, `models` | Local process/user; Commander argument validation | UC-002, UC-003 |
|
||||
| Library | exports from `src/index.ts` | Typed package API | UC-001–UC-004 |
|
||||
| HTTP health/config | `/health`, `/api/health`, `/api/preflight`, `/api/llm/status` | Loopback Host/origin guards; local operator | UC-003 |
|
||||
| HTTP mission | `/api/mission/start|stop|pause|resume|status|report` | Request validation plus target/approval gates | UC-001, UC-003 |
|
||||
| HTTP tools/approval | `/api/tools/*`, `/api/approvals/*`, arsenal routes | Scope, risk classification, explicit approval | UC-001 |
|
||||
| HTTP white-box | `/api/whitebox/analyze` | Input containment and parser limits | UC-004 |
|
||||
| HTTP evidence/findings | `/api/evidence`, `/api/findings`, retest routes | Local artifact/evidence policy | UC-001 |
|
||||
| HTTP agents/providers | `/api/models`, `/api/llm/*`, `/api/agents/local/*` | Provider configuration and bounded subprocess calls | UC-002 |
|
||||
| MCP | `security_recon` | Input schema; stdio; deliberately narrow | UC-003 |
|
||||
|
||||
The security model assumes the operator host, model output, external tools, target responses, browser requests, and imported repositories may all introduce risk. Default loopback exposure reduces remote attack surface but does not protect against malicious local content or DNS rebinding without Host/origin validation. Scope containment reduces accidental or model-driven egress but must be propagated through every network-capable adapter. Approval gates reduce the chance of executing high-impact actions without human intent. Evidence/provenance controls reduce false reporting but do not make model output trustworthy.
|
||||
No general remote-user authentication or multi-tenant authorization system is claimed. Loopback exposure is the default boundary; changing it requires an explicit deployment threat model.
|
||||
|
||||
Key failure modes are scope bypass, credential misrouting, shell/argument injection, unsafe parsing, evidence confusion, prompt injection from target/code content, and false maturity claims. CI and smoke tests cover important deterministic contracts; manual/release review remains necessary for new tools and trust-boundary changes.
|
||||
## 9. Deployment and Process Architecture
|
||||
|
||||
## 8. Quality and Operations
|
||||
| Environment | Purpose | Artifact / exposure | Persistence | Owner |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Local development | Build/test/operator use | Node package/process; loopback HTTP optional | Host config/artifacts | Developer/operator |
|
||||
| CI | Deterministic quality gates | GitHub Actions, Node 22 | Workflow artifacts/caches per policy | Maintainer |
|
||||
| Local Docker | Reproducible application runtime | Compose service; host binds loopback port | Mounted reports/evidence | Operator |
|
||||
| Isolated challenge/tool containers | CTF/tool execution | Per-harness containers | Controlled staging/results | Evaluator/operator |
|
||||
| Hosted staging/production/DR | Not part of current baseline | N/A | N/A | Requires new ADR/SAD |
|
||||
|
||||
The project favors deterministic CI that does not require live targets or paid models. Live tests and benchmarks are separated or opt-in. Timeouts bound slow local agents and task/planning calls. Doctor, preflight, smoke, capability, and gauntlet scripts provide layered diagnostics. The current codebase does not establish fleet-level SLOs, centralized observability, on-call, or disaster-recovery objectives, so those are unknown rather than missing implementation promises.
|
||||
### 9.1 Process Types and State
|
||||
|
||||
## 9. Vision Relationship
|
||||
| Process | Entry point | Scale/concurrency | State | Shutdown/recovery |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| CLI/library | `src/cli.ts` / package API | One invocation; internal tasks | Memory + local artifacts | Process exit; rerun with retained artifacts where supported |
|
||||
| HTTP/War Room | `src/server.ts` | One local process; async requests/tasks | Memory, config, browser/local artifacts | Explicit mission stop/pause; container/process restart loses volatile state |
|
||||
| MCP | `src/mcp-server.ts` | One stdio client process | Process-local | Client restarts process |
|
||||
| External tool | Adapter-built subprocess | Per action, bounded by policy/timeout | stdout/stderr and generated files | Terminate on timeout; classify failure explicitly |
|
||||
| Connected agent | Local-agent subprocess | Per bounded request | Agent process plus normalized result | Timeout/kill/fallback behavior recorded |
|
||||
|
||||
`VISION.md` describes seven research directions: cognitive architecture, swarm dynamics, adversarial machine learning, continuous autonomous operations, knowledge architecture, distributed/edge execution, and evaluation science. The current system strongly realizes evaluation science, partially realizes cognitive/swarm/knowledge ideas, contains research experiments for self-improvement, and leaves persistent autonomy and distributed execution largely future-facing. The authoritative classification and evidence are in `vision-alignment.md`.
|
||||
Startup/shutdown/resource targets are not currently measured and therefore are open NFR-11 work. The server binds `T3MP3ST_HOST`/`T3MP3ST_PORT` with `127.0.0.1:3333` defaults outside Docker. Configuration comes from environment and local config with validation/defaults. Logs/diagnostics go to process streams and UI/event channels; no centralized telemetry or SLO error budget exists.
|
||||
|
||||
Architectural evolution must preserve a two-axis view: maturity of implementation and alignment with direction. A feature can align with the vision while remaining experimental; conversely, a stable implementation need not imply completion of the broader vision vector.
|
||||
Backing resources are provider URLs, local agent executables, target URLs, SOCKS proxy configuration, and filesystem paths. Their locators come from configuration/arguments; credentials must not be embedded in repository files or forwarded across origin boundaries.
|
||||
|
||||
## 10. Architectural Risks and Evolution Rules
|
||||
## 10. Cross-Cutting Concerns
|
||||
|
||||
The main structural risk is unsafe coupling across delivery, orchestration, provider, and tool layers in a single process. The mitigation is typed shared contracts, narrow adapters, trust-boundary tests, and ADR review. The main product risk is confusing benchmark success with general system maturity. The mitigation is corpus-scoped claims, re-derivation, maturity labels, and the alignment matrix.
|
||||
### 10.1 Authentication and Authorization
|
||||
|
||||
Future changes require an ADR when they add a delivery surface, persistence system, privilege boundary, generic execution mechanism, network egress class, provider contract, or maturity promotion. A hosted/multi-tenant mode, distributed worker architecture, or autonomous persistent operation requires a new SAD baseline rather than an amendment that assumes current trust boundaries still apply.
|
||||
T3MP3ST does not implement general end-user identity/RBAC for a hosted service. Authority comes from the local operator and explicit target/RoE context. Provider credentials authenticate to provider endpoints; target credentials are exact-origin bound. Dangerous tool operations use risk-based approval. Remote/multi-user exposure is outside the baseline.
|
||||
|
||||
## 11. Traceability
|
||||
### 10.2 Cache Architecture
|
||||
|
||||
| Use case | Architectural coverage |
|
||||
No distributed application cache exists. Browser localStorage and in-process maps are local state, not authoritative shared caches. Any future shared cache requires key, TTL, invalidation, poisoning, classification, and availability design.
|
||||
|
||||
### 10.3 Observability
|
||||
|
||||
Mission events, diagnostics, health/preflight/doctor/smoke commands, test output, evidence, reports, and benchmark receipts are the current observability surfaces. Errors distinguish validation, provider, timeout, tool/process, parse, and verification outcomes where implemented. Central metrics, distributed tracing, alerting, and hosted SLOs are N/A for the current local baseline.
|
||||
|
||||
### 10.4 Configuration and Secrets
|
||||
|
||||
Environment variables and the local `conf` store provide settings; model/provider registry code supplies declared defaults/capabilities. Invalid critical settings fail explicitly. API keys and target credentials are restricted data and must be redacted. Release/update behavior protects declared local paths.
|
||||
|
||||
### 10.5 Rate Limiting and Resource Control
|
||||
|
||||
Per-user remote rate limiting is N/A because the default server is loopback-only. Finite timeouts, ingest limits, task controls, and provider/tool bounds are required resource controls. Any remote shared deployment must add authentication, rate-limit scope/algorithm/storage, quotas, and abuse monitoring.
|
||||
|
||||
### 10.6 Error Handling
|
||||
|
||||
Adapters validate input and return surface-appropriate errors; untrusted provider/tool output is not success until parsed and, for claims/findings, verified. Retries occur only through declared bounded fallback paths. Sensitive values are redacted. There is no durable distributed dead-letter queue; failed local tasks remain explicit mission/diagnostic outcomes.
|
||||
|
||||
## 11. Technology Choices
|
||||
|
||||
| Dependency | Version | Purpose | License / policy note |
|
||||
| --- | --- | --- | --- |
|
||||
| Node.js | `>=18.0.0` (CI uses 22) | Runtime | Project engine constraint |
|
||||
| TypeScript | `^5.3.3` | Typed implementation | Apache-2.0 |
|
||||
| Express | `^4.18.2` | Local HTTP/War Room server | MIT |
|
||||
| MCP SDK | `^1.25.1` | MCP stdio server | MIT |
|
||||
| Commander / Inquirer | `^12.1.0` / `^9.2.15` | CLI surfaces | MIT |
|
||||
| AJV | `^8.20.0` | JSON/schema validation | MIT |
|
||||
| Undici / socks | `^8.7.0` / `^2.8.9` | HTTP and proxy networking | MIT |
|
||||
| web-tree-sitter / grammars | `0.25.10` / `0.1.13` | Multi-language structural parsing | MIT / package-specific grammar licenses |
|
||||
| Vitest / coverage-v8 | `^4.1.9` / `4.1.9` | Tests and coverage | MIT |
|
||||
| Docker / Compose | Environment-managed | Local/containerized execution | External runtime; pin images/actions per threat model |
|
||||
|
||||
`package.json` and the lockfile are authoritative for the complete dependency/version set. License/supply-chain review must use those files; this table records architecturally significant dependencies only.
|
||||
|
||||
## 12. UC → Architecture Traceability
|
||||
|
||||
| UC | Containers / components | External entry | Verification |
|
||||
| --- | --- | --- | --- |
|
||||
| UC-001 | Mission, Admiral, Operators, Target, Approval, Arsenal, Evidence | CLI/library/HTTP mission and tool routes | Scope, approval, credential, mission, evidence tests |
|
||||
| UC-002 | Config registry, LLM backbone, local-agent adapter | Setup/models, LLM/local-agent routes | Provider, routing, local-agent, fallback tests |
|
||||
| UC-003 | CLI, library, HTTP/War Room, MCP adapters | All declared surfaces | Build/type, API hardening, validation tests |
|
||||
| UC-004 | Code ingest, parsers, white-box, analysis, evidence | White-box route/library path | Ingest limit, grammar, adversarial, containment tests |
|
||||
| UC-005 | Benchmarks, graders, receipts, claim/provenance/anti-fitting gates | Package scripts and CI | Claim/finding/fitting/provenance tests |
|
||||
|
||||
## 13. US → Architecture Traceability
|
||||
|
||||
| Stories | Implementing components / endpoint groups | Status |
|
||||
| --- | --- | --- |
|
||||
| US-01, US-04 | Target, scope, ApprovalController, Arsenal; approval/tool routes | Implemented |
|
||||
| US-02 | MissionControl, CLI, HTTP mission routes, War Room | Implemented |
|
||||
| US-03 | Config, provider registry, local-agent adapter/routes | Implemented |
|
||||
| US-05 | EvidenceVault, live gate, analysis/reporting/verifiers | Implemented |
|
||||
| US-06 | MCP `security_recon` adapter | Implemented, intentionally narrow |
|
||||
| US-07 | Ingest, Tree-sitter parsers, white-box analysis | Experimental |
|
||||
| US-08, US-10 | Claim/provenance/prompt/anti-fitting scripts and CI | Implemented |
|
||||
| US-09 | README/FEATURES, ADR-005, vision-alignment | Implemented manually; automation gap |
|
||||
|
||||
## 14. NFR → Architecture Traceability
|
||||
|
||||
| NFR | Tactic / component | Verification |
|
||||
| --- | --- | --- |
|
||||
| NFR-01/02/05 | Target context, scope gate, approval, exact-origin credentials | Arsenal scope/approval and target-header/redaction tests |
|
||||
| NFR-03 | Evidence gate, receipts, claim/finding verification | Claim, finding, provenance CI gates |
|
||||
| NFR-04 | Loopback bind plus Host/origin guards | Local API hardening tests and Compose review |
|
||||
| NFR-06/07 | One Node package, typed adapters/contracts | CI build/type/provider/contract tests |
|
||||
| NFR-08 | Fixtures/mocks/committed receipts; live work separated | Required deterministic CI jobs |
|
||||
| NFR-09 | Maturity matrix, stub/count honesty, claim verification | Tests plus documentation audit (automation gap) |
|
||||
| NFR-10 | Protected-path update behavior | Update self-tests |
|
||||
| NFR-11 | Finite timeouts and bounded ingest | Timeout/fallback/limit tests; percentile baseline open |
|
||||
| NFR-12 | ADR triggers and traceability matrices | Architecture review and artifact audit |
|
||||
|
||||
## Appendix A — Glossary
|
||||
|
||||
| Term | Definition |
|
||||
| --- | --- |
|
||||
| UC-001 | Mission/orchestration, target/OPSEC, arsenal, evidence/reporting |
|
||||
| UC-002 | Model and agent provider adapters |
|
||||
| UC-003 | CLI/library, War Room/HTTP, MCP runtime views |
|
||||
| UC-004 | Reconnaissance and source-ingest components |
|
||||
| UC-005 | Benchmark/provenance subsystem and CI |
|
||||
| Arsenal | Catalog and runtime for internal/external security tools. |
|
||||
| Receipt | Retained structured record supporting an evaluation or claim. |
|
||||
| RoE | Rules of engagement defining authorized target activity. |
|
||||
| War Room | Local browser interface served by the HTTP adapter. |
|
||||
| Current state | Capability supported by implementation/configuration/tests/evidence, not merely vision text. |
|
||||
|
||||
## 12. Accepted Decisions
|
||||
## Appendix B — Open Architectural Questions
|
||||
|
||||
This baseline is governed by:
|
||||
| # | Question | Owner | Target | Status |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | What are measured source-ingest and concurrent-mission resource/latency baselines? | Test/architecture steward | Construction | Open |
|
||||
| 2 | Is package repository metadata intentionally different from canonical tracker identity? | Release integrator | Iteration 001 | Open |
|
||||
| 3 | What machine-checkable format will enforce maturity consistency? | Architecture/evaluation steward | Construction | Open |
|
||||
| 4 | Is every network-capable adapter mapped to a scope/approval test? | Safety reviewer | Construction | Open |
|
||||
|
||||
- ADR-001 — modular monolith and local-first deployment
|
||||
- ADR-002 — provider-neutral reasoning backbone
|
||||
- ADR-003 — scope and approval enforcement below model output
|
||||
- ADR-004 — evidence-derived public claims
|
||||
- ADR-005 — explicit separation of current state from research vision
|
||||
## Appendix C — Review Incorporation Log
|
||||
|
||||
| Date | Reviewer | Feedback | Resolution | Sections |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 2026-07-20 | Brownfield baseline process | Separate implemented architecture from research vision | Accepted; ADR-005 and alignment matrix | 1, 2, 12–14 |
|
||||
| 2026-07-21 | Artifact standards audit | Add template reasoning, diagrams, interfaces, data, sequences, deployment, matrices, open questions, and citable references | Incorporated | All |
|
||||
|
||||
## References
|
||||
|
||||
- @.aiwg/requirements/UC-001.md — Scoped mission behavior.
|
||||
- @.aiwg/requirements/UC-002.md — Provider/local-agent behavior.
|
||||
- @.aiwg/requirements/UC-003.md — Delivery-surface behavior.
|
||||
- @.aiwg/requirements/UC-004.md — Source-analysis behavior.
|
||||
- @.aiwg/requirements/UC-005.md — Claim reproduction behavior.
|
||||
- @.aiwg/requirements/user-stories.md — Story-level coverage.
|
||||
- @.aiwg/requirements/nfr-register.md — Quality constraints and pass conditions.
|
||||
- @.aiwg/architecture/adr-001.md — Local-first modular-monolith decision.
|
||||
- @.aiwg/architecture/adr-002.md — Provider-neutral reasoning decision.
|
||||
- @.aiwg/architecture/adr-003.md — Deterministic safety boundary.
|
||||
- @.aiwg/architecture/adr-004.md — Evidence-derived claim boundary.
|
||||
- @.aiwg/architecture/adr-005.md — Current-state/vision separation.
|
||||
- @src/types/index.ts — Canonical domain types.
|
||||
- @src/server.ts — HTTP implementation and exposure controls.
|
||||
- @src/mcp-server.ts — MCP contract.
|
||||
- @package.json — Runtime, scripts, dependency, and license metadata.
|
||||
- @.github/workflows/ci.yml — Automated quality/evidence gates.
|
||||
@@ -1,8 +1,17 @@
|
||||
# Vision-to-Code Alignment Matrix
|
||||
|
||||
**Reference:** `VISION.md`
|
||||
**Reference:** `VISION.md`
|
||||
**Rule:** Alignment is directional; only code/tests/receipts establish implementation maturity.
|
||||
|
||||
## Assessment Method
|
||||
|
||||
- **Implemented:** Supported operational path with current code, deterministic shared-safety tests, and maintained documentation.
|
||||
- **Partial / experimental:** Working code or bounded benchmark evidence exists, but reliability, breadth, safety coverage, or support is incomplete.
|
||||
- **Research:** Evidence answers a defined research question but is not a supported general product contract.
|
||||
- **Future:** Direction is documented without an implemented, tested operational path.
|
||||
|
||||
Evidence confidence is bounded by the cited source: source code establishes presence, tests establish behavior under their cases, and a receipt establishes only its declared corpus/harness/model/metric. No row is a substitute for a requirement, threat model, or release decision.
|
||||
|
||||
| Vision vector | Current maturity | Current evidence | Gap / next architectural proof |
|
||||
| --- | --- | --- | --- |
|
||||
| Cognitive architecture | Partial / experimental | Admiral planning, context packs, prompts, operator roles, adjudication | Demonstrate durable reasoning-state architecture and comparative outcomes beyond prompt composition |
|
||||
@@ -20,3 +29,32 @@
|
||||
3. Promotion to stable requires deterministic safety tests, an operational path, documentation, and a reproducible receipt.
|
||||
4. Persistent autonomy, distributed execution, or shared knowledge services trigger new threat models and ADRs.
|
||||
5. The SAD is updated from implementation evidence; the vision is not reverse-engineered into fictitious components.
|
||||
|
||||
## Promotion Checklist
|
||||
|
||||
A capability may move toward stable only when all applicable items are evidenced:
|
||||
|
||||
- [ ] User goal, acceptance behavior, and NFRs are identified.
|
||||
- [ ] Operational entry path and failure/rollback behavior are documented.
|
||||
- [ ] Shared authorization, scope, secret, evidence, and approval controls apply.
|
||||
- [ ] Deterministic positive and adversarial tests cover critical paths.
|
||||
- [ ] Benchmark/claim evidence is reproducible and scoped.
|
||||
- [ ] SAD/API/deployment/test documentation and traceability are updated.
|
||||
- [ ] Architecture, safety, test, and evaluation reviewers record approval.
|
||||
|
||||
## Review Triggers
|
||||
|
||||
Review this matrix when a public feature label changes, a benchmark/receipt is added or regraded, a new provider/tool/domain is exposed, a roadmap capability gains an operational path, or the SAD/ADRs change a maturity boundary. Persistent autonomy, distributed execution, hosted multi-tenancy, and shared knowledge services require new threat/architecture work before promotion.
|
||||
|
||||
## References
|
||||
|
||||
- @VISION.md — Directional research source.
|
||||
- @README.md — Primary product claims and maturity labels.
|
||||
- @FEATURES.md — Feature inventory and maturity surface.
|
||||
- @.aiwg/architecture/software-architecture-doc.md — Current implemented architecture.
|
||||
- @.aiwg/architecture/adr-004.md — Evidence-derived claims policy.
|
||||
- @.aiwg/architecture/adr-005.md — Current-state/vision separation decision.
|
||||
- @.aiwg/requirements/UC-005.md — Reproduction and maturity acceptance behavior.
|
||||
- @.aiwg/requirements/nfr-register.md — NFR-03 and NFR-09.
|
||||
- @scripts/verify-claims.mjs — Claim derivation implementation.
|
||||
- @src/__tests__/stub-honesty.test.ts — Stub/maturity regression evidence.
|
||||
Reference in new issue
Block a user