Joseph Magly
afc9dad1b2
* docs(policy): allow low-impact contributor courtesy cleanup * chore(aiwg): migrate to shared BT6 maintainer plugin * chore: reconcile delivery validation
125 lines
6.9 KiB
Markdown
125 lines
6.9 KiB
Markdown
# ADR-002 — Keep the Reasoning Backbone Provider-Neutral
|
|
|
|
## Metadata
|
|
|
|
- **ADR ID:** ADR-002
|
|
- **Status:** Accepted (retrospective baseline)
|
|
- **Decision date:** 2026-07-20
|
|
- **Authors:** Architecture baseline process
|
|
- **Reviewers:** Maintainer, integration, and safety-review roles; named human approval is not recorded
|
|
- **Supersedes:** None
|
|
- **Current level:** L3 — Accepted; multiple adapter paths are implemented
|
|
|
|
## Context
|
|
|
|
### Problem
|
|
|
|
Operators may use hosted model APIs, local OpenAI-compatible servers, or already-authenticated coding-agent CLIs. These providers differ in authentication, model names, base URLs, timeouts, tool-calling formats, and failure behavior. Binding mission orchestration to one provider would weaken local/keyless operation and make provider output an accidental trust boundary.
|
|
|
|
### Constraints
|
|
|
|
- No single provider exposes a stable universal capability contract.
|
|
- Local agent and local inference paths must remain viable without new cloud credentials.
|
|
- Provider responses are nondeterministic and may include unsafe tool proposals.
|
|
- Provider-specific behavior must remain observable rather than erased by a lowest-common-denominator abstraction.
|
|
- Secrets must not cross provider/base-URL boundaries or appear in diagnostics.
|
|
|
|
### Stakeholders
|
|
|
|
- Operators need provider choice and fallback clarity.
|
|
- Integrators need a stable common interface with explicit capability differences.
|
|
- Safety reviewers need deterministic controls below model output.
|
|
- Maintainers need provider-specific tests without duplicating mission logic.
|
|
|
|
## Source Verification and Claim Tracking
|
|
|
|
| Claim | Source | Verification |
|
|
| --- | --- | --- |
|
|
| Hosted, local OpenAI-compatible, and connected-agent paths exist | Configuration, LLM, and local-agent modules | Direct code and provider test inspection |
|
|
| Provider/model compatibility is registry-driven | Provider/model registry | Registry tests |
|
|
| Local-agent tool proposals use shared runtime controls | Local-agent and arsenal integration | Tool-calling and approval/scope tests |
|
|
|
|
## Decision
|
|
|
|
Resolve provider/model differences behind shared configuration and LLM/agent adapter contracts. Preserve provider capability metadata and explicit routing/fallback behavior. Treat all provider output as untrusted proposals: target authorization, scope, dangerous-action approval, argument validation, evidence, and reporting policy remain in T3MP3ST.
|
|
|
|
Provider-specific adapters may expose additional capability only through declared contract extensions and tests. A fallback must be configured, bounded, and recorded; silent provider switching is not acceptable for evidence-bearing work.
|
|
|
|
## Alternatives Considered
|
|
|
|
### Standardize on one hosted provider
|
|
|
|
- **Pros:** Smaller adapter/test matrix and simpler support.
|
|
- **Cons:** Vendor dependency, credential requirement, weaker local operation, and provider outage concentration.
|
|
- **Why not chosen:** Conflicts with the existing product promise and implemented local-agent/provider diversity.
|
|
|
|
### Let each workflow call providers directly
|
|
|
|
- **Pros:** Maximum provider-specific freedom.
|
|
- **Cons:** Duplicated authentication, retry, timeout, secret, and response logic; safety behavior drifts by workflow.
|
|
- **Why not chosen:** Makes provider behavior inseparable from domain and safety logic.
|
|
|
|
### Normalize every provider to the lowest common feature set
|
|
|
|
- **Pros:** Superficially uniform orchestration.
|
|
- **Cons:** Hides meaningful capability and error differences and can misroute tool behavior.
|
|
- **Why not chosen:** The common contract must normalize transport, not erase capabilities.
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
|
|
- Users retain hosted, local, and connected-agent choices.
|
|
- Mission/orchestration code depends on stable internal contracts.
|
|
- Safety controls remain provider-independent.
|
|
|
|
### Negative
|
|
|
|
- Compatibility, fallback, timeout, and secret-routing test matrices grow with every provider.
|
|
- Provider-specific capabilities require explicit extension and documentation.
|
|
- Similar-looking model names or base URLs can still create configuration hazards.
|
|
|
|
## Risks and Mitigations
|
|
|
|
| Risk | Likelihood / impact | Mitigation |
|
|
| --- | --- | --- |
|
|
| Credential sent to wrong endpoint | Low / Critical | Explicit provider/base-URL routing, environment/static tests, redaction |
|
|
| Silent fallback changes evidence identity | Medium / High | Record primary/fallback outcomes and model/provider identity |
|
|
| Adapter reports malformed output as success | Medium / High | Schema/response validation and explicit error feedback |
|
|
| Lowest-common-denominator abstraction hides capability | Medium / Medium | Capability registry and provider-specific contract tests |
|
|
|
|
## Implementation and Verification
|
|
|
|
- **Configuration:** `src/config/` resolves provider, model, base URL, secret source, and defaults.
|
|
- **Hosted/local interface:** `src/llm/index.ts` normalizes reasoning requests and responses.
|
|
- **Connected agents:** `src/agent/local-agents.ts` discovers and invokes supported authenticated agent CLIs.
|
|
- **Concurrency/state:** Requests are bounded tasks; no provider response becomes shared durable state until mission/evidence code records it.
|
|
- **Testing:** Provider-model registry, provider-specific adapters, base-URL routing, local-agent selection/path/tool-calling, timeout/fallback, and redaction.
|
|
- **Regression guard:** A new provider is incomplete until configuration, capability metadata, error/timeout, secret, and tool-proposal behavior are tested.
|
|
|
|
## Definition of Done
|
|
|
|
- [x] Provider-neutral contract and trust boundary documented.
|
|
- [x] Hosted, local inference, and connected-agent implementations identified.
|
|
- [x] Alternatives and security risks recorded.
|
|
- [ ] Named maintainer/integration approval is recorded.
|
|
- [ ] Evidence receipts consistently capture provider/model/fallback identity across all benchmark paths.
|
|
|
|
## References
|
|
|
|
- @.aiwg/requirements/UC-002.md — Provider and local-agent use case.
|
|
- @.aiwg/requirements/nfr-register.md — Secret, portability, compatibility, testability, and timeout constraints.
|
|
- @.aiwg/architecture/software-architecture-doc.md — Provider adapter and trust-boundary views.
|
|
- @src/config/index.ts — Runtime configuration.
|
|
- @src/config/provider-models.ts — Provider/model capability registry.
|
|
- @src/llm/index.ts — LLM adapter interface.
|
|
- @src/agent/local-agents.ts — Connected-agent adapter.
|
|
- @src/__tests__/provider-models.test.ts — Compatibility registry evidence.
|
|
- @src/__tests__/provider-base-url-routing-static.test.ts — Endpoint-routing evidence.
|
|
- @src/__tests__/local-agent-tool-calling.test.ts — Tool-proposal integration evidence.
|
|
- @scripts/test-model-fallback.mjs — Fallback behavior evidence.
|
|
|
|
## Review and Approval
|
|
|
|
This is a retrospective description of implemented adapter policy. “Accepted” means it governs the current baseline; it does not fabricate historical approval. Future provider additions or contract changes require maintainer/integration review and an updated evidence matrix.
|