Make the contributor, CI, package, and tagged-release contracts agree with the shared BT6 baseline while preserving T3MP3ST's project-specific integrity checks. Exclude workspace-only material from published packages and keep local-agent detection tests host-independent.\n\nCloses #159
Add user and developer documentation plus Pagenary docsite publishing support.
Includes Getting Started, Developer Guide, API and MCP references, Pagenary tenant metadata, npm scripts for sync/build/check/serve, and validation coverage for the generated docsite.
Closes#102
Integrate SOCKS5 attack-traffic proxying, egress leak visibility, and local-model hardening. Preserve direct local/LAN model traffic while the proxy is active and reconcile the branch with current upstream.
* feat: add richer JSON Schema types and AJV validation utility
- Extended ToolParameter with enum, items, properties, additionalProperties (max depth 3)
- Added ToolError class with ToolErrorCategory enum
- Added ToolValidationError interface
- Created AJV-based validateToolArgs() with lazy loading and schema depth enforcement
- 57 new tests covering types and validation
* feat: add validation gate, parallel execution, and structured error feedback
- Arsenal.execute() validates args against JSON Schema before handler execution
- Tool execution errors categorized (scope_denied, tool_not_found, timeout, execution_error, validation_error)
- ToolError import fix in agent (was missing, causing silent ReferenceError)
- Semaphore-based parallel tool execution with configurable concurrency (default 5)
- Results maintain original call order regardless of completion order
- Structured error feedback to LLM with category only (no raw error messages)
- Scope/approval gates checked per-call before execution
- 35 new tests covering validation, parallel execution, and error feedback
* feat: add Ollama native function calling with probe + cache + fallback
- Added nativeTools config option to LLMConfig
- LocalAdapter probes model support for native tools on first call
- Probe result cached per model name to avoid re-probing
- Native tools formatted as Ollama /api/chat tools parameter
- Tool calls parsed from Ollama's tool_calls response format
- Falls back to text-based renderToolContract when native not supported
- OpenAI-compatible wire format also supported (for vLLM, LM Studio)
- 10 new tests covering probe, cache, fallback, and config overrides
* fix(tools): harden validation and native fallback
---------
Co-authored-by: sronaal <sronaal@users.noreply.github.com>
Co-authored-by: Joseph Magly <1159087+jmagly@users.noreply.github.com>
* chore(ingest): pin web-tree-sitter@0.25.10 + tree-sitter-wasms@0.1.13; add multilang-ingest design docs
* feat(ingest): multi-language grammar registry (web-tree-sitter, 8 langs, fail-open); 100% cov
* feat(ingest): language-aware param-name splitter (py/go/ts/java/c); 100% cov
* feat(ingest): tree-sitter CodeBlock extractor with wall-clock parse budget + fail-open
* feat(ingest): cross-lang sinks + multilang config + wire tree-sitter into ingestRepository (sync)
* feat(ingest): repoint white-box production callers to multilang config + prod regression test
* server-init-wiring: load multi-language grammars once at bootstrap
initGrammars() runs after LLM init, before app.listen. Internally
fail-open (never rejects): on failure the grammar registry stays empty
and ingest falls back to the Python regex parser.
* adversarial-tests: hostile-input hardening for untrusted target source
Malformed per-language source, wrong-language content, binary/non-UTF8,
empty, unicode identifiers, 20k-func large file (wall-clock budget),
deeply nested input, and a symlink-loop crawl — all must fail-open, never
throw or spin. Drives parseFileMultiLang + ingestRepository/crawl.
* readme-status + docs: flip white-box status to multi-language ingest
README source-code rows now state web-tree-sitter multi-language ingest
(fail-open to Python regex). whitebox.ts scope caveat + no-source error
message updated off the stale Python-only wording.
* coverage-setup: enforce 100% coverage gate on the three new ingest files
vitest.config.ts adds a coverage-only block (v8, perFile thresholds=100)
scoped to ts-grammars.ts, param-split.ts, ts-parse.ts. Test discovery and
env stay on defaults. Edited files gate on new-branch coverage via the
regression suites; the server.ts initGrammars bootstrap line is the
documented smoke-covered carve-out.
* review: fix WASM heap leak, language-scoped fail-open, sink recall/precision
Adversarial security + strict code review (two independent subagents)
returned one blocking bug and several correctness findings; remediated:
BLOCKING #1 — native WASM memory leak: web-tree-sitter Tree/Parser objects
were never .delete()'d. Both reviewers reproduced a process-global wedge:
after ~7 large parses the shared WASM heap exhausts, every subsequent parse
aborts (caught by fail-open), and non-Python ingest silently returns []
forever. Fix: free the tree in a finally; delete the parser in resetParser.
Regression test parses ~8MB through the real shared parser 12x.
regex, which mis-parses brace/class syntax into corrupt blocks (wrong line
spans, lost methods). Now non-.py fail-open returns [] (honest absence); the
Python regex only ever runs on .py. Isolated ts-parse-fallback test.
http(s).request, axios(config), C exec-family); negative lookbehind stops
bare system(/popen(/fetch( from matching Python receivers (os.system etc.
still matched, corpus rankings unchanged). Recall+precision tests added.
operator sees non-Python degradation.
(one-line call-graph edges, no decorators, C++ member methods).
* review round 2: fix sink-regex regressions, real timeout test, deterministic leak guard
Third review round (correctness + security re-attack + test-quality) against
the round-1 fixes surfaced issues in the fixes themselves; remediated:
- Sink-regex regressions I introduced: (1) my fetch lookbehind dropped
window.fetch/this.fetch (top JS SSRF idiom) — reverted to \bfetch\(;
(2) bare .Get/.Post/.Do flagged db.Get/cache.Get/queue.Do as SSRF —
receiver-qualified to [Cc]lient. Recall+precision tests cover both;
regex is ReDoS-free (timed).
- Placebo leak test: the 12x function-dense loop did NOT wedge the parser
on machines with heap headroom (proven: passed with tree.delete removed).
Replaced with a deterministic vi.spyOn(Tree.prototype,'delete') assertion,
verified RED without the fix.
- Real (non-mocked) parse-timeout test: parseFileMultiLang gains an injectable
budgetMs; a 0ms budget drives the real progressCallback cancellation to [].
- runWhiteboxAnalysis swap now has coverage (spy on orchestrator.run) so the
2nd production caller can't silently revert to the Python-only config.
- Un-vacuoused weak tests: unicode now asserts the extracted block; nested/
wrong-language wrapped in invariants() + timing bounds.
- CI now runs npm run test:coverage (the 100% gate was defined but unenforced).
- Security: documented maxFileBytes as the control bounding a single parse's
peak WASM memory below the 2GB heap-abort ceiling.
- Stale 'fail-open to Python regex' docs fixed (ts-parse, whitebox, server,
adversarial); circular-import note added; temp-dir cleanup; honest coverage
wording in spec + config.
* docs(ingest): align fallback-contract text with the shipped language-scoped fail-open
Review feedback (#75, @jmagly): the design docs + one ts-grammars comment
still described the pre-implementation 'fall back to the Python regex'
model for non-Python files. The shipped contract is language-scoped: .py
uses parseFile; any non-.py file with a missing/unsupported/failed grammar
or a parse timeout returns [] (never the Python regex, which would mis-parse
brace/class syntax into corrupt blocks). Updated every stale reference in
multilang-ingest-spec.md, multilang-ingest-plan.md, and ts-grammars.ts so
the docs match the security boundary. Docs/comment only — no code change.
* fix(test): make the ReDoS wall-clock guard robust under coverage instrumentation
Review feedback (#75, @jmagly): the CI step this PR adds (npm run
test:coverage = vitest run --coverage) runs the full suite under V8
coverage, and that overhead pushes the pathological-brace DoS timing guard
in local-agent-tool-calling.test.ts:69 over its <300ms bound (320 < 300
fails on CI). Raise the bound to 1000ms — still 2x below the ~2100ms
quadratic-ReDoS regression it guards against, so it fails hard if the
regression returns, but no longer false-fails under coverage/slow CI.
Chose the reviewer's option (a) (robust bound) over scoping test:coverage,
so the gate stays a simple full-suite run and no test-file enumeration can
rot. Verified: npm run test:coverage is green and the 3 new files stay at
100%.
* fix(ingest): cover pointer-returning C functions
---------
Co-authored-by: Joseph Magly <1159087+jmagly@users.noreply.github.com>
Joseph Magly