main
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Lyubomir Bozhinov
|
41f0c14039 |
fix(obsidivm): actionable error when the range service is unreachable (#156) (#158)
* test(obsidivm): RED — bridge should give an actionable error when the service is down The obsidivm:* benchmarks fetch from the OBSIDIVM range service; when it is not running the bridge throws a bare 'fetch failed' (surfaced as 'FATAL: fetch failed'), naming neither the service, the URL, nor how to start it. Pins that a connection failure becomes an actionable, self-service error. Fails on main: the message is 'fetch failed'. Refs #156 * fix(obsidivm): actionable error when the range service is unreachable The obsidivm:* benchmarks fetch from the OBSIDIVM range service; when it is not running the bridge's shared call() let the raw 'fetch failed' propagate, which surfaced as an opaque 'FATAL: fetch failed' (#156) naming neither the service, the URL, nor how to start it — even in --hunter stub, whose grading step still needs it. Wrap the fetch in call() (the single choke point for every OBSIDIVM request) so a connection failure becomes: 'OBSIDIVM service unreachable at <url> ... Start it per docs/OBSIDIVM.md (python3 range.py, listens on :4200), or set OBSIDIVM_URL', flagged err.unreachable to distinguish it from an HTTP-status error; a timeout is reported distinctly. Clarify in the bench --help/usage that every hunter mode (stub included) needs the OBSIDIVM service, live/t3mp3st also a key. Adds scripts/obsidivm-bridge.d.mts so the new test types cleanly. Closes #156 * fix(obsidivm): distinguish invalid-URL from unreachable; test timeout path Adversarial review of the diff found two gaps: - A malformed OBSIDIVM_URL (ERR_INVALID_URL) was folded into the 'service unreachable — is it running?' path, misdirecting the user to start a service when the real fix is the URL. Now branched to a distinct 'Invalid OBSIDIVM base URL … set OBSIDIVM_URL' error, no err.unreachable flag. - The timeout (AbortError) branch had no coverage. Added a test using a never-responding TCP server + short timeoutMs asserting 'timed out after Nms' and NOT unreachable; and a test for the invalid-URL branch. Also completed the .d.mts stub with the cloudgoat* methods it was missing. Refs #156 |
||
|
Lyubomir Bozhinov
|
6d4e01756a |
feat(binary): detect memcpy/memmove with a non-constant length (#151)
* test(binary): RED — no rule for memcpy/memmove with a non-constant length Adds two corpus fixtures + direct assertions for a memcpy/memmove sink: - memcpy-wire-length (bad): a wire-decoded length into a fixed buffer, expect B-MEMCPY. Fails on main — the ruleset has no memcpy rule. - memcpy-bounded-control (clean): two safe memcpys (sizeof-bounded + a constant length). This forces a DISCRIMINATING rule: a naive /memcpy/ match false-positives here and fails the zero-false-positive gate. Refs #150 * feat(binary): detect memcpy/memmove with a non-constant length Adds B-MEMCPY to the decompiled-vuln ruleset — the classic unbounded-copy overflow. The framework's own reasoning layer treats this as its textbook target (src/orchestration/prompts.ts:60,80 — 'a wire-controlled length flowing into an unchecked memcpy'; orchestration/index.ts:11), but the static pre-filter had no rule for it: strcpy was covered, its memcpy(dst, src, len) sibling was not. The rule inspects the length (3rd) argument and fires only when it is non-constant — a variable, struct field, deref, or computed/decoded value — staying silent on a sizeof(...) or numeric-literal length. Validated against a 22-case dangerous/safe matrix (perfect discrimination). Corpus 6+2 -> 7+3; the new sizeof/constant-bounded control keeps the rule honest — a bare /memcpy/ match would false-positive on it. Closes #150 * fix(binary): stop B-MEMCPY false-positive on nested-call args; disclose limits Adversarial review found a real false positive: a comma inside a nested call in arg 1/2 (memcpy(dst, get_src(a,b), sizeof(dst))) mis-split the naive [^,]+,[^,]+, arg list, so a sizeof-BOUNDED call fired. Restricting args 1-2 to [^,()]+ (paren-free) removes it — verified the bounded nested-call form is now silent while every common non-constant length still fires. The trade is a memcpy whose 1st/2nd arg is itself a call is not matched (an acceptable miss for a directional pre-filter, now disclosed). Rule comment rewritten to honestly enumerate the directional misses (leading- digit computed length, sizeof-plus-term, nested-call args, inline comments, __memcpy_chk/memcpy_s/wmemcpy variants). Test expanded into an in-tree discrimination matrix (fires / silent / pinned disclosed-limit cases) so the scope boundary is auditable and any future tightening surfaces deliberately. Refs #150 * fix(binary): match pointer casts in B-MEMCPY args; correct disclosed limits A second adversarial pass on the shipped rule caught that the previous paren-free tightening ([^,()]+) over-corrected: it also stopped matching pointer CASTS on args 1-2 — memcpy((void *)dst, (void *)src, len) — which are the most common memcpy shape in real decompiled output. The rule was silently missing the main case. Replace the arg matcher with one level of BALANCED parens, (?:[^,()]|\([^()]*\))+, so a cast (void *)dst and a comma-bearing call get(a,b) are each consumed as ONE arg. This fires on casts again, still fires on a call-bearing arg with a non-constant length (now a true positive, previously a disclosed miss), and keeps the original nested-call-with-bounded-length false positive silent. Verified no catastrophic backtracking on pathological input. Also corrects the disclosed-limits comment: the previous 'arg is itself a call' miss is now only DOUBLY-nested parens, and the inline-comment note was backwards (a comment obscuring a sizeof/literal false-fires, it does not cause a miss) — restated accurately, with decompiled output not emitting inline comments noted. Tests pin the cast fires and the variant exclusions. Refs #150 |
||
|
Lyubomir Bozhinov
|
9020e5fc08 |
feat(ingest): extract JS/TS arrow and function-expression definitions (#142)
* test(ingest): RED — JS/TS arrow and function-expression definitions are not extracted Pins issue #141 in both directions: every named-value function form must be extracted with its name, parameters and span, and the shapes carrying no name-to-function binding (IIFE, destructured, HOC-wrapped) must stay unmatched. Also covers the end-to-end consequence — an arrow-bound sink ranking attack_surface, and an arrow-bound handler becoming an entry point. Fails on main: the .js/.ts/.tsx def-queries bind declarations only. Refs #141 * feat(ingest): extract JS/TS arrow and function-expression definitions The .js/.ts/.tsx def-queries bound declarations only, so every definition bound to a value produced no CodeBlock and its sinks never reached classification or ranking. Adds the name-to-function-value shapes to all three queries: const f = (a) => {} / function (a) {} variable_declarator const f = a => ... un-parenthesized single param obj.f = ... / module.exports.handler assignment_expression onmessage = e => ... bare-identifier assignment { f: (a) => {} } pair class K { f = (a) => {} } field_definition / public_field_definition Query-only — no parser, pipeline or CodeBlock-shape change. nodeToCodeBlock already handles any @def/@name/@params triple and kindOf maps these to 'function'. Property-assigned and object-literal forms are named by the property, consistent with how method_definition is named: that is the name buildCallGraph and the entry-point heuristic match on. Not matched, by design: HOC-wrapped values (const h = withAuth(fn)) and anonymous callbacks (app.get('/x', (req,res) => ...)) have no name-to-function binding, and inventing one is worse than honest absence. whitebox.ts caveat block updated to match. Refs #141 * fix(ingest): capture #private class-field arrows; keep same-line block ids distinct Two defects found by adversarial review of the arrow-extraction change: - `class C { #handler = (x) => eval(x) }` yielded nothing: a private field's name is a `private_property_identifier`, not the `property_identifier` the class-field pattern required. A sink in a private handler is exactly as dangerous as one in a public handler, so it is captured (name keeps the `#`). - A block id is `path::name@line`, so two same-named definitions starting on the same line collide, and buildCallGraph keys on that id — the two blocks silently share one callers/callees entry even when only one carries a sink. Rare for declarations, routine once value-bound functions are extracted (minified bundles pack many `{ handler: … }` onto one line). The later block is now disambiguated by column; the first keeps the plain id, so no existing id changes and the Python path is untouched. Refs #141 |
||
|
Lyubomir Bozhinov
|
32b81708ea |
test(ui): add semantic defect gate for inline scripts (#111 hardening) (#113)
#111 shipped 8 bad-auto-merge scars into the SPA past a CI that only ever lints/tests `src/` — the 27k-line docs/index.html had (and still has) no static gate. The parse test added with the #111 fix catches outright syntax failures; this adds the *semantic* layer that parsing misses. It runs ESLint's high-precision "possible problem" rules programmatically over each inline <script> (no new dependency — `eslint` is already a devDependency; no eslint.config or CI change — rides the existing `npm test`): no-redeclare, no-unreachable, no-dupe-keys, no-dupe-args, no-dupe-else-if, no-duplicate-case, no-const-assign, no-import-assign, no-class-assign — the exact shapes a conflict-concatenating merge leaves behind. Tight, always-a-bug rules, each script linted in isolation → zero false positives on the real file. no-undef / no-unused-vars are excluded (the SPA relies on cross-<script> and browser globals a per-block lint can't see) and no-func-assign is excluded (the app legitimately monkey-patches navigateTo). Verified: 0 findings on the fixed tree, and it rejects the pre-fix main (catches the duplicate `response`). Full suite green. Also drops a now-unused eslint-disable directive in the parse test. |
||
|
Lyubomir BozhinovandJoseph Magly
|
151d3555c6 |
feat(ingest): multi-language white-box ingest (web-tree-sitter) (#75)
* chore(ingest): pin web-tree-sitter@0.25.10 + tree-sitter-wasms@0.1.13; add multilang-ingest design docs * feat(ingest): multi-language grammar registry (web-tree-sitter, 8 langs, fail-open); 100% cov * feat(ingest): language-aware param-name splitter (py/go/ts/java/c); 100% cov * feat(ingest): tree-sitter CodeBlock extractor with wall-clock parse budget + fail-open * feat(ingest): cross-lang sinks + multilang config + wire tree-sitter into ingestRepository (sync) * feat(ingest): repoint white-box production callers to multilang config + prod regression test * server-init-wiring: load multi-language grammars once at bootstrap initGrammars() runs after LLM init, before app.listen. Internally fail-open (never rejects): on failure the grammar registry stays empty and ingest falls back to the Python regex parser. * adversarial-tests: hostile-input hardening for untrusted target source Malformed per-language source, wrong-language content, binary/non-UTF8, empty, unicode identifiers, 20k-func large file (wall-clock budget), deeply nested input, and a symlink-loop crawl — all must fail-open, never throw or spin. Drives parseFileMultiLang + ingestRepository/crawl. * readme-status + docs: flip white-box status to multi-language ingest README source-code rows now state web-tree-sitter multi-language ingest (fail-open to Python regex). whitebox.ts scope caveat + no-source error message updated off the stale Python-only wording. * coverage-setup: enforce 100% coverage gate on the three new ingest files vitest.config.ts adds a coverage-only block (v8, perFile thresholds=100) scoped to ts-grammars.ts, param-split.ts, ts-parse.ts. Test discovery and env stay on defaults. Edited files gate on new-branch coverage via the regression suites; the server.ts initGrammars bootstrap line is the documented smoke-covered carve-out. * review: fix WASM heap leak, language-scoped fail-open, sink recall/precision Adversarial security + strict code review (two independent subagents) returned one blocking bug and several correctness findings; remediated: BLOCKING #1 — native WASM memory leak: web-tree-sitter Tree/Parser objects were never .delete()'d. Both reviewers reproduced a process-global wedge: after ~7 large parses the shared WASM heap exhausts, every subsequent parse aborts (caught by fail-open), and non-Python ingest silently returns [] forever. Fix: free the tree in a finally; delete the parser in resetParser. Regression test parses ~8MB through the real shared parser 12x. regex, which mis-parses brace/class syntax into corrupt blocks (wrong line spans, lost methods). Now non-.py fail-open returns [] (honest absence); the Python regex only ever runs on .py. Isolated ts-parse-fallback test. http(s).request, axios(config), C exec-family); negative lookbehind stops bare system(/popen(/fetch( from matching Python receivers (os.system etc. still matched, corpus rankings unchanged). Recall+precision tests added. operator sees non-Python degradation. (one-line call-graph edges, no decorators, C++ member methods). * review round 2: fix sink-regex regressions, real timeout test, deterministic leak guard Third review round (correctness + security re-attack + test-quality) against the round-1 fixes surfaced issues in the fixes themselves; remediated: - Sink-regex regressions I introduced: (1) my fetch lookbehind dropped window.fetch/this.fetch (top JS SSRF idiom) — reverted to \bfetch\(; (2) bare .Get/.Post/.Do flagged db.Get/cache.Get/queue.Do as SSRF — receiver-qualified to [Cc]lient. Recall+precision tests cover both; regex is ReDoS-free (timed). - Placebo leak test: the 12x function-dense loop did NOT wedge the parser on machines with heap headroom (proven: passed with tree.delete removed). Replaced with a deterministic vi.spyOn(Tree.prototype,'delete') assertion, verified RED without the fix. - Real (non-mocked) parse-timeout test: parseFileMultiLang gains an injectable budgetMs; a 0ms budget drives the real progressCallback cancellation to []. - runWhiteboxAnalysis swap now has coverage (spy on orchestrator.run) so the 2nd production caller can't silently revert to the Python-only config. - Un-vacuoused weak tests: unicode now asserts the extracted block; nested/ wrong-language wrapped in invariants() + timing bounds. - CI now runs npm run test:coverage (the 100% gate was defined but unenforced). - Security: documented maxFileBytes as the control bounding a single parse's peak WASM memory below the 2GB heap-abort ceiling. - Stale 'fail-open to Python regex' docs fixed (ts-parse, whitebox, server, adversarial); circular-import note added; temp-dir cleanup; honest coverage wording in spec + config. * docs(ingest): align fallback-contract text with the shipped language-scoped fail-open Review feedback (#75, @jmagly): the design docs + one ts-grammars comment still described the pre-implementation 'fall back to the Python regex' model for non-Python files. The shipped contract is language-scoped: .py uses parseFile; any non-.py file with a missing/unsupported/failed grammar or a parse timeout returns [] (never the Python regex, which would mis-parse brace/class syntax into corrupt blocks). Updated every stale reference in multilang-ingest-spec.md, multilang-ingest-plan.md, and ts-grammars.ts so the docs match the security boundary. Docs/comment only — no code change. * fix(test): make the ReDoS wall-clock guard robust under coverage instrumentation Review feedback (#75, @jmagly): the CI step this PR adds (npm run test:coverage = vitest run --coverage) runs the full suite under V8 coverage, and that overhead pushes the pathological-brace DoS timing guard in local-agent-tool-calling.test.ts:69 over its <300ms bound (320 < 300 fails on CI). Raise the bound to 1000ms — still 2x below the ~2100ms quadratic-ReDoS regression it guards against, so it fails hard if the regression returns, but no longer false-fails under coverage/slow CI. Chose the reviewer's option (a) (robust bound) over scoping test:coverage, so the gate stays a simple full-suite run and no test-file enumeration can rot. Verified: npm run test:coverage is green and the 3 new files stay at 100%. * fix(ingest): cover pointer-returning C functions --------- Co-authored-by: Joseph Magly <1159087+jmagly@users.noreply.github.com> |
||
|
Lyubomir BozhinovandJoseph Magly
|
9cce68e804 |
feat(settings): Universal API Config panel with server-side model fetch (#90) (#101)
* feat(settings): Universal API Config panel with server-side model fetch (#90) Consolidates provider configuration into one "Universal API Config" panel in Settings — provider dropdown (OpenAI-compatible / Anthropic), base URL, API key, model with a live "Fetch models" button, and max context tokens — above the existing per-provider key sections. Backward-compatible by design: the panel reads and writes the SAME state.settings keys the individual sections already use (${provider}Key, selectedModel), so nothing migrates and nothing is lost; the individual sections keep working and stay in sync. - src/config/provider-models.ts (new): listProviderModels queries a provider's own model endpoint (OpenAI-compatible Bearer /models, Anthropic x-api-key /v1/models, OpenRouter public list); resolveModels wraps it fail-open. - POST /api/models (new route): server-side so the browser dodges provider CORS. Binds key+baseUrl by source — a caller-supplied base URL must bring its own key, so the server never lends its configured key to a body-chosen URL. Always responds, failing open to the static AVAILABLE_MODELS list on a missing key, network error, or unknown provider (incl. local-agent) — never hangs. - Model options are built via the DOM API (createElement/textContent), never innerHTML, so untrusted provider-returned model ids cannot inject markup. Scope notes: augment-not-replace (removing the individual sections is deferred to maintainer preference); base URL + context cap are stored and used for model discovery and server/agent flows — in-browser mission routing still uses OpenRouter/Venice, matching the existing "not wired" hints; the "Model Reasoning (Eino)" sub-proposal is a separate follow-up pending a mapping. Verified on macOS: 12 unit tests (RED->GREEN) for the model module, full suite 462 green, and Playwright QA (17 checks) driving the real panel — render, provider switch, persistence across reload, backward-compat, and both the static and live Fetch paths. Refs #90 * fix(settings): harden model list fetch --------- Co-authored-by: Joseph Magly <1159087+jmagly@users.noreply.github.com> |
||
|
Lyubomir Bozhinov
|
5b4ac700ad |
fix(agents): detect local CLIs via well-known install dirs (#83)
A T3MP3ST server launched from Finder, the desktop app, or any non-interactive shell inherits launchd's minimal PATH (/usr/bin:/bin:/usr/sbin:/sbin). The local-agent detector resolved `claude`/`codex`/`hermes` against that PATH via execFile, so an installed, authed CLI in ~/.local/bin (Claude Code's native installer), Homebrew, npm-global, or a version-manager shim (mise/nvm/fnm) threw ENOENT and was reported installed:false — the Settings "Connect Local Agents" checkboxes went dead and it read like a UI bug. Add resolveBin(): pure-fs (no shell) resolution over PATH plus a curated set of well-known install dirs under the real agent home (agentHome(), so a HOME redirect can't hide them), with an executable-file access check and fail-open to the bare name — an unresolved name yields the same ENOENT → installed:false as before, so true-absence detection is unchanged. Route the detector and both spawn sites (runLocalAgent, localAgentChat) through it, so a CLI detected under a minimal PATH also actually spawns. Bound the keychain auth-probe with a 2s timeout so a locked/headless keychain (the same non-interactive launch context) can't hang detection. POSIX/macOS-scoped: resolveBin short-circuits on win32 — Windows resolution (where.exe / .cmd shims) is handled by the complementary open PR #18. Verified on macOS: RED->GREEN unit + integration tests (resolver, detector wiring, and both spawn sites), and manual QA driving the real Web UI under a reproduced minimal PATH (Claude Code flips from "Not installed" to detected). Refs #78 — fixes the macOS/POSIX variant. The original report was a Windows Sandbox environment (PR #18's domain), so this intentionally does not auto-close that issue. |