JuliusBrussee/caveman

What it measures: How tangled the control flow is — methods with many branches are hard to test and change.

Method: Cyclomatic complexity per method (1 + decision points), computed exhaustively across production source; test projects separated by convention. Deterministic.

Maturity: Documented → Verified → Prevented · effective 5.0 / 10 · rule-coverage 100% · ceiling Prevented

17 method(s) exceeded the cyclomatic complexity threshold of 15; the worst was uninstall at 81.

+ 11 more group(s) — more in Appendix A; the complete list is findings.md.

What to do

1. Resolve the 1 uninstall (cyclomatic 81) finding(s) in Cyclomatic Complexity — start with install.js. — One of this dimension's main actionable groups (1 warning-level).
2. Resolve the 1 parseModeChange (cyclomatic 54) finding(s) in Cyclomatic Complexity — start with caveman-parse.js. — One of this dimension's main actionable groups (1 warning-level).
3. Resolve the 1 installOpencode (cyclomatic 48) finding(s) in Cyclomatic Complexity — start with install.js. — One of this dimension's main actionable groups (1 warning-level).
4. Enforce Cyclomatic Complexity in CI to reach Verified (currently Documented). — Hardens enforcement from Documented toward Prevented — provenance only; does not change the score.

Detailed fixes: d1_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: How hard the code is for a person to follow, beyond raw branching.

Method: Cognitive complexity per method (Sonar-style nesting-penalized score), computed exhaustively over production code, excluding test projects. Deterministic.

Maturity: Documented → Verified → Prevented · effective 2.0 / 10 · rule-coverage 100% · ceiling Prevented

29 method(s) exceeded the cognitive complexity threshold of 15; the worst was uninstall at 203.

+ 21 more group(s) — more in Appendix A; the complete list is findings.md.

What to do

1. Resolve the 1 uninstall (cognitive 203) finding(s) in Cognitive Complexity — start with install.js. — One of this dimension's main actionable groups (1 warning-level).
2. Resolve the 1 parseModeChange (cognitive 90) finding(s) in Cognitive Complexity — start with caveman-parse.js. — One of this dimension's main actionable groups (1 warning-level).
3. Resolve the 1 installOpencode (cognitive 84) finding(s) in Cognitive Complexity — start with install.js. — One of this dimension's main actionable groups (1 warning-level).
4. Enforce Cognitive Complexity in CI to reach Verified (currently Documented). — Hardens enforcement from Documented toward Prevented — provenance only; does not change the score.

Detailed fixes: d2_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Over-large classes that try to do too much ("god classes").

Method: God-class detection by line and method-count thresholds per logical type (partial classes unified), filtered for generated code and registration/contract false positives. Deterministic.

Maturity: Documented → Verified → Prevented · effective 2.5 / 10 · rule-coverage 100% · ceiling Prevented

3 god class(es) detected.

What to do

1. Resolve the 3 FileTooLong finding(s) in God Classes — start with caveman-stats.js, e2e.freshinstall.test.mjs, install.js. — One of this dimension's main actionable groups (3 warning-level).
2. Enforce God Classes in CI to reach Verified (currently Documented). — Hardens enforcement from Documented toward Prevented — provenance only; does not change the score.

Detailed fixes: d3_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Copy-pasted code that should be shared instead.

Method: Code duplication via token-stream sliding windows with type-aware normalization (locals masked, type names preserved), density-scored per KLoC of production code. Deterministic.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Verified

0 duplicated block group(s) detected.

✓ On the Gold path — maintain.

Detailed fixes: d4_recommendation.md.

What it measures: Whether volatile projects sit underneath others that depend on them (so their churn ripples upward), and whether project dependencies form cycles. A widely-depended-on but stable shared/kernel project is healthy, not penalised.

Method: Dependency cycles via elementary-DFS over real .csproj references, plus Martin instability (afferent/efferent) per project. Exhaustive over the reference graph, deterministic.

Coverage: Exhaustive · type-level: afferent/efferent coupling + cycles computed over every production type — the population is all types, not a name convention.

Maturity: Documented → Verified → Prevented · effective 6.9 / 10 · rule-coverage 100% · ceiling Prevented

0 projects, 0 dependency cycle(s), 0 unstable depended-on project(s).

What to do

1. Enforce Coupling in CI to reach Verified (currently Documented). — Hardens enforcement from Documented toward Prevented — provenance only; does not change the score.

Detailed fixes: d5_recommendation.md.

What it measures: Whether the test suite has a healthy mix of unit / integration / end-to-end tests.

Method: Test projects classified (Unit/Integration/BDD/E2E) from compiled metadata; test methods counted exhaustively across projects with placement-agnostic disk fallback. Deterministic.

Maturity: Documented → Verified → Prevented · effective 0.0 / 10 · rule-coverage 100% · ceiling Documented

No test projects found.

What to do

1. Resolve the 1 No tests found finding(s) in Test Distribution. — One of this dimension's main actionable groups (1 recommendation-level).

Detailed fixes: d9_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Whether dependencies are current, secure, and not bloated.

Method: Manifest scan via dotnet list package across all projects; worst-signal-per-package deduction (saturating for vulnerabilities, capped-linear for deprecation/outdated) per KLoC. Exhaustive, deterministic.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Verified

0 outdated, 0 vulnerable, 0 deprecated packages.

✓ On the Gold path — maintain.

Detailed fixes: d12_recommendation.md.

What it measures: Whether any secrets (keys, tokens, passwords) have leaked into the code.

Method: In-process native secret scanner (entropy plus signature patterns) across all tracked files; no external tool. A clean result is a measured 10, not no-data zero. Deterministic.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Prevented

Secret scan ran and found no leaked secrets.

✓ On the Gold path — maintain.

Detailed fixes: d13_recommendation.md.

What it measures: Files that change often and are also complex — the riskiest hotspots.

Method: Per production file churn times cyclomatic complexity over a rolling window, computed from git and Roslyn/JS/Razor analysis. Exhaustive, deterministic per commit date.

Maturity: Documented → Verified → Prevented · effective 9.7 / 10 · rule-coverage 100% · ceiling Documented

Top hotspots: src/hooks/caveman-mode-tracker.js (7×35=245); src/hooks/caveman-stats.js (6×31=186); cli/install.js (2×81=162)

✓ On the Gold path — maintain.

Detailed fixes: d15_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Whether knowledge is concentrated in too few people (the "bus factor").

Method: Living knowledge per author via time-decayed commit attribution (6-month half-life, focus weighting) across largest source files. Deterministic, avoids blame's mechanical-refactor false positives.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

No source file's living knowledge is concentrated in a single author.

✓ On the Gold path — maintain.

Detailed fixes: d16_recommendation.md.

What it measures: Whether the project's documentation is clear, complete, and useful.

Method: Judged by language model at low temperature (0.0-0.1) on a deterministic doc sample (READMEs plus first 25 architecture docs), with two-pass stability filtering. Advisory, sampled.

Maturity: Documented → Verified → Prevented · effective Exemplary / 10 · rule-coverage 100% · ceiling Documented

Caveman's README is a polished, multi-page document that opens with an eye-catching 'why use many token when few do trick' hook and a strong value proposition. It includes a star badge, links to every major section (before/after, install, levels, what you get, benchmarks, the whole cave, caveman 2), and a dedicated Evals chapter explaining how it measures real skill compression across three arms with a control arm that isolates skill contribution from generic terseness. The hooks README adds bundled-session-start, user-prompt-submit, and statusline scripts for automatic activation without setup, which is a clear value-add.

✓ On the Gold path — maintain.

Detailed fixes: d19_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Whether names — types, methods, variables — are clear and consistent.

Method: Judged by language model at low temperature (0.0-0.1) on a deterministic random symbol sample (fixed size, not exhaustive), with disclosed confidence band. Advisory, sampled.

Maturity: Documented → Verified → Prevented · effective Exemplary / 10 · rule-coverage 100% · ceiling Verified

0 naming inconsistencies across 0 sampled symbols.

✓ On the Gold path — maintain.

Detailed fixes: d21_recommendation.md.

What it measures: Whether each project is a focused, coherent unit rather than an oversized grab-bag.

Method: Project size overshoot penalties (LoC / public-type count / namespace count, 2-of-3 flag) weighted by log magnitude. Exhaustive across projects, deterministic, LLM-independent.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

No projects to assess.

✓ On the Gold path — maintain.

Detailed fixes: d26_recommendation.md.

What it measures: How far you must trace to follow a call — low indirection and co-located slices read easier.

Method: Call indirection (interface hops, cross-namespace calls, slice-locality scaled) over a sampled set of method invocations, size-aware baseline. Sampled; confidence discounted by symbol-resolution gaps.

Coverage: Slice locality from the first namespace segments, SAMPLED (≤400 methods) — not exhaustive.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

Too little code to assess navigability.

✓ On the Gold path — maintain.

Detailed fixes: d27_recommendation.md.

What it measures: Whether any secrets were ever committed — scanned across the full git history, not just now.

Method: Git-history secret scan via gitleaks detect over full history in an isolated checkout; each match flagged High. Exhaustive; degrades cleanly when tool absent.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

gitleaks scanned the full history AND the current working tree and found no secrets.

✓ On the Gold path — maintain.

Detailed fixes: d28_recommendation.md.

What it measures: Real static-analysis (SAST) findings — likely security bugs in the code, any language.

Method: Polyglot static analysis via semgrep across the repo using the pinned, image-baked p/security-audit + p/owasp-top-ten rulesets (no scan-time registry fetch); severity rules (ERROR/WARNING/INFO) map to a full-band severity-weighted score. Exhaustive, deterministic; degrades on parse failure.

Coverage: semgrep pattern rules over all files — exhaustive for the rule set, blind to classes of bug without a rule (clean = no rule matched).

Maturity: Documented → Verified → Prevented · effective 2.0 / 10 · rule-coverage 100% · ceiling Documented

10 finding(s): 0 critical, 9 high, 1 medium, 0 low.

What to do

1. Resolve the 9 High finding(s) in Static Analysis (SAST) — start with ci.yml (5), install.js (3), sync-skill.yml. — One of this dimension's main actionable groups (9 issue-level).
2. Resolve the 1 Medium finding(s) in Static Analysis (SAST) — start with llm_run.py. — One of this dimension's main actionable groups (1 warning-level).

Detailed fixes: d29_recommendation.md · top locations in Appendix A, every location in findings.md.

What it measures: Whether JavaScript/npm dependencies have known published vulnerabilities (CVEs) — the npm ecosystem's biggest risk.

Method: JS/npm CVE scan via trivy fs --scanners vuln over JS manifests (package.json/yarn.lock/pnpm-lock/bun.lockb); 0-10 tight normalizer. NotApplicable without JS manifests. Exhaustive, deterministic.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

No known-vulnerable JS/npm dependencies.

✓ On the Gold path — maintain.

Detailed fixes: d33_recommendation.md.

What it measures: Whether anyone still has living knowledge of each file, or it has been orphaned — last understood long ago by someone now gone quiet. The sibling of the bus factor: D16 asks who owns it, D34 asks whether anyone still knows it.

Method: File orphaning as total living-knowledge decay below one focused-commit's worth within a year, computed per-file from the D16 decay model. Exhaustive, deterministic over fixed history.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

Every significant source file has living knowledge — recently and meaningfully worked.

✓ On the Gold path — maintain.

Detailed fixes: d34_recommendation.md.

What it measures: Whether files that change together actually belong together — pairs that repeatedly co-change in git history despite having no explicit code dependency, surfacing the hidden/logical coupling (and boundaries in the wrong place) a static scan can't see.

Method: Pairwise co-occurrence over the per-commit file sets in git history (production source only — tests and generated dropped): Degree-of-Coupling = shared ÷ min individual revisions, reported above noise floors (each file ≥10 revisions, ≥5 shared commits, ≥50% strength); sweeping commits excluded. Deterministic over fixed history.

Coverage: Population: PRODUCTION source files only — test and generated files are dropped before pairing, so a class co-changing with its own test (trivially ~100%) can't drown the real production↔production coupling. Pairs ranked by Degree-of-Coupling; coupling through a build step, config, or non-source file isn't seen.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

No strong hidden change-coupling between production files.

✓ On the Gold path — maintain.

Detailed fixes: d35_recommendation.md.

What it measures: Whether the repository publishes a coordinated-vulnerability-disclosure policy (SECURITY.md or security.txt) with a reporting contact, so finders know how to report a vulnerability. Presence of a policy file with a contact, not whether the policy is adequate or honoured.

Method: Vulnerability-disclosure policy read deterministically from the repo: a SECURITY.md (root/.github/docs) or .well-known/security.txt / security.txt, regex-checked for a reporting contact (email / URL / mailto). Present + contact → 10; present without a contact → 4; NotApplicable when no policy file exists (it may live off-repo). Detects the policy file's presence + contact, not its adequacy.

Maturity: Documented → Verified → Prevented · effective 10.0 / 10 · rule-coverage 100% · ceiling Documented

A vulnerability-disclosure policy (SECURITY.md) is published with a reporting contact.

✓ On the Gold path — maintain.

Detailed fixes: d37_recommendation.md.

Other · Architecture — Whether the codebase has a recognisable, scale-appropriate structure (a named architectural style, or modular enough for its size) rather than being an ad-hoc ball of mud.

Method: Roslyn plus csproj analysis: architecture style detection (DDD, clean, vertical-slice, CQRS) and structure fitness for repo size. Deterministic.

Maturity · Maturity — Whether the repo and its projects have a README, and whether it's substantive and current.

Method: Filesystem scan: README presence, word count, and headings for depth; git history for staleness. Exhaustive across root and project dirs, deterministic.

What to do

• Add a 'Testing' section to the root README — how to run the test suite.

Maturity · Maturity — Whether key decisions (ADRs) and the high-level shape (C4/diagrams) are written down.

Method: Filesystem scan: ADR folder/naming conventions or content, plus Mermaid/PlantUML/C4/architecture.md discovery. Exhaustive, deterministic.

• No Architecture Decision Records found — decisions aren't captured for future maintainers.

What to do

• Start an ADR log (docs/adr/) recording significant decisions and their rationale.

Maturity · Maturity — Whether the repo is organised deliberately — src/test separation and consistent project naming.

Method: Filesystem scan: src/test folder separation and namespace-prefix consistency (majority RootNamespace agreement). Exhaustive across projects, deterministic.

Maturity · Maturity — Whether the README actually describes the code that exists (LLM-judged, advisory).

Method: Judged by language model at low temperature: README accuracy versus actual projects, within a disclosed tolerance. Advisory, not a measured number.

• README claims '30+ agents' but INSTALL.md lists only Claude Code and one Gemini CLI extension
• claims '65% output tokens saved on prose replies' but the benchmarks table shows 87% for chat-style prompts

What to do

• Reconcile the README with reality: README claims '30+ agents' but INSTALL.md lists only Claude Code and one Gemini CLI extension; claims '65% output tokens saved on prose replies' but the benchmarks table shows 87% for chat-style prompts.

Readiness · Readiness — Whether an automated pipeline builds and tests every change.

Method: Filesystem scan: CI workflow files (.github/workflows, .gitlab-ci.yml, etc.) for build and test stages. Exhaustive, deterministic.

Readiness · Readiness — Whether the code is diagnosable in production — structured logging, tracing/metrics, health checks.

Method: Filesystem/Roslyn scan: structured-logging frameworks (Serilog, NLog), OpenTelemetry, and health-check endpoint patterns. Exhaustive, deterministic.

• No ILogger/Serilog usage found — production issues will be hard to diagnose.

What to do

• Adopt ILogger (or Serilog) and log at meaningful points across the projects.
• Consider OpenTelemetry tracing/metrics and a health-check endpoint for operability.

Readiness · Readiness — Whether SAST, secret/dependency scanning and performance benchmarking are wired in (presence, not runtime).

Method: Filesystem/Roslyn scan: CodeQL, Dependabot, secret-scanning, and BenchmarkDotNet presence in pipelines and projects. Exhaustive, deterministic.

• No static application security testing (CodeQL / security analyzers / codehealth) detected.

What to do

• Add a SAST step (e.g. CodeQL) or a security analyzer package.
• Enable Dependabot/Renovate or a dependency-review gate.
• Add gitleaks/trufflehog in CI to block PRs that introduce committed secrets.

React / JS · Code Health — How much of the frontend is typed TypeScript vs untyped JavaScript.

Method: Frontend file inventory: the share of typed TypeScript vs untyped JavaScript across the source tree. Deterministic, exhaustive over frontend files.

• 0 typed · 32 plain JS

What to do

• Migrate the remaining .js/.jsx files to TypeScript.

React / JS · Code Health — Copy-pasted token-identical blocks across the frontend (the D4 clone algorithm over JS/TS tokens, D-386).

Method: Copy-pasted token-identical blocks across the frontend (the D4 clone algorithm run over JS/TS tokens). Deterministic.

• src/hooks/caveman-config.js:116 · src/hooks/caveman-config.js:276 — caveman-config.js:116
• cli/install.js:198 · src/tools/caveman-init.js:45 — install.js:198
• src/plugins/opencode/plugin.js:45 · src/plugins/opencode/plugin.js:77 — plugin.js:45
• tests/installer/e2e.freshinstall.test.mjs:195 · tests/test_mode_tracker_stdin.js:160 — e2e.freshinstall.test.mjs:195
• tests/installer/opencode-agent.test.mjs:49 · tests/installer/opencode-agent.test.mjs:69 — opencode-agent.test.mjs:49
• tests/installer/e2e.freshinstall.test.mjs:234 · tests/installer/e2e.freshinstall.test.mjs:344 · tests/installer/e2e.freshinstall.test.mjs:384 · tests/installer/e2e.freshinstall.test.mjs:402 — e2e.freshinstall.test.mjs:234
• tests/installer/e2e.freshinstall.test.mjs:429 · tests/installer/e2e.freshinstall.test.mjs:449 · tests/installer/e2e.freshinstall.test.mjs:664 — e2e.freshinstall.test.mjs:429
• tests/test_caveman_parse.js:128 · tests/test_caveman_parse.js:150 — test_caveman_parse.js:128

What to do

• Extract the duplicated blocks into shared functions/components.

React / JS · Architecture — Conformance to the detected frontend architecture layout (feature-sliced / layered src) plus cross-package deep-import rules (D-386).

Method: Conformance to the detected frontend layout (feature-sliced / layered src) plus cross-package deep-import rules, over the module graph. Deterministic.

• tests/test_cavecrew_model_overrides.js:13 reaches into another package with a relative path (../src/hooks/cavecrew-model-overrides) — import the package by name instead. — test_cavecrew_model_overrides.js:13
• tests/test_caveman_parse.js:16 reaches into another package with a relative path (../src/hooks/caveman-parse) — import the package by name instead. — test_caveman_parse.js:16
• tests/test_repo_local_config.js:21 reaches into another package with a relative path (../src/hooks/caveman-config) — import the package by name instead. — test_repo_local_config.js:21
• tests/test_symlink_flag.js:13 reaches into another package with a relative path (../src/hooks/caveman-config) — import the package by name instead. — test_symlink_flag.js:13

What to do

• Fix the listed violations: import through public entries (package exports / slice index), never reach into another layer or package's internals.

React / JS · Code Health — Per-function cyclomatic/cognitive complexity from the token-level function scanner (D-386) — real branching, not a regex heuristic.

Method: Per-function cyclomatic/cognitive complexity from a token-level function scanner (real branching, not a regex heuristic), computed over every frontend function. Deterministic.

• Branch-heavy code is where defects cluster — extract decisions into smaller functions. (×8) — install.js:1184, caveman-parse.js:55, install.js:711, …

What to do

• Break down the listed branch-heavy functions; aim P95 cyclomatic ≤ 5.

React / JS · Code Health — How many components/modules exceed the large-file threshold.

Method: Components/modules exceeding the large-file threshold, counted exhaustively across the frontend source tree. Deterministic.

• 3 file(s) over 400 lines

What to do

• Split the oversized components into smaller, focused ones.

React / JS · Readiness — Static test reachability (D-386): the share of production files reachable from any test via the import graph — measured without running anything.

Method: Static test reachability: the share of production files reachable from any test via the import graph — measured without running anything. Deterministic.

• No test imports this module directly or transitively — its behavior is unverified. (×8) — install.js, caveman-stats.js, openclaw.js, …

What to do

• Add tests that import the unreached modules (directly or through their public entry).

React / JS · Readiness — Whether the project wires up test, lint and typecheck — detected from each package.json script's COMMAND (eslint / tsc / vitest / jest / playwright), not just its name, and corroborated against CI-workflow invocations so a tool run only in CI still counts.

Method: package.json scanned for test/lint/typecheck script wiring. Deterministic presence check.

• test ✓ · lint ✗ · typecheck ✗

What to do

• Add the missing tooling (eslint, tsc) as package.json scripts and run them in CI.

React / JS · Code Health — Files unreachable from every application/tooling/test entry point, and exports nothing imports (module-graph reachability, D-386).

Method: Dead code: files unreachable from every application/tooling/test entry point plus exports nothing imports, via module-graph reachability. Deterministic, exhaustive over the import graph.

• 6 file(s) (~1648 LoC) were excluded from dead-code analysis — declare main/module/exports or a conventional entry (src/index.*, an index.html script) so reachability can see this package. — hooks
• Unreachable from the 4 application, 0 tooling and 17 test entry point(s) detected in this repo. Gate removals on your build/type-check — an undetected custom entry would make these reachable.
• no import path from any entry point (4 application, 0 tooling, 17 test roots considered) — caveman-init.js

What to do

• Delete the dead files and unused exports — every line is maintenance cost and rebuild-estimate inflation with zero runtime value.

React / JS · Readiness — npm dependency truthfulness (D-386): unused dependencies, imports not declared anywhere, and type-/test-only packages shipped as production deps.

Method: npm dependency truthfulness: unused dependencies, imports declared nowhere, and type-/test-only packages shipped as production deps — from the manifest + import graph. Deterministic.

React / JS · Architecture — Import cycles in the module graph (D-386) — files that can only be understood and changed together.

Method: Import cycles in the module graph, detected exhaustively over JS/TS imports (the same cycle detection as the .NET coupling dimension). Deterministic.

Other · Code Health — Whether the code avoids sync-over-async (deadlock-prone blocking on tasks) and async void.

Method: Roslyn syntax scan: async methods scanned for .Wait()/.GetAwaiter().GetResult() and async-void outside event handlers. Deterministic, hard fact per invocation.

Other · Code Health — Whether exceptions are handled rather than silently swallowed or rethrown with lost stack traces.

Method: Roslyn syntax scan: every catch clause counted; empty catches and bare rethrows flagged. Population is all catch clauses, not estimated. Deterministic, hard fact.

Other · Code Health — Whether log calls use message templates (queryable) rather than interpolated strings.

Method: Roslyn syntax scan: every log call-site counted; interpolated-string first-argument violations flagged. Population is all log calls, not estimated. Deterministic.