Commit Graph
2 Commits
Author SHA1 Message Date
Furkan Reha f32d7c320d fix(skills): anchor citations on the named line, and treat repo text as evidence (#2976)
Ports four corrections these two skills received upstream after a second trial
run against a real repository. #2951 merged the snapshot taken before them. The
bundled scripts are already identical to their upstream versions and are not
touched here.

1. Neither skill told the agent that text read out of the audited repository is
   data rather than instruction. These skills exist to read untrusted
   repositories, so a README, a code comment, a commit message or a dependency
   manifest reached the model with no framing -- and a line claiming a file is
   approved, or telling the audit to skip a module, reads exactly like a
   guardrail. Both skills now carry the rule and report such text as a finding
   instead of following it.

2. The citation rule allowed anchors to land beside the symbol rather than on
   it: the blank line above a definition, a decorator, or a line inside a
   multi-line literal. In one trialled file every anchor sat two lines above the
   def it named. The rule is now a single applicable test -- the line you cite
   must literally contain the thing you name, and a cited range must contain it
   on the first line. Quoted text is cited at the line the quoted characters are
   on, because a comment has its own line number and it is usually not the line
   of the code beside it.

3. "Never restate a count without the raw output in front of you" was ignored
   twice in that trial, so the rule flips from prohibition to requirement: any
   number stated must appear under Checks Run next to the command that produced
   it. Unwilling to show the command means describing the pattern rather than
   counting it.

4. Both Related Skills sections said the skill is one of seven and that the
   other five cover the remaining ground. Six, not five. Each section now names
   its sibling in this repository and links the remaining five out.

Front matter is unchanged, so the generated README tables do not move.
2026-09-08 09:48:27 +10:00
Furkan RehaandAaron Powell 5f7e3d0c68 Add test-gap-audit and docs-sync-audit skills (#2951)
* feat(skills): add test-gap-audit and docs-sync-audit

Two repo-agnostic review skills that answer questions the existing testing and
documentation skills do not.

test-gap-audit asks which behaviour is not covered, rather than how to write a
test in a given framework. Given no scope it audits the whole repository,
inventories the testable surfaces, and reports which routes, services, jobs and
contracts have no tests, too few assertions, or only indirect coverage. It
bundles coverage_map.py, which detects the test framework and naming convention,
then matches every source file to tests by name, by mirrored path, and by what
the test files actually import, and ranks the unmatched by risk keyword and size.

docs-sync-audit compares what the docs claim against what the code does. It
bundles docs_drift.py, which checks documented npm scripts and make targets
against the ones that exist, relative Markdown links against the filesystem, and
environment variable names in both directions. It also reports a documented
setting that is read only inside a module nothing imports, which is configuration
that reads as working but cannot take effect.

Both are read-only: they report and do not edit unless asked. Both emit the same
contract, so a finding always carries a P0-P3 severity and a path:line you can
open. Both scripts are Python standard library only, install nothing, and are
accelerators rather than requirements, so each skill still works when the script
cannot run.

The existing testing and docs skills here are framework-specific, which is where
most of the value is. These are the repo-agnostic complement: pytest-coverage
raises coverage inside a pytest project, and this decides where coverage is
missing across a repository regardless of language.

* fix: satisfy codespell and regenerate all generated docs

Two CI failures on the first push.

codespell flagged `testng` and `shouldBe` in coverage_map.py. Both are
legitimate identifiers rather than typos: TestNG is the Java test framework the
script detects by name, and shouldBe is the Kotlin and Scala assertion method
matched by its assertion-detection regex. Added both to ignore-words-list with a
comment each, following the convention already used for the other entries.

validate-readme failed because I had reverted docs/README.agents.md. `npm start`
rewrites a Dynatrace MCP URL there from re-fetched external plugin data, which is
unrelated to these skills, so I had excluded it to keep the diff scoped. That was
wrong: the check regenerates every generated file and compares, so the commit has
to carry whatever the build produces. Restored.

---------

Co-authored-by: Aaron Powell <me@aaron-powell.com>
2026-09-07 14:08:49 +10:00