Files
awesome-copilot/skills/webmcpify/references/heal.md
T
Jonas Tüchler 03fb5fc96e Add the WebMCPify agent skill 🤖🤖🤖 (#2400)
* feat(skills): add webmcpify skill

* style(skills): quote webmcpify description
2026-07-23 10:50:25 -07:00

70 lines
6.4 KiB
Markdown

# Heal — failure taxonomy → fixes
Work one failed tool at a time. Re-verify after each fix. The triggering verify
failure counts as attempt 0; each fix cycle increments `attempts`. At `attempts`
= 3 → mark `skipped` with a blocker note (this is an explicit escalation to the
human in the final report, not a silent drop) and move on. **Never** widen the
diff, disable a check, or fake a return value to force a pass. **Mutating
tools:** run the manifest `cleanup` between attempts — retrying a mutation
without cleanup duplicates data.
**Heal fixes implementations, not contracts.** The manifest is the
human-approved contract: if the correct fix would change a tool's `inputSchema`,
`description`, `mutating` class, `annotations`, or `expect`, take it back to the
gate as a mini re-approval — never silently edit the manifest to match the code.
## Taxonomy
| Symptom | Likely cause | Fix |
|---|---|---|
| Tool absent from enumeration | **Registration is async** — the test asserted before `registerTool()` settled; or registration never ran (bootstrap not reached, view not mounted) or wrong Chrome build/flags | FIRST make the test poll (`waitForTool`) or await `toolchange` — only if it still fails, trace the registration call; confirm `isWebMCPAvailable()` in the test env; current Chrome + `--enable-features=WebMCP,WebMCPTesting` |
| Whole scope absent | A registration in the batch rejected (duplicate name, invalid schema, policy) — the runtime rolls back the entire scope | Check console for the `onError` report; fix the offending tool contract |
| Tool absent after route change | Scope disposed by navigation (over-scoping) | Move to static app-level registration unless genuinely view-bound |
| Declarative tool missing | `toolname` typo, frame without `allow="tools"`, or page sends `Origin-Agent-Cluster: ?0` | Fix attribute; check Permissions-Policy `tools` and origin-keying headers |
| Schema mismatch (declarative) | Control lacks `name`, description not resolvable, unsupported control type in this build | Add `name`/`toolparamdescription`/`label[for]`; unsupported controls → switch that form to imperative |
| Schema mismatch (imperative) | Manifest and code drifted | Make code match the approved manifest; if the manifest was wrong, that's a contract change — take it back to the gate for re-approval (see above), never silently update it |
| Assertion compares object to string | Enumerated `inputSchema` is a stringified JSON Schema | `JSON.parse` before comparing (see `verify.md`) |
| `executeTool` returns `null` unexpectedly | The execution navigated (normal for submit-navigating declarative forms) | Assert on the post-navigation page instead of the return value |
| `executeTool` rejects | Schema violation or declarative-validation failure — rejection IS the failure signal for these | For invalid-input tests on declarative tools, assert rejection, not an `"ERROR:"` string |
| Mutating declarative execution hangs until timeout | Chrome fills the form, then **pauses the execution awaiting a real submit interaction** — awaiting `executeTool` alone deadlocks | Use the concurrent pattern in the spec template: start `executeTool` unawaited → wait for the agent-filled value → click submit → await. **NEVER heal by adding `toolautosubmit`** (ground rule 5) |
| Backend rejects the harness with 403/CORS despite correct auth | The endpoint **allow-lists the production `Origin`** (mailers, form gateways) — the localhost harness origin is refused before the tool logic runs, and no local fix exists | Verify the live path with the env-gated server-side replay (§Origin-allow-listed endpoints below), only with the production side-effect approval recorded in `approval.productionSideEffect` (see §Origin-allow-listed endpoints below); without it, mark the live path `skipped` with a blocker note |
| Execution times out / canned success while UI still loading | Completion event fired before the async work finished, or listener missing/wrong event name | Fire `tool-completion-<requestId>` with `{ ok, message/error }` AFTER awaiting the real work (`runtime.md` contract) |
| Returns success but UI unchanged | `execute()` bypassed the real UI path (parallel implementation) | Rewrite to call the same handler/store action/endpoint the UI uses |
| Invalid input resolves successfully (imperative) | Missing in-code validation | Validate strictly in code; return `"ERROR: <what/how to fix>"` |
| Fetch-submitted form: agent gets nothing | `preventDefault()` without `respondWith()` | Add the `e.agentInvoked → e.respondWith(promise)` bridge |
| Works manually, fails in Playwright | Headless, missing flags, or profile without the flag | Headed + flags; persistent context; `xvfb-run` in CI |
| 401/403 from `execute()` in test | Tool registered outside the authenticated scope, or test session lacks the role in the manifest `auth` field | Role-scope the registration; sign in with the recorded fixture |
| Flaky: passes alone, fails in suite | Shared state between tool executions | Isolate test data per tool run (use `cleanup`); don't reorder tests to hide it |
## Origin-allow-listed endpoints — the replay pattern
Some production backends (mailers, form gateways) allow-list the production
`Origin` header and refuse everything else — the localhost harness can never
exercise the live path directly. When (and only when) the gate approved the real
production side effect (`approval.productionSideEffect`), verify the live path
with an env-gated replay: intercept the app's own request in Playwright and
re-issue it server-side (Node context — not subject to browser CORS) with the
production `Origin`:
```ts
// Env-gated: runs only with WEBMCP_LIVE_MUTATIONS=1 — never default-on in CI.
if (process.env.WEBMCP_LIVE_MUTATIONS === '1') {
await page.route('**/api/contact', async (route) => {
const response = await context.request.fetch(route.request(), {
headers: { ...route.request().headers(), origin: 'https://example.com' }, // the prod Origin
});
await route.fulfill({ response });
});
}
```
This causes a REAL production side effect. Mark every payload
`[webmcpify verification]`, run the manifest `cleanup`, list the effect in
`report.md`, and never enable the gate by default in CI.
## After healing
Re-run verification once for **all** tools with status `integrated` or `verified`
(not only the healed ones) — healing one tool can unregister or break another;
scope collisions are the classic case. Only then evaluate the exit condition.