Files
awesome-copilot/skills/webmcpify/references/heal.md
T
Jonas Tüchler 03fb5fc96e Add the WebMCPify agent skill 🤖🤖🤖 (#2400)
* feat(skills): add webmcpify skill

* style(skills): quote webmcpify description
2026-07-23 10:50:25 -07:00

6.4 KiB

Heal — failure taxonomy → fixes

Work one failed tool at a time. Re-verify after each fix. The triggering verify failure counts as attempt 0; each fix cycle increments attempts. At attempts = 3 → mark skipped with a blocker note (this is an explicit escalation to the human in the final report, not a silent drop) and move on. Never widen the diff, disable a check, or fake a return value to force a pass. Mutating tools: run the manifest cleanup between attempts — retrying a mutation without cleanup duplicates data.

Heal fixes implementations, not contracts. The manifest is the human-approved contract: if the correct fix would change a tool's inputSchema, description, mutating class, annotations, or expect, take it back to the gate as a mini re-approval — never silently edit the manifest to match the code.

Taxonomy

Symptom Likely cause Fix
Tool absent from enumeration Registration is async — the test asserted before registerTool() settled; or registration never ran (bootstrap not reached, view not mounted) or wrong Chrome build/flags FIRST make the test poll (waitForTool) or await toolchange — only if it still fails, trace the registration call; confirm isWebMCPAvailable() in the test env; current Chrome + --enable-features=WebMCP,WebMCPTesting
Whole scope absent A registration in the batch rejected (duplicate name, invalid schema, policy) — the runtime rolls back the entire scope Check console for the onError report; fix the offending tool contract
Tool absent after route change Scope disposed by navigation (over-scoping) Move to static app-level registration unless genuinely view-bound
Declarative tool missing toolname typo, frame without allow="tools", or page sends Origin-Agent-Cluster: ?0 Fix attribute; check Permissions-Policy tools and origin-keying headers
Schema mismatch (declarative) Control lacks name, description not resolvable, unsupported control type in this build Add name/toolparamdescription/label[for]; unsupported controls → switch that form to imperative
Schema mismatch (imperative) Manifest and code drifted Make code match the approved manifest; if the manifest was wrong, that's a contract change — take it back to the gate for re-approval (see above), never silently update it
Assertion compares object to string Enumerated inputSchema is a stringified JSON Schema JSON.parse before comparing (see verify.md)
executeTool returns null unexpectedly The execution navigated (normal for submit-navigating declarative forms) Assert on the post-navigation page instead of the return value
executeTool rejects Schema violation or declarative-validation failure — rejection IS the failure signal for these For invalid-input tests on declarative tools, assert rejection, not an "ERROR:" string
Mutating declarative execution hangs until timeout Chrome fills the form, then pauses the execution awaiting a real submit interaction — awaiting executeTool alone deadlocks Use the concurrent pattern in the spec template: start executeTool unawaited → wait for the agent-filled value → click submit → await. NEVER heal by adding toolautosubmit (ground rule 5)
Backend rejects the harness with 403/CORS despite correct auth The endpoint allow-lists the production Origin (mailers, form gateways) — the localhost harness origin is refused before the tool logic runs, and no local fix exists Verify the live path with the env-gated server-side replay (§Origin-allow-listed endpoints below), only with the production side-effect approval recorded in approval.productionSideEffect (see §Origin-allow-listed endpoints below); without it, mark the live path skipped with a blocker note
Execution times out / canned success while UI still loading Completion event fired before the async work finished, or listener missing/wrong event name Fire tool-completion-<requestId> with { ok, message/error } AFTER awaiting the real work (runtime.md contract)
Returns success but UI unchanged execute() bypassed the real UI path (parallel implementation) Rewrite to call the same handler/store action/endpoint the UI uses
Invalid input resolves successfully (imperative) Missing in-code validation Validate strictly in code; return "ERROR: <what/how to fix>"
Fetch-submitted form: agent gets nothing preventDefault() without respondWith() Add the e.agentInvoked → e.respondWith(promise) bridge
Works manually, fails in Playwright Headless, missing flags, or profile without the flag Headed + flags; persistent context; xvfb-run in CI
401/403 from execute() in test Tool registered outside the authenticated scope, or test session lacks the role in the manifest auth field Role-scope the registration; sign in with the recorded fixture
Flaky: passes alone, fails in suite Shared state between tool executions Isolate test data per tool run (use cleanup); don't reorder tests to hide it

Origin-allow-listed endpoints — the replay pattern

Some production backends (mailers, form gateways) allow-list the production Origin header and refuse everything else — the localhost harness can never exercise the live path directly. When (and only when) the gate approved the real production side effect (approval.productionSideEffect), verify the live path with an env-gated replay: intercept the app's own request in Playwright and re-issue it server-side (Node context — not subject to browser CORS) with the production Origin:

// Env-gated: runs only with WEBMCP_LIVE_MUTATIONS=1 — never default-on in CI.
if (process.env.WEBMCP_LIVE_MUTATIONS === '1') {
  await page.route('**/api/contact', async (route) => {
    const response = await context.request.fetch(route.request(), {
      headers: { ...route.request().headers(), origin: 'https://example.com' }, // the prod Origin
    });
    await route.fulfill({ response });
  });
}

This causes a REAL production side effect. Mark every payload [webmcpify verification], run the manifest cleanup, list the effect in report.md, and never enable the gate by default in CI.

After healing

Re-run verification once for all tools with status integrated or verified (not only the healed ones) — healing one tool can unregister or break another; scope collisions are the classic case. Only then evaluate the exit condition.