Add doublecheck plugin: three-layer verification pipeline for AI output (#978)

* Add doublecheck plugin: three-layer verification pipeline for AI output

Adds a new plugin that helps users verify AI-generated content before
acting on it. Designed for sensitive contexts (legal, medical, financial,
compliance) where hallucinations carry real consequences.

Three verification layers:
- Self-Audit: extracts verifiable claims, checks internal consistency
- Source Verification: web searches per claim, produces URLs for human review
- Adversarial Review: assumes errors exist, checks hallucination patterns

Supports persistent mode (auto-verifies every factual response inline)
and one-shot mode (full report on specific text). Confidence ratings:
VERIFIED, PLAUSIBLE, UNVERIFIED, DISPUTED, FABRICATION RISK.

Includes:
- Skill (skills/doublecheck/) with bundled report template
- Agent (agents/doublecheck.agent.md) for interactive verification
- Plugin package (plugins/doublecheck/) bundling both

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: fix tools YAML format, remove materialized artifacts

- Fix tools frontmatter in agents/doublecheck.agent.md to use standard
  YAML list format instead of flow sequence with trailing comma
- Remove plugins/doublecheck/agents/ and plugins/doublecheck/skills/
  from tracking; these paths are in .gitignore as CI-materialized
  artifacts that should not be committed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Dan Velton
2026-03-11 18:35:43 -07:00
committed by GitHub
parent f8c2b32140
commit eb7d223446
9 changed files with 597 additions and 0 deletions

View File

@@ -113,6 +113,12 @@
"description": "A focused set of prompts, instructions, and a chat mode to help triage incidents and respond quickly with DevOps tools and Azure resources.",
"version": "1.0.0"
},
{
"name": "doublecheck",
"source": "doublecheck",
"description": "Three-layer verification pipeline for AI output. Extracts claims, finds sources, and flags hallucination risks so humans can verify before acting.",
"version": "1.0.0"
},
{
"name": "edge-ai-tasks",
"source": "edge-ai-tasks",