Commit Graph

6 Commits

Author SHA1 Message Date
Aaron Powell 8ae5a99109 Enforce external plugin ref/sha consistency (#2463)
* Enforce external plugin ref/sha consistency

Extract shared ref/sha normalization and consistency checks into eng/lib and reuse them in intake plus quality gate flows.

Add a dedicated ref/sha consistency quality gate surfaced in PR/intake summaries, and add targeted tests for matching and mismatched refs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 6afe21ad-eafa-4c90-a1f2-053dedac7625

* Address review: tree/blob ref errors and PR workflow ref/sha column

- resolveCommitShaAtReadRef: classify rev-parse failure as 'fail'
  instead of 'infra_error' because a successfully-fetched ref that
  doesn't dereference to a commit is a submitter problem, not infra.
- validateRemoteRepository (intake): treat HTTP 422 from the commit
  endpoint as a submitter error; all other non-404 errors remain
  transient warnings requiring maintainer re-run.
- external-plugin-pr-quality-gates.yml: add ref/sha consistency
  column to the per-plugin quality table and failure details block.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6afe21ad-eafa-4c90-a1f2-053dedac7625

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6afe21ad-eafa-4c90-a1f2-053dedac7625
2026-07-29 15:28:13 +10:00
Aaron Powell 8e137c3c09 Harden external plugin PR quality gate rendering and name validation (#2444)
* Harden external plugin PR quality gates

Reference: https://github.com/github/awesome-copilot/pull/2398

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a79923be-c65f-4d51-8fe3-a86e05fd02f1

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot-Session: a79923be-c65f-4d51-8fe3-a86e05fd02f1
2026-07-27 15:57:06 +10:00
Aaron Powell 40665c23b7 Migrate extension plugin materialization to extensions container (#2334)
* Migrate extension plugin materialization layout

Materialize extension plugins into a dedicated extensions/ container, validate the new manifest convention, and bump extension plugin versions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d26008fb-9928-4ba7-b7c8-36f35320f7c1

* Keep extension manifests source-compatible

Restore source extension manifests to "extensions": "." while preserving materialization-time rewrite to "extensions" in distribution output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d26008fb-9928-4ba7-b7c8-36f35320f7c1

* Validate canvas extension layout for external submissions

Add intake and quality-gate checks for canvas-tagged external plugins so they must include extensions/extension.mjs and optional manifest extensions is validated when present.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d26008fb-9928-4ba7-b7c8-36f35320f7c1

* Fix plugin clean extension pass and typo guard text

Declare EXTENSIONS_DIR in clean-materialized-plugins and run extension cleanup once after plugin cleanup. Also normalize misspelled-key detection strings to satisfy spelling checks without changing validation behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d26008fb-9928-4ba7-b7c8-36f35320f7c1

* Proper codespell fix

* Separate canvas structure quality gate status

Track canvas structure as its own gate status and output, include it in aggregate summaries, and enforce Git object types so extensions/ is a tree and extensions/extension.mjs is a blob.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d26008fb-9928-4ba7-b7c8-36f35320f7c1

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-17 18:04:03 +10:00
Aaron Powell 61dda50523 Add version parity checks to external plugin quality gates (#2307)
* Add external plugin version-match gate

Enforce external.json version matching against remote plugin.json for source ref and/or sha in shared quality gates, and surface the new gate status/output in intake and PR workflows.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8aa3e98d-1873-4cab-8866-1b2efd0f24ad

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-16 12:39:32 +10:00
Aaron Powell 28c3a14af4 Switch skill CI validation workflows to vally lint (#2030)
* Switch skill CI checks to vally lint

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Adding Vally to allowed words

* case sensitivity

* Migrate external plugin quality gates from skill-validator to vally lint

Replace the downloaded skill-validator binary with
px @microsoft/vally-cli lint
in the external plugin quality gates pipeline:

- Remove downloadSkillValidator() and SKILL_VALIDATOR_ARCHIVE_URL constant
- Replace uildSkillValidatorArgs() + 
unSkillValidatorGate() with
  uildVallyLintArgs() + 
unVallyLintGate() that run
px vally-cli lint
  per resolved skill directory (falling back to the full plugin root when no
  specific skill paths can be resolved from plugin.json)
- Rename result keys skill_validator_status / skill_validator_output
  to ally_lint_status / ally_lint_output throughout both
  ng/external-plugin-quality-gates.mjs and ng/external-plugin-intake.mjs
- Update PR comment markdown to show 'vally lint' instead of 'skill-validator'
- Update CONTRIBUTING.md prose references accordingly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use @microsoft/vally library directly instead of vally-cli subprocess

Replace the npx-spawned vally-cli process with a direct call to the
@microsoft/vally core library in the external plugin quality gates scripts:

- Add @microsoft/vally as a devDependency in package.json
- Import runLint and LintConsoleReporter from @microsoft/vally
- Replace runVallyLintGate() process spawn with async API call:
  - runLint({ rootPath }) returns structured LintResults
  - LintConsoleReporter with a Writable capture stream collects
    text output without printing to stdout
- Make runExternalPluginQualityGates() async (propagated to
  runExternalPluginPrQualityGates() and both main entry points)
- Use Promise.all in runExternalPluginPrQualityGates() for parallel
  plugin checks
- Fix remaining skill_validator_status reference in pr-quality-gates
  summary string (now vally-lint=...) and YAML workflow table header
- Add 'npm install @microsoft/vally' step to both calling workflows

This removes a layer of indirection (Node -> npx -> CLI -> library)
and replaces it with a direct in-process library call, which is faster,
more reliable, and gives structured access to lint results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-30 14:07:45 +10:00
Aaron Powell a34c98bfbf Automate external plugin update PR quality checks (#2005)
* Add PR quality gates for external plugin updates

Automate external plugin update PR review by running skill-validator and install smoke checks against changed entries in plugins/external.json. Sync PR workflow-state labels and upsert a marker-based status comment with source tree links for each changed plugin.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Tighten external plugin PR workflow permissions

Scope write permissions to the PR synchronization job, keep the quality-gate job read-only, and handle no-op and detection-failure states explicitly. Also fix source tree link encoding for refs, SHAs, and plugin paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix external plugin workflow job steps

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
2026-06-16 15:40:21 +10:00