mirror of
https://github.com/github/awesome-copilot.git
synced 2026-03-19 07:35:17 +00:00
GHAS Pack - Agent Skills for GitHub Advanced Security - Includes Dependabot, CodeQL, and Secret Scanning (#1049)
* feat: add dependabot skill * feat: add codeql skill * feat: add secret-scanning skill * feat: run start and update docs * fix: replace deprecated @dependabot merge example with native auto-merge guidance The usage example still showed @dependabot merge despite the Jan 2026 deprecation. Replaced with gh pr merge --auto and GitHub UI auto-merge. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
170
skills/codeql/references/alert-management.md
Normal file
170
skills/codeql/references/alert-management.md
Normal file
@@ -0,0 +1,170 @@
|
||||
# CodeQL Alert Management Reference
|
||||
|
||||
Guide for understanding, triaging, dismissing, and resolving code scanning alerts generated by CodeQL.
|
||||
|
||||
## Alert Severity Levels
|
||||
|
||||
### Standard Severity
|
||||
|
||||
All code scanning alerts have one of these severity levels:
|
||||
|
||||
| Level | Description |
|
||||
|---|---|
|
||||
| `Error` | High-confidence, high-impact issues that should be fixed |
|
||||
| `Warning` | Moderate-confidence or moderate-impact issues |
|
||||
| `Note` | Low-confidence or informational findings |
|
||||
|
||||
### Security Severity
|
||||
|
||||
Security alerts additionally have a security severity derived from CVSS scores:
|
||||
|
||||
| Level | CVSS Score Range | Description |
|
||||
|---|---|---|
|
||||
| `Critical` | > 9.0 | Severe vulnerabilities requiring immediate attention |
|
||||
| `High` | 7.0 – 8.9 | Significant vulnerabilities that should be prioritized |
|
||||
| `Medium` | 4.0 – 6.9 | Moderate vulnerabilities to address in normal workflow |
|
||||
| `Low` | 0.1 – 3.9 | Minor issues with limited security impact |
|
||||
|
||||
When a security severity is present, it takes precedence over the standard severity for display and sorting.
|
||||
|
||||
### How Security Severity Is Calculated
|
||||
|
||||
For each CodeQL security query added to the Default or Extended suite:
|
||||
1. All CVEs matching the query's CWE tags are identified
|
||||
2. The 75th percentile of CVSS scores for those CVEs is calculated
|
||||
3. That score becomes the query's security severity
|
||||
4. The numerical score maps to Critical/High/Medium/Low per CVSS definitions
|
||||
|
||||
## Alert Labels
|
||||
|
||||
Alerts in non-application code receive category labels:
|
||||
|
||||
| Label | Description |
|
||||
|---|---|
|
||||
| **Generated** | Code generated by the build process |
|
||||
| **Test** | Test code (detected by file path) |
|
||||
| **Library** | Library or third-party code |
|
||||
| **Documentation** | Documentation files |
|
||||
|
||||
These labels are assigned automatically based on file paths. They cannot be manually overridden.
|
||||
|
||||
## Alert Triage in Pull Requests
|
||||
|
||||
### How PR Alerts Work
|
||||
|
||||
- Alerts appear as annotations in the **Conversation** tab and **Files changed** tab
|
||||
- The **Code scanning results** check summarizes all findings
|
||||
- Alerts only appear in a PR if ALL identified lines exist in the PR diff
|
||||
- New alerts on changed lines are shown; pre-existing alerts are not
|
||||
|
||||
### PR Check Failure Behavior
|
||||
|
||||
By default, the check fails if alerts have severity of `error`, `critical`, or `high`. Override this threshold via repository Settings → Rules → Rulesets → Code scanning.
|
||||
|
||||
### Merge Protection
|
||||
|
||||
Configure rulesets to block PR merging when:
|
||||
- A required tool finds alerts matching the severity threshold
|
||||
- A required tool's analysis is still in progress
|
||||
- A required tool is not configured for the repository
|
||||
|
||||
## Copilot Autofix
|
||||
|
||||
GitHub Copilot Autofix automatically generates fix suggestions for CodeQL alerts in pull requests.
|
||||
|
||||
### Availability
|
||||
- Free for all public repositories
|
||||
- Available for private repos with GitHub Code Security license
|
||||
- No Copilot subscription required
|
||||
- Supports a subset of CodeQL queries (not all)
|
||||
|
||||
### How It Works
|
||||
1. Code scanning detects an alert in a PR
|
||||
2. Alert information is sent to the LLM for analysis
|
||||
3. Fix suggestions are posted as PR comments with inline code changes
|
||||
4. Developers review, edit, and commit the suggested fix
|
||||
|
||||
### Using Autofix Suggestions
|
||||
- Click **Edit** to apply the fix directly on GitHub or via GitHub CLI
|
||||
- Use **View autofix patch** to apply locally
|
||||
- Always review and test the fix before committing
|
||||
- The fix may include changes to files not in the original PR diff (e.g., adding a dependency to `package.json`)
|
||||
|
||||
### Dismissing Autofix
|
||||
Click **Dismiss suggestion** on the comment to reject a suggestion.
|
||||
|
||||
## Dismissing Alerts
|
||||
|
||||
### When to Dismiss
|
||||
|
||||
Dismiss alerts when:
|
||||
- The finding is a false positive (code uses a pattern CodeQL doesn't recognize as safe)
|
||||
- The code is used only for testing and risk is acceptable
|
||||
- The effort to fix is greater than the benefit
|
||||
|
||||
### Dismissal Reasons
|
||||
|
||||
Choose the appropriate reason — it affects whether the query continues running:
|
||||
|
||||
| Reason | When to Use |
|
||||
|---|---|
|
||||
| **False positive** | The alert is incorrect; the code is actually safe |
|
||||
| **Won't fix** | The risk is accepted or the code is being deprecated |
|
||||
| **Used in tests** | The vulnerable pattern is only in test code |
|
||||
|
||||
### Dismissal Comments
|
||||
- Add a comment explaining the dismissal rationale
|
||||
- Comments are stored in the alert timeline for audit/compliance
|
||||
- Accessible via REST API at `alerts/{alert_number}` → `dismissed_comment`
|
||||
|
||||
### Contributing Improvements
|
||||
For false positives from unsupported sanitization libraries, consider contributing to the CodeQL repository to improve analysis accuracy.
|
||||
|
||||
## Resolving Alerts
|
||||
|
||||
### Fix and Re-scan
|
||||
1. Fix the vulnerability in the source code
|
||||
2. Commit and push the changes
|
||||
3. The next code scanning run will verify the fix
|
||||
4. Alert is automatically closed when the fix is confirmed
|
||||
|
||||
### Removing Stale Configurations
|
||||
If alerts persist from old/disabled configurations:
|
||||
1. Navigate to the alert's **Affected branches** section
|
||||
2. Identify stale configurations
|
||||
3. Delete the stale configuration to remove outdated alerts
|
||||
|
||||
## Alert Data Flow
|
||||
|
||||
For `path-problem` queries, alerts include data flow information:
|
||||
|
||||
- **Source** — where untrusted data enters (e.g., user input)
|
||||
- **Sink** — where the data is used unsafely (e.g., SQL query, HTML output)
|
||||
- **Path** — the intermediate steps data takes from source to sink
|
||||
|
||||
Click **Show paths** on alert annotations to visualize the full data flow.
|
||||
|
||||
## Multi-Configuration Alerts
|
||||
|
||||
When multiple code scanning configurations analyze the same file:
|
||||
- The same problem detected by the same query appears as a single alert
|
||||
- The **Affected branches** section shows which configurations found the alert
|
||||
- Different configurations may show different statuses
|
||||
- Re-run out-of-date configurations to synchronize alert statuses
|
||||
|
||||
## Viewing Alerts
|
||||
|
||||
### Repository Security Tab
|
||||
- Navigate to **Security** → **Code scanning alerts**
|
||||
- Filter by: tool, severity, rule, branch, state
|
||||
- Click an alert to see full details, affected branches, and data flow
|
||||
|
||||
### Pull Request Checks
|
||||
- View **Code scanning results** check in the PR
|
||||
- Click **View all branch alerts** for the full alert list
|
||||
- Annotations appear inline in **Files changed**
|
||||
|
||||
### REST API
|
||||
- `GET /repos/{owner}/{repo}/code-scanning/alerts` — list alerts
|
||||
- `GET /repos/{owner}/{repo}/code-scanning/alerts/{alert_number}` — get alert details
|
||||
- `PATCH /repos/{owner}/{repo}/code-scanning/alerts/{alert_number}` — update alert status
|
||||
283
skills/codeql/references/cli-commands.md
Normal file
283
skills/codeql/references/cli-commands.md
Normal file
@@ -0,0 +1,283 @@
|
||||
# CodeQL CLI Command Reference
|
||||
|
||||
Detailed reference for the CodeQL CLI — installation, database creation, analysis, SARIF upload, and CI integration.
|
||||
|
||||
## Installation
|
||||
|
||||
### Download the CodeQL Bundle
|
||||
|
||||
Always download the CodeQL bundle (CLI + precompiled queries) from:
|
||||
**https://github.com/github/codeql-action/releases**
|
||||
|
||||
The bundle includes:
|
||||
- CodeQL CLI product
|
||||
- Compatible queries and libraries from `github/codeql`
|
||||
- Precompiled query plans for faster analysis
|
||||
|
||||
### Platform-Specific Bundles
|
||||
|
||||
| Platform | File |
|
||||
|---|---|
|
||||
| All platforms | `codeql-bundle.tar.zst` |
|
||||
| Linux | `codeql-bundle-linux64.tar.zst` |
|
||||
| macOS | `codeql-bundle-osx64.tar.zst` |
|
||||
| Windows | `codeql-bundle-win64.tar.zst` |
|
||||
|
||||
> `.tar.gz` variants are also available for systems without Zstandard support.
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
# Extract the bundle
|
||||
tar xf codeql-bundle-linux64.tar.zst
|
||||
|
||||
# Add to PATH
|
||||
export PATH="$HOME/codeql:$PATH"
|
||||
|
||||
# Verify installation
|
||||
codeql resolve packs
|
||||
codeql resolve languages
|
||||
```
|
||||
|
||||
`codeql resolve packs` should list available query packs for all supported languages. If packs are missing, verify you downloaded the bundle (not standalone CLI).
|
||||
|
||||
### CI System Setup
|
||||
|
||||
Ensure the full CodeQL bundle contents are available on every CI server:
|
||||
- Copy from a central location and extract on each server, or
|
||||
- Use the GitHub REST API to download the bundle dynamically per run
|
||||
|
||||
## Core Commands
|
||||
|
||||
### `codeql database create`
|
||||
|
||||
Create a CodeQL database from source code.
|
||||
|
||||
```bash
|
||||
# Basic usage (interpreted language)
|
||||
codeql database create <output-dir> \
|
||||
--language=<language> \
|
||||
--source-root=<source-dir>
|
||||
|
||||
# Compiled language with build command
|
||||
codeql database create <output-dir> \
|
||||
--language=java-kotlin \
|
||||
--command='./gradlew build' \
|
||||
--source-root=.
|
||||
|
||||
# Multiple languages (cluster mode)
|
||||
codeql database create <output-dir> \
|
||||
--db-cluster \
|
||||
--language=java,python,javascript-typescript \
|
||||
--command='./build.sh' \
|
||||
--source-root=.
|
||||
```
|
||||
|
||||
**Key flags:**
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `--language=<lang>` | Language to extract (required). Use CodeQL language identifiers. |
|
||||
| `--source-root=<dir>` | Root directory of source code (default: current directory) |
|
||||
| `--command=<cmd>` | Build command for compiled languages |
|
||||
| `--db-cluster` | Create databases for multiple languages in one pass |
|
||||
| `--overwrite` | Overwrite existing database directory |
|
||||
| `--threads=<n>` | Number of threads for extraction (default: 1; use 0 for all available cores) |
|
||||
| `--ram=<mb>` | RAM limit in MB for extraction |
|
||||
|
||||
### `codeql database analyze`
|
||||
|
||||
Run queries against a CodeQL database and produce SARIF output.
|
||||
|
||||
```bash
|
||||
codeql database analyze <database-dir> \
|
||||
<query-suite-or-pack> \
|
||||
--format=sarif-latest \
|
||||
--sarif-category=<category> \
|
||||
--output=<output-file>
|
||||
```
|
||||
|
||||
**Key flags:**
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `--format=sarif-latest` | Output format (use `sarif-latest` for current SARIF v2.1.0) |
|
||||
| `--sarif-category=<cat>` | Category tag for the SARIF results (important for multi-language repos) |
|
||||
| `--output=<file>` | Output file path for SARIF results |
|
||||
| `--threads=<n>` | Number of threads for analysis |
|
||||
| `--ram=<mb>` | RAM limit in MB |
|
||||
| `--sarif-add-file-contents` | Include source file contents in SARIF output |
|
||||
| `--ungroup-results` | Disable result grouping (each occurrence reported separately) |
|
||||
| `--no-download` | Skip downloading query packs (use only locally available packs) |
|
||||
|
||||
**Common query suites:**
|
||||
|
||||
| Suite | Description |
|
||||
|---|---|
|
||||
| `<lang>-code-scanning.qls` | Standard code scanning queries |
|
||||
| `<lang>-security-extended.qls` | Extended security queries |
|
||||
| `<lang>-security-and-quality.qls` | Security + code quality queries |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```bash
|
||||
# JavaScript analysis with extended security
|
||||
codeql database analyze codeql-db/javascript-typescript \
|
||||
javascript-typescript-security-extended.qls \
|
||||
--format=sarif-latest \
|
||||
--sarif-category=javascript \
|
||||
--output=js-results.sarif
|
||||
|
||||
# Java analysis with all available threads
|
||||
codeql database analyze codeql-db/java-kotlin \
|
||||
java-kotlin-code-scanning.qls \
|
||||
--format=sarif-latest \
|
||||
--sarif-category=java \
|
||||
--output=java-results.sarif \
|
||||
--threads=0
|
||||
|
||||
# Include file contents in SARIF
|
||||
codeql database analyze codeql-db \
|
||||
javascript-typescript-code-scanning.qls \
|
||||
--format=sarif-latest \
|
||||
--output=results.sarif \
|
||||
--sarif-add-file-contents
|
||||
```
|
||||
|
||||
### `codeql github upload-results`
|
||||
|
||||
Upload SARIF results to GitHub code scanning.
|
||||
|
||||
```bash
|
||||
codeql github upload-results \
|
||||
--repository=<owner/repo> \
|
||||
--ref=<git-ref> \
|
||||
--commit=<commit-sha> \
|
||||
--sarif=<sarif-file>
|
||||
```
|
||||
|
||||
**Key flags:**
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `--repository=<owner/repo>` | Target GitHub repository |
|
||||
| `--ref=<ref>` | Git ref (e.g., `refs/heads/main`, `refs/pull/42/head`) |
|
||||
| `--commit=<sha>` | Full commit SHA |
|
||||
| `--sarif=<file>` | Path to SARIF file |
|
||||
| `--github-url=<url>` | GitHub instance URL (for GHES; defaults to github.com) |
|
||||
| `--github-auth-stdin` | Read auth token from stdin instead of `GITHUB_TOKEN` env var |
|
||||
|
||||
**Authentication:** Set `GITHUB_TOKEN` environment variable with a token that has `security-events: write` scope, or use `--github-auth-stdin`.
|
||||
|
||||
### `codeql resolve packs`
|
||||
|
||||
List available query packs:
|
||||
|
||||
```bash
|
||||
codeql resolve packs
|
||||
```
|
||||
|
||||
Use to verify installation and diagnose missing packs. Available since CLI v2.19.0 (earlier versions: use `codeql resolve qlpacks`).
|
||||
|
||||
### `codeql resolve languages`
|
||||
|
||||
List supported languages:
|
||||
|
||||
```bash
|
||||
codeql resolve languages
|
||||
```
|
||||
|
||||
Shows which language extractors are available in the current installation.
|
||||
|
||||
### `codeql database bundle`
|
||||
|
||||
Create a relocatable archive of a CodeQL database for sharing or troubleshooting:
|
||||
|
||||
```bash
|
||||
codeql database bundle <database-dir> \
|
||||
--output=<archive-file>
|
||||
```
|
||||
|
||||
Useful for sharing databases with team members or GitHub Support.
|
||||
|
||||
## CLI Server Mode
|
||||
|
||||
### `codeql execute cli-server`
|
||||
|
||||
Run a persistent server to avoid repeated JVM initialization when executing multiple commands:
|
||||
|
||||
```bash
|
||||
codeql execute cli-server [options]
|
||||
```
|
||||
|
||||
**Key flags:**
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `-v, --verbose` | Increase progress messages |
|
||||
| `-q, --quiet` | Decrease progress messages |
|
||||
| `--verbosity=<level>` | Set verbosity: `errors`, `warnings`, `progress`, `progress+`, `progress++`, `progress+++` |
|
||||
| `--logdir=<dir>` | Write detailed logs to directory |
|
||||
| `--common-caches=<dir>` | Location for persistent cached data (default: `~/.codeql`) |
|
||||
| `-J=<opt>` | Pass option to the JVM |
|
||||
|
||||
The server accepts commands via stdin and returns results, keeping the JVM warm between commands. Primarily useful in CI environments running multiple sequential CodeQL commands.
|
||||
|
||||
## CI Integration Pattern
|
||||
|
||||
### Complete CI Script Example
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
REPO="my-org/my-repo"
|
||||
REF="refs/heads/main"
|
||||
COMMIT=$(git rev-parse HEAD)
|
||||
LANGUAGES=("javascript-typescript" "python")
|
||||
|
||||
# Create databases for all languages
|
||||
codeql database create codeql-dbs \
|
||||
--db-cluster \
|
||||
--source-root=. \
|
||||
--language=$(IFS=,; echo "${LANGUAGES[*]}")
|
||||
|
||||
# Analyze each language and upload results
|
||||
for lang in "${LANGUAGES[@]}"; do
|
||||
echo "Analyzing $lang..."
|
||||
|
||||
codeql database analyze "codeql-dbs/$lang" \
|
||||
"${lang}-security-extended.qls" \
|
||||
--format=sarif-latest \
|
||||
--sarif-category="$lang" \
|
||||
--output="${lang}-results.sarif" \
|
||||
--threads=0
|
||||
|
||||
codeql github upload-results \
|
||||
--repository="$REPO" \
|
||||
--ref="$REF" \
|
||||
--commit="$COMMIT" \
|
||||
--sarif="${lang}-results.sarif"
|
||||
|
||||
echo "$lang analysis uploaded."
|
||||
done
|
||||
```
|
||||
|
||||
### External CI Systems
|
||||
|
||||
For CI systems other than GitHub Actions:
|
||||
1. Install the CodeQL bundle on CI runners
|
||||
2. Run `codeql database create` with appropriate build commands
|
||||
3. Run `codeql database analyze` to generate SARIF
|
||||
4. Run `codeql github upload-results` to push results to GitHub
|
||||
5. Set `GITHUB_TOKEN` with `security-events: write` permission
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Purpose |
|
||||
|---|---|
|
||||
| `GITHUB_TOKEN` | Authentication for `github upload-results` |
|
||||
| `CODEQL_EXTRACTOR_<LANG>_OPTION_<KEY>` | Extractor configuration (e.g., `CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_TESTS=true`) |
|
||||
| `CODEQL_EXTRACTOR_CPP_AUTOINSTALL_DEPENDENCIES` | Auto-install C/C++ build dependencies on Ubuntu |
|
||||
| `CODEQL_RAM` | Override default RAM allocation for analysis |
|
||||
| `CODEQL_THREADS` | Override default thread count |
|
||||
284
skills/codeql/references/compiled-languages.md
Normal file
284
skills/codeql/references/compiled-languages.md
Normal file
@@ -0,0 +1,284 @@
|
||||
# CodeQL Build Modes for Compiled Languages
|
||||
|
||||
Detailed reference for how CodeQL handles compiled language analysis, including build modes, autobuild behavior, runner requirements, and hardware specifications.
|
||||
|
||||
## Build Modes Overview
|
||||
|
||||
CodeQL offers three build modes for compiled languages:
|
||||
|
||||
| Mode | Description | When to Use |
|
||||
|---|---|---|
|
||||
| `none` | Analyze source without building. Dependencies inferred heuristically. | Default setup; quick scans; interpreted-like analysis |
|
||||
| `autobuild` | Automatically detect and run the build system. | When `none` produces inaccurate results; when Kotlin code is present |
|
||||
| `manual` | User provides explicit build commands. | Complex build systems; autobuild failures; custom build requirements |
|
||||
|
||||
## C/C++
|
||||
|
||||
### Supported Build Modes
|
||||
`none`, `autobuild`, `manual`
|
||||
|
||||
**Default setup mode:** `none`
|
||||
|
||||
### No Build (`none`)
|
||||
- Infers compilation units through source file extensions
|
||||
- Compilation flags and include paths inferred by inspecting the codebase
|
||||
- No working build command needed
|
||||
|
||||
**Accuracy considerations:**
|
||||
- May be less accurate if code depends heavily on custom macros/defines not in existing headers
|
||||
- May miss accuracy when codebase has many external dependencies
|
||||
|
||||
**Improving accuracy:**
|
||||
- Place custom macros/defines in header files included by source files
|
||||
- Ensure external dependencies (headers) are available in system include directories or workspace
|
||||
- Run extraction on the target platform (e.g., Windows runner for Windows projects)
|
||||
|
||||
### Autobuild
|
||||
|
||||
**Windows autodetection:**
|
||||
1. Invoke `MSBuild.exe` on `.sln` or `.vcxproj` closest to root
|
||||
2. If multiple files at same depth, attempts to build all
|
||||
3. Falls back to build scripts: `build.bat`, `build.cmd`, `build.exe`
|
||||
|
||||
**Linux/macOS autodetection:**
|
||||
1. Look for build system in root directory
|
||||
2. If not found, search subdirectories for unique build system
|
||||
3. Run appropriate configure/build command
|
||||
|
||||
**Supported build systems:** MSBuild, Autoconf, Make, CMake, qmake, Meson, Waf, SCons, Linux Kbuild, build scripts
|
||||
|
||||
### Runner Requirements (C/C++)
|
||||
- **Ubuntu:** `gcc` compiler; may need `clang` or `msvc`. Build tools: `msbuild`, `make`, `cmake`, `bazel`. Utilities: `python`, `perl`, `lex`, `yacc`.
|
||||
- **Auto-install dependencies:** Set `CODEQL_EXTRACTOR_CPP_AUTOINSTALL_DEPENDENCIES=true` (enabled by default on GitHub-hosted; disabled on self-hosted). Requires Ubuntu with passwordless `sudo apt-get`.
|
||||
- **Windows:** `powershell.exe` in PATH
|
||||
|
||||
## C\#
|
||||
|
||||
### Supported Build Modes
|
||||
`none`, `autobuild`, `manual`
|
||||
|
||||
**Default setup mode:** `none`
|
||||
|
||||
### No Build (`none`)
|
||||
- Restores dependencies using heuristics from: `*.csproj`, `*.sln`, `nuget.config`, `packages.config`, `global.json`, `project.assets.json`
|
||||
- Uses private NuGet feeds if configured for the organization
|
||||
- Generates additional source files for accuracy:
|
||||
- Global `using` directives (implicit `using` feature)
|
||||
- ASP.NET Core `.cshtml` → `.cs` conversion
|
||||
|
||||
**Accuracy considerations:**
|
||||
- Requires internet access or private NuGet feed
|
||||
- Multiple versions of same NuGet dependency may cause issues (CodeQL picks newer version)
|
||||
- Multiple .NET framework versions may affect accuracy
|
||||
- Colliding class names cause missing method call targets
|
||||
|
||||
### Autobuild
|
||||
|
||||
**Windows autodetection:**
|
||||
1. `dotnet build` on `.sln` or `.csproj` closest to root
|
||||
2. `MSBuild.exe` on solution/project files
|
||||
3. Build scripts: `build.bat`, `build.cmd`, `build.exe`
|
||||
|
||||
**Linux/macOS autodetection:**
|
||||
1. `dotnet build` on `.sln` or `.csproj` closest to root
|
||||
2. `MSbuild` on solution/project files
|
||||
3. Build scripts: `build`, `build.sh`
|
||||
|
||||
### Injected Compiler Flags (Manual Builds)
|
||||
|
||||
The CodeQL tracer injects these flags into C# compiler invocations:
|
||||
|
||||
| Flag | Purpose |
|
||||
|---|---|
|
||||
| `/p:MvcBuildViews=true` | Precompile ASP.NET MVC views for security analysis |
|
||||
| `/p:UseSharedCompilation=false` | Disable shared compilation server (required for tracer inspection) |
|
||||
| `/p:EmitCompilerGeneratedFiles=true` | Write generated source files to disk for extraction |
|
||||
|
||||
> `/p:EmitCompilerGeneratedFiles=true` may cause issues with legacy projects or `.sqlproj` files.
|
||||
|
||||
### Runner Requirements (C#)
|
||||
- **.NET Core:** .NET SDK (for `dotnet`)
|
||||
- **.NET Framework (Windows):** Microsoft Build Tools + NuGet CLI
|
||||
- **.NET Framework (Linux/macOS):** Mono Runtime (`mono`, `msbuild`, `nuget`)
|
||||
- **`build-mode: none`:** Requires internet access or private NuGet feed
|
||||
|
||||
## Go
|
||||
|
||||
### Supported Build Modes
|
||||
`autobuild`, `manual` (no `none` mode)
|
||||
|
||||
**Default setup mode:** `autobuild`
|
||||
|
||||
### Autobuild
|
||||
|
||||
Autodetection sequence:
|
||||
1. Invoke `make`, `ninja`, `./build`, or `./build.sh` until one succeeds and `go list ./...` works
|
||||
2. If none succeed, look for `go.mod` (`go get`), `Gopkg.toml` (`dep ensure -v`), or `glide.yaml` (`glide install`)
|
||||
3. If no dependency managers found, rearrange directory for `GOPATH` and use `go get`
|
||||
4. Extract all Go code (similar to `go build ./...`)
|
||||
|
||||
**Default setup** automatically detects `go.mod` and installs compatible Go version.
|
||||
|
||||
### Extractor Options
|
||||
|
||||
| Environment Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_TESTS` | `false` | Include `_test.go` files in analysis |
|
||||
| `CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_VENDOR_DIRS` | `false` | Include `vendor/` directories |
|
||||
|
||||
## Java/Kotlin
|
||||
|
||||
### Supported Build Modes
|
||||
- **Java:** `none`, `autobuild`, `manual`
|
||||
- **Kotlin:** `autobuild`, `manual` (no `none` mode)
|
||||
|
||||
**Default setup mode:**
|
||||
- Java only: `none`
|
||||
- Kotlin or Java+Kotlin: `autobuild`
|
||||
|
||||
> If Kotlin code is added to a repo using `none` mode, disable and re-enable default setup to switch to `autobuild`.
|
||||
|
||||
### No Build (`none`) — Java Only
|
||||
- Runs Gradle or Maven for dependency information (not actual build)
|
||||
- Queries each root build file; prefers newer dependency versions on clash
|
||||
- Uses private Maven registries if configured
|
||||
|
||||
**Accuracy considerations:**
|
||||
- Build scripts that can't be queried for dependencies may cause inaccurate guesses
|
||||
- Code generated during normal build process will be missed
|
||||
- Multiple versions of same dependency (CodeQL picks newer)
|
||||
- Multiple JDK versions — CodeQL uses highest found; lower-version files may be partially analyzed
|
||||
- Colliding class names cause missing method call targets
|
||||
|
||||
### Autobuild
|
||||
|
||||
**Autodetection sequence:**
|
||||
1. Search root directory for Gradle, Maven, Ant build files
|
||||
2. Run first found (Gradle preferred over Maven)
|
||||
3. Otherwise, search for build scripts
|
||||
|
||||
**Build systems:** Gradle, Maven, Ant
|
||||
|
||||
### Runner Requirements (Java)
|
||||
- JDK (appropriate version for the project)
|
||||
- Gradle and/or Maven
|
||||
- Internet access or private artifact repository (for `none` mode)
|
||||
|
||||
## Rust
|
||||
|
||||
### Supported Build Modes
|
||||
`none`, `autobuild`, `manual`
|
||||
|
||||
**Default setup mode:** `none`
|
||||
|
||||
## Swift
|
||||
|
||||
### Supported Build Modes
|
||||
`autobuild`, `manual` (no `none` mode)
|
||||
|
||||
**Default setup mode:** `autobuild`
|
||||
|
||||
**Runner requirement:** macOS runners only. Not supported on Actions Runner Controller (ARC) — Linux only.
|
||||
|
||||
> macOS runners are more expensive; consider scanning only the build step to optimize cost.
|
||||
|
||||
## Multi-Language Matrix Examples
|
||||
|
||||
### Mixed Build Modes
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- language: c-cpp
|
||||
build-mode: manual
|
||||
- language: csharp
|
||||
build-mode: autobuild
|
||||
- language: java-kotlin
|
||||
build-mode: none
|
||||
```
|
||||
|
||||
### Conditional Manual Build Steps
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: ${{ matrix.build-mode }}
|
||||
|
||||
- if: matrix.build-mode == 'manual'
|
||||
name: Build C/C++ code
|
||||
run: |
|
||||
make bootstrap
|
||||
make release
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v4
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
```
|
||||
|
||||
### OS-Specific Runners
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- language: javascript-typescript
|
||||
build-mode: none
|
||||
runner: ubuntu-latest
|
||||
- language: swift
|
||||
build-mode: autobuild
|
||||
runner: macos-latest
|
||||
- language: csharp
|
||||
build-mode: autobuild
|
||||
runner: windows-latest
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
runs-on: ${{ matrix.runner }}
|
||||
```
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
### Recommended Specifications (Self-Hosted Runners)
|
||||
|
||||
| Codebase Size | Lines of Code | RAM | CPU Cores | Disk |
|
||||
|---|---|---|---|---|
|
||||
| Small | < 100K | 8 GB+ | 2 | SSD, ≥14 GB |
|
||||
| Medium | 100K – 1M | 16 GB+ | 4–8 | SSD, ≥14 GB |
|
||||
| Large | > 1M | 64 GB+ | 8 | SSD, ≥14 GB |
|
||||
|
||||
### Performance Tips
|
||||
- Use SSD storage for all codebase sizes
|
||||
- Ensure enough disk space for checkout + build + CodeQL data
|
||||
- Use `--threads=0` to use all available CPU cores
|
||||
- Enable dependency caching to reduce analysis time
|
||||
- Consider `none` build mode where accuracy is acceptable — significantly faster than `autobuild`
|
||||
|
||||
## Dependency Caching
|
||||
|
||||
### Advanced Setup Workflows
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: java-kotlin
|
||||
dependency-caching: true
|
||||
```
|
||||
|
||||
| Value | Behavior |
|
||||
|---|---|
|
||||
| `false` / `none` / `off` | Disabled (default for advanced setup) |
|
||||
| `restore` | Restore existing caches only |
|
||||
| `store` | Store new caches only |
|
||||
| `true` / `full` / `on` | Restore and store caches |
|
||||
|
||||
Default setup on GitHub-hosted runners has caching enabled automatically.
|
||||
265
skills/codeql/references/sarif-output.md
Normal file
265
skills/codeql/references/sarif-output.md
Normal file
@@ -0,0 +1,265 @@
|
||||
# CodeQL SARIF Output Reference
|
||||
|
||||
Detailed reference for the SARIF v2.1.0 output produced by CodeQL analysis. Use this when interpreting or processing CodeQL scan results.
|
||||
|
||||
## About SARIF
|
||||
|
||||
SARIF (Static Analysis Results Interchange Format) is a standardized JSON format for representing static analysis tool output. CodeQL produces SARIF v2.1.0 (specification: `sarifv2.1.0`).
|
||||
|
||||
- Specification: [OASIS SARIF v2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html)
|
||||
- Schema: [sarif-schema-2.1.0.json](https://docs.oasis-open.org/sarif/sarif/v2.1.0/errata01/os/schemas/sarif-schema-2.1.0.json)
|
||||
- Format type: `sarifv2.1.0` (passed to `--format` flag)
|
||||
|
||||
## Top-Level Structure
|
||||
|
||||
### `sarifLog` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `$schema` | ✅ | Link to the SARIF schema |
|
||||
| `version` | ✅ | SARIF specification version (`"2.1.0"`) |
|
||||
| `runs` | ✅ | Array containing a single `run` object per language |
|
||||
|
||||
### `run` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `tool` | ✅ | Tool information (`toolComponent`) |
|
||||
| `artifacts` | ✅ | Array of artifact objects for every file referenced in a result |
|
||||
| `results` | ✅ | Array of `result` objects |
|
||||
| `newLineSequences` | ✅ | Newline character sequences |
|
||||
| `columnKind` | ✅ | Column counting method |
|
||||
| `properties` | ✅ | Contains `semmle.formatSpecifier` identifying the format |
|
||||
|
||||
## Tool Information
|
||||
|
||||
### `tool` Object
|
||||
|
||||
Contains a single `driver` property.
|
||||
|
||||
### `toolComponent` Object (Driver)
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `name` | ✅ | `"CodeQL command-line toolchain"` |
|
||||
| `organization` | ✅ | `"GitHub"` |
|
||||
| `version` | ✅ | CodeQL release version (e.g., `"2.19.0"`) |
|
||||
| `rules` | ✅ | Array of `reportingDescriptor` objects for available/run rules |
|
||||
|
||||
## Rules
|
||||
|
||||
### `reportingDescriptor` Object (Rule)
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `id` | ✅ | Rule identifier from `@id` query property (e.g., `cpp/unsafe-format-string`). Uses `@opaqueid` if defined. |
|
||||
| `name` | ✅ | Same as `@id` property from the query |
|
||||
| `shortDescription` | ✅ | From `@name` query property |
|
||||
| `fullDescription` | ✅ | From `@description` query property |
|
||||
| `defaultConfiguration` | ❌ | `reportingConfiguration` with `enabled` (true/false) and `level` based on `@severity`. Omitted if no `@severity` specified. |
|
||||
|
||||
### Severity Mapping
|
||||
|
||||
| CodeQL `@severity` | SARIF `level` |
|
||||
|---|---|
|
||||
| `error` | `error` |
|
||||
| `warning` | `warning` |
|
||||
| `recommendation` | `note` |
|
||||
|
||||
## Results
|
||||
|
||||
### `result` Object
|
||||
|
||||
By default, results are grouped by unique message format string and primary location. Two results at the same location with the same message appear as a single result. Disable grouping with `--ungroup-results`.
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `ruleId` | ✅ | Rule identifier (matches `reportingDescriptor.id`) |
|
||||
| `ruleIndex` | ✅ | Index into the `rules` array |
|
||||
| `message` | ✅ | Problem description. May contain SARIF "Message with placeholder" linking to `relatedLocations`. |
|
||||
| `locations` | ✅ | Array containing a single `location` object |
|
||||
| `partialFingerprints` | ✅ | Dictionary with at least `primaryLocationLineHash` for deduplication |
|
||||
| `codeFlows` | ❌ | Populated for `@kind path-problem` queries with one or more `codeFlow` objects |
|
||||
| `relatedLocations` | ❌ | Populated when message has placeholder options; each unique location included once |
|
||||
| `suppressions` | ❌ | If suppressed: single `suppression` object with `@kind: IN_SOURCE`. If not suppressed but other results are: empty array. Otherwise: not set. |
|
||||
|
||||
### Fingerprints
|
||||
|
||||
`partialFingerprints` contains:
|
||||
- `primaryLocationLineHash` — fingerprint based on the context of the primary location
|
||||
|
||||
Used by GitHub to track alerts across commits and avoid duplicate notifications.
|
||||
|
||||
## Locations
|
||||
|
||||
### `location` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `physicalLocation` | ✅ | Physical file location |
|
||||
| `id` | ❌ | Present in `relatedLocations` array |
|
||||
| `message` | ❌ | Present in `relatedLocations` and `threadFlowLocation.location` |
|
||||
|
||||
### `physicalLocation` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `artifactLocation` | ✅ | File reference |
|
||||
| `region` | ❌ | Present for text file locations |
|
||||
| `contextRegion` | ❌ | Present when location has an associated snippet |
|
||||
|
||||
### `region` Object
|
||||
|
||||
Two types of regions may be produced:
|
||||
|
||||
**Line/Column Offset Regions:**
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `startLine` | ✅ | Starting line number |
|
||||
| `startColumn` | ❌ | Omitted if equal to default value of 1 |
|
||||
| `endLine` | ❌ | Omitted if identical to `startLine` |
|
||||
| `endColumn` | ✅ | Ending column number |
|
||||
| `snippet` | ❌ | Source code snippet |
|
||||
|
||||
**Character Offset Regions:**
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `charOffset` | ✅ | Character offset from start of file |
|
||||
| `charLength` | ✅ | Length in characters |
|
||||
| `snippet` | ❌ | Source code snippet |
|
||||
|
||||
> Consumers should handle both region types robustly.
|
||||
|
||||
## Artifacts
|
||||
|
||||
### `artifact` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `location` | ✅ | `artifactLocation` object |
|
||||
| `index` | ✅ | Index of the artifact |
|
||||
| `contents` | ❌ | Populated with `artifactContent` when using `--sarif-add-file-contents` |
|
||||
|
||||
### `artifactLocation` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `uri` | ✅ | File path (relative or absolute) |
|
||||
| `index` | ✅ | Index reference |
|
||||
| `uriBaseId` | ❌ | Set when file is relative to a known abstract location (e.g., source root) |
|
||||
|
||||
## Code Flows (Path Problems)
|
||||
|
||||
For queries of `@kind path-problem`, results include code flow information showing the data flow path.
|
||||
|
||||
### `codeFlow` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `threadFlows` | ✅ | Array of `threadFlow` objects |
|
||||
|
||||
### `threadFlow` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `locations` | ✅ | Array of `threadFlowLocation` objects |
|
||||
|
||||
### `threadFlowLocation` Object
|
||||
|
||||
| Property | Always Generated | Description |
|
||||
|---|:---:|---|
|
||||
| `location` | ✅ | A `location` object for this step in the flow |
|
||||
|
||||
## Automation Details
|
||||
|
||||
The `category` value from `github/codeql-action/analyze` appears as `<run>.automationDetails.id` in the SARIF output.
|
||||
|
||||
Example:
|
||||
```json
|
||||
{
|
||||
"automationDetails": {
|
||||
"id": "/language:javascript-typescript"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Key CLI Flags for SARIF
|
||||
|
||||
| Flag | Effect |
|
||||
|---|---|
|
||||
| `--format=sarif-latest` | Produce SARIF v2.1.0 output |
|
||||
| `--sarif-category=<cat>` | Set `automationDetails.id` for result categorization |
|
||||
| `--sarif-add-file-contents` | Include source file content in `artifact.contents` |
|
||||
| `--ungroup-results` | Report every occurrence separately (no deduplication by location + message) |
|
||||
| `--output=<file>` | Write SARIF to specified file |
|
||||
|
||||
## Third-Party SARIF Support
|
||||
|
||||
When uploading SARIF from non-CodeQL tools, ensure these properties are populated for best results on GitHub.
|
||||
|
||||
### Recommended `reportingDescriptor` Properties
|
||||
|
||||
| Property | Required | Description |
|
||||
|---|:---:|---|
|
||||
| `id` | ✅ | Unique rule identifier |
|
||||
| `name` | ❌ | Rule name (max 255 chars) |
|
||||
| `shortDescription.text` | ✅ | Concise description (max 1024 chars) |
|
||||
| `fullDescription.text` | ✅ | Full description (max 1024 chars) |
|
||||
| `defaultConfiguration.level` | ❌ | Default severity: `note`, `warning`, `error` |
|
||||
| `help.text` | ✅ | Documentation in text format |
|
||||
| `help.markdown` | ❌ | Documentation in Markdown (displayed if available) |
|
||||
| `properties.tags[]` | ❌ | Tags for filtering (e.g., `security`) |
|
||||
| `properties.precision` | ❌ | `very-high`, `high`, `medium`, `low` — affects display ordering |
|
||||
| `properties.problem.severity` | ❌ | Non-security severity: `error`, `warning`, `recommendation` |
|
||||
| `properties.security-severity` | ❌ | Score 0.0–10.0 for security queries. Maps to: >9.0=critical, 7.0–8.9=high, 4.0–6.9=medium, 0.1–3.9=low |
|
||||
|
||||
### Source File Location Requirements
|
||||
|
||||
- Use relative paths (relative to repository root) when possible
|
||||
- Absolute URIs are converted to relative using the source root
|
||||
- Source root can be set via:
|
||||
- `checkout_path` input to `github/codeql-action/analyze`
|
||||
- `checkout_uri` parameter to SARIF upload API
|
||||
- `invocations[0].workingDirectory.uri` in the SARIF file
|
||||
- Consistent file paths are required across runs for fingerprint stability
|
||||
- Symlinked files must use resolved (non-symlink) URIs
|
||||
|
||||
### Fingerprint Requirements
|
||||
|
||||
- `partialFingerprints` with `primaryLocationLineHash` prevents duplicate alerts across commits
|
||||
- CodeQL SARIF automatically includes fingerprints
|
||||
- Third-party SARIF: the `upload-sarif` action computes fingerprints if missing
|
||||
- API uploads without fingerprints may produce duplicate alerts
|
||||
|
||||
## Upload Limits
|
||||
|
||||
### File Size
|
||||
- Maximum: **10 MB** (gzip-compressed)
|
||||
- If too large: reduce query scope, remove `--sarif-add-file-contents`, or split into multiple uploads
|
||||
|
||||
### Object Count Limits
|
||||
|
||||
| Object | Maximum |
|
||||
|---|---|
|
||||
| Runs per file | 20 |
|
||||
| Results per run | 25,000 |
|
||||
| Rules per run | 25,000 |
|
||||
| Tool extensions per run | 100 |
|
||||
| Thread flow locations per result | 10,000 |
|
||||
| Locations per result | 1,000 |
|
||||
| Tags per rule | 20 |
|
||||
|
||||
Files exceeding these limits are rejected. Split analysis across multiple SARIF uploads with different `--sarif-category` values.
|
||||
|
||||
### Validation
|
||||
|
||||
Validate SARIF files before upload using the [Microsoft SARIF validator](https://sarifweb.azurewebsites.net/).
|
||||
|
||||
## Backwards Compatibility
|
||||
|
||||
- Fields marked "always generated" will never be removed in future versions
|
||||
- Fields not always generated may change circumstances under which they appear
|
||||
- New fields may be added without breaking changes
|
||||
- Consumers should be robust to both presence and absence of optional fields
|
||||
259
skills/codeql/references/troubleshooting.md
Normal file
259
skills/codeql/references/troubleshooting.md
Normal file
@@ -0,0 +1,259 @@
|
||||
# CodeQL Troubleshooting Reference
|
||||
|
||||
Comprehensive guide for diagnosing and resolving CodeQL analysis errors, SARIF upload issues, and common configuration problems.
|
||||
|
||||
## Build and Analysis Errors
|
||||
|
||||
### "No source code was seen during the build"
|
||||
|
||||
**Cause:** CodeQL extractor did not find any source files during database creation.
|
||||
|
||||
**Solutions:**
|
||||
- Verify the `--source-root` points to the correct directory
|
||||
- For compiled languages, ensure the build command actually compiles source files
|
||||
- Check that `autobuild` is detecting the correct build system
|
||||
- Switch from `autobuild` to `manual` build mode with explicit build commands
|
||||
- Verify the language specified matches the actual source code language
|
||||
|
||||
### Automatic Build Failed
|
||||
|
||||
**Cause:** `autobuild` could not detect or run the project's build system.
|
||||
|
||||
**Solutions:**
|
||||
- Switch to `build-mode: manual` and provide explicit build commands
|
||||
- Ensure all build dependencies are installed on the runner
|
||||
- For C/C++: verify `gcc`, `make`, `cmake`, or `msbuild` are available
|
||||
- For C#: verify `.NET SDK` or `MSBuild` is installed
|
||||
- For Java: verify `gradle` or `maven` is installed
|
||||
- Check the autobuild logs for the specific detection step that failed
|
||||
|
||||
### C# Compiler Unexpectedly Failing
|
||||
|
||||
**Cause:** The CodeQL tracer injects compiler flags that may conflict with project configuration.
|
||||
|
||||
**Details:** CodeQL injects `/p:EmitCompilerGeneratedFiles=true` which can cause issues with:
|
||||
- Legacy .NET Framework projects
|
||||
- Projects using `.sqlproj` files
|
||||
|
||||
**Solutions:**
|
||||
- Add `<EmitCompilerGeneratedFiles>false</EmitCompilerGeneratedFiles>` to problematic project files
|
||||
- Use `build-mode: none` for C# if build accuracy is acceptable
|
||||
- Exclude problematic projects from the CodeQL analysis
|
||||
|
||||
### Analysis Takes Too Long
|
||||
|
||||
**Cause:** Large codebase, complex queries, or insufficient resources.
|
||||
|
||||
**Solutions:**
|
||||
- Use `build-mode: none` where accuracy is acceptable (significantly faster)
|
||||
- Enable dependency caching: `dependency-caching: true`
|
||||
- Set `timeout-minutes` on the job to prevent hung workflows
|
||||
- Use `--threads=0` (CLI) to use all available CPU cores
|
||||
- Reduce query scope: use `default` suite instead of `security-and-quality`
|
||||
- For self-hosted runners, ensure hardware meets recommendations:
|
||||
- Small (<100K LOC): 8 GB RAM, 2 cores
|
||||
- Medium (100K–1M LOC): 16 GB RAM, 4–8 cores
|
||||
- Large (>1M LOC): 64 GB RAM, 8 cores
|
||||
- Configure larger GitHub-hosted runners if available
|
||||
- Use `paths` in config file to limit analyzed directories
|
||||
|
||||
### CodeQL Scanned Fewer Lines Than Expected
|
||||
|
||||
**Cause:** Build command didn't compile all source files, or `build-mode: none` missed generated code.
|
||||
|
||||
**Solutions:**
|
||||
- Switch from `none` to `autobuild` or `manual` build mode
|
||||
- Ensure the build command compiles the full codebase (not just a subset)
|
||||
- Check the code scanning logs for extraction metrics:
|
||||
- Lines of code in codebase (baseline)
|
||||
- Lines of code extracted
|
||||
- Lines excluding auto-generated files
|
||||
- Verify language detection includes all expected languages
|
||||
|
||||
### Kotlin Detected in No-Build Mode
|
||||
|
||||
**Cause:** Repository uses `build-mode: none` (Java only) but also contains Kotlin code.
|
||||
|
||||
**Solutions:**
|
||||
- Disable default setup and re-enable it (switches to `autobuild`)
|
||||
- Or switch to advanced setup with `build-mode: autobuild` for `java-kotlin`
|
||||
- Kotlin requires a build to be analyzed; `none` mode only works for Java
|
||||
|
||||
## Permission and Access Errors
|
||||
|
||||
### Error: 403 "Resource not accessible by integration"
|
||||
|
||||
**Cause:** `GITHUB_TOKEN` lacks required permissions.
|
||||
|
||||
**Solutions:**
|
||||
- Add explicit permissions to the workflow:
|
||||
```yaml
|
||||
permissions:
|
||||
security-events: write
|
||||
contents: read
|
||||
actions: read
|
||||
```
|
||||
- For Dependabot PRs, use `pull_request_target` instead of `pull_request`
|
||||
- Verify the repository has GitHub Code Security enabled (for private repos)
|
||||
|
||||
### Cannot Enable CodeQL in a Private Repository
|
||||
|
||||
**Cause:** GitHub Code Security is not enabled.
|
||||
|
||||
**Solution:** Enable GitHub Code Security in repository Settings → Advanced Security.
|
||||
|
||||
### Error: "GitHub Code Security or Advanced Security must be enabled"
|
||||
|
||||
**Cause:** Attempting to use code scanning on a private repo without the required license.
|
||||
|
||||
**Solutions:**
|
||||
- Enable GitHub Code Security for the repository
|
||||
- Contact organization admin to enable Advanced Security
|
||||
|
||||
## Configuration Errors
|
||||
|
||||
### Two CodeQL Workflows Running
|
||||
|
||||
**Cause:** Both default setup and a pre-existing `codeql.yml` workflow are active.
|
||||
|
||||
**Solutions:**
|
||||
- Disable default setup if using advanced setup, or
|
||||
- Delete the old workflow file if using default setup
|
||||
- Check repository Settings → Advanced Security for active configurations
|
||||
|
||||
### Some Languages Not Analyzed
|
||||
|
||||
**Cause:** Matrix configuration doesn't include all languages.
|
||||
|
||||
**Solutions:**
|
||||
- Add missing languages to the `matrix.include` array
|
||||
- Verify language identifiers are correct (e.g., `javascript-typescript` not just `javascript`)
|
||||
- Check that each language has an appropriate `build-mode`
|
||||
|
||||
### Unclear What Triggered a Workflow Run
|
||||
|
||||
**Solutions:**
|
||||
- Check the tool status page in repository Settings → Advanced Security
|
||||
- Review workflow run logs for trigger event details
|
||||
- Look at the `on:` triggers in the workflow file
|
||||
|
||||
### Error: "is not a .ql file, .qls file, a directory, or a query pack specification"
|
||||
|
||||
**Cause:** Invalid query or pack reference in the workflow.
|
||||
|
||||
**Solutions:**
|
||||
- Verify query pack names and versions exist
|
||||
- Use correct format: `owner/pack-name@version` or `owner/pack-name:path/to/query.ql`
|
||||
- Run `codeql resolve packs` to verify available packs
|
||||
|
||||
## Resource Errors
|
||||
|
||||
### "Out of disk" or "Out of memory"
|
||||
|
||||
**Cause:** Runner lacks sufficient resources for the analysis.
|
||||
|
||||
**Solutions:**
|
||||
- Use larger GitHub-hosted runners (if available)
|
||||
- For self-hosted runners, increase RAM and disk (SSD with ≥14 GB)
|
||||
- Reduce analysis scope with `paths` configuration
|
||||
- Analyze fewer languages per job
|
||||
- Use `build-mode: none` to reduce resource usage
|
||||
|
||||
### Extraction Errors in Database
|
||||
|
||||
**Cause:** Some source files couldn't be processed by the CodeQL extractor.
|
||||
|
||||
**Solutions:**
|
||||
- Check extraction metrics in workflow logs for error counts
|
||||
- Enable debug logging for detailed extraction diagnostics
|
||||
- Verify source files are syntactically valid
|
||||
- Ensure all build dependencies are available
|
||||
|
||||
## Logging and Debugging
|
||||
|
||||
### Enable Debug Logging
|
||||
|
||||
To get more detailed diagnostic information:
|
||||
|
||||
**GitHub Actions:**
|
||||
1. Re-run the workflow with debug logging enabled
|
||||
2. In the workflow run, click "Re-run jobs" → "Enable debug logging"
|
||||
|
||||
**CodeQL CLI:**
|
||||
```bash
|
||||
codeql database create my-db \
|
||||
--language=javascript-typescript \
|
||||
--verbosity=progress++ \
|
||||
--logdir=codeql-logs
|
||||
```
|
||||
|
||||
**Verbosity levels:** `errors`, `warnings`, `progress`, `progress+`, `progress++`, `progress+++`
|
||||
|
||||
### Code Scanning Log Metrics
|
||||
|
||||
Workflow logs include summary metrics:
|
||||
- **Lines of code in codebase** — baseline before extraction
|
||||
- **Lines of code in CodeQL database** — extracted including external libraries
|
||||
- **Lines excluding auto-generated files** — net analyzed code
|
||||
- **Extraction success/error/warning counts** — per-file extraction results
|
||||
|
||||
### Private Registry Diagnostics
|
||||
|
||||
For `build-mode: none` with private package registries:
|
||||
- Check the "Setup proxy for registries" step in workflow logs
|
||||
- Look for `Credentials loaded for the following registries:` message
|
||||
- Verify organization-level private registry configuration
|
||||
- Ensure internet access is available for dependency resolution
|
||||
|
||||
## SARIF Upload Errors
|
||||
|
||||
### SARIF File Too Large
|
||||
|
||||
**Limit:** 10 MB maximum (gzip-compressed).
|
||||
|
||||
**Solutions:**
|
||||
- Focus on the most important query suites (use `default` instead of `security-and-quality`)
|
||||
- Reduce the number of queries via configuration
|
||||
- Split analysis into multiple jobs with separate SARIF uploads
|
||||
- Remove `--sarif-add-file-contents` flag
|
||||
|
||||
### SARIF Results Exceed Limits
|
||||
|
||||
GitHub enforces limits on SARIF data objects:
|
||||
|
||||
| Object | Maximum |
|
||||
|---|---|
|
||||
| Runs per file | 20 |
|
||||
| Results per run | 25,000 |
|
||||
| Rules per run | 25,000 |
|
||||
| Tool extensions per run | 100 |
|
||||
| Thread flow locations per result | 10,000 |
|
||||
| Location per result | 1,000 |
|
||||
| Tags per rule | 20 |
|
||||
|
||||
**Solutions:**
|
||||
- Reduce query scope to focus on high-impact rules
|
||||
- Split analysis across multiple SARIF uploads with different `--sarif-category`
|
||||
- Disable noisy queries that produce many results
|
||||
|
||||
### SARIF File Invalid
|
||||
|
||||
**Solutions:**
|
||||
- Validate against the [Microsoft SARIF validator](https://sarifweb.azurewebsites.net/)
|
||||
- Ensure `version` is `"2.1.0"` and `$schema` points to the correct schema
|
||||
- Verify required properties (`runs`, `tool.driver`, `results`) are present
|
||||
|
||||
### Upload Rejected: Default Setup Enabled
|
||||
|
||||
**Cause:** Cannot upload CodeQL-generated SARIF when default setup is active.
|
||||
|
||||
**Solutions:**
|
||||
- Disable default setup before uploading via CLI/API
|
||||
- Or switch to using default setup exclusively (no manual uploads)
|
||||
|
||||
### Missing Authentication Token
|
||||
|
||||
**Solutions:**
|
||||
- Set `GITHUB_TOKEN` environment variable with `security-events: write` scope
|
||||
- Or use `--github-auth-stdin` to pipe the token
|
||||
- For GitHub Actions: the token is automatically available via `${{ secrets.GITHUB_TOKEN }}`
|
||||
398
skills/codeql/references/workflow-configuration.md
Normal file
398
skills/codeql/references/workflow-configuration.md
Normal file
@@ -0,0 +1,398 @@
|
||||
# CodeQL Workflow Configuration Reference
|
||||
|
||||
Detailed reference for configuring CodeQL analysis via GitHub Actions workflows. This supplements the procedural guidance in SKILL.md.
|
||||
|
||||
## Trigger Configuration
|
||||
|
||||
### Push Trigger
|
||||
|
||||
Scan on every push to specified branches:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
push:
|
||||
branches: [main, protected]
|
||||
```
|
||||
|
||||
- Code scanning is triggered on every push to the listed branches
|
||||
- The workflow must exist on the target branch for scanning to activate
|
||||
- Results appear in the repository Security tab
|
||||
- When push results map to an open PR, alerts also appear as PR annotations
|
||||
|
||||
### Pull Request Trigger
|
||||
|
||||
Scan merge commits of pull requests:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
```
|
||||
|
||||
- Scans the PR's merge commit (not the head commit) for more accurate results
|
||||
- For private fork PRs, enable "Run workflows from fork pull requests" in repository settings
|
||||
- Results appear as PR check annotations
|
||||
|
||||
### Schedule Trigger
|
||||
|
||||
Periodic scans on the default branch:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
schedule:
|
||||
- cron: '20 14 * * 1' # Monday 14:20 UTC
|
||||
```
|
||||
|
||||
- Only triggers if the workflow file exists on the default branch
|
||||
- Catches newly discovered vulnerabilities even without active development
|
||||
|
||||
### Merge Group Trigger
|
||||
|
||||
Required when using merge queues:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
merge_group:
|
||||
```
|
||||
|
||||
### Path Filtering
|
||||
|
||||
Control when the workflow runs based on changed files:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- '**/*.md'
|
||||
- '**/*.txt'
|
||||
- 'docs/**'
|
||||
```
|
||||
|
||||
Or use `paths` to only trigger on specific directories:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'src/**'
|
||||
- 'apps/**'
|
||||
```
|
||||
|
||||
> **Important:** `paths-ignore` and `paths` control whether the workflow runs. When the workflow does run, it analyzes ALL changed files in the PR (including those matched by `paths-ignore`), unless files are excluded via the CodeQL configuration file's `paths-ignore`.
|
||||
|
||||
### Workflow Dispatch (Manual Trigger)
|
||||
|
||||
```yaml
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
language:
|
||||
description: 'Language to analyze'
|
||||
required: true
|
||||
default: 'javascript-typescript'
|
||||
```
|
||||
|
||||
## Runner and OS Configuration
|
||||
|
||||
### GitHub-Hosted Runners
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
analyze:
|
||||
runs-on: ubuntu-latest # Also: windows-latest, macos-latest
|
||||
```
|
||||
|
||||
- `ubuntu-latest` — most common, recommended for most languages
|
||||
- `macos-latest` — required for Swift analysis
|
||||
- `windows-latest` — required for some C/C++ and C# projects using MSBuild
|
||||
|
||||
### Self-Hosted Runners
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
analyze:
|
||||
runs-on: [self-hosted, ubuntu-latest]
|
||||
```
|
||||
|
||||
Requirements for self-hosted runners:
|
||||
- Git must be in the PATH
|
||||
- SSD with ≥14 GB disk space recommended
|
||||
- See hardware requirements table in SKILL.md
|
||||
|
||||
### Timeout Configuration
|
||||
|
||||
Prevent hung workflows:
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
analyze:
|
||||
timeout-minutes: 120
|
||||
```
|
||||
|
||||
## Language and Build Mode Matrix
|
||||
|
||||
### Standard Matrix Pattern
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- language: javascript-typescript
|
||||
build-mode: none
|
||||
- language: python
|
||||
build-mode: none
|
||||
- language: java-kotlin
|
||||
build-mode: none
|
||||
- language: c-cpp
|
||||
build-mode: autobuild
|
||||
```
|
||||
|
||||
### Multi-Language Repository with Mixed Build Modes
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- language: c-cpp
|
||||
build-mode: manual
|
||||
- language: csharp
|
||||
build-mode: autobuild
|
||||
- language: java-kotlin
|
||||
build-mode: none
|
||||
```
|
||||
|
||||
### Build Mode Summary
|
||||
|
||||
| Language | `none` | `autobuild` | `manual` | Default Setup Mode |
|
||||
|---|:---:|:---:|:---:|---|
|
||||
| C/C++ | ✅ | ✅ | ✅ | `none` |
|
||||
| C# | ✅ | ✅ | ✅ | `none` |
|
||||
| Go | ❌ | ✅ | ✅ | `autobuild` |
|
||||
| Java | ✅ | ✅ | ✅ | `none` |
|
||||
| Kotlin | ❌ | ✅ | ✅ | `autobuild` |
|
||||
| Python | ✅ | ❌ | ❌ | `none` |
|
||||
| Ruby | ✅ | ❌ | ❌ | `none` |
|
||||
| Rust | ✅ | ✅ | ✅ | `none` |
|
||||
| Swift | ❌ | ✅ | ✅ | `autobuild` |
|
||||
| JavaScript/TypeScript | ✅ | ❌ | ❌ | `none` |
|
||||
| GitHub Actions | ✅ | ❌ | ❌ | `none` |
|
||||
|
||||
## CodeQL Database Location
|
||||
|
||||
Override the default database location:
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
db-location: '${{ github.runner_temp }}/my_location'
|
||||
```
|
||||
|
||||
- Default: `${{ github.runner_temp }}/codeql_databases`
|
||||
- Path must be writable and either not exist or be an empty directory
|
||||
- On self-hosted runners, ensure cleanup between runs
|
||||
|
||||
## Query Suites and Packs
|
||||
|
||||
### Built-In Query Suites
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
queries: security-extended
|
||||
```
|
||||
|
||||
Options:
|
||||
- (default) — standard security queries
|
||||
- `security-extended` — additional security queries with slightly higher false-positive rate
|
||||
- `security-and-quality` — security plus code quality queries
|
||||
|
||||
### Custom Query Packs
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
packs: |
|
||||
codeql/javascript-queries:AlertSuppression.ql
|
||||
codeql/javascript-queries:~1.0.0
|
||||
my-org/my-custom-pack@1.2.3
|
||||
```
|
||||
|
||||
### Model Packs
|
||||
|
||||
Extend CodeQL coverage for custom libraries/frameworks:
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
packs: my-org/my-model-pack
|
||||
```
|
||||
|
||||
## Analysis Category
|
||||
|
||||
Distinguish between multiple analyses for the same commit:
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/analyze@v4
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
```
|
||||
|
||||
### Monorepo Category Patterns
|
||||
|
||||
```yaml
|
||||
# Per language (default auto-generated pattern)
|
||||
category: "/language:${{ matrix.language }}"
|
||||
|
||||
# Per component
|
||||
category: "/language:${{ matrix.language }}/component:frontend"
|
||||
|
||||
# Per app in monorepo
|
||||
category: "/language:javascript-typescript/app:blog"
|
||||
```
|
||||
|
||||
The `category` value appears as `<run>.automationDetails.id` in the SARIF output.
|
||||
|
||||
## CodeQL Configuration File
|
||||
|
||||
Create `.github/codeql/codeql-config.yml` for advanced path and query configuration:
|
||||
|
||||
```yaml
|
||||
name: "CodeQL Configuration"
|
||||
|
||||
# Directories to scan
|
||||
paths:
|
||||
- apps/
|
||||
- services/
|
||||
- packages/
|
||||
|
||||
# Directories to exclude
|
||||
paths-ignore:
|
||||
- node_modules/
|
||||
- '**/test/**'
|
||||
- '**/fixtures/**'
|
||||
- '**/*.test.ts'
|
||||
|
||||
# Additional queries
|
||||
queries:
|
||||
- uses: security-extended
|
||||
- uses: security-and-quality
|
||||
|
||||
# Custom query packs
|
||||
packs:
|
||||
javascript-typescript:
|
||||
- codeql/javascript-queries
|
||||
python:
|
||||
- codeql/python-queries
|
||||
```
|
||||
|
||||
Reference in the workflow:
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
config-file: .github/codeql/codeql-config.yml
|
||||
```
|
||||
|
||||
## Dependency Caching
|
||||
|
||||
Enable caching to speed up dependency resolution:
|
||||
|
||||
```yaml
|
||||
- uses: github/codeql-action/init@v4
|
||||
with:
|
||||
dependency-caching: true
|
||||
```
|
||||
|
||||
Values:
|
||||
- `false` / `none` / `off` — disabled (default for advanced setup)
|
||||
- `restore` — only restore existing caches
|
||||
- `store` — only store new caches
|
||||
- `true` / `full` / `on` — restore and store caches
|
||||
|
||||
> Default setup on GitHub-hosted runners has caching enabled automatically.
|
||||
|
||||
## Alert Severity and Merge Protection
|
||||
|
||||
Use repository rulesets to block PRs based on code scanning alerts:
|
||||
|
||||
- A required tool finds an alert matching the defined severity threshold
|
||||
- A required tool's analysis is still in progress
|
||||
- A required tool is not configured for the repository
|
||||
|
||||
Configure via repository Settings → Rules → Rulesets → Code scanning.
|
||||
|
||||
## Concurrency Control
|
||||
|
||||
Prevent duplicate workflow runs:
|
||||
|
||||
```yaml
|
||||
concurrency:
|
||||
group: codeql-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
```
|
||||
|
||||
## Complete Workflow Example
|
||||
|
||||
```yaml
|
||||
name: "CodeQL Analysis"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: '30 6 * * 1'
|
||||
|
||||
permissions:
|
||||
security-events: write
|
||||
contents: read
|
||||
actions: read
|
||||
|
||||
concurrency:
|
||||
group: codeql-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze (${{ matrix.language }})
|
||||
runs-on: ${{ matrix.language == 'swift' && 'macos-latest' || 'ubuntu-latest' }}
|
||||
timeout-minutes: 120
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- language: javascript-typescript
|
||||
build-mode: none
|
||||
- language: python
|
||||
build-mode: none
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: ${{ matrix.build-mode }}
|
||||
queries: security-extended
|
||||
dependency-caching: true
|
||||
|
||||
- if: matrix.build-mode == 'manual'
|
||||
name: Manual Build
|
||||
run: |
|
||||
echo 'Replace with actual build commands'
|
||||
exit 1
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v4
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
```
|
||||
Reference in New Issue
Block a user