# Run-State Schema (v1.5.6)

*Authoritative schema for `quality/run_state.jsonl`, `quality/PROGRESS.md`, and `Calibration Cycles/<cycle>/run_state.jsonl`. The playbook AI writes these files directly via the file-tool layer; the orchestrator AI reads them to drive multi-benchmark calibration cycles.*

*Companion to: `docs/design/QPB_v1.5.5_Design.md` ("Design — Run-state event taxonomy" section).*

---

## File locations and ownership

- `<benchmark>/quality/run_state.jsonl` — per-run event log. Append-only. Written by the AI executing the playbook.
- `<benchmark>/quality/PROGRESS.md` — human-readable run status. Atomically rewritten by the AI on each event.
- `Calibration Cycles/<cycle>/run_state.jsonl` — cycle-level event log. Append-only. Written by the orchestrator AI.

All three live in the bind-mounted workspace owned by the user. The AI writes via Edit/Write file tools, never via shell redirection or `tee` (which routes through a different UID layer in some sandbox runtimes).

---

## Schema versioning

Every `run_state.jsonl` opens with an `_index` event recording `schema_version`. Current version: `"1.5.6"`. Schema bumps preserve backward compatibility — older files remain readable by newer parsers. Breaking schema changes bump the major number.

---

## Required fields (every event)

Every event object MUST have:

- `ts` — ISO 8601 UTC timestamp with `Z` suffix (e.g. `"2026-05-15T14:32:01Z"`). Sub-second precision allowed but not required.
- `event` — string, the event-type name. Must match one of the names listed in `_index.event_types`.

Events MAY have additional fields per their type's spec below. Unknown fields are tolerated by readers (forward-compatible).

---

## Per-run events (`<benchmark>/quality/run_state.jsonl`)

### `_index`

ALWAYS the first line. Records schema metadata.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | Always `"_index"` |
| `ts` | string | yes | ISO 8601 UTC |
| `schema_version` | string | yes | `"1.5.6"` |
| `event_types` | array of string | yes | Every event type this file uses |
| `benchmark` | string | yes | E.g. `"chi-1.3.45"`, `"virtio-1.5.1"` |
| `lever_state` | string | yes | E.g. `"pre-pattern7"`, `"post-pattern7"`, `"baseline"` |
| `started_at` | string | yes | ISO 8601 UTC, equals `ts` of this event |

### `run_start`

Marks the beginning of a playbook run.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"run_start"` |
| `ts` | string | yes | |
| `runner` | string | yes | One of `"claude"`, `"codex"`, `"copilot"`, `"cursor"` |
| `playbook_version` | string | yes | E.g. `"1.5.6-pre"`, `"1.5.6"` (matches `bin.benchmark_lib.RELEASE_VERSION`) |
| `target_path` | string | yes | Relative path to benchmark target |

### `phase_start`

Marks the beginning of one of the six playbook phases.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"phase_start"` |
| `ts` | string | yes | |
| `phase` | integer | yes | 1, 2, 3, 4, 5, or 6 |

### `pattern_walked`

Phase 1 only. Records that one of the seven exploration patterns was walked.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"pattern_walked"` |
| `ts` | string | yes | |
| `phase` | integer | yes | Always 1 |
| `pattern` | integer | yes | 1 through 7 |
| `findings_count` | integer | yes | Number of findings produced by this pattern |
| `duration_seconds` | number | optional | Wall-clock for this pattern walk |

### `pass_started` / `pass_ended`

Phase 4 only. Records start/end of one of the four skill-derivation passes.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"pass_started"` or `"pass_ended"` |
| `ts` | string | yes | |
| `phase` | integer | yes | Always 4 |
| `pass` | string | yes | One of `"A"`, `"B"`, `"C"`, `"D"` |
| `output_artifact` | string | optional | Relative path to pass artifact (on `pass_ended`) |

### `finding_logged`

Records that a finding (skill-divergence, code-bug, etc.) was logged in the current phase.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"finding_logged"` |
| `ts` | string | yes | |
| `phase` | integer | yes | 1-6 |
| `finding_id` | string | yes | E.g. `"BUG-007"`, `"REQ-042"` |
| `category` | string | yes | E.g. `"code-bug"`, `"skill-divergence"`, `"missing-citation"`, `"prose-to-code-mismatch"` |

### `artifact_written`

Records that an artifact file was produced/updated.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"artifact_written"` |
| `ts` | string | yes | |
| `relative_path` | string | yes | Path relative to benchmark target (e.g. `"quality/EXPLORATION.md"`) |
| `byte_size` | integer | optional | Size of the file at write time |
| `line_count` | integer | optional | Line count |

### `gate_check`

Records the outcome of a single quality-gate check.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"gate_check"` |
| `ts` | string | yes | |
| `gate_name` | string | yes | Identifier from `quality_gate.py` |
| `verdict` | string | yes | One of `"pass"`, `"fail"`, `"warn"`, `"skip"` |
| `reason` | string | optional | Human-readable explanation |

### `phase_end`

Marks the end of a phase. Cross-validated against the phase's expected artifacts before being written (see "Cross-validation rules" below).

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"phase_end"` |
| `ts` | string | yes | |
| `phase` | integer | yes | 1-6 |
| `key_counts` | object | yes | Phase-specific counts (see below) |
| `artifacts_produced` | array of string | yes | Relative paths of artifacts produced this phase |
| `duration_seconds` | number | optional | Wall-clock for the whole phase |

`key_counts` per phase:

- Phase 1: `{"findings_total": N, "patterns_walked": M}` (M should be 7 for full Phase 1)
- Phase 2: `{"findings_promoted": N, "findings_dropped": M}`
- Phase 3: `{"bugs_identified": N, "bug_writeups": M}`
- Phase 4: `{"req_count": N, "uc_count": M, "passes_complete": K}` (K should be 4)
- Phase 5: `{"gate_checks_total": N, "gate_failures": M}`
- Phase 6: `{"bugs_md_count": N, "gate_verdict": "pass|fail|partial"}`

### `error`

Records an error during the run.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"error"` |
| `ts` | string | yes | |
| `phase` | integer | optional | If error is phase-scoped |
| `message` | string | yes | Human-readable description |
| `recoverable` | boolean | yes | If true, the run will retry the affected phase; if false, the run is aborting |

### `documentation_state`

v1.5.6+. Records the documentation-availability state at Phase 1 entry. Currently the only emitted state is `"code_only"`, indicating that `reference_docs/` and `reference_docs/cite/` carry no recognized plaintext content (`.md` or `.txt`) and Phase 1 is proceeding in code-only mode (see `references/code-only-mode.md`). A `"with_docs"` value is reserved for future explicit emission; today the absence of a `documentation_state` event implies docs were present.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"documentation_state"` |
| `ts` | string | yes | |
| `state` | string | yes | Currently `"code_only"`. Future values may include `"with_docs"`. |
| `reason` | string | yes | Free-form (e.g. `"reference_docs/ empty"`) |

When `documentation_state state="code_only"` is emitted, the playbook also prepends a "Documentation status: code-only mode" section to `quality/EXPLORATION.md` and adds a "Documentation state: code_only" line to `quality/PROGRESS.md` so the downgrade is visible to anyone reading either artifact. New runs adding the `documentation_state` event must include it in the `_index.event_types` list.

### `aborted_missing_docs`

v1.5.6+. Records that the run aborted at Phase 1 entry because `--require-docs` was set and `reference_docs/` was empty. Mutually exclusive with `documentation_state state="code_only"` for the same Phase 1 entry — `--require-docs` is the opt-IN abort path; the absence of the flag preserves the documented code-only-mode downgrade. After this event the runner returns non-zero without invoking any LLM work, so no `phase_start phase=1` is recorded.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"aborted_missing_docs"` |
| `ts` | string | yes | |
| `reason` | string | yes | Free-form (e.g. `"reference_docs/ empty and --require-docs set"`) |

When `aborted_missing_docs` is emitted, the playbook also writes an `ERROR: aborted_missing_docs — <reason>` block to `quality/PROGRESS.md` so the abort is visible without reading the JSONL. New runs that pass `--require-docs` against an empty `reference_docs/` must include `aborted_missing_docs` in the `_index.event_types` list.

### `run_end`

Marks the end of the playbook run.

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"run_end"` |
| `ts` | string | yes | |
| `status` | string | yes | One of `"success"`, `"aborted"`, `"failed"` |
| `total_findings` | integer | optional | Sum across all phases |
| `final_verdict` | string | optional | The Phase 6 gate verdict |

---

## Cycle-level events (`Calibration Cycles/<cycle>/run_state.jsonl`)

### `_index` (cycle-level)

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"_index"` |
| `ts` | string | yes | |
| `schema_version` | string | yes | `"1.5.6"` |
| `event_types` | array of string | yes | |
| `cycle_name` | string | yes | E.g. `"2026-05-15-pattern7-displacement-recovery"` |
| `lever_under_test` | string | yes | E.g. `"lever-1-exploration-breadth-depth"` |
| `benchmarks` | array of string | yes | Cycle's pinned benchmark list |
| `iteration` | integer | yes | Iteration ordinal (1, 2, or 3 — see iterate-cap) |

### `cycle_start`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"cycle_start"` |
| `ts` | string | yes | |
| `hypothesis` | string | yes | The cycle's testable hypothesis |
| `noise_floor_threshold` | number | yes | Recall delta below this is treated as noise (default 0.05) |

### `benchmark_start`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"benchmark_start"` |
| `ts` | string | yes | |
| `benchmark` | string | yes | |
| `lever_state` | string | yes | `"pre-lever"` or `"post-lever"` |

### `lever_change_applied`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"lever_change_applied"` |
| `ts` | string | yes | |
| `lever_id` | string | yes | E.g. `"lever-1-exploration-breadth-depth"` |
| `files_changed` | array of string | yes | Paths relative to QPB repo root |
| `commit_sha` | string | yes | Commit SHA on the implementing branch |
| `description` | string | yes | What the change is (e.g. `"Pattern 7 budget cap 3-5 → 2-3"`) |

### `lever_change_reverted`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"lever_change_reverted"` |
| `ts` | string | yes | |
| `files_changed` | array of string | yes | |
| `commit_sha` | string | optional | Null/absent if revert is uncommitted |

### `benchmark_end`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"benchmark_end"` |
| `ts` | string | yes | |
| `benchmark` | string | yes | |
| `lever_state` | string | yes | |
| `recall` | number | yes | 0.0-1.0 |
| `bugs_found` | array of string | yes | Bug IDs found this run |
| `bugs_missed` | array of string | yes | Bug IDs in baseline missed this run |
| `historical_baseline_path` | string | yes | Path to the baseline BUGS.md used for recall computation |

### `cycle_end`

| Field | Type | Required | Notes |
|---|---|---|---|
| `event` | string | yes | `"cycle_end"` |
| `ts` | string | yes | |
| `verdict` | string | yes | One of `"ship"`, `"revert"`, `"iterate"`, `"halt-iterate-cap"` |
| `recall_before` | object | yes | Per-benchmark recall before lever change |
| `recall_after` | object | yes | Per-benchmark recall after lever change |
| `delta` | object | yes | Per-benchmark delta (recall_after - recall_before) |
| `cross_benchmark_check` | object | yes | `{"clean": bool, "regressions": [list of bench/bug pairs that regressed]}` |

---

## Cross-validation rules (per `phase_end`)

The AI verifies these conditions before appending a `phase_end` event. If any check fails, the AI appends an `error` event with `recoverable: true` and re-runs the failing phase.

| Phase | Required conditions |
|---|---|
| 1 | `quality/EXPLORATION.md` exists, ≥ 120 lines (aligned with the Phase 2 startup gate in `bin/run_playbook.check_phase_gate`), contains at least one finding section (regex `^##\s+(Finding\|Open Exploration Findings\|\d+\.)` — accepts `## Finding ...`, the SKILL-prescribed exact heading `## Open Exploration Findings`, and numbered `## N.` headings) |
| 2 | All nine fixed-name Generate-contract artifacts exist non-empty under `quality/`: `REQUIREMENTS.md`, `QUALITY.md`, `CONTRACTS.md`, `COVERAGE_MATRIX.md`, `COMPLETENESS_REPORT.md`, `RUN_CODE_REVIEW.md`, `RUN_INTEGRATION_TESTS.md`, `RUN_SPEC_AUDIT.md`, `RUN_TDD_TESTS.md`. Plus at least one non-empty `quality/test_functional.<ext>` (extension varies by primary language). Pre-v1.5.6 this row described the v1.5.5-design triage model (`EXPLORATION_MERGED.md` / `triage.md`); that mapping was never adopted by shipped SKILL.md / orchestrator_protocol.md / agent files, which always documented Phase 2 as Generate. |
| 3 | `quality/code_reviews/` directory contains at least one review file. If `quality/BUGS.md` has any `### BUG-` heading, `quality/patches/` contains at least one `BUG-*-regression-test.patch` file. Pre-v1.5.6 this row checked `quality/RUN_CODE_REVIEW.md` (a Phase 2 Generate output, not a Phase 3 review result) — same v1.5.5-design / shipped-Generate drift class as the Phase 2 row. Cluster B reconciled. |
| 4 | `quality/spec_audits/` directory contains at least one `*-triage.md` file AND at least one `*-auditor-*.md` file (per orchestrator_protocol.md naming convention). When neither name pattern matches, the validator falls back to a weaker "≥2 files" check — older bootstrap runs with arbitrary `.md` names still pass; the gate at Phase 6 enforces deeper conformance. Pre-v1.5.6 this row checked `quality/REQUIREMENTS.md` + `COVERAGE_MATRIX.md` (Phase 2 outputs) — same v1.5.5-design drift class. Cluster B reconciled. |
| 5 | If `quality/BUGS.md` has confirmed `### BUG-` entries: `quality/results/tdd-results.json` exists non-empty; for every confirmed bug, `quality/writeups/BUG-NNN.md` exists AND `quality/results/BUG-NNN.red.log` exists. With no confirmed bugs the row is vacuously satisfied. Pre-v1.5.6 this row checked `quality/results/quality-gate.log` (a Phase 6 output) — same v1.5.5-design drift class. Cluster B reconciled. |
| 6 | `quality/results/quality-gate.log` exists non-empty AND `quality/PROGRESS.md` contains a `Terminal Gate Verification` section (the orchestrator-protocol marker that Phase 6 ran the script-verified gate to completion). Pre-v1.5.6 this row checked `quality/BUGS.md` + `quality/INDEX.md` — BUGS.md is a Phase 3 output, INDEX.md was never adopted in the shipped contract. Same v1.5.5-design drift class. Cluster B reconciled. |

The `run_end` event additionally requires: all 6 `phase_end` events present in the log; the final BUGS.md count matches `phase_end phase=6 key_counts.bugs_md_count`.

---

## Resume semantics

When an AI session starts on a run directory:

1. If `quality/run_state.jsonl` does not exist: fresh run. Write `_index` + `run_start` + `phase_start phase=1`.
2. If it exists: read all events. Find the last `phase_start` not followed by a matching `phase_end`. Call it the "in-progress phase".
3. Verify the in-progress phase's expected artifacts (per cross-validation rules above):
   - If artifacts complete: append the missing `phase_end` event and proceed to the next phase. Note: this is the "session crashed mid-phase but the work is done" recovery path.
   - If artifacts incomplete: re-run that phase from scratch. The prior session left a partial state that can't be safely resumed.
4. If all 6 `phase_end` events are present but no `run_end`: append `run_end status=success` and finalize.

The policy is "trust artifacts more than events." If events claim phase 4 done but `REQUIREMENTS.md` doesn't exist, the AI re-runs phase 4. If events stop mid-phase but artifacts are complete, the AI catches up the events.

---

## PROGRESS.md format

Atomically rewritten on every event. Markdown.

```markdown
# QPB Run Progress

**Started:** 2026-05-15T14:32:01Z  **Benchmark:** chi-1.5.1  **Lever:** post-pattern7
**Runner:** claude  **Playbook version:** 1.5.6

## Phases

- [x] Phase 1 — Explore (10:10, 12 findings, patterns 1-7 walked)
- [x] Phase 2 — Generate (0:42, 9 artifacts produced)
- [x] Phase 3 — Code Review (15:31, 6 bugs identified)
- [x] Phase 4 — Spec Audit (3 auditors, 1 triage)
- [ ] Phase 5 — Reconciliation *(in progress, started 14:58:31Z)*
- [ ] Phase 6 — Verify

## Recent events (last 10)

- 2026-05-15T14:58:31Z — phase_start phase=5
- 2026-05-15T14:58:30Z — phase_end phase=4 passes=[A,B,C,D] req_count=89
- 2026-05-15T14:42:11Z — phase_end phase=1 findings=12

## Artifacts produced

- quality/EXPLORATION.md (12,034 bytes)
- quality/REQUIREMENTS.md (28,891 bytes)
- quality/COVERAGE_MATRIX.md (3,022 bytes)
```

Sections (header, phase checklist, recent events, artifacts produced) are required. Phase checklist uses `[x]` for complete phases (with summary stats), `[ ]` for incomplete, with in-progress phase noted explicitly with start time. Recent events shows last 10 event lines from `run_state.jsonl` in human-readable form. Artifacts produced shows files written this run with byte sizes.

---

## Format invariants (enforced by `bin/run_state_lib.py` validators)

1. `_index` is line 1.
2. Every line is valid JSON (one object per line).
3. Every event has `ts` and `event` fields.
4. Every `event` value appears in `_index.event_types`.
5. Append-only: events are added, never edited. Editing a prior event is a schema violation.
6. `phase_start` and `phase_end` events for a given phase appear at most once per run (no out-of-order or duplicate phase markers).
7. `run_start` is the second line (after `_index`); `run_end` is the last line if the run completed.

Validators are read-only checks. They surface violations as findings; they don't auto-correct.