# 06 — Development & Operations

**Audience:** developers and operators of the Library (v0.22.188, `package.json`). **Method:** every claim below was measured from the code or the governing document in this tree on 2026-09-22, cited as `file:line`. Where something is a *plan* rather than shipped code it is marked **PLANNED** or **DESIGN**. Normative keywords follow RFC 2119.

---

## 1. The async job system

### 1.1 Two queues, two workers

| Queue table | Consumer | Claim style | Evidence |
|---|---|---|---|
| `registry.async_jobs` | `pdl-job-worker` (`scripts/job-worker.ts`) | poll every `JOB_WORKER_POLL_MS` (default 1000 ms), `FOR UPDATE SKIP LOCKED` (STANDARD-006 P2) | `scripts/job-worker.ts:13-14`, `:71`, `:700` |
| `conduit.analysis_jobs` | `pdl-worker` (`scripts/worker.ts`) and `pdl-worker-large` (`scripts/worker-large.ts`) | analysis loop: claim → fork child → done | `scripts/worker.ts:290`, `:588`, `:1148`; `scripts/worker-large.ts:1-8` |

The generic queue lifecycle is `queued → running → (completed | failed | cancelled)` (`scripts/job-worker.ts:16`). The worker is stateless and restartable: on restart, `running` rows older than 5 minutes with no recent `updated_at` are reaped to `failed` (`:24-26`, `reapStaleRunning`, `:617` exempts legitimately long kinds such as `crawler.project_run`).

### 1.2 Job kinds and handlers

Handlers live in one map, `const HANDLERS: Record<string, JobHandler>` at `scripts/job-worker.ts:105` (the map closes at `:430`). Kind names are **imported from config**, never typed in the worker (headline rule) — with five string-literal exceptions (three crawler kinds and two `dataset_analysis.*` kinds). Measured: 16 config-keyed entries plus 5 literals = **21 kinds** (`grep -nE '^\s{2}(\[|")' scripts/job-worker.ts`, lines 105-430; `portal-to-ontology-development-writeup.md:102` counts 21 independently).

| Kind (resolved string) | Config source | Handler line |
|---|---|---|
| `workspace_schema_maintenance` | `config/rag.config.ts:1141` | `:109` |
| `mdm.resolution_run` | `config/mdm-resolution.config.ts:65` | `:121` |
| `mdm.config_train` | `config/mdm-resolution.config.ts:251` | `:140` |
| `mdm.rule_run` / `mdm.rule_backfill` | `config/mdm-stewardship.config.ts:338-339` | `:163` / `:170` |
| `mdm.proposal_prepare` | `config/mdm-preparer.config.ts:77` | `:183` |
| `mdm.recommendation_refresh` | `config/recommendation-generation.config.ts:70` | `:195` |
| `mdm.recommendation_attribution` | `config/recommendation-attribution.config.ts:47` | `:211` |
| `mdm.quality_axis_refresh` | `config/mdm-quality.config.ts:473` | `:226` |
| `rag.usage_rollup` | `config/rag-telemetry.config.ts:99` | `:246` |
| `corpus_ingest` / `corpus_reindex` / `corpus_resplit` | `config/rag.config.ts:2099,2112,2151` | `:257` / `:277` / `:296` |
| `corpus_fetch` | `config/rag.config.ts:2467` | `:305` |
| `catalog.terms_materialize` | `config/data-catalog.config.ts:3693` | `:310` |
| `crawler.project_run` / `crawler.source_run` / `crawler.portal_discover` | literals (`config/crawler.config.ts:228` for the first) | `:316` / `:334` / `:362` |
| `dataset_analysis.preflight` | literal (the `dataset_ingest.*` handler twins were deleted at LIB-BUG-287, `:373-377`) | `:378` |
| `dataset_ingest.physical_load` | `config/physical-masking.config.ts:78` | `:389` |
| `dataset_analysis.submit` | literal (thin delegation to the conduit analysis queue — the `async_jobs` row shadows the conduit job and bridges status back) | `:404` |

A handler receives `(params, isCancelled, job)`. Cancellation is cooperative: the worker checks `cancel_requested` each poll and a handler halts at its next checkpoint (`:18-22`); crawler handlers refuse with `cancelled_before_start` if cancelled while queued (`:324`, `:335`, `:363`).

### 1.3 Lanes

`config/job-lanes.config.ts` declares named lanes; each runs **at most one job at a time on its own DB connection**, and the implicit `default` lane runs every kind not claimed (`:10-17`). Measured lanes (`:36-49`):

| Lane | Kinds | Why |
|---|---|---|
| `bulk` | `corpus_ingest`, `corpus_reindex`, `corpus_resplit`, `workspace_schema_maintenance` | LIB-BUG-698 head-of-line; LIB-BUG-791/766 single-slot mutual exclusion |
| `CORPUS_UPLOAD_CONFIG.fetchLane.laneName` | `corpus_fetch` | multi-GB pulls must not starve small ingests (LIB-RAG-1 D3 F13) |
| `default` | everything else (MDM, analysis, masking, catalog) | pre-lane semantics preserved |

Enrolling a heavy kind is one config row, no worker edit (`:16-17`). The same file declares `SYSTEM_ENQUEUED_JOB_KINDS` (`:80`; rationale `:54-79`): kinds the platform enqueues with `created_by IS NULL`, which the `job` umbrella MUST NOT expose to a non-admin keyless caller (LIB-BUG-946 V-9).

### 1.4 Durable ticks

Time-gated schedulers (quality-axis refresh, recommendation refresh/attribution, RAG usage report) are gated on `registry.scheduled_tick_state` (`scripts/job-worker.ts:43`, `:819-825`) via `armDurableTick(...)` (defined `:828`; armed at `:841`, `:960`, `:979`, `:991` — one per named tick). Policy lives in `config/scheduled-ticks.config.ts`: `pollCadence: "PT5M"`, `claimGrace: "PT10S"`, `maxCatchUpWindows: 1`, `durabilityRequiredAtOrAbove: "PT30M"` (ISO 8601 durations, `:51-60`); tick names at `:88-92`. Operators: a healthy boot logs `durable-tick gate healthy (no degraded ticks)`; `durable-tick DEGRADED` means the schema is behind the code — migrate first (LIB-BUG-1029; CLAUDE.md release step 5).

### 1.5 The `job` umbrella (retry, cancel, correlation ids)

`scripts/mcp-server/umbrellas/job.ts` commands (measured `case` labels): `get` (`:172`), `list_mine` (`:287`), `cancel` (`:334`), `retry` (`:396`), `history` (`:446`), `list_all` (`:504`), `stats` (`:533`), `reset_circuit_breaker` (`:552`). Read-only `get` is open to any caller; listing/mutation of the analysis queue is admin-only (`:168`). `retry` is owner-scoped (`asyncOwnerScope`, LIB-BUG-428) and honours the failed/cancelled-only contract (`:396-410`); the analysis-queue status vocabulary is `pending|running|completed|failed|retry|cancelled` (`:130`).

**Failures carry a correlation id.** On handler failure the worker mints `randomUUID()` and stores `{...err, correlation_id}` in the row while logging `failure correlation <id> for <short_id> (<code>)` (`scripts/job-worker.ts:588-596`); reaped rows each get their own id (`:656`, `:664`). Operators: grep `pm2 logs pdl-job-worker` for the id a caller reports.

---

## 2. pm2 processes (`ecosystem.config.cjs`)

All processes run from the PRIMARY tree `/home/datashield/library`; worktree builds never deploy. `DATABASE_URL` is declared once (`:13`) and `PDL_EXPECTED_DATABASE` is derived from it so every process fails closed on a foreign `current_database()` (`:14`, LIB-BUG-719). Never `pm2 reload --update-env` (`:12`, `:339`, LIB-BUG-719). The three auxiliary workers are sourced from `lib/workers/*.ts`, not `scripts/` (`scripts/build-scripts.sh:84-86`).

| Process | Entry (`args`) | Role | Lines |
|---|---|---|---|
| `pdl-web` | `npm run start`, PORT 3002 | Next.js UI + REST bridge (`/api/*`, `/api/health`) | `:28-30` |
| `pdl-mcp` | `dist-scripts/mcp-server.mjs --http`, MCP_PORT 3100 | MCP server (all umbrellas); `UNIVERSAL_MIDDLEWARE_ENABLED="true"` | `:64-66` |
| `pdl-worker` | `dist-scripts/worker.mjs` (16 GB heap) | analysis loop over `conduit.analysis_jobs`, crons, crawler queue, child ingest | `:105-108` |
| `pdl-worker-large` | `dist-scripts/worker-large.mjs` (18 GB heap, 4 h drain) | large/high-priority analysis only | `:146-151`, `:182-183` |
| `pdl-anomaly-detector` | `dist-scripts/anomaly-detector.mjs` (`lib/workers/anomaly-detector.ts`) | polls `mcp_call_log` every 60 s; last-5-min rate > 5× the rolling 60-min baseline → `registry.alerts_queue` INSERT (`anomaly-detector.ts:4-8`) | `:191-193` |
| `pdl-job-worker` | `dist-scripts/job-worker.mjs` (2 GB heap) | §1 generic queue + durable ticks; MUST restart every release (LIB-BUG-565) | `:218-221` |
| `pdl-alerts-dispatcher` | `dist-scripts/alerts-dispatcher.mjs` (`lib/workers/alerts-dispatcher.ts`) | drains `alerts_queue`; only the `internal` channel (structured log line) delivers — `slack`/`email` are STUBS that record `channel_not_implemented` and dispose the row (`alerts-dispatcher.ts:9-13`, `:179`) | `:258-260` |
| `pdl-billing-reconcile` | `dist-scripts/billing-reconcile.mjs` (`lib/workers/billing-reconcile.ts`), `cron_restart "0 6 * * *"` | one-shot daily reconciliation of `mcp_call_log` against `api_usage_daily` / `api_usage_mcp_daily`; each drift row → `alerts_queue` (`billing-reconcile.ts:5-17`) | `:290-305` |

Which bundle each consumer needs, and the reload verb/order, is declared in `config/bundle-consumers.config.ts` (`CONSUMER_RELOAD_POLICY`) and consumed by `npm run release:reloads` (§3).

---

## 3. The release process as it actually runs

Authoritative text: `CLAUDE.md` § "canonical release-event sequence". **TAG LAST.** Summary of the sequence with the commands a runner types:

| Step | Command | Notes |
|---|---|---|
| 0 | `npm run release-lock:acquire -- --holder <s> --purpose "release vX.Y.Z"` | STANDARD-007; `release-lock:check` from the PRIMARY tree with `RELEASE_LOCK_HOLDER` set (`package.json:328-331`) |
| 0 | `npm run release:build-cycle -- open --version vX.Y.Z --input '{...}'` | writes `registry.build_cycles` (`scripts/release-pipeline/13-build-cycle.ts`, `package.json:347`) |
| 2a0 | ledger reconciliation | only if DDL was applied out of band under another filename |
| 2a0.1 | `npm run release:must-declare -- --sprint <branch>` | generates the must-declare list into `docs/audit/vX.Y.Z-merge-ledger.md` (`package.json:342`) |
| 2a1 | `RELEASE_LOCK_HOLDER=<s> npm run release:migrate` | applies pending migrations INSIDE the lock, before gates (LIB-BUG-1057; `package.json:339`) |
| 2a2 | `npm run release:gates` | the HARD gate chain; **205 links at v0.22.188** (measured: `node -e "…split('&&').length"`). Never hand-edit; use `npm run gates:splice` (`package.json:341`). Then `release:build-cycle -- gates` (state-less; per-link exits + red attribution; LIB-BUG-1166) and `-- merge` |
| 2b | `RELEASE_VERSION=vX.Y.Z npm run release:gen` → `release:snapshot` → `release:diff` → release notes | writes `docs/versions/vX.Y.Z/` (write-once) |
| 2b2/2b3 | `npm run audit:snapshot-version-agreement`, `RELEASE_VERSION=… npm run audit:test-prompt-shape-validity` | HARD, run after snapshot and before publish |
| 2c | `npm run release:publish -- --snapshot=docs/versions/vX.Y.Z --version=vX.Y.Z` | auto-commit |
| 2d/2e | version-surface grep audit → `package-bump` | bump commit |
| 3-4 | PAUSE "ready to tag at SHA"; operator `git tag -a vX.Y.Z <sha>` | `build-cycle tag` REFUSES without `gates_final_exit` |
| 5 | `npm run release:migrate` again | expected `PLAN: nothing pending — no-op` |
| 6 | `npm run release:reloads` (`scripts/release-reload-chain.ts`, `package.json:306`) | derived from `CONSUMER_RELOAD_POLICY`; terminates in strict `audit:bundle-freshness`. NO hand restarts |
| 6b | `release:build-cycle -- deploy / verify / report` | then `audit:build-cycle-completeness --strict`, `audit:bug-provenance-coverage --strict` |
| 7 | `npm run release:report` (T-8) | LAST; `--snapshot` required (`scripts/release-pipeline/07c-gen-release-report.ts:16,113`); saved via the `report` umbrella at slug `release-vX.Y.Z` (`:173-219`); checked by `audit:release-report-currency` |

Builds: `scripts/build-scripts.sh` produces every `dist-scripts/*.mjs` (esbuild, type-stripped) and is REQUIRED before any bundle-consumer reload; `env NODE_ENV=production npm run build` produces `.next/` for `pdl-web` (LIB-BUG-201 disaggregation, LIB-BUG-266).

### 3.1 Dev-gov rules a contributor MUST know

| Rule | Where it is written | What it means for you |
|---|---|---|
| Independent close vet is non-waivable | CLAUDE.md META-CALIBRATION 7; `docs/CC-DIRECTIVES.md:263` (Directive 9) | every substantive change ships only after an adversarial vet by a session other than its author; live verification exercises the real queue/door path |
| Findings define classes, not instances | CLAUDE.md META-CALIBRATION 8 | grep every sibling site of a finding before closing it; re-vets hunt unswept siblings first |
| Must-declare | `config/release-ledger-contract.config.ts`; `audit:must-declare-completeness` | every merged bundle ledger declares its release-notes section or a written waiver; the gate blocks otherwise |
| `.describe()` coverage | CLAUDE.md rules 7/7b/7c; `scripts/release-pipeline/_lib/enforced-describe-umbrellas.ts` | enrolled umbrellas HARD-BLOCK the release on any undescribed required param; new umbrellas enroll at creation |
| No new MCP tools | `docs/audit/portal-migration-1-ratification.md:55` (cited by writeup §6 iv-a); `npm run audit:umbrella-inventory` | new capability is a *command* on an existing umbrella; the tool count is reconciled against `config/mcp-tiers.config.ts` |
| Worktree preflight | `npm run agent:preflight -- --branch <b> --base <ref> [--detach <sha>] --role builder\|vetter\|runner` (`package.json:340`) | creates `/home/datashield/library-wt-<name>`; refuses under an ambient `DATABASE_URL`; check `ls -ld node_modules` before any install (symlink hazard) |
| Hooks | `.husky/pre-commit` (anti-sprawl: `audit:duplicate-detector --staged`, `audit:justification-required --staged`; every run logs to `.husky/.audit-log`); `.husky/pre-merge-commit` (LIB-BUG-1273: clean merges fire this hook) | `--no-verify` is PROHIBITED by policy; a commit-count vs log-count mismatch is the bypass signal |
| Path-scoped commits, `--no-ff` merges, never delete sprint branches | CLAUDE.md "Sprint forensic ledger discipline" | the sprint branch is the forensic ledger |
| Standards | `docs/standards/STANDARD-001…011` | 001 report scroll, 004 server-side authz, 006 concurrency (row-scoped patterns P1-P5, never table locks), 007 release lock, 008 master-detail surfaces, 009 privacy/masking, 010 connection paths; 011 (catalog north star) is still `-DRAFT` |

---

## 4. The bug registry (`artifact` umbrella)

Bugs are registry-backed rows in `registry.bugs` with a generated file; **ids come only from the writer**. Measured from the tool description in `scripts/mcp-server/umbrellas/artifact.ts`:

| Command | Line | Behaviour |
|---|---|---|
| `bug_submit({title, description, severity?, umbrella?, sprint?, discovered_by?, reproducer?, affected_versions?, related_bugs?, metadata?, auto_commit?})` | `:129` | canonical filer: DB-sequence id `LIB-BUG-NNN`, tmp+rename atomic write, INSERT, optional auto-commit. Replaces client-picked ids (LIB-BUG-129..138 collisions) |
| `bug_get({id})` | `:130` | body + `provenance` (introduced_in/by/how, detected_by, resolved_in/by, regression_evidence, bisect_owed) + `final_resolution` |
| `bug_list({umbrella?, sprint?, status?, severity?, limit?, cursor?})` | `:131` | paginated, id-DESC, cursor = last id |
| `bug_update({id, status?, severity?, metadata?, body_append?})` | `:132` | a flip to `resolved` writes the provenance record and renders `## Final resolution`; pass `metadata.resolved_in` at release events (CLAUDE.md 6b) |

`artifact.create({type:'bug'})` FORWARDS to `bug_submit` (`:119`); `artifact.update`/`bulk_update`/`convert` REJECT bug ids (`:123-126`). Directive 1 (`docs/CC-DIRECTIVES.md:10`) governs the bug file lifecycle.

---

## 5. The Development section (being built)

Sources: `docs/audit/dev-section-charter.md` §3-§4 and `docs/audit/portal-to-ontology-development-writeup.md` §1, §6. Status at v0.22.188: **step 0 ratified in documents; no step-i code has been measured as shipped in this tree.** Treat everything below as the plan of record.

### 5.1 Architecture (charter §3, `dev-section-charter.md:28-46`)

```
Development (nav section, config row)
 ├─ Monitor   — read models: Releases (build_cycles) · Bugs · Estate · Jobs        [step i]
 ├─ Pipelines — DEFINITIONS (registry.pipelines + append-only revisions)
 │             + RUNS (lineage.runs, one per node execution)                       [ii, iii-a, iv]
 └─ Lineage   — in-house canvas, third VIEW_SOURCE `openlineage_runs`, one pipeline run  [iii-b]
```

Rules (RFC 2119, charter §3): definitions reference catalog assets by id and MUST NOT carry a value (`scanForValueLeak`); the run ledger MUST be the OpenLineage store — no second run table; the executor MUST be one job kind `pipeline.run` on the existing queue (§1), cadence via the durable tick; every surface follows STANDARD-008.

### 5.2 Step i — the slice being built (PLANNED; writeup §6 rows i-a…i-f)

| Sub-step | What | Wire | Migrations |
|---|---|---|---|
| i-a | Nav section `config/development-nav.config.ts` (mirror `data-catalog-nav.config.ts:180-213`) + `app-nav.config.ts` edits + nav test | none | none |
| i-b | Development app shell `app/app/development/` + `sections/index.ts` totality assert + `lib/contracts.ts` | none | none |
| i-c | Bridge door `app/api/app/development/[verb]/route.ts` derived from a wire contract + `COMMAND_AUTH`; RFC 9457 refusals | REST over EXISTING MCP commands | none |
| i-d | Monitor lanes **Bugs** (`artifact.bug_list`), **Jobs** (`job.list_all`/`stats`), **Estate** (link to catalog bridge) via `useSectionFeed`; STANDARD-009 counts-only | REST only | none |
| i-e | **Releases** lane: ONE read command over `registry.build_cycles` on an existing umbrella (the single wire addition; 7b enrolment + must-declare) | 1 addition | none |
| i-f | Blank-lineage defect: set `anchor` on `app-nav.config.ts:464-473` (`anchor: null` at `:470`) (page requires `start`, `app/data-catalog/lineage/page.tsx:100-104`) | none | none |

Step i total ≈ 455k naive tokens (≈ 680k at ×1.5) + vet ≈ 200k + runner ≈ 300k; zero migrations (charter §4).

### 5.3 Roadmap ii-iv (PLANNED — NOT SHIPPED)

| Step | What | Gate |
|---|---|---|
| ii-a | `registry.pipelines` + `pipeline_revisions` + `config/pipeline-model.config.ts`; CRUD on `data_catalog` (1 migration) | D-3 ownership + append-only; D-8 retire `docs/specs/pipeline-monitor-ui-spec.md` first |
| ii-b | OpenLineage store M-1 + M-2 + `config/openlineage.config.ts` (2 migrations, +1 contingent) | O-1..O-6 ruled; D-5 heal-first |
| ii-c | real `lib/server/lineage/emit.ts` (does not exist today; `lib/data_catalog/extract/lineage.ts:109-116` is a self-declared stub); gate → block | D-5 |
| iii-a | Port the Portal editor UI (real 9,020-line React-Flow front end, backend absent) onto ii-a and `@xyflow/react@12` | D-5 design session; gated on ii-a |
| iii-b | third `VIEW_SOURCES` entry `openlineage_runs` (`config/lineage-view.config.ts:401`) + facet renderer | O-6, LIB-BUG-1209 |
| iv-a | executor: job kind `pipeline.run` in `HANDLERS` + lane row; status via existing `job.get/list_all/history` | W1 `catalogWrite()` shipped; D-6 |
| iv-b | ONE enqueue seam over the 17 non-test files carrying `INSERT INTO registry.async_jobs` (58 raw occurrences; writeup §6 E-5) | D-6 |
| iv-c | W1 sinks | out of scope |

Sequencing constraints (writeup §6): ii-a before iii-a; D-8 before ii-a; heal O-1..O-6 in code before ii-b; ii-b before iii-b; W1 before iv-a; iv-b before or with iv-a.

---

## 6. Lineage today and the OpenLineage roadmap

### 6.1 Today: the `edges` umbrella over `asset_edges`

`scripts/mcp-server/umbrellas/edges.ts` is **read-only by contract** — there is no edge-write command (`:4`, `:51`); relationships are written by the pipelines that own them (the corpus splitter writes `part_of`/`cites_section`). Commands (`:55-92`): `predicates`, `neighbors` (depth-1), `traverse` (bounded BFS, every hop gated), `view` (layered graph for a canvas), `impact` (reverse closure). Storage: `migrations/registry/0190_asset_edges.sql`; `config/edges.config.ts:17` states the one rule — `asset_edges` stores ONLY otherwise-discarded relationships; everything else is `read_through` from its owning table (`:42`, `:64`). Measured state (writeup §1): lineage is real for RAG chunks (17,888 document→document edges) and **absent for data** — the `lineage.*` schema has 0 tables.

### 6.2 DESIGN: OpenLineage-native lineage (`docs/specs/lineage-openlineage-design.md`)

D1 canonical model Dataset / Job / Run (`:62-101`); D2 ONE run plane `lineage.datasets` / `runs` / `io`, relationship plane stays in `asset_edges` (`:111`); D3 emitter enrolment contract (`:174`); D4 wire: one read, one new command, **no new tool** (`:228`); D5 asset detail = three lists, no new tables (`:270`). Operator rulings 2026-09-16 (`:346-367`):

| Decision | Ruling |
|---|---|
| O-1 namespaces | `opaque` (`datashield://provider/<id>`) by default; per-provider `interop` opt-in |
| O-2 Marquez | not hosted this sprint; `edges.export` targets its ingest format |
| O-3 retention | keep all runs 90 days, then fold to latest run per (job, dataset, io) + monthly counts |
| O-4 golden records | Datasets by *type*, never per record (STANDARD-009) |
| O-5 backfill | none in L1a |
| O-6 facets | standard `dataQualityMetrics` carries counts only; richer analysis travels as custom value-free `datashield_*` facets (`:369+`) |

None of D1-D7 is implemented at v0.22.188 (`lib/server/lineage/emit.ts` absent; writeup §1).

---

## 7. Observability surfaces

| Umbrella | File | Commands (measured) | Reads |
|---|---|---|---|
| `timeline` | `scripts/mcp-server/umbrellas/timeline.ts` | `make` `:131`, `get` `:136`, `render` `:150`, `publish` `:155`, `list` `:238`, `lineage` `:248`, `analysis` `:282`, `super` `:316` | `registry.timeline_spec`, `registry.storyline` (`:143`, `:161`), PHI determinations gate (`:173`) |
| `mcp_log` | `scripts/mcp-server/umbrellas/mcp_log.ts` | `query` `:270`, `stats` `:381` | `registry.mcp_call_log` (`:348`), `registry.mcp_sessions` (`:173`), account scoping via `registry.api_keys` (`:309`) |
| `health_check` | `scripts/mcp-server/umbrellas/health_check.ts` | **no `command` discriminator** — a flat object (`:206`) taking `mode` (`:222`) = `quick` (default, any caller) \| `full` (admin-tier: doctor report, config, hardware, queue, table sizes, egress) \| `worker` (admin-tier: heartbeat analytics) and `format` (`:223`) = `json` \| `text`. STANDARD-003 extracted umbrella; v0.22.14 R2 sweep gave it the canonical compact MCP error envelope (`:1-6`); REST twin `GET /api/health` on pdl-web (CLAUDE.md preflight) | DB connectivity + latency, migration status (`pending` is always a COUNT), worker-heartbeat freshness, dataset + active-key counts, `rollout_mode` |
| alerts | no `alerts` umbrella exists; the surface is `stewardship.alerts_queue` (`scripts/mcp-server/umbrellas/stewardship.ts:835`) | queue rows produced by `pdl-anomaly-detector` and `pdl-billing-reconcile`, drained by `pdl-alerts-dispatcher` (§2) | `registry.alerts_queue` |

The MCP surface uses the compact canonical MCP error envelope; problem+json (RFC 9457) is REST-only (DEC-1 / LIB-BUG-318; `lib/server/problem.ts`).

---

## 8. Where to read release notes and reports

| Artifact | Location | Produced by |
|---|---|---|
| Release notes + capability manifest + per-tool docs + diffs | `docs/versions/vX.Y.Z/` (`RELEASE-NOTES.md`, `capability-manifest.json`, `diff-*.json`, `tools/`) — write-once, 1 directory per tag | `release:gen` / `release:snapshot` / `release:diff` (§3 step 2b) |
| Release event ledger | `docs/audit/vX.Y.Z-release-event.md` (default `--event-ledger`, `07c-gen-release-report.ts:20`) | the release runner, during the event |
| Merge ledger with must-declare block | `docs/audit/vX.Y.Z-merge-ledger.md` | `release:must-declare` |
| T-8 release report | `report` umbrella, slug `release-vX.Y.Z` (`07c-gen-release-report.ts:173-219`); also the local file the script writes | `npm run release:report`, generated LAST (§3 step 7) |
| Build-cycle record | `registry.build_cycles` (read via `release:build-cycle`; the PLANNED i-e Releases lane) | `13-build-cycle.ts` |
| Fresh-client test prompt per release | `docs/audit/vX.Y.Z-fresh-client-test-prompt.md` (e.g. v0.22.188) | operator requirement, every cycle close |
| Live discovery | `_hello` → `ux.help` / `ux.docs_get` / `ux.versions_list` with `version` | pdl-mcp |

Operator quick checks: `pm2 list && curl -s localhost:3002/api/health`; `git status && git log --oneline -3`; `npm run release-lock:status`; `npm run release:reloads -- --plan`; `npm run release:migrate -- --plan`.