v0.48.0 — SLICE-2: collection field schema + central validation (§22.4a)

Collections declare a `fields:` schema in `.collection.yaml`; entries carry
typed metadata (enum/tags/text). New central `metadata_schema` module parses
the schema leniently (INV-3) and validates entry values — advisory at read
(corpus mirror flags violations as `metadata_malformed` without hard-failing),
the enforcement point for the write boundary (edit endpoints land SLICE-4/5).

- app/metadata_schema.py: parse_fields (lenient/normalizing) + validate
- registry.parse_collection_manifest reads `fields:` into collection config
- collections.get_collection unpacks `fields`; served by the collection API
- cache._refresh_collection_corpus validates each entry advisory-only

Non-breaking, opt-in: no `fields:` → unchanged (INV-5); default `document`
collection declares none (N=1 unchanged). No DB migration — schema rides in
collections.config_json. SLICE-2 of
docs/design/2026-06-06-configurable-collection-metadata.md §7.2.

Backend suite green (601 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/design/plans/2026-06-07-slice2-collection-field-schema.md

@@ -0,0 +1,109 @@
+# Implementation plan — SLICE-2: collection field schema + central validation
+
+**Slice:** SLICE-2 of
+[`docs/design/2026-06-06-configurable-collection-metadata.md`](../2026-06-06-configurable-collection-metadata.md)
+§7.2. **Session:** OHM-0085. **Branch:** `worktree-metadata-slice2-schema`.
+
+## Goal / Definition of Done (from the design)
+
+- `.collection.yaml` `fields:` block is parsed and stored.
+- `metadata_schema.validate` exists — **advisory at read**, the enforcement
+  point **at write** (write endpoints land in SLICE-4/5; this slice supplies and
+  read-wires the function).
+- The schema is served via the collection API (`GET …/collections/{id}`).
+- A collection with **no `fields:`** behaves exactly as today (INV-5) — the
+  §22.13 default `document` collection sees zero change.
+
+## Design decisions (this slice)
+
+- **Field-def shape** (design §6.3): `fields:` is an ordered mapping
+  `{name → {type, values?, label?}}`. `type ∈ {enum, tags, text}` (v1).
+  Order is preserved for facet display (SLICE-3).
+  - `enum` — single scalar; requires a non-empty `values:` list.
+  - `tags` — list; `values:` optional (controlled when present, free-form
+    otherwise).
+  - `text` — free string scalar.
+- **`ref` and `multi-enum` are out of v1** (design §2 future; Q2 leans "later").
+  Unknown field types are **ignored with a warning** (design §6.3), not fatal.
+- **Lenient schema parsing.** A malformed `fields:` block or an individual bad
+  field def is skipped with a warning, never raised — a typo in one field must
+  not nuke the collection mirror (INV-3 spirit). Structural manifest errors
+  (`type`, `visibility`) keep raising `RegistryError` as before.
+- **No DB migration.** The normalized `fields` schema rides in the existing
+  `collections.config_json` column, exactly like `enabled_models`.
+- **`values` source for validation.** An entry's full metadata mapping
+  (`metadata.metadata_dict(entry)` = `to_frontmatter_dict`) — `tags` come from
+  the known `Entry.tags`; custom fields (e.g. `priority`) come from
+  `Entry.extra`. Undeclared keys are forward-compat (INV-7) and never flagged.
+
+## Tasks
+
+### 1. `app/metadata_schema.py` (new) — the one place that knows field shapes
+- `VALID_FIELD_TYPES = {"enum", "tags", "text"}`.
+- `parse_fields(raw) -> dict[str, dict]` — lenient/normalizing. Returns an
+  ordered mapping of `{name: {"type", "values"?, "label"?}}`. Skips: non-mapping
+  block; non-mapping field def; unknown/missing type; `enum` without a non-empty
+  `values:` list. Logs a warning per skip. Pure (no I/O).
+- `@dataclass Problem(field, code, message)` + `as_dict()`.
+- `validate(values: dict, fields: dict[str, dict]) -> list[Problem]` — for each
+  **declared** field, validate the entry's value (absent is OK):
+  - `enum`: scalar ∈ `values:` else `not-in-values`; a list/non-scalar →
+    `wrong-type`.
+  - `tags`: must be a list (`wrong-type` otherwise); if controlled, each member
+    ∈ `values:` else `not-in-values`.
+  - `text`: must be a scalar string (`wrong-type` otherwise).
+  Undeclared keys ignored (INV-7). Never raises.
+- **Tests** `tests/test_metadata_schema.py`: parse (each type, missing block,
+  enum-without-values skipped, unknown type skipped, order preserved); validate
+  (happy, enum bad value, tags uncontrolled-ok, tags controlled-bad, text wrong
+  type, undeclared key ignored, empty schema → no problems).
+
+### 2. `registry.py` — parse `fields:` into the collection config
+- In `parse_collection_manifest`, after the existing keys: if `raw.get("fields")`
+  present, `cfg["fields"] = metadata_schema.parse_fields(raw["fields"])` (only
+  set when non-empty). Flows into `CollectionEntry.config` →
+  `config_json` via the existing `json.dumps(ce.config)` in
+  `_upsert_named_collection`.
+- **Default collection** (`apply_registry`) currently writes the `collections`
+  row **without** `config_json`. A default collection's `fields:` would live in
+  `projects.yaml`? No — the design says `fields:` is a `.collection.yaml` block.
+  The default collection has no `.collection.yaml`. **Decision:** default-
+  collection field schemas are out of this slice's happy path (the N=1 default is
+  `document` with no fields, INV-5). Leave `apply_registry` untouched; only
+  named collections (with a `.collection.yaml`) carry `fields:`. Documented as a
+  known limitation (a deployment wanting fields on its primary corpus declares a
+  named collection — consistent with the design's opt-in story).
+- **Tests** extend `tests/test_collection_registry.py`: a manifest with a
+  `fields:` block round-trips into `get_collection(...)["fields"]`; a manifest
+  with a bad field def still upserts (lenient).
+
+### 3. `collections.py` — unpack `fields` on read + serve via API
+- Add `_fields_from_config(config_json) -> dict | None` (mirror
+  `_enabled_models_from_config`).
+- `get_collection` sets `out["fields"] = _fields_from_config(config_json)`
+  (alongside `enabled_models`). `api_collections.get_col` then serves it with no
+  change. `list_collections` left as-is (facets are SLICE-3).
+- **Tests** extend `tests/test_collection_helpers.py`: `get_collection` exposes
+  `fields`; a no-fields collection → `fields is None` (INV-5).
+
+### 4. `cache.py` — advisory validation at ingest (INV-3)
+- In `_refresh_collection_corpus`, fetch the collection's `fields` schema once
+  (`collections.get_collection(collection_id)`); for each entry, if a schema is
+  present, `problems = metadata_schema.validate(metadata.metadata_dict(entry),
+  fields)` and OR any problems into `metadata_malformed` (warn-log a summary).
+  No schema → behavior identical to today (INV-5).
+- **Tests** `tests/test_metadata_cache.py` (extend): an entry violating an enum
+  field ingests with `metadata_malformed = 1`; a conforming entry → `0`; a
+  collection with no schema → `0` regardless of extra keys.
+
+### 5. Verify · version · ship
+- `pytest` full backend suite green (575 baseline + new).
+- Bump `VERSION` + `frontend/package.json` → **0.48.0**; CHANGELOG minor entry
+  (§20) — non-breaking, opt-in, N=1 unchanged; note write-enforcement lands with
+  SLICE-4/5.
+- Commit (cite design §7.2 SLICE-2), PR on Gitea `origin`, merge to `main`.
+
+## Invariants honored
+INV-3 (read never hard-fails — advisory malformed), INV-5 (no-`fields:`
+unchanged), INV-7 (undeclared keys ride along, never flagged), INV-8 (additive,
+read-mostly; no write-path fork — write enforcement is a later slice).