v0.48.0 — SLICE-2: collection field schema + central validation (§22.4a)

Collections declare a `fields:` schema in `.collection.yaml`; entries carry
typed metadata (enum/tags/text). New central `metadata_schema` module parses
the schema leniently (INV-3) and validates entry values — advisory at read
(corpus mirror flags violations as `metadata_malformed` without hard-failing),
the enforcement point for the write boundary (edit endpoints land SLICE-4/5).

- app/metadata_schema.py: parse_fields (lenient/normalizing) + validate
- registry.parse_collection_manifest reads `fields:` into collection config
- collections.get_collection unpacks `fields`; served by the collection API
- cache._refresh_collection_corpus validates each entry advisory-only

Non-breaking, opt-in: no `fields:` → unchanged (INV-5); default `document`
collection declares none (N=1 unchanged). No DB migration — schema rides in
collections.config_json. SLICE-2 of
docs/design/2026-06-06-configurable-collection-metadata.md §7.2.

Backend suite green (601 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ben Stull
2026-06-07 17:06:04 -07:00
parent 98c276a662
commit e336e31812
12 changed files with 616 additions and 3 deletions
+52
View File
@@ -23,6 +23,58 @@ skip versions are the composition of each intervening adjacent
release's steps in order — no A-to-B path is pre-computed beyond
that.
## 0.48.0 — 2026-06-07
**Minor — collection field schema + central validation (§22.4a SLICE-2).** A
collection can now declare a small **field schema** in its `.collection.yaml`
(`fields:` block), so its entries carry structured, typed metadata — `priority`,
`tags`, and any custom fields the deployment defines. The schema is mirrored into
the collection record and served on the collection API; a new central validator
checks each entry's stored values against it. This is SLICE-2 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2), building on the SLICE-1 sidecars; faceted filtering (SLICE-3) and
the edit UIs (SLICE-4/5) follow.
Non-breaking and opt-in: a collection with **no `fields:`** behaves exactly as
before (INV-5), and the §22.13 default `document` collection declares none
(**N=1 sees no change**). No DB migration — the normalized schema rides in the
existing `collections.config_json` column.
Added:
- **`app/metadata_schema.py`** — the one place that knows a collection's field
shapes. `parse_fields(raw)` normalizes a `.collection.yaml` `fields:` block
**leniently** (INV-3): a bad block or a bad field def (non-mapping, unknown
type, `enum` without a non-empty `values:` list) is skipped with a warning,
never fatal — a typo in one field can't drop the whole collection from the
mirror. `validate(values, fields)` returns advisory `Problem`s (empty =
clean). v1 field types: **`enum`** (single value, controlled by a required
`values:`), **`tags`** (a list; free-form unless `values:` given), **`text`**
(a free string). `ref` / `multi-enum` are deferred (design §2). Keys an entry
carries that the schema does not declare ride along untouched, never flagged
(INV-7).
- **Registry ingest** (`registry.parse_collection_manifest`) reads the `fields:`
block into the collection config (→ `config_json`), beside `enabled_models`.
- **Collection read + API** (`collections.get_collection`) unpacks the schema
and `GET /api/projects/{id}/collections/{cid}` serves it as `fields` (`null`
when unset).
- **Advisory validation at ingest** — the corpus mirror
(`cache._refresh_collection_corpus`) validates each entry against its
collection's schema and flags a violation as `metadata_malformed` **without
blocking the read** (INV-3), OR-ed onto the SLICE-1 sidecar-syntax check.
Write-boundary **enforcement** (a 422 on the metadata-edit endpoints) lands
with those endpoints in SLICE-4/5.
### Upgrade steps (0.47.0 → 0.48.0)
- **No migration, no operator action.** A deployment opts in per collection by
adding a `fields:` block to that collection's `.collection.yaml`; until it
does, behavior is byte-for-byte unchanged.
- A collection's field schema is edited in git for v1 (in-app schema management
is deferred, design D8). After editing `.collection.yaml`, the registry mirror
picks the schema up on its next webhook / reconciler sweep.
- No config change. No content change is required.
## 0.47.0 — 2026-06-07
**Minor — metadata sidecars: storage + dual-read + migration tool (§22.4a