benstull/rfc-app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Spec v0.1.3: supersede per-type-surfaces; split personas; harvest patterns

Browse Source 
- Supersede docs/design/2026-06-06-per-type-surfaces.md (banner added there);
  harvest validation seam, malformed-metadata flag, unknown-fields-ride-along,
  engine-unchanged invariant, N=1 document backcompat
- bdd coverage kept as a future per-type surface over a generic `ref` field
- Schema model = pure collection-config (not type-driven)
- SLICE-0 added: amend binding SPEC.md §22.4a (frontmatter→sidecar,
  type-driven→collection-configured, defer surfaces)
- Split personas: §6 Business Actors/Roles (solution-agnostic) + §10 Product
  Personas (mapped to business roles); renumber accordingly

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ben Stull
2026-06-06 07:34:15 -07:00
parent 3c910e89ab
commit 561cd73760
2 changed files with 158 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/2026-06-06-configurable-collection-metadata.md
+144 -109
View File
@@ -5,8 +5,8 @@
| **Author(s)** | Ben Stull |
| **Reviewers / approvers** | Ben Stull |
| **Status** | `draft` |
| **Version** | v0.1.2 |
| **Source artifacts** | Reference modeled: retired **BDD Release Planner** (`wiggleverse/wiggleverse-ecomm-bdd-release-planner-app`, RETIRED 2026-06-04) · Related spec: [`docs/design/2026-06-05-three-tier-projects-collections.md`](./2026-06-05-three-tier-projects-collections.md) (§22) · Corpus: ecomm Shopify-modeled BDD (`wiggleverse-ecomm-meta/research/shopify`, ~1,238 scenarios) · Supersedes: — |
| **Version** | v0.1.3 |
| **Source artifacts** | Reference modeled: retired **BDD Release Planner** (`wiggleverse/wiggleverse-ecomm-bdd-release-planner-app`, RETIRED 2026-06-04) · Related: [`2026-06-05-three-tier-projects-collections.md`](./2026-06-05-three-tier-projects-collections.md) (§22) · Corpus: ecomm Shopify-modeled BDD (`wiggleverse-ecomm-meta/research/shopify`, ~1,238 scenarios) · **Supersedes:** [`2026-06-06-per-type-surfaces.md`](./2026-06-06-per-type-surfaces.md) |
**Change log**
@@ -14,7 +14,8 @@
| --- | --- | --- | --- |
| 2026-06-06 | v0.1.0 | Initial draft from discovery session OHM-0079.0 | Ben Stull |
| 2026-06-06 | v0.1.1 | Executive Summary → value-only; add §4 Pain Points; renumber per Solution Design template revision | Ben Stull |
| 2026-06-06 | v0.1.2 | Reshuffle (Business Outcomes → §5, business cluster); outcomes restated as business (adoption/diversity), not solution outputs; Business Use Cases rewritten as solution-agnostic actor-goal scenarios (no product reference) | Ben Stull |
| 2026-06-06 | v0.1.2 | Reshuffle (Business Outcomes → §5); outcomes restated as business (adoption/diversity); Business Use Cases → solution-agnostic actor-goal scenarios | Ben Stull |
| 2026-06-06 | v0.1.3 | Supersede the per-type-surfaces draft (harvest validation seam, malformed flag, unknown-field-rides-along, engine-unchanged, N=1 backcompat; bdd coverage kept as a future surface; §22.4a amendment work item); split personas into Business Actors (§6) + Product Personas (§10) | Ben Stull |
---
@@ -44,40 +45,42 @@ rfc-app cannot express or surface structured signal about its content. Tags are
## 5. Targeted Business Outcomes
Business outcomes for rfc-app as a platform — adoption, reach, and diversity of use — **not** solution outputs. (Whether documents carry a priority is a solution output, tracked as a slice's Definition of Done in §13, not here.) These are the business change we expect once the pains in §4 are relieved.
Business outcomes for rfc-app as a platform — adoption, reach, and diversity of use — **not** solution outputs. (Whether documents carry a priority is a solution output, tracked as a slice's Definition of Done in §14, not here.) These are the business change we expect once the pains in §4 are relieved.
| Outcome | Success metric | Baseline → Target | Guardrail (must not regress) | How / when measured |
| --- | --- | --- | --- | --- |
| Teams blocked by missing structured attributes (priority / structured tags) now adopt rfc-app | # organizations on rfc-app; # active users | internal deployments only → external orgs onboard | existing deployments don't churn | deployment registry + usage analytics; quarterly |
| Teams blocked by missing structured attributes now adopt rfc-app | # organizations on rfc-app; # active users | internal deployments only → external orgs onboard | existing deployments don't churn | deployment registry + usage analytics; quarterly |
| The platform hosts a wider variety of workflows and document types | # distinct document/collection types & field schemas in use; # distinct workflow patterns | today's handful (philosophy RFCs, BDD corpus) → broader mix | existing types' experience unchanged | type/schema census across deployments; quarterly |
| Corpus planning happens on-platform rather than in side tools | share of prioritisation/planning done in rfc-app vs spreadsheets | largely off-platform → on-platform | — | operator interviews + usage signals; quarterly |
## 6. Stakeholders / Personas / Actors
## 6. Business Actors / Roles
| Actor | Type | Goal in this design |
| --- | --- | --- |
| Corpus contributor | persona | Set priority and tags on a scenario when proposing/curating it |
| Release planner (operator) | persona | Bulk-prioritise/tag many scenarios quickly to shape a plan |
| Collection owner | operator | Declare the metadata fields a collection supports |
| Prospective adopter (org/team) | persona | Run a new document-type workflow on rfc-app |
| Downstream consumer | system | Read structured per-scenario metadata from the git corpus (e.g. an external release planner) |
| Reader | persona | Read clean scenario docs; filter the catalog by priority/tag |
Real-world roles, **solution-agnostic** — they exist whether or not rfc-app does. The Business Use Cases (§9) are about these roles; the Product Personas (§10) map onto them.
| Role | Responsible for (in the business) |
| --- | --- |
| Standards owner | Owns an organization's RFC / standards / requirements process; decides what's tracked and how |
| Release planner | Decides what work belongs in upcoming releases |
| Requirements author | Proposes and curates the requirements (e.g. BDD scenarios) |
| Requirements consumer | A person or downstream tool that plans or builds from the requirements |
| Reader | Anyone navigating the corpus to find what's relevant to them |
## 7. Scope
- **In scope:** per-collection field schema in `.collection.yaml` (`enum`, `tags`, `text`); per-entry sidecar (`<slug>.meta.yaml`) as source of truth with pure-prose doc bodies; schema-derived faceted left-pane filtering with counts; schema-derived metadata form (detail panel); single + bulk tag/untag with direct commit for authorized roles; dual-read compatibility + a one-shot frontmatter→sidecar migration tool + lazy migration on write.
- **Out of scope:** release ordering, ship status, roadmap emission, `RELEASE-PLAN.md` generation — these stay downstream, consuming sidecars from git. In-app management of field definitions / controlled vocabularies; corpus-wide tag rename/merge/delete. Sub-document (per-scenario-within-a-file) grain. A whole-corpus metadata export endpoint.
- **Non-goals:** rebuilding a bespoke "release" entity in rfc-app — releases are not modeled here at all; metadata is the only primitive.
- **In scope:** per-collection field schema in `.collection.yaml` (`enum`, `tags`, `text`); per-entry sidecar (`<slug>.meta.yaml`) as source of truth with pure-prose doc bodies; central schema validation (advisory at read, enforced at write); a "malformed metadata" catalog flag; schema-derived faceted left-pane filtering with counts; schema-derived metadata form (detail panel); single + bulk tag/untag with direct commit for authorized roles; dual-read compatibility + a one-shot frontmatter→sidecar migration tool + lazy migration on write; an amendment to the binding `SPEC.md` §22.4a contract (see §14 SLICE-0).
- **Out of scope:** release ordering, ship status, roadmap emission, `RELEASE-PLAN.md` generation, and the `specification` *release-planning surface* — all downstream, consuming sidecars from git. In-app management of field definitions / controlled vocabularies; corpus-wide tag rename/merge/delete. Sub-document grain. A whole-corpus metadata export endpoint.
- **Future (recorded, not v1):** a **bdd coverage surface** — a `verifies`-style **`ref` field type** plus a read-derived view mapping features to the spec sections they exercise and flagging gaps (harvested from the superseded per-type-surfaces draft, Part B.2). Built on the generic field machinery; deferred pending product questions (§16 Q4).
- **Non-goals:** rebuilding a bespoke "release" entity in rfc-app — releases are not modeled here at all; metadata is the only primitive. Hard-coding fields per collection *type* — schemas are collection-configured, not type-driven (D9).
## 8. Assumptions · Constraints · Dependencies
- **Assumptions:** git remains the content source of truth and downstream consumers can read the corpus from git; the BDD grain is one markdown file per scenario (already true for the ecomm corpus), so no sub-document parsing is needed.
- **Constraints:** rfc-app is a framework hosting multiple deployments — the upgrade must be **mechanical and non-breaking**, with §20 changelog/upgrade-steps; the hard secrets rule (§6.3) holds; metadata edits must respect scope-role authorization (§22 Part B / S3).
- **Dependencies:** the S3 scope-role resolver (`auth.effective_scope_role`) for edit authorization; the existing git write-through path used by `edit-meta` (§9.5); the §22 collection model (`.collection.yaml`, `cached_rfcs` keyed by `(collection_id, slug)`).
- **Constraints:** rfc-app is a framework hosting multiple deployments — the upgrade must be **mechanical and non-breaking**, with §20 changelog/upgrade-steps; the hard secrets rule (§6.3) holds; metadata edits must respect scope-role authorization (§22 Part B / S3); the §22.4a "engine unchanged" rule holds (INV-8).
- **Dependencies:** the S3 scope-role resolver (`auth.effective_scope_role`); the existing git write-through path used by `edit-meta` (§9.5); the §22 collection model (`.collection.yaml`, `cached_rfcs` keyed by `(collection_id, slug)`); the binding `SPEC.md` §22.4a contract, which this design amends (SLICE-0).
## 9. Business Use Cases
Business use cases are **solution-agnostic**: they state what an actor wants to accomplish, *why* (the value), and what *success* looks like — with **no reference to rfc-app or any product**. Each is true regardless of how it's met; in principle a person could satisfy it by hand (a concierge first pass) before any software is built, and only later is it automated. They open with "As a … I can … so that …" and state acceptance in business terms.
Business use cases are **solution-agnostic**: they state what an actor (§6) wants to accomplish, *why* (value), and what *success* looks like — with **no reference to rfc-app or any product**. Each could in principle be satisfied by a person by hand (a concierge first pass) before any software. Form: "As a … I can … so that …" + a business-centric scenario.
**BUC-1 — As a release planner, I can prioritise the requirements in a body of work, so that I can decide what belongs in upcoming releases.**
```gherkin
@@ -125,11 +128,22 @@ Scenario: BUC-5 — Find what matters
```
- **Acceptance:** a person narrows a large corpus to the relevant, important subset for their task.
## 10. Product Use Cases
## 10. Product Personas
rfc-app's user types — each an embodiment of one or more Business Roles (§6). The Product Use Cases (§11) are about these personas.
| Product persona | In rfc-app | Maps to business role(s) |
| --- | --- | --- |
| Collection Owner | scope-role Owner on a collection/project; declares the collection's `fields:` schema (by editing `.collection.yaml`) | Standards owner |
| Contributor | scope-role contributor; sets metadata (single + bulk), proposes/curates entries | Requirements author; Release planner |
| Reader | viewer; browses and filters the catalog | Reader |
| Downstream consumer | an external system reading sidecars + `.collection.yaml` from git | Requirements consumer |
## 11. Product Use Cases
```gherkin
Scenario: PUC-1 — Set priority/tags on a scenario (realizes BUC-1, BUC-4)
Given I am a contributor viewing a scenario whose collection defines priority and tags
Given I am a Contributor viewing a scenario whose collection defines priority and tags
When I choose P0 in the priority control and add the tag "checkout"
Then the metadata panel reflects P0 and the checkout tag
And the change is committed directly to the scenario's sidecar
@@ -146,8 +160,8 @@ Scenario: PUC-3 — Filter the catalog by facet (realizes BUC-5, BUC-1)
Then the catalog shows only scenarios matching both
And each facet value shows its result count
Scenario: PUC-4 — A collection declares its fields (realizes BUC-4)
Given a collection owner edits .collection.yaml to add a priority enum field
Scenario: PUC-4 — A Collection Owner declares fields (realizes BUC-4)
Given a Collection Owner edits .collection.yaml to add a priority enum field
When the collection is re-ingested
Then the priority filter and the priority form control appear automatically
@@ -156,42 +170,48 @@ Scenario: PUC-5 — Migrate a collection to clean docs (product-only; enables BU
When the operator runs the frontmatter→sidecar migration
Then each doc body becomes pure prose and a sidecar holds its metadata
And rfc-app reads the collection identically before and after
Scenario: PUC-6 — A malformed entry is visibly fixable (realizes BUC-3)
Given a stored entry whose metadata fails its collection's schema
When the catalog renders
Then the entry still loads (read never hard-fails)
And it is flagged "malformed metadata" so a Contributor can fix it
```
## 11. UX Layout
## 12. UX Layout
### 11.1 Screen: Catalog (left pane) (serves PUC-3)
### 12.1 Screen: Catalog (left pane) (serves PUC-3, PUC-6)
- **Purpose:** browse and filter a collection's entries.
- **Layout (top → bottom):**
- **Search:** full-text box (existing).
- **Faceted filter groups** (one per schema field + state): each is a collapsible group showing per-value **result counts** and multi-select checkboxes; `tags`-type fields include a "filter values…" search box to stay usable at 30+ values. (Chosen layout: faceted groups with counts — validated in brainstorming over flat chips.)
- **States:** happy: facets with counts · empty: "no entries match these filters" with a clear-filters action · loading: skeleton facets · error: "couldn't load facets" with retry.
- **Layout (top → bottom):** full-text search (existing); **faceted filter groups** (one per schema field + state): each a collapsible group with per-value **result counts** and multi-select checkboxes; `tags`-type fields include a "filter values…" search box to stay usable at 30+ values. (Chosen: faceted groups with counts — validated in brainstorming over flat chips.)
- **States:** happy: facets with counts · empty: "no entries match these filters" + clear-filters · loading: skeleton facets · error: retry · **malformed:** entries failing their schema carry a fixable "malformed metadata" marker (parallel to the §22.4c `unreviewed` flag), and are filterable.
### 11.2 Screen: Scenario detail — metadata panel (serves PUC-1)
### 12.2 Screen: Scenario detail — metadata panel (serves PUC-1)
- **Purpose:** view/edit one entry's metadata.
- **Layout:** a panel rendering one control per schema field — `enum` → single-select; `tags` → removable chips + add-tag input (with existing AI suggest where applicable); `text` → text input. The document body renders below as pure prose; metadata never appears inline in the body.
- **States:** read (role without edit) shows values, no controls · edit (authorized) shows controls · saving: inline spinner · error: field-level validation message (e.g. "P5 is not an allowed priority").
- **Layout:** one control per schema field — `enum` → single-select; `tags` → removable chips + add-tag input (with existing AI suggest where applicable); `text` → text input. The document body renders below as pure prose; metadata never appears inline.
- **States:** read (no edit role) shows values, no controls · edit (authorized) shows controls · saving: inline spinner · error: field-level validation message (e.g. "P5 is not an allowed priority").
### 11.3 Screen: Catalog — bulk action bar (serves PUC-2)
### 12.3 Screen: Catalog — bulk action bar (serves PUC-2)
- **Purpose:** apply a field value to many entries at once.
- **Layout:** selecting ≥1 row reveals a sticky action bar: "*N* selected · Set priority ▾ · Add tag ▾ · Remove tag ▾ · Clear". Each action targets one field; applying commits once.
- **States:** none selected: bar hidden · applying: bar shows progress · partial failure: toast naming entries that failed validation, others applied.
- **Layout:** selecting ≥1 row reveals a sticky bar: "*N* selected · Set priority ▾ · Add tag ▾ · Remove tag ▾ · Clear". Each action targets one field; applying commits once.
- **States:** none selected: hidden · applying: progress · partial failure: toast naming entries that failed validation, others applied.
## 12. Technical Design
## 13. Technical Design
### 12.1 Invariants
### 13.1 Invariants
- **INV-1:** The sidecar (`<slug>.meta.yaml`) is the source of truth for entry metadata; `cached_rfcs` is a derived index, fully rebuildable from git.
- **INV-2:** A document body (`.md`) never contains rfc-app metadata once migrated; metadata lives only in the sidecar.
- **INV-3:** Reading a collection never hard-fails on bad metadata — an invalid value against the schema surfaces as a warning and the entry still loads.
- **INV-4:** Metadata writes are authorized by scope-role (contributor+ on the collection); content-body edits keep their existing PR-review path.
- **INV-5:** A collection with no `fields:` block behaves exactly as today (free-form `tags` only) — the feature is additive and opt-in per collection.
- **INV-3:** Reading a collection never hard-fails on bad metadata — an invalid value against the schema surfaces as a warning, the entry still loads, and the catalog flags it (§12.1).
- **INV-4:** Metadata writes are authorized by scope-role (contributor+ on the collection) and validated at the write boundary; content-body edits keep their existing PR-review path.
- **INV-5:** A collection with no `fields:` block behaves exactly as today (free-form `tags` only). The §22.13 generated **default collection is `document`** with no fields → **N=1 deployments see zero change** (the load-bearing backcompat guarantee).
- **INV-6:** Dual-read holds throughout: parser reads the sidecar if present, else legacy top-of-doc frontmatter, with identical resulting in-memory records.
- **INV-7:** Unknown / forward-compat keys in a sidecar **ride along untouched** — never dropped on read or rewrite, and never reported as malformed.
- **INV-8:** **Engine unchanged** (§22.4a) — this feature is additive and read-mostly; it never forks the content write path, the propose→branch→PR→graduate lifecycle, threads/flags/chat, or the storage model. Metadata edits reuse the existing `edit-meta` git write-through.
### 12.2 High-level architecture
### 13.2 High-level architecture
```mermaid
flowchart LR
@@ -200,31 +220,33 @@ flowchart LR
MD[slug.md<br/>prose body]
SC[slug.meta.yaml<br/>values]
end
CY --> ING[ingest / parser<br/>validate vs schema]
CY --> ING[ingest / parser<br/>lenient, type-agnostic]
MD --> ING
SC --> ING
ING --> DB[(cached_rfcs<br/>values + facet counts)]
ING --> VAL[metadata_schema.validate<br/>advisory at read]
VAL --> DB[(cached_rfcs<br/>values + facet counts + malformed flag)]
DB --> API[API: schema · list+filter · facets · edit]
API --> FILT[left-pane faceted filters]
API --> PANEL[detail metadata panel]
API --> BULK[bulk select bar]
PANEL -->|direct commit| SC
BULK -->|1 commit| SC
PANEL -->|validate + direct commit| SC
BULK -->|validate + 1 commit| SC
SC -.read from git.-> CONS[downstream consumers]
```
- **ingest/parser** — owns reading `.collection.yaml` schema + sidecars (or legacy frontmatter), validating values, and rebuilding `cached_rfcs`; must never treat the DB as authoritative.
- **API** — owns serving the schema, filtered lists with facet counts, and metadata edits; must never write metadata anywhere but the sidecar in git.
- **ingest/parser** — reads `.collection.yaml` schema + sidecars (or legacy frontmatter), stays **lenient and type-agnostic** (unknown keys ride along, INV-7); rebuilds `cached_rfcs`; never authoritative.
- **`metadata_schema.validate(values, fields) → [problems]`** — the **one** place that knows a collection's required/forbidden fields and each field's shape (the seam harvested from the per-type-surfaces draft, modeled on `registry.py`). **Advisory at ingest** (warn + malformed flag, INV-3); **enforced at the write boundary** (single/bulk edit, INV-4).
- **API** — serves the schema, filtered lists with facet counts + malformed flag, and metadata edits; never writes metadata anywhere but the sidecar in git.
### 12.3 Data model & ownership
### 13.3 Data model & ownership
| Entity | Owned by | Key fields | System of record |
| --- | --- | --- | --- |
| Collection field schema | collection owner | `fields: {name → {type, values?, label}}` in `.collection.yaml` | git |
| Entry metadata values | contributor | sidecar `<slug>.meta.yaml`: lifecycle (`slug,title,state,owners,…`) + schema fields (`priority,tags,…`) | git (sidecar) |
| Derived index | ingest | per-entry values + facet aggregations | `cached_rfcs` (SQLite, derived) |
| Collection field schema | Collection Owner | `fields: {name → {type, values?, label}}` in `.collection.yaml` | git |
| Entry metadata values | Contributor | sidecar `<slug>.meta.yaml`: lifecycle (`slug,title,state,owners,…`) + schema fields (`priority,tags,…`) + any forward-compat keys (INV-7) | git (sidecar) |
| Derived index | ingest | per-entry values + facet aggregations + `malformed` flag | `cached_rfcs` (SQLite, derived) |
**Field types (v1):** `enum` (single-select; controlled by required `values:`), `tags` (multi-value; free-form unless `values:` given), `text` (free string). Unknown types are ignored with a warning (forward-compat).
**Field types (v1):** `enum` (single-select; controlled by required `values:`), `tags` (multi-value; free-form unless `values:` given), `text` (free string). **Future:** `ref` (a typed cross-entry link, the basis for the deferred bdd `verifies`/coverage surface — §7 Future, §16 Q4). Unknown types are ignored with a warning (forward-compat).
**Sidecar example:**
```yaml
@@ -237,14 +259,14 @@ tags: [dashboard, analytics]
owner: hasan
```
### 12.4 Interfaces & contracts
### 13.4 Interfaces & contracts
- **`GET /api/projects/<p>/collections/<c>`** — out: collection incl. `fields` schema. Errors: 404.
- **`GET /api/projects/<p>/collections/<c>/rfcs`** — in: filter params (`?priority=P0&tags=checkout&state=active`, repeatable for multi-value/OR-within-field, AND across fields) · out: entries with metadata values **+ `facets: {field → {value → count}}`**. Errors: 400 on unknown field.
- **`POST /api/projects/<p>/collections/<c>/rfcs/<slug>/meta`** — in: `{field: value, …}` · out: updated values · effect: validate vs schema → write sidecar → commit directly → re-ingest entry. Errors: 403 (role), 422 (invalid value).
- **`POST /api/projects/<p>/collections/<c>/meta/bulk`** — in: `{slugs: […], op: set|add|remove, field, value}` · out: `{applied: […], rejected: [{slug, reason}]}` · effect: validate → write N sidecars → **one** commit → re-ingest. Errors: 403, 422.
- **`GET /api/projects/<p>/collections/<c>/rfcs`** — in: filter params (`?priority=P0&tags=checkout&state=active`; repeatable → OR within a field, AND across fields; `?malformed=true`) · out: entries with values + per-entry `malformed` flag + `facets: {field → {value → count}}`. Errors: 400 on unknown field.
- **`POST /api/projects/<p>/collections/<c>/rfcs/<slug>/meta`** — in: `{field: value, …}` · effect: `metadata_schema.validate` → write sidecar → commit directly → re-ingest. Errors: 403 (role), 422 (invalid value).
- **`POST /api/projects/<p>/collections/<c>/meta/bulk`** — in: `{slugs, op: set|add|remove, field, value}` · out: `{applied, rejected:[{slug,reason}]}` · effect: validate → write N sidecars → **one** commit → re-ingest. Errors: 403, 422.
### 12.5 Per–Product-Use-Case design
### 13.5 Per–Product-Use-Case design
#### PUC-2 — Bulk tag/untag
@@ -253,121 +275,130 @@ sequenceDiagram
actor U as Contributor
participant C as Catalog UI
participant A as API
participant V as metadata_schema
participant G as Git
participant D as cached_rfcs
U->>C: select rows, "Set priority P1"
C->>A: POST /meta/bulk {slugs, set, priority, P1}
A->>A: authz (scope-role) + validate vs schema
A->>A: authz (scope-role)
A->>V: validate values vs schema
A->>G: write N sidecars, 1 commit
A->>D: re-ingest affected entries
A-->>C: {applied, rejected}
C-->>U: rows show P1; toast on any rejected
```
- **Implementation:** reuse the `edit-meta` git write-through, extended to (a) target the sidecar rather than frontmatter and (b) batch N files into one commit. Honors INV-1/INV-4.
- **Implementation:** reuse the `edit-meta` git write-through, extended to target the sidecar and batch N files into one commit. Honors INV-1/INV-4/INV-8.
#### PUC-5 — Migration
- **Implementation:** a tool (framework CLI verb / `tools/` script) walks a collection, and for each entry with legacy frontmatter, writes `<slug>.meta.yaml` from the frontmatter and rewrites `<slug>.md` to the body only — one commit per collection, idempotent. Dual-read (INV-6) means this can run anytime; lazy migration converts stragglers on their first metadata edit.
- **Implementation:** a tool (framework CLI verb / `tools/` script) walks a collection; for each entry with legacy frontmatter it writes `<slug>.meta.yaml` and rewrites `<slug>.md` to the body only — one commit per collection, idempotent, preserving unknown keys (INV-7). Dual-read (INV-6) means it can run anytime; lazy migration converts stragglers on first metadata edit.
### 12.6 Non-functional requirements & cross-cutting concerns
### 13.6 Non-functional requirements & cross-cutting concerns
- **Security & privacy:** metadata edits gated by `auth.effective_scope_role` (contributor+ on the collection); no secrets in sidecars; git history records authorship.
- **Performance & scale:** facet counts computed from the derived DB; must stay responsive at ~1.2k entries with dozens of tag values (indexed value columns / aggregation query).
- **Availability & resilience:** bad metadata never blocks read (INV-3); a failed re-ingest leaves git authoritative and is recoverable by full rebuild.
- **Observability:** log each metadata commit (actor, field, entry count); warn-log schema validation failures encountered on ingest.
- **Accessibility:** facet groups and form controls keyboard-navigable; checkboxes labelled with value + count.
- **Security & privacy:** edits gated by `auth.effective_scope_role`; no secrets in sidecars; git history records authorship.
- **Performance & scale:** facet counts computed from the derived DB; responsive at ~1.2k entries with dozens of tag values (indexed value columns / aggregation).
- **Availability & resilience:** bad metadata never blocks read (INV-3); a failed re-ingest leaves git authoritative, recoverable by full rebuild.
- **Observability:** log each metadata commit (actor, field, entry count); warn-log + count schema-validation failures on ingest.
- **Accessibility:** facet groups and form controls keyboard-navigable; checkboxes labelled value + count.
### 12.7 Key decisions & alternatives considered
### 13.7 Key decisions & alternatives considered
| Decision | Chosen | Alternatives | Why |
| --- | --- | --- | --- |
| Release modeling | Metadata only; releases downstream | First-class release entity in rfc-app; release-typed tags with behavior | Operator pulled ordering/ship-status out of rfc-app; metadata is the only needed primitive |
| Tag system shape | One generic typed-field system (Approach A) | Releases first-class + simple tags; namespaced facets | Tags/priority/custom are all just fields; one mechanism |
| Metadata storage | Sidecar per entry | Top-of-doc frontmatter (today); end-of-doc block; collection index file; DB-only | Clean docs + git-visible to consumers + locality per scenario |
| Left-pane filtering | Faceted groups with counts | Flat facet chips | Scales to the ~1.2k-scenario, many-tag ecomm corpus |
| Edit governance | Direct commit for authorized roles (bulk = 1 commit) | PR per change | Bulk planning is impractical via PR-per-toggle |
| Mgmt UI | Deferred; edit `.collection.yaml` in git | In-app field/vocab management in v1 | Smallest coherent v1 |
| Release modeling | Metadata only; releases downstream | First-class release entity; release-typed tags with behavior | Operator pulled ordering/ship-status out of rfc-app |
| Tag system shape | One generic typed-field system (Approach A) | Releases first-class + simple tags; namespaced facets | Tags/priority/custom are all just fields |
| Schema model (D9) | **Pure collection-config** | Type-driven hard-coded schemas (per-type-surfaces draft) | Flexible, data-driven; a type shouldn't fix the fields |
| Metadata storage | Sidecar per entry | Frontmatter (today + per-type-surfaces draft); end-of-doc; index file; DB-only | Clean docs + git-visible + per-entry locality |
| Left-pane filtering | Faceted groups with counts | Flat facet chips | Scales to the ~1.2k-scenario, many-tag corpus |
| Edit governance | Direct commit for authorized roles (bulk = 1 commit) | PR per change | Bulk planning impractical via PR-per-toggle |
| bdd coverage (D10) | Future per-type surface over a `ref` field | Build now; drop entirely | Valuable but not v1; needs product questions (Q4) |
### 12.8 Testing strategy
### 13.8 Testing strategy
Unit tests for: schema parsing (`fields:` block, all types, missing block); sidecar read/write round-trip; dual-read equivalence (frontmatter vs sidecar produce identical records); schema validation (reject bad enum, accept free-form tag); facet aggregation; bulk op (set/add/remove, single commit, partial-rejection). Two-tier local-Docker→PPE for the API + git write-through. "Tested" = the PUC acceptance scenarios pass plus the migration is proven idempotent and reversible-on-read.
Unit: schema parsing (all types, missing block); sidecar round-trip incl. unknown-key preservation (INV-7); dual-read equivalence (INV-6); validation (reject bad enum, accept free-form tag); malformed-flag derivation; facet aggregation; bulk op (set/add/remove, single commit, partial-rejection). Two-tier local-Docker→PPE for API + git write-through. "Tested" = the PUC acceptance scenarios pass + migration proven idempotent and reversible-on-read.
### 12.9 Failure modes, rollback & flags
### 13.9 Failure modes, rollback & flags
- **Failure mode:** invalid value committed out-of-band → on ingest, warn + load entry with the raw value flagged (INV-3); not surfaced as a filter facet count error.
- **Failure mode:** re-ingest fails after commit → git is authoritative; full rebuild recovers.
- **Migration rollback:** dual-read means an un-migrated or partially-migrated corpus still works; the migration commit is revertible.
- **Feature flag:** the feature is inherently opt-in per collection (INV-5) — no global flag needed; absent a `fields:` block, behavior is unchanged.
- **Invalid value committed out-of-band** → ingest warns + loads with the value flagged malformed (INV-3).
- **Re-ingest fails after commit** → git authoritative; full rebuild recovers.
- **Migration rollback:** dual-read keeps an un-/partly-migrated corpus working; the migration commit is revertible.
- **Feature flag:** inherently opt-in per collection (INV-5) — no global flag; absent a `fields:` block, behavior is unchanged.
## 13. Delivery Plan
## 14. Delivery Plan
### 13.1 Approach / strategy
### 14.1 Approach / strategy
Build the storage/compat foundation first (sidecars + dual-read + migration) so nothing breaks, then the schema, then read (filtering), then write (single, bulk). Each slice is shippable and non-breaking.
Amend the binding contract first, then build storage/compat, then schema, then read, then write. Each build slice is shippable and non-breaking.
### 13.2 Slicing plan
### 14.2 Slicing plan
#### SLICE-1 — Sidecar storage + dual-read + migration → completes PUC-5
#### SLICE-0 — Amend `SPEC.md` §22.4a (contract) → unblocks the rest
- **Depends on:** —
- **Definition of done:** parser reads sidecar-if-present else legacy frontmatter (INV-6); migration tool splits frontmatter→sidecar idempotently; existing collections load byte-identically; tests green.
- **Definition of done:** §22.4a reframed — item 1 (entry schema) is **collection-configured sidecar fields**, not type-driven frontmatter; item 3 (type surfaces) deferred to a future design (bdd coverage recorded); the per-type-surfaces draft marked superseded; §20 changelog entry. *Doc-only; no code.*
#### SLICE-2 — Collection field schema + validation → completes PUC-4
#### SLICE-1 — Sidecar storage + dual-read + migration → completes PUC-5, PUC-6
- **Depends on:** SLICE-0
- **DoD:** parser reads sidecar-else-legacy (INV-6), preserves unknown keys (INV-7); migration tool idempotent; existing collections load byte-identically; malformed flag derived; tests green.
#### SLICE-2 — Collection field schema + central validation → completes PUC-4
- **Depends on:** SLICE-1
- **Definition of done:** `.collection.yaml fields:` parsed (`enum`/`tags`/`text`); values validated on read (warn) and on write (reject); schema served via the collection API; no-`fields:` collections unchanged (INV-5).
- **DoD:** `.collection.yaml fields:` parsed; `metadata_schema.validate` advisory at read / enforced at write; schema served via the collection API; no-`fields:` collections unchanged (INV-5).
#### SLICE-3 — Faceted left-pane filtering (read) → completes PUC-3
- **Depends on:** SLICE-2
- **Definition of done:** list endpoint returns facet counts + honors filter params; left pane renders faceted groups with counts and tag-value search; filters compose (AND across fields).
- **DoD:** list endpoint returns facet counts + honors filter params (incl. `malformed`); left pane renders faceted groups with counts + tag-value search; filters compose (AND across fields).
#### SLICE-4 — Single-entry metadata edit → completes PUC-1
- **Depends on:** SLICE-2
- **Definition of done:** detail metadata panel renders schema controls; `POST …/meta` validates, direct-commits the sidecar, re-ingests; authorized by scope-role (INV-4); lazy-migrates a legacy entry on first edit.
- **DoD:** detail panel renders schema controls; `POST …/meta` validates, direct-commits, re-ingests; scope-role gated (INV-4); lazy-migrates a legacy entry on first edit.
#### SLICE-5 — Bulk tag/untag → completes PUC-2
- **Depends on:** SLICE-3, SLICE-4
- **Definition of done:** multi-select + bulk action bar; `POST …/meta/bulk` applies set/add/remove as one commit; partial-rejection reported.
- **DoD:** multi-select + bulk bar; `POST …/meta/bulk` applies set/add/remove as one commit; partial-rejection reported.
### 13.3 Rollout / launch plan
### 14.3 Rollout / launch plan
Pre-v1, single production: ship slices in order to the RFC deployment; each minor-version bump carries §20 changelog + upgrade steps. The opt-in-per-collection nature (INV-5) means a deployment adopts it only when it declares a `fields:` block and (optionally) runs the migration.
Pre-v1, single production: ship slices in order to the RFC deployment; each minor bump carries §20 changelog + upgrade steps. Opt-in per collection (INV-5): a deployment adopts it only by declaring a `fields:` block and (optionally) running the migration.
### 13.4 Risks & mitigations
### 14.4 Risks & mitigations
| Risk | L/I | Mitigation |
| --- | --- | --- |
| Frontmatter→sidecar migration corrupts content | L/H | Dual-read; idempotent, revertible migration; body-byte-identity test |
| Doubling file count (sidecars) clutters corpus | M/L | Docs stay clean; sidecars are small/co-located; acceptable for one-file-per-scenario corpora |
| Direct-commit metadata edits bypass review | M/M | Scope-role gate (INV-4); content-body edits still PR'd; full git audit trail |
| Overlap/conflict with §22 S6 "type modules" | M/M | Position as §23, generalizing S6's per-type frontmatter; reconcile at the S6 SPEC merge |
| Amending binding §22.4a destabilises a shipped contract | M/M | SLICE-0 doc-only, reviewed; dual-read keeps runtime non-breaking; supersede note preserves rationale |
| Frontmatter→sidecar migration corrupts content | L/H | Dual-read; idempotent, revertible migration; body-byte-identity + unknown-key tests |
| Doubling file count (sidecars) clutters corpus | M/L | Docs stay clean; sidecars small/co-located |
| Direct-commit metadata edits bypass review | M/M | Scope-role gate (INV-4); content-body edits still PR'd; git audit trail |
| Facet aggregation slow at scale | L/M | Compute from indexed derived DB; measure at ~1.2k entries |
## 14. Traceability matrix
## 15. Traceability matrix
| Pain | Business UC | Product UC | Slice | Tests |
| --- | --- | --- | --- | --- |
| PP-5 | BUC-3 | PUC-5 | SLICE-1 | `test_dual_read_equiv`, `test_migration_idempotent` |
| PP-7 | BUC-4 | PUC-4 | SLICE-2 | `test_schema_parse`, `test_validation` |
| — (contract) | — | — | SLICE-0 | (doc review) |
| PP-5 | BUC-3 | PUC-5, PUC-6 | SLICE-1 | `test_dual_read_equiv`, `test_migration_idempotent`, `test_unknown_keys_preserved` |
| PP-7 | BUC-4 | PUC-4 | SLICE-2 | `test_schema_parse`, `test_validate` |
| PP-2 | BUC-5, BUC-1 | PUC-3 | SLICE-3 | `test_facet_counts`, `test_filter_compose` |
| PP-1, PP-3 | BUC-1, BUC-4 | PUC-1 | SLICE-4 | `test_single_meta_commit`, `test_authz` |
| PP-4 | BUC-2 | PUC-2 | SLICE-5 | `test_bulk_one_commit`, `test_partial_reject` |
| PP-6 | BUC-3 | (consumer reads git) | — | `test_sidecar_schema_stable` |
## 15. Open Questions & Decisions log
## 16. Open Questions & Decisions log
**Open**
| # | Question | Owner | Blocks |
| --- | --- | --- | --- |
| Q1 | Do downstream consumers read sidecars from git, via rfc-app API, or both? (leaning git) | Ben | nothing v1 |
| Q2 | Should a `multi-enum` type (multi-select controlled) ship in v1 or later? | Ben | SLICE-2 scope |
| Q3 | Exact §23 placement / reconciliation with §22 S6 type-modules | Ben | SPEC merge |
| Q1 | Do downstream consumers read sidecars from git, via API, or both? (leaning git) | Ben | nothing v1 |
| Q2 | Ship `multi-enum` (multi-select controlled) in v1 or later? | Ben | SLICE-2 scope |
| Q3 | Exact §22.4a amendment wording + the future-surfaces home | Ben | SLICE-0 |
| Q4 | bdd coverage: `ref` field grammar + does a coverage view honor §22's no-cross-collection-join rule as hyperlinks? | Ben | future surface |
**Resolved**
| # | Decision | Resolution | Date |
| --- | --- | --- | --- |
| D1 | Release behaviors (ordering, ship status, roadmap emit) | Out of rfc-app; downstream | 2026-06-06 |
| D1 | Release behaviors | Out of rfc-app; downstream | 2026-06-06 |
| D2 | Tag system shape | Approach A — one generic typed-field system | 2026-06-06 |
| D3 | Metadata grain | Per entry (corpus already one file per scenario) | 2026-06-06 |
| D4 | Schema location | `.collection.yaml` `fields:` block | 2026-06-06 |
@@ -375,12 +406,16 @@ Pre-v1, single production: ship slices in order to the RFC deployment; each mino
| D6 | Left-pane filtering | Faceted groups with counts | 2026-06-06 |
| D7 | Edit governance | Direct commit for authorized roles; bulk = 1 commit | 2026-06-06 |
| D8 | Management scope | Deferred; edit `.collection.yaml` in git for v1 | 2026-06-06 |
| D9 | Schema model | Pure collection-config; **not** type-driven (supersedes per-type-surfaces) | 2026-06-06 |
| D10 | bdd coverage | Keep as a **future** per-type surface over a `ref` field; not v1 | 2026-06-06 |
| D11 | per-type-surfaces draft | **Superseded** by this spec; §22.4a to be amended (SLICE-0) | 2026-06-06 |
## 16. Glossary & References
## 17. Glossary & References
- **Sidecar** — `<slug>.meta.yaml`, the per-entry metadata file that is the source of truth; keeps the `.md` body pure prose.
- **Field schema** — the `fields:` block in `.collection.yaml` declaring a collection's typed metadata fields.
- **Facet** — a schema field surfaced as a left-pane filter group with per-value counts.
- **Malformed metadata** — stored values that fail their collection's schema; flagged in the catalog, never a hard read failure (INV-3).
- **Downstream consumer** — an external tool (e.g. a release planner) that reads corpus metadata from git; rfc-app does not model releases.
- **References:** retired BDD Release Planner (`wiggleverse-ecomm-bdd-release-planner-app`); §22 three-tier design (`docs/design/2026-06-05-three-tier-projects-collections.md`); SPEC §7.1 (left-pane filter), §9.5 (edit-meta), §20 (versioning), §22 Part B / S3 (scope-role).
- **References:** retired BDD Release Planner; superseded per-type-surfaces draft (`2026-06-06-per-type-surfaces.md`); §22 three-tier design; `SPEC.md` §7.1 (left-pane filter), §9.5 (edit-meta), §20 (versioning), §22.4a (per-type contract — to be amended), §22 Part B / S3 (scope-role).
```
docs/design/2026-06-06-per-type-surfaces.md
+14
View File
@@ -1,5 +1,19 @@
# Draft spec — §22.4a per-type surfaces (the last S6 item)
> # ⛔ SUPERSEDED (2026-06-06)
>
> This draft is **superseded by**
> [`2026-06-06-configurable-collection-metadata.md`](./2026-06-06-configurable-collection-metadata.md),
> which reframes §22.4a item 1 as **collection-configured** metadata in
> **sidecars** (not type-driven frontmatter) and defers item 3's surfaces.
> Harvested into the successor: the validation seam (A.1), the malformed-metadata
> catalog flag (A.5), unknown-fields-ride-along (C.1), the engine-unchanged rule
> (§0), and the N=1 `document` backcompat anchor (A.2). The **bdd coverage**
> capability (`feature`/`verifies` → coverage view, Part B.2) is preserved there
> as a *future* per-type surface over a generic `ref` field. The binding
> `SPEC.md` §22.4a contract is to be amended by the successor's SLICE-0. Kept for
> historical rationale; do not build from this document.
> **Status:** discovery/spec pass — *not yet sliced into a shipped release.*
> Author session: 0083 (2026-06-06). This document is the spec pass the §22 S6
> remainder called for: it specifies **§22.4a item 1** (the per-type entry