Merge pull request '§22 S6 (last item): spec pass for per-type surfaces (§22.4a items 1 & 3)' (#25) from spec/s7-per-type-surfaces into main

2026-06-06 09:16:07 +00:00
parent 1c17fecea3 281dd29e62
commit b7e23a01f8
2 changed files with 361 additions and 0 deletions
@@ -614,6 +614,15 @@ means the deployment runs and either gains a capability or provably loses none
  place). **Usable end-state:** the model is fully realized and merged into
  `SPEC.md`. **Completes:** type-specific scenarios (added in S6, beyond Part C's
  role focus).
  - *Shipped in S6 core (v0.45.0):* the SPEC merge, per-collection
    `enabled_models`, the type-driven entry noun (§22.4a item 2).
  - *Shipped as the S6 remainder (v0.46.0):* request-to-join + the
    cross-collection inbox (§22.8).
  - *Spec'd, not yet built — the last S6 item:* the per-type **frontmatter
    schemas** (§22.4a item 1) and **surfaces** (§22.4a item 3, the
    `specification` release-planning + `bdd` scenario/coverage views). The
    discovery/spec pass + BDD scenarios + slicing (S7a–S7c) are in
    [`2026-06-06-per-type-surfaces.md`](./2026-06-06-per-type-surfaces.md).
 ### Slice → scenario index (the inverse of the `@S<n>` tags)
@@ -0,0 +1,352 @@
 # Draft spec — §22.4a per-type surfaces (the last S6 item)
 > **Status:** discovery/spec pass — *not yet sliced into a shipped release.*
 > Author session: 0083 (2026-06-06). This document is the spec pass the §22 S6
 > remainder called for: it specifies **§22.4a item 1** (the per-type entry
 > **frontmatter schema**) and **§22.4a item 3** (the per-type **surfaces**) for
 > the `specification` and `bdd` collection types, with BDD-style acceptance
 > scenarios and a delivery slicing, so a later coding session can build them
 > against a written contract rather than improvising.
 >
 > It is the sibling of
 > [`2026-06-05-three-tier-projects-collections.md`](./2026-06-05-three-tier-projects-collections.md)
 > (the three-tier model, S1–S6 core, shipped through v0.46.0) and refines, in
 > implementable detail, what `SPEC.md` §22.4a states at the contract level.
 > §22.4a **item 2** (the type-driven entry noun / terminology) shipped in the S6
 > core (v0.45.0) and is out of scope here.
 ## 0. Why this is its own pass
 `SPEC.md` §22.4a says a collection's immutable `type` selects exactly three
 things: (1) the entry **frontmatter schema**, (2) the **terminology**, and (3)
 the **type-specific surfaces**. The S6 core shipped (2) and merged the contract
 into `SPEC.md`; it also shipped the type *plumbing* — `collections.type` is
 immutable, validated (`registry.VALID_TYPES = {document, specification, bdd}`),
 and drives the entry noun. What it did **not** ship is any *behavior* keyed on
 type beyond the noun: every type today parses the same §2 baseline frontmatter
 (`backend/app/entry.py` is type-agnostic and lenient about unknown keys) and
 renders the same §7 catalog with no type-specific surface.
 Items 1 and 3 were deliberately deferred at v0.45.0 (CHANGELOG: "they want a
 discovery/spec pass first, lacking BDD scenarios in Part C"). The three-tier
 design doc's Part C scenarios are all about **roles** (C.1–C.3); there are no
 scenarios describing what a `specification` release-planning view *does* or what
 a `bdd` coverage view *shows*. This document supplies them.
 The governing constraint from §22.4a, which every proposal below honors:
 > Type does not change the **engine** — every type uses the same content repo
 > (§22.3), the same propose→branch→PR→discuss→graduate lifecycle (§§9–13), the
 > same threads, flags, and chat. … the engine itself treats every entry as
 > markdown + frontmatter regardless of type.
 So a per-type surface is **additive and read-mostly**: it reads the (now
 type-aware) frontmatter and presents a derived view. It never forks the write
 path, the PR lifecycle, or the storage model.
 ---
 # Part A — Item 1: the per-type frontmatter schema
 ## A.1 Where validation lives today, and where it should land
 `entry.py:parse()` reads a fixed set of §2 baseline fields and is **lenient**:
 unknown keys are ignored, future fields ride along untouched (its own docstring
 says so). That leniency is the seam. The per-type schema is layered as a
 **validator**, not a parser rewrite:
 - `entry.py` keeps parsing the union of all known fields into the `Entry`
  dataclass (add the new optional fields below; absent → `None`/default, exactly
  as `models`/`funder`/`unreviewed` already do). The parser stays type-agnostic.
 - A new **`entry_schema.py`** module exposes `validate(entry, collection_type)
  -> list[str]` returning human-readable problems (empty = valid). It is the one
  place that knows which fields a type **requires**, which it **forbids**, and
  the **enum/shape** of each.
 - Validation is **advisory at parse, enforced at the write boundary.** The
  propose/edit/PR-merge paths (§9.1, §22.4b) call `entry_schema.validate` and
  surface problems the way the propose modal already surfaces field errors. A
  malformed historical file still *parses* (we never hard-fail a read — a
  deployment's existing corpus must keep loading), but the catalog flags it
  (§A.4) and the next write must fix it.
 This mirrors how visibility/initial_state are validated centrally in
 `registry.py` rather than at each call site.
 ## A.2 `document` — unchanged (the §2 baseline)
 `document` is the baseline: the §2 fields exactly as today
 (`slug, title, state, id, repo, proposed_by, proposed_at, owners, arbiters,
 tags`, plus the §6.6/§6.7 `models`/`funder` and §22.4c `unreviewed`/`reviewed_*`).
 No new fields, no type-specific surface. The §22.13 generated default collection
 is `document`, so **N=1 deployments see zero change** — the load-bearing
 backcompat guarantee.
 ## A.3 `specification` — versioned-spec metadata
 A `specification` entry is a versioned technical spec (the archetype is this
 framework's own `SPEC.md`). Frontmatter **adds** (all optional at parse,
 required/validated per A.1 at write):
 | Field | Shape | Meaning | Required when |
 |---|---|---|---|
 | `spec_version` | semver string (`MAJOR.MINOR.PATCH`) | the entry's own version | `state = active` |
 | `lifecycle` | enum `draft \| active \| superseded` | spec lifecycle, **orthogonal to** the §2.4 entry `state` | always (defaults `draft`) |
 | `supersedes` | list of slugs (in this collection) | specs this one replaces | optional |
 Notes / decisions:
 - **`lifecycle` ≠ `state`.** The §2.4 `state` (super-draft/active/withdrawn) is
  the *engine's* workflow position; `lifecycle` is the *spec's* editorial status.
  An `active` (graduated) entry can be `lifecycle: draft` (published but not yet
  ratified) or `superseded`. Keeping them orthogonal avoids overloading the
  shared state machine (the §22.4a "engine unchanged" rule).
 - **`supersedes` is validated as in-collection slugs** (§22.14 §2: slugs are
  unique *per collection*). A `superseded` lifecycle with no inbound
  `supersedes` from a newer entry is a soft warning in the surface, not a write
  error (the replacement may land later).
 - `spec_version` uses the same semver vocabulary as the framework `VERSION`/§20
  so the release surface (A.5 / Part B) can sort and group.
 ## A.4 `bdd` — feature/scenario metadata
 A `bdd` entry states a feature as Given/When/Then scenarios. Frontmatter
 **adds**:
 | Field | Shape | Meaning | Required when |
 |---|---|---|---|
 | `feature` | string | the feature's one-line statement (the "In order to / As a / I want" intent) | `state = active` |
 | `verifies` | list of refs | the `specification` entries/sections this feature exercises | optional |
 | `scenarios` | derived, **not** frontmatter | count/list parsed from the body's `Scenario:` blocks | n/a |
 Notes / decisions:
 - **`verifies` is a cross-collection ref.** A ref is `"<collection>/<slug>"` or
  `"<collection>/<slug>#<anchor>"`. The default `<collection>` is a sibling
  `specification` collection in the same project; an unqualified `<slug>` means
  "a spec slug in this project's specification collection" (resolved at render).
  This is the one place a `bdd` surface reaches across collections — and §22's
  "no app surface joins across collections" rule (`SPEC.md` line 5016) is
  **honored**: `verifies` is a *declared link rendered as a hyperlink*, not a
  query that fuses two corpora. The coverage view (B.2) aggregates these links
  but each entry still lives in exactly one collection.
 - **Scenarios are parsed from the body, not frontmatter.** Gherkin-style
  `Scenario:` / `Given`/`When`/`Then` lines in the markdown body are the source
  of truth; the surface counts and lists them. This keeps the authoring
  experience plain-markdown (the engine's invariant) — no structured
  scenario-editor write path.
 - `bdd` collections default `initial_state: active` (§22.4b) so a feature lands
  active-but-`unreviewed`; the schema validator therefore requires `feature` for
  active entries, which is every freshly-landed `bdd` entry.
 ## A.5 Schema surfacing in the existing chrome
 Item-1 work is mostly invisible plumbing, but two small surfaces make it real
 without waiting for Part B:
 1. **Propose/edit validation** — the propose modal and edit-branch flow run
   `entry_schema.validate` for the collection's type and block submit on errors
   (e.g. proposing into a `specification` collection without a `lifecycle`).
 2. **A "malformed frontmatter" catalog flag** — the §7 catalog marks entries
   whose stored frontmatter fails its type's schema (parallel to the §22.4c
   `unreviewed` filter), so a corpus migrated from `document`→… or hand-edited
   is visibly fixable.
 ---
 # Part B — Item 3: the type-specific surfaces
 A surface is an **additional view** layered on the shared §7 catalog +
 §8 entry view, selected on `collection.type`. It is read-derived from
 frontmatter + body; it adds no write path the engine doesn't already have.
 ## B.1 `specification` → the release-planning surface
 §22.4a: "group entries/changes into versioned releases with a changelog +
 §20-style upgrade-steps per release." Concretely, a per-collection
 **Releases** view at `/p/<project>/c/<collection>/releases`:
 - **A release** is a named, ordered version (e.g. `0.46.0`) with: the set of
  spec entries at a given `spec_version`/`lifecycle`, a changelog body, and an
  optional upgrade-steps block (the §20.4 RFC-2119 convention reused verbatim).
 - **Source of truth = the content repo**, per §22.2/§22.3. A release is a file
  in the collection's subfolder (proposal: `releases/<version>.md`,
  frontmatter `version` + `released_at` + `entries: [slug@spec_version, …]`,
  body = changelog + upgrade-steps). The registry mirror caches a `releases`
  table the way it caches `collections` — git is truth, the table is a cache
  (§22.2 "never written except by the mirror").
 - **The view** lists releases newest-first; each expands to its changelog +
  upgrade-steps and the entries it cut. An Owner (scope-role, §22.6) can cut a
  new release (a bot-committed file, exactly like create-collection commits a
  manifest — §22 S5 pattern); contributors read.
 - **Reuse, don't reinvent:** the changelog + upgrade-steps renderer is the same
  markdown the framework's own `CHANGELOG.md`/§20.4 uses; the "cut a release"
  write is the §22 S5 bot-commit-then-mirror pattern.
 Deliberately **out of this surface** (deferred): cross-release diffing, automated
 version bumping, dependency graphs between specs. The MVP is "see the releases,
 their changelog, their upgrade-steps, and what each contained."
 ## B.2 `bdd` → the scenario/acceptance + coverage surfaces
 §22.4a: "a scenario/acceptance view and a coverage view mapping features to the
 spec sections they exercise." Two read-derived views:
 1. **Scenario/acceptance view** (per entry, on the §8 entry page): renders the
   body's parsed `Scenario:` blocks as a structured checklist — each scenario's
   Given/When/Then, plus the entry's `feature` line as the header. No new
   storage; pure body parse. This is the `bdd` analogue of the `document`
   entry's prose view.
 2. **Coverage view** (per collection, at
   `/p/<project>/c/<collection>/coverage`): a matrix of **features → the spec
   entries/sections they `verifies`**. Rows are this collection's `bdd` entries;
   columns (or grouped rows) are the referenced `specification` entries. Cells
   show "covered / declared-but-spec-missing / spec-section-with-no-feature".
   The view aggregates the `verifies` links (A.4) across the collection but
   renders each as a hyperlink into the spec collection — it does not fuse the
   corpora (the §22 cross-collection rule, B/A.4).
 Deliberately **out of this surface** (deferred): executing scenarios, CI/test
 result ingestion, auto-detecting coverage from code. The MVP maps *declared*
 coverage (`verifies`), surfacing gaps for humans to close.
 ## B.3 How a surface is selected and routed
 - The collection payload already carries `type` and `entry_noun`
  (`GET /api/projects/:id/collections/:cid`). The frontend's `ProjectLayout` /
  collection chrome reads `type` and mounts the type's surface routes
  (`releases` for `specification`; `coverage` for `bdd`) alongside the shared
  catalog. `document` mounts none.
 - Backend: a per-type router group (`api_releases.py`, `api_coverage.py`)
  guarded by the same §22.5 read gates as the rest of the collection; the
  release-cut write reuses `auth.is_collection_superuser` / the S5 bot pattern.
 - The "per-type module the framework selects on `collection.type`" (§22.4a) is
  realized as: backend `entry_schema.py` (item 1) + the two router groups
  (item 3), and frontend a `typeModules[type]` map of `{ schema, surfaces }`.
  Adding a future type = a new map entry + enum value, no rebuild (§22.4a "open
  set").
 ---
 # Part C — Behavioral scenarios (BDD)
 > Tagged for the proposed slices in Part D (`@S7a` = item 1 schemas; `@S7b` =
 > specification releases; `@S7c` = bdd surfaces). These are the acceptance gate
 > the implementing session writes tests against, in the Part C style of the
 > three-tier doc.
 ## C.1 Per-type frontmatter schema (`@S7a`)
 ```gherkin
 Scenario: document collection is unchanged
  Given a "document" collection
  When a contributor proposes an entry with the §2 baseline frontmatter only
  Then the proposal is accepted with no schema error
 Scenario: specification entry requires a lifecycle
  Given a "specification" collection
  When a contributor proposes an entry with no `lifecycle`
  Then it defaults to lifecycle "draft" and is accepted
  And when an Owner graduates it to active without a `spec_version`
  Then the write is blocked with "spec_version is required for an active specification"
 Scenario: bdd entry requires a feature statement once active
  Given a "bdd" collection whose initial_state is "active"
  When a contributor proposes an entry with no `feature`
  Then the write is blocked with "feature is required for a bdd entry"
 Scenario: unknown future field still rides along
  Given any collection
  When an entry carries a frontmatter key no schema names
  Then it parses unchanged and is not reported as malformed
 Scenario: malformed existing entry loads but is flagged
  Given a stored specification entry missing a required field
  When the catalog renders
  Then the entry still loads (read never hard-fails)
  And the catalog marks it "malformed frontmatter"
 ```
 ## C.2 specification release-planning surface (`@S7b`)
 ```gherkin
 Scenario: an Owner cuts a release
  Given a "specification" collection with two active entries
  When the collection Owner cuts release "1.0.0" with a changelog and upgrade-steps
  Then a releases/1.0.0.md file is committed to the content repo
  And the registry mirror caches the release
  And the Releases view lists "1.0.0" newest-first with its changelog + upgrade-steps
 Scenario: a contributor reads releases but cannot cut one
  Given a contributor (not Owner) in the collection
  Then the Releases view is read-only (no "Cut release" control)
 Scenario: upgrade-steps render with the §20.4 convention
  Given a release whose body uses MUST/SHOULD/MAY upgrade-steps
  Then they render with the same normative-language styling as CHANGELOG.md
 ```
 ## C.3 bdd scenario + coverage surfaces (`@S7c`)
 ```gherkin
 Scenario: an entry's scenarios render as an acceptance checklist
  Given a "bdd" entry whose body has two Scenario: blocks
  When the entry page renders
  Then it shows the `feature` header and both scenarios' Given/When/Then
 Scenario: coverage maps features to the specs they verify
  Given a "bdd" entry that `verifies: ["spec/auth#sessions"]`
  And a sibling "specification" collection "spec" containing entry "auth"
  When the coverage view renders
  Then a row links the feature to spec/auth#sessions as covered
 Scenario: a declared ref to a missing spec is surfaced as a gap
  Given a "bdd" entry that `verifies: ["spec/ghost"]` where no such spec exists
  Then the coverage view marks that ref "declared but spec missing"
 Scenario: coverage does not fuse corpora
  Then each cell is a hyperlink into the spec collection
  And no entry from the spec collection is listed as if it belonged to the bdd collection
 ```
 ---
 # Part D — Delivery slicing
 Each slice lands a usable increment + a runnable acceptance gate (`--tags @S7x`),
 in the three-tier doc's slicing style. Suggested order (item 1 first — the
 surfaces read its fields):
 - **S7a — per-type frontmatter schema (item 1).** `entry_schema.py` +
  the new optional `Entry` fields + write-boundary validation + the catalog
  "malformed" flag. **Usable:** proposing into a typed collection is validated;
  N=1 `document` unchanged. **Completes:** `@S7a` (C.1). *Non-breaking, additive.*
 - **S7b — specification release planning (item 3a).** `releases/<v>.md` storage
  + registry mirror + `api_releases.py` + the Releases view + cut-release write.
  **Usable:** a spec collection has versioned releases with changelog +
  upgrade-steps. **Completes:** `@S7b` (C.2).
 - **S7c — bdd scenario + coverage surfaces (item 3b).** Body scenario parser +
  the per-entry acceptance view + the per-collection coverage view +
  `api_coverage.py`. **Usable:** a bdd collection shows scenarios and declared
  coverage. **Completes:** `@S7c` (C.3).
 All three are **additive** (new optional frontmatter, new tables that are pure
 caches, new read views): each is a minor, non-breaking release, and a `document`
 N=1 deployment is unaffected by any of them. None touches the engine, the PR
 lifecycle, or the role model — they consume the §22 three-tier + §22.6 role
 work already shipped.
 ## D.1 Open questions for the implementing session
 1. **Release identity vs. entry `spec_version`.** Should a release's `entries`
   pin exact `slug@spec_version` (immutable snapshot) or just slugs (live)? This
   doc proposes the pinned snapshot; confirm against a real spec-collection
   workflow before building S7b.
 2. **`verifies` ref grammar.** `"<collection>/<slug>#<anchor>"` is proposed;
   anchor resolution into a spec entry's section needs the spec body to carry
   stable anchors. May want a lightweight `## §n` anchor convention on
   `specification` entries first.
 3. **Whether `lifecycle` belongs in the shared state machine after all.** Kept
   orthogonal here; revisit if product wants `superseded` to gate the catalog.
 These are genuine product decisions a discovery/spec session (or the operator)
 should settle before S7b/S7c code; S7a (schemas) is unblocked and buildable now.