Compare commits

..

63 Commits

Author SHA1 Message Date
Ben Stull f7b93d797c Merge pull request 'feat(§8.12/§8.3): main-view Ask cuts an edit branch and runs the question there (v0.54.0)' (#49) from fix-main-view-ask into main 2026-06-09 11:20:30 +00:00
Ben Stull 24596842ea feat(§8.12/§8.3): main-view Ask cuts an edit branch and runs the question there (v0.54.0)
The AI "Ask" affordance (selection tooltip + prompt bar) had no response
surface on an entry's canonical `main` view: RFCView renders the human-
discussion panel there, not the chat panel, so a chat turn posted from the
reading view had nowhere to render — asking-while-reading silently did
nothing. AI chat is an editing activity (a turn can emit <change> proposals)
and only runs on an edit branch.

Option B: when Ask is invoked from main, transparently cut an edit branch via
the same dispatch as Start Contributing (promote-to-branch for active,
start-edit-branch for super-draft; idempotent — reuse an existing edit
branch), navigate onto it so the chat panel mounts, and run the question
(text + selected quote) as the branch's first chat turn once its view and
main_thread_id resolve. The turn fires from the message-load effect's
continuation (not a parallel effect) so a late message load can't clobber the
optimistic turn; a live ref keeps the latest submitChatTurn closure in reach.

Degrades gracefully: signed-out → login; a viewer who can't contribute has the
cut rejected server-side and the error surfaced (no spurious branch); asking
from a branch already, and Flag on main (human discussion thread), unchanged.

Frontend-only; the branch chat endpoints already work on a branch. SPEC §8.3
records the behavior; new RFCView unit tests + an e2e spec cover the flow.
backend 677 / frontend 66 / e2e 5 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 04:19:20 -07:00
Ben Stull ff88be2e91 Merge pull request 'fix(§22/G-15): three-tier-aware branch/edit/body subsystem (v0.53.0)' (#47) from fix-g15-three-tier-write-paths into main 2026-06-09 05:40:59 +00:00
Ben Stull 4e7410f90b fix(§22/G-15): make the branch/edit/body subsystem three-tier aware (v0.53.0)
§22 migrated the READ/catalog path to the three-tier (project/collection)
model but the WRITE/branch/body subsystem still hardcoded the default
project's default collection — resolving every meta-resident entry to the
default content repo at rfcs/<slug>.md, ignoring the entry's project (its own
content repo) and collection (a <subfolder>/rfcs/ prefix). An entry outside
the default collection rendered a blank canonical body and every edit/PR/
body-write path hit the wrong file.

- New single resolver: projects.content_repo_for_collection() +
  projects.entry_location(config, cid, slug) -> (org, repo, md_path)
  (collection -> project -> content_repo + subfolder; falls back to the
  default repo for a legacy/unknown collection).
- Every entry write/read path resolves via it: api_branches (body GET + all
  branch write paths), api_prs (pr-draft/open/merge/withdraw/review/
  resolution-branch), api_graduation (graduate/claim/retire/unretire +
  orchestrator + state-flip), api.py mark-reviewed + idea-PR merge/decline/
  withdraw + proposal preview, api_metadata. bot.open_metadata_pr and
  bot.mark_entry_reviewed gained a file_path param. refresh_meta_branches,
  the webhook corpus-refresh dispatch, and the hygiene branch-delete now
  span every project's content repo, not just the default.
- New additive collection-scoped body-read routes:
  GET /api/projects/{pid}/collections/{cid}/rfcs/{slug}/main and
  .../branches/{branch} disambiguate a slug across collections (G-5) and read
  the entry's own repo. Slug-only routes kept (now collection-aware via the
  cached row). Frontend getRFCMain/getBranch take optional pid+cid; RFCView
  threads them (mirrors the v0.52.1 entry-detail fix).

No migration, no config change. Existing default-collection entries
unaffected. Tests: backend resolver + collection-scoped branch/body + graduate-
in-subfolder write path; frontend api unit. backend 677 / frontend 60 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 22:39:46 -07:00
Ben Stull afa8d26378 Merge pull request 'fix(routing): 404 page for unmatched routes (v0.52.3)' (#45) from fix-404-unmatched-routes into main 2026-06-09 03:14:39 +00:00
Ben Stull 948ee88160 fix(routing): render a 404 page for unmatched routes (v0.52.3)
The top-level catch-all silently redirected unknown paths to "/", and the
nested /admin/*, /p/:projectId/*, and /docs/* route groups had no catch-all
(invalid subpaths rendered a blank pane). Add a shared NotFound component and
wire it into all four route groups so a bad/typo'd URL shows a clear 404 with
a link home. Client-side 404 UI (SPA still serves HTTP 200).

Patch 0.52.2 → 0.52.3; CHANGELOG updated. Caught by the operator on PPE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 20:14:25 -07:00
Ben Stull 7bcf784d06 Merge pull request 'fix(admin): absolute sidebar nav links (v0.52.2)' (#44) from fix-admin-nav-relative-links into main 2026-06-09 03:08:25 +00:00
Ben Stull 8e207a60e6 fix(admin): absolute sidebar nav links so /admin URLs don't accumulate (v0.52.2)
The /admin left-rail NavLinks used relative targets (to="users", etc.). Under
the /admin/* nested route a relative link resolves against the current URL, so
each click appended a segment — Users→Allowlist→Graduation yielded
/admin/users/allowlist/graduation instead of /admin/graduation. Use absolute
targets (/admin/<tab>) + `end` for exact active matching.

Patch 0.52.1 → 0.52.2; CHANGELOG updated. Caught by the operator on PPE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 20:07:44 -07:00
Ben Stull fbaa975b5c Merge pull request 'fix(§22.4a): scope RFCView entry-detail fetch to its collection (v0.52.1)' (#43) from fix-collection-scoped-entry-detail into main 2026-06-08 13:45:42 +00:00
Ben Stull ba37da927a fix(§22.4a): scope RFCView entry-detail fetch to its collection (v0.52.1)
The §9 deployed-environment E2E harness (0.52.0), run against a PPE host
with per-collection-isolated content, surfaced a latent multi-collection
bug: RFCView computed the collection id from the route but called
getRFC(pid, slug) without it, so a named-collection entry was always
fetched via the project default-collection route — which 404s for an entry
that exists only in a named collection ("Error: Not found"; metadata panel
absent). Local/Tier-1 stacks masked it (same slug also reachable via the
default collection). Thread cid through all three getRFC call sites; re-run
the load effect on collection change.

Harness/test-infra (not in the deployed artifact):
- e2e: pre-record cookie consent via addInitScript (lib/fixtures.js) so the
  bottom-fixed consent banner can't intercept catalog row-select clicks on
  the slower deployed edge.
- testing/seed-ppe.sh: fail loudly on any non-2xx Gitea response (a
  swallowed 403 org-repo create had reached the deploy as a 502).
- testing/ppe-deploy-and-test.sh: seed via the Keychain admin token
  (write:organization needed to create the PPE repos); store the E2E secret
  newline-free; read EXPECT_VERSION from VERSION.

Patch bump 0.52.0 → 0.52.1; CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 06:45:01 -07:00
Ben Stull 9c8035bdbd test(e2e): one-shot PPE deploy+E2E resume script
Runs the whole §9 PPE stage non-interactively after the operator's gcloud
reauth: bot-token-seed the PPE repos -> ensure E2E secret -> deploy ->
wait for health=0.52.0 + bdd-collection sync -> run metadata.spec.js
against the deployed host. Idempotent; secrets fetched from SM, never
echoed. Test/ops infra (not in the deployed artifact).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 00:01:19 -07:00
Ben Stull fd123da6a3 test(e2e): harden harness for deployed runs (retries, serialize, banner)
Test-infra only (e2e/ is not in the deployed artifact). Refines the
v0.52.0 deployed-env harness:
- retries:2 + on-first-retry trace now meaningful (was dead: retries
  defaulted to 0); de-risks timing flakes over the public edge.
- workers:1 — the metadata specs run in order and write real commits;
  parallel workers would race on shared state and concurrent bot pushes.
- deployed timeouts bumped (60s/20s) when E2E_TEST_AUTH_SECRET is set.
- dismissCookies forcibly removes any lingering consent-banner node so it
  can't intercept catalog-footer checkbox clicks (the recurring flake).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:56:32 -07:00
Ben Stull 96e2214213 Merge pull request 'v0.52.0 — deployed-environment E2E harness + gated test-auth' (#42) from ppe-e2e-deployed-harness into main 2026-06-08 06:36:18 +00:00
Ben Stull 83eafe72ee feat(e2e): deployed-environment E2E harness + gated test-auth (v0.52.0)
The §9 pipeline's PPE+E2E stage was unreachable: e2e/metadata.spec.js was
bound to Tier-1-only scaffolding (docker-seeded faceted collection,
SQLite-injected owner, Mailpit OTC sink). This makes the same suite run
against a deployed host.

- backend: POST /auth/test/login — fail-closed, secret-gated, single-
  identity owner test-login (404 unless E2E_TEST_AUTH_SECRET +
  E2E_TEST_AUTH_EMAIL both set; constant-time compare; loud startup warn).
  6 vertical tests; backend 665 green. Documented in backend/.env.example.
- e2e/lib/auth.js: branch on E2E_TEST_AUTH_SECRET (deployed test-login vs
  local Mailpit OTC); OWNER_EMAIL from E2E_OWNER_EMAIL. Spec unchanged so
  the localhost Tier-1 path keeps working.
- testing/seed-ppe.sh: seed a dedicated, prod-untouching PPE registry +
  content repo (faceted bdd collection) — real OHM content never touched.
- docs/design/2026-06-07-deployed-env-e2e-harness.md; CHANGELOG; VERSION
  + frontend/package.json -> 0.52.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:35:53 -07:00
Ben Stull dd9ceff69e Merge pull request 'v0.51.1 — collection-id divergence fix + faceted-catalog scoping fix + metadata E2E' (#41) from fix-mig029-collection-divergence into main 2026-06-08 05:30:47 +00:00
Ben Stull 52f465b4dd release: v0.51.1 — collection-id divergence + faceted-catalog scoping fixes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:30:09 -07:00
Ben Stull 2fc7029bd9 test(e2e): §22-current Tier-1 harness + metadata E2E (SLICE-3/4/5)
Modernize the Tier-1 stack to the three-tier app and add browser coverage for
the §22.4a metadata UI, closing the E2E gap deferred across SLICE-3/4/5:
- seed-gitea.sh: create a REGISTRY_REPO with projects.yaml + a faceted named
  collection (.collection.yaml fields: priority enum + tags) seeded with three
  metadata-bearing entries; register content+registry webhooks; self-guarding
  (skip if a prior token still works) so a dependency-triggered re-run can't
  remint and invalidate the backend's token.
- .env.tier1: REGISTRY_REPO/DEFAULT_PROJECT_ID; disable OTC cooldown + lift the
  per-IP auth limiter for the single-IP test runner.
- docker-compose: pin backend image; backend-seed inserts a granted owner the
  OTC path can sign in as (write paths need contributor+).
- Makefile: two-phase tier1-up (seed to completion, then create backend so it
  reads the populated token env); robust down; e2e-fresh = down+up+e2e (the
  canonical run, since the edit/bulk specs mutate the seeded corpus).
- metadata.spec.js: SLICE-3 faceted filter (anon), SLICE-4 edit panel (owner),
  SLICE-5 bulk bar (owner). 4 passed against a fresh stack.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:29:11 -07:00
Ben Stull 9e1b7ce34f feat(ratelimit): env-tunable per-IP auth limiter budgets
Add RATELIMIT_OTC_REQUEST_MAX / RATELIMIT_VERIFY_MAX / RATELIMIT_CHECK_MAX
(default to the existing secure values; non-positive/unparseable → default).
Lets a test/PPE stack that drives the auth endpoints repeatedly from one IP
lift the budget; production leaves them unset. Mirrors the existing
OTC_REQUEST_COOLDOWN_SECONDS knob.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:28:57 -07:00
Ben Stull cbaba76345 fix(§22): catalog scopes to the named collection in the URL (E2E-caught)
useCollectionId read useParams().collectionId, but the Catalog renders at
/p/:projectId/* — outside the c/:collectionId route — so it always fell back
to the default collection. The faceted filter (SLICE-3) and bulk action bar
(SLICE-5) therefore never scoped to a named (fields-bearing) collection in the
deployed app; the Catalog unit test had mocked useCollectionId, hiding it.
Resolve the /c/<id>/ segment from the pathname when the route param isn't in
scope. Caught by the new Playwright metadata suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:28:57 -07:00
Ben Stull 620926b834 fix(§22): heal mig-029 vs registry-mirror collection-id divergence
On a deployment that already had >=2 projects when migration 029 ran, 029
seeds the default project's collection id as the project id (e.g. 'ohm'),
but the registry mirror expects 'default' -> it inserted a duplicate empty
'default' collection, orphaning the entries.

Adds projects.reconcile_default_collection_id() -- the collection-grain twin
of restamp_default_project -- run at startup BEFORE the mirror so it merges
onto the canonical 'default' collection instead of duplicating. Renames the
divergent collection id and cascades collection_id across all keyed tables
(FK-off atomic rename + foreign_key_check). Idempotent; no-op on fresh /
single-project / already-aligned deploys. Seam tests show the duplicate forms
without the fix and merges with it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 21:54:52 -07:00
Ben Stull 4ac3955056 Merge pull request 'v0.51.0 — SLICE-5: bulk tag/untag metadata (§22.4a PUC-2)' (#35) from slice5-bulk-metadata into main 2026-06-08 04:04:21 +00:00
Ben Stull 7886840362 fix(slice5): validate raw metadata input + prune stale bulk selections
Code-review follow-ups:
- Validate the raw submitted value before apply_values coerces it, in both
  the single-edit and bulk endpoints — a scalar set onto a tags field now
  rejects (422 / rejected) instead of silently char-splitting into a list.
- Catalog prunes bulk selections to the currently-visible (filtered) entries
  after each list fetch, so the bulk bar can't act on rows the user has
  filtered out of view.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 21:03:29 -07:00
Ben Stull 8eee907893 docs(slice5): record SLICE-5 bulk edit shipped in SPEC §22.4a status
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:52:48 -07:00
Ben Stull a2b55f94ce release(slice5): v0.51.0 — bulk tag/untag metadata (§22.4a PUC-2 SLICE-5)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:52:16 -07:00
Ben Stull edbf68909a feat(slice5): catalog multi-select + bulk action bar (§22.4a PUC-2)
Faceted catalog rows are selectable for contributors; a sticky bulk bar
(BulkActionBar) drives set/add/remove gestures from the collection fields
schema, calls bulkEntryMeta (one commit), toasts applied/skipped counts,
and re-fetches. Non-contributors and legacy (no-fields) collections see
no selection (INV-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:51:48 -07:00
Ben Stull 282706d7ef feat(slice5): POST .../meta/bulk one-commit bulk metadata edit (§22.4a PUC-2)
set/add/remove ops reusing the SLICE-4 sidecar write-through; per-entry
partial-rejection; contributor+ gated (INV-4); validated at the write
boundary. Tests cover one-commit, add/remove, partial reject, authz,
and op/field/empty guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:48:54 -07:00
Ben Stull eaf69cd05c Merge pull request 'v0.50.0 — SLICE-4: single-entry metadata edit (§22.4a)' (#34) from worktree-metadata-slice4 into main 2026-06-08 02:18:21 +00:00
Ben Stull 0d2fdfacf2 release(slice4): v0.50.0 — single-entry metadata edit + sidecar-aware writes (§22.4a SLICE-4)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:17:25 -07:00
Ben Stull abd17a6cc8 feat(slice4): schema-driven metadata edit panel in RFCView (§22.4a PUC-1 §5.2)
MetadataFieldsPanel renders one control per declared field (enum→select,
tags→chips, text→input), read-only without contribute access, saving changed
values via saveEntryMeta (direct sidecar commit). Wired into the canonical
main view, gated on the collection declaring fields (INV-5). saveEntryMeta
API client added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:14:13 -07:00
Ben Stull 46c957cff5 feat(slice4): Owner-gated collection migrate endpoint (§22.4a PUC-5)
POST .../collections/{cid}/migrate — now safe to ship since all write paths
are sidecar-aware. Idempotent, one commit per collection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:01:18 -07:00
Ben Stull d687a65470 feat(slice4): POST .../meta single-entry edit endpoint + GET RFC meta/can_edit_meta (§22.4a PUC-1)
Direct-commit to the sidecar (D7), contributor+ gated (INV-4), schema-validated
at the write boundary, lazy-migrates a legacy entry, re-ingests. GET RFC now
returns the per-entry meta mapping and a can_edit_meta capability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:00:40 -07:00
Ben Stull aee9b582e5 fix(slice4): make all entry write paths sidecar-aware (§22.4a carried from SLICE-1)
graduate, claim, retire/unretire, _read_meta_entry, mark_entry_reviewed,
body extract/wrap (api_branches + api_prs replay) now dual-read and write
metadata to the sidecar via write_entry_files + bot.commit_entry_files/
open_entry_pr — a migrated body-only .md no longer crashes entry.parse or
re-grows frontmatter; legacy entries lazy-migrate on first metadata write.
Existing tests updated to assert the sidecar (INV-2 clean docs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:57:54 -07:00
Ben Stull 734290f344 feat(slice4): bot.commit_entry_files + open_entry_pr multi-file primitives (§22.4a)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:46:09 -07:00
Ben Stull 49981e2d6e feat(slice4): metadata sidecar git read/write helpers — dual-read + lazy-migrate ops (§22.4a)
apply_values, EntryGitState, read_entry_from_git, write_entry_files, sidecar_path_for.
Plus the SLICE-4 implementation plan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:44:14 -07:00
Ben Stull 7ece6d348b Merge pull request 'v0.49.0 — SLICE-3: faceted left-pane filtering (§22.4a)' (#33) from worktree-metadata-slice3-facets into main 2026-06-08 00:44:54 +00:00
Ben Stull 6bb6d654fa release(slice3): v0.49.0 — faceted left-pane filtering (§22.4a SLICE-3) 2026-06-07 17:43:56 -07:00
Ben Stull 276a625997 test(slice3): Catalog faceted-pane component test (§22.4a PUC-3) 2026-06-07 17:41:04 -07:00
Ben Stull ae3afe2f48 feat(slice3): faceted catalog pane gated on collection fields (§22.4a §5.1) 2026-06-07 17:40:29 -07:00
Ben Stull ae083bfcaa feat(slice3): FacetGroups faceted-pane component (§22.4a §5.1) 2026-06-07 17:38:35 -07:00
Ben Stull bcce40d2cb feat(slice3): listRFCs accepts facet selections, returns {items,facets} (§22.4a) 2026-06-07 17:38:16 -07:00
Ben Stull 7b269e11c4 test(slice3): integration tests for faceted list endpoint (§22.4a PUC-3) 2026-06-07 17:37:46 -07:00
Ben Stull 27061c30b0 feat(slice3): collection list endpoint returns facets + honours filters (§22.4a) 2026-06-07 17:35:26 -07:00
Ben Stull 14ea3c0cce feat(slice3): pure facet field-set + filter/count helper (§22.4a) 2026-06-07 17:34:33 -07:00
Ben Stull 644bf35d89 feat(slice3): persist entry metadata to cached_rfcs.meta_json at ingest (§22.4a) 2026-06-07 17:33:47 -07:00
Ben Stull 3636fa5afd feat(slice3): migration 034 — cached_rfcs.meta_json for facet values (§22.4a) 2026-06-07 17:33:11 -07:00
Ben Stull 1bcf8aa77e Merge pull request 'v0.48.0 — SLICE-2: collection field schema + central validation (§22.4a)' (#32) from worktree-metadata-slice2-schema into main 2026-06-08 00:06:41 +00:00
Ben Stull e336e31812 v0.48.0 — SLICE-2: collection field schema + central validation (§22.4a)
Collections declare a `fields:` schema in `.collection.yaml`; entries carry
typed metadata (enum/tags/text). New central `metadata_schema` module parses
the schema leniently (INV-3) and validates entry values — advisory at read
(corpus mirror flags violations as `metadata_malformed` without hard-failing),
the enforcement point for the write boundary (edit endpoints land SLICE-4/5).

- app/metadata_schema.py: parse_fields (lenient/normalizing) + validate
- registry.parse_collection_manifest reads `fields:` into collection config
- collections.get_collection unpacks `fields`; served by the collection API
- cache._refresh_collection_corpus validates each entry advisory-only

Non-breaking, opt-in: no `fields:` → unchanged (INV-5); default `document`
collection declares none (N=1 unchanged). No DB migration — schema rides in
collections.config_json. SLICE-2 of
docs/design/2026-06-06-configurable-collection-metadata.md §7.2.

Backend suite green (601 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:06:04 -07:00
Ben Stull 98c276a662 Merge pull request 'v0.47.0 — SLICE-1: metadata sidecars (storage + dual-read + migration tool)' (#31) from worktree-metadata-slice1-sidecar into main 2026-06-07 15:24:55 +00:00
Ben Stull f05ee59763 v0.47.0 — SLICE-1: metadata sidecars — storage + dual-read + migration tool
§22.4a SLICE-1 of docs/design/2026-06-06-configurable-collection-metadata.md
(§7.2). Entry metadata can live in a per-entry `<slug>.meta.yaml` sidecar with
the `.md` kept as pure prose (INV-2). Additive and non-breaking — with no
sidecars present every corpus stays on the legacy frontmatter path,
byte-identical (N=1 unchanged).

- Dual-read (app/metadata.py `read_entry`) — sidecar-else-legacy-frontmatter,
  identical records (INV-6); unknown/forward-compat keys ride along through
  parse→serialize and migration (INV-7, `Entry.extra`). A degenerate sidecar
  (malformed/empty/slug-less) never drops the entry — slug backstopped from the
  filename stem, flagged not lost (INV-3).
- Migration tool (`metadata.migrate_collection`) — idempotent, one ChangeFiles
  commit per collection (new `gitea.change_files`). Tested as a function; its
  Owner-gated operator trigger is DEFERRED to SLICE-4 (write paths must become
  sidecar-aware first — see the design's SLICE-4 note + INV-8). No production
  trigger ships here, so no corpus is rewritten.
- Malformed flag — migration 033 adds `cached_rfcs.metadata_malformed`
  (additive); the corpus mirror derives it; catalog list + entry-detail APIs
  surface `metadata_malformed`.
- INV-7 at graduation — graduation now carries `Entry.extra` through the rebuild
  instead of dropping forward-compat keys.

Gate: backend 575 passed (28 new: test_metadata / _migration / _cache +
graduation extra-preservation). Frontend untouched. CHANGELOG 0.47.0 +
upgrade-steps; VERSION + frontend/package.json -> 0.47.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 08:24:27 -07:00
Ben Stull b1acc2382d Merge pull request 'v0.46.2 — SLICE-0: §22.4a contract amendment (entry metadata is collection-configured, not type-driven)' (#30) from worktree-metadata-slice0-spec into main 2026-06-07 14:33:21 +00:00
Ben Stull 5cb5f4a4a2 v0.46.2 — SLICE-0: §22.4a contract amendment (entry metadata is collection-configured, not type-driven)
Reframes binding SPEC.md §22.4a so a collection's entry metadata schema is
collection-configured — a `fields:` schema in `.collection.yaml` plus per-entry
`<slug>.meta.yaml` sidecars — rather than a frontmatter schema hard-wired to the
collection's `type`. Item 3's type-specific surfaces (release planning;
bdd scenario/coverage views) are deferred to a future design; bdd coverage is
recorded as a future `ref`-field surface rendered as hyperlinks (no cross-
collection corpus fusion). `type` still selects terminology (entry noun,
v0.45.0) + default initial_state/review posture (§22.4b-c).

SLICE-0 of docs/design/2026-06-06-configurable-collection-metadata.md (§7.2);
supersedes the per-type-surfaces draft (D11). Doc-only: per-type frontmatter
validation was never implemented, so no operator action, no schema/behavior
change. Sidecar storage + validation + UI arrive in SLICE-1+.

- SPEC.md §22.4a reframed; document/specification/bdd bullets updated; metadata
  amendment blockquote added.
- SPEC.md §2 and §22 forward-pointer blockquotes: "type-dependent frontmatter
  schema" -> "collection-configured, not type-driven (§22.4a, as amended)".
- CHANGELOG 0.46.2; VERSION + frontend/package.json -> 0.46.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 07:32:27 -07:00
Ben Stull 36cb6187eb Merge pull request 'preview.env.example: flotilla-core overlay set (retire shim ref)' (#29) from fix/preview-env-flotilla-core into main 2026-06-07 05:52:03 +00:00
Ben Stull 677c5eb72f preview.env.example: flotilla-core overlay set (not the retired per-app shim)
The example comment referenced 'ohm-rfc-app-flotilla overlay set' — the retired
per-app shim. flotilla-core is the operator CLI for all deployments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 22:44:09 -07:00
Ben Stull 077563ea47 Merge pull request 'docs(spec): Corpus Tree — universal directory-tree left pane (Solution Design)' (#28) from docs/corpus-tree-spec into main 2026-06-07 00:55:02 +00:00
Ben Stull dc5345cef4 docs(spec): Corpus Tree — universal directory-tree left pane (Solution Design)
Solution Design for making the left pane a universal git-directory tree:
host existing documentation repos as path-addressed corpora, full
governance lifecycle per file, dual-mode (structure/flat) pane replacing
the §7 Catalog, zero-migration onboarding ("no record = active").

Discovery output of ohm session 0081.0. Conforms to handbook §3.3
Solution Design standard. Sliced SLICE-1..4 in §7.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:47:24 -07:00
Ben Stull ee74a39b62 Merge pull request 'v0.46.1 — fix migration 029 for the §22.13 re-stamp aftermath (OHM deploy fault)' (#27) from fix/migration-029-orphan-collection into main 2026-06-06 19:03:20 +00:00
Ben Stull 2f507e5721 v0.46.1 — fix migration 029 for the §22.13 re-stamp aftermath (OHM deploy fault)
Deploying the three-tier series onto OHM crash-looped on migration 029:
`NOT NULL constraint failed: cached_branches__new.collection_id`, then a UNIQUE
collision. Root cause: the v0.39.0 default→ohm re-stamp updated
cached_rfcs.project_id but NOT the entry-satellite tables, so ~1.3k
cached_branches rows were stranded at project_id='default' — which 029's
per-project collection backfill can't map (NULL), some of which also duplicate
freshly-re-mirrored 'ohm' rows (UNIQUE), and some of which reference entries
that no longer exist.

Fix — a repair prologue at the top of 029 (no schema change):
- drop stale rows that duplicate an already-correctly-stamped row (keep the
  fresh copy) for the branch-keyed tables;
- re-derive each satellite's project_id from its entry (cached_rfcs, by slug);
- drop rows whose entry no longer exists (stale cache, rebuildable from gitea).
A no-op on clean/fresh deployments (empty or consistent satellites).

Validated against a snapshot of the live OHM DB: 029→032 apply cleanly, zero
NULL collection_id, FK check clean, watches/RFCs/branch_visibility preserved,
cached_branches 1397 → 1291 (−67 no-RFC, −39 dups). Fresh-install path
unchanged: existing 029 suite green + a new regression test for the
stale/dup/orphan shape. Full backend suite 547 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:02:53 -07:00
Ben Stull 9785782532 Merge pull request 'Discovery spec: configurable collection metadata (clean-doc tagging)' (#26) from docs/collection-metadata-design into main 2026-06-06 18:43:26 +00:00
Ben Stull 43a002c6aa Spec v0.1.6: §7.1 execution convention (one slice, one session, just-in-time)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 11:19:28 -07:00
Ben Stull 8ce3e5792d Spec v0.1.5: define Business Actors (§1.3) before Problem/Pain reference them
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 08:12:20 -07:00
Ben Stull 1be4a2edbf Spec v0.1.4: two-part structure — Business Context (§1) + Solution Proposal (§2)
- §1 Business Context holds the whole business lens (1.1–1.9), solution-agnostic
- §2 Solution Proposal introduces the chosen approach (and justifies build over
  a manual alternative); may be non-software in general
- Renumber product/engineering lenses to §§3–7

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 08:02:33 -07:00
Ben Stull 561cd73760 Spec v0.1.3: supersede per-type-surfaces; split personas; harvest patterns
- Supersede docs/design/2026-06-06-per-type-surfaces.md (banner added there);
  harvest validation seam, malformed-metadata flag, unknown-fields-ride-along,
  engine-unchanged invariant, N=1 document backcompat
- bdd coverage kept as a future per-type surface over a generic `ref` field
- Schema model = pure collection-config (not type-driven)
- SLICE-0 added: amend binding SPEC.md §22.4a (frontmatter→sidecar,
  type-driven→collection-configured, defer surfaces)
- Split personas: §6 Business Actors/Roles (solution-agnostic) + §10 Product
  Personas (mapped to business roles); renumber accordingly

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 07:34:15 -07:00
Ben Stull 3c910e89ab Revise spec to template: value-only summary, Pain Points, business framing
- Executive Summary → value-only (no solution)
- New §4 Pain Points (PP-1..PP-7)
- Business Outcomes → §5, restated as business outcomes (adoption/diversity),
  not solution outputs
- Business Use Cases rewritten as solution-agnostic actor-goal scenarios
- Renumber per the Solution Design template revision

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 07:15:35 -07:00
89 changed files with 11514 additions and 595 deletions
+469
View File
@@ -23,6 +23,475 @@ skip versions are the composition of each intervening adjacent
release's steps in order — no A-to-B path is pre-computed beyond
that.
## 0.54.0 — 2026-06-09
**Minor — the AI "Ask" affordance now works from an entry's canonical view
(§8.12); no operator action required.**
AI chat is an editing activity (a turn can emit proposed edits) and only runs
on an edit branch. On an entry's canonical `main` view the right column is the
human-discussion panel, not the chat panel — so invoking "Ask" (the selection
tooltip or the prompt bar) from the reading view pushed a chat turn that had
nowhere to render: it silently did nothing. Surfaced by dogfooding a non-default
project's spec entry on a live deployment.
- **Asking while reading "just works" (Option B).** When Ask is invoked from
the canonical view, the app transparently cuts an edit branch via the same
dispatch as Start Contributing (`start-edit-branch` for a super-draft,
`promote-to-branch` for an active RFC — idempotent, so it reuses an existing
edit branch rather than double-cutting), navigates onto that branch so the
chat panel mounts, and runs the question (text + the selected quote) there
once the branch's chat thread has resolved. The viewer lands in the chat with
the streaming answer, quote intact.
- **Graceful degradation.** A signed-out viewer keeps the read-only path
(sign-in prompt; Ask disabled) and a viewer who cannot contribute has the
branch cut rejected server-side and the error surfaced — no silent no-op and
no spurious branch. Asking from a branch already, and Flag on the canonical
view (which still opens a human discussion thread), are unchanged.
Frontend-only: the branch chat endpoints already worked on a branch. No
database migration and no configuration change.
## 0.53.0 — 2026-06-09
**Minor — the branch/edit/body subsystem is now three-tier (project /
collection) aware (gap G-15); no operator action required.**
§22 migrated the READ/catalog path to the three-tier model, but the
WRITE/branch/body subsystem still hardcoded the default project's default
collection: it resolved every meta-resident entry to the default project's
content repo at `rfcs/<slug>.md`, ignoring the entry's project (its own
content repo) and its collection (a `<subfolder>/rfcs/` prefix). As a result
an entry outside the default project's default collection rendered a **blank
canonical body** (the git-backed `GET …/branches/main` read the wrong file)
and every edit / PR / body-write path hit the wrong repo. Surfaced by
dogfooding a separate project on a live deployment.
- **Single resolver.** New `projects.content_repo_for_collection(cid)` and
`projects.entry_location(config, cid, slug) → (org, repo, md_path)` resolve
an entry's git location from its collection (collection → project →
content_repo, plus the collection subfolder), falling back to the default
project's repo for a legacy / unknown collection. Every write/read path now
shares this resolver instead of the hardcoded default.
- **Every entry write/read path is collection-resolved:** the branch-body GET
and all branch write paths (`api_branches`: start-edit-branch, accept /
decline / reask, manual-flush, promote-to-branch, metadata, branch chat),
the PR family (`api_prs`: pr-draft / open-pr / merge / withdraw / review /
resolution-branch), graduation (`api_graduation`: graduate / claim / retire
/ unretire and the module-level orchestrator + state-flip helpers),
mark-reviewed, the idea-PR merge / decline / withdraw + proposal preview,
and metadata edit/bulk/migrate (`api_metadata`). `bot.open_metadata_pr` and
`bot.mark_entry_reviewed` gained a `file_path` parameter (was hardcoded to
the repo root). The branch cache (`refresh_meta_branches`), the webhook
corpus-refresh dispatch, and the hygiene branch-delete now span **every**
project's content repo, not just the default.
- **Collection-scoped body-read routes** (new, additive):
`GET /api/projects/{pid}/collections/{cid}/rfcs/{slug}/main` and
`…/rfcs/{slug}/branches/{branch}` disambiguate a slug that exists in two
collections (G-5) and read the entry's own repo. The slug-only routes are
kept and are now collection-aware via the entry's cached row, so existing
clients are unaffected.
- **Frontend.** `getRFCMain` / `getBranch` take optional `projectId` +
`collectionId` and call the scoped routes when present (RFCView threads its
`pid`/`cid`, mirroring the v0.52.1 entry-detail fix); both fall back to the
slug-only routes otherwise.
No database migration and no configuration change. Existing default-collection
entries are unaffected (the resolver returns the default repo and a repo-root
path for them).
_Known boundary:_ the branch / thread / change / PR cache tables remain
slug-keyed, so a slug that genuinely exists in two collections is still
ambiguous on the slug-only *write* routes; the collection-scoped body-read
routes resolve it for the canonical view, and the resolver makes every write
target the correct repo via the entry's own collection. Full slug
de-duplication across collections on the write family is tracked separately.
## 0.52.3 — 2026-06-09
**Patch — unmatched routes render a 404 instead of a blank page or a
silent redirect home (no operator action required).**
- **The top-level catch-all redirected any unknown path to `/`**, and the
nested `/admin/*`, `/p/:projectId/*`, and `/docs/*` route groups had no
catch-all (so an invalid subpath rendered an empty pane). Added a shared
`NotFound` 404 page and wired it into all four route groups, so a bad or
typo'd URL — including a stale `/admin/users/allowlist/graduation`-style
path — now shows a clear "page not found" with a link back to the catalog.
(Client-side 404 UI; the SPA still serves over HTTP 200, as before.)
## 0.52.2 — 2026-06-09
**Patch — admin sidebar nav links no longer accumulate URL segments (no
operator action required).**
- **The `/admin` left-rail links were relative**, so under the `/admin/*`
nested route each click resolved against the current URL and *appended* a
segment — e.g. clicking Users → Allowlist → Graduation produced
`/admin/users/allowlist/graduation` instead of `/admin/graduation`. Fixed:
the rail `NavLink`s now use absolute targets (`/admin/<tab>`) with `end`
for exact active-state matching.
## 0.52.1 — 2026-06-08
**Patch — collection-scoped entry-detail fetch fix (no operator action
required).**
Caught by the §9 deployed-environment E2E harness (0.52.0) running against
a PPE host whose content is cleanly isolated per collection:
- **Entry detail in a named collection 404'd ("Error: Not found") and its
metadata panel never rendered.** `RFCView` computed the collection id
from the route but called `getRFC(pid, slug)` without it, so an entry was
always fetched via the project's *default*-collection route
(`/api/projects/<pid>/rfcs/<slug>`). For an entry that lives only in a
named collection that route 404s. The bug was latent since the
multi-collection work — local/Tier-1 stacks masked it because the same
slug was also reachable through the default collection; a deployment with
per-collection-isolated content surfaces it. Fixed: all three `getRFC`
call sites in `RFCView` now pass the collection id (and the load effect
re-runs on collection change).
Test-only (not in the deployed artifact): the deployed-env E2E harness now
pre-records cookie consent via `addInitScript` so the bottom-fixed consent
banner can't intercept catalog row-select clicks on the slower deployed
edge, and `testing/seed-ppe.sh` fails loudly on any non-2xx Gitea response
(a swallowed 403 had let a missing-repo seed reach the deploy as a 502).
## 0.52.0 — 2026-06-07
**Minor — deployed-environment E2E harness (new opt-in test-auth surface;
no operator action required for existing deployments).**
The §9 pipeline's middle stage — running the Playwright E2E suite against
a *deployed* pre-prod host — was unreachable because the suite was bound
to local-only scaffolding (a docker-seeded faceted collection, a
SQLite-injected owner, and Mailpit for the OTC code). This release makes
the same suite run green against a deployed host.
- **New gated test-auth endpoint, `POST /auth/test/login`.** Mints an
authenticated **owner** session for a single pre-configured identity, to
let the E2E suite sign in without a mail sink. It is **fail-closed**:
returns `404` unless **both** `E2E_TEST_AUTH_SECRET` and
`E2E_TEST_AUTH_EMAIL` are set (so it is inert in production, which sets
neither); requires the secret in the `X-Test-Auth-Secret` header
(constant-time compare; wrong/absent → `404`); only mints the one
configured email (any other → `403`); and logs a `WARNING` at startup
when enabled. **Leave both env vars unset on production.**
- **E2E harness parameterized for deployed runs.** `e2e/lib/auth.js`
branches on `E2E_TEST_AUTH_SECRET` (deployed test-login vs. the local
Mailpit OTC path); `OWNER_EMAIL` reads `E2E_OWNER_EMAIL`. `BASE_URL` was
already honored. The local Tier-1 path is unchanged.
- **`testing/seed-ppe.sh`** seeds a dedicated, prod-untouching PPE
registry + content repo (faceted `bdd` collection) so the deployed
fixtures never leak onto real content.
Design note: `docs/design/2026-06-07-deployed-env-e2e-harness.md`.
## 0.51.1 — 2026-06-07
**Patch — two §22 correctness fixes (no operator action required).**
- **Faceted catalog / bulk bar now scope to the named collection in the URL.**
`useCollectionId` read the route param, but the catalog renders above the
`c/:collectionId` route, so it always fell back to the default collection —
the faceted filter (§22.4a SLICE-3) and bulk action bar (SLICE-5) never
scoped to a named, `fields:`-bearing collection in the deployed app. It now
resolves the `/c/<id>/` segment from the path. Caught by a new end-to-end
Playwright suite exercising the metadata UI against a real browser + Gitea.
- **Heal the migration-029 vs registry-mirror collection-id divergence.** On a
deployment that already held ≥2 projects when migration 029 ran, the default
project's collection was seeded as the project id while the mirror expects
`'default'`, so the mirror inserted a duplicate empty collection. A new
startup reconciler (`projects.reconcile_default_collection_id`, the
collection-grain twin of the §22.13 re-stamp) renames the divergent
collection to `'default'` before the mirror runs. Idempotent; a no-op on
fresh / single-project / already-aligned deployments (incl. the default
`document` deployment).
Also: the per-IP auth rate-limiter budgets are now env-overridable
(`RATELIMIT_OTC_REQUEST_MAX` / `_VERIFY_MAX` / `_CHECK_MAX`) for test/PPE stacks
that drive auth from one IP; production keeps the secure defaults. The Tier-1
test harness is brought current with the §22 three-tier app.
## 0.51.0 — 2026-06-07
**Minor — bulk tag/untag metadata edit (§22.4a SLICE-5).**
An authorized user can now apply one metadata field change to many catalog
entries at once. A new endpoint
`POST /api/projects/{id}/collections/{cid}/meta/bulk` takes
`{slugs, op: set|add|remove, field, value}`, validates each entry against the
collection's `fields:` schema at the write boundary (INV-4), writes the passing
entries' `<slug>.meta.yaml` **sidecars** in a **single commit** (D7: bulk = one
commit, reusing the SLICE-4 sidecar write-through), re-ingests, and returns
`{applied, rejected}` so partial failures (missing entry, invalid value) are
reported without sinking the batch. `set` works for any field; `add`/`remove`
operate on a `tags`-type field. In the faceted catalog, contributors now get a
per-row selection checkbox and a sticky bulk action bar (one "Set ▾" control per
enum field, add/remove-tag for tags fields); a successful apply toasts the
applied/skipped counts and refreshes the list. This completes PUC-2 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§5.3, §6.4/§6.5).
No deployment action required. The endpoint and the bulk bar are additive and
opt-in per collection: a collection with no `fields:` block (INV-5 — the
default `document` collection) shows no selection UI and the endpoint returns
`422`, so an N=1 deployment sees zero change.
## 0.50.0 — 2026-06-07
**Minor — single-entry metadata edit + sidecar-aware writes (§22.4a SLICE-4).**
An authorized user can now edit one entry's schema-defined metadata directly from
its detail view. A new endpoint
`POST /api/projects/{id}/collections/{cid}/rfcs/{slug}/meta` takes `{values:{…}}`,
validates them against the collection's `fields:` schema at the write boundary
(INV-4), writes them to the entry's `<slug>.meta.yaml` **sidecar** with a **direct
commit** (D7 — no PR for authorized roles), and re-ingests. A legacy entry is
**lazy-migrated** to a clean body-only `.md` + sidecar on its first metadata edit.
The detail view renders one control per declared field (enum→select, tags→chips,
text→input), read-only without contribute access; `GET` on an entry now returns
its `meta` mapping and a `can_edit_meta` capability. This completes PUC-1 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2); bulk edit (SLICE-5) follows.
Carried from SLICE-1: **every entry write path is now sidecar-aware.** Graduation,
ownership claim, retire/un-retire, mark-reviewed, the body-edit accept/flush
wrappers, and the PR-replay wrappers all dual-read an entry (so a migrated
body-only `.md` no longer crashes `entry.parse`) and write metadata changes to the
sidecar, keeping the `.md` body pure (INV-2) and never re-growing frontmatter; a
legacy entry lazy-migrates on its first metadata-bearing write. With the write
paths safe, the **Owner-gated collection-migration endpoint**
`POST /api/projects/{id}/collections/{cid}/migrate` now ships (PUC-5): it converts
a collection's legacy-frontmatter entries to clean body-only `.md` + sidecars in
one commit, idempotently.
Non-breaking and opt-in (INV-5): a collection with **no `fields:`** exposes no
edit panel and the edit endpoint returns 422 ("no editable fields"); the §22.13
default `document` collection declares none — so **N=1 deployments see no change**.
**Upgrade steps**
- No schema migration. The edit and migrate endpoints are additive; the sidecar
storage layer (mig 033) and `meta_json` index (mig 034) shipped in 0.47.0/0.49.0.
- A deployment adopts single-entry editing by declaring an `enum`/`tags`/`text`
`fields:` block in a collection's `.collection.yaml` (SLICE-2). Contributors+
on that collection (§22 Part B / S3 scope roles) may then edit from the detail
view; the change is a direct commit to the entry's sidecar.
- Operators MAY run `POST …/collections/{cid}/migrate` (collection Owner only) to
convert a collection's existing entries to clean body-only docs + sidecars up
front. It is idempotent and safe to re-run; dual-read means an un-migrated
collection keeps working, and entries lazy-migrate on their first metadata edit
regardless. **No data migration is required.**
## 0.49.0 — 2026-06-07
**Minor — faceted left-pane filtering (§22.4a SLICE-3).** A collection that
declares a metadata field schema (`fields:`, SLICE-2) now gets a **faceted
catalog left pane**: one collapsible filter group per `enum`/`tags` field plus
state, each with per-value **result counts** and multi-select checkboxes, and a
"filter values…" search box on `tags` groups so they stay usable at many values.
Filters compose — OR within a field, AND across fields — and a "malformed
metadata only" toggle surfaces entries failing their schema (INV-3). This is
SLICE-3 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2); the single-entry and bulk metadata-edit UIs (SLICE-4/5) follow.
The collection-scoped list endpoint
(`GET /api/projects/{id}/collections/{cid}/rfcs`, and the project-scoped
default-collection alias) now honours filter query params
(`?priority=P0&tags=checkout&state=active&malformed=true`), returns a
`facets: {field → {value → count}}` block with drill-down counts, and includes
each entry's metadata `meta` mapping; an unknown filter field is a 400. Migration
034 adds the additive `cached_rfcs.meta_json` column that persists per-entry
metadata values for the index.
Non-breaking and opt-in (INV-5): a collection with **no `fields:`** keeps the
existing state-chip catalog unchanged, and the §22.13 default `document`
collection declares none — so **N=1 deployments see no change**. The unscoped
cross-collection `GET /api/rfcs` is untouched.
**Upgrade steps**
- Migration 034 (`cached_rfcs.meta_json`) applies automatically on startup
(additive, nullable). It is populated lazily as the corpus reconciler /
content-repo webhooks re-ingest each collection; until an entry is re-ingested
its `meta_json` is NULL and it contributes no facet values. Operators wanting
facets populated immediately MAY trigger a corpus refresh (the reconciler sweep
on next startup does this). **No data migration is required.**
- A deployment opts a collection into faceting by declaring an `enum`/`tags`
`fields:` block in its `.collection.yaml` (SLICE-2). A collection with no
`fields:` keeps the existing state-chip catalog unchanged.
## 0.48.0 — 2026-06-07
**Minor — collection field schema + central validation (§22.4a SLICE-2).** A
collection can now declare a small **field schema** in its `.collection.yaml`
(`fields:` block), so its entries carry structured, typed metadata — `priority`,
`tags`, and any custom fields the deployment defines. The schema is mirrored into
the collection record and served on the collection API; a new central validator
checks each entry's stored values against it. This is SLICE-2 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2), building on the SLICE-1 sidecars; faceted filtering (SLICE-3) and
the edit UIs (SLICE-4/5) follow.
Non-breaking and opt-in: a collection with **no `fields:`** behaves exactly as
before (INV-5), and the §22.13 default `document` collection declares none
(**N=1 sees no change**). No DB migration — the normalized schema rides in the
existing `collections.config_json` column.
Added:
- **`app/metadata_schema.py`** — the one place that knows a collection's field
shapes. `parse_fields(raw)` normalizes a `.collection.yaml` `fields:` block
**leniently** (INV-3): a bad block or a bad field def (non-mapping, unknown
type, `enum` without a non-empty `values:` list) is skipped with a warning,
never fatal — a typo in one field can't drop the whole collection from the
mirror. `validate(values, fields)` returns advisory `Problem`s (empty =
clean). v1 field types: **`enum`** (single value, controlled by a required
`values:`), **`tags`** (a list; free-form unless `values:` given), **`text`**
(a free string). `ref` / `multi-enum` are deferred (design §2). Keys an entry
carries that the schema does not declare ride along untouched, never flagged
(INV-7).
- **Registry ingest** (`registry.parse_collection_manifest`) reads the `fields:`
block into the collection config (→ `config_json`), beside `enabled_models`.
- **Collection read + API** (`collections.get_collection`) unpacks the schema
and `GET /api/projects/{id}/collections/{cid}` serves it as `fields` (`null`
when unset).
- **Advisory validation at ingest** — the corpus mirror
(`cache._refresh_collection_corpus`) validates each entry against its
collection's schema and flags a violation as `metadata_malformed` **without
blocking the read** (INV-3), OR-ed onto the SLICE-1 sidecar-syntax check.
Write-boundary **enforcement** (a 422 on the metadata-edit endpoints) lands
with those endpoints in SLICE-4/5.
### Upgrade steps (0.47.0 → 0.48.0)
- **No migration, no operator action.** A deployment opts in per collection by
adding a `fields:` block to that collection's `.collection.yaml`; until it
does, behavior is byte-for-byte unchanged.
- A collection's field schema is edited in git for v1 (in-app schema management
is deferred, design D8). After editing `.collection.yaml`, the registry mirror
picks the schema up on its next webhook / reconciler sweep.
- No config change. No content change is required.
## 0.47.0 — 2026-06-07
**Minor — metadata sidecars: storage + dual-read + migration tool (§22.4a
SLICE-1).** Entry metadata can now live in a per-entry `<slug>.meta.yaml`
**sidecar**, with the `.md` kept as pure prose (INV-2). The corpus mirror reads
the sidecar when present and falls back to legacy top-of-document frontmatter
otherwise (**dual-read**, INV-6), so existing corpora load byte-identically.
This is SLICE-1 of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2); the collection `fields:` schema + validation (SLICE-2), faceted
filtering (SLICE-3), and the edit UIs (SLICE-4/5) follow.
Non-breaking and opt-in: a collection with no sidecars behaves exactly as
before, and the §22.13 default collection is untouched (**N=1 sees no change**).
Added:
- **Dual-read parser** (`app/metadata.py`) — `read_entry(md, sidecar)` yields
identical records whether metadata comes from a sidecar or legacy
frontmatter (INV-6); a malformed sidecar never hard-fails a read — the entry
still loads and is flagged (INV-3). Unknown / forward-compat keys ride along
untouched through parse→serialize and the migration (INV-7;
`Entry.extra`).
- **Frontmatter→sidecar migration tool** (`metadata.migrate_collection`) — a
deterministic, idempotent tool that lifts a collection's legacy entries into
sidecars + body-only `.md`s in **one commit** (new Gitea `change_files`
batch); a fully-migrated collection is a no-op. The **operator trigger** for
it (an Owner-gated endpoint) is intentionally **deferred to SLICE-4**: the
propose/graduate/mark-reviewed/edit write paths still read `.md` frontmatter
directly, so they must become sidecar-aware before a corpus is migrated in
production. Until then the tool is shippable groundwork, not yet wired to a
production trigger (INV-8: the engine write paths are unchanged this slice).
- **Malformed-metadata flag** — migration `033_metadata_malformed.sql` adds
`cached_rfcs.metadata_malformed` (additive); the corpus mirror derives it and
the catalog + entry-detail APIs surface `metadata_malformed`. A degenerate
sidecar (malformed / empty / slug-less) never drops the entry — it loads with
its slug backstopped from the filename and is flagged (INV-3).
- **INV-7 at graduation** — graduation now carries an entry's unknown /
forward-compat frontmatter keys through the rebuild instead of dropping them.
### Upgrade steps (0.46.2 → 0.47.0)
- The framework **MUST** apply migration `033_metadata_malformed.sql` — it runs
automatically at startup (additive column, no rebuild, default `0`).
- **No operator action** otherwise. With no sidecars present (the default after
this upgrade) every corpus stays on the legacy frontmatter path, byte-for-byte
as before. The frontmatter→sidecar migration is **not** operator-triggerable
yet (its endpoint lands in SLICE-4); dual-read makes the storage change
invisible until then.
- No config change. No content change is required.
## 0.46.2 — 2026-06-07
**Patch — `SPEC.md` §22.4a contract amendment: entry metadata is
collection-configured, not type-driven (doc-only).** Reframes the binding
§22.4a "collection type" contract so a collection's entry **metadata schema**
is **collection-configured** — a `fields:` schema declared in
`.collection.yaml` plus per-entry `<slug>.meta.yaml` **sidecars** — rather than
a frontmatter schema hard-wired to the collection's `type`. Item 3's
**type-specific surfaces** (release planning for `specification`;
scenario/coverage views for `bdd`) are **deferred** to a future design; the
`bdd` coverage capability is recorded there as a future `ref`-field surface
rendered as hyperlinks (never fusing corpora across collections). What `type`
still selects is the entry-noun terminology (item 2, shipped v0.45.0) and the
default `initial_state` / review posture (§22.4bc).
This is **SLICE-0** of the
[Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
design (§7.2) — the contract amendment that unblocks the build slices. It
supersedes the [per-type-surfaces](./docs/design/2026-06-06-per-type-surfaces.md)
draft (D11; banner already on that doc).
**No operator action; no schema or behavior change.** Per-type frontmatter
validation was never implemented — the engine treats every entry as markdown +
frontmatter regardless of type — so amending the contract changes no runtime
behavior, no migration, and no deployment config. Sidecar storage, the
collection `fields:` schema, validation, and the metadata UI arrive in later
slices (SLICE-1+).
Changed:
- **`SPEC.md` §22.4a** — reframed: `type` selects terminology + default
`initial_state`/review posture; entry metadata is collection-configured
(`fields:` + `<slug>.meta.yaml` sidecars); type surfaces deferred. The
`document`/`specification`/`bdd` bullets updated accordingly; a metadata
amendment blockquote records the supersession.
- **`SPEC.md` §2 and §22 forward-pointers** — the two amendment blockquotes
that said "entry frontmatter schema is type-dependent" now read
"collection-configured, not type-driven (§22.4a, as amended)".
## 0.46.1 — 2026-06-06
**Patch — migration 029 hardening for the §22.13 re-stamp aftermath.** Fixes a
crash deploying the three-tier series (v0.40.0+) onto a deployment that went
through the v0.39.0 `default``<id>` project re-stamp: the re-stamp updated
`cached_rfcs.project_id` but **not** the entry-satellite tables, leaving rows at
the stale `project_id` that migration 029's per-project collection backfill could
not map (`NOT NULL constraint failed: cached_branches__new.collection_id`), plus
stale rows that duplicate freshly-re-mirrored ones (`UNIQUE` collision) and stale
rows whose entry no longer exists. No operator action; **no schema change** — 029
gains a repair prologue only.
Fixed:
- **Migration 029 repair prologue** — before rekeying, each entry-satellite table
(`cached_branches`, `branch_visibility`, `stars`, `watches`, `pr_seen`, …) has
its `project_id` re-derived from its entry (`cached_rfcs`, by slug); rows whose
entry no longer exists are dropped (stale cache, rebuildable from gitea), and
stale rows that duplicate an already-correctly-stamped row are dropped (keeping
the fresh copy). A no-op on a clean/fresh deployment (empty or already-
consistent satellites), so fresh installs are unaffected — the existing 029
test suite passes unchanged, plus a new regression test for the stale/dup/
orphan shape.
No upgrade steps: applying 029 (now repaired) is automatic on deploy; the repair
only mutates the rebuildable `cached_*` caches.
## 0.46.0 — 2026-06-06
**Minor (non-breaking) — §22 three-tier refactor, slice S6 (remainder):
+15 -1
View File
@@ -1,9 +1,18 @@
.PHONY: tier1-up tier1-down tier1-logs fe-unit e2e e2e-install
.PHONY: tier1-up tier1-down tier1-logs fe-unit e2e e2e-install e2e-fresh
# Two-phase: run the Gitea seed to completion FIRST so it writes the bot token /
# OAuth creds into generated/.env.tier1.generated, THEN create the backend/web —
# compose snapshots env_file at container-create time, so the backend must be
# created after the seed has populated it. The touch seeds an empty placeholder
# for compose's up-front env_file existence check on a clean checkout.
tier1-up:
touch testing/generated/.env.tier1.generated
docker compose -f testing/docker-compose.yml up --build -d gitea-seed
docker compose -f testing/docker-compose.yml wait gitea-seed
docker compose -f testing/docker-compose.yml up --build -d
tier1-down:
touch testing/generated/.env.tier1.generated
docker compose -f testing/docker-compose.yml down -v
tier1-logs:
@@ -17,3 +26,8 @@ e2e-install:
e2e:
cd e2e && BASE_URL=$${BASE_URL:-http://localhost:8080} MAILSINK_URL=$${MAILSINK_URL:-http://localhost:8025} npm run e2e
# Canonical run: the metadata specs mutate the seeded corpus (edit/bulk write
# real commits), so they assume a freshly-seeded stack. This brings the stack
# down, back up (re-seeds), and runs the suite once — the shape CI uses.
e2e-fresh: tier1-down tier1-up e2e
+99 -31
View File
@@ -121,9 +121,11 @@ live in the meta repo — they live in the app database (see §5).
> **Three-tier amendment (v0.45.0 — see §22).** Slugs are unique **per
> collection** (§22.4): `model/intro` and `specs/intro` coexist. The entry
> frontmatter schema is **type-dependent** on the collection's `type`
> (§22.4a) — `document` keeps the fields below; `specification` and `bdd`
> add their type metadata. The §2.3 `RFC-NNNN` `max+1` allocation is
> **metadata schema is collection-configured** (§22.4a, as amended by
> v0.46.2) — each collection declares a `fields:` schema in its
> `.collection.yaml` and stores per-entry values in a `<slug>.meta.yaml`
> sidecar; the fields below are the `document` baseline, **not**
> type-driven. The §2.3 `RFC-NNNN` `max+1` allocation is
> **removed** (the slug is the identity); pre-change `id` values survive as
> frozen legacy labels. New `active`-entry frontmatter: `unreviewed` (bool)
> and the `reviewed_at` / `reviewed_by` provenance pair (§22.4c).
@@ -895,7 +897,13 @@ a hierarchy on the user that gets in the way of finding by title.
- **Filter chip strip** — multi-select, AND-combined. Chips:
`State: super-draft | active | withdrawn`, `My RFCs` (I'm an owner
or arbiter), `Has open PRs`, `Unclaimed` (super-drafts with empty
`owners:`), `Tag: …`.
`owners:`), `Tag: …`. A collection that declares a metadata field
schema (§22.4a) replaces this chip strip with **faceted filter
groups** — one collapsible group per `enum`/`tags` field plus state,
each showing per-value result counts and multi-select checkboxes
(OR within a field, AND across fields); see the Configurable
Collection Metadata design. A collection with no `fields:` schema
keeps the chip strip described here unchanged.
### 7.2 The list rows
@@ -1014,6 +1022,20 @@ the user on it in contribute mode. New-branch naming defaults to an
auto-generated value (user-renamable); the exact format is an
implementation detail.
The AI chat is an editing activity — a turn can emit `<change>`
proposals — so it runs on an edit branch, not on read-only main, whose
right column is the human-discussion surface (§8.12 is branch-scoped).
Invoking the AI **Ask** affordance (the selection tooltip's prompt, or
the prompt bar) from main therefore transparently cuts an edit branch
via the same dispatch as "Start Contributing" (promote-to-branch for an
active RFC, start-edit-branch for a super-draft per §9.5; idempotent, so
an existing edit branch is reused rather than a second one cut), lands
the user on it, and runs the question — text plus any selected quote —
as the branch's first chat turn. A viewer who cannot contribute has the
branch cut rejected and the error surfaced (no branch is created); a
signed-out viewer keeps the §8.7 read-only path. **Flag** from main is
unaffected — it opens a human discussion thread on main, not a branch.
Discuss vs. contribute is an *intent* affordance, not a *permission*
affordance. A user without contribute access to a branch sees the
toggle disabled, with a sign-in or request-access path (see §8.7).
@@ -5125,39 +5147,83 @@ never used for routing or lookup. New entries are never assigned one.
### 22.4a Collection type
> **Metadata amendment (v0.46.2 — Configurable Collection Metadata).** Item 1
> below originally made the **entry metadata schema type-driven** — a
> `document`/`specification`/`bdd` frontmatter schema baked into a per-type
> module. That is **superseded**: entry metadata is **collection-configured**,
> not type-driven. Each collection declares a `fields:` schema in its
> `.collection.yaml`, and per-entry values live in a `<slug>.meta.yaml`
> **sidecar** (the `.md` body stays pure prose; a parser reads the sidecar
> else legacy top-of-doc frontmatter). See the
> [Configurable Collection Metadata](./docs/design/2026-06-06-configurable-collection-metadata.md)
> design, which supersedes the
> [per-type-surfaces](./docs/design/2026-06-06-per-type-surfaces.md) draft
> (D11). Item 3's **type-specific surfaces** (release planning for
> `specification`; scenario/coverage views for `bdd`) are **deferred** to a
> future design; the `bdd` **coverage** capability is recorded there as a
> future `ref`-field surface that maps features to the spec entries they
> verify **as hyperlinks**, honoring the §22 rule against fusing corpora
> across collections. What `type` still selects is the **terminology** (item 2,
> the entry noun, shipped v0.45.0) and the **default `initial_state` / review
> posture** (§22.4bc).
>
> **Shipped status.** The design lands incrementally: sidecar storage +
> dual-read (SLICE-1, v0.47.0), the collection `fields:` schema + central
> validation (SLICE-2, v0.48.0), faceted left-pane filtering (SLICE-3, v0.49.0),
> and **single-entry metadata edit (SLICE-4, v0.50.0)** — the direct-commit
> `POST …/rfcs/{slug}/meta` editor (contributor+, validated at the write
> boundary), the schema-driven detail panel, all entry **write paths made
> sidecar-aware** (a migrated body-only `.md` never re-grows frontmatter), and
> the Owner-gated `POST …/collections/{cid}/migrate` endpoint. **Bulk
> tag/untag (SLICE-5, v0.51.0)** completes the design: the
> `POST …/collections/{cid}/meta/bulk` endpoint (`{slugs, op: set|add|remove,
> field, value}`) applies one field change to many entries' sidecars in a
> **single commit** (D7), validated per entry at the write boundary with
> partial-rejection reporting (`{applied, rejected}`), plus the catalog's
> row multi-select + sticky bulk action bar. See §9.5 for the edit-metadata
> write-through this reuses.
Every collection declares a `type` in its `.collection.yaml` manifest
(§22.2), chosen at creation and **immutable**: one of `document`,
`specification`, or `bdd`. Type does not change the engine — every type uses
the same content repo (§22.3), the same propose→branch→PR→discuss→graduate
lifecycle (§§913), the same threads, flags, and chat. Type selects exactly
three things:
lifecycle (§§913), the same threads, flags, and chat. Type selects:
1. the **entry frontmatter schema** the collection validates entries against (§2);
2. the **terminology** the chrome uses for an entry (the §8.1 noun, catalog labels);
3. the set of **type-specific surfaces** layered on top of the shared §7 catalog.
1. the **terminology** the chrome uses for an entry (the §8.1 noun, catalog
labels) — the entry noun, shipped v0.45.0;
2. the **default `initial_state`** a new entry lands in, and its review
posture (§22.4b, §22.4c).
Type-specific behavior is implemented as a per-type module the framework
selects on `collection.type`; the engine itself treats every entry as
markdown + frontmatter regardless of type. `type` is an **open set** in shape
— a future type is a new module plus a new allowed enum value, no schema
rebuild. The type names and their behavior are framework concepts (like role
names), not deployment content: a deployment picks which type each collection
is, but does not define or rename types.
Entry **metadata** is **not** selected by type — it is **collection-configured**
(a `.collection.yaml` `fields:` schema + per-entry `<slug>.meta.yaml`
sidecars; see the amendment above, the Configurable Collection Metadata
design, and the §2 baseline). **Type-specific surfaces** layered on the shared
§7 catalog are **deferred** to a future design.
Type-specific behavior, where it exists, is implemented as a per-type module
the framework selects on `collection.type`; the engine itself treats every
entry as markdown + a metadata sidecar regardless of type. `type` is an
**open set** in shape — a future type is a new module plus a new allowed enum
value, no schema rebuild. The type names and their behavior are framework
concepts (like role names), not deployment content: a deployment picks which
type each collection is, but does not define or rename types.
- **`document`** — long-form normative prose (OHM: a model of principles and
definitions). Frontmatter is the §2 baseline. No type-specific surfaces. The
§22.13 generated default collection is a `document` collection, so the N=1
case is unchanged.
definitions). Metadata is the §2 baseline; no collection-configured `fields:`
are required. The §22.13 generated default collection is a `document`
collection with no `fields:`, so the N=1 case is unchanged.
- **`specification`** — a versioned technical specification (this framework's
own `SPEC.md` is the archetype). Frontmatter adds spec metadata (`version`,
lifecycle `status` of draft/active/superseded, `supersedes`). Type-specific
surface — **release planning:** group entries/changes into versioned
releases with a changelog + §20-style upgrade-steps per release.
own `SPEC.md` is the archetype). A deployment that wants spec metadata
(`version`, lifecycle `status` of draft/active/superseded, `supersedes`)
declares those as collection `fields:`. **Release planning** grouping
entries into versioned releases with a changelog + §20-style upgrade-steps
is a **deferred** type surface.
- **`bdd`** — behavior-driven feature specs: each entry states a feature as
Given/When/Then scenarios with acceptance criteria. Frontmatter adds feature
metadata and an optional link to the `specification` entries a feature
verifies. Type-specific surface: a scenario/acceptance view and a coverage
view mapping features to the spec sections they exercise.
Given/When/Then scenarios with acceptance criteria. Feature metadata is
declared as collection `fields:`. The **scenario/acceptance view** and a
**coverage view** (mapping features to the spec entries they verify via a
future `ref` field, rendered as hyperlinks — never fusing corpora across
collections) are **deferred** type surfaces.
### 22.4b Initial state of a new entry
@@ -5400,10 +5466,12 @@ The single-corpus sections defer to §22; the load-bearing reinterpretations:
a project's repo (§22.3). The bot and app-owned-authorization paragraphs are
unchanged and now read org-wide.
- **§2 Schema / §2.3 IDs.** Slugs are unique **per collection**; the entry
frontmatter schema is **type-dependent** (§22.4a). The `RFC-NNNN` `max+1`
allocation is **removed** — the slug is the identity (§22.4). New
`active`-entry fields: `unreviewed` (bool) and the `reviewed_at`/
`reviewed_by` provenance pair (§22.4c).
**metadata schema is collection-configured**, not type-driven (§22.4a, as
amended by v0.46.2) — a `.collection.yaml` `fields:` schema + per-entry
`<slug>.meta.yaml` sidecars. The `RFC-NNNN` `max+1` allocation is
**removed** — the slug is the identity (§22.4). New `active`-entry fields:
`unreviewed` (bool) and the `reviewed_at`/`reviewed_by` provenance pair
(§22.4c).
- **§2.4 State machine.** The `(no entry) ─[idea-PR merged]→` transition
targets the collection's `initial_state` (§22.4b); a new `active
─[mark-reviewed, Owner]→ active` self-transition clears the `unreviewed`
+1 -1
View File
@@ -1 +1 @@
0.46.0
0.54.0
+13
View File
@@ -130,3 +130,16 @@ CLOUDFLARE_TURNSTILE_SECRET=
# config drift surfaces as a loud 500 rather than a silent abuse-
# defense disablement.
TURNSTILE_REQUIRED=false
# --- Deployed-environment E2E test auth (v0.52.0) ---
# DANGER: NEVER set these on a production deployment. Together they
# enable `POST /auth/test/login`, which mints an authenticated OWNER
# session for the one configured email without any OTC/email round trip
# — it exists only to run the Playwright E2E suite against a deployed
# pre-prod (PPE) host that has no Mailpit sink. The route is fail-closed:
# it returns 404 unless BOTH vars below are set, requires the caller to
# present E2E_TEST_AUTH_SECRET in the `X-Test-Auth-Secret` header
# (constant-time compare), and only ever mints the single configured
# email (any other → 403). Leave BOTH unset everywhere except PPE.
# E2E_TEST_AUTH_EMAIL=e2e-owner@example.test
# E2E_TEST_AUTH_SECRET= # a Secret Manager ref on real deployments; never a literal here
+93 -16
View File
@@ -29,6 +29,7 @@ from . import (
api_invitations,
api_join_requests,
api_memberships,
api_metadata,
api_notifications,
api_prs,
auth,
@@ -41,6 +42,7 @@ from . import (
docs_specs,
entry as entry_mod,
cache,
facets,
funder,
health,
notify,
@@ -129,6 +131,8 @@ def make_router(
router.include_router(api_prs.make_router(config, gitea, bot, providers))
# Slice 5: §13 graduation + §13.1 claim.
router.include_router(api_graduation.make_router(config, gitea, bot))
# §22.4a SLICE-4/5: entry metadata edit + Owner-gated collection migrate.
router.include_router(api_metadata.make_router(config, gitea, bot))
# Slice 6: §15 notifications surface (inbox, watches, prefs,
# quiet hours, per-user mute, email unsubscribe, bounce webhook).
router.include_router(api_notifications.make_router(config))
@@ -651,6 +655,7 @@ def make_router(
f"""
SELECT r.slug, r.title, r.state, r.rfc_id, r.repo,
r.owners_json, r.arbiters_json, r.tags_json,
r.metadata_malformed,
r.last_main_commit_at, r.last_entry_commit_at, r.updated_at
FROM cached_rfcs r JOIN collections c ON c.id = r.collection_id
WHERE r.state IN ('super-draft', 'active')
@@ -684,6 +689,7 @@ def make_router(
"last_active_at": r["last_main_commit_at"] or r["last_entry_commit_at"] or r["updated_at"],
"starred_by_me": r["slug"] in starred,
"has_open_prs": False, # wired in Slice 2 when per-RFC repos exist
"metadata_malformed": bool(r["metadata_malformed"]),
}
)
return {"items": items}
@@ -718,6 +724,9 @@ def make_router(
(slug,),
).fetchone()
payload["proposed_use_case"] = uc["use_case"] if uc else None
# §22.4a SLICE-4: contributor+ on the entry's collection may edit metadata.
payload["can_edit_meta"] = bool(
auth.can_contribute_in_collection(viewer, auth.collection_of_rfc(slug)))
return payload
# ---------------------------------------------------------------
@@ -735,16 +744,41 @@ def make_router(
raise HTTPException(404, "Not found")
def _list_rfcs_for_collection(
collection_id: str, viewer, unreviewed: str | None
collection_id: str, viewer, unreviewed: str | None,
query_params=None,
) -> dict[str, Any]:
viewer_id = viewer.user_id if viewer else None
# §22.4a SLICE-3: the collection's declared field schema drives the facet
# set (None when undeclared → no facets, INV-5).
col = collections_mod.get_collection(collection_id)
fields_schema = (col or {}).get("fields") or None
# Parse + validate filter selections from the query string. Unknown
# field → 400 (§6.4). `unreviewed` keeps its existing meaning; an
# empty-valued selection is ignored, not an error (plan decision 6).
selections: dict[str, set[str]] = {}
only_malformed = False
if query_params is not None:
allowed = facets.allowed_filter_keys(fields_schema)
facet_names = {n for n, _ in facets.facet_fields(fields_schema)}
for key in query_params.keys():
if key not in allowed:
raise HTTPException(400, f"unknown filter field {key!r}")
if (query_params.get("malformed") or "").lower() in ("1", "true", "yes"):
only_malformed = True
for name in facet_names:
vals = {v for v in query_params.getlist(name) if v != ""}
if vals:
selections[name] = vals
unreviewed_clause = ""
if unreviewed is not None and unreviewed.lower() in ("1", "true", "yes"):
unreviewed_clause = " AND unreviewed = 1 AND state = 'active'"
rows = db.conn().execute(
f"""
SELECT slug, title, state, rfc_id, repo,
owners_json, arbiters_json, tags_json,
owners_json, arbiters_json, tags_json, metadata_malformed,
meta_json,
last_main_commit_at, last_entry_commit_at, updated_at
FROM cached_rfcs
WHERE state IN ('super-draft', 'active')
@@ -762,8 +796,16 @@ def make_router(
(viewer_id, collection_id),
)
}
items = [
{
# Build entry dicts the facet helper understands (state + malformed +
# parsed meta), preserving SQL order.
entries = []
for r in rows:
try:
meta = json.loads(r["meta_json"]) if r["meta_json"] else {}
except (TypeError, ValueError):
meta = {}
entries.append({
"slug": r["slug"],
"title": r["title"],
"state": r["state"],
@@ -775,10 +817,14 @@ def make_router(
"last_active_at": r["last_main_commit_at"] or r["last_entry_commit_at"] or r["updated_at"],
"starred_by_me": r["slug"] in starred,
"has_open_prs": False,
}
for r in rows
]
return {"items": items}
"metadata_malformed": bool(r["metadata_malformed"]),
"meta": meta,
})
filtered, facet_counts = facets.filter_and_count(
entries, fields_schema, selections, only_malformed=only_malformed
)
return {"items": filtered, "facets": facet_counts}
def _get_rfc_for_collection(collection_id: str, slug: str, viewer) -> dict[str, Any]:
row = db.conn().execute(
@@ -799,6 +845,9 @@ def make_router(
(slug, collection_id),
).fetchone()
payload["proposed_use_case"] = uc["use_case"] if uc else None
# §22.4a SLICE-4: contributor+ on the collection may edit metadata (INV-4).
payload["can_edit_meta"] = bool(
auth.can_contribute_in_collection(viewer, collection_id))
return payload
@router.get("/api/projects/{project_id}/rfcs")
@@ -810,7 +859,9 @@ def make_router(
auth.require_project_readable(viewer, project_id)
# §22 S1: the project-scoped route serves the default collection.
collection_id = collections_mod.default_collection_id(project_id)
return _list_rfcs_for_collection(collection_id, viewer, unreviewed)
return _list_rfcs_for_collection(
collection_id, viewer, unreviewed, query_params=request.query_params
)
@router.get("/api/projects/{project_id}/rfcs/{slug}")
async def get_project_rfc(project_id: str, slug: str, request: Request) -> dict[str, Any]:
@@ -832,7 +883,9 @@ def make_router(
_require_collection_in_project(collection_id, project_id)
# §22.5 (S3): a hidden/gated collection 404s to a non-scope-role viewer.
auth.require_collection_readable(viewer, collection_id)
return _list_rfcs_for_collection(collection_id, viewer, unreviewed)
return _list_rfcs_for_collection(
collection_id, viewer, unreviewed, query_params=request.query_params
)
@router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}")
async def get_collection_rfc(
@@ -866,14 +919,18 @@ def make_router(
raise HTTPException(404, "Not found")
if row["state"] != "active" or not row["unreviewed"]:
raise HTTPException(409, "Entry is not an unreviewed active entry")
# §22/G-15: write to the entry's project content_repo + collection
# subfolder, not the deployment default.
org, meta_repo, md_path = projects_mod.entry_location(config, collection_id, slug)
try:
await bot.mark_entry_reviewed(
viewer.as_actor(),
org=config.gitea_org,
meta_repo=(projects_mod.default_content_repo(config) or ""),
org=org,
meta_repo=meta_repo,
slug=slug,
reviewed_by=viewer.gitea_login,
reviewed_at=entry_mod.today(),
file_path=md_path,
)
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
@@ -978,7 +1035,19 @@ def make_router(
# Read the proposed entry file from the head branch.
slug = row["rfc_slug"]
head = row["head_branch"]
result = await gitea.read_file(config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref=head)
# §22/G-15: the proposal lives in its project's content_repo under its
# collection's `<subfolder>/rfcs/`. cached_prs carries project_id but not
# collection_id, so resolve the repo from the project and locate the file
# by trying each of the project's collection subfolders (default first).
repo = (projects_mod.content_repo(row["project_id"])
or projects_mod.default_content_repo(config) or "")
result = None
for col in collections_mod.list_collections(row["project_id"], include_unlisted=True):
sub = col["subfolder"] or ""
cand = f"{sub}/rfcs/{slug}.md" if sub else f"rfcs/{slug}.md"
result = await gitea.read_file(config.gitea_org, repo, cand, ref=head)
if result:
break
entry_payload: dict[str, Any] | None = None
if result:
text, _sha = result
@@ -1192,7 +1261,9 @@ def make_router(
await bot.merge_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=(projects_mod.default_content_repo(config) or ""),
# §22/G-15: the idea PR lives in its project's content_repo.
meta_repo=(projects_mod.content_repo(row["project_id"])
or projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
)
@@ -1212,7 +1283,8 @@ def make_router(
await bot.decline_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=(projects_mod.default_content_repo(config) or ""),
meta_repo=(projects_mod.content_repo(row["project_id"])
or projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
comment=body.comment,
@@ -1236,7 +1308,8 @@ def make_router(
await bot.withdraw_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=(projects_mod.default_content_repo(config) or ""),
meta_repo=(projects_mod.content_repo(row["project_id"])
or projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
)
@@ -1342,6 +1415,10 @@ def _serialize_rfc(row) -> dict[str, Any]:
"arbiters": json.loads(row["arbiters_json"] or "[]"),
"tags": json.loads(row["tags_json"] or "[]"),
"body": row["body"] or "",
"metadata_malformed": bool(row["metadata_malformed"]),
# §22.4a SLICE-4: the full per-entry metadata mapping (known + custom
# fields) so the detail panel can render schema-driven controls.
"meta": json.loads(row["meta_json"] or "{}"),
}
+107 -35
View File
@@ -29,7 +29,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, projects as projects_mod
from . import auth, cache, chat as chat_layer, collections as collections_mod, db, entry as entry_mod, funder, metadata as metadata_mod, models_resolver, projects as projects_mod
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -40,6 +40,37 @@ log = logging.getLogger(__name__)
RFC_FILE_PATH = "RFC.md"
# ---------------------------------------------------------------------------
# §22.4a SLICE-4: sidecar-aware body extract/wrap (pure, unit-testable)
# ---------------------------------------------------------------------------
def _extract_body_pure(rfc, file_contents: str, branch: str, *, is_meta: bool) -> str:
"""Editable body of an entry file. Meta-resident files carry a frontmatter
envelope (legacy) or are already body-only (migrated, §22.4a); per-RFC repo
files are body-only. Dual-read tolerant: a body-only `.md` returns as-is."""
if not is_meta:
return file_contents
return metadata_mod.strip_frontmatter(file_contents)
def _wrap_body_pure(rfc, prior_contents: str, new_body: str, branch: str, *, is_meta: bool) -> str:
"""Inverse of `_extract_body_pure`. Under §22.4a the body lives in the `.md`
and metadata in the sidecar, so wrapping is identity for body-only files —
frontmatter is never re-grown here. A legacy un-migrated meta file still has
its metadata in the `.md` frontmatter (no sidecar yet), so preserve it rather
than silently dropping it on a pure body edit; it is migrated to body-only on
its next *metadata* edit."""
nb = new_body if new_body.endswith("\n") else new_body + "\n"
if not is_meta:
return nb
if entry_mod.FRONTMATTER_RE.match(prior_contents):
e = entry_mod.parse(prior_contents)
e.body = nb
return entry_mod.serialize(e)
return nb
# ---------------------------------------------------------------------------
# Request bodies
# ---------------------------------------------------------------------------
@@ -139,10 +170,7 @@ def make_router(
# open edit branches, open meta-repo body-edit and metadata PRs.
# -------------------------------------------------------------------
@router.get("/api/rfcs/{slug}/main")
async def get_rfc_main(slug: str, request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
rfc = _require_rfc(slug, viewer)
async def _main_payload(rfc, slug: str, viewer) -> dict[str, Any]:
if rfc["state"] not in ("active", "super-draft"):
raise HTTPException(409, f"RFC is {rfc['state']}")
@@ -263,6 +291,23 @@ def make_router(
"pre_graduation_history": pre_grad,
}
@router.get("/api/rfcs/{slug}/main")
async def get_rfc_main(slug: str, request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
return await _main_payload(_require_rfc(slug, viewer), slug, viewer)
@router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}/main")
async def get_rfc_main_scoped(
project_id: str, collection_id: str, slug: str, request: Request,
) -> dict[str, Any]:
# §22/G-15: collection-scoped canonical-body read. Disambiguates a slug
# that exists in two collections (G-5) and resolves the entry's own
# content repo / subfolder via `_repo_for`/`_file_path_for`.
viewer = auth.current_user(request)
_check_collection_in_project(project_id, collection_id)
rfc = _require_rfc(slug, viewer, collection_id=collection_id)
return await _main_payload(rfc, slug, viewer)
# The bare `GET /api/rfcs/<slug>/branches/<branch>` is declared
# at the *bottom* of this router so the more-specific deeper GET
# routes — `branches/{branch:path}/threads` and
@@ -458,6 +503,7 @@ def make_router(
org=owner,
meta_repo=repo,
slug=slug,
file_path=path,
new_file_contents=new_content,
prior_sha=prior_sha,
pr_title=pr_title,
@@ -1017,10 +1063,7 @@ def make_router(
# else, including slashed branch names like `foo/bar`.
# -------------------------------------------------------------------
@router.get("/api/rfcs/{slug}/branches/{branch:path}")
async def get_branch_view(slug: str, branch: str, request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
rfc = _require_rfc_with_repo(slug, viewer)
async def _branch_view_payload(rfc, slug: str, branch: str, viewer) -> dict[str, Any]:
if not _can_read_branch(slug, branch, viewer):
raise HTTPException(403, "Branch is private")
@@ -1087,12 +1130,48 @@ def make_router(
"capabilities": capabilities,
}
@router.get("/api/rfcs/{slug}/branches/{branch:path}")
async def get_branch_view(slug: str, branch: str, request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
rfc = _require_rfc_with_repo(slug, viewer)
return await _branch_view_payload(rfc, slug, branch, viewer)
@router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}/branches/{branch:path}")
async def get_branch_view_scoped(
project_id: str, collection_id: str, slug: str, branch: str, request: Request,
) -> dict[str, Any]:
# §22/G-15: collection-scoped branch-body read (the canonical-body GET
# RFCView renders). Disambiguates a slug across collections (G-5) and
# reads the entry's own content repo / subfolder.
viewer = auth.current_user(request)
_check_collection_in_project(project_id, collection_id)
rfc = _require_rfc_with_repo(slug, viewer, collection_id=collection_id)
return await _branch_view_payload(rfc, slug, branch, viewer)
# ------------------------------------------------------------------
# Permission + state helpers (closures, share `config` etc.)
# ------------------------------------------------------------------
def _require_rfc(slug: str, viewer):
row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone()
def _check_collection_in_project(project_id: str, collection_id: str) -> None:
"""§22/G-15: a collection-scoped route 404s when the collection isn't in
the named project (matches api_metadata's guard)."""
if collections_mod.project_of_collection(collection_id) != project_id:
raise HTTPException(404, "Collection not in project")
def _require_rfc(slug: str, viewer, collection_id: str | None = None):
# §22/G-15: when a collection_id is supplied (the collection-scoped
# body-read routes), scope the lookup to that collection so a slug that
# exists in two collections resolves unambiguously (G-5); otherwise the
# legacy slug-only lookup picks the entry by slug alone.
if collection_id is not None:
row = db.conn().execute(
"SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id "
"FROM cached_rfcs WHERE slug = ? AND collection_id = ?",
(slug, collection_id)).fetchone()
else:
row = db.conn().execute(
"SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id "
"FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone()
if row is None:
raise HTTPException(404, "RFC not found")
# §22.5 visibility gate (subtractive, §22.7): a gated project's entries
@@ -1100,13 +1179,13 @@ def make_router(
auth.require_project_readable(viewer, row["project_id"])
return row
def _require_rfc_with_repo(slug: str, viewer):
def _require_rfc_with_repo(slug: str, viewer, collection_id: str | None = None):
"""Used by every branch-scoped endpoint. Under the meta-only
topology (§1) the meta repo is the implicit target for every
entry — super-draft and active alike — so there is no per-RFC
repo check. The name is retained for call-site stability; a
withdrawn entry is still rejected."""
row = _require_rfc(slug, viewer)
row = _require_rfc(slug, viewer, collection_id)
if row["state"] == "withdrawn":
raise HTTPException(409, "RFC is withdrawn")
return row
@@ -1152,39 +1231,32 @@ def make_router(
return _is_meta_branch_name(branch)
def _repo_for(rfc, branch: str = "main") -> tuple[str, str]:
# §22/G-15: a meta-resident entry's repo is its COLLECTION's project
# content_repo (collection → project → content_repo), not the deployment
# default — so an entry in a non-default project reads/writes its own
# repo. `entry_location` falls back to the default repo for a legacy /
# unknown collection, preserving the single-corpus behaviour.
if _is_meta_target(rfc, branch):
return config.gitea_org, (projects_mod.default_content_repo(config) or "")
org, repo, _ = projects_mod.entry_location(config, rfc["collection_id"], rfc["slug"])
return org, repo
owner, repo = rfc["repo"].split("/", 1)
return owner, repo
def _file_path_for(rfc, branch: str = "main") -> str:
# §22/G-15: path is the collection's `<subfolder>/rfcs/<slug>.md`
# (repo root `rfcs/<slug>.md` for a default collection).
if _is_meta_target(rfc, branch):
return f"rfcs/{rfc['slug']}.md"
_, _, path = projects_mod.entry_location(config, rfc["collection_id"], rfc["slug"])
return path
return RFC_FILE_PATH
def _extract_body(rfc, file_contents: str, branch: str = "main") -> str:
"""For super-draft entries (and active-RFC pre-graduation reads
per §9.8) the file on disk is the full frontmatter+body envelope;
the editable body is entry.body. For active RFCs reading their
per-RFC repo the file is just RFC.md and the whole thing is body."""
if not _is_meta_target(rfc, branch):
return file_contents
try:
entry = entry_mod.parse(file_contents)
except Exception:
return file_contents
return entry.body
return _extract_body_pure(
rfc, file_contents, branch, is_meta=_is_meta_target(rfc, branch))
def _wrap_body(rfc, prior_contents: str, new_body: str, branch: str = "main") -> str:
"""Inverse of _extract_body: re-wrap a new body into the entry
envelope, preserving the prior frontmatter exactly."""
if not _is_meta_target(rfc, branch):
return new_body
entry = entry_mod.parse(prior_contents)
# Ensure exactly one trailing newline so the serializer's
# round-trip is stable.
entry.body = new_body if new_body.endswith("\n") else new_body + "\n"
return entry_mod.serialize(entry)
return _wrap_body_pure(
rfc, prior_contents, new_body, branch, is_meta=_is_meta_target(rfc, branch))
async def _refresh_cache_for(rfc) -> None:
if _is_meta_resident(rfc):
+74 -67
View File
@@ -42,7 +42,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from . import auth, cache, db, entry as entry_mod, projects as projects_mod
from . import auth, cache, db, entry as entry_mod, metadata as metadata_mod, projects as projects_mod
from .bot import Actor, Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -341,19 +341,18 @@ def make_router(
if _rfc_id_taken(rfc_id, excluding_slug=slug):
raise HTTPException(409, f"Integer ID {rfc_id} is already taken")
# Read the meta-repo entry once — we need the file's sha for the
# graduation PR's update_file call and the body to carry through
# Dual-read the meta-repo entry once (§22.4a sidecar-aware) — we need its
# git state for the graduation commit and the body to carry through
# unchanged (meta-only keeps the body in the entry, §13.3).
fetched = await gitea.read_file(
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
meta_text, meta_sha = fetched
try:
super_draft_entry = entry_mod.parse(meta_text)
except Exception as e:
raise HTTPException(500, f"Meta entry malformed: {e}")
# §22/G-15: resolve the repo+path from the entry's COLLECTION, not the
# deployment default, so an entry in a non-default project graduates in
# its own content repo / collection subfolder.
org, meta_repo, md_path = projects_mod.entry_location(
config, rfc["collection_id"], slug)
st = await metadata_mod.read_entry_from_git(gitea, org, meta_repo, md_path)
if st is None:
raise HTTPException(409, f"Meta entry {md_path} not found on main")
super_draft_entry = st.entry
arbiters = json.loads(rfc["arbiters_json"] or "[]") or owners[:1]
@@ -376,8 +375,12 @@ def make_router(
models=super_draft_entry.models,
funder=super_draft_entry.funder,
body=super_draft_entry.body,
# INV-7 (§22.4a): carry forward-compat / unknown frontmatter keys
# through graduation rather than dropping them on the rebuild.
extra=dict(super_draft_entry.extra),
)
graduated_contents = entry_mod.serialize(graduated_entry)
graduation_files = metadata_mod.write_entry_files(
md_path, graduated_entry, st)
state = _new_active(
slug, rfc_id=rfc_id, owners=owners, arbiters=arbiters,
@@ -397,8 +400,8 @@ def make_router(
coro = _orchestrate(
config=config, gitea=gitea, bot=bot,
actor=viewer.as_actor(), state=state,
graduated_contents=graduated_contents,
meta_file_sha=meta_sha,
meta_repo=meta_repo,
graduation_files=graduation_files,
)
if request.query_params.get("_sync") == "1":
await coro
@@ -478,26 +481,23 @@ def make_router(
if already:
raise HTTPException(409, f"A claim PR is already open: #{already['pr_number']}")
fetched = await gitea.read_file(
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
meta_text, meta_sha = fetched
try:
ent = entry_mod.parse(meta_text)
except Exception as e:
raise HTTPException(500, f"Meta entry malformed: {e}")
if viewer.gitea_login in ent.owners:
# §22/G-15: resolve repo+path from the entry's collection, not the default.
org, meta_repo, md_path = projects_mod.entry_location(
config, rfc["collection_id"], slug)
st = await metadata_mod.read_entry_from_git(gitea, org, meta_repo, md_path)
if st is None:
raise HTTPException(409, f"Meta entry {md_path} not found on main")
if viewer.gitea_login in st.entry.owners:
return {"ok": True, "noop": True}
ent.owners = ent.owners + [viewer.gitea_login]
new_contents = entry_mod.serialize(ent)
ent = metadata_mod.apply_values(
st.entry, {"owners": st.entry.owners + [viewer.gitea_login]})
files = metadata_mod.write_entry_files(md_path, ent, st)
try:
pr = await bot.open_claim_pr(
viewer.as_actor(),
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
org=org, meta_repo=meta_repo,
slug=slug,
new_file_contents=new_contents, prior_sha=meta_sha,
files=files,
)
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
@@ -524,11 +524,15 @@ def make_router(
403, "Only this RFC's owners or a site owner may retire it"
)
prior_state = rfc["state"]
entry, sha = await _read_meta_entry(slug)
entry.state = "retired"
# §22/G-15: resolve repo+path from the entry's collection, not the default.
org, meta_repo, md_path = projects_mod.entry_location(
config, rfc["collection_id"], slug)
st = await _read_meta_entry(org, meta_repo, md_path)
entry = metadata_mod.apply_values(st.entry, {"state": "retired"})
files = metadata_mod.write_entry_files(md_path, entry, st)
await _run_state_flip(
config=config, gitea=gitea, bot=bot, actor=viewer.as_actor(),
slug=slug, new_contents=entry_mod.serialize(entry), prior_sha=sha,
meta_repo=meta_repo, slug=slug, files=files,
verb="retire", target_state="retired",
)
_audit(
@@ -552,13 +556,17 @@ def make_router(
viewer = auth.require_user(request)
if viewer.role != "owner":
raise HTTPException(403, "Only a site owner may un-retire an RFC")
_require_retired(slug)
rfc = _require_retired(slug)
restored = _prior_state_before_retire(slug)
entry, sha = await _read_meta_entry(slug)
entry.state = restored
# §22/G-15: resolve repo+path from the entry's collection, not the default.
org, meta_repo, md_path = projects_mod.entry_location(
config, rfc["collection_id"], slug)
st = await _read_meta_entry(org, meta_repo, md_path)
entry = metadata_mod.apply_values(st.entry, {"state": restored})
files = metadata_mod.write_entry_files(md_path, entry, st)
await _run_state_flip(
config=config, gitea=gitea, bot=bot, actor=viewer.as_actor(),
slug=slug, new_contents=entry_mod.serialize(entry), prior_sha=sha,
meta_repo=meta_repo, slug=slug, files=files,
verb="unretire", target_state=restored,
)
_audit(
@@ -599,17 +607,15 @@ def make_router(
raise HTTPException(409, f"RFC is {row['state']}, not retired")
return row
async def _read_meta_entry(slug: str) -> tuple[entry_mod.Entry, str]:
fetched = await gitea.read_file(
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
text, file_sha = fetched
try:
return entry_mod.parse(text), file_sha
except Exception as e:
raise HTTPException(500, f"Meta entry malformed: {e}")
async def _read_meta_entry(org: str, repo: str, md_path: str):
"""Dual-read an entry from meta-main → EntryGitState (sidecar-aware,
§22.4a). A migrated body-only `.md` reads cleanly; never raises on bad
metadata (INV-3). `org`/`repo`/`md_path` are collection-resolved by the
caller (§22/G-15) so a non-default project's entry reads its own repo."""
st = await metadata_mod.read_entry_from_git(gitea, org, repo, md_path)
if st is None:
raise HTTPException(409, f"Meta entry {md_path} not found on main")
return st
async def _refresh_catalog() -> None:
# Inline refresh so the catalog reflects the flip immediately; the
@@ -637,8 +643,8 @@ async def _orchestrate(
bot: Bot,
actor: Actor,
state: GraduationState,
graduated_contents: str,
meta_file_sha: str,
meta_repo: str,
graduation_files: list[dict],
) -> None:
"""Open the flip PR, then merge it. Two steps, no transaction:
@@ -656,10 +662,9 @@ async def _orchestrate(
try:
pr = await bot.open_graduation_pr(
actor,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
org=config.gitea_org, meta_repo=meta_repo,
slug=state.slug,
new_file_contents=graduated_contents,
prior_sha=meta_file_sha,
files=graduation_files,
rfc_id=state.rfc_id,
owners=state.owners,
)
@@ -676,14 +681,15 @@ async def _orchestrate(
try:
await bot.merge_graduation_pr(
actor,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
org=config.gitea_org, meta_repo=meta_repo,
pr_number=state.new_pr_number,
head_branch=state.graduation_branch or "",
slug=state.slug, rfc_id=state.rfc_id,
)
except GiteaError as e:
await _fail(state, "merge_pr", f"Gitea: {e.detail}")
await _cleanup_unmerged(config=config, bot=bot, actor=actor, state=state)
await _cleanup_unmerged(
config=config, bot=bot, actor=actor, state=state, meta_repo=meta_repo)
await _finish_failed(state, failed_at="merge_pr", on_behalf_of=actor.gitea_login)
return
await _done(state, "merge_pr", f"PR #{state.new_pr_number} merged")
@@ -726,7 +732,7 @@ async def _orchestrate(
async def _cleanup_unmerged(
*, config: Config, bot: Bot, actor: Actor, state: GraduationState,
*, config: Config, bot: Bot, actor: Actor, state: GraduationState, meta_repo: str,
) -> None:
"""A merge failure leaves the flip PR open on its `graduate-<slug>-<hex>`
branch. Close the PR and delete the branch so failed attempts don't
@@ -738,7 +744,7 @@ async def _cleanup_unmerged(
try:
await bot.close_graduation_pr(
actor,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
org=config.gitea_org, meta_repo=meta_repo,
pr_number=state.new_pr_number,
head_branch=state.graduation_branch or "",
slug=state.slug, reason="graduation merge failed",
@@ -751,7 +757,7 @@ async def _cleanup_unmerged(
await bot.delete_branch(
actor,
owner=config.gitea_org,
repo=(projects_mod.default_content_repo(config) or ""),
repo=meta_repo,
branch=branch_name,
slug=state.slug,
action_kind="delete_post_merge_branch",
@@ -846,13 +852,14 @@ async def _run_state_flip(
gitea: Gitea,
bot: Bot,
actor: Actor,
meta_repo: str,
slug: str,
new_contents: str,
prior_sha: str,
files: list[dict],
verb: str,
target_state: str,
) -> None:
"""§13.7: open + merge a retire / un-retire frontmatter flip PR. Runs
"""§13.7: open + merge a retire / un-retire state-flip PR. The flip is
written to the entry's metadata sidecar (§22.4a) via `files` ops. Runs
inline (no SSE — the flip is a single quick state change, unlike the
multi-step graduation that streams progress). On an open failure
nothing was created; on a merge failure the half-open PR/branch is
@@ -861,8 +868,8 @@ async def _run_state_flip(
try:
pr = await bot.open_retire_flip_pr(
actor,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=slug, new_file_contents=new_contents, prior_sha=prior_sha,
org=config.gitea_org, meta_repo=meta_repo,
slug=slug, files=files,
verb=verb, target_state=target_state,
)
except GiteaError as e:
@@ -872,7 +879,7 @@ async def _run_state_flip(
try:
await bot.merge_retire_flip_pr(
actor,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
org=config.gitea_org, meta_repo=meta_repo,
pr_number=pr_number, head_branch=head_branch,
slug=slug, verb=verb,
)
@@ -881,12 +888,12 @@ async def _run_state_flip(
# accumulate (mirrors graduation's `_cleanup_unmerged`).
try:
await bot.close_graduation_pr(
actor, org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
actor, org=config.gitea_org, meta_repo=meta_repo,
pr_number=pr_number, head_branch=head_branch,
slug=slug, reason=f"{verb} merge failed",
)
await bot.delete_branch(
actor, owner=config.gitea_org, repo=(projects_mod.default_content_repo(config) or ""),
actor, owner=config.gitea_org, repo=meta_repo,
branch=head_branch, slug=slug,
action_kind="delete_post_merge_branch",
reason=f"{verb} merge failed",
+215
View File
@@ -0,0 +1,215 @@
"""§22.4a SLICE-4/5 — entry metadata edit endpoints.
`POST .../rfcs/<slug>/meta` writes schema-defined metadata to an entry's sidecar
with a direct commit (D7: direct commit for authorized roles), validated against
the collection's field schema at the write boundary (INV-4), lazy-migrating a
legacy entry to a clean body-only `.md` on first edit. The Owner-gated
`metadata.migrate_collection` operator endpoint also lives here (SLICE-4 carried
work); SLICE-5's bulk endpoint will join it.
"""
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from . import (auth, cache, collections as collections_mod,
metadata as metadata_mod, metadata_schema,
projects as projects_mod)
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
class MetaEditBody(BaseModel):
values: dict[str, Any]
class BulkMetaBody(BaseModel):
slugs: list[str]
op: str
field: str
value: Any = None
def _apply_op(entry: Any, op: str, field: str, value: Any) -> Any:
"""Return the new value for `field` after applying `op` to `entry`.
`set` → `value`; `add`/`remove` operate on the entry's current tags-list
value for `field` (the route restricts add/remove to tags-type fields).
"""
if op == "set":
return value
current = metadata_mod.metadata_dict(entry).get(field) or []
if not isinstance(current, list):
current = [current]
if op == "add":
return current if value in current else [*current, value]
if op == "remove":
return [x for x in current if x != value]
return value # unreachable; op validated by the route
def make_router(config: Config, gitea: Gitea, bot: Bot) -> APIRouter:
router = APIRouter()
def _content_repo(collection_id: str) -> tuple[str, str]:
# §22/G-15: the COLLECTION's project content_repo, not the deployment
# default — an entry in a non-default project writes its own repo.
# Falls back to the default repo for an unknown collection.
repo = (projects_mod.content_repo_for_collection(collection_id)
or (projects_mod.default_content_repo(config) or ""))
return config.gitea_org, repo
def _md_path(collection_id: str, slug: str) -> str:
sub = collections_mod.subfolder_of(collection_id) or ""
rfcs_dir = f"{sub}/rfcs" if sub else "rfcs"
return f"{rfcs_dir}/{slug}.md"
@router.post("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}/meta")
async def edit_meta(
project_id: str, collection_id: str, slug: str,
body: MetaEditBody, request: Request,
) -> dict[str, Any]:
viewer = auth.current_user(request)
if collections_mod.project_of_collection(collection_id) != project_id:
raise HTTPException(404, "Collection not in project")
# INV-4: contributor+ on the collection (returns False for anonymous).
if not auth.can_contribute_in_collection(viewer, collection_id):
raise HTTPException(403, "Contributor access required to edit metadata")
col = collections_mod.get_collection(collection_id)
fields = (col or {}).get("fields") or {}
if not fields:
raise HTTPException(422, "Collection declares no editable fields")
if not body.values:
raise HTTPException(422, "Provide at least one field value")
unknown = [k for k in body.values if k not in fields]
if unknown:
raise HTTPException(422, f"Unknown field(s): {', '.join(sorted(unknown))}")
org, repo = _content_repo(collection_id)
md_path = _md_path(collection_id, slug)
st = await metadata_mod.read_entry_from_git(gitea, org, repo, md_path)
if st is None:
raise HTTPException(404, f"{md_path} not found")
# Validate the *raw* submitted values (INV-4): catch a type mismatch
# before `apply_values` coerces it — e.g. a scalar handed to a `tags`
# field would otherwise char-split into a valid-looking list.
problems = metadata_schema.validate(body.values, fields)
if problems:
raise HTTPException(422, {"problems": [p.as_dict() for p in problems]})
new_entry = metadata_mod.apply_values(st.entry, body.values)
files = metadata_mod.write_entry_files(md_path, new_entry, st)
try:
await bot.commit_entry_files(
viewer.as_actor(), org=org, repo=repo, files=files,
message=f"Edit metadata: {slug}", branch="main")
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
await cache.refresh_meta_repo(config, gitea)
return {
"ok": True, "slug": slug,
"meta": metadata_mod.metadata_dict(new_entry),
}
@router.post("/api/projects/{project_id}/collections/{collection_id}/meta/bulk")
async def bulk_meta(
project_id: str, collection_id: str,
body: BulkMetaBody, request: Request,
) -> dict[str, Any]:
"""§22.4a PUC-2 (SLICE-5): apply one field op to many entries at once.
`set` works for any field; `add`/`remove` operate on a tags-type field.
Each passing entry's metadata is validated at the write boundary
(INV-4) and its sidecar staged; all stage into **one** commit (D7:
bulk = 1 commit, reusing the SLICE-4 sidecar write-through). Entries
that are missing or fail validation are reported in `rejected`; the
rest in `applied`. A no-op (value unchanged) is applied without writing.
"""
viewer = auth.current_user(request)
if collections_mod.project_of_collection(collection_id) != project_id:
raise HTTPException(404, "Collection not in project")
# INV-4: contributor+ on the collection (returns False for anonymous).
if not auth.can_contribute_in_collection(viewer, collection_id):
raise HTTPException(403, "Contributor access required to edit metadata")
col = collections_mod.get_collection(collection_id)
fields = (col or {}).get("fields") or {}
if not fields:
raise HTTPException(422, "Collection declares no editable fields")
if not body.slugs:
raise HTTPException(422, "Provide at least one entry")
if body.op not in ("set", "add", "remove"):
raise HTTPException(422, f"Unknown op: {body.op}")
if body.field not in fields:
raise HTTPException(422, f"Unknown field: {body.field}")
if body.op in ("add", "remove") and fields[body.field].get("type") != "tags":
raise HTTPException(422, f"op {body.op} requires a tags field")
org, repo = _content_repo(collection_id)
applied: list[str] = []
rejected: list[dict[str, str]] = []
all_ops: list[dict[str, Any]] = []
for slug in body.slugs:
md_path = _md_path(collection_id, slug)
st = await metadata_mod.read_entry_from_git(gitea, org, repo, md_path)
if st is None:
rejected.append({"slug": slug, "reason": "not found"})
continue
new_value = _apply_op(st.entry, body.op, body.field, body.value)
# Validate the *raw* new value before coercion (see edit_meta) so a
# scalar `set` onto a tags field is rejected, not char-split.
problems = metadata_schema.validate({body.field: new_value}, fields)
if problems:
rejected.append({"slug": slug,
"reason": "; ".join(p.message for p in problems)})
continue
applied.append(slug)
new_entry = metadata_mod.apply_values(st.entry, {body.field: new_value})
if metadata_mod.metadata_dict(new_entry) != metadata_mod.metadata_dict(st.entry):
all_ops.extend(metadata_mod.write_entry_files(md_path, new_entry, st))
committed = False
if all_ops:
n = len(applied)
msg = f"Bulk {body.op} {body.field}: {n} entr{'y' if n == 1 else 'ies'}"
try:
await bot.commit_entry_files(
viewer.as_actor(), org=org, repo=repo, files=all_ops,
message=msg, branch="main")
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
committed = True
await cache.refresh_meta_repo(config, gitea)
return {"ok": True, "applied": applied,
"rejected": rejected, "committed": committed}
@router.post("/api/projects/{project_id}/collections/{collection_id}/migrate")
async def migrate(
project_id: str, collection_id: str, request: Request
) -> dict[str, Any]:
"""§22.4a PUC-5: migrate a collection's legacy-frontmatter entries to
clean body-only `.md` + sidecars, one commit per collection. Owner-gated
operator action. Safe to ship now that every entry write path is
sidecar-aware (SLICE-4 carried work). Idempotent."""
viewer = auth.current_user(request)
if collections_mod.project_of_collection(collection_id) != project_id:
raise HTTPException(404, "Collection not in project")
if not auth.is_collection_superuser(viewer, collection_id):
raise HTTPException(403, "Owner access required to migrate a collection")
org, repo = _content_repo(collection_id)
subfolder = collections_mod.subfolder_of(collection_id) or ""
try:
result = await metadata_mod.migrate_collection(
gitea, org=org, repo=repo, subfolder=subfolder,
actor=viewer.as_actor())
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
if result["committed"]:
await cache.refresh_meta_repo(config, gitea)
return result
return router
+22 -11
View File
@@ -23,7 +23,7 @@ from typing import Any
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, projects as projects_mod, rfc_links
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, metadata as metadata_mod, models_resolver, projects as projects_mod, rfc_links
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -690,14 +690,20 @@ def make_router(
return not rfc["repo"]
def _owner_repo(rfc) -> tuple[str, str]:
# §22/G-15: a meta-resident entry's repo is its COLLECTION's project
# content_repo, not the deployment default (entry_location falls back to
# the default for a legacy/unknown collection).
if _is_meta_resident(rfc):
return config.gitea_org, (projects_mod.default_content_repo(config) or "")
org, repo, _ = projects_mod.entry_location(config, rfc["collection_id"], rfc["slug"])
return org, repo
owner, repo = rfc["repo"].split("/", 1)
return owner, repo
def _file_path_for(rfc) -> str:
# §22/G-15: the collection's `<subfolder>/rfcs/<slug>.md`.
if _is_meta_resident(rfc):
return f"rfcs/{rfc['slug']}.md"
_, _, path = projects_mod.entry_location(config, rfc["collection_id"], rfc["slug"])
return path
return RFC_FILE_PATH
def _extract_body(rfc, file_contents: str) -> str:
@@ -1009,20 +1015,25 @@ async def _replay_changes(
def _extract_body_for_replay(is_super_draft: bool, content: str) -> str:
# §22.4a SLICE-4: a meta-resident entry may be legacy (frontmatter+body) or
# migrated (body-only). strip_frontmatter handles both without raising.
if not is_super_draft:
return content
try:
return entry_mod.parse(content).body
except Exception:
return content
return metadata_mod.strip_frontmatter(content)
def _wrap_body_for_replay(is_super_draft: bool, prior_content: str, new_body: str) -> str:
# §22.4a SLICE-4: identity for body-only (migrated) files — never re-grow
# frontmatter; preserve a legacy file's frontmatter until its next metadata
# edit migrates it.
nb = new_body if new_body.endswith("\n") else new_body + "\n"
if not is_super_draft:
return new_body
entry = entry_mod.parse(prior_content)
entry.body = new_body if new_body.endswith("\n") else new_body + "\n"
return entry_mod.serialize(entry)
return nb
if entry_mod.FRONTMATTER_RE.match(prior_content):
entry = entry_mod.parse(prior_content)
entry.body = nb
return entry_mod.serialize(entry)
return nb
def _resolution_branch_name(original_branch: str) -> str:
+100 -85
View File
@@ -27,7 +27,7 @@ import json
import logging
from dataclasses import dataclass
from . import db, entry as entry_mod, notify
from . import db, entry as entry_mod, metadata as metadata_mod, notify
from .gitea import Gitea, GiteaError
log = logging.getLogger(__name__)
@@ -404,6 +404,43 @@ class Bot:
pr_number=pr_number,
)
# ----- Entry sidecar writes (§22.4a SLICE-4) -----
async def commit_entry_files(
self, actor: Actor, *, org: str, repo: str,
files: list[dict], message: str, branch: str = "main",
) -> dict:
"""Commit a set of entry file ops (sidecar + body-only `.md`, from
`metadata.write_entry_files`) in one commit. Used by the direct-commit
metadata paths and, on a branch, by `open_entry_pr`."""
return await self._gitea.change_files(
org, repo, files=files,
message=_stamp_single(message, actor), branch=branch,
author_name=actor.display_name,
author_email=actor.email or f"{actor.gitea_login}@users.noreply",
)
async def open_entry_pr(
self, actor: Actor, *, org: str, repo: str, slug: str,
files: list[dict], pr_title: str, pr_description: str,
branch_prefix: str = "metadata",
) -> dict:
"""Create a branch, commit entry file ops there, and open a PR — the
sidecar-aware successor to `open_metadata_pr`'s single-file write."""
import secrets
branch = f"{branch_prefix}-{slug}-{secrets.token_hex(3)}"
await self._gitea.create_branch(org, repo, branch, from_branch="main")
await self.commit_entry_files(
actor, org=org, repo=repo, files=files,
message=pr_title, branch=branch)
_subject, pr_body = _stamp("", pr_description, actor)
pr = await self._gitea.create_pull(
org, repo, title=pr_title, body=pr_body, head=branch, base="main")
_log(actor, "open_entry_pr", rfc_slug=slug, branch_name=branch,
pr_number=pr["number"], details={"pr_title": pr_title})
return pr
# ----- Meta repo: metadata-pane PRs (§9.5) -----
async def open_metadata_pr(
@@ -413,17 +450,19 @@ class Bot:
org: str,
meta_repo: str,
slug: str,
file_path: str,
new_file_contents: str,
prior_sha: str,
pr_title: str,
pr_description: str,
) -> dict:
"""Per §9.5: a metadata-pane edit (title or tags) on a super-draft
opens a tiny meta-repo PR that touches only the frontmatter of
`rfcs/<slug>.md`. One commit, one PR, easy to triage. The branch
name uses the dash-separated `metadata-<slug>-<6hex>` shape same
routing-friendly form Slice 4 picked for edit branches per the
§19.2 path-routing candidate.
opens a tiny meta-repo PR that touches only the frontmatter of the
entry's `.md`. `file_path` is the collection-resolved path (§22/G-15:
`<subfolder>/rfcs/<slug>.md`), not assumed to be at the repo root. One
commit, one PR, easy to triage. The branch name uses the dash-separated
`metadata-<slug>-<6hex>` shape same routing-friendly form Slice 4
picked for edit branches per the §19.2 path-routing candidate.
"""
import secrets
@@ -434,7 +473,7 @@ class Bot:
result = await self._gitea.update_file(
org,
meta_repo,
f"rfcs/{slug}.md",
file_path,
content=new_file_contents,
sha=prior_sha,
message=commit_message,
@@ -819,35 +858,29 @@ class Bot:
org: str,
meta_repo: str,
slug: str,
new_file_contents: str,
prior_sha: str,
files: list[dict],
rfc_id: str | None,
owners: list[str],
) -> dict:
"""§13.3 (meta-only): open a PR against the meta repo that flips the
entry's frontmatter to `state: active` with the graduation stamps
and **optionally** the integer `id`, **keeping the body
unchanged** (§1 meta-only topology; no repo is created and no body
is stripped). When `rfc_id` is None the entry graduates without a
number (id stays null, slug is canonical per §2.3, §13.2). Branch
name uses the `graduate-<slug>-<6hex>` shape dash-separated like
the other meta-repo branches per the §19.2 path-routing candidate.
entry to `state: active` with the graduation stamps and **optionally**
the integer `id`, **keeping the body unchanged** (§1 meta-only
topology; no repo is created and no body is stripped). The graduation
metadata is written to the entry's sidecar (§22.4a) via `files`; a legacy
`.md` is lazy-migrated to body-only in the same commit. When `rfc_id` is
None the entry graduates without a number (id stays null, slug is
canonical per §2.3, §13.2). Branch name uses the `graduate-<slug>-<6hex>`
shape dash-separated like the other meta-repo branches per the §19.2
path-routing candidate.
"""
import secrets
branch = f"graduate-{slug}-{secrets.token_hex(3)}"
await self._gitea.create_branch(org, meta_repo, branch, from_branch="main")
ae = actor.email or f"{actor.gitea_login}@users.noreply"
commit_subject = f"Graduate {slug}{rfc_id}" if rfc_id else f"Graduate {slug} (no number)"
commit_message = _stamp_single(commit_subject, actor)
result = await self._gitea.update_file(
org, meta_repo, f"rfcs/{slug}.md",
content=new_file_contents,
sha=prior_sha,
message=commit_message,
branch=branch,
author_name=actor.display_name, author_email=ae,
)
result = await self.commit_entry_files(
actor, org=org, repo=meta_repo, files=files,
message=commit_subject, branch=branch)
commit_sha = (
result.get("commit", {}).get("sha")
or result.get("content", {}).get("sha")
@@ -936,35 +969,26 @@ class Bot:
org: str,
meta_repo: str,
slug: str,
new_file_contents: str,
prior_sha: str,
files: list[dict],
verb: str,
target_state: str,
) -> dict:
"""§13.7: open a PR flipping `rfcs/<slug>.md` to `state:
<target_state>` for retire (`verb='retire'`, target `retired`)
or un-retire (`verb='unretire'`, target the restored prior state).
Only the frontmatter `state` changes; the body and every other
field (including the integer `id`) are kept, so an un-retire
restores the entry exactly. Branch shape mirrors graduation's
`<verb>-<slug>-<6hex>`.
"""§13.7: open a PR flipping an entry to `state: <target_state>` — for
retire (`verb='retire'`, target `retired`) or un-retire
(`verb='unretire'`, target the restored prior state). The `state` change
is written to the entry's metadata sidecar (§22.4a), keeping the `.md`
body and every other field, so an un-retire restores the entry exactly.
`files` come from `metadata.write_entry_files`. Branch shape mirrors
graduation's `<verb>-<slug>-<6hex>`.
"""
import secrets
branch = f"{verb}-{slug}-{secrets.token_hex(3)}"
await self._gitea.create_branch(org, meta_repo, branch, from_branch="main")
ae = actor.email or f"{actor.gitea_login}@users.noreply"
verb_title = "Retire" if verb == "retire" else "Un-retire"
commit_subject = f"{verb_title} {slug}"
commit_message = _stamp_single(commit_subject, actor)
result = await self._gitea.update_file(
org, meta_repo, f"rfcs/{slug}.md",
content=new_file_contents,
sha=prior_sha,
message=commit_message,
branch=branch,
author_name=actor.display_name, author_email=ae,
)
result = await self.commit_entry_files(
actor, org=org, repo=meta_repo, files=files,
message=f"{verb_title} {slug}", branch=branch)
commit_sha = (
result.get("commit", {}).get("sha")
or result.get("content", {}).get("sha")
@@ -1134,30 +1158,23 @@ class Bot:
org: str,
meta_repo: str,
slug: str,
new_file_contents: str,
prior_sha: str,
files: list[dict],
) -> dict:
"""§13.1: open a PR adding the actor to the entry's `owners:` list.
Touches only the frontmatter of `rfcs/<slug>.md`. Branch shape is
`claim/<slug>` single attempt per super-draft per actor (Gitea
refuses duplicate branch creation, which is the right behavior:
if the claim is still open, point the contributor at the existing
PR rather than opening a second one).
Writes the updated `owners:` to the entry's metadata sidecar (§22.4a)
via `files`; a legacy `.md` is lazy-migrated to body-only in the same
commit. Branch shape is `claim/<slug>` single attempt per super-draft
per actor (Gitea refuses duplicate branch creation, which is the right
behavior: if the claim is still open, point the contributor at the
existing PR rather than opening a second one).
"""
branch = f"claim/{slug}"
await self._gitea.create_branch(org, meta_repo, branch, from_branch="main")
ae = actor.email or f"{actor.gitea_login}@users.noreply"
commit_subject = f"Claim ownership of {slug} for {actor.gitea_login}"
commit_message = _stamp_single(commit_subject, actor)
result = await self._gitea.update_file(
org, meta_repo, f"rfcs/{slug}.md",
content=new_file_contents,
sha=prior_sha,
message=commit_message,
branch=branch,
author_name=actor.display_name, author_email=ae,
)
result = await self.commit_entry_files(
actor, org=org, repo=meta_repo, files=files,
message=commit_subject, branch=branch)
commit_sha = (
result.get("commit", {}).get("sha")
or result.get("content", {}).get("sha")
@@ -1195,30 +1212,28 @@ class Bot:
slug: str,
reviewed_by: str,
reviewed_at: str,
file_path: str | None = None,
) -> None:
"""Clear §22.4c unreviewed on an active entry by rewriting its
frontmatter on main. Stamps the commit with the §6.5 On-behalf-of
trailer and writes an actions-log row, mirroring the graduation
stamp's bot-write shape."""
path = f"rfcs/{slug}.md"
result = await self._gitea.read_file(org, meta_repo, path, ref="main")
if result is None:
"""Clear §22.4c unreviewed on an active entry by writing its metadata
sidecar on main (§22.4a). Dual-reads the entry (so a migrated body-only
`.md` doesn't crash) and lazy-migrates a legacy `.md` to body-only in the
same commit. Stamps the §6.5 On-behalf-of trailer and writes an
actions-log row, mirroring the graduation stamp's bot-write shape.
`file_path` is the collection-resolved entry path (§22/G-15:
`<subfolder>/rfcs/<slug>.md`); it defaults to the repo-root
`rfcs/<slug>.md` for the legacy single-corpus / default-collection case."""
path = file_path or f"rfcs/{slug}.md"
st = await metadata_mod.read_entry_from_git(self._gitea, org, meta_repo, path)
if st is None:
raise GiteaError(404, f"{path} not found")
text, sha = result
e = entry_mod.parse(text)
e.unreviewed = False
e.reviewed_at = reviewed_at
e.reviewed_by = reviewed_by
commit_message = _stamp_single(f"Mark {slug} reviewed", actor)
result = await self._gitea.update_file(
org, meta_repo, path,
content=entry_mod.serialize(e),
sha=sha,
message=commit_message,
branch="main",
author_name=actor.display_name,
author_email=actor.email or f"{actor.gitea_login}@users.noreply",
)
e = metadata_mod.apply_values(st.entry, {
"unreviewed": False, "reviewed_at": reviewed_at, "reviewed_by": reviewed_by,
})
files = metadata_mod.write_entry_files(path, e, st)
result = await self.commit_entry_files(
actor, org=org, repo=meta_repo, files=files,
message=f"Mark {slug} reviewed", branch="main")
commit_sha = (
result.get("commit", {}).get("sha")
or result.get("content", {}).get("sha")
+130 -59
View File
@@ -27,7 +27,15 @@ import asyncio
import json
import logging
from . import db, entry as entry_mod, projects as projects_mod, registry as registry_mod
from . import (
collections as collections_mod,
db,
entry as entry_mod,
metadata as metadata_mod,
metadata_schema,
projects as projects_mod,
registry as registry_mod,
)
from .config import Config
from .gitea import Gitea, GiteaError
@@ -77,6 +85,22 @@ async def _refresh_collection_corpus(
project_id, collection_id, rfcs_dir, e)
return
# §22.4a SLICE-2: a collection may declare a metadata field schema. Fetch it
# once for the whole corpus pass; entries whose stored values fail it are
# flagged malformed advisory-only (INV-3) — the read never hard-fails. A
# collection with no schema validates nothing (INV-5, the default unchanged).
col = collections_mod.get_collection(collection_id)
fields_schema = (col or {}).get("fields") or None
# §22.4a SLICE-1: an entry's metadata may live in a `<slug>.meta.yaml`
# sidecar (the source of truth) with the `.md` kept as pure prose. Map the
# sidecars surfaced by this listing so each `.md` can dual-read its sibling.
sidecar_path_by_slug = {
metadata_mod.slug_of_sidecar(f["name"]): f["path"]
for f in files
if f.get("type") == "file" and metadata_mod.is_sidecar(f.get("name", ""))
}
seen_slugs: set[str] = set()
for f in files:
if f.get("type") != "file" or not f.get("name", "").endswith(".md"):
@@ -85,8 +109,14 @@ async def _refresh_collection_corpus(
if not result:
continue
text, sha = result
stem = f["name"][:-len(".md")]
sidecar_text: str | None = None
sidecar_path = sidecar_path_by_slug.get(stem)
if sidecar_path:
sc_result = await gitea.read_file(org, repo, sidecar_path, ref="main")
sidecar_text = sc_result[0] if sc_result else None
try:
entry = entry_mod.parse(text)
entry, malformed = metadata_mod.read_entry(text, sidecar_text, fallback_slug=stem)
except Exception as parse_err:
log.warning("refresh_meta_repo: %s/%s: skipping %s: %s",
project_id, collection_id, f["path"], parse_err)
@@ -95,8 +125,24 @@ async def _refresh_collection_corpus(
log.warning("refresh_meta_repo: %s/%s: skipping %s: missing slug",
project_id, collection_id, f["path"])
continue
if malformed:
log.warning("refresh_meta_repo: %s/%s: %s has malformed metadata sidecar",
project_id, collection_id, f["path"])
# §22.4a SLICE-2: advisory schema validation (INV-3). A schema violation
# flags the entry malformed without blocking the read, OR-ed onto any
# sidecar-syntax malformation above.
if fields_schema:
problems = metadata_schema.validate(
metadata_mod.metadata_dict(entry), fields_schema
)
if problems:
malformed = True
log.warning("refresh_meta_repo: %s/%s: %s fails its field schema: %s",
project_id, collection_id, f["path"],
"; ".join(p.message for p in problems))
seen_slugs.add(entry.slug)
_upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id)
_upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id,
metadata_malformed=malformed)
# Entries removed from a collection's rfcs/ — the spec keeps withdrawn entries
# as historical record (§3), so this fires only for out-of-band deletes;
@@ -112,13 +158,22 @@ async def _refresh_collection_corpus(
project_id, collection_id, missing)
def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str = "default") -> None:
def _upsert_cached_rfc(
entry: entry_mod.Entry,
body_sha: str,
collection_id: str = "default",
metadata_malformed: bool = False,
) -> None:
# §6.6: models_json stays NULL when the frontmatter key is absent
# (inherit operator universe) and '[]' for the explicit opt-out.
models_json = json.dumps(entry.models) if entry.models is not None else None
# §6.7: funder_login mirrors the optional `funder:` frontmatter
# field. NULL means absent — operator credentials are used.
funder_login = entry.funder or None
# §22.4a SLICE-3: persist the full per-entry metadata mapping (known keys +
# extra, never the body) so facet/filter can read any declared field. Stored
# via metadata_dict so the sidecar's forward-compat keys (INV-7) ride along.
meta_json = json.dumps(metadata_mod.metadata_dict(entry))
db.conn().execute(
"""
INSERT INTO cached_rfcs
@@ -126,8 +181,8 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str
graduated_at, graduated_by, owners_json, arbiters_json, tags_json,
models_json, funder_login, body, body_sha,
unreviewed, reviewed_at, reviewed_by, collection_id,
last_entry_commit_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now'))
metadata_malformed, meta_json, last_entry_commit_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now'))
ON CONFLICT(collection_id, slug) DO UPDATE SET
title = excluded.title,
state = excluded.state,
@@ -147,6 +202,8 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str
unreviewed = excluded.unreviewed,
reviewed_at = excluded.reviewed_at,
reviewed_by = excluded.reviewed_by,
metadata_malformed = excluded.metadata_malformed,
meta_json = excluded.meta_json,
last_entry_commit_at = datetime('now'),
updated_at = datetime('now')
""",
@@ -171,6 +228,8 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str
entry.reviewed_at,
entry.reviewed_by,
collection_id,
1 if metadata_malformed else 0,
meta_json,
),
)
@@ -367,73 +426,85 @@ async def refresh_meta_branches(config: Config, gitea: Gitea) -> None:
of slashes per the §19.2 path-routing candidate.
"""
org = config.gitea_org
repo = projects_mod.default_content_repo(config)
if not repo:
log.warning("refresh_meta_branches: default project has no content_repo yet; skipping")
return
try:
branches = await gitea.list_branches(org, repo)
except GiteaError as e:
log.warning("refresh_meta_branches: %s", e)
# §22/G-15: scan EVERY project's content_repo (not just the default), so an
# entry in a non-default project gets its edit branches + synthesized `main`
# row cached and its branch dropdown / has-commits-ahead check work. Mirrors
# refresh_meta_pulls' per-project loop.
prows = db.conn().execute(
"SELECT id, content_repo FROM projects WHERE content_repo IS NOT NULL AND content_repo != ''"
).fetchall()
if not prows:
log.warning("refresh_meta_branches: no projects with a content_repo yet; skipping")
return
meta_main_sha = ""
meta_main_ts = None
edit_keys_seen: set[tuple[str, str]] = set()
for b in branches:
name = b.get("name") or ""
head_sha = (b.get("commit") or {}).get("id") or ""
last_commit_at = (b.get("commit") or {}).get("timestamp")
if name == "main":
meta_main_sha = head_sha
meta_main_ts = last_commit_at
for prow in prows:
repo = prow["content_repo"]
try:
branches = await gitea.list_branches(org, repo)
except GiteaError as e:
log.warning("refresh_meta_branches: %s (%s)", e, repo)
continue
slug = _slug_from_branch_name(name)
if not slug:
continue
rfc = db.conn().execute(
"SELECT state, repo FROM cached_rfcs WHERE slug = ?", (slug,)
).fetchone()
# Meta-only topology (§1): edit branches live on the meta repo for
# every meta-resident entry — super-drafts and active RFCs alike
# (active RFCs are graduated in place and keep editing here, §13).
# A legacy per-RFC repo (repo set) is the only thing excluded.
if not rfc or rfc["repo"] or rfc["state"] not in ("super-draft", "active"):
continue
edit_keys_seen.add((slug, name))
db.conn().execute(
"""
INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at)
VALUES (?, ?, ?, 'open', ?)
ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET
head_sha = excluded.head_sha,
state = CASE WHEN cached_branches.state = 'closed' THEN 'closed' ELSE 'open' END,
last_commit_at = excluded.last_commit_at
""",
(slug, name, head_sha, last_commit_at),
)
# Synthesize a per-slug `main` row for every super-draft entry, so the
# §10.1 has-commits-ahead check in api_prs.py works uniformly. The
# head_sha is the meta-repo main's tip — every super-draft edit branch
# diverges from this single point.
if meta_main_sha:
super_drafts = db.conn().execute(
"SELECT slug FROM cached_rfcs "
"WHERE repo IS NULL AND state IN ('super-draft', 'active')"
).fetchall()
for r in super_drafts:
meta_main_sha = ""
meta_main_ts = None
for b in branches:
name = b.get("name") or ""
head_sha = (b.get("commit") or {}).get("id") or ""
last_commit_at = (b.get("commit") or {}).get("timestamp")
if name == "main":
meta_main_sha = head_sha
meta_main_ts = last_commit_at
continue
slug = _slug_from_branch_name(name)
if not slug:
continue
rfc = db.conn().execute(
"SELECT state, repo FROM cached_rfcs WHERE slug = ?", (slug,)
).fetchone()
# Meta-only topology (§1): edit branches live on the content repo for
# every meta-resident entry — super-drafts and active RFCs alike
# (active RFCs are graduated in place and keep editing here, §13).
# A legacy per-RFC repo (repo set) is the only thing excluded.
if not rfc or rfc["repo"] or rfc["state"] not in ("super-draft", "active"):
continue
edit_keys_seen.add((slug, name))
db.conn().execute(
"""
INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at)
VALUES (?, 'main', ?, 'open', ?)
VALUES (?, ?, ?, 'open', ?)
ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET
head_sha = excluded.head_sha,
state = CASE WHEN cached_branches.state = 'closed' THEN 'closed' ELSE 'open' END,
last_commit_at = excluded.last_commit_at
""",
(r["slug"], meta_main_sha, meta_main_ts),
(slug, name, head_sha, last_commit_at),
)
# Synthesize a per-slug `main` row for this project's super-draft/active
# entries, so the §10.1 has-commits-ahead check works uniformly. The
# head_sha is this content repo's main tip — every edit branch in the
# project diverges from that single point.
if meta_main_sha:
super_drafts = db.conn().execute(
"SELECT r.slug AS slug FROM cached_rfcs r "
"JOIN collections c ON c.id = r.collection_id "
"WHERE r.repo IS NULL AND r.state IN ('super-draft', 'active') "
" AND c.project_id = ?",
(prow["id"],),
).fetchall()
for r in super_drafts:
db.conn().execute(
"""
INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at)
VALUES (?, 'main', ?, 'open', ?)
ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET
head_sha = excluded.head_sha,
last_commit_at = excluded.last_commit_at
""",
(r["slug"], meta_main_sha, meta_main_ts),
)
# Mark previously-known edit branches that disappeared as deleted per
# §11.5 / §12. Keep the row so chat history survives the branch's
# deletion in Gitea.
+21 -1
View File
@@ -45,6 +45,21 @@ def _enabled_models_from_config(config_json: str | None) -> list[str] | None:
return [str(m) for m in em] if isinstance(em, list) else None
def _fields_from_config(config_json: str | None) -> dict | None:
"""§22.4a SLICE-2 per-collection metadata field schema from a `config_json`
blob, or None when the collection declares no `fields:`. The stored value is
already normalized by `metadata_schema.parse_fields` at ingest, so it's
served verbatim."""
if not config_json:
return None
try:
cfg = json.loads(config_json)
except (json.JSONDecodeError, TypeError):
return None
fields = cfg.get("fields") if isinstance(cfg, dict) else None
return fields if isinstance(fields, dict) and fields else None
def default_collection_id(project_id: str) -> str:
"""The id of a project's default (S1: sole) collection. Falls back to the
literal 'default' when the project has no collection row yet."""
@@ -102,7 +117,12 @@ def get_collection(collection_id: str) -> dict | None:
if row is None:
return None
out = dict(row)
out["enabled_models"] = _enabled_models_from_config(out.pop("config_json", None))
config_json = out.pop("config_json", None)
out["enabled_models"] = _enabled_models_from_config(config_json)
# §22.4a SLICE-2: serve the collection's metadata field schema (None when
# the collection declares no `fields:` — INV-5, the default `document`
# collection is unaffected).
out["fields"] = _fields_from_config(config_json)
out["entry_noun"] = entry_noun(out["type"])
return out
+44 -2
View File
@@ -58,6 +58,20 @@ class Entry:
reviewed_at: str | None = None
reviewed_by: str | None = None
body: str = ""
# §22.4a (configurable collection metadata, SLICE-1): frontmatter / sidecar
# keys outside the known set above are preserved here verbatim so they ride
# along untouched through a parse→serialize round-trip and the
# frontmatter→sidecar migration (INV-7). Includes future collection-`fields:`
# schema values, which the engine does not interpret.
extra: dict[str, Any] = field(default_factory=dict)
# Frontmatter keys the Entry models explicitly; everything else is `extra`.
KNOWN_KEYS = {
"slug", "title", "state", "id", "repo", "proposed_by", "proposed_at",
"graduated_at", "graduated_by", "owners", "arbiters", "tags", "models",
"funder", "unreviewed", "reviewed_at", "reviewed_by",
}
def parse(text: str) -> Entry:
@@ -66,6 +80,16 @@ def parse(text: str) -> Entry:
raise ValueError("Entry file missing frontmatter")
fm = yaml.safe_load(match.group(1)) or {}
body = match.group(2).lstrip("\n")
return from_frontmatter(fm, body)
def from_frontmatter(fm: dict[str, Any], body: str = "") -> Entry:
"""Build an Entry from an already-parsed metadata mapping + body.
Shared by `parse()` (legacy `.md` frontmatter) and the SLICE-1 dual-read
sidecar path (`metadata.read_entry`), so both produce identical records
(INV-6). `fm` keys outside `KNOWN_KEYS` are preserved on `Entry.extra`.
"""
raw_models = fm.get("models", _ABSENT)
if raw_models is _ABSENT or raw_models is None:
models: list[str] | None = None
@@ -74,6 +98,7 @@ def parse(text: str) -> Entry:
raw_funder = fm.get("funder")
funder = str(raw_funder).strip() if raw_funder else None
unreviewed = bool(fm.get("unreviewed") or False)
extra = {k: v for k, v in fm.items() if k not in KNOWN_KEYS}
return Entry(
slug=str(fm.get("slug") or ""),
title=str(fm.get("title") or ""),
@@ -93,11 +118,18 @@ def parse(text: str) -> Entry:
reviewed_at=fm.get("reviewed_at") or None,
reviewed_by=fm.get("reviewed_by") or None,
body=body,
extra=extra,
)
def serialize(entry: Entry) -> str:
"""Emit canonical entry file text — frontmatter then body."""
def to_frontmatter_dict(entry: Entry) -> dict[str, Any]:
"""The canonical ordered metadata mapping for an entry.
Shared by `serialize()` (which wraps it in `---` fences over the body) and
the SLICE-1 sidecar writer (`metadata.sidecar_yaml`, which emits the same
mapping as a standalone `<slug>.meta.yaml`). Known keys first in canonical
order, then `extra` (INV-7).
"""
fm: dict[str, Any] = {
"slug": entry.slug,
"title": entry.title,
@@ -129,6 +161,16 @@ def serialize(entry: Entry) -> str:
fm["reviewed_at"] = entry.reviewed_at
if entry.reviewed_by:
fm["reviewed_by"] = entry.reviewed_by
# INV-7: forward-compat / unknown keys ride along after the known ones.
for k, v in entry.extra.items():
if k not in fm:
fm[k] = v
return fm
def serialize(entry: Entry) -> str:
"""Emit canonical entry file text — frontmatter then body."""
fm = to_frontmatter_dict(entry)
yaml_text = yaml.safe_dump(fm, sort_keys=False, default_flow_style=False).rstrip()
body = entry.body.lstrip("\n")
if body:
+110
View File
@@ -0,0 +1,110 @@
"""§22.4a SLICE-3 — faceted catalog filtering + counts (read).
Pure functions over already-mirrored entries; no I/O, no DB. An "entry" here is
a plain dict carrying at least:
- "state": the lifecycle state column,
- "metadata_malformed": bool,
- "meta": the per-entry metadata mapping (from cached_rfcs.meta_json).
Facetable fields (§5.1, plan decision 1): a collection's declared `enum` and
`tags` fields, in declaration order, plus the built-in `state` facet appended
last but only when the collection declares a schema (INV-5: a no-`fields:`
collection has no facets at all, so the frontend keeps its legacy chips). `text`
fields are not faceted in v1 (they get a detail control in SLICE-4).
Counts use drill-down semantics (plan decision 2): the count for a value of
field F is taken over entries matching every OTHER field's selection (and the
malformed toggle), not F's own — so within-field values stay switchable (OR
within a field, AND across fields). The returned items list applies ALL
selections.
"""
from __future__ import annotations
from typing import Any
# enum + tags are facetable; text is rendered as a detail control (SLICE-4).
FACETABLE_TYPES = {"enum", "tags"}
def facet_fields(fields: dict[str, dict] | None) -> list[tuple[str, str]]:
"""Ordered `[(name, type), ...]` facetable from the schema, `state` last.
Empty when the collection declares no schema (INV-5)."""
if not fields:
return []
out = [
(name, spec.get("type"))
for name, spec in fields.items()
if spec.get("type") in FACETABLE_TYPES
]
out.append(("state", "enum"))
return out
def allowed_filter_keys(fields: dict[str, dict] | None) -> set[str]:
"""Query-param keys the collection-scoped list accepts (plan decision 6)."""
keys = {name for name, _ in facet_fields(fields)}
keys.update({"unreviewed", "malformed"})
return keys
def _values_for(entry: dict[str, Any], name: str, ftype: str) -> list[str]:
"""The facet value(s) an entry contributes for field `name` (str-cast)."""
if name == "state":
v = entry.get("state")
return [str(v)] if v else []
meta = entry.get("meta") or {}
v = meta.get(name)
if v is None:
return []
if ftype == "tags":
return [str(x) for x in v] if isinstance(v, list) else []
return [str(v)]
def _matches(entry: dict[str, Any], name: str, ftype: str, selected: set[str]) -> bool:
if not selected:
return True
return bool(set(_values_for(entry, name, ftype)) & selected) # OR within field
def filter_and_count(
entries: list[dict[str, Any]],
fields: dict[str, dict] | None,
selections: dict[str, set[str]],
only_malformed: bool = False,
) -> tuple[list[dict[str, Any]], dict[str, dict[str, int]]]:
"""Filter `entries` by `selections` and compute drill-down facet counts.
`selections` maps a facet field name the set of selected values (OR within
the field; AND across fields). `only_malformed` narrows items and counts to
entries flagged malformed (INV-3). Returns `(items, facets)` where
`facets = {field: {value: count}}`. With no schema `([all passing], {})`.
"""
facetable = facet_fields(fields)
def passes_malformed(e: dict[str, Any]) -> bool:
return (not only_malformed) or bool(e.get("metadata_malformed"))
items = [
e for e in entries
if passes_malformed(e)
and all(_matches(e, n, t, selections.get(n, set())) for n, t in facetable)
]
facets: dict[str, dict[str, int]] = {}
for name, ftype in facetable:
counts: dict[str, int] = {}
for e in entries:
if not passes_malformed(e):
continue
if not all(
_matches(e, on, ot, selections.get(on, set()))
for on, ot in facetable
if on != name
):
continue
for val in _values_for(e, name, ftype):
counts[val] = counts.get(val, 0) + 1
facets[name] = counts
return items, facets
+34
View File
@@ -193,6 +193,40 @@ class Gitea:
resp = await self._request("PUT", f"/repos/{owner}/{repo}/contents/{path}", json=body)
return resp.json()
async def change_files(
self,
owner: str,
repo: str,
*,
files: list[dict[str, Any]],
message: str,
branch: str,
author_name: str | None = None,
author_email: str | None = None,
) -> dict:
"""Create/update/delete several files in ONE commit (Gitea ChangeFiles).
Each `files` entry is `{"operation": "create"|"update"|"delete",
"path": str, "content": str (for create/update), "sha": str (required
for update/delete)}`. Plaintext `content` is base64-encoded here.
Backs the §22.4a frontmattersidecar migration's "one commit per
collection" (`metadata_migrate`).
"""
out_files: list[dict[str, Any]] = []
for f in files:
item: dict[str, Any] = {"operation": f["operation"], "path": f["path"]}
if "content" in f and f["content"] is not None:
item["content"] = base64.b64encode(f["content"].encode("utf-8")).decode("ascii")
if f.get("sha"):
item["sha"] = f["sha"]
out_files.append(item)
body: dict[str, Any] = {"message": message, "branch": branch, "files": out_files}
if author_name and author_email:
body["author"] = {"name": author_name, "email": author_email}
body["committer"] = {"name": author_name, "email": author_email}
resp = await self._request("POST", f"/repos/{owner}/{repo}/contents", json=body)
return resp.json()
# ----- Pull requests -----
async def list_pulls(self, owner: str, repo: str, state: str = "open") -> list[dict]:
+5 -4
View File
@@ -293,20 +293,21 @@ async def _delete_branch_via_bot(
(we leave the branch row in place a subsequent reconciler sweep
will reconcile or the operator can intervene)."""
rfc = db.conn().execute(
"SELECT state, repo FROM cached_rfcs WHERE slug = ?", (slug,)
"SELECT state, repo, collection_id FROM cached_rfcs WHERE slug = ?", (slug,)
).fetchone()
if rfc is None:
log.warning("hygiene: cannot delete %s/%s — slug missing from cache", slug, branch)
return False
if not rfc["repo"]:
repo = projects_mod.default_content_repo(config)
# §22/G-15: the edit/graduation branch lives on the entry's COLLECTION's
# project content_repo, not the deployment default.
owner, repo, _ = projects_mod.entry_location(config, rfc["collection_id"], slug)
if not repo:
log.warning(
"hygiene: default project has no content_repo; skipping branch delete for %s/%s",
"hygiene: no content_repo resolved; skipping branch delete for %s/%s",
slug, branch,
)
return False
owner = config.gitea_org
elif "/" in rfc["repo"]:
owner, repo = rfc["repo"].split("/", 1)
else:
+81
View File
@@ -66,6 +66,13 @@ class OtcVerifyBody(BaseModel):
trust_device: bool = False
class TestLoginBody(BaseModel):
# The single configured test identity to sign in as. Must equal
# E2E_TEST_AUTH_EMAIL (case-insensitive) or the request is refused —
# see `/auth/test/login`.
email: str = Field(min_length=3, max_length=320)
class PasscodeSetBody(BaseModel):
passcode: str = Field(min_length=1, max_length=64)
@@ -101,7 +108,26 @@ async def lifespan(app: FastAPI):
config = load_config()
db.run_migrations(config)
db.init(config)
# v0.52.0: shout if the deployed-env E2E test-auth shortcut is live.
# It mints owner sessions for one configured identity (see
# `/auth/test/login`); it must only ever be on for a pre-prod (PPE)
# host. A loud startup line means an accidental prod enablement is
# visible in the logs rather than silent.
if os.environ.get("E2E_TEST_AUTH_SECRET", "").strip() and os.environ.get(
"E2E_TEST_AUTH_EMAIL", ""
).strip():
log.warning(
"E2E TEST-AUTH IS ENABLED: POST /auth/test/login will mint an owner "
"session for %s. This must NEVER be set on production.",
os.environ["E2E_TEST_AUTH_EMAIL"].strip(),
)
gitea = Gitea(config)
# §22 framework heal: reconcile a divergent default-project collection id
# (migration 029's ≥2-projects seed names it after the project, e.g. 'ohm',
# but the mirror expects 'default') BEFORE the mirror runs, so the mirror
# merges onto the canonical 'default' collection instead of duplicating it.
# Idempotent no-op on fresh / single-project / already-aligned deployments.
projects.reconcile_default_collection_id(config)
# §22.2: mirror the registry before anything reads projects/content_repo.
# First boot has no last-good rows, so a missing/invalid registry is fatal
# (loud-fail per separation-of-concerns); the reconciler sweep keeps it
@@ -380,6 +406,61 @@ def _oauth_router(config) -> APIRouter:
"needs_profile": needs_profile,
}
# ---------------------------------------------------------------
# v0.52.0: deployed-environment E2E test-auth shortcut.
#
# Running the Playwright E2E suite against a *deployed* environment
# (PPE) is the §9 pre-prod gate. But the deployed env has neither of
# the two scaffolds the Tier-1 docker stack relies on for auth: a
# Mailpit sink to read the OTC code from, and direct SQLite access to
# inject a granted-owner row. This endpoint replaces both with a
# single gated gesture: it mints an authenticated OWNER session for
# one pre-configured throwaway identity.
#
# It is FAIL-CLOSED and must never function in production:
# * 404 unless BOTH `E2E_TEST_AUTH_SECRET` and `E2E_TEST_AUTH_EMAIL`
# are set — a prod deployment that sets neither cannot be coaxed
# into minting a session, and the route is invisible.
# * The caller must present the shared secret in `X-Test-Auth-Secret`
# (constant-time compare); a wrong/absent secret 404s (the route
# does not advertise itself to an unauthenticated caller).
# * Only the one configured email may be minted; any other address
# is refused (403). So an enabled PPE exposes exactly one
# throwaway owner identity, with the secret as the trust boundary.
#
# The hard secrets rule (§6.3) holds: the secret is a Secret Manager
# ref injected as env on the VM (never a literal in the repo), and the
# E2E runner presents it from SM at runtime (never echoed).
@router.post("/auth/test/login")
async def test_login(body: TestLoginBody, request: Request):
secret = os.environ.get("E2E_TEST_AUTH_SECRET", "").strip()
configured_email = os.environ.get("E2E_TEST_AUTH_EMAIL", "").strip()
# Feature off (the default, incl. production): route is invisible.
if not secret or not configured_email:
raise HTTPException(404, "Not Found")
presented = request.headers.get("x-test-auth-secret", "")
if not secrets.compare_digest(presented, secret):
# Don't reveal that the route exists to a caller without the
# secret — mirror the "off" shape exactly.
raise HTTPException(404, "Not Found")
if body.email.strip().lower() != configured_email.lower():
raise HTTPException(403, "email not permitted")
# Provision-or-link the row, then force it to a granted owner so
# the metadata write paths (SLICE-4/5) accept it — the deployed
# equivalent of the Tier-1 docker-compose backend-seed owner row.
user = otc.provision_or_link_user(body.email)
db.conn().execute(
"UPDATE users SET role = 'owner', permission_state = 'granted', "
"last_seen_at = datetime('now') WHERE id = ?",
(user.user_id,),
)
db.conn().commit()
user.role = "owner"
user.permission_state = "granted"
auth.store_session(request, user)
return {"ok": True}
# ---------------------------------------------------------------
# v0.10.0: user-set passcodes after OTC (§6.2, roadmap item #8).
#
+311
View File
@@ -0,0 +1,311 @@
"""§22.4a configurable collection metadata — sidecar storage + dual-read.
SLICE-1 of docs/design/2026-06-06-configurable-collection-metadata.md.
Entry metadata is collection-configured and stored in a per-entry sidecar,
`<slug>.meta.yaml`, with the `.md` body kept as pure prose (INV-2). This module
is the storage/compat layer:
- the **dual-read** parser (`read_entry`) read the sidecar if present, else
legacy top-of-document frontmatter, with identical resulting records
(INV-6);
- sidecar (de)serialization that preserves unknown / forward-compat keys
(INV-7), reusing `entry`'s canonical field semantics;
- lenient parsing that never hard-fails a read a malformed sidecar surfaces
a flag, not an exception (INV-3).
The collection `fields:` schema and per-field validation are SLICE-2; faceted
filtering and the edit UIs are later slices. This module interprets no field
values it only moves metadata between git and in-memory `Entry` records.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
import yaml
from . import entry as entry_mod
from .entry import Entry
SIDECAR_SUFFIX = ".meta.yaml"
# ----- filename helpers -----
def sidecar_name(slug: str) -> str:
"""The sidecar filename for an entry whose markdown is `<slug>.md`."""
return f"{slug}{SIDECAR_SUFFIX}"
def is_sidecar(name: str) -> bool:
return name.endswith(SIDECAR_SUFFIX)
def slug_of_sidecar(name: str) -> str:
"""The entry stem for a `<slug>.meta.yaml` filename."""
return name[: -len(SIDECAR_SUFFIX)] if is_sidecar(name) else name
def sidecar_path_for(md_path: str) -> str:
"""The sidecar path sibling to a `<dir>/<slug>.md` entry file."""
assert md_path.endswith(".md"), md_path
return md_path[: -len(".md")] + SIDECAR_SUFFIX
# ----- metadata <-> sidecar -----
def metadata_dict(entry: Entry) -> dict[str, Any]:
"""The full metadata mapping for an entry (known fields + `extra`)."""
return entry_mod.to_frontmatter_dict(entry)
def sidecar_yaml(entry: Entry) -> str:
"""Render an entry's metadata as standalone `<slug>.meta.yaml` text."""
return yaml.safe_dump(
metadata_dict(entry), sort_keys=False, default_flow_style=False
)
def parse_sidecar(text: str) -> tuple[dict[str, Any], bool]:
"""Parse sidecar YAML leniently → `(values, malformed)`.
`malformed` is True when the text is not a YAML mapping (a list, a scalar,
or a YAML syntax error). An empty / whitespace-only sidecar is an empty
mapping, not malformed. Never raises (INV-3).
"""
try:
raw = yaml.safe_load(text)
except yaml.YAMLError:
return {}, True
if raw is None:
return {}, False
if not isinstance(raw, dict):
return {}, True
return raw, False
# ----- frontmatter stripping (INV-2) -----
def strip_frontmatter(md_text: str) -> str:
"""Return the prose body of a `.md`, dropping a leading `---…---` block.
A migrated entry's `.md` is body-only and passes through unchanged. A
not-yet-migrated `.md` still carrying frontmatter yields just its body, so
the dual-read body is the same either way.
"""
match = entry_mod.FRONTMATTER_RE.match(md_text)
if not match:
return md_text
return match.group(2).lstrip("\n")
# ----- dual-read (INV-6) -----
def read_entry(
md_text: str, sidecar_text: str | None, *, fallback_slug: str | None = None
) -> tuple[Entry, bool]:
"""Read an entry from its `.md` and optional sidecar → `(Entry, malformed)`.
- **Sidecar present and well-formed (non-empty):** metadata comes from the
sidecar; the body is the `.md` stripped of any leading frontmatter. The
sidecar is the source of truth (INV-1) and wins over stale `.md`
frontmatter.
- **Sidecar present but malformed:** the entry still loads from the legacy
`.md` frontmatter (if any) and is flagged `malformed` (INV-3).
- **Sidecar present but empty:** it has nothing to override with, so fall
back to the `.md` frontmatter (not flagged).
- **No sidecar:** the legacy path parse the `.md` frontmatter (INV-6).
`fallback_slug` (typically the filename stem) backstops the entry's slug
whenever the metadata source lacks one so a degenerate sidecar never
yields a slug-less record the caller has to silently drop (INV-3).
"""
def _with_slug(entry: Entry) -> Entry:
if not entry.slug and fallback_slug:
entry.slug = fallback_slug
return entry
if sidecar_text is None:
return _with_slug(entry_mod.parse(md_text)), False
values, malformed = parse_sidecar(sidecar_text)
if malformed or not values:
# Malformed or empty sidecar: load from the legacy .md so the entry
# still loads; flag only when the sidecar was actually malformed.
try:
entry = entry_mod.parse(md_text)
except ValueError:
entry = entry_mod.from_frontmatter({}, strip_frontmatter(md_text))
return _with_slug(entry), malformed
body = strip_frontmatter(md_text)
return _with_slug(entry_mod.from_frontmatter(values, body)), False
# ----- value editing (SLICE-4) -----
def apply_values(entry: Entry, values: dict[str, Any]) -> Entry:
"""Return a new Entry with `values` merged over the entry's metadata.
Known keys (`tags`, `state`, `reviewed_by`, ) land on their typed fields;
unknown keys land on `extra` (INV-7). The body is carried through unchanged
this mutates metadata only. Unspecified keys are preserved.
"""
merged = metadata_dict(entry)
merged.update(values)
return entry_mod.from_frontmatter(merged, entry.body)
# ----- git-aware read/write (SLICE-4) -----
@dataclass
class EntryGitState:
"""An entry's on-disk state across its `.md` and optional sidecar.
Captured by `read_entry_from_git` and consumed by `write_entry_files` to
decide create-vs-update for the sidecar and whether the `.md` still needs
its frontmatter stripped (lazy migration).
"""
entry: Entry
md_text: str
md_sha: str
sidecar_text: str | None
sidecar_sha: str | None
malformed: bool
async def read_entry_from_git(
gitea: Any, org: str, repo: str, md_path: str, *, ref: str = "main"
) -> "EntryGitState | None":
"""Dual-read an entry from git → `EntryGitState`, or None if the `.md` is
missing. Reads the `.md` and its sibling sidecar (if any); never raises on
bad metadata (INV-3)."""
md = await gitea.read_file(org, repo, md_path, ref=ref)
if md is None:
return None
md_text, md_sha = md
sc_path = sidecar_path_for(md_path)
sc = await gitea.read_file(org, repo, sc_path, ref=ref)
sidecar_text, sidecar_sha = (sc[0], sc[1]) if sc else (None, None)
stem = md_path.rsplit("/", 1)[-1][: -len(".md")]
entry, malformed = read_entry(md_text, sidecar_text, fallback_slug=stem)
return EntryGitState(
entry=entry, md_text=md_text, md_sha=md_sha,
sidecar_text=sidecar_text, sidecar_sha=sidecar_sha, malformed=malformed,
)
def _md_has_frontmatter(md_text: str) -> bool:
return entry_mod.FRONTMATTER_RE.match(md_text) is not None
def write_entry_files(
md_path: str, entry: Entry, state: "EntryGitState"
) -> list[dict[str, Any]]:
"""Produce `change_files` ops that persist `entry`'s metadata to its sidecar
and keep the `.md` as pure prose (INV-1/INV-2).
- Sidecar: `create` when none existed, else `update` at its prior sha.
- `.md`: rewritten body-only **only when it still carries frontmatter**
(lazy migration, INV-6); an already-clean body is left untouched.
"""
sc_path = sidecar_path_for(md_path)
ops: list[dict[str, Any]] = []
sc_op: dict[str, Any] = {
"operation": "update" if state.sidecar_sha else "create",
"path": sc_path,
"content": sidecar_yaml(entry),
}
if state.sidecar_sha:
sc_op["sha"] = state.sidecar_sha
ops.append(sc_op)
if _md_has_frontmatter(state.md_text):
body = strip_frontmatter(state.md_text)
new_md = body if (body == "" or body.endswith("\n")) else body + "\n"
ops.append({
"operation": "update", "path": md_path,
"content": new_md, "sha": state.md_sha,
})
return ops
# ----- frontmatter -> sidecar migration (PUC-5) -----
async def migrate_collection(
gitea: Any,
*,
org: str,
repo: str,
subfolder: str = "",
actor: Any = None,
branch: str = "main",
) -> dict[str, Any]:
"""Migrate a collection's legacy-frontmatter entries to sidecars.
Walks `<subfolder>/rfcs`; for each `<slug>.md` that has legacy frontmatter
and **no** `<slug>.meta.yaml` sibling yet, it stages two file changes
create the sidecar (the entry's metadata, unknown keys preserved, INV-7)
and rewrite the `.md` to body-only (INV-2) and commits all of them in a
single ChangeFiles commit (§6.5: one commit per collection).
Idempotent: an entry that already has a sidecar is skipped; a second run
with nothing left to migrate makes no commit. Returns
`{"migrated": [...], "skipped": [...], "committed": bool}`.
"""
rfcs_dir = f"{subfolder}/rfcs" if subfolder else "rfcs"
listing = await gitea.list_dir(org, repo, rfcs_dir, ref=branch)
names = {f.get("name") for f in listing if f.get("type") == "file"}
ops: list[dict[str, Any]] = []
migrated: list[str] = []
skipped: list[str] = []
for f in listing:
if f.get("type") != "file" or not f.get("name", "").endswith(".md"):
continue
slug = f["name"][:-len(".md")]
if sidecar_name(slug) in names:
skipped.append(slug) # already migrated
continue
result = await gitea.read_file(org, repo, f["path"], ref=branch)
if not result:
continue
text, sha = result
try:
e = entry_mod.parse(text)
except ValueError:
# No frontmatter to lift (e.g. an already-clean body without a
# sidecar) — nothing to migrate; leave it untouched.
skipped.append(slug)
continue
body = strip_frontmatter(text)
new_md = body if (body == "" or body.endswith("\n")) else body + "\n"
ops.append({
"operation": "create",
"path": f"{rfcs_dir}/{sidecar_name(slug)}",
"content": sidecar_yaml(e),
})
ops.append({
"operation": "update",
"path": f["path"],
"content": new_md,
"sha": sha,
})
migrated.append(slug)
committed = False
if ops:
n = len(migrated)
message = f"Migrate {n} entr{'y' if n == 1 else 'ies'} to metadata sidecars (§22.4a)"
kwargs: dict[str, Any] = {}
if actor is not None:
kwargs = {
"author_name": actor.display_name,
"author_email": actor.email or f"{actor.gitea_login}@users.noreply",
}
await gitea.change_files(
org, repo, files=ops, message=message, branch=branch, **kwargs
)
committed = True
return {"migrated": migrated, "skipped": skipped, "committed": committed}
+147
View File
@@ -0,0 +1,147 @@
"""§22.4a configurable collection metadata — field schema + central validation.
SLICE-2 of docs/design/2026-06-06-configurable-collection-metadata.md.
A collection declares a small **field schema** in its `.collection.yaml`
(`fields:` block) so its entries can carry structured metadata priority, tags,
and any custom fields the deployment defines. This module is the **one place**
that knows a collection's field shapes (modeled on `registry.py`):
- `parse_fields` normalize + validate the declared schema, leniently: a bad
block or a bad field def is skipped with a warning, never raised, so a typo
in one field can't nuke the collection mirror (INV-3 spirit). The normalized
schema is a plain, JSON-serializable mapping that rides in
`collections.config_json` (no DB migration) and is served verbatim by the
collection API.
- `validate` check an entry's stored values against the schema, returning a
list of advisory `Problem`s. Empty list = clean. Used **advisory at read**
(the corpus mirror flags a non-empty result as `metadata_malformed`, INV-3)
and is the enforcement point at the **write** boundary (the metadata-edit
endpoints land in SLICE-4/5).
Field types (v1): `enum` (single scalar, controlled by a required `values:`
list), `tags` (a list; free-form unless `values:` given), `text` (a free
string). `ref` / `multi-enum` are future (design §2, Q2). Unknown types are
ignored with a warning. Keys an entry carries that the schema does **not**
declare ride along untouched and are never flagged (INV-7).
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import Any
log = logging.getLogger(__name__)
# §6.3 v1 field types. `ref` (typed cross-entry link) and `multi-enum` are
# deferred (design §2, Q2) — declared with an unknown type they're skipped.
VALID_FIELD_TYPES = {"enum", "tags", "text"}
@dataclass
class Problem:
"""One advisory schema-validation problem against a declared field.
`code` is a stable machine token (`not-in-values`, `wrong-type`); `message`
is human-facing (surfaced at the write boundary in SLICE-4/5)."""
field: str
code: str
message: str
def as_dict(self) -> dict[str, str]:
return {"field": self.field, "code": self.code, "message": self.message}
# ----- schema parsing (lenient) -----
def parse_fields(raw: Any) -> dict[str, dict]:
"""Normalize a `.collection.yaml` `fields:` block → `{name: {type, ...}}`.
Pure (no I/O). Lenient (INV-3): a non-mapping block yields `{}`; an
individual field def that is not a mapping, has an unknown/missing `type`, or
is an `enum` without a non-empty `values:` list is **skipped with a warning**
never raised. Order is preserved (facet display order, SLICE-3). The
result is plain dicts so it serializes straight into `config_json` and the
collection API.
"""
if not isinstance(raw, dict):
if raw is not None:
log.warning("metadata_schema: fields block is not a mapping (%s); ignoring",
type(raw).__name__)
return {}
out: dict[str, dict] = {}
for name, spec in raw.items():
if not isinstance(spec, dict):
log.warning("metadata_schema: field %r def is not a mapping; skipping", name)
continue
ftype = str(spec.get("type") or "").strip()
if ftype not in VALID_FIELD_TYPES:
log.warning("metadata_schema: field %r has unknown type %r; skipping",
name, ftype)
continue
values = spec.get("values")
norm_values: list[str] | None = None
if values is not None:
if not isinstance(values, list):
log.warning("metadata_schema: field %r values is not a list; ignoring",
name)
else:
norm_values = [str(v) for v in values]
if ftype == "enum" and not norm_values:
log.warning("metadata_schema: enum field %r needs a non-empty values "
"list; skipping", name)
continue
field_def: dict[str, Any] = {"type": ftype}
if norm_values is not None:
field_def["values"] = norm_values
label = spec.get("label")
if label:
field_def["label"] = str(label)
out[str(name)] = field_def
return out
# ----- value validation (advisory) -----
def _is_scalar(v: Any) -> bool:
return isinstance(v, (str, int, float, bool))
def validate(values: dict[str, Any], fields: dict[str, dict]) -> list[Problem]:
"""Check an entry's metadata `values` against a collection's field schema.
Returns advisory `Problem`s (empty = clean). Only **declared** fields are
checked; a field the entry omits is fine (no required fields in v1), and a
key the schema doesn't declare rides along untouched (INV-7). With an empty
schema, everything is clean (INV-5). Never raises (INV-3).
"""
problems: list[Problem] = []
for name, spec in fields.items():
if name not in values:
continue
value = values[name]
if value is None:
continue
ftype = spec.get("type")
allowed = spec.get("values")
if ftype == "enum":
if not _is_scalar(value):
problems.append(Problem(name, "wrong-type",
f"{name!r} must be a single value, got {type(value).__name__}"))
elif allowed is not None and str(value) not in allowed:
problems.append(Problem(name, "not-in-values",
f"{name!r} value {value!r} is not one of {allowed}"))
elif ftype == "tags":
if not isinstance(value, list):
problems.append(Problem(name, "wrong-type",
f"{name!r} must be a list, got {type(value).__name__}"))
elif allowed is not None:
for member in value:
if str(member) not in allowed:
problems.append(Problem(name, "not-in-values",
f"{name!r} value {member!r} is not one of {allowed}"))
elif ftype == "text":
if not _is_scalar(value):
problems.append(Problem(name, "wrong-type",
f"{name!r} must be a string, got {type(value).__name__}"))
return problems
+120
View File
@@ -92,6 +92,95 @@ def restamp_default_project(config: Config) -> None:
DEFAULT_PROJECT_ID, target, len(pid_tables))
def reconcile_default_collection_id(config: Config) -> None:
"""Heal the §22 migration-029 vs registry-mirror divergence for the default
project's collection id on a multi-project deployment.
Migration 029 seeds each project's default collection id as the literal
'default' only when the DB holds a single project at migration time; with
2 projects it falls back to the *project id* (avoiding a PK collision
029 can't read DEFAULT_PROJECT_ID, there is no env in SQL). But the registry
mirror (`registry._default_collection_id`) expects the deployment's default
project to own the collection id 'default'. On an upgrade whose DB already
held 2 projects when 029 ran, the default project's collection is therefore
named after the project (e.g. 'ohm'), and the next mirror would INSERT a
second, empty 'default' collection instead of merging duplicating the
default corpus and orphaning the entries (which point at 'ohm').
This is the collection-grain twin of `restamp_default_project`. Run at
startup BEFORE the registry mirror so the canonical 'default' collection
already exists when the mirror upserts (merge, not duplicate). Renames the
divergent collection's id to 'default' and cascades `collection_id` across
every collection-keyed table, with FK enforcement off for the atomic rename
and a `foreign_key_check` backstop before commit. Idempotent; a no-op on
fresh / single-project / already-aligned deployments.
"""
from .collections import DEFAULT_COLLECTION_ID
target = resolved_default_id(config)
if target == DEFAULT_COLLECTION_ID: # default project already owns 'default'
return
conn = db.conn()
# The 029 ≥2-projects seed names the default project's collection after the
# project itself; the canonical id the mirror expects is 'default'.
divergent = conn.execute(
"SELECT 1 FROM collections WHERE id = ? AND project_id = ? LIMIT 1",
(target, target),
).fetchone()
if not divergent:
return
if conn.execute(
"SELECT 1 FROM collections WHERE id = ? LIMIT 1", (DEFAULT_COLLECTION_ID,)
).fetchone():
# A 'default' collection already exists (e.g. a prior buggy mirror left a
# duplicate). Don't auto-merge data — that needs care; leave both for
# operator cleanup and log loudly.
log.warning(
"reconcile: default project %r owns both a %r and a 'default' "
"collection; skipping auto-rename (manual merge required)",
target, target,
)
return
cid_tables = [
t["name"]
for t in conn.execute("SELECT name FROM sqlite_master WHERE type='table'")
if any(c["name"] == "collection_id"
for c in conn.execute(f"PRAGMA table_info({t['name']})"))
]
conn.execute("PRAGMA foreign_keys = OFF")
try:
conn.execute("BEGIN")
conn.execute(
"UPDATE collections SET id = ? WHERE id = ?",
(DEFAULT_COLLECTION_ID, target),
)
for t in cid_tables:
conn.execute(
f"UPDATE {t} SET collection_id = ? WHERE collection_id = ?",
(DEFAULT_COLLECTION_ID, target),
)
violations = conn.execute("PRAGMA foreign_key_check").fetchall()
if violations:
conn.execute("ROLLBACK")
raise RuntimeError(
f"reconcile left foreign-key violations: {[tuple(v) for v in violations]}"
)
conn.execute("COMMIT")
except Exception:
try:
conn.execute("ROLLBACK")
except Exception:
pass
raise
finally:
conn.execute("PRAGMA foreign_keys = ON")
log.info(
"reconcile: renamed default-project collection %r -> 'default' across %d tables",
target, len(cid_tables),
)
def default_content_repo(config: Config) -> str | None:
"""The content repo the single-corpus mirror reads, from the default
project's row (filled by the registry mirror). Replaces the retired
@@ -112,6 +201,37 @@ def content_repo(project_id: str) -> str | None:
return row["content_repo"] if row and row["content_repo"] else None
def content_repo_for_collection(collection_id: str) -> str | None:
"""The content repo a collection's entries live in (§22 three-tier write
path, G-15): collection project content_repo. None if the collection or
its project is unknown/unset. The per-collection successor to
`default_content_repo` for the WRITE path an entry in a non-default
project must read/write that project's repo, not the deployment default."""
from . import collections as collections_mod
pid = collections_mod.project_of_collection(collection_id)
return content_repo(pid) if pid else None
def entry_location(config: Config, collection_id: str, slug: str) -> tuple[str, str, str]:
"""The git location `(gitea_org, content_repo, md_path)` of an entry, resolved
from its collection (§22 three-tier, G-15).
Repo: the collection's project content_repo, falling back to the deployment
default project's repo when the collection (or its project) is unknown — so a
legacy/single-corpus entry still resolves to a usable location rather than an
empty repo. Path: `<subfolder>/rfcs/<slug>.md`, or `rfcs/<slug>.md` at the
repo root for a default (subfolder-less) collection.
This is the single resolver the branch/edit/body/metadata/graduation write
paths share, replacing the hardcoded `default_content_repo` + `rfcs/<slug>.md`.
"""
from . import collections as collections_mod
repo = content_repo_for_collection(collection_id) or (default_content_repo(config) or "")
sub = collections_mod.subfolder_of(collection_id)
rfcs_dir = f"{sub}/rfcs" if sub else "rfcs"
return config.gitea_org, repo, f"{rfcs_dir}/{slug}.md"
def project_initial_state(project_id: str) -> str:
"""§22.4b landing state for new entries in a project's default collection
(the per-corpus field moved down to the collection in migration 029).
+22 -3
View File
@@ -19,11 +19,27 @@ lockouts). Both layers run together.
"""
from __future__ import annotations
import os
import threading
import time
from collections import defaultdict, deque
def _max_events(env_name: str, default: int) -> int:
"""Per-limiter budget, overridable via env (e.g. a test/PPE stack that
drives the auth endpoints repeatedly from one IP). Production leaves these
unset and gets the secure defaults below. A non-positive / unparseable
value falls back to the default."""
raw = os.environ.get(env_name, "").strip()
if not raw:
return default
try:
n = int(raw)
except ValueError:
return default
return n if n > 0 else default
class SlidingWindowLimiter:
"""Allow at most `max_events` per `window_seconds` per key.
@@ -66,12 +82,15 @@ class SlidingWindowLimiter:
# * verify: 10 attempts / 5 min / IP across the auth verify surfaces.
# * otc request: 5 sends / 5 min / IP (Turnstile is the primary gate;
# this is defense in depth against a solved-challenge replay loop).
verify_limiter = SlidingWindowLimiter(max_events=10, window_seconds=300)
otc_request_limiter = SlidingWindowLimiter(max_events=5, window_seconds=300)
verify_limiter = SlidingWindowLimiter(
max_events=_max_events("RATELIMIT_VERIFY_MAX", 10), window_seconds=300)
otc_request_limiter = SlidingWindowLimiter(
max_events=_max_events("RATELIMIT_OTC_REQUEST_MAX", 5), window_seconds=300)
# /auth/passcode/check is an anonymous has-passcode oracle (audit 0026 L3).
# It's a legitimate Login-flow affordance, so the budget is generous —
# enough for a human typing emails, tight enough to stop bulk scraping.
check_limiter = SlidingWindowLimiter(max_events=30, window_seconds=300)
check_limiter = SlidingWindowLimiter(
max_events=_max_events("RATELIMIT_CHECK_MAX", 30), window_seconds=300)
def _reset_all_for_tests() -> None:
+8
View File
@@ -18,6 +18,7 @@ from dataclasses import dataclass, field
import yaml
from . import db
from . import metadata_schema
from .config import Config
from .gitea import Gitea
@@ -159,6 +160,13 @@ def parse_collection_manifest(text: str) -> CollectionEntry:
if not isinstance(em, list):
raise RegistryError("collection enabled_models must be a list")
cfg["enabled_models"] = [str(m) for m in em]
# §22.4a SLICE-2: the collection's metadata field schema. Parsed leniently —
# a bad field def is skipped with a warning, never fatal (INV-3), so a typo
# in one field can't drop the whole collection from the mirror. Stored only
# when at least one valid field survives.
fields = metadata_schema.parse_fields(raw.get("fields"))
if fields:
cfg["fields"] = fields
return CollectionEntry(ctype, vis, initial_state, name, cfg)
+12 -5
View File
@@ -80,10 +80,17 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
payload = {}
repo_full = (payload.get("repository") or {}).get("full_name") or ""
registry_full = f"{config.gitea_org}/{config.registry_repo}"
content_repo = projects_mod.default_content_repo(config)
if not content_repo:
log.warning("webhook: default project content_repo is unknown; corpus refresh skipped")
content_full = f"{config.gitea_org}/{content_repo}" if content_repo else None
# §22/G-15: a corpus push can land on ANY project's content_repo, not
# just the default — recognise the full set so a non-default project's
# push triggers the (multi-project) corpus/branch/PR refresh.
content_fulls = {
f"{config.gitea_org}/{r['content_repo']}"
for r in db.conn().execute(
"SELECT content_repo FROM projects "
"WHERE content_repo IS NOT NULL AND content_repo != ''")
}
if not content_fulls:
log.warning("webhook: no project content_repo is known; corpus refresh skipped")
try:
if repo_full == registry_full:
# §22.2: a registry-repo push re-mirrors the projects table.
@@ -94,7 +101,7 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
await registry_mod.refresh_registry(config, gitea)
except registry_mod.RegistryError:
log.exception("registry webhook: invalid projects.yaml; keeping last-good")
elif content_full and (repo_full == content_full or not repo_full):
elif content_fulls and (repo_full in content_fulls or not repo_full):
await cache.refresh_meta_repo(config, gitea)
await cache.refresh_meta_branches(config, gitea)
await cache.refresh_meta_pulls(config, gitea)
+47
View File
@@ -26,6 +26,53 @@
-- they carry a project-grain tag, untouched in S1. See
-- docs/design/2026-06-05-three-tier-projects-collections.md §A.6 / Part E.
-- ── §22.13 repair: re-stamp stale satellite project_id before rekeying ──────
-- The §22.13 default→ohm re-stamp (v0.39.0, `projects.restamp_default_project`)
-- updated `cached_rfcs.project_id` but NOT the entry-satellite tables, leaving
-- rows with a stale `project_id` (e.g. 'default') that the per-project collection
-- backfill below cannot map — the subquery returns NULL and the NOT NULL rebuild
-- fails (`cached_branches__new.collection_id`). Before rebuilding, re-derive each
-- satellite's `project_id` from its entry (`cached_rfcs`, joined by slug — slugs
-- are unique per collection and, pre-rebuild, globally), and drop rows whose
-- entry no longer exists (stale cache; the `cached_*` tables are rebuildable from
-- gitea). On a clean/fresh deployment every satellite is empty or already
-- consistent, so this whole block is a no-op. (Discovered on the OHM data:
-- ~1.3k `cached_branches` rows stranded at project_id='default'.)
-- First drop stale rows that DUPLICATE an already-correctly-stamped row (the same
-- branch cached under both the stale and the real project_id) — re-stamping them
-- would collide on the (project_id, rfc_slug, branch_name) key. The correctly-
-- stamped copy is kept (it carries the current head_sha / visibility). Only the
-- branch-keyed tables can hold such a pair; the others key on (rfc_slug,user_id)
-- /(scope,pr_number) and have no stale data here, so they need no dedup.
DELETE FROM cached_branches WHERE project_id NOT IN (SELECT id FROM projects)
AND EXISTS (SELECT 1 FROM cached_branches o WHERE o.rfc_slug = cached_branches.rfc_slug AND o.branch_name = cached_branches.branch_name AND o.project_id IN (SELECT id FROM projects));
DELETE FROM branch_visibility WHERE project_id NOT IN (SELECT id FROM projects)
AND EXISTS (SELECT 1 FROM branch_visibility o WHERE o.rfc_slug = branch_visibility.rfc_slug AND o.branch_name = branch_visibility.branch_name AND o.project_id IN (SELECT id FROM projects));
UPDATE rfc_invitations SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = rfc_invitations.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM rfc_invitations WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE cached_branches SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = cached_branches.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM cached_branches WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE branch_visibility SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = branch_visibility.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM branch_visibility WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE branch_contribute_grants SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = branch_contribute_grants.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM branch_contribute_grants WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE stars SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = stars.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM stars WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE watches SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = watches.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM watches WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE pr_seen SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = pr_seen.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM pr_seen WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE branch_chat_seen SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = branch_chat_seen.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM branch_chat_seen WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE funder_consents SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = funder_consents.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM funder_consents WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE rfc_collaborators SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = rfc_collaborators.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM rfc_collaborators WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE contribution_requests SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = contribution_requests.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM contribution_requests WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
UPDATE proposed_use_cases SET project_id = (SELECT r.project_id FROM cached_rfcs r WHERE r.slug = proposed_use_cases.rfc_slug) WHERE rfc_slug IN (SELECT slug FROM cached_rfcs);
DELETE FROM proposed_use_cases WHERE rfc_slug NOT IN (SELECT slug FROM cached_rfcs);
-- ── collections: the new typed-corpus grain beneath projects ───────────────
CREATE TABLE collections (
id TEXT NOT NULL,
@@ -0,0 +1,8 @@
-- §22.4a SLICE-1 — configurable collection metadata: malformed-sidecar flag.
--
-- The corpus mirror reads an entry's metadata from its `<slug>.meta.yaml`
-- sidecar (dual-read: sidecar-else-legacy-frontmatter). A sidecar that does not
-- parse as a YAML mapping never hard-fails the read (INV-3) — the entry still
-- loads (from the legacy `.md` frontmatter if present) and this derived flag
-- marks it so the catalog can surface it. Additive only — no rebuild. 0 = ok.
ALTER TABLE cached_rfcs ADD COLUMN metadata_malformed INTEGER NOT NULL DEFAULT 0;
@@ -0,0 +1,12 @@
-- §22.4a SLICE-3 — configurable collection metadata: cache per-entry values.
--
-- Faceted filtering (§5.1) needs each entry's metadata values (priority, custom
-- enum/tags fields) to compute facet counts and honour filter params. Today the
-- mirror keeps only `tags_json` + the lifecycle columns and drops `Entry.extra`,
-- so a declared field's values are unrecoverable. This column persists the full
-- per-entry metadata mapping (`metadata.metadata_dict(entry)`, known keys +
-- extra, never the body) as JSON, so `app/facets.py` can read any declared
-- field uniformly. Additive + nullable — no rebuild. NULL = not yet re-ingested
-- (the reconciler/webhook fills it on the next sweep) → that entry contributes
-- no facet values until then. SLICE-4/5 edit panels read the same column.
ALTER TABLE cached_rfcs ADD COLUMN meta_json TEXT;
+62
View File
@@ -0,0 +1,62 @@
"""SLICE-4 — bot multi-file commit + PR primitives."""
from __future__ import annotations
import asyncio
from app import gitea as gitea_mod, metadata
from app.bot import Actor, Bot
from app.config import load_config
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea,
provision_user_row,
tmp_env,
)
LEGACY = "---\nslug: alpha\ntitle: Alpha\nstate: active\ntags:\n- one\n---\n\nBody.\n"
def _actor():
return Actor(user_id=1, gitea_login="ben.stull", display_name="Ben", email="ben@x.io")
def test_commit_entry_files_direct_to_main(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
gitea = gitea_mod.Gitea(load_config())
bot = Bot(gitea)
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": LEGACY, "sha": "s1"}
st = asyncio.run(metadata.read_entry_from_git(gitea, "wiggleverse", "meta", "rfcs/alpha.md"))
e2 = metadata.apply_values(st.entry, {"tags": ["two"]})
ops = metadata.write_entry_files("rfcs/alpha.md", e2, st)
asyncio.run(bot.commit_entry_files(
_actor(), org="wiggleverse", repo="meta", files=ops,
message="Edit metadata", branch="main"))
sc = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"]
assert "two" in sc
# .md is now body-only
assert "---" not in fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
def test_open_entry_pr_commits_on_branch_and_opens_pr(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
gitea = gitea_mod.Gitea(load_config())
bot = Bot(gitea)
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": LEGACY, "sha": "s1"}
with TestClient(app): # lifespan inits the DB
provision_user_row(user_id=1, login="ben.stull", role="owner")
st = asyncio.run(metadata.read_entry_from_git(gitea, "wiggleverse", "meta", "rfcs/alpha.md"))
e2 = metadata.apply_values(st.entry, {"tags": ["two"]})
ops = metadata.write_entry_files("rfcs/alpha.md", e2, st)
pr = asyncio.run(bot.open_entry_pr(
_actor(), org="wiggleverse", repo="meta", slug="alpha", files=ops,
pr_title="Metadata: Alpha", pr_description="edit", branch_prefix="metadata"))
assert pr["number"] >= 1
head = pr["head"]["ref"]
assert head.startswith("metadata-alpha-")
# committed on the branch, main untouched
assert ("wiggleverse", "meta", head, "rfcs/alpha.meta.yaml") in fake.files
assert ("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml") not in fake.files
+21
View File
@@ -2,6 +2,7 @@
subfolder_of."""
from __future__ import annotations
import json
import tempfile
from pathlib import Path
@@ -62,3 +63,23 @@ def test_get_collection_and_subfolder():
assert collections_mod.subfolder_of("features") == "features"
assert collections_mod.subfolder_of("default") == ""
assert collections_mod.get_collection("nope") is None
# ---- §22.4a SLICE-2: field schema served on the collection ----
def test_get_collection_fields_none_when_unset():
# INV-5: a collection with no `fields:` exposes fields=None (the default
# `document` collection sees zero change).
_db()
_seed()
assert collections_mod.get_collection("features")["fields"] is None
def test_get_collection_exposes_field_schema():
_db()
_seed()
schema = {"priority": {"type": "enum", "values": ["P0", "P1"]}}
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'features'",
(json.dumps({"fields": schema}),))
assert collections_mod.get_collection("features")["fields"] == schema
+30
View File
@@ -61,6 +61,36 @@ def test_parse_collection_manifest_rejects_bad_visibility():
registry.parse_collection_manifest("type: bdd\nvisibility: nope\n")
# ---- §22.4a SLICE-2: a `fields:` block flows into the collection config ----
def test_parse_collection_manifest_reads_field_schema():
doc = registry.parse_collection_manifest(
"type: bdd\n"
"fields:\n"
" priority:\n"
" type: enum\n"
" values: [P0, P1, P2]\n"
" tags:\n"
" type: tags\n"
)
assert doc.config["fields"] == {
"priority": {"type": "enum", "values": ["P0", "P1", "P2"]},
"tags": {"type": "tags"},
}
def test_parse_collection_manifest_lenient_on_bad_field():
# A bad field def is skipped (INV-3), the manifest still parses, and a
# manifest with no surviving fields carries no `fields` config key at all.
doc = registry.parse_collection_manifest(
"type: bdd\n"
"fields:\n"
" broken:\n"
" type: ref\n"
)
assert "fields" not in doc.config
# --- mirror discovery ---------------------------------------------------------
@@ -0,0 +1,165 @@
"""End-to-end integration tests for the deployed-environment E2E
test-auth path (`POST /auth/test/login`).
This endpoint is a **deliberately gated auth shortcut** for running the
Playwright E2E suite against a *deployed* environment (PPE) that has no
Mailpit OTC sink and no direct SQLite access to inject an owner row
the two scaffolds the Tier-1 stack relies on. It mints an authenticated
**owner** session for a single, pre-configured test identity, but ONLY
when the deployment has explicitly opted in by setting BOTH
`E2E_TEST_AUTH_SECRET` and `E2E_TEST_AUTH_EMAIL`. It is fail-closed:
* Off by default with neither (or only one) env var set, the route
is invisible (404), so a production deployment that never sets them
cannot be coaxed into minting a session.
* Even when enabled, it requires the caller to present the shared
secret in the `X-Test-Auth-Secret` header (constant-time compare),
and it will only mint a session for the one configured email any
other address is refused (403). So the blast radius of an enabled
PPE is a single throwaway owner identity, and the secret is the
trust boundary.
The tests below pin every branch of that gate.
"""
from __future__ import annotations
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
tmp_env,
)
SECRET = "ppe-e2e-shared-secret-value"
EMAIL = "e2e-owner@example.test"
def test_test_login_is_404_when_disabled(app_with_fake_gitea):
"""Neither env var set (the default, incl. production) → the route
does not exist."""
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/auth/test/login",
json={"email": EMAIL},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 404, r.text
def test_test_login_is_404_when_only_email_is_set(app_with_fake_gitea, monkeypatch):
"""Half-configured (email but no secret) must NOT open the route —
a framework auth shortcut gated only by a known email would be far
too weak."""
from fastapi.testclient import TestClient
monkeypatch.setenv("E2E_TEST_AUTH_EMAIL", EMAIL)
monkeypatch.delenv("E2E_TEST_AUTH_SECRET", raising=False)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/auth/test/login",
json={"email": EMAIL},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 404, r.text
def test_test_login_refuses_wrong_secret(app_with_fake_gitea, monkeypatch):
"""Enabled, but a bad/absent secret → 404 (don't advertise the
route's existence to an unauthenticated caller)."""
from fastapi.testclient import TestClient
monkeypatch.setenv("E2E_TEST_AUTH_SECRET", SECRET)
monkeypatch.setenv("E2E_TEST_AUTH_EMAIL", EMAIL)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
# Wrong secret.
r = client.post(
"/auth/test/login",
json={"email": EMAIL},
headers={"X-Test-Auth-Secret": "not-the-secret"},
)
assert r.status_code == 404, r.text
# Absent secret.
r = client.post("/auth/test/login", json={"email": EMAIL})
assert r.status_code == 404, r.text
def test_test_login_refuses_unconfigured_email(app_with_fake_gitea, monkeypatch):
"""Right secret but an email other than the single configured
identity 403. Even a secret-bearer can only mint the one test
owner."""
from fastapi.testclient import TestClient
monkeypatch.setenv("E2E_TEST_AUTH_SECRET", SECRET)
monkeypatch.setenv("E2E_TEST_AUTH_EMAIL", EMAIL)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/auth/test/login",
json={"email": "someone-else@example.test"},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 403, r.text
def test_test_login_mints_owner_session(app_with_fake_gitea, monkeypatch):
"""The happy path: right secret + configured email → an authenticated
session whose user is a GRANTED OWNER (so the metadata write paths
SLICE-4 edit, SLICE-5 bulk accept it), persisted on a fresh
`users` row."""
from fastapi.testclient import TestClient
monkeypatch.setenv("E2E_TEST_AUTH_SECRET", SECRET)
monkeypatch.setenv("E2E_TEST_AUTH_EMAIL", EMAIL)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/auth/test/login",
json={"email": EMAIL},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 200, r.text
# The session cookie now surfaces an authenticated owner.
me = client.get("/api/auth/me").json()
assert me["authenticated"] is True
assert me["user"]["email"] == EMAIL
assert me["user"]["role"] == "owner"
assert me["user"]["permission_state"] == "granted"
# Idempotent: a second login reuses the same row (still owner).
r = client.post(
"/auth/test/login",
json={"email": EMAIL},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 200, r.text
from app import db
rows = db.conn().execute(
"SELECT role, permission_state FROM users WHERE email = ? COLLATE NOCASE",
(EMAIL,),
).fetchall()
assert len(rows) == 1
assert rows[0]["role"] == "owner"
assert rows[0]["permission_state"] == "granted"
def test_test_login_is_case_insensitive_on_email(app_with_fake_gitea, monkeypatch):
"""The configured-email check matches case-insensitively, mirroring
how the rest of the auth stack treats email."""
from fastapi.testclient import TestClient
monkeypatch.setenv("E2E_TEST_AUTH_SECRET", SECRET)
monkeypatch.setenv("E2E_TEST_AUTH_EMAIL", EMAIL)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/auth/test/login",
json={"email": EMAIL.upper()},
headers={"X-Test-Auth-Secret": SECRET},
)
assert r.status_code == 200, r.text
+87
View File
@@ -0,0 +1,87 @@
"""§22.4a SLICE-3 — pure facet field-set + filter/count (PUC-3).
Per docs/design/2026-06-06-configurable-collection-metadata.md §5.1, §6.4.
"""
from app import facets
PRIORITY = {"priority": {"type": "enum", "values": ["P0", "P1", "P2"]}}
SCHEMA = {
"priority": {"type": "enum", "values": ["P0", "P1", "P2"]},
"tags": {"type": "tags"},
"owner": {"type": "text"},
}
def _e(slug, state="active", malformed=False, **meta):
return {"slug": slug, "state": state, "metadata_malformed": malformed, "meta": meta}
def test_facet_fields_orders_declared_then_state_skips_text():
# enum + tags in declaration order, text skipped, state appended last.
assert facets.facet_fields(SCHEMA) == [
("priority", "enum"), ("tags", "tags"), ("state", "enum")]
def test_no_schema_yields_no_facets():
# INV-5: a collection with no fields has no facets (frontend keeps chips).
assert facets.facet_fields(None) == []
assert facets.facet_fields({}) == []
def test_filter_and_count_basic_counts():
entries = [
_e("a", priority="P0", tags=["checkout"]),
_e("b", priority="P0", tags=["cart"]),
_e("c", priority="P1", tags=["checkout", "cart"]),
]
items, fac = facets.filter_and_count(entries, SCHEMA, {})
assert {i["slug"] for i in items} == {"a", "b", "c"}
assert fac["priority"] == {"P0": 2, "P1": 1}
assert fac["tags"] == {"checkout": 2, "cart": 2}
assert fac["state"] == {"active": 3}
def test_filter_compose_or_within_and_across():
entries = [
_e("a", priority="P0", tags=["checkout"]), # P0 + checkout
_e("b", priority="P0", tags=["cart"]), # P0, no checkout
_e("c", priority="P1", tags=["checkout"]), # checkout, not P0
]
# priority=P0 AND tags=checkout → only "a".
items, _ = facets.filter_and_count(
entries, SCHEMA, {"priority": {"P0"}, "tags": {"checkout"}})
assert {i["slug"] for i in items} == {"a"}
def test_drilldown_counts_exclude_own_field_selection():
entries = [
_e("a", priority="P0"),
_e("b", priority="P1"),
_e("c", priority="P1"),
]
# With P0 selected, the priority facet still counts P1 over the set that
# ignores priority's own selection — so P1 stays switchable.
_, fac = facets.filter_and_count(entries, SCHEMA, {"priority": {"P0"}})
assert fac["priority"] == {"P0": 1, "P1": 2}
def test_malformed_toggle_narrows_items_and_counts():
entries = [
_e("a", state="active", malformed=True, priority="P9"),
_e("b", state="active", malformed=False, priority="P0"),
]
items, fac = facets.filter_and_count(entries, SCHEMA, {}, only_malformed=True)
assert {i["slug"] for i in items} == {"a"}
assert fac["priority"] == {"P9": 1}
def test_missing_value_contributes_no_facet_value():
entries = [_e("a", priority="P0"), _e("b")] # b has no priority
_, fac = facets.filter_and_count(entries, SCHEMA, {})
assert fac["priority"] == {"P0": 1}
def test_allowed_filter_keys():
assert facets.allowed_filter_keys(PRIORITY) == {"priority", "state",
"unreviewed", "malformed"}
+134
View File
@@ -0,0 +1,134 @@
"""§22.4a SLICE-3 integration — faceted list endpoint (PUC-3, §6.4).
Through the real API: schema-declared facets, filter params (OR within / AND
across), drill-down counts, malformed toggle, unknown-field 400, and meta_json
persistence. Reuses the fake-Gitea harness from test_metadata_cache.
"""
from __future__ import annotations
import asyncio
import json
import yaml
from fastapi.testclient import TestClient
from app import cache, db, gitea as gitea_mod
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
tmp_env,
)
# The fake-Gitea harness seeds a single project whose id is the literal
# 'default' (see test_s1_collection_grain_vertical), served at
# /api/projects/default/rfcs.
PID = "default"
def _refresh():
cfg = load_config()
asyncio.run(cache.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
def _set_default_fields_schema(schema):
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'default'",
(json.dumps({"fields": schema}),))
def _seed(fake, slug, *, state="active", **front):
fm = {"slug": slug, "title": slug.title(), "state": state, **front}
body = yaml.safe_dump(fm, sort_keys=False).strip()
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {
"content": f"---\n{body}\n---\n\nBody.\n", "sha": slug}
def test_facets_and_counts_returned(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_default_fields_schema({
"priority": {"type": "enum", "values": ["P0", "P1", "P2"]},
"tags": {"type": "tags"},
})
_seed(fake, "a", priority="P0", tags=["checkout"])
_seed(fake, "b", priority="P0", tags=["cart"])
_seed(fake, "c", priority="P1", tags=["checkout", "cart"])
_refresh()
res = client.get(f"/api/projects/{PID}/rfcs")
assert res.status_code == 200
body = res.json()
assert {i["slug"] for i in body["items"]} == {"a", "b", "c"}
assert body["facets"]["priority"] == {"P0": 2, "P1": 1}
assert body["facets"]["tags"] == {"checkout": 2, "cart": 2}
assert body["facets"]["state"] == {"active": 3}
def test_filter_params_compose(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_default_fields_schema({
"priority": {"type": "enum", "values": ["P0", "P1"]},
"tags": {"type": "tags"},
})
_seed(fake, "a", priority="P0", tags=["checkout"])
_seed(fake, "b", priority="P0", tags=["cart"])
_seed(fake, "c", priority="P1", tags=["checkout"])
_refresh()
res = client.get(
f"/api/projects/{PID}/rfcs",
params={"priority": "P0", "tags": "checkout"})
assert {i["slug"] for i in res.json()["items"]} == {"a"}
def test_unknown_filter_field_400(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_default_fields_schema(
{"priority": {"type": "enum", "values": ["P0"]}})
_seed(fake, "a", priority="P0")
_refresh()
res = client.get(f"/api/projects/{PID}/rfcs",
params={"nonsense": "x"})
assert res.status_code == 400
def test_malformed_toggle(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_default_fields_schema(
{"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed(fake, "good", priority="P0")
_seed(fake, "bad", priority="P9") # not in values → malformed (INV-3)
_refresh()
res = client.get(f"/api/projects/{PID}/rfcs",
params={"malformed": "true"})
slugs = {i["slug"] for i in res.json()["items"]}
assert slugs == {"bad"}
def test_no_schema_no_facets(app_with_fake_gitea):
# INV-5: the default document collection (no fields) returns empty facets.
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed(fake, "plain", tags=["whatever"])
_refresh()
body = client.get(f"/api/projects/{PID}/rfcs").json()
assert body["facets"] == {}
def test_meta_json_persisted_at_ingest(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app):
_seed(fake, "withmeta", priority="P0", tags=["x"])
_refresh()
row = db.conn().execute(
"SELECT meta_json FROM cached_rfcs WHERE slug = 'withmeta'"
).fetchone()
meta = json.loads(row["meta_json"])
assert meta["priority"] == "P0"
assert meta["tags"] == ["x"]
@@ -0,0 +1,192 @@
"""G-15 — the branch/body subsystem is three-tier (project/collection) aware.
Before G-15 the branch-body GET resolved every meta-resident entry to the
default project's content repo at `rfcs/<slug>.md`, so an entry in a named
collection (subfolder) or a non-default project's repo rendered a BLANK
canonical body and its edit/PR/body-write paths hit the wrong file. These tests
seed entries outside the default collection and assert the collection-scoped
body-read routes (and the now-collection-aware slug-only routes) read the
correct repo + subfolder.
"""
from __future__ import annotations
import asyncio
from app import cache as cache_mod, db, gitea as gitea_mod
from app.config import load_config
from fastapi.testclient import TestClient # noqa: E402
from test_propose_vertical import ( # noqa: E402,F401
app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as,
)
def _entry_md(slug, title, state="active"):
return f"---\nslug: {slug}\ntitle: {title}\nstate: {state}\n---\nthe canonical body\n"
def _add_features_collection():
"""A named 'features' collection (subfolder 'features') under the seeded
default project same content repo ('meta'), distinct subfolder."""
db.conn().execute(
"INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, "
"initial_state, visibility, name, created_at, updated_at) VALUES "
"('features','default','document','features','super-draft','public','Features', "
"datetime('now'), datetime('now'))")
def _add_distinct_project():
"""A second project with its OWN content repo + a default (root) collection
the live OHM dogfood shape (rfc-app project at rfc-app-content)."""
db.conn().execute(
"INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) "
"VALUES ('rfc-app','RFC App','rfc-app-content','public', datetime('now'))")
db.conn().execute(
"INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, "
"initial_state, visibility, name, created_at, updated_at) VALUES "
"('rfc-app','rfc-app','document','','super-draft','public','RFC App', "
"datetime('now'), datetime('now'))")
def _mirror():
cfg = load_config()
asyncio.run(cache_mod.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
# --- named collection (subfolder) under the default project -------------------
def test_branch_view_named_collection_reads_subfolder(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_add_features_collection()
fake.files[("wiggleverse", "meta", "main", "features/rfcs/feat.md")] = {
"content": _entry_md("feat", "Feature Entry"), "sha": "sf"}
_mirror()
# cached_rfcs is keyed by the named collection.
assert db.conn().execute(
"SELECT collection_id FROM cached_rfcs WHERE slug='feat'"
).fetchone()["collection_id"] == "features"
# Collection-scoped branch view renders the body (was blank pre-G-15).
r = client.get(
"/api/projects/default/collections/features/rfcs/feat/branches/main")
assert r.status_code == 200, r.text
assert r.json()["body"] == "the canonical body\n"
# The slug-only legacy route is now collection-aware too: it resolves
# the entry's own subfolder via the cached row, so it also renders.
r2 = client.get("/api/rfcs/feat/branches/main")
assert r2.status_code == 200, r2.text
assert r2.json()["body"] == "the canonical body\n"
def test_main_view_named_collection_scoped_route(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_add_features_collection()
fake.files[("wiggleverse", "meta", "main", "features/rfcs/feat.md")] = {
"content": _entry_md("feat", "Feature Entry"), "sha": "sf"}
_mirror()
r = client.get("/api/projects/default/collections/features/rfcs/feat/main")
assert r.status_code == 200, r.text
assert r.json()["title"] == "Feature Entry"
# --- non-default project with a DISTINCT content repo (the OHM dogfood) -------
def test_branch_view_distinct_project_repo(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_add_distinct_project()
fake.files[("wiggleverse", "rfc-app-content", "main", "rfcs/scoped.md")] = {
"content": _entry_md("scoped", "Scoped Admin IA"), "sha": "s1"}
_mirror()
assert db.conn().execute(
"SELECT collection_id FROM cached_rfcs WHERE slug='scoped'"
).fetchone()["collection_id"] == "rfc-app"
# Scoped read resolves the entry's OWN content repo (rfc-app-content).
r = client.get(
"/api/projects/rfc-app/collections/rfc-app/rfcs/scoped/branches/main")
assert r.status_code == 200, r.text
assert r.json()["body"] == "the canonical body\n"
# And the slug-only route resolves the right repo via the cached row.
r2 = client.get("/api/rfcs/scoped/branches/main")
assert r2.status_code == 200, r2.text
assert r2.json()["body"] == "the canonical body\n"
# --- guards + regression ------------------------------------------------------
def test_scoped_branch_route_404_for_collection_outside_project(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
r = client.get(
"/api/projects/default/collections/nope/rfcs/x/branches/main")
assert r.status_code == 404
def _seed_super_draft_in_collection(fake, *, slug, collection_id, subfolder, owners):
import json as _json
import yaml
md_path = f"{subfolder}/rfcs/{slug}.md" if subfolder else f"rfcs/{slug}.md"
fm = {"slug": slug, "title": slug.title(), "state": "super-draft", "id": None,
"repo": None, "proposed_by": owners[0], "proposed_at": "2026-05-23",
"graduated_at": None, "graduated_by": None,
"owners": owners, "arbiters": owners[:1], "tags": []}
body = "the body\n"
text = f"---\n{yaml.safe_dump(fm, sort_keys=False).rstrip()}\n---\n\n{body}"
sha = fake._next_sha()
fake.files[("wiggleverse", "meta", "main", md_path)] = {"content": text, "sha": sha}
db.conn().execute(
"INSERT OR REPLACE INTO cached_rfcs (slug, title, state, rfc_id, repo, "
"proposed_by, proposed_at, owners_json, arbiters_json, tags_json, body, "
"body_sha, collection_id, last_main_commit_at, last_entry_commit_at) "
"VALUES (?,?, 'super-draft', NULL, NULL, ?, '2026-05-23', ?, ?, '[]', ?, ?, ?, "
"datetime('now'), datetime('now'))",
(slug, slug.title(), owners[0], _json.dumps(owners), _json.dumps(owners[:1]),
body, sha, collection_id))
def test_graduate_in_named_collection_writes_to_subfolder(app_with_fake_gitea):
"""G-15 write path: graduating a super-draft that lives in a named
collection flips the entry in that collection's `<subfolder>/rfcs/<slug>.md`,
not the default `rfcs/<slug>.md`."""
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_add_features_collection()
provision_user_row(user_id=1, login="ben", role="owner")
_seed_super_draft_in_collection(
fake, slug="gradme", collection_id="features", subfolder="features",
owners=["ben"])
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben",
role="owner", email="ben@x")
r = client.post("/api/rfcs/gradme/graduate?_sync=1",
json={"rfc_id": "RFC-0007", "owners": ["ben"]})
assert r.status_code == 200, r.text
# The flip landed in the collection's subfolder, not the repo root.
sc = fake.files.get(
("wiggleverse", "meta", "main", "features/rfcs/gradme.meta.yaml"))
assert sc is not None, "sidecar not written under the collection subfolder"
import yaml as _yaml
assert _yaml.safe_load(sc["content"])["state"] == "active"
# Nothing was written to the default repo-root path.
assert ("wiggleverse", "meta", "main", "rfcs/gradme.meta.yaml") not in fake.files
def test_default_collection_entry_still_renders(app_with_fake_gitea):
"""Regression: the default-collection path (repo root `rfcs/<slug>.md` in
the default content repo) is unchanged by the G-15 resolution."""
app, fake = app_with_fake_gitea
with TestClient(app) as client:
fake.files[("wiggleverse", "meta", "main", "rfcs/base.md")] = {
"content": _entry_md("base", "Baseline"), "sha": "sb"}
_mirror()
r = client.get("/api/rfcs/base/branches/main")
assert r.status_code == 200, r.text
assert r.json()["body"] == "the canonical body\n"
r2 = client.get("/api/projects/default/collections/default/rfcs/base/branches/main")
assert r2.status_code == 200, r2.text
assert r2.json()["body"] == "the canonical body\n"
+108
View File
@@ -0,0 +1,108 @@
"""G-15 — the §22 three-tier write-path resolver.
`projects.content_repo_for_collection` and `projects.entry_location` resolve an
entry's git location (org, content_repo, md_path) from its *collection*
(collection project content_repo, plus the collection subfolder) instead of
the deployment default. This is the keystone the branch/edit/body/graduation
write paths share so an entry outside the default project's default collection
reads/writes the correct file.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from app import collections as collections_mod, db, projects as projects_mod
from app.config import Config
def _db() -> Config:
cfg = Config(
gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="wiggleverse",
registry_repo="registry", oauth_client_id="x",
oauth_client_secret="x", app_url="x", secret_key="x",
database_path=Path(tempfile.mkdtemp(prefix="g15loc-")) / "t.db",
owner_gitea_login="x", webhook_secret="x",
)
db.run_migrations(cfg)
if db._CONN is not None:
db._CONN.close()
db._CONN = None
db.init(cfg)
return cfg
def _seed():
# Default project (its content_repo is the deployment default) + a second
# project with a DISTINCT content_repo, each with a default + a named
# (subfolder) collection.
db.conn().execute(
"INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) "
"VALUES ('default','Default','meta','public', datetime('now'))")
db.conn().execute(
"INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) "
"VALUES ('rfc-app','RFC App','rfc-app-content','public', datetime('now'))")
rows = [
("default", "default", ""),
("features", "default", "features"),
("rfc-app", "rfc-app", ""),
("specs", "rfc-app", "specs"),
]
for cid, pid, sub in rows:
db.conn().execute(
"INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, "
"initial_state, visibility, created_at, updated_at) VALUES "
"(?,?, 'document', ?, 'super-draft','public', datetime('now'), datetime('now'))",
(cid, pid, sub))
def test_content_repo_for_collection_resolves_per_project():
_db()
_seed()
# Default project's collections → the default content repo.
assert projects_mod.content_repo_for_collection("default") == "meta"
assert projects_mod.content_repo_for_collection("features") == "meta"
# The second project's collections → its own content repo.
assert projects_mod.content_repo_for_collection("rfc-app") == "rfc-app-content"
assert projects_mod.content_repo_for_collection("specs") == "rfc-app-content"
def test_content_repo_for_collection_unknown_is_none():
_db()
_seed()
assert projects_mod.content_repo_for_collection("nope") is None
def test_entry_location_default_collection_repo_root():
cfg = _db()
_seed()
org, repo, path = projects_mod.entry_location(cfg, "default", "alpha")
assert (org, repo, path) == ("wiggleverse", "meta", "rfcs/alpha.md")
def test_entry_location_named_collection_uses_subfolder():
cfg = _db()
_seed()
org, repo, path = projects_mod.entry_location(cfg, "features", "beta")
assert (org, repo, path) == ("wiggleverse", "meta", "features/rfcs/beta.md")
def test_entry_location_other_project_distinct_repo():
cfg = _db()
_seed()
# Named collection in a non-default project: distinct repo AND subfolder.
org, repo, path = projects_mod.entry_location(cfg, "specs", "gamma")
assert (org, repo, path) == ("wiggleverse", "rfc-app-content", "specs/rfcs/gamma.md")
# Default collection of the non-default project: distinct repo, repo root.
org, repo, path = projects_mod.entry_location(cfg, "rfc-app", "delta")
assert (org, repo, path) == ("wiggleverse", "rfc-app-content", "rfcs/delta.md")
def test_entry_location_unknown_collection_falls_back_to_default_repo():
cfg = _db()
_seed()
# An entry whose collection_id is missing/unknown must still resolve to a
# usable location (the deployment default repo, repo root) rather than an
# empty repo — the legacy single-corpus behaviour.
org, repo, path = projects_mod.entry_location(cfg, "nope", "epsilon")
assert (org, repo, path) == ("wiggleverse", "meta", "rfcs/epsilon.md")
+45 -10
View File
@@ -48,6 +48,18 @@ PITCH = (
)
def _entry_from_git(fake, slug, branch="main"):
"""§22.4a SLICE-4: read an entry's combined metadata+body from git via the
dual-read parser graduation/claim now write metadata to the sidecar and
keep the body in the `.md`, so an Entry is reconstructed from both."""
from app import metadata
md = fake.files[("wiggleverse", "meta", branch, f"rfcs/{slug}.md")]["content"]
sc = fake.files.get(
("wiggleverse", "meta", branch, f"rfcs/{slug}.meta.yaml"), {}).get("content")
e, _ = metadata.read_entry(md, sc, fallback_slug=slug)
return e
def seed_owned_super_draft(fake: FakeGitea, *, slug: str, title: str, pitch: str,
owners: list[str], arbiters: list[str] | None = None,
proposed_by: str = "alice", tags: list[str] | None = None) -> None:
@@ -190,9 +202,8 @@ def test_graduate_happy_path_flips_in_place_keeping_body(app_with_fake_gitea):
k[1].startswith("rfc-0042") for k in fake.repos
), f"a per-RFC repo was created: {fake.repos}"
# Meta entry on main: state flipped, body KEPT, repo null.
meta_text = fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
graduated = entry_mod.parse(meta_text)
# Meta entry on main: state flipped (sidecar), body KEPT (.md), repo null.
graduated = _entry_from_git(fake, "ohm")
assert graduated.state == "active"
assert graduated.id == "RFC-0042"
assert graduated.repo is None
@@ -224,6 +235,34 @@ def test_graduate_happy_path_flips_in_place_keeping_body(app_with_fake_gitea):
assert gone not in kinds, f"retired audit row present: {gone}"
def test_graduate_preserves_unknown_frontmatter_keys(app_with_fake_gitea):
"""§22.4a INV-7: a forward-compat / unknown frontmatter key on the
super-draft entry must ride through the graduation rebuild, not be dropped."""
from fastapi.testclient import TestClient
from app import entry as entry_mod
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=1, login="ben", role="owner")
seed_owned_super_draft(fake, slug="ohm", title="OHM", pitch=PITCH,
owners=["ben"], arbiters=["ben"])
# Inject an unknown key into the seeded entry's frontmatter.
key = ("wiggleverse", "meta", "main", "rfcs/ohm.md")
e = entry_mod.parse(fake.files[key]["content"])
e.extra["priority"] = "P1"
fake.files[key]["content"] = entry_mod.serialize(e)
sign_in_as(client, user_id=1, gitea_login="ben",
display_name="Ben", role="owner", email="ben@test")
r = client.post("/api/rfcs/ohm/graduate?_sync=1",
json={"rfc_id": "RFC-0042", "owners": ["ben"]})
assert r.status_code == 200, r.text
graduated = _entry_from_git(fake, "ohm")
assert graduated.state == "active"
assert graduated.extra.get("priority") == "P1"
def test_graduate_coexists_with_open_body_edit_pr(app_with_fake_gitea):
"""§9.8 (meta-only): an open meta-repo body-edit PR no longer blocks
graduation the body is kept, so they coexist. /check stays
@@ -571,8 +610,7 @@ def test_graduate_without_number_flips_to_active_null_id_by_slug(app_with_fake_g
assert d["rfc_id"] is None
# Meta entry: active, id null, body kept, graduation stamped.
meta_text = fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
graduated = entry_mod.parse(meta_text)
graduated = _entry_from_git(fake, "ohm")
assert graduated.state == "active"
assert graduated.id is None
assert graduated.graduated_by == "ben"
@@ -621,9 +659,7 @@ def test_graduate_with_number_unchanged_when_id_absent_field(app_with_fake_gitea
r = client.post("/api/rfcs/ohm/graduate?_sync=1", json={"owners": ["ben"]})
assert r.status_code == 200, r.text
assert r.json()["rfc_id"] is None
graduated = entry_mod.parse(
fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
)
graduated = _entry_from_git(fake, "ohm")
assert graduated.state == "active"
assert graduated.id is None
@@ -649,8 +685,7 @@ def test_claim_opens_meta_pr(app_with_fake_gitea):
d = r.json()
assert d["branch_name"] == "claim/ohm"
text = fake.files[("wiggleverse", "meta", "claim/ohm", "rfcs/ohm.md")]["content"]
ent = entry_mod.parse(text)
ent = _entry_from_git(fake, "ohm", branch="claim/ohm")
assert "alice" in ent.owners
row = db.conn().execute(
+8 -5
View File
@@ -47,12 +47,15 @@ def test_mark_reviewed_clears_flag(app_with_fake_gitea):
assert row["unreviewed"] == 0
assert row["reviewed_by"] == "ben"
assert row["reviewed_at"] # provenance stamped
# git-side: the entry file on main was rewritten with the cleared flag.
from app import entry as entry_mod
# git-side (§22.4a SLICE-4): the cleared flag now lands in the metadata
# sidecar and the `.md` is lazy-migrated to a clean body-only file (INV-2).
import yaml
sidecar = fake.files[("wiggleverse", "meta", "main", "rfcs/feat.meta.yaml")]["content"]
sc = yaml.safe_load(sidecar)
assert not sc.get("unreviewed") # cleared (omitted when False)
assert sc.get("reviewed_by") == "ben"
written = fake.files[("wiggleverse", "meta", "main", "rfcs/feat.md")]["content"]
e = entry_mod.parse(written)
assert e.unreviewed is False
assert e.reviewed_by == "ben"
assert "---" not in written # body-only, no frontmatter
def test_mark_reviewed_forbidden_for_non_superuser(app_with_fake_gitea):
+223
View File
@@ -0,0 +1,223 @@
"""SLICE-1 unit tests — sidecar metadata: dual-read, unknown-key preservation,
frontmatter stripping, malformed detection.
Pure functions only (no DB / no Gitea). Per
docs/design/2026-06-06-configurable-collection-metadata.md §7.2 (SLICE-1) and
INV-6 (dual-read), INV-7 (unknown keys ride along), INV-2 (clean body).
"""
from __future__ import annotations
import yaml
from app import entry as entry_mod
from app import metadata
LEGACY_MD = """---
slug: view-metrics
title: View today's metrics
state: active
owners:
- ben.stull
tags:
- dashboard
- analytics
priority: P1
owner: hasan
---
This is the prose body.
Second paragraph.
"""
# ---- INV-7: unknown keys ride along ----
def test_parse_preserves_unknown_keys_in_extra():
e = entry_mod.parse(LEGACY_MD)
assert e.extra == {"priority": "P1", "owner": "hasan"}
def test_serialize_round_trip_preserves_unknown_keys():
e = entry_mod.parse(LEGACY_MD)
text = entry_mod.serialize(e)
e2 = entry_mod.parse(text)
assert e2.extra == {"priority": "P1", "owner": "hasan"}
assert e2.tags == ["dashboard", "analytics"]
assert e2.owners == ["ben.stull"]
def test_known_keys_never_leak_into_extra():
e = entry_mod.parse(LEGACY_MD)
for known in ("slug", "title", "state", "owners", "tags"):
assert known not in e.extra
# ---- metadata_dict / sidecar_yaml ----
def test_metadata_dict_merges_known_and_extra():
e = entry_mod.parse(LEGACY_MD)
d = metadata.metadata_dict(e)
assert d["slug"] == "view-metrics"
assert d["title"] == "View today's metrics"
assert d["state"] == "active"
assert d["tags"] == ["dashboard", "analytics"]
# forward-compat keys present
assert d["priority"] == "P1"
assert d["owner"] == "hasan"
def test_sidecar_yaml_is_parseable_and_has_no_frontmatter_fences():
e = entry_mod.parse(LEGACY_MD)
sc = metadata.sidecar_yaml(e)
assert "---" not in sc.splitlines()[0]
loaded = yaml.safe_load(sc)
assert loaded["slug"] == "view-metrics"
assert loaded["priority"] == "P1"
# ---- strip_frontmatter (INV-2) ----
def test_strip_frontmatter_removes_leading_block():
body = metadata.strip_frontmatter(LEGACY_MD)
assert body.startswith("This is the prose body.")
assert "slug:" not in body
assert "priority:" not in body
def test_strip_frontmatter_passthrough_when_no_frontmatter():
plain = "Just a body.\n\nNo frontmatter here.\n"
assert metadata.strip_frontmatter(plain).strip() == plain.strip()
# ---- parse_sidecar (malformed detection, INV-3) ----
def test_parse_sidecar_good():
values, malformed = metadata.parse_sidecar("slug: a\ntitle: A\npriority: P0\n")
assert malformed is False
assert values == {"slug": "a", "title": "A", "priority": "P0"}
def test_parse_sidecar_non_mapping_is_malformed():
values, malformed = metadata.parse_sidecar("- just\n- a\n- list\n")
assert malformed is True
assert values == {}
def test_parse_sidecar_invalid_yaml_is_malformed():
values, malformed = metadata.parse_sidecar("slug: : : not yaml\n bad: [unclosed\n")
assert malformed is True
assert values == {}
def test_parse_sidecar_empty_is_empty_not_malformed():
values, malformed = metadata.parse_sidecar("")
assert malformed is False
assert values == {}
# ---- read_entry dual-read equivalence (INV-6) ----
def test_dual_read_sidecar_matches_legacy():
legacy_entry, legacy_bad = metadata.read_entry(LEGACY_MD, None)
# The migrated form: body-only .md + a sidecar holding the metadata.
migrated_md = metadata.strip_frontmatter(LEGACY_MD)
sidecar_text = metadata.sidecar_yaml(legacy_entry)
sidecar_entry, sidecar_bad = metadata.read_entry(migrated_md, sidecar_text)
assert legacy_bad is False
assert sidecar_bad is False
# Identical resulting records (INV-6).
assert sidecar_entry.slug == legacy_entry.slug
assert sidecar_entry.title == legacy_entry.title
assert sidecar_entry.state == legacy_entry.state
assert sidecar_entry.owners == legacy_entry.owners
assert sidecar_entry.tags == legacy_entry.tags
assert sidecar_entry.extra == legacy_entry.extra
assert sidecar_entry.body.strip() == legacy_entry.body.strip()
def test_read_entry_sidecar_takes_precedence_over_md_frontmatter():
# A not-yet-migrated .md still carrying frontmatter, plus a sidecar that
# disagrees: the sidecar wins for metadata; the body comes from the .md.
md_with_fm = "---\nslug: old\ntitle: Old Title\nstate: super-draft\n---\n\nBody.\n"
sidecar = "slug: new\ntitle: New Title\nstate: active\n"
e, malformed = metadata.read_entry(md_with_fm, sidecar)
assert malformed is False
assert e.title == "New Title"
assert e.state == "active"
assert e.body.strip() == "Body."
def test_read_entry_malformed_sidecar_still_loads_entry():
# INV-3: a malformed sidecar never hard-fails the read. The entry loads
# (from the .md frontmatter if present) and is flagged malformed.
md = "---\nslug: x\ntitle: X\nstate: active\n---\n\nBody.\n"
e, malformed = metadata.read_entry(md, "- not a mapping\n")
assert malformed is True
assert e.slug == "x"
assert e.title == "X"
assert e.body.strip() == "Body."
# ---- dual-read robustness: degenerate sidecars never drop the entry ----
def test_empty_sidecar_falls_back_to_md_frontmatter():
# An empty sidecar has no metadata to override with — keep the .md's.
md = "---\nslug: keep\ntitle: Keep Me\nstate: active\n---\n\nBody.\n"
e, malformed = metadata.read_entry(md, "", fallback_slug="keep")
assert malformed is False
assert e.slug == "keep"
assert e.title == "Keep Me"
def test_malformed_sidecar_on_body_only_md_loads_with_fallback_slug():
# The .md is already body-only (migrated) and the sidecar is corrupt:
# the entry must still load (INV-3), taking its slug from the filename stem.
e, malformed = metadata.read_entry("Just a body.\n", "- a\n- list\n", fallback_slug="foo")
assert malformed is True
assert e.slug == "foo"
def test_slugless_sidecar_uses_fallback_slug():
md = "Body only.\n"
sidecar = "title: No Slug Here\nstate: active\n"
e, malformed = metadata.read_entry(md, sidecar, fallback_slug="bar")
assert malformed is False
assert e.slug == "bar"
assert e.title == "No Slug Here"
# ---- sidecar filename helpers ----
def test_sidecar_filename_helpers():
assert metadata.sidecar_name("view-metrics") == "view-metrics.meta.yaml"
assert metadata.is_sidecar("view-metrics.meta.yaml") is True
assert metadata.is_sidecar("view-metrics.md") is False
assert metadata.slug_of_sidecar("view-metrics.meta.yaml") == "view-metrics"
# ---- SLICE-4: sidecar_path_for + apply_values ----
def test_sidecar_path_for_derives_sibling():
assert metadata.sidecar_path_for("rfcs/alpha.md") == "rfcs/alpha.meta.yaml"
assert metadata.sidecar_path_for("x/y/beta.md") == "x/y/beta.meta.yaml"
def test_apply_values_updates_known_and_extra_fields():
e = entry_mod.parse(LEGACY_MD) # has tags + extra priority/owner
e2 = metadata.apply_values(e, {"tags": ["x"], "priority": "P0", "owner": "sam"})
assert e2.tags == ["x"]
assert e2.extra["priority"] == "P0"
assert e2.extra["owner"] == "sam"
# body preserved unchanged
assert e2.body == e.body
def test_apply_values_preserves_unspecified_keys():
e = entry_mod.parse(LEGACY_MD)
e2 = metadata.apply_values(e, {"priority": "P0"})
assert e2.tags == e.tags # untouched
assert e2.extra["owner"] == "hasan" # untouched
@@ -0,0 +1,232 @@
"""SLICE-5 — bulk metadata edit endpoint (PUC-2, §6.4/§6.5).
Through the real API: contributor+ gating (INV-4), set/add/remove ops,
validation at the write boundary, one commit for N sidecars (D7), and
partial-rejection reporting. Reuses the fake-Gitea harness.
"""
from __future__ import annotations
import asyncio
import json
import yaml
from fastapi.testclient import TestClient
from app import cache, db, gitea as gitea_mod
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
PID = "default"
CID = "default"
BASE = f"/api/projects/{PID}/collections/{CID}"
def _refresh():
cfg = load_config()
asyncio.run(cache.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
def _set_fields(schema):
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'default'",
(json.dumps({"fields": schema}),))
def _seed_legacy(fake, slug, *, state="active", **front):
fm = {"slug": slug, "title": slug.title(), "state": state, **front}
body = yaml.safe_dump(fm, sort_keys=False).strip()
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {
"content": f"---\n{body}\n---\n\nBody.\n", "sha": slug}
def _login_owner(client):
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
def _has_sidecar(fake, slug):
return ("wiggleverse", "meta", "main", f"rfcs/{slug}.meta.yaml") in fake.files
def _sidecar(fake, slug):
return yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.meta.yaml")]["content"])
# ---- Task 1: happy path, one commit ----
def test_bulk_set_applies_to_all_and_one_commit(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1", "P2"]}})
_seed_legacy(fake, "a", priority="P2")
_seed_legacy(fake, "b", priority="P1")
_refresh()
_login_owner(client)
commits_before = fake.change_files_calls
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "b"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 200, r.text
body = r.json()
assert set(body["applied"]) == {"a", "b"}
assert body["rejected"] == []
assert body["committed"] is True
# exactly one ChangeFiles commit covered both entries (D7)
assert fake.change_files_calls - commits_before == 1
assert _sidecar(fake, "a")["priority"] == "P0"
assert _sidecar(fake, "b")["priority"] == "P0"
# ---- Task 2: add/remove tags ----
def test_bulk_add_tag(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x"])
_seed_legacy(fake, "b", tags=["x", "y"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "b"], "op": "add",
"field": "tags", "value": "y"})
assert r.status_code == 200, r.text
assert set(r.json()["applied"]) == {"a", "b"}
# "a" gained y; "b" already had y (no-op write skipped → no sidecar written)
assert _sidecar(fake, "a")["tags"] == ["x", "y"]
assert not _has_sidecar(fake, "b")
def test_bulk_remove_tag(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x", "y"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "remove",
"field": "tags", "value": "x"})
assert r.status_code == 200, r.text
assert r.json()["applied"] == ["a"]
assert _sidecar(fake, "a")["tags"] == ["y"]
def test_bulk_set_scalar_on_tags_rejected(app_with_fake_gitea):
# A scalar `set` onto a tags field must reject, not char-split into a list.
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "tags", "value": "checkout"})
assert r.status_code == 200, r.text
assert r.json()["applied"] == []
assert len(r.json()["rejected"]) == 1
assert r.json()["committed"] is False
assert not _has_sidecar(fake, "a")
def test_bulk_set_list_on_tags_ok(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "tags", "value": ["x", "y"]})
assert r.status_code == 200, r.text
assert r.json()["applied"] == ["a"]
assert _sidecar(fake, "a")["tags"] == ["x", "y"]
def test_bulk_add_remove_requires_tags_field(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "add",
"field": "priority", "value": "z"})
assert r.status_code == 422, r.text
# ---- Task 3: partial rejection, authz, validation guards ----
def test_bulk_partial_reject_missing_entry(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "ghost"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 200, r.text
body = r.json()
assert body["applied"] == ["a"]
assert body["rejected"] == [{"slug": "ghost", "reason": "not found"}]
assert body["committed"] is True
assert _sidecar(fake, "a")["priority"] == "P0"
def test_bulk_invalid_value_rejects_all_no_commit(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "priority", "value": "ZZZ"})
assert r.status_code == 200, r.text
assert r.json()["applied"] == []
assert len(r.json()["rejected"]) == 1
assert r.json()["committed"] is False
assert not _has_sidecar(fake, "a")
def test_bulk_forbidden_for_anonymous(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 403, r.text
def test_bulk_unknown_field_op_and_empty(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
_login_owner(client)
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "nope", "value": "P0"}).status_code == 422
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "frobnicate",
"field": "priority", "value": "P0"}).status_code == 422
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": [], "op": "set",
"field": "priority", "value": "P0"}).status_code == 422
+177
View File
@@ -0,0 +1,177 @@
"""SLICE-1 integration — the corpus mirror reads sidecars (dual-read) and
derives the malformed flag (PUC-6, INV-3/INV-6).
Per docs/design/2026-06-06-configurable-collection-metadata.md §6.2-6.3.
"""
from __future__ import annotations
import asyncio
import json
from fastapi.testclient import TestClient
from app import cache, db, gitea as gitea_mod
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
tmp_env,
)
def _refresh():
cfg = load_config()
asyncio.run(cache.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
def _row(slug):
return db.conn().execute(
"SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)
).fetchone()
def test_mirror_reads_metadata_from_sidecar(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app):
# A migrated entry: body-only .md + a sidecar holding the metadata.
fake.files[("wiggleverse", "meta", "main", "rfcs/sidecar-one.md")] = {
"content": "Just the prose body.\n", "sha": "s1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/sidecar-one.meta.yaml")] = {
"content": "slug: sidecar-one\ntitle: From Sidecar\nstate: active\ntags:\n- alpha\n",
"sha": "m1"}
_refresh()
row = _row("sidecar-one")
assert row is not None
assert row["title"] == "From Sidecar"
assert row["state"] == "active"
assert row["body"].strip() == "Just the prose body."
assert row["metadata_malformed"] == 0
def test_malformed_sidecar_flags_but_still_loads(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app):
# .md still has frontmatter; sidecar is malformed (a list, not a map).
fake.files[("wiggleverse", "meta", "main", "rfcs/bad-meta.md")] = {
"content": "---\nslug: bad-meta\ntitle: Legacy Title\nstate: active\n---\n\nBody.\n",
"sha": "b1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/bad-meta.meta.yaml")] = {
"content": "- not\n- a\n- mapping\n", "sha": "b2"}
_refresh()
row = _row("bad-meta")
assert row is not None # INV-3: still loads
assert row["metadata_malformed"] == 1
# Falls back to the legacy .md frontmatter for the metadata.
assert row["title"] == "Legacy Title"
def test_malformed_flag_surfaces_in_catalog_api(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
fake.files[("wiggleverse", "meta", "main", "rfcs/flagged.md")] = {
"content": "---\nslug: flagged\ntitle: Flagged\nstate: active\n---\n\nB.\n",
"sha": "f1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/flagged.meta.yaml")] = {
"content": "just a scalar\n", "sha": "f2"}
fake.files[("wiggleverse", "meta", "main", "rfcs/clean.md")] = {
"content": "---\nslug: clean\ntitle: Clean\nstate: active\n---\n\nB.\n",
"sha": "c1"}
_refresh()
items = {i["slug"]: i for i in client.get("/api/rfcs").json()["items"]}
assert items["flagged"]["metadata_malformed"] is True
assert items["clean"]["metadata_malformed"] is False
# And on the detail view.
assert client.get("/api/rfcs/flagged").json()["metadata_malformed"] is True
def test_malformed_sidecar_on_migrated_entry_still_loads_flagged(app_with_fake_gitea):
# INV-3 regression: a migrated (body-only .md) entry whose sidecar is
# corrupt must NOT vanish from the catalog — it loads (slug from the
# filename stem) and is flagged malformed.
app, fake = app_with_fake_gitea
with TestClient(app):
fake.files[("wiggleverse", "meta", "main", "rfcs/orphaned.md")] = {
"content": "Just the body, no frontmatter.\n", "sha": "o1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/orphaned.meta.yaml")] = {
"content": "- corrupt\n- list\n", "sha": "o2"}
_refresh()
row = _row("orphaned")
assert row is not None # did not vanish
assert row["metadata_malformed"] == 1
assert row["body"].strip() == "Just the body, no frontmatter."
def _set_default_fields_schema(schema):
# apply_registry leaves the default collection's config_json untouched, so a
# schema set here survives a corpus refresh (refresh_meta_repo only).
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'default'",
(json.dumps({"fields": schema}),))
# ---- §22.4a SLICE-2: advisory schema validation at ingest (INV-3) ----
def test_schema_violation_flags_malformed(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app):
_set_default_fields_schema(
{"priority": {"type": "enum", "values": ["P0", "P1", "P2"]}})
fake.files[("wiggleverse", "meta", "main", "rfcs/bad-prio.md")] = {
"content": "---\nslug: bad-prio\ntitle: Bad\nstate: active\npriority: P9\n---\n\nB.\n",
"sha": "bp1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/good-prio.md")] = {
"content": "---\nslug: good-prio\ntitle: Good\nstate: active\npriority: P0\n---\n\nB.\n",
"sha": "gp1"}
_refresh()
assert _row("bad-prio")["metadata_malformed"] == 1 # INV-3: flagged
assert _row("bad-prio")["title"] == "Bad" # still loads
assert _row("good-prio")["metadata_malformed"] == 0
def test_schema_ignores_undeclared_keys(app_with_fake_gitea):
# INV-7: keys the schema doesn't declare ride along and never flag malformed.
app, fake = app_with_fake_gitea
with TestClient(app):
_set_default_fields_schema(
{"priority": {"type": "enum", "values": ["P0", "P1"]}})
fake.files[("wiggleverse", "meta", "main", "rfcs/extra-key.md")] = {
"content": "---\nslug: extra-key\ntitle: Extra\nstate: active\nowner: hasan\n---\n\nB.\n",
"sha": "ek1"}
_refresh()
assert _row("extra-key")["metadata_malformed"] == 0
def test_no_schema_never_flags(app_with_fake_gitea):
# INV-5: a collection with no field schema validates nothing, even when an
# entry carries values that would fail a schema if one existed.
app, fake = app_with_fake_gitea
with TestClient(app):
fake.files[("wiggleverse", "meta", "main", "rfcs/anything.md")] = {
"content": "---\nslug: anything\ntitle: Any\nstate: active\npriority: whatever\n---\n\nB.\n",
"sha": "an1"}
_refresh()
assert _row("anything")["metadata_malformed"] == 0
def test_legacy_collection_without_sidecars_unchanged(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app):
fake.files[("wiggleverse", "meta", "main", "rfcs/legacy.md")] = {
"content": "---\nslug: legacy\ntitle: Legacy\nstate: super-draft\n---\n\nPitch.\n",
"sha": "l1"}
_refresh()
row = _row("legacy")
assert row is not None
assert row["title"] == "Legacy"
assert row["state"] == "super-draft"
assert row["body"].strip() == "Pitch."
assert row["metadata_malformed"] == 0
@@ -0,0 +1,190 @@
"""SLICE-4 — single-entry metadata edit endpoint (PUC-1, §6.4).
Through the real API: contributor+ gating (INV-4), schema validation at the
write boundary, direct commit to the sidecar with lazy migration, re-ingest,
and the GET RFC `meta` + `can_edit_meta` exposure. Reuses the fake-Gitea harness.
"""
from __future__ import annotations
import asyncio
import json
import yaml
from fastapi.testclient import TestClient
from app import cache, db, gitea as gitea_mod
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
PID = "default"
CID = "default"
META = f"/api/projects/{PID}/collections/{CID}"
def _refresh():
cfg = load_config()
asyncio.run(cache.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
def _set_fields(schema):
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'default'",
(json.dumps({"fields": schema}),))
def _seed_legacy(fake, slug, *, state="active", **front):
fm = {"slug": slug, "title": slug.title(), "state": state, **front}
body = yaml.safe_dump(fm, sort_keys=False).strip()
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {
"content": f"---\n{body}\n---\n\nBody.\n", "sha": slug}
def _seed_migrated(fake, slug, sidecar):
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {
"content": "Body.\n", "sha": f"{slug}-md"}
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.meta.yaml")] = {
"content": sidecar, "sha": f"{slug}-sc"}
def _login_owner(client):
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
def test_edit_meta_sets_value_commits_sidecar_and_lazy_migrates(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1", "P2"]},
"tags": {"type": "tags"}})
_seed_legacy(fake, "a", priority="P1", tags=["x"])
_refresh()
_login_owner(client)
r = client.post(f"{META}/rfcs/a/meta", json={"values": {"priority": "P0"}})
assert r.status_code == 200, r.text
assert r.json()["meta"]["priority"] == "P0"
# sidecar written, .md lazy-migrated to body-only
sc = yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/a.meta.yaml")]["content"])
assert sc["priority"] == "P0"
assert "---" not in fake.files[("wiggleverse", "meta", "main", "rfcs/a.md")]["content"]
# cache reflects the new value
row = db.conn().execute(
"SELECT meta_json FROM cached_rfcs WHERE slug='a'").fetchone()
assert json.loads(row["meta_json"])["priority"] == "P0"
def test_edit_meta_rejects_value_outside_enum(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_refresh()
_login_owner(client)
r = client.post(f"{META}/rfcs/a/meta", json={"values": {"priority": "ZZZ"}})
assert r.status_code == 422, r.text
# nothing committed
assert ("wiggleverse", "meta", "main", "rfcs/a.meta.yaml") not in fake.files
def test_edit_meta_rejects_unknown_field(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
_login_owner(client)
r = client.post(f"{META}/rfcs/a/meta", json={"values": {"nope": "x"}})
assert r.status_code == 422, r.text
def test_edit_meta_forbidden_for_anonymous(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
r = client.post(f"{META}/rfcs/a/meta", json={"values": {"priority": "P0"}})
assert r.status_code == 403, r.text
def test_edit_meta_on_already_migrated_entry(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_migrated(fake, "a", "slug: a\ntitle: A\nstate: active\npriority: P1\n")
_refresh()
_login_owner(client)
r = client.post(f"{META}/rfcs/a/meta", json={"values": {"priority": "P0"}})
assert r.status_code == 200, r.text
sc = yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/a.meta.yaml")]["content"])
assert sc["priority"] == "P0"
# .md untouched (still body-only)
assert fake.files[("wiggleverse", "meta", "main", "rfcs/a.md")]["content"] == "Body.\n"
def test_get_rfc_exposes_meta_and_can_edit(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_refresh()
# anonymous: meta present, can_edit_meta False
r = client.get(f"{META}/rfcs/a")
assert r.status_code == 200, r.text
assert r.json()["meta"]["priority"] == "P1"
assert r.json()["can_edit_meta"] is False
# owner: can_edit_meta True
_login_owner(client)
r = client.get(f"{META}/rfcs/a")
assert r.json()["can_edit_meta"] is True
# ---- Owner-gated collection migrate endpoint (PUC-5) ----
def test_migrate_collection_endpoint_owner(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_legacy(fake, "a", priority="P1", tags=["x"])
_seed_legacy(fake, "b", priority="P0")
_refresh()
_login_owner(client)
r = client.post(f"{META}/migrate")
assert r.status_code == 200, r.text
assert r.json()["committed"] is True
assert set(r.json()["migrated"]) == {"a", "b"}
# both entries now body-only + sidecar
for slug in ("a", "b"):
assert "---" not in fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")]["content"]
assert ("wiggleverse", "meta", "main", f"rfcs/{slug}.meta.yaml") in fake.files
def test_migrate_collection_forbidden_for_contributor(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_legacy(fake, "a", priority="P1")
_refresh()
provision_user_row(user_id=2, login="carol", role="contributor")
sign_in_as(client, user_id=2, gitea_login="carol",
display_name="Carol", role="contributor")
r = client.post(f"{META}/migrate")
assert r.status_code == 403, r.text
def test_migrate_collection_idempotent(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_legacy(fake, "a", priority="P1")
_refresh()
_login_owner(client)
assert client.post(f"{META}/migrate").json()["committed"] is True
# second run: nothing left to migrate
r2 = client.post(f"{META}/migrate")
assert r2.status_code == 200, r2.text
assert r2.json()["committed"] is False
+108
View File
@@ -0,0 +1,108 @@
"""SLICE-4 — git-aware sidecar read/write helpers.
Uses the FakeGitea from the propose-vertical fixtures (no network).
"""
from __future__ import annotations
import asyncio
import yaml
from app import gitea as gitea_mod, metadata
from app.config import load_config
from test_propose_vertical import app_with_fake_gitea, tmp_env # noqa: F401
LEGACY = """---
slug: alpha
title: Alpha
state: active
owners:
- ben.stull
tags:
- one
priority: P1
---
Alpha body.
"""
MIGRATED_MD = "Alpha body.\n"
MIGRATED_SIDECAR = """slug: alpha
title: Alpha
state: active
owners:
- ben.stull
tags:
- one
priority: P1
"""
def _gitea():
return gitea_mod.Gitea(load_config())
def test_read_entry_from_git_legacy(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": LEGACY, "sha": "s1"}
st = asyncio.run(metadata.read_entry_from_git(
_gitea(), "wiggleverse", "meta", "rfcs/alpha.md"))
assert st is not None
assert st.entry.slug == "alpha"
assert st.entry.extra["priority"] == "P1"
assert st.sidecar_sha is None # no sidecar yet
assert st.malformed is False
def test_read_entry_from_git_migrated(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": MIGRATED_MD, "sha": "s1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")] = {
"content": MIGRATED_SIDECAR, "sha": "s2"}
st = asyncio.run(metadata.read_entry_from_git(
_gitea(), "wiggleverse", "meta", "rfcs/alpha.md"))
assert st.entry.extra["priority"] == "P1" # from sidecar
assert st.entry.body == "Alpha body.\n"
assert st.sidecar_sha == "s2"
def test_read_entry_from_git_missing(app_with_fake_gitea):
st = asyncio.run(metadata.read_entry_from_git(
_gitea(), "wiggleverse", "meta", "rfcs/nope.md"))
assert st is None
def test_write_entry_files_lazy_migrates_legacy(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": LEGACY, "sha": "s1"}
st = asyncio.run(metadata.read_entry_from_git(
_gitea(), "wiggleverse", "meta", "rfcs/alpha.md"))
e2 = metadata.apply_values(st.entry, {"priority": "P0"})
ops = metadata.write_entry_files("rfcs/alpha.md", e2, st)
paths = {o["path"]: o for o in ops}
# sidecar created, .md rewritten body-only
assert "rfcs/alpha.meta.yaml" in paths
assert paths["rfcs/alpha.meta.yaml"]["operation"] == "create"
assert paths["rfcs/alpha.md"]["operation"] == "update"
assert "---" not in paths["rfcs/alpha.md"]["content"] # INV-2 clean body
assert yaml.safe_load(paths["rfcs/alpha.meta.yaml"]["content"])["priority"] == "P0"
def test_write_entry_files_already_migrated_touches_sidecar_only(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": MIGRATED_MD, "sha": "s1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")] = {
"content": MIGRATED_SIDECAR, "sha": "s2"}
st = asyncio.run(metadata.read_entry_from_git(
_gitea(), "wiggleverse", "meta", "rfcs/alpha.md"))
e2 = metadata.apply_values(st.entry, {"priority": "P0"})
ops = metadata.write_entry_files("rfcs/alpha.md", e2, st)
paths = {o["path"]: o for o in ops}
assert set(paths) == {"rfcs/alpha.meta.yaml"} # .md untouched
assert paths["rfcs/alpha.meta.yaml"]["operation"] == "update"
assert paths["rfcs/alpha.meta.yaml"]["sha"] == "s2"
+135
View File
@@ -0,0 +1,135 @@
"""SLICE-1 integration — frontmatter→sidecar migration tool (PUC-5).
Per docs/design/2026-06-06-configurable-collection-metadata.md §6.5 / §7.2:
a tool walks a collection; for each entry with legacy frontmatter it writes
`<slug>.meta.yaml` and rewrites `<slug>.md` to the body only one commit per
collection, idempotent, preserving unknown keys (INV-7).
"""
from __future__ import annotations
import asyncio
import yaml
from app import gitea as gitea_mod, metadata
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
tmp_env,
)
ALPHA_MD = """---
slug: alpha
title: Alpha
state: active
owners:
- ben.stull
tags:
- one
priority: P1
---
Alpha body prose.
"""
BETA_MD = """---
slug: beta
title: Beta
state: super-draft
---
Beta body prose.
"""
def _seed_entries(fake):
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": ALPHA_MD, "sha": "a0001"}
fake.files[("wiggleverse", "meta", "main", "rfcs/beta.md")] = {
"content": BETA_MD, "sha": "b0001"}
def _run_migration(subfolder=""):
cfg = load_config()
gitea = gitea_mod.Gitea(cfg)
return asyncio.run(
metadata.migrate_collection(
gitea, org="wiggleverse", repo="meta", subfolder=subfolder
)
)
def test_migration_writes_sidecars_and_strips_bodies(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
_seed_entries(fake)
summary = _run_migration()
assert sorted(summary["migrated"]) == ["alpha", "beta"]
# Sidecars now exist.
alpha_sc = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"]
beta_sc = fake.files[("wiggleverse", "meta", "main", "rfcs/beta.meta.yaml")]["content"]
alpha_vals = yaml.safe_load(alpha_sc)
assert alpha_vals["slug"] == "alpha"
assert alpha_vals["title"] == "Alpha"
assert alpha_vals["tags"] == ["one"]
# INV-7: the unknown key rides along into the sidecar.
assert alpha_vals["priority"] == "P1"
# .md bodies are stripped of frontmatter (INV-2).
alpha_md = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
assert "---" not in alpha_md
assert "priority:" not in alpha_md
assert alpha_md.strip() == "Alpha body prose."
assert beta_sc # beta got a sidecar too
def test_migration_is_idempotent(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
_seed_entries(fake)
first = _run_migration()
assert sorted(first["migrated"]) == ["alpha", "beta"]
assert first["committed"] is True
commits_after_first = fake._commit_counter
alpha_md_after_first = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
second = _run_migration()
assert second["migrated"] == []
assert second["committed"] is False
# No new commit; files untouched.
assert fake._commit_counter == commits_after_first
assert fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"] == alpha_md_after_first
def test_migration_one_commit_for_whole_collection(app_with_fake_gitea):
_app, fake = app_with_fake_gitea
_seed_entries(fake)
before = fake._commit_counter
_run_migration()
# Two entries migrated in exactly one commit (ChangeFiles batch).
assert fake._commit_counter == before + 1
def test_migration_dual_read_equivalence_after_migrate(app_with_fake_gitea):
"""An entry reads identically before and after migration (INV-6)."""
_app, fake = app_with_fake_gitea
_seed_entries(fake)
before, _ = metadata.read_entry(ALPHA_MD, None)
_run_migration()
md = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
sc = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"]
after, malformed = metadata.read_entry(md, sc)
assert malformed is False
assert after.slug == before.slug
assert after.title == before.title
assert after.state == before.state
assert after.tags == before.tags
assert after.extra == before.extra
assert after.body.strip() == before.body.strip()
+149
View File
@@ -0,0 +1,149 @@
"""SLICE-2 unit tests — collection field schema + central validation.
Pure functions only (no DB / no Gitea). Per
docs/design/2026-06-06-configurable-collection-metadata.md §7.2 (SLICE-2):
`metadata_schema.parse_fields` (lenient schema parsing) and
`metadata_schema.validate` (advisory at read / enforcement point at write).
Honors INV-3 (never hard-fails), INV-5 (no-fields unchanged), INV-7 (undeclared
keys ride along).
"""
from __future__ import annotations
from app import metadata_schema as ms
# ---- parse_fields: normalization + leniency ----
def test_parse_fields_each_type():
raw = {
"priority": {"type": "enum", "values": ["P0", "P1", "P2"], "label": "Priority"},
"tags": {"type": "tags"},
"owner": {"type": "text"},
}
fields = ms.parse_fields(raw)
assert list(fields) == ["priority", "tags", "owner"] # order preserved
assert fields["priority"] == {
"type": "enum",
"values": ["P0", "P1", "P2"],
"label": "Priority",
}
assert fields["tags"] == {"type": "tags"}
assert fields["owner"] == {"type": "text"}
def test_parse_fields_controlled_tags_keeps_values():
fields = ms.parse_fields({"area": {"type": "tags", "values": ["a", "b"]}})
assert fields["area"] == {"type": "tags", "values": ["a", "b"]}
def test_parse_fields_missing_block_is_empty():
assert ms.parse_fields(None) == {}
assert ms.parse_fields({}) == {}
def test_parse_fields_non_mapping_block_skipped():
assert ms.parse_fields(["not", "a", "mapping"]) == {}
assert ms.parse_fields("nope") == {}
def test_parse_fields_enum_without_values_skipped():
# enum requires a non-empty values list — skipped, not fatal.
assert ms.parse_fields({"p": {"type": "enum"}}) == {}
assert ms.parse_fields({"p": {"type": "enum", "values": []}}) == {}
def test_parse_fields_unknown_type_skipped():
fields = ms.parse_fields(
{"good": {"type": "text"}, "bad": {"type": "ref"}, "huh": {"type": "frob"}}
)
assert list(fields) == ["good"]
def test_parse_fields_non_mapping_def_skipped():
fields = ms.parse_fields({"good": {"type": "text"}, "bad": "scalar"})
assert list(fields) == ["good"]
def test_parse_fields_values_coerced_to_str_list():
fields = ms.parse_fields({"p": {"type": "enum", "values": [0, 1, 2]}})
assert fields["p"]["values"] == ["0", "1", "2"]
# ---- validate: advisory problem reporting ----
SCHEMA = ms.parse_fields(
{
"priority": {"type": "enum", "values": ["P0", "P1", "P2"]},
"tags": {"type": "tags"},
"area": {"type": "tags", "values": ["checkout", "cart"]},
"owner": {"type": "text"},
}
)
def test_validate_happy():
values = {
"priority": "P0",
"tags": ["anything", "free"],
"area": ["checkout"],
"owner": "ben",
}
assert ms.validate(values, SCHEMA) == []
def test_validate_absent_fields_ok():
# A declared field that the entry omits is fine (no required fields in v1).
assert ms.validate({}, SCHEMA) == []
def test_validate_enum_bad_value():
problems = ms.validate({"priority": "P9"}, SCHEMA)
assert [p.field for p in problems] == ["priority"]
assert problems[0].code == "not-in-values"
def test_validate_enum_wrong_type():
problems = ms.validate({"priority": ["P0"]}, SCHEMA)
assert problems[0].field == "priority"
assert problems[0].code == "wrong-type"
def test_validate_tags_free_form_ok():
assert ms.validate({"tags": ["x", "y", "z"]}, SCHEMA) == []
def test_validate_tags_wrong_type():
problems = ms.validate({"tags": "notalist"}, SCHEMA)
assert problems[0].field == "tags"
assert problems[0].code == "wrong-type"
def test_validate_controlled_tags_bad_member():
problems = ms.validate({"area": ["checkout", "nope"]}, SCHEMA)
assert problems[0].field == "area"
assert problems[0].code == "not-in-values"
def test_validate_text_wrong_type():
problems = ms.validate({"owner": ["a", "b"]}, SCHEMA)
assert problems[0].field == "owner"
assert problems[0].code == "wrong-type"
def test_validate_undeclared_keys_ignored():
# INV-7: keys outside the schema ride along untouched, never flagged.
assert ms.validate({"random": "value", "slug": "x", "title": "y"}, SCHEMA) == []
def test_validate_empty_schema_no_problems():
# INV-5: a collection with no fields validates everything as clean.
assert ms.validate({"priority": "anything", "x": 1}, {}) == []
def test_problem_as_dict():
p = ms.Problem(field="priority", code="not-in-values", message="bad")
assert p.as_dict() == {
"field": "priority",
"code": "not-in-values",
"message": "bad",
}
+145
View File
@@ -0,0 +1,145 @@
"""SLICE-4 — write paths are sidecar-aware: a migrated (body-only `.md` +
sidecar) entry never crashes `entry.parse` nor re-grows frontmatter."""
from __future__ import annotations
import asyncio
from app import gitea as gitea_mod, metadata
from app.bot import Actor, Bot
from app.config import load_config
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea,
provision_user_row,
tmp_env,
)
BODY_ONLY = "Alpha prose body.\n"
SIDECAR = ("slug: alpha\ntitle: Alpha\nstate: active\n"
"owners:\n- ben.stull\ntags:\n- one\npriority: P1\n")
def _seed_migrated(fake, repo="meta"):
fake.files[("wiggleverse", repo, "main", "rfcs/alpha.md")] = {
"content": BODY_ONLY, "sha": "m1"}
fake.files[("wiggleverse", repo, "main", "rfcs/alpha.meta.yaml")] = {
"content": SIDECAR, "sha": "m2"}
def _actor():
return Actor(user_id=1, gitea_login="ben.stull", display_name="Ben", email="ben@x.io")
def test_mark_entry_reviewed_on_migrated_entry(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
_seed_migrated(fake)
gitea = gitea_mod.Gitea(load_config())
bot = Bot(gitea)
with TestClient(app):
provision_user_row(user_id=1, login="ben.stull", role="owner")
# Must not raise (legacy code parsed body-only .md → ValueError).
asyncio.run(bot.mark_entry_reviewed(
_actor(), org="wiggleverse", meta_repo="meta", slug="alpha",
reviewed_by="ben.stull", reviewed_at="2026-06-07"))
md = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
sc = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"]
assert "---" not in md # body stays clean (no re-grown FM)
assert "reviewed_by: ben.stull" in sc # review stamp landed in the sidecar
# ---- Task 3.2: body extract/wrap helpers (api_branches) ----
def test_extract_wrap_body_on_body_only_md():
from app import api_branches
rfc = {"state": "super-draft", "repo": None, "slug": "alpha", "collection_id": "default"}
body = api_branches._extract_body_pure(rfc, BODY_ONLY, "main", is_meta=True)
assert body == BODY_ONLY
wrapped = api_branches._wrap_body_pure(rfc, BODY_ONLY, "new body\n", "main", is_meta=True)
assert wrapped == "new body\n" # stays clean — no re-grown frontmatter
def test_wrap_body_preserves_legacy_frontmatter():
from app import api_branches
legacy = "---\nslug: alpha\ntitle: Alpha\nstate: active\n---\n\nold body\n"
rfc = {"state": "super-draft", "repo": None, "slug": "alpha", "collection_id": "default"}
wrapped = api_branches._wrap_body_pure(rfc, legacy, "new body\n", "main", is_meta=True)
assert wrapped.startswith("---") # legacy frontmatter preserved
assert "new body" in wrapped
# ---- Task 3.3: PR-replay body wrappers (api_prs) ----
def test_replay_wrappers_on_body_only():
from app import api_prs
assert api_prs._extract_body_for_replay(True, BODY_ONLY) == BODY_ONLY
out = api_prs._wrap_body_for_replay(True, BODY_ONLY, "new\n")
assert out == "new\n" # clean, no re-grown frontmatter
legacy = "---\nslug: a\ntitle: A\nstate: active\n---\n\nold\n"
out2 = api_prs._wrap_body_for_replay(True, legacy, "new\n")
assert out2.startswith("---") # legacy preserved
# ---- Task 3.4: retire a fully-migrated (body-only + sidecar) entry ----
def test_retire_already_migrated_entry_does_not_crash(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import cache, db
from app.config import load_config
app, fake = app_with_fake_gitea
_seed_migrated(fake)
with TestClient(app) as client:
provision_user_row(user_id=1, login="ben", role="owner")
from test_propose_vertical import sign_in_as
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
# Ingest the migrated entry so the catalog/cache knows it.
asyncio.run(cache.refresh_meta_repo(load_config(), gitea_mod.Gitea(load_config())))
assert db.conn().execute(
"SELECT state FROM cached_rfcs WHERE slug='alpha'").fetchone()["state"] == "active"
r = client.post("/api/rfcs/alpha/retire")
assert r.status_code == 200, r.text
assert r.json()["state"] == "retired"
import yaml as _yaml
sc = _yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"])
assert sc["state"] == "retired"
md = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
assert "---" not in md # body stayed clean
# ---- Task 3.5: graduate a fully-migrated super-draft entry ----
SUPER_SIDECAR = ("slug: alpha\ntitle: Alpha\nstate: super-draft\n"
"owners:\n- ben\ntags:\n- one\npriority: P1\n")
def test_graduate_already_migrated_super_draft(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import cache, db
from app.config import load_config
app, fake = app_with_fake_gitea
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")] = {
"content": BODY_ONLY, "sha": "m1"}
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")] = {
"content": SUPER_SIDECAR, "sha": "m2"}
with TestClient(app) as client:
from test_propose_vertical import sign_in_as
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
asyncio.run(cache.refresh_meta_repo(load_config(), gitea_mod.Gitea(load_config())))
assert db.conn().execute(
"SELECT state FROM cached_rfcs WHERE slug='alpha'").fetchone()["state"] == "super-draft"
r = client.post("/api/rfcs/alpha/graduate?_sync=1",
json={"rfc_id": "RFC-0007", "owners": ["ben"]})
assert r.status_code == 200, r.text
import yaml as _yaml
sc = _yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.meta.yaml")]["content"])
assert sc["state"] == "active"
assert sc["id"] == "RFC-0007"
assert sc.get("priority") == "P1" # INV-7 carried through graduation
md = fake.files[("wiggleverse", "meta", "main", "rfcs/alpha.md")]["content"]
assert "---" not in md # body stayed clean
@@ -137,3 +137,63 @@ def test_memberships_table_replaces_project_members():
conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('bogus','default',9,'owner')")
with pytest.raises(sqlite3.IntegrityError):
conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('project','default',9,'viewer')")
# ── regression: §22.13 satellite re-stamp repair (the OHM-data deploy fault) ──
# Reproduces the shape that crashed the v0.46.0 deploy: the default→ohm re-stamp
# updated cached_rfcs but left cached_branches at the stale project_id='default',
# with (a) a stale row duplicating a freshly-stamped one, (b) a stale row with no
# fresh counterpart, and (c) a stale row whose RFC no longer exists. 029 must
# repair all three rather than hit NOT NULL / UNIQUE on the rebuild.
def _apply_through(path, ceiling):
conn = sqlite3.connect(path, isolation_level=None)
conn.row_factory = sqlite3.Row
conn.execute("CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY, applied_at TEXT NOT NULL DEFAULT (datetime('now')))")
done = {r["version"] for r in conn.execute("SELECT version FROM schema_migrations")}
for p in sorted(db.MIGRATIONS_DIR.glob("*.sql")):
v = p.stem
if v in done or v > ceiling:
continue
sql = p.read_text()
if "-- migrate:no-foreign-keys" in sql:
conn.execute("PRAGMA foreign_keys = OFF")
conn.executescript("BEGIN; " + sql + "; COMMIT;")
conn.execute("PRAGMA foreign_keys = ON")
else:
conn.executescript("BEGIN; " + sql + "; COMMIT;")
conn.execute("INSERT INTO schema_migrations (version) VALUES (?)", (v,))
return conn
def test_029_repairs_stale_duplicate_and_orphan_satellite_rows():
d = tempfile.mkdtemp()
path = str(Path(d) / "t.db")
conn = _apply_through(path, "028_project_scoped_keys")
# simulate the §22.13 re-stamp having renamed the default project + its RFCs
# to 'ohm', but NOT the satellite tables (the actual prod fault).
conn.execute("UPDATE projects SET id='ohm' WHERE id='default'")
conn.execute("INSERT INTO cached_rfcs (slug, title, state, project_id) VALUES ('human','Human','active','ohm')")
conn.execute("INSERT INTO cached_branches (rfc_slug, branch_name, project_id) VALUES ('human','main','ohm')") # fresh/correct
conn.execute("INSERT INTO cached_branches (rfc_slug, branch_name, project_id) VALUES ('human','main','default')") # stale DUP of the fresh one
conn.execute("INSERT INTO cached_branches (rfc_slug, branch_name, project_id) VALUES ('human','edit-1','default')")# stale, unique -> re-stamp+keep
conn.execute("INSERT INTO cached_branches (rfc_slug, branch_name, project_id) VALUES ('ghost','main','default')") # no live RFC -> drop
conn.close()
# apply 029+ (the patched migration). Must NOT raise.
db.run_migrations(_Cfg(path))
conn = db.connect(path)
rows = conn.execute(
"SELECT rfc_slug, branch_name, collection_id FROM cached_branches"
).fetchall()
got = {(r["rfc_slug"], r["branch_name"]) for r in rows}
# every surviving row mapped to a collection (the single-project 'default' one)
assert all(r["collection_id"] is not None for r in rows)
assert {r["collection_id"] for r in rows} == {"default"}
# the duplicate collapsed to exactly one human/main
assert len([r for r in rows if (r["rfc_slug"], r["branch_name"]) == ("human", "main")]) == 1
# the unique stale row survived (re-stamped)
assert ("human", "edit-1") in got
# the no-RFC stale row was dropped
assert ("ghost", "main") not in got
+26
View File
@@ -54,6 +54,9 @@ class FakeGitea:
self.repos: set[tuple[str, str]] = set()
self._pr_counter = 0
self._commit_counter = 0
# count of batch ChangeFiles commits (one per /contents POST with a
# files[] array) — lets tests assert "N files, one commit" (§22.4a D7).
self.change_files_calls = 0
self._seed_repo("wiggleverse", "meta")
# §22 M3: the deployment's project registry. Startup refresh_registry
# reads projects.yaml here; the single 'default' project's content_repo
@@ -280,6 +283,29 @@ class FakeGitea:
return httpx.Response(200, json=children)
return httpx.Response(404, json={"message": "not found"})
# POST /repos/{owner}/{repo}/contents — ChangeFiles (batch, one commit).
# §22.4a SLICE-1: the frontmatter→sidecar migration writes N files in a
# single commit. Matches the no-path /contents route (the per-path POST
# below needs a /contents/<path> suffix).
m_batch = re.fullmatch(r"/repos/([^/]+)/([^/]+)/contents/?", path)
if method == "POST" and m_batch:
owner, repo = m_batch.groups()
branch = payload["branch"]
self.change_files_calls += 1
sha = self._next_sha()
for f in payload["files"]:
op = f["operation"]
fpath = f["path"]
if op == "delete":
self.files.pop((owner, repo, branch, fpath), None)
else:
content = base64.b64decode(f["content"]).decode()
self.files[(owner, repo, branch, fpath)] = {"content": content, "sha": sha}
br = self.branches[(owner, repo)].setdefault(branch, {})
br["sha"] = sha
br["ts"] = "2026-05-23T00:00:00Z"
return httpx.Response(201, json={"commit": {"sha": sha}})
# POST /repos/{owner}/{repo}/contents/{path}
m = re.fullmatch(r"/repos/([^/]+)/([^/]+)/contents/(.+)", path)
if method == "POST" and m:
+44
View File
@@ -0,0 +1,44 @@
"""The per-IP limiter budgets are env-overridable (test/PPE stacks drive the
auth endpoints repeatedly from one IP); production leaves them unset and keeps
the secure defaults. A non-positive / unparseable value falls back."""
from __future__ import annotations
import importlib
import pytest
def _reload_with(monkeypatch, **env):
for k, v in env.items():
if v is None:
monkeypatch.delenv(k, raising=False)
else:
monkeypatch.setenv(k, v)
import app.ratelimit as ratelimit
return importlib.reload(ratelimit)
@pytest.fixture(autouse=True)
def _restore():
yield
# Leave the module in its default state for other tests.
import app.ratelimit as ratelimit
importlib.reload(ratelimit)
def test_defaults_when_unset(monkeypatch):
rl = _reload_with(monkeypatch, RATELIMIT_OTC_REQUEST_MAX=None, RATELIMIT_VERIFY_MAX=None)
assert rl.otc_request_limiter.max_events == 5
assert rl.verify_limiter.max_events == 10
def test_env_override(monkeypatch):
rl = _reload_with(monkeypatch, RATELIMIT_OTC_REQUEST_MAX="1000", RATELIMIT_VERIFY_MAX="250")
assert rl.otc_request_limiter.max_events == 1000
assert rl.verify_limiter.max_events == 250
def test_bad_value_falls_back_to_default(monkeypatch):
rl = _reload_with(monkeypatch, RATELIMIT_OTC_REQUEST_MAX="nope", RATELIMIT_VERIFY_MAX="0")
assert rl.otc_request_limiter.max_events == 5 # unparseable → default
assert rl.verify_limiter.max_events == 10 # non-positive → default
@@ -0,0 +1,156 @@
"""§22 framework bug — migration-029 vs registry-mirror collection-id
divergence for a multi-project deployment's default project.
Migration 029 seeds the default project's collection id as the literal
'default' only when one project exists at migration time; with 2 projects it
falls back to the *project id*. The registry mirror expects the default
project to own the collection id 'default'. On an upgrade whose DB already
held 2 projects when 029 ran, the default project's collection is therefore
named after the project (e.g. 'ohm'), and the next mirror would INSERT a
second, empty 'default' collection. `reconcile_default_collection_id` heals
the divergence at startup, before the mirror, so the mirror merges instead of
duplicating. This is the collection-grain twin of `restamp_default_project`.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
import app.db as db
from app import projects, registry
_TWO_PROJECT_REGISTRY = """
deployment:
name: Open Human Model
tagline: t
projects:
- id: ohm
name: Open Human Model
type: document
content_repo: ohm-content
visibility: public
- id: ecomm
name: Ecomm
type: bdd
content_repo: ecomm-content
visibility: public
"""
def _collection_ids_for(conn, project_id):
return {r["id"] for r in conn.execute(
"SELECT id FROM collections WHERE project_id = ?", (project_id,))}
class _Cfg:
def __init__(self, path, default_id):
self.database_path = path
self.default_project_id = default_id
def _divergent_multiproject(monkeypatch, default_id="ohm"):
"""A DB in the post-029 divergent state: the default project ('ohm') owns a
collection whose id is the project id (the 029 2-projects seed), a second
project ('ecomm') owns its own collection, and NO 'default' collection
exists. Entry rows point at the divergent collection_id='ohm'."""
path = str(Path(tempfile.mkdtemp()) / "t.db")
cfg = _Cfg(path, default_id)
db.run_migrations(cfg) # seeds bootstrap 'default' project + 'default' collection
monkeypatch.setattr(db, "_CONN", db.connect(path))
conn = db.conn()
# Drop the single-project bootstrap seed and rebuild the divergent
# multi-project state 029 would have produced on an upgrade.
conn.execute("DELETE FROM collections")
conn.execute("DELETE FROM projects")
conn.execute("INSERT INTO projects (id,name,content_repo,visibility) "
"VALUES ('ohm','Open Human Model','ohm-content','public')")
conn.execute("INSERT INTO projects (id,name,content_repo,visibility) "
"VALUES ('ecomm','Ecomm','ecomm-content','public')")
# 029 ≥2-projects seed: collection id == project id.
conn.execute("INSERT INTO collections (id,project_id,type,subfolder,initial_state,visibility,name) "
"VALUES ('ohm','ohm','document','','super-draft','public','Open Human Model')")
conn.execute("INSERT INTO collections (id,project_id,type,subfolder,initial_state,visibility,name) "
"VALUES ('ecomm','ecomm','bdd','ecomm','super-draft','public','Ecomm')")
conn.execute("INSERT INTO users (id,gitea_login,display_name,role) VALUES (1,'a','A','contributor')")
# Entry data for the default project lives in the divergent 'ohm' collection.
conn.execute("INSERT INTO cached_rfcs (slug,title,state,collection_id) VALUES ('human','Human','active','ohm')")
conn.execute("INSERT INTO rfc_collaborators (rfc_slug,user_id,role_in_rfc,collection_id) "
"VALUES ('human',1,'contributor','ohm')")
conn.execute("INSERT INTO stars (user_id,rfc_slug,collection_id) VALUES (1,'human','ohm')")
# The 'ecomm' collection has its own entry.
conn.execute("INSERT INTO cached_rfcs (slug,title,state,collection_id) VALUES ('cart','Cart','active','ecomm')")
return cfg, conn
def test_reconcile_renames_divergent_default_collection_to_default(monkeypatch):
cfg, conn = _divergent_multiproject(monkeypatch, default_id="ohm")
projects.reconcile_default_collection_id(cfg)
# The default project's collection id is now the canonical 'default'.
assert conn.execute("SELECT 1 FROM collections WHERE id='ohm'").fetchone() is None
row = conn.execute("SELECT project_id FROM collections WHERE id='default'").fetchone()
assert row is not None and row["project_id"] == "ohm"
# Entry rows cascaded onto 'default'.
assert conn.execute("SELECT collection_id FROM cached_rfcs WHERE slug='human'").fetchone()["collection_id"] == "default"
assert conn.execute("SELECT collection_id FROM rfc_collaborators WHERE rfc_slug='human'").fetchone()["collection_id"] == "default"
assert conn.execute("SELECT collection_id FROM stars WHERE rfc_slug='human'").fetchone()["collection_id"] == "default"
# The non-default 'ecomm' collection is untouched (mirror + 029 agree on it).
assert conn.execute("SELECT 1 FROM collections WHERE id='ecomm'").fetchone() is not None
assert conn.execute("SELECT collection_id FROM cached_rfcs WHERE slug='cart'").fetchone()["collection_id"] == "ecomm"
# FK integrity intact after the rename.
assert conn.execute("PRAGMA foreign_key_check").fetchall() == []
def test_reconcile_is_idempotent(monkeypatch):
cfg, conn = _divergent_multiproject(monkeypatch, default_id="ohm")
projects.reconcile_default_collection_id(cfg)
projects.reconcile_default_collection_id(cfg) # second call: already aligned → no-op
assert conn.execute("SELECT project_id FROM collections WHERE id='default'").fetchone()["project_id"] == "ohm"
assert conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE collection_id='default'").fetchone()["c"] == 1
def test_reconcile_noop_when_default_id_is_literal_default(monkeypatch):
# Single-project deployment, no DEFAULT_PROJECT_ID: 029 already seeded
# 'default' and the mirror agrees — nothing to reconcile.
path = str(Path(tempfile.mkdtemp()) / "t.db")
cfg = _Cfg(path, "") # resolves to 'default'
db.run_migrations(cfg)
monkeypatch.setattr(db, "_CONN", db.connect(path))
conn = db.conn()
projects.reconcile_default_collection_id(cfg)
assert conn.execute("SELECT 1 FROM collections WHERE id='default'").fetchone() is not None
def test_mirror_duplicates_without_reconcile(monkeypatch):
# Demonstrates the bug: the mirror on the divergent state inserts a SECOND
# 'default' collection for the default project (alongside the 029 'ohm').
cfg, conn = _divergent_multiproject(monkeypatch, default_id="ohm")
doc = registry.parse_registry(_TWO_PROJECT_REGISTRY)
registry.apply_registry(doc, registry_sha="s1", default_id="ohm")
assert _collection_ids_for(conn, "ohm") == {"ohm", "default"} # duplicate!
def test_reconcile_then_mirror_merges_no_duplicate(monkeypatch):
# With the fix: reconcile before the mirror → the mirror merges onto the
# canonical 'default' collection; the default project owns exactly one.
cfg, conn = _divergent_multiproject(monkeypatch, default_id="ohm")
projects.reconcile_default_collection_id(cfg)
doc = registry.parse_registry(_TWO_PROJECT_REGISTRY)
registry.apply_registry(doc, registry_sha="s1", default_id="ohm")
assert _collection_ids_for(conn, "ohm") == {"default"}
assert _collection_ids_for(conn, "ecomm") == {"ecomm"}
# The default project's corpus entry is intact under 'default'.
assert conn.execute("SELECT collection_id FROM cached_rfcs WHERE slug='human'").fetchone()["collection_id"] == "default"
# The mirror refreshed the merged collection's metadata (type from registry).
assert conn.execute("SELECT type FROM collections WHERE id='default'").fetchone()["type"] == "document"
def test_reconcile_skips_when_default_collection_already_exists(monkeypatch):
# A prior buggy mirror already created a 'default' collection alongside the
# divergent 'ohm' one: don't auto-merge data — leave both for operator cleanup.
cfg, conn = _divergent_multiproject(monkeypatch, default_id="ohm")
conn.execute("INSERT INTO collections (id,project_id,type,subfolder,initial_state,visibility,name) "
"VALUES ('default','ohm','document','','super-draft','public','dup')")
projects.reconcile_default_collection_id(cfg)
# Both still present (no destructive auto-merge).
assert conn.execute("SELECT 1 FROM collections WHERE id='ohm'").fetchone() is not None
assert conn.execute("SELECT 1 FROM collections WHERE id='default'").fetchone() is not None
+13 -9
View File
@@ -57,12 +57,15 @@ def test_rfc_owner_can_retire_and_entry_leaves_every_surface(app_with_fake_gitea
assert r.status_code == 200, r.text
assert r.json()["state"] == "retired"
# Meta entry on main: state retired, body + fields kept.
meta = entry_mod.parse(
fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
# §22.4a SLICE-4: the state flip lands in the metadata sidecar and the
# `.md` is lazy-migrated to a clean body-only file (INV-2). Fields kept.
import yaml as _yaml
sc = _yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.meta.yaml")]["content"]
)
assert meta.state == "retired"
assert "carol" in meta.owners
assert sc["state"] == "retired"
assert "carol" in sc["owners"]
assert "---" not in fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
# Cache flipped; gone from the catalog.
cached = db.conn().execute(
@@ -177,11 +180,12 @@ def test_site_owner_can_retire_active_and_unretire_restores_active_with_id(app_w
assert r.status_code == 200, r.text
assert r.json()["state"] == "active"
meta = entry_mod.parse(
fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")]["content"]
import yaml as _yaml
sc = _yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.meta.yaml")]["content"]
)
assert meta.state == "active"
assert meta.id == "RFC-0042"
assert sc["state"] == "active"
assert sc["id"] == "RFC-0042"
cached = db.conn().execute(
"SELECT state, rfc_id FROM cached_rfcs WHERE slug = 'ohm'"
+1 -1
View File
@@ -5,7 +5,7 @@
#
# while IFS='=' read -r k v; do
# [ -n "$k" ] && case "$k" in \#*) ;; *) \
# ohm-rfc-app-flotilla overlay set ohm-rfc-app "$k=$v" --preview ;; esac
# flotilla-core overlay set <deployment> "$k=$v" --preview ;; esac
# done < deploy/preview/preview.env.example
#
# CRITICAL (§15 / §3 invariant 1): a preview resolves ZERO real secret bytes.
@@ -5,105 +5,167 @@
| **Author(s)** | Ben Stull |
| **Reviewers / approvers** | Ben Stull |
| **Status** | `draft` |
| **Version** | v0.1.0 |
| **Source artifacts** | Reference modeled: retired **BDD Release Planner** (`wiggleverse/wiggleverse-ecomm-bdd-release-planner-app`, RETIRED 2026-06-04) · Related spec: [`docs/design/2026-06-05-three-tier-projects-collections.md`](./2026-06-05-three-tier-projects-collections.md) (§22) · Corpus: ecomm Shopify-modeled BDD (`wiggleverse-ecomm-meta/research/shopify`, ~1,238 scenarios) · Supersedes: |
| **Version** | v0.1.6 |
| **Source artifacts** | Reference modeled: retired **BDD Release Planner** (`wiggleverse/wiggleverse-ecomm-bdd-release-planner-app`, RETIRED 2026-06-04) · Related: [`2026-06-05-three-tier-projects-collections.md`](./2026-06-05-three-tier-projects-collections.md) (§22) · Corpus: ecomm Shopify-modeled BDD (`wiggleverse-ecomm-meta/research/shopify`, ~1,238 scenarios) · **Supersedes:** [`2026-06-06-per-type-surfaces.md`](./2026-06-06-per-type-surfaces.md) |
**Change log**
| Date | Version | Change | By |
| --- | --- | --- | --- |
| 2026-06-06 | v0.1.0 | Initial draft from discovery session OHM-0079.0 | Ben Stull |
| 2026-06-06 | v0.1.1 | Value-only Executive Summary; add Pain Points | Ben Stull |
| 2026-06-06 | v0.1.2 | Business Outcomes restated as business (adoption/diversity); Business Use Cases → solution-agnostic | Ben Stull |
| 2026-06-06 | v0.1.3 | Supersede per-type-surfaces draft (harvest patterns; bdd coverage future; §22.4a amendment); split Business Actors / Product Personas | Ben Stull |
| 2026-06-06 | v0.1.4 | Two-part restructure: §1 Business Context (solution-agnostic, 1.11.9) + §2 Solution Proposal; renumber | Ben Stull |
| 2026-06-06 | v0.1.5 | Move Business Actors to §1.3 (define roles before Problem/Pain reference them) | Ben Stull |
| 2026-06-06 | v0.1.6 | §7.1 execution convention — each slice is its own writing-plans→executing-plans coding session, plans just-in-time | Ben Stull |
---
## 1. Executive Summary
## 1. Business Context
rfc-app gains a generic, **per-collection-configurable metadata system**. A collection declares its fields — `priority`, `tags`, and arbitrary custom fields — in its `.collection.yaml`; each entry's *values* live in a **sidecar** (`<slug>.meta.yaml`) so the document body stays pure prose. rfc-app renders schema-driven forms and **faceted left-pane filters**, and supports fast **single + bulk tag/untag** via direct commits for authorized roles. This restores the *corpus-annotation* capability of the retired BDD Release Planner inside the framework, while keeping release *planning* (ordering, ship status, roadmap emission) a downstream concern that reads the sidecars straight from git.
*The business lens — solution-agnostic throughout. No mechanism is proposed until §2.*
## 2. Business Context
### 1.1 Executive Summary
The framework hosts RFC standardization for multiple deployments. One deployment hosts the ecomm BDD corpus — ~1,238 Shopify-modeled scenarios, one markdown file per scenario, slugged by feature ID (`DD-FF-NNNN-slug`). A standalone **BDD Release Planner** previously let operators search that corpus, attach metadata (priority P0P3, owner, status), cluster scenarios into named releases, and emit each release as a roadmap phase. §22 (three-tier projects/collections) absorbed the planner's *corpus hosting* into rfc-app (the corpus now runs as a `bdd` project on the RFC deployment) and the planner was retired — but its *annotation* half (priority/tags on scenarios, filtering, bulk assignment) was never rebuilt. Operators planning work, and downstream tools consuming the corpus, currently have no structured, framework-native way to set or read that metadata.
A deployment's corpus is only as valuable as the ability of the people running it to prioritise it, navigate it, and act on it — and as valuable as the downstream tools that can read structured signal out of it. Today that value is stranded: operators and contributors can't rank what matters or find content by what matters, and the tools meant to plan and build from the corpus have nothing structured to consume. The value at stake is **lower-friction corpus planning** for operators and contributors, **broader adoption** by teams whose document types the platform couldn't previously serve, and **a corpus external tooling can consume without bespoke glue**. *(Value summary; the solution is proposed in §2.)*
## 3. Problem Statement
### 1.2 Background
Today rfc-app metadata is weak and intrusive:
The framework hosts RFC standardization for multiple deployments. One deployment hosts the ecomm BDD corpus — ~1,238 Shopify-modeled scenarios, one markdown file per scenario, slugged by feature ID (`DD-FF-NNNN-slug`). A standalone **BDD Release Planner** previously let operators search that corpus, attach metadata (priority P0P3, owner, status), cluster scenarios into named releases, and emit each release as a roadmap phase. §22 (three-tier projects/collections) absorbed the planner's *corpus hosting* into rfc-app (the corpus now runs as a `bdd` project on the RFC deployment) and the planner was retired — but its *annotation* half (priority/tags on scenarios, filtering, bulk assignment) was never rebuilt. Teams evaluating rfc-app for *other* document types often need structured attributes (a priority, a status, domain tags) the platform can't yet express — so they go elsewhere.
- **Tags are free-form strings** with **no filtering** — §7.1 specifies a `Tag:` filter chip that was never implemented.
- **No priority**, and no way for a collection to declare any other structured field.
- **No per-collection schema** — every collection gets the same fixed frontmatter shape; a `bdd` collection cannot say "scenarios have a P0P3 priority."
- **Metadata pollutes the document body** — a large YAML frontmatter block sits at the top of every doc, mixing rfc-app lifecycle bookkeeping with content metadata.
- **No bulk workflow** — annotating hundreds of scenarios one PR at a time is impractical, so the planner's core gesture has no analog.
### 1.3 Business Actors / Roles
Result: corpus authors can't meaningfully prioritise/tag, downstream consumers have nothing structured to read, and documents aren't clean.
Real-world roles, **solution-agnostic** — they exist whether or not rfc-app does. They are defined here, before the Problem (§1.4) and Pain Points (§1.5) reference them; the Business Use Cases (§1.9) are about these roles, and the Product Personas (§3) map onto them.
## 4. Stakeholders / Personas / Actors
| Role | Responsible for (in the business) |
| --- | --- |
| Standards owner | Owns an organization's RFC / standards / requirements process; decides what's tracked and how |
| Release planner | Decides what work belongs in upcoming releases |
| Requirements author | Proposes and curates the requirements (e.g. BDD scenarios) |
| Requirements consumer | A person or downstream tool that plans or builds from the requirements |
| Reader | Anyone navigating the corpus to find what's relevant to them |
| Actor | Type | Goal in this design |
| --- | --- | --- |
| Corpus contributor | persona | Set priority and tags on a scenario when proposing/curating it |
| Release planner (operator) | persona | Bulk-prioritise/tag many scenarios quickly to shape a plan |
| Collection owner | operator | Declare the metadata fields a collection supports |
| Downstream consumer | system | Read structured per-scenario metadata from the git corpus (e.g. an external release planner) |
| Reader | persona | Read clean scenario docs; filter the catalog by priority/tag |
### 1.4 Problem Statement
## 5. Scope
rfc-app cannot express or surface structured signal about its content. Tags are free-form strings with no filtering; there is no notion of priority or any other collection-defined attribute; the catalog is a flat list; and what little metadata exists is mixed into the top of every document. As a result, a corpus cannot be prioritised, navigated by attribute, planned in bulk, or cleanly consumed by downstream tools — and teams whose workflows depend on such attributes cannot adopt the platform at all.
- **In scope:** per-collection field schema in `.collection.yaml` (`enum`, `tags`, `text`); per-entry sidecar (`<slug>.meta.yaml`) as source of truth with pure-prose doc bodies; schema-derived faceted left-pane filtering with counts; schema-derived metadata form (detail panel); single + bulk tag/untag with direct commit for authorized roles; dual-read compatibility + a one-shot frontmatter→sidecar migration tool + lazy migration on write.
- **Out of scope:** release ordering, ship status, roadmap emission, `RELEASE-PLAN.md` generation — these stay downstream, consuming sidecars from git. In-app management of field definitions / controlled vocabularies; corpus-wide tag rename/merge/delete. Sub-document (per-scenario-within-a-file) grain. A whole-corpus metadata export endpoint.
- **Non-goals:** rebuilding a bespoke "release" entity in rfc-app — releases are not modeled here at all; metadata is the only primitive.
### 1.5 Pain Points
## 6. Assumptions · Constraints · Dependencies
| # | Pain | Who feels it | Cost / frequency today |
| --- | --- | --- | --- |
| PP-1 | Scenarios carry no priority, so triage and planning happen off-platform, in spreadsheets and memory | Release planner, contributor | Every planning cycle; signal lives off-platform and goes stale |
| PP-2 | The catalog is a flat, unfilterable list — at ~1,200 scenarios, "show me the P0 checkout scenarios" is impractical | Reader, release planner | Every browse/triage; finding the right work is slow and error-prone |
| PP-3 | Tags are free-form with no filtering payoff, so they're decorative and go unmaintained | Contributor | Ongoing; the one existing affordance rots |
| PP-4 | Annotating many scenarios means opening many PRs, so bulk planning has no home in the tool | Release planner | Every batch; the core planning gesture is effectively impossible |
| PP-5 | rfc-app metadata clutters the top of every document, hurting readability and making the corpus awkward to consume cleanly | Reader, downstream consumer | Every read; every downstream integration |
| PP-6 | Downstream tools have no structured signal to read — the retired planner's capability left a gap | Downstream consumer | Continuous since the planner's retirement |
| PP-7 | Teams whose document types need structured attributes can't model them, so they don't adopt rfc-app | Prospective adopter (org/team) | Every evaluation that ends in "not yet" |
- **Assumptions:** git remains the content source of truth and downstream consumers can read the corpus from git; the BDD grain is one markdown file per scenario (already true for the ecomm corpus), so no sub-document parsing is needed.
- **Constraints:** rfc-app is a framework hosting multiple deployments — the upgrade must be **mechanical and non-breaking**, with §20 changelog/upgrade-steps; the hard secrets rule (§6.3) holds; metadata edits must respect scope-role authorization (§22 Part B / S3).
- **Dependencies:** the S3 scope-role resolver (`auth.effective_scope_role`) for edit authorization; the existing git write-through path used by `edit-meta` (§9.5); the §22 collection model (`.collection.yaml`, `cached_rfcs` keyed by `(collection_id, slug)`).
### 1.6 Targeted Business Outcomes
## 7. Targeted Business Outcomes
Business outcomes for rfc-app as a platform — adoption, reach, and diversity of use — **not** solution outputs. (Whether documents carry a priority is a solution output, tracked as a slice's Definition of Done in §7, not here.)
| Outcome | Success metric | Baseline → Target | Guardrail (must not regress) | How / when measured |
| --- | --- | --- | --- | --- |
| Authors can prioritise/tag scenarios | scenarios carrying a priority | 0 → corpus-wide | doc bodies stay prose-clean | corpus inspection after rollout |
| Corpus is filterable | left-pane facet filters available | none → priority+tags+state | filter latency acceptable at ~1.2k entries | manual + perf check |
| Downstream tools can consume metadata | sidecars readable from git | none → all migrated entries | sidecar schema stable | consumer integration |
| Clean docs | docs with no rfc-app frontmatter | 0% → 100% (post-migration) | dual-read keeps legacy working | post-migration inspection |
| Teams blocked by missing structured attributes now adopt rfc-app | # organizations on rfc-app; # active users | internal deployments only → external orgs onboard | existing deployments don't churn | deployment registry + usage analytics; quarterly |
| The platform hosts a wider variety of workflows and document types | # distinct document/collection types & field schemas in use | today's handful → broader mix | existing types' experience unchanged | type/schema census; quarterly |
| Corpus planning happens on-platform rather than in side tools | share of prioritisation/planning done in rfc-app vs spreadsheets | largely off-platform → on-platform | — | operator interviews + usage signals; quarterly |
## 8. Business Use Cases
### 1.7 Scope (business)
- **In scope:** the corpus can carry per-item importance and categorisation; people can find items by those attributes; the signal is captured durably and is consumable by other people and tools; teams with new document types can express the attributes their workflow needs.
- **Out of scope (business):** deciding *what* a given deployment's priorities or categories should be (that's the deployment's editorial choice); release sequencing and ship tracking as a business process (stays a downstream/operator concern).
- **Non-goals:** modelling "releases" as a first-class business object inside the platform.
*(Solution-specific scope/non-goals are in §2.)*
### 1.8 Assumptions · Constraints · Dependencies
- **Assumptions:** git remains the content source of truth and downstream consumers can read the corpus from git; the BDD grain is one markdown file per scenario (already true for the ecomm corpus).
- **Constraints:** rfc-app is a framework hosting multiple deployments — any change must be **mechanical and non-breaking**, with §20 changelog/upgrade-steps; the hard secrets rule (§6.3) holds; edits must respect scope-role authorization (§22 Part B / S3); the §22.4a "engine unchanged" rule holds (INV-8).
- **Dependencies:** the S3 scope-role resolver (`auth.effective_scope_role`); the existing git write-through used by `edit-meta` (§9.5); the §22 collection model; the binding `SPEC.md` §22.4a contract, which §2's solution amends (§7 SLICE-0).
### 1.9 Business Use Cases
Solution-agnostic: what an actor (§1.3) wants to accomplish, *why* (value), and what *success* looks like — **no reference to any product**. Each could be satisfied by a person by hand before any software. Form: "As a … I can … so that …".
**BUC-1 — As a release planner, I can prioritise the requirements in a body of work, so that I can decide what belongs in upcoming releases.**
```gherkin
Scenario: BUC-1 — A contributor prioritises a scenario
Given a bdd collection whose schema defines a priority field
When a contributor marks a scenario as P0
Then the scenario's metadata records priority P0
And the document body is unchanged prose
Scenario: BUC-2 — A planner batch-prioritises a set of scenarios
Given a contributor has selected 40 scenarios
When they set priority P1 on the selection
Then all 40 carry priority P1
And the change lands as a single auditable commit
Scenario: BUC-3 — A downstream tool consumes priorities
Given scenarios carry priority and tags in their sidecars
When an external release planner reads the corpus from git
Then it can group and order scenarios using that metadata
And rfc-app did not need to model releases at all
Scenario: BUC-4 — Documents stay clean
Given a migrated collection
When a reader opens a scenario document
Then they see only prose, with no rfc-app metadata block
Scenario: BUC-1 — Prioritise to plan releases
Given a body of requirements of varying importance
When the planner weighs which matter most
Then they hold a ranking of those requirements by importance
And can decide a release's contents from it
```
- **Acceptance:** the planner can select and justify the next release's contents from the relative importance of the work.
- **BUC-1 acceptance:** the scenario's `priority` value is `P0` and its `.md` body is byte-identical to before.
- **BUC-2 acceptance:** 40 sidecars updated; exactly one commit; authorship recorded.
- **BUC-3 acceptance:** sidecars + `.collection.yaml` are sufficient for a consumer to read priority/tags without rfc-app's API.
- **BUC-4 acceptance:** no `---` frontmatter remains in migrated docs.
**BUC-2 — As a planner facing a large body of requirements, I can organise and triage it within a normal working session, so that planning actually gets done rather than deferred or improvised.**
```gherkin
Scenario: BUC-2 — Triage at scale
Given more requirements than can be weighed one at a time
When the planner ranks and groups them in bulk
Then the body of work reflects those decisions without per-item drudgery
```
- **Acceptance:** a planner moves from an unsorted corpus to a prioritised plan in one sitting.
## 9. Product Use Cases
**BUC-3 — As a team, I want the importance and categorisation of our requirements captured durably and shareably, so that other people and tools can plan from it without re-deriving it.**
```gherkin
Scenario: BUC-3 — Durable, shareable signal
Given requirements that have been weighed and categorised
When someone or something else needs to plan from them
Then they can read what matters and why without asking the original author
```
- **Acceptance:** a second party — person or tool — can pick up the work and plan from it unaided.
**BUC-4 — As a team with a specialised body of documents, I can capture the attributes that make them actionable (importance, status, category), so that I can manage that work the way my domain requires.**
```gherkin
Scenario: BUC-4 — Manage a domain's work on its own terms
Given documents whose usefulness depends on domain-specific attributes
When the team records and works with those attributes
Then they can run their workflow with the distinctions it depends on
```
- **Acceptance:** the team can capture and act on the distinctions their domain requires — success is them choosing to manage the work this way.
**BUC-5 — As someone consuming a large corpus, I can find the items that matter to my current purpose, so that I act on the right things instead of wading through everything.**
```gherkin
Scenario: BUC-5 — Find what matters
Given a large body of items
When the consumer looks for the important ones for their task
Then they can locate them quickly
```
- **Acceptance:** a person narrows a large corpus to the relevant, important subset for their task.
---
## 2. Solution Proposal
**The solution is to build it into rfc-app.** Give every collection a small, declared **field schema** (in its `.collection.yaml`) so it can carry structured metadata — priority, tags, and any custom fields the deployment defines. Store each entry's values in a **clean sidecar** file so the document body stays pure prose. rfc-app then **renders those fields as forms, filters the catalog by them (faceted, with counts), and lets authorized users tag in single and bulk gestures** committed straight to git; downstream tools read the values from the sidecars directly. It is one generic mechanism — tags and priority are just *fields* — not per-type special-casing and not a bespoke "release" entity.
**Why a software solution (and not a manual one).** A non-build alternative — operators maintaining priorities/tags in a shared spreadsheet — was considered and rejected: it leaves the corpus unfilterable in-tool (PP-2), keeps documents and the side-sheet out of sync, produces no durable git-readable signal for downstream tools (PP-5/PP-6), and does nothing for the adoption outcome (§1.6, PP-7). The value only lands if the structure lives with the content.
**Solution-specific scope.** *Out:* release ordering, ship status, roadmap emission, the `specification` release-planning surface — all downstream, reading sidecars from git. In-app management of field definitions (edit `.collection.yaml` in git for v1); corpus-wide tag rename/merge/delete; sub-document grain; a whole-corpus export endpoint. *Future (recorded, not v1):* a **bdd coverage surface** — a `verifies`-style **`ref` field type** plus a read-derived view mapping features to the spec sections they exercise (harvested from the superseded per-type-surfaces draft); deferred pending §9 Q4. This solution **amends the binding `SPEC.md` §22.4a contract** (§7 SLICE-0).
*(The Product and Engineering sections below — §§37 — elaborate this build. They would be replaced by an operational plan if the chosen solution were non-software.)*
---
## 3. Product Personas
rfc-app's user types — each an embodiment of one or more Business Roles (§1.3). The Product Use Cases (§4) are about these personas.
| Product persona | In rfc-app | Maps to business role(s) |
| --- | --- | --- |
| Collection Owner | scope-role Owner; declares the collection's `fields:` schema (edits `.collection.yaml`) | Standards owner |
| Contributor | scope-role contributor; sets metadata (single + bulk), proposes/curates entries | Requirements author; Release planner |
| Reader | viewer; browses and filters the catalog | Reader |
| Downstream consumer | an external system reading sidecars + `.collection.yaml` from git | Requirements consumer |
## 4. Product Use Cases
```gherkin
Scenario: PUC-1 — Set priority/tags on a scenario (realizes BUC-1)
Given I am a contributor viewing a scenario whose collection defines priority and tags
Scenario: PUC-1 — Set priority/tags on a scenario (realizes BUC-1, BUC-4)
Given I am a Contributor viewing a scenario whose collection defines priority and tags
When I choose P0 in the priority control and add the tag "checkout"
Then the metadata panel reflects P0 and the checkout tag
And the change is committed directly to the scenario's sidecar
@@ -114,58 +176,64 @@ Scenario: PUC-2 — Bulk tag/untag from the catalog (realizes BUC-2)
Then every selected scenario shows P1
And the bulk change is one commit
Scenario: PUC-3 — Filter the catalog by facet (realizes BUC-1/BUC-3)
Scenario: PUC-3 — Filter the catalog by facet (realizes BUC-5, BUC-1)
Given the left pane shows faceted filters generated from the collection schema
When I check Priority P0 and tag "checkout"
Then the catalog shows only scenarios matching both
And each facet value shows its result count
Scenario: PUC-4 — A collection declares its fields (product-only)
Given a collection owner edits .collection.yaml to add a priority enum field
Scenario: PUC-4 — A Collection Owner declares fields (realizes BUC-4)
Given a Collection Owner edits .collection.yaml to add a priority enum field
When the collection is re-ingested
Then the priority filter and the priority form control appear automatically
Scenario: PUC-5 — Migrate a collection to clean docs (product-only)
Scenario: PUC-5 — Migrate a collection to clean docs (product-only; enables BUC-3)
Given a collection whose docs still carry top-of-doc frontmatter
When the operator runs the frontmatter→sidecar migration
Then each doc body becomes pure prose and a sidecar holds its metadata
And rfc-app reads the collection identically before and after
Scenario: PUC-6 — A malformed entry is visibly fixable (realizes BUC-3)
Given a stored entry whose metadata fails its collection's schema
When the catalog renders
Then the entry still loads (read never hard-fails)
And it is flagged "malformed metadata" so a Contributor can fix it
```
## 10. UX Layout
## 5. UX Layout
### 10.1 Screen: Catalog (left pane) (serves PUC-3)
### 5.1 Screen: Catalog (left pane) (serves PUC-3, PUC-6)
- **Purpose:** browse and filter a collection's entries.
- **Layout (top → bottom):**
- **Search:** full-text box (existing).
- **Faceted filter groups** (one per schema field + state): each is a collapsible group showing per-value **result counts** and multi-select checkboxes; `tags`-type fields include a "filter values…" search box to stay usable at 30+ values. (Chosen layout: faceted groups with counts — validated in brainstorming over flat chips.)
- **States:** happy: facets with counts · empty: "no entries match these filters" with a clear-filters action · loading: skeleton facets · error: "couldn't load facets" with retry.
- **Layout (top → bottom):** full-text search (existing); **faceted filter groups** (one per schema field + state): each a collapsible group with per-value **result counts** and multi-select checkboxes; `tags`-type fields include a "filter values…" search box to stay usable at 30+ values.
- **States:** happy: facets with counts · empty: "no entries match" + clear-filters · loading: skeleton facets · error: retry · **malformed:** entries failing their schema carry a fixable marker (parallel to §22.4c `unreviewed`) and are filterable.
### 10.2 Screen: Scenario detail — metadata panel (serves PUC-1)
### 5.2 Screen: Scenario detail — metadata panel (serves PUC-1)
- **Purpose:** view/edit one entry's metadata.
- **Layout:** a panel rendering one control per schema field — `enum` → single-select; `tags` → removable chips + add-tag input (with existing AI suggest where applicable); `text` → text input. The document body renders below as pure prose; metadata never appears inline in the body.
- **States:** read (role without edit) shows values, no controls · edit (authorized) shows controls · saving: inline spinner · error: field-level validation message (e.g. "P5 is not an allowed priority").
- **Layout:** one control per schema field — `enum` → single-select; `tags` → removable chips + add-tag input (with existing AI suggest); `text` → text input. The body renders below as pure prose; metadata never appears inline.
- **States:** read (no edit role) shows values · edit (authorized) shows controls · saving: spinner · error: field-level validation message.
### 10.3 Screen: Catalog — bulk action bar (serves PUC-2)
### 5.3 Screen: Catalog — bulk action bar (serves PUC-2)
- **Purpose:** apply a field value to many entries at once.
- **Layout:** selecting ≥1 row reveals a sticky action bar: "*N* selected · Set priority ▾ · Add tag ▾ · Remove tag ▾ · Clear". Each action targets one field; applying commits once.
- **States:** none selected: bar hidden · applying: bar shows progress · partial failure: toast naming entries that failed validation, others applied.
- **Layout:** selecting ≥1 row reveals a sticky bar: "*N* selected · Set priority ▾ · Add tag ▾ · Remove tag ▾ · Clear". Applying commits once.
- **States:** none selected: hidden · applying: progress · partial failure: toast naming entries that failed validation, others applied.
## 11. Technical Design
## 6. Technical Design
### 11.1 Invariants
### 6.1 Invariants
- **INV-1:** The sidecar (`<slug>.meta.yaml`) is the source of truth for entry metadata; `cached_rfcs` is a derived index, fully rebuildable from git.
- **INV-2:** A document body (`.md`) never contains rfc-app metadata once migrated; metadata lives only in the sidecar.
- **INV-3:** Reading a collection never hard-fails on bad metadata — an invalid value against the schema surfaces as a warning and the entry still loads.
- **INV-4:** Metadata writes are authorized by scope-role (contributor+ on the collection); content-body edits keep their existing PR-review path.
- **INV-5:** A collection with no `fields:` block behaves exactly as today (free-form `tags` only) — the feature is additive and opt-in per collection.
- **INV-6:** Dual-read holds throughout: parser reads the sidecar if present, else legacy top-of-doc frontmatter, with identical resulting in-memory records.
- **INV-3:** Reading a collection never hard-fails on bad metadata — an invalid value surfaces as a warning, the entry still loads, and the catalog flags it (§5.1).
- **INV-4:** Metadata writes are authorized by scope-role (contributor+ on the collection) and validated at the write boundary; content-body edits keep their existing PR-review path.
- **INV-5:** A collection with no `fields:` block behaves exactly as today (free-form `tags` only). The §22.13 generated **default collection is `document`** with no fields → **N=1 deployments see zero change**.
- **INV-6:** Dual-read: parser reads the sidecar if present, else legacy top-of-doc frontmatter, with identical resulting in-memory records.
- **INV-7:** Unknown / forward-compat keys in a sidecar **ride along untouched** — never dropped on read or rewrite, never reported as malformed.
- **INV-8:** **Engine unchanged** (§22.4a) — additive and read-mostly; never forks the content write path, the propose→branch→PR→graduate lifecycle, threads/flags/chat, or the storage model. Metadata edits reuse the existing `edit-meta` git write-through.
### 11.2 High-level architecture
### 6.2 High-level architecture
```mermaid
flowchart LR
@@ -174,31 +242,33 @@ flowchart LR
MD[slug.md<br/>prose body]
SC[slug.meta.yaml<br/>values]
end
CY --> ING[ingest / parser<br/>validate vs schema]
CY --> ING[ingest / parser<br/>lenient, type-agnostic]
MD --> ING
SC --> ING
ING --> DB[(cached_rfcs<br/>values + facet counts)]
ING --> VAL[metadata_schema.validate<br/>advisory at read]
VAL --> DB[(cached_rfcs<br/>values + facet counts + malformed)]
DB --> API[API: schema · list+filter · facets · edit]
API --> FILT[left-pane faceted filters]
API --> PANEL[detail metadata panel]
API --> BULK[bulk select bar]
PANEL -->|direct commit| SC
BULK -->|1 commit| SC
PANEL -->|validate + direct commit| SC
BULK -->|validate + 1 commit| SC
SC -.read from git.-> CONS[downstream consumers]
```
- **ingest/parser** owns reading `.collection.yaml` schema + sidecars (or legacy frontmatter), validating values, and rebuilding `cached_rfcs`; must never treat the DB as authoritative.
- **API** — owns serving the schema, filtered lists with facet counts, and metadata edits; must never write metadata anywhere but the sidecar in git.
- **ingest/parser** — reads `.collection.yaml` schema + sidecars (or legacy frontmatter), stays lenient/type-agnostic (INV-7); rebuilds `cached_rfcs`; never authoritative.
- **`metadata_schema.validate(values, fields) → [problems]`** — the one place that knows a collection's required/forbidden fields and each field's shape (modeled on `registry.py`). Advisory at ingest (warn + malformed flag, INV-3); enforced at the write boundary (INV-4).
- **API** — serves the schema, filtered lists with facet counts + malformed flag, and metadata edits; never writes metadata anywhere but the sidecar.
### 11.3 Data model & ownership
### 6.3 Data model & ownership
| Entity | Owned by | Key fields | System of record |
| --- | --- | --- | --- |
| Collection field schema | collection owner | `fields: {name → {type, values?, label}}` in `.collection.yaml` | git |
| Entry metadata values | contributor | sidecar `<slug>.meta.yaml`: lifecycle (`slug,title,state,owners,…`) + schema fields (`priority,tags,…`) | git (sidecar) |
| Derived index | ingest | per-entry values + facet aggregations | `cached_rfcs` (SQLite, derived) |
| Collection field schema | Collection Owner | `fields: {name → {type, values?, label}}` in `.collection.yaml` | git |
| Entry metadata values | Contributor | sidecar `<slug>.meta.yaml`: lifecycle + schema fields + forward-compat keys (INV-7) | git (sidecar) |
| Derived index | ingest | per-entry values + facet aggregations + `malformed` flag | `cached_rfcs` (SQLite, derived) |
**Field types (v1):** `enum` (single-select; controlled by required `values:`), `tags` (multi-value; free-form unless `values:` given), `text` (free string). Unknown types are ignored with a warning (forward-compat).
**Field types (v1):** `enum` (single-select; controlled by required `values:`), `tags` (multi-value; free-form unless `values:` given), `text` (free string). **Future:** `ref` (a typed cross-entry link — basis for the deferred bdd `verifies`/coverage surface; §2, §9 Q4). Unknown types ignored with a warning.
**Sidecar example:**
```yaml
@@ -211,14 +281,14 @@ tags: [dashboard, analytics]
owner: hasan
```
### 11.4 Interfaces & contracts
### 6.4 Interfaces & contracts
- **`GET /api/projects/<p>/collections/<c>`** — out: collection incl. `fields` schema. Errors: 404.
- **`GET /api/projects/<p>/collections/<c>/rfcs`** — in: filter params (`?priority=P0&tags=checkout&state=active`, repeatable for multi-value/OR-within-field, AND across fields) · out: entries with metadata values **+ `facets: {field → {value → count}}`**. Errors: 400 on unknown field.
- **`POST /api/projects/<p>/collections/<c>/rfcs/<slug>/meta`** — in: `{field: value, …}` · out: updated values · effect: validate vs schema → write sidecar → commit directly → re-ingest entry. Errors: 403 (role), 422 (invalid value).
- **`POST /api/projects/<p>/collections/<c>/meta/bulk`** — in: `{slugs: […], op: set|add|remove, field, value}` · out: `{applied: […], rejected: [{slug, reason}]}` · effect: validate → write N sidecars → **one** commit → re-ingest. Errors: 403, 422.
- **`GET /collections/<c>`** — out: collection incl. `fields` schema.
- **`GET /collections/<c>/rfcs`** — in: filter params (`?priority=P0&tags=checkout&state=active`; OR within a field, AND across fields; `?malformed=true`) · out: entries with values + per-entry `malformed` + `facets: {field → {value → count}}`. Errors: 400 unknown field.
- **`POST /rfcs/<slug>/meta`** — in: `{field: value}` · effect: validate → write sidecar → direct commit → re-ingest. Errors: 403, 422.
- **`POST /collections/<c>/meta/bulk`** — in: `{slugs, op: set|add|remove, field, value}` · out: `{applied, rejected}` · effect: validate → write N sidecars → one commit → re-ingest. Errors: 403, 422.
### 11.5 PerProduct-Use-Case design
### 6.5 PerProduct-Use-Case design
#### PUC-2 — Bulk tag/untag
@@ -227,121 +297,143 @@ sequenceDiagram
actor U as Contributor
participant C as Catalog UI
participant A as API
participant V as metadata_schema
participant G as Git
participant D as cached_rfcs
U->>C: select rows, "Set priority P1"
C->>A: POST /meta/bulk {slugs, set, priority, P1}
A->>A: authz (scope-role) + validate vs schema
A->>A: authz (scope-role)
A->>V: validate values vs schema
A->>G: write N sidecars, 1 commit
A->>D: re-ingest affected entries
A-->>C: {applied, rejected}
C-->>U: rows show P1; toast on any rejected
```
- **Implementation:** reuse the `edit-meta` git write-through, extended to (a) target the sidecar rather than frontmatter and (b) batch N files into one commit. Honors INV-1/INV-4.
- **Implementation:** reuse the `edit-meta` git write-through, extended to target the sidecar and batch N files into one commit. Honors INV-1/INV-4/INV-8.
#### PUC-5 — Migration
- **Implementation:** a tool (framework CLI verb / `tools/` script) walks a collection, and for each entry with legacy frontmatter, writes `<slug>.meta.yaml` from the frontmatter and rewrites `<slug>.md` to the body only — one commit per collection, idempotent. Dual-read (INV-6) means this can run anytime; lazy migration converts stragglers on their first metadata edit.
- **Implementation:** a tool walks a collection; for each entry with legacy frontmatter it writes `<slug>.meta.yaml` and rewrites `<slug>.md` to the body only — one commit per collection, idempotent, preserving unknown keys (INV-7). Dual-read (INV-6) lets it run anytime; lazy migration converts stragglers on first metadata edit.
### 11.6 Non-functional requirements & cross-cutting concerns
### 6.6 Non-functional requirements & cross-cutting concerns
- **Security & privacy:** metadata edits gated by `auth.effective_scope_role` (contributor+ on the collection); no secrets in sidecars; git history records authorship.
- **Performance & scale:** facet counts computed from the derived DB; must stay responsive at ~1.2k entries with dozens of tag values (indexed value columns / aggregation query).
- **Availability & resilience:** bad metadata never blocks read (INV-3); a failed re-ingest leaves git authoritative and is recoverable by full rebuild.
- **Observability:** log each metadata commit (actor, field, entry count); warn-log schema validation failures encountered on ingest.
- **Accessibility:** facet groups and form controls keyboard-navigable; checkboxes labelled with value + count.
- **Security & privacy:** edits gated by `auth.effective_scope_role`; no secrets in sidecars; git history records authorship.
- **Performance & scale:** facet counts from the derived DB; responsive at ~1.2k entries with dozens of tag values.
- **Availability & resilience:** bad metadata never blocks read (INV-3); failed re-ingest leaves git authoritative, recoverable by rebuild.
- **Observability:** log each metadata commit; warn-log + count schema-validation failures on ingest.
- **Accessibility:** facet groups and form controls keyboard-navigable; checkboxes labelled value + count.
### 11.7 Key decisions & alternatives considered
### 6.7 Key decisions & alternatives considered
| Decision | Chosen | Alternatives | Why |
| --- | --- | --- | --- |
| Release modeling | Metadata only; releases downstream | First-class release entity in rfc-app; release-typed tags with behavior | Operator pulled ordering/ship-status out of rfc-app; metadata is the only needed primitive |
| Tag system shape | One generic typed-field system (Approach A) | Releases first-class + simple tags; namespaced facets | Tags/priority/custom are all just fields; one mechanism |
| Metadata storage | Sidecar per entry | Top-of-doc frontmatter (today); end-of-doc block; collection index file; DB-only | Clean docs + git-visible to consumers + locality per scenario |
| Left-pane filtering | Faceted groups with counts | Flat facet chips | Scales to the ~1.2k-scenario, many-tag ecomm corpus |
| Edit governance | Direct commit for authorized roles (bulk = 1 commit) | PR per change | Bulk planning is impractical via PR-per-toggle |
| Mgmt UI | Deferred; edit `.collection.yaml` in git | In-app field/vocab management in v1 | Smallest coherent v1 |
| Solution type | Build into rfc-app | Manual (shared spreadsheet) | Manual leaves corpus unfilterable, out of sync, no git-readable signal (§2) |
| Release modeling | Metadata only; releases downstream | First-class release entity | Operator pulled ordering/ship-status out of rfc-app |
| Tag system shape | One generic typed-field system | Releases first-class + simple tags; namespaced facets | Tags/priority/custom are all just fields |
| Schema model (D9) | Pure collection-config | Type-driven hard-coded schemas (per-type-surfaces draft) | Flexible, data-driven |
| Metadata storage | Sidecar per entry | Frontmatter; end-of-doc; index file; DB-only | Clean docs + git-visible + locality |
| Left-pane filtering | Faceted groups with counts | Flat facet chips | Scales to ~1.2k-scenario, many-tag corpus |
| Edit governance | Direct commit for authorized roles | PR per change | Bulk planning impractical via PR-per-toggle |
| bdd coverage (D10) | Future per-type surface over a `ref` field | Build now; drop | Valuable but not v1; needs Q4 |
### 11.8 Testing strategy
### 6.8 Testing strategy
Unit tests for: schema parsing (`fields:` block, all types, missing block); sidecar read/write round-trip; dual-read equivalence (frontmatter vs sidecar produce identical records); schema validation (reject bad enum, accept free-form tag); facet aggregation; bulk op (set/add/remove, single commit, partial-rejection). Two-tier local-Docker→PPE for the API + git write-through. "Tested" = the PUC acceptance scenarios pass plus the migration is proven idempotent and reversible-on-read.
Unit: schema parsing (all types, missing block); sidecar round-trip incl. unknown-key preservation (INV-7); dual-read equivalence (INV-6); validation; malformed-flag; facet aggregation; bulk op (single commit, partial-rejection). Two-tier local-Docker→PPE for API + git write-through. "Tested" = PUC acceptance scenarios pass + migration proven idempotent and reversible-on-read.
### 11.9 Failure modes, rollback & flags
### 6.9 Failure modes, rollback & flags
- **Failure mode:** invalid value committed out-of-band → on ingest, warn + load entry with the raw value flagged (INV-3); not surfaced as a filter facet count error.
- **Failure mode:** re-ingest fails after commit → git is authoritative; full rebuild recovers.
- **Migration rollback:** dual-read means an un-migrated or partially-migrated corpus still works; the migration commit is revertible.
- **Feature flag:** the feature is inherently opt-in per collection (INV-5) — no global flag needed; absent a `fields:` block, behavior is unchanged.
- **Invalid value committed out-of-band** → ingest warns + loads with the value flagged malformed (INV-3).
- **Re-ingest fails after commit** → git authoritative; full rebuild recovers.
- **Migration rollback:** dual-read keeps an un-/partly-migrated corpus working; the migration commit is revertible.
- **Feature flag:** inherently opt-in per collection (INV-5) — no global flag.
## 12. Delivery Plan
## 7. Delivery Plan
### 12.1 Approach / strategy
### 7.1 Approach / strategy
Build the storage/compat foundation first (sidecars + dual-read + migration) so nothing breaks, then the schema, then read (filtering), then write (single, bulk). Each slice is shippable and non-breaking.
Amend the binding contract first, then build storage/compat, then schema, then read, then write. Each build slice is shippable and non-breaking.
### 12.2 Slicing plan
**Execution convention.** Each slice is taken as **its own coding session**`writing-plans → executing-plans → verify → ship/deploy → merge + version bump` — in dependency order, with the slice's implementation plan written **just-in-time** at the start of that session, not up front (later slices' plans depend on the code earlier slices land). `brainstorming` ran once to produce this spec and recurs only if a slice proves the spec wrong. A slice's **Definition of Done** (§7.2) is the signal to advance the `Next /goal:` cursor to the next slice. SLICE-0 is doc-only (no implementation plan).
#### SLICE-1 — Sidecar storage + dual-read + migration → completes PUC-5
### 7.2 Slicing plan
#### SLICE-0 — Amend `SPEC.md` §22.4a (contract) → unblocks the rest
- **Depends on:**
- **DoD:** parser reads sidecar-if-present else legacy frontmatter (INV-6); migration tool splits frontmatter→sidecar idempotently; existing collections load byte-identically; tests green.
- **Definition of done:** §22.4a reframed — item 1 (entry schema) is **collection-configured sidecar fields**, not type-driven frontmatter; item 3 (type surfaces) deferred to a future design (bdd coverage recorded); per-type-surfaces draft marked superseded; §20 changelog. *Doc-only; no code.*
#### SLICE-2Collection field schema + validation → completes PUC-4
#### SLICE-1Sidecar storage + dual-read + migration → completes PUC-5, PUC-6
- **Depends on:** SLICE-0
- **DoD:** parser reads sidecar-else-legacy (INV-6), preserves unknown keys (INV-7); migration tool idempotent; existing collections load byte-identically; malformed flag derived; tests green.
#### SLICE-2 — Collection field schema + central validation → completes PUC-4
- **Depends on:** SLICE-1
- **DoD:** `.collection.yaml fields:` parsed (`enum`/`tags`/`text`); values validated on read (warn) and on write (reject); schema served via the collection API; no-`fields:` collections unchanged (INV-5).
- **DoD:** `.collection.yaml fields:` parsed; `metadata_schema.validate` advisory at read / enforced at write; schema served via the collection API; no-`fields:` collections unchanged (INV-5).
#### SLICE-3 — Faceted left-pane filtering (read) → completes PUC-3
- **Depends on:** SLICE-2
- **DoD:** list endpoint returns facet counts + honors filter params; left pane renders faceted groups with counts and tag-value search; filters compose (AND across fields).
- **DoD:** list endpoint returns facet counts + honors filter params (incl. `malformed`); left pane renders faceted groups with counts + tag-value search; filters compose.
#### SLICE-4 — Single-entry metadata edit → completes PUC-1
- **Depends on:** SLICE-2
- **DoD:** detail metadata panel renders schema controls; `POST …/meta` validates, direct-commits the sidecar, re-ingests; authorized by scope-role (INV-4); lazy-migrates a legacy entry on first edit.
- **DoD:** detail panel renders schema controls; `POST …/meta` validates, direct-commits, re-ingests; scope-role gated (INV-4); lazy-migrates a legacy entry on first edit.
- **Carried from SLICE-1 (deferred there):** make the **write paths**
sidecar-aware — every site that today does `entry.parse(<slug>.md)` and
serializes back into the `.md` must read/write metadata via the sidecar so a
migrated (body-only) entry doesn't crash or re-grow frontmatter. The known
sites: graduation + claim + `_read_meta_entry` (`api_graduation.py`),
`mark_entry_reviewed` (`bot.py`), body-edit / accept-change wrappers
(`api_branches.py` `_wrap_body`/`_extract_body`), and the PR-replay wrappers
(`api_prs.py`). Only once these are sidecar-aware should the **operator
trigger** for `metadata.migrate_collection` (the Owner-gated migrate endpoint)
ship.
#### SLICE-5 — Bulk tag/untag → completes PUC-2
- **Depends on:** SLICE-3, SLICE-4
- **DoD:** multi-select + bulk action bar; `POST …/meta/bulk` applies set/add/remove as one commit; partial-rejection reported.
- **DoD:** multi-select + bulk bar; `POST …/meta/bulk` applies set/add/remove as one commit; partial-rejection reported.
### 12.3 Rollout / launch plan
### 7.3 Rollout / launch plan
Pre-v1, single production: ship slices in order to the RFC deployment; each minor-version bump carries §20 changelog + upgrade steps. The opt-in-per-collection nature (INV-5) means a deployment adopts it only when it declares a `fields:` block and (optionally) runs the migration.
Pre-v1, single production: ship slices in order; each minor bump carries §20 changelog + upgrade steps. Opt-in per collection (INV-5): a deployment adopts it only by declaring a `fields:` block and (optionally) running the migration.
### 12.4 Risks & mitigations
### 7.4 Risks & mitigations
| Risk | L/I | Mitigation |
| --- | --- | --- |
| Frontmatter→sidecar migration corrupts content | L/H | Dual-read; idempotent, revertible migration; body-byte-identity test |
| Doubling file count (sidecars) clutters corpus | M/L | Docs stay clean; sidecars are small/co-located; acceptable for one-file-per-scenario corpora |
| Direct-commit metadata edits bypass review | M/M | Scope-role gate (INV-4); content-body edits still PR'd; full git audit trail |
| Overlap/conflict with §22 S6 "type modules" | M/M | Position as §23, generalizing S6's per-type frontmatter; reconcile at the S6 SPEC merge |
| Amending binding §22.4a destabilises a shipped contract | M/M | SLICE-0 doc-only, reviewed; dual-read keeps runtime non-breaking; supersede note preserves rationale |
| Frontmatter→sidecar migration corrupts content | L/H | Dual-read; idempotent, revertible migration; body-byte-identity + unknown-key tests |
| Doubling file count (sidecars) clutters corpus | M/L | Docs stay clean; sidecars small/co-located |
| Direct-commit metadata edits bypass review | M/M | Scope-role gate (INV-4); content-body edits still PR'd; git audit trail |
| Facet aggregation slow at scale | L/M | Compute from indexed derived DB; measure at ~1.2k entries |
## 13. Traceability matrix
## 8. Traceability matrix
| Business UC | Product UC | Slice | Tests |
| --- | --- | --- | --- |
| BUC-4 | PUC-5 | SLICE-1 | `test_dual_read_equiv`, `test_migration_idempotent` |
| — (product-only) | PUC-4 | SLICE-2 | `test_schema_parse`, `test_validation` |
| BUC-1/BUC-3 | PUC-3 | SLICE-3 | `test_facet_counts`, `test_filter_compose` |
| BUC-1 | PUC-1 | SLICE-4 | `test_single_meta_commit`, `test_authz` |
| BUC-2 | PUC-2 | SLICE-5 | `test_bulk_one_commit`, `test_partial_reject` |
| BUC-3 | (consumer reads git) | — | `test_sidecar_schema_stable` |
| Pain | Business UC | Product UC | Slice | Tests |
| --- | --- | --- | --- | --- |
| — (contract) | — | — | SLICE-0 | (doc review) |
| PP-5 | BUC-3 | PUC-5, PUC-6 | SLICE-1 | `test_dual_read_equiv`, `test_migration_idempotent`, `test_unknown_keys_preserved` |
| PP-7 | BUC-4 | PUC-4 | SLICE-2 | `test_schema_parse`, `test_validate` |
| PP-2 | BUC-5, BUC-1 | PUC-3 | SLICE-3 | `test_facet_counts`, `test_filter_compose` |
| PP-1, PP-3 | BUC-1, BUC-4 | PUC-1 | SLICE-4 | `test_single_meta_commit`, `test_authz` |
| PP-4 | BUC-2 | PUC-2 | SLICE-5 | `test_bulk_one_commit`, `test_partial_reject` |
| PP-6 | BUC-3 | (consumer reads git) | — | `test_sidecar_schema_stable` |
## 14. Open Questions & Decisions log
## 9. Open Questions & Decisions log
**Open**
| # | Question | Owner | Blocks |
| --- | --- | --- | --- |
| Q1 | Do downstream consumers read sidecars from git, via rfc-app API, or both? (leaning git) | Ben | nothing v1 |
| Q2 | Should a `multi-enum` type (multi-select controlled) ship in v1 or later? | Ben | SLICE-2 scope |
| Q3 | Exact §23 placement / reconciliation with §22 S6 type-modules | Ben | SPEC merge |
| Q1 | Do downstream consumers read sidecars from git, via API, or both? (leaning git) | Ben | nothing v1 |
| Q2 | Ship `multi-enum` (multi-select controlled) in v1 or later? | Ben | SLICE-2 scope |
| Q3 | Exact §22.4a amendment wording + the future-surfaces home | Ben | SLICE-0 |
| Q4 | bdd coverage: `ref` field grammar + a coverage view honoring §22's no-cross-collection-join rule as hyperlinks | Ben | future surface |
**Resolved**
| # | Decision | Resolution | Date |
| --- | --- | --- | --- |
| D1 | Release behaviors (ordering, ship status, roadmap emit) | Out of rfc-app; downstream | 2026-06-06 |
| D1 | Release behaviors | Out of rfc-app; downstream | 2026-06-06 |
| D2 | Tag system shape | Approach A — one generic typed-field system | 2026-06-06 |
| D3 | Metadata grain | Per entry (corpus already one file per scenario) | 2026-06-06 |
| D4 | Schema location | `.collection.yaml` `fields:` block | 2026-06-06 |
@@ -349,11 +441,16 @@ Pre-v1, single production: ship slices in order to the RFC deployment; each mino
| D6 | Left-pane filtering | Faceted groups with counts | 2026-06-06 |
| D7 | Edit governance | Direct commit for authorized roles; bulk = 1 commit | 2026-06-06 |
| D8 | Management scope | Deferred; edit `.collection.yaml` in git for v1 | 2026-06-06 |
| D9 | Schema model | Pure collection-config; not type-driven | 2026-06-06 |
| D10 | bdd coverage | Future per-type surface over a `ref` field; not v1 | 2026-06-06 |
| D11 | per-type-surfaces draft | Superseded; §22.4a to be amended (SLICE-0) | 2026-06-06 |
## 15. Glossary & References
## 10. Glossary & References
- **Sidecar**`<slug>.meta.yaml`, the per-entry metadata file that is the source of truth; keeps the `.md` body pure prose.
- **Field schema** — the `fields:` block in `.collection.yaml` declaring a collection's typed metadata fields.
- **Facet** — a schema field surfaced as a left-pane filter group with per-value counts.
- **Downstream consumer** — an external tool (e.g. a release planner) that reads corpus metadata from git; rfc-app does not model releases.
- **References:** retired BDD Release Planner (`wiggleverse-ecomm-bdd-release-planner-app`); §22 three-tier design (`docs/design/2026-06-05-three-tier-projects-collections.md`); SPEC §7.1 (left-pane filter), §9.5 (edit-meta), §20 (versioning), §22 Part B / S3 (scope-role).
- **Malformed metadata** — stored values that fail their collection's schema; flagged in the catalog, never a hard read failure (INV-3).
- **Downstream consumer** — an external tool that reads corpus metadata from git; rfc-app does not model releases.
- **References:** retired BDD Release Planner; superseded per-type-surfaces draft (`2026-06-06-per-type-surfaces.md`); §22 three-tier design; `SPEC.md` §7.1 (left-pane filter), §9.5 (edit-meta), §20 (versioning), §22.4a (per-type contract — to be amended), §22 Part B / S3 (scope-role).
```
@@ -1,5 +1,19 @@
# Draft spec — §22.4a per-type surfaces (the last S6 item)
> # ⛔ SUPERSEDED (2026-06-06)
>
> This draft is **superseded by**
> [`2026-06-06-configurable-collection-metadata.md`](./2026-06-06-configurable-collection-metadata.md),
> which reframes §22.4a item 1 as **collection-configured** metadata in
> **sidecars** (not type-driven frontmatter) and defers item 3's surfaces.
> Harvested into the successor: the validation seam (A.1), the malformed-metadata
> catalog flag (A.5), unknown-fields-ride-along (C.1), the engine-unchanged rule
> (§0), and the N=1 `document` backcompat anchor (A.2). The **bdd coverage**
> capability (`feature`/`verifies` → coverage view, Part B.2) is preserved there
> as a *future* per-type surface over a generic `ref` field. The binding
> `SPEC.md` §22.4a contract is to be amended by the successor's SLICE-0. Kept for
> historical rationale; do not build from this document.
> **Status:** discovery/spec pass — *not yet sliced into a shipped release.*
> Author session: 0083 (2026-06-06). This document is the spec pass the §22 S6
> remainder called for: it specifies **§22.4a item 1** (the per-type entry
@@ -0,0 +1,121 @@
# Deployed-environment E2E harness (PPE)
**Date:** 2026-06-07 · **Version:** v0.52.0 · **Status:** implemented
## Why
The §9 deployment pipeline is `localhost + E2E → PPE + E2E → prod`. The
middle stage — running the Playwright E2E suite against a *deployed*
pre-prod host (`https://rfc-ppe.wiggleverse.org`) — was unreachable
because the suite (`e2e/metadata.spec.js`, SLICE-3/4/5 of the
configurable-collection-metadata work) was bound to three scaffolds that
exist only in the local Tier-1 docker stack:
1. **A faceted `bdd` collection**, seeded into a throwaway Gitea by
`testing/seed-gitea.sh`.
2. **A granted-owner identity** (`e2e-owner@example.test`), injected
directly into SQLite by the docker-compose `backend-seed` step.
3. **Mailpit**, the SMTP sink the OTC sign-in reads the one-time code
from.
PPE has none of these: it runs against the real `git.wiggleverse.org`
(shared with prod), has no direct DB access, and has no mail sink. This
note records how each coupling is replaced so the *same* spec runs green
against both localhost and PPE.
## The three seams
### 1. Auth — a gated test-login endpoint (the framework change)
A new backend route, `POST /auth/test/login`, replaces both the Mailpit
OTC dance *and* the SQLite owner injection with one gesture: it mints an
authenticated **owner** session for a single pre-configured identity.
It is the framework's only auth bypass, so it is **fail-closed** and must
never function in production:
- **Off by default.** It returns `404` unless **both**
`E2E_TEST_AUTH_SECRET` and `E2E_TEST_AUTH_EMAIL` are set. A production
deployment sets neither, so the route is invisible and inert.
- **Secret-gated.** The caller must present `E2E_TEST_AUTH_SECRET` in the
`X-Test-Auth-Secret` header, compared in constant time. A wrong/absent
secret returns `404` (it does not advertise the route's existence).
- **Single identity.** It will only mint the one configured
`E2E_TEST_AUTH_EMAIL` (case-insensitive); any other address is `403`.
So an enabled PPE exposes exactly one throwaway owner, with the secret
as the trust boundary.
- **Loud at startup.** When enabled, the app logs a `WARNING` at boot, so
an accidental prod enablement is visible rather than silent.
On success it provision-or-links the row (reusing `otc.provision_or_link_user`),
forces it to `role='owner', permission_state='granted'` (the deployed
equivalent of the Tier-1 owner-seed), and stores the session exactly like
the OTC verify path.
**Why an endpoint rather than alternatives.** Reading the OTC code from
the VM's journald (the email adapter logs the envelope to stdout when
SMTP is unconfigured) would couple the test harness to `gcloud` SSH at
runtime — slow, brittle, and operator-cred-bound. Running Mailpit on the
VM and exposing its API publicly is more infra and its own exposure
surface. A default-off, secret-gated endpoint is the portable engineering
seam: it works for *any* deployed environment, needs no SSH, and the
secrets rule (§6.3) is honored — the secret is a Secret Manager ref
injected as VM env, never a literal.
The hard-secrets caveat: the E2E runner presents the secret by resolving
it from Secret Manager at runtime (command substitution), never echoing
it.
### 2. Content — a dedicated PPE registry + content repo
PPE shares the prod Gitea org (`wiggleverse`) and, until now, prod's
registry (`rfc-registry`) and default project (`ohm`). Seeding a faceted
test collection into that shared registry would surface it on **prod**.
So PPE gets its **own**, prod-untouching fixtures:
- `wiggleverse/rfc-registry-ppe` — PPE's project registry. Prod keeps
`rfc-registry`, so prod is never affected.
- `wiggleverse/rfc-app-ppe-content` — one project `ohm` (document) with a
default collection entry plus a faceted `bdd` named collection
(`priority` enum + `tags`) and three entries, mirroring the Tier-1
seed. The E2E path `/p/ohm/c/bdd` therefore resolves identically on
both environments.
PPE is pointed at it with `overlay set rfc-app-ppe
REGISTRY_REPO=rfc-registry-ppe`. The startup reconciler sweep loads the
content into `cached_rfcs` (incl. `meta_json` for facets) on the next
deploy — no webhook needed for the initial load. The seed is scripted in
`testing/seed-ppe.sh` (idempotent; `RESEED=1` restores entry values for a
re-run). Repo *creation* is a one-time operator gesture (the
`write:repository` Keychain token cannot create org repos; create the two
empty repos in the Gitea UI or re-scope the PAT).
### 3. Parameterization — one spec, two environments
- `e2e/playwright.config.js` already honors `BASE_URL`
(default `http://localhost:8080`); PPE sets
`BASE_URL=https://rfc-ppe.wiggleverse.org`.
- `e2e/lib/auth.js` branches on `E2E_TEST_AUTH_SECRET`: set → use
`/auth/test/login`; unset → the original Mailpit OTC path. `OWNER_EMAIL`
reads `E2E_OWNER_EMAIL` (PPE points it at `E2E_TEST_AUTH_EMAIL`) or the
Tier-1 default. The spec itself is unchanged, so the localhost Tier-1
path keeps working.
## PPE version
The harness *requires* the test-login endpoint to exist in the deployed
build, so PPE must run a framework version that contains it — **v0.52.0**,
not v0.51.1. PPE is pinned ahead of prod via its own
`ben/ohm-rfc/.rfc-app-version.ppe` (prod stays on `.rfc-app-version`),
realizing the "PPE stages newer versions first" note the
`deployment.ppe.toml` always anticipated.
## Known limitations
- **Re-runnability.** SLICE-4/5 mutate the seeded entries (commit
sidecars). A clean run needs seed-state preconditions; re-run after
`RESEED=1` + a cache refresh (next reconciler sweep or a redeploy).
Unlike Tier-1's `make e2e-fresh`, PPE has no per-run teardown.
- **smoke.spec.js** stays Tier-1-only (anonymous OTC smoke through
Mailpit); only `metadata.spec.js` runs against PPE.
@@ -0,0 +1,108 @@
# SLICE-1 plan — sidecar storage + dual-read + migration + malformed flag
Just-in-time implementation plan for **SLICE-1** of
[Configurable Collection Metadata](../2026-06-06-configurable-collection-metadata.md)
(§7.2). Authored at the start of the SLICE-1 coding session (session 0084,
2026-06-07), against the code SLICE-0 (v0.46.2) landed.
## Scope (and non-scope)
**In:** the storage/compat layer only — sidecar files become the source of
truth for entry metadata, with a dual-read parser, an idempotent
frontmatter→sidecar migration tool, and a derived `metadata_malformed` flag.
**Out (later slices):** the `.collection.yaml` `fields:` schema + validation
(SLICE-2), faceted filtering (SLICE-3), the edit/bulk UIs (SLICE-4/5). SLICE-1
maps sidecar values onto the **existing** typed `cached_rfcs` columns; it does
not add per-field schema columns or facet aggregation.
## Invariants honored
- **INV-6 dual-read:** parser reads the sidecar if present, else legacy
top-of-doc frontmatter, with identical resulting in-memory records.
- **INV-7 unknown keys ride along:** preserved through parse→serialize and
through the migration (never dropped).
- **INV-2:** a migrated `.md` body contains no metadata.
- **INV-1:** `cached_rfcs` stays a derived, rebuildable index; the sidecar in
git is the source of truth.
- **INV-3:** bad metadata never hard-fails a read — the entry still loads and
the catalog flags it (`metadata_malformed`).
- **INV-5 / byte-identity:** a collection with no sidecars behaves exactly as
today (legacy frontmatter path); existing entries load identically.
## Components
1. **`app/entry.py` — unknown-key preservation (INV-7).** Add
`extra: dict[str, Any]` to `Entry`. `parse()` collects frontmatter keys
outside the known set into `extra`; `serialize()` re-emits them after the
known keys. Makes frontmatter round-trips lossless.
2. **`app/metadata.py` — new module (sidecar concerns).**
- `SIDECAR_SUFFIX = ".meta.yaml"`; `sidecar_name(slug)`,
`is_sidecar(name)`, `slug_of_sidecar(name)`.
- `metadata_dict(entry) -> dict` — the full metadata mapping (known
emit-rules + `extra`), shared by the sidecar writer and the frontmatter
serializer.
- `sidecar_yaml(entry) -> str` — canonical YAML for a sidecar from
`metadata_dict`.
- `strip_frontmatter(md_text) -> str` — body-only (drops a leading
`---…---` block if present; whole text otherwise).
- `parse_sidecar(text) -> tuple[dict, bool]` — lenient: `(values, malformed)`;
non-mapping / YAML error → `({}, True)`.
- `read_entry(md_text, sidecar_text|None) -> tuple[Entry, bool]` — dual-read:
sidecar present → metadata from sidecar values, body from
`strip_frontmatter(md_text)`, `malformed` from `parse_sidecar`; absent →
`entry.parse(md_text)`, `malformed=False`.
3. **`app/gitea.py``change_files(...)` batch commit.** `POST
/repos/{owner}/{repo}/contents` (Gitea ChangeFiles) with a `files[]` array
of `{operation, path, content(b64), sha?}` — one commit for N files. Backs
the migration's "one commit per collection".
4. **`metadata.migrate_collection(gitea, org, repo, subfolder, actor)`.**
Lists `<subfolder>/rfcs`; for each `<slug>.md` **without** a `<slug>.meta.yaml`
sibling and **with** legacy frontmatter, batch: create the sidecar
(`metadata_dict` → YAML) + update the `.md` to body-only. One ChangeFiles
commit per collection. Idempotent (skip entries already migrated; no-op when
none remain). Returns a summary (`migrated`, `skipped`, `committed`).
> **Deferred (decided mid-slice, after code review):** the **operator
> trigger** for this tool (an Owner-gated endpoint) is held back to SLICE-4.
> The propose/graduate/mark-reviewed/edit write paths still `entry.parse` the
> `.md` directly, so migrating a corpus to body-only `.md`s before those
> paths are sidecar-aware would break them (crash / re-introduce
> frontmatter). SLICE-1 ships the tool as tested groundwork; SLICE-4 makes
> the write paths sidecar-aware (and adds lazy migration) and is where the
> trigger belongs (INV-8: engine write paths unchanged this slice).
5. **`app/cache.py` — dual-read in `_refresh_collection_corpus`.** Build a
sidecar-by-stem map from the dir listing; for each `.md`, read its sidecar
sibling (if any), `metadata.read_entry(...)`, thread `malformed` into
`_upsert_cached_rfc(metadata_malformed=…)`.
6. **`backend/migrations/033_metadata_malformed.sql`** — additive
`ALTER TABLE cached_rfcs ADD COLUMN metadata_malformed INTEGER NOT NULL DEFAULT 0`.
7. **`app/api.py` — surface the flag.** Add `metadata_malformed` (bool) to the
two catalog list dicts and `get_rfc`/`_get_rfc_for_collection`. (Frontend
badge + `?malformed=` filter are SLICE-3.)
## Tests (TDD — write first)
- `test_metadata.py` (unit, pure): dual-read equivalence (sidecar vs legacy →
identical Entry); unknown-key preservation through parse→serialize and
through `metadata_dict`; `strip_frontmatter` (with/without frontmatter);
`parse_sidecar` malformed cases.
- `test_metadata_migration.py` (integration, FakeGitea): migrate a collection
→ sidecars written + `.md` bodies stripped + one commit; **idempotent**
(second run is a no-op); unknown keys preserved in the sidecar.
- extend the cache/propose vertical: a collection with a sidecar mirrors from
the sidecar; a malformed sidecar sets `metadata_malformed` and still loads
the entry (INV-3); a no-sidecar collection is byte-identical to today.
## Release
Minor bump **0.46.2 → 0.47.0** (new functionality: sidecar storage + migration
tool; non-breaking — additive migration 033, dual-read keeps legacy corpora
working, opt-in). §20 CHANGELOG + upgrade-steps: migration 033 auto-applies;
running the migration tool per collection is optional (**MAY**).
@@ -0,0 +1,109 @@
# Implementation plan — SLICE-2: collection field schema + central validation
**Slice:** SLICE-2 of
[`docs/design/2026-06-06-configurable-collection-metadata.md`](../2026-06-06-configurable-collection-metadata.md)
§7.2. **Session:** OHM-0085. **Branch:** `worktree-metadata-slice2-schema`.
## Goal / Definition of Done (from the design)
- `.collection.yaml` `fields:` block is parsed and stored.
- `metadata_schema.validate` exists — **advisory at read**, the enforcement
point **at write** (write endpoints land in SLICE-4/5; this slice supplies and
read-wires the function).
- The schema is served via the collection API (`GET …/collections/{id}`).
- A collection with **no `fields:`** behaves exactly as today (INV-5) — the
§22.13 default `document` collection sees zero change.
## Design decisions (this slice)
- **Field-def shape** (design §6.3): `fields:` is an ordered mapping
`{name → {type, values?, label?}}`. `type ∈ {enum, tags, text}` (v1).
Order is preserved for facet display (SLICE-3).
- `enum` — single scalar; requires a non-empty `values:` list.
- `tags` — list; `values:` optional (controlled when present, free-form
otherwise).
- `text` — free string scalar.
- **`ref` and `multi-enum` are out of v1** (design §2 future; Q2 leans "later").
Unknown field types are **ignored with a warning** (design §6.3), not fatal.
- **Lenient schema parsing.** A malformed `fields:` block or an individual bad
field def is skipped with a warning, never raised — a typo in one field must
not nuke the collection mirror (INV-3 spirit). Structural manifest errors
(`type`, `visibility`) keep raising `RegistryError` as before.
- **No DB migration.** The normalized `fields` schema rides in the existing
`collections.config_json` column, exactly like `enabled_models`.
- **`values` source for validation.** An entry's full metadata mapping
(`metadata.metadata_dict(entry)` = `to_frontmatter_dict`) — `tags` come from
the known `Entry.tags`; custom fields (e.g. `priority`) come from
`Entry.extra`. Undeclared keys are forward-compat (INV-7) and never flagged.
## Tasks
### 1. `app/metadata_schema.py` (new) — the one place that knows field shapes
- `VALID_FIELD_TYPES = {"enum", "tags", "text"}`.
- `parse_fields(raw) -> dict[str, dict]` — lenient/normalizing. Returns an
ordered mapping of `{name: {"type", "values"?, "label"?}}`. Skips: non-mapping
block; non-mapping field def; unknown/missing type; `enum` without a non-empty
`values:` list. Logs a warning per skip. Pure (no I/O).
- `@dataclass Problem(field, code, message)` + `as_dict()`.
- `validate(values: dict, fields: dict[str, dict]) -> list[Problem]` — for each
**declared** field, validate the entry's value (absent is OK):
- `enum`: scalar ∈ `values:` else `not-in-values`; a list/non-scalar →
`wrong-type`.
- `tags`: must be a list (`wrong-type` otherwise); if controlled, each member
`values:` else `not-in-values`.
- `text`: must be a scalar string (`wrong-type` otherwise).
Undeclared keys ignored (INV-7). Never raises.
- **Tests** `tests/test_metadata_schema.py`: parse (each type, missing block,
enum-without-values skipped, unknown type skipped, order preserved); validate
(happy, enum bad value, tags uncontrolled-ok, tags controlled-bad, text wrong
type, undeclared key ignored, empty schema → no problems).
### 2. `registry.py` — parse `fields:` into the collection config
- In `parse_collection_manifest`, after the existing keys: if `raw.get("fields")`
present, `cfg["fields"] = metadata_schema.parse_fields(raw["fields"])` (only
set when non-empty). Flows into `CollectionEntry.config`
`config_json` via the existing `json.dumps(ce.config)` in
`_upsert_named_collection`.
- **Default collection** (`apply_registry`) currently writes the `collections`
row **without** `config_json`. A default collection's `fields:` would live in
`projects.yaml`? No — the design says `fields:` is a `.collection.yaml` block.
The default collection has no `.collection.yaml`. **Decision:** default-
collection field schemas are out of this slice's happy path (the N=1 default is
`document` with no fields, INV-5). Leave `apply_registry` untouched; only
named collections (with a `.collection.yaml`) carry `fields:`. Documented as a
known limitation (a deployment wanting fields on its primary corpus declares a
named collection — consistent with the design's opt-in story).
- **Tests** extend `tests/test_collection_registry.py`: a manifest with a
`fields:` block round-trips into `get_collection(...)["fields"]`; a manifest
with a bad field def still upserts (lenient).
### 3. `collections.py` — unpack `fields` on read + serve via API
- Add `_fields_from_config(config_json) -> dict | None` (mirror
`_enabled_models_from_config`).
- `get_collection` sets `out["fields"] = _fields_from_config(config_json)`
(alongside `enabled_models`). `api_collections.get_col` then serves it with no
change. `list_collections` left as-is (facets are SLICE-3).
- **Tests** extend `tests/test_collection_helpers.py`: `get_collection` exposes
`fields`; a no-fields collection → `fields is None` (INV-5).
### 4. `cache.py` — advisory validation at ingest (INV-3)
- In `_refresh_collection_corpus`, fetch the collection's `fields` schema once
(`collections.get_collection(collection_id)`); for each entry, if a schema is
present, `problems = metadata_schema.validate(metadata.metadata_dict(entry),
fields)` and OR any problems into `metadata_malformed` (warn-log a summary).
No schema → behavior identical to today (INV-5).
- **Tests** `tests/test_metadata_cache.py` (extend): an entry violating an enum
field ingests with `metadata_malformed = 1`; a conforming entry → `0`; a
collection with no schema → `0` regardless of extra keys.
### 5. Verify · version · ship
- `pytest` full backend suite green (575 baseline + new).
- Bump `VERSION` + `frontend/package.json`**0.48.0**; CHANGELOG minor entry
(§20) — non-breaking, opt-in, N=1 unchanged; note write-enforcement lands with
SLICE-4/5.
- Commit (cite design §7.2 SLICE-2), PR on Gitea `origin`, merge to `main`.
## Invariants honored
INV-3 (read never hard-fails — advisory malformed), INV-5 (no-`fields:`
unchanged), INV-7 (undeclared keys ride along, never flagged), INV-8 (additive,
read-mostly; no write-path fork — write enforcement is a later slice).
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,802 @@
# SLICE-5 — Bulk tag/untag Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let an authorized user apply one metadata field change (set / add-tag / remove-tag) to many catalog entries at once, committed as a single git commit, with per-entry partial-rejection reported — completing PUC-2 of the [Configurable Collection Metadata](../../design/2026-06-06-configurable-collection-metadata.md) design (§6.4, §6.5, D7).
**Architecture:** A new backend endpoint `POST /api/projects/{pid}/collections/{cid}/meta/bulk` reuses the SLICE-4 sidecar-aware git helpers (`metadata.read_entry_from_git` / `apply_values` / `write_entry_files`) and the `bot.commit_entry_files` multi-file primitive: read each entry, apply the op, validate at the write boundary (INV-4), collect file ops for the passing entries, and commit them all in **one** `change_files` call (D7: bulk = 1 commit). Entries that fail validation or are missing are reported in `rejected`, others in `applied`. The frontend adds row multi-select + a sticky bulk action bar in `Catalog.jsx` (faceted, contributor+ only), driven by the collection `fields:` schema, calling a new `bulkEntryMeta` API helper and toasting partial rejections.
**Tech Stack:** Python / FastAPI / pytest (backend); React / Vitest (frontend); Gitea ChangeFiles for the one-commit write.
---
## File Structure
- `backend/app/api_metadata.py`**modify**: add the `bulk_meta` route + a `BulkMetaBody` model alongside the existing single-edit and migrate routes. Add a small pure helper `_apply_op(entry, op, field, value)` (or inline) computing the new field value for set/add/remove.
- `backend/tests/test_metadata_bulk_endpoint.py`**create**: endpoint tests (one commit, partial reject, authz, op validation), reusing the `test_propose_vertical` fake-Gitea harness like `test_metadata_edit_endpoint.py`.
- `frontend/src/api.js`**modify**: add `bulkEntryMeta(projectId, collectionId, { slugs, op, field, value })`.
- `frontend/src/components/BulkActionBar.jsx`**create**: the sticky bar (N selected · Set <enum> ▾ · Add/Remove tag · Clear), driven by `fields`.
- `frontend/src/components/BulkActionBar.test.jsx`**create**: render + interaction unit tests.
- `frontend/src/components/Catalog.jsx`**modify**: per-row selection checkboxes (faceted + contributor only), selection state, render `BulkActionBar`, apply handler + toast, re-fetch on success.
- `CHANGELOG.md`, `VERSION`, `frontend/package.json`**modify**: v0.51.0 release entry + version bump.
---
## Backend op semantics (reference for all backend tasks)
Endpoint: `POST /api/projects/{pid}/collections/{cid}/meta/bulk`
Body: `{ "slugs": ["a","b"], "op": "set"|"add"|"remove", "field": "priority", "value": <any> }`
Response (200): `{ "ok": true, "applied": ["a"], "rejected": [{"slug":"b","reason":"..."}], "committed": true }`
Rules:
- **Authz:** `auth.can_contribute_in_collection(viewer, collection_id)` — else 403 (anonymous → 403). Same gate as the single-entry edit.
- **Collection-in-project:** mismatch → 404. No `fields:` → 422. Empty `slugs` → 422. `op` not in `{set,add,remove}` → 422. `field` not in schema → 422.
- **`set`** valid for any field; new value = `value` as given.
- **`add` / `remove`** valid only for `tags`-type fields (else 422); they read the entry's current list and append / drop the single `value`.
- **Per entry:** read via `read_entry_from_git`; missing `.md` → reject `{slug, reason:"not found"}`. Apply op → `apply_values`. Validate `metadata_schema.validate(metadata_dict(new_entry), fields)`; problems → reject `{slug, reason:"<problem messages>"}`. If the resulting metadata equals the old, count as applied but emit **no** file op (avoid redundant writes). Otherwise collect `write_entry_files(md_path, new_entry, state)` ops.
- **Commit:** concatenate all passing entries' file ops into one list; if non-empty, one `bot.commit_entry_files(..., message="Bulk <op> <field>: N entries")`; `committed=true`. If empty (all rejected, or all no-op) → no commit, `committed=false`.
- **Re-ingest:** `cache.refresh_meta_repo` once when committed.
---
## Task 1: Bulk endpoint — happy path, one commit
**Files:**
- Modify: `backend/app/api_metadata.py`
- Test: `backend/tests/test_metadata_bulk_endpoint.py`
- [ ] **Step 1: Write the failing test**
Create `backend/tests/test_metadata_bulk_endpoint.py`:
```python
"""SLICE-5 — bulk metadata edit endpoint (PUC-2, §6.4/§6.5).
Through the real API: contributor+ gating (INV-4), set/add/remove ops,
validation at the write boundary, one commit for N sidecars (D7), and
partial-rejection reporting. Reuses the fake-Gitea harness.
"""
from __future__ import annotations
import asyncio
import json
import yaml
from fastapi.testclient import TestClient
from app import cache, db, gitea as gitea_mod
from app.config import load_config
from test_propose_vertical import ( # noqa: F401 (fixtures)
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
PID = "default"
CID = "default"
BASE = f"/api/projects/{PID}/collections/{CID}"
def _refresh():
cfg = load_config()
asyncio.run(cache.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg)))
def _set_fields(schema):
db.conn().execute(
"UPDATE collections SET config_json = ? WHERE id = 'default'",
(json.dumps({"fields": schema}),))
def _seed_legacy(fake, slug, *, state="active", **front):
fm = {"slug": slug, "title": slug.title(), "state": state, **front}
body = yaml.safe_dump(fm, sort_keys=False).strip()
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {
"content": f"---\n{body}\n---\n\nBody.\n", "sha": slug}
def _login_owner(client):
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
def _sidecar(fake, slug):
return yaml.safe_load(
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.meta.yaml")]["content"])
def test_bulk_set_applies_to_all_and_one_commit(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1", "P2"]}})
_seed_legacy(fake, "a", priority="P2")
_seed_legacy(fake, "b", priority="P1")
_refresh()
_login_owner(client)
commits_before = fake.change_files_calls
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "b"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 200, r.text
body = r.json()
assert set(body["applied"]) == {"a", "b"}
assert body["rejected"] == []
assert body["committed"] is True
# exactly one ChangeFiles commit covered both entries (D7)
assert fake.change_files_calls - commits_before == 1
assert _sidecar(fake, "a")["priority"] == "P0"
assert _sidecar(fake, "b")["priority"] == "P0"
```
> NOTE: the test asserts `fake.change_files_calls` — a counter on the fake Gitea. If the fake does not already expose one, Step 3 adds it (see Task 1 Step 3a).
- [ ] **Step 2: Run test to verify it fails**
Run: `cd backend && python -m pytest tests/test_metadata_bulk_endpoint.py::test_bulk_set_applies_to_all_and_one_commit -v`
Expected: FAIL (404 — route not defined, or AttributeError on `change_files_calls`).
- [ ] **Step 3a: Add a commit counter to the fake Gitea (only if missing)**
Inspect the fake's `change_files` in `backend/tests/test_propose_vertical.py`. If it has no call counter, add one. Find the fake class's `change_files` method and increment a counter:
```python
async def change_files(self, owner, repo, *, files, message, branch,
author_name=None, author_email=None):
self.change_files_calls = getattr(self, "change_files_calls", 0) + 1
# ... existing body unchanged ...
```
If a counter already exists under another name, use that name in the test instead and skip this step.
- [ ] **Step 3b: Implement the bulk route**
In `backend/app/api_metadata.py`, add the request model near `MetaEditBody`:
```python
class BulkMetaBody(BaseModel):
slugs: list[str]
op: str
field: str
value: Any = None
```
Add the pure op helper above `make_router` (module level):
```python
def _apply_op(entry: Any, op: str, field: str, value: Any) -> Any:
"""Return the new value for `field` after applying `op` to `entry`.
set → `value`; add/remove operate on the entry's current tags-list value
for `field` (the route restricts add/remove to tags-type fields).
"""
if op == "set":
return value
current = metadata_mod.metadata_dict(entry).get(field) or []
if not isinstance(current, list):
current = [current]
if op == "add":
return current if value in current else [*current, value]
if op == "remove":
return [x for x in current if x != value]
return value # unreachable; op validated by the route
```
Add the route inside `make_router`, after the single-entry `edit_meta` route:
```python
@router.post("/api/projects/{project_id}/collections/{collection_id}/meta/bulk")
async def bulk_meta(
project_id: str, collection_id: str,
body: BulkMetaBody, request: Request,
) -> dict[str, Any]:
viewer = auth.current_user(request)
if collections_mod.project_of_collection(collection_id) != project_id:
raise HTTPException(404, "Collection not in project")
if not auth.can_contribute_in_collection(viewer, collection_id):
raise HTTPException(403, "Contributor access required to edit metadata")
col = collections_mod.get_collection(collection_id)
fields = (col or {}).get("fields") or {}
if not fields:
raise HTTPException(422, "Collection declares no editable fields")
if not body.slugs:
raise HTTPException(422, "Provide at least one entry")
if body.op not in ("set", "add", "remove"):
raise HTTPException(422, f"Unknown op: {body.op}")
if body.field not in fields:
raise HTTPException(422, f"Unknown field: {body.field}")
if body.op in ("add", "remove") and fields[body.field].get("type") != "tags":
raise HTTPException(422, f"op {body.op} requires a tags field")
org, repo = _content_repo()
applied: list[str] = []
rejected: list[dict[str, str]] = []
all_ops: list[dict[str, Any]] = []
for slug in body.slugs:
md_path = _md_path(collection_id, slug)
st = await metadata_mod.read_entry_from_git(gitea, org, repo, md_path)
if st is None:
rejected.append({"slug": slug, "reason": "not found"})
continue
new_value = _apply_op(st.entry, body.op, body.field, body.value)
new_entry = metadata_mod.apply_values(st.entry, {body.field: new_value})
problems = metadata_schema.validate(
metadata_mod.metadata_dict(new_entry), fields)
if problems:
rejected.append({"slug": slug,
"reason": "; ".join(p.message for p in problems)})
continue
applied.append(slug)
if metadata_mod.metadata_dict(new_entry) != metadata_mod.metadata_dict(st.entry):
all_ops.extend(metadata_mod.write_entry_files(md_path, new_entry, st))
committed = False
if all_ops:
n = len(applied)
msg = f"Bulk {body.op} {body.field}: {n} entr{'y' if n == 1 else 'ies'}"
try:
await bot.commit_entry_files(
viewer.as_actor(), org=org, repo=repo, files=all_ops,
message=msg, branch="main")
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
committed = True
await cache.refresh_meta_repo(config, gitea)
return {"ok": True, "applied": applied,
"rejected": rejected, "committed": committed}
```
> `Problem` exposes `.message` (see `metadata_schema.Problem`/`as_dict`); confirm the attribute name when implementing and adjust the join if it differs.
- [ ] **Step 4: Run test to verify it passes**
Run: `cd backend && python -m pytest tests/test_metadata_bulk_endpoint.py::test_bulk_set_applies_to_all_and_one_commit -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/app/api_metadata.py backend/tests/test_metadata_bulk_endpoint.py backend/tests/test_propose_vertical.py
git commit -m "feat(slice5): POST .../meta/bulk one-commit bulk metadata edit (§22.4a PUC-2)"
```
---
## Task 2: Bulk add/remove tags
**Files:**
- Test: `backend/tests/test_metadata_bulk_endpoint.py`
- [ ] **Step 1: Write the failing tests**
Append:
```python
def test_bulk_add_tag(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x"])
_seed_legacy(fake, "b", tags=["x", "y"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "b"], "op": "add",
"field": "tags", "value": "y"})
assert r.status_code == 200, r.text
assert set(r.json()["applied"]) == {"a", "b"}
assert _sidecar(fake, "a")["tags"] == ["x", "y"]
# idempotent: "b" already had y → unchanged, still no duplicate
assert _sidecar(fake, "b")["tags"] == ["x", "y"] or "b" not in [
# b may be a no-op write skip; the cache still shows ["x","y"]
]
def test_bulk_remove_tag(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"tags": {"type": "tags"}})
_seed_legacy(fake, "a", tags=["x", "y"])
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "remove",
"field": "tags", "value": "x"})
assert r.status_code == 200, r.text
assert r.json()["applied"] == ["a"]
assert _sidecar(fake, "a")["tags"] == ["y"]
def test_bulk_add_remove_requires_tags_field(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "add",
"field": "priority", "value": "z"})
assert r.status_code == 422, r.text
```
> Simplify the `test_bulk_add_tag` "b" assertion to whatever the no-op-skip semantics produce — verify the sidecar for "b" still reads `["x", "y"]` if a sidecar was written, and don't assert a sidecar exists for "b" if it was a pure no-op. Adjust after observing the first run.
- [ ] **Step 2: Run to verify they fail/pass appropriately**
Run: `cd backend && python -m pytest tests/test_metadata_bulk_endpoint.py -v -k "add or remove"`
Expected: pass for the logic implemented in Task 1; fix the `test_bulk_add_tag` "b" assertion to match the no-op-skip behavior actually observed.
- [ ] **Step 3: Finalize assertions**
Edit the `test_bulk_add_tag` "b" branch to a concrete assertion based on the observed behavior (sidecar absent for a pure no-op, or present and equal to `["x","y"]`).
- [ ] **Step 4: Run to verify all pass**
Run: `cd backend && python -m pytest tests/test_metadata_bulk_endpoint.py -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add backend/tests/test_metadata_bulk_endpoint.py
git commit -m "test(slice5): bulk add/remove tag ops + tags-field guard"
```
---
## Task 3: Partial rejection, authz, op validation
**Files:**
- Test: `backend/tests/test_metadata_bulk_endpoint.py`
- [ ] **Step 1: Write the failing tests**
Append:
```python
def test_bulk_partial_reject_invalid_value(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_seed_legacy(fake, "missing-source", priority="P1") # has source
_refresh()
_login_owner(client)
# invalid value rejects ALL (set value is global), so use a per-entry
# rejection: one slug exists, one does not.
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a", "ghost"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 200, r.text
body = r.json()
assert body["applied"] == ["a"]
assert body["rejected"] == [{"slug": "ghost", "reason": "not found"}]
assert body["committed"] is True
assert _sidecar(fake, "a")["priority"] == "P0"
def test_bulk_invalid_value_rejects_all_no_commit(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0", "P1"]}})
_seed_legacy(fake, "a", priority="P1")
_refresh()
_login_owner(client)
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "priority", "value": "ZZZ"})
assert r.status_code == 200, r.text
assert r.json()["applied"] == []
assert len(r.json()["rejected"]) == 1
assert r.json()["committed"] is False
assert ("wiggleverse", "meta", "main", "rfcs/a.meta.yaml") not in fake.files
def test_bulk_forbidden_for_anonymous(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
r = client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "priority", "value": "P0"})
assert r.status_code == 403, r.text
def test_bulk_unknown_field_and_op(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_set_fields({"priority": {"type": "enum", "values": ["P0"]}})
_seed_legacy(fake, "a", priority="P0")
_refresh()
_login_owner(client)
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "set",
"field": "nope", "value": "P0"}).status_code == 422
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": ["a"], "op": "frobnicate",
"field": "priority", "value": "P0"}).status_code == 422
assert client.post(f"{BASE}/meta/bulk",
json={"slugs": [], "op": "set",
"field": "priority", "value": "P0"}).status_code == 422
```
- [ ] **Step 2: Run to verify pass**
Run: `cd backend && python -m pytest tests/test_metadata_bulk_endpoint.py -v`
Expected: PASS (logic from Task 1 covers these). Fix any assertion mismatches.
- [ ] **Step 3: Commit**
```bash
git add backend/tests/test_metadata_bulk_endpoint.py
git commit -m "test(slice5): bulk partial-reject, authz, validation guards"
```
---
## Task 4: Frontend API helper
**Files:**
- Modify: `frontend/src/api.js`
- [ ] **Step 1: Add the helper**
After `saveEntryMeta` in `frontend/src/api.js`:
```javascript
// §22.4a SLICE-5 (PUC-2): apply one field op (set | add | remove) to many
// entries at once — one commit server-side; returns { applied, rejected }.
export async function bulkEntryMeta(projectId, collectionId, { slugs, op, field, value }) {
const res = await fetch(
`/api/projects/${projectId}/collections/${collectionId}/meta/bulk`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ slugs, op, field, value }),
},
)
return jsonOrThrow(res)
}
```
- [ ] **Step 2: Commit**
```bash
git add frontend/src/api.js
git commit -m "feat(slice5): bulkEntryMeta API helper"
```
---
## Task 5: BulkActionBar component
**Files:**
- Create: `frontend/src/components/BulkActionBar.jsx`
- Test: `frontend/src/components/BulkActionBar.test.jsx`
- [ ] **Step 1: Write the failing test**
Create `frontend/src/components/BulkActionBar.test.jsx`:
```javascript
import { render, screen, fireEvent } from '@testing-library/react'
import { describe, it, expect, vi } from 'vitest'
import BulkActionBar from './BulkActionBar.jsx'
const FIELDS = {
priority: { type: 'enum', values: ['P0', 'P1', 'P2'], label: 'Priority' },
tags: { type: 'tags', label: 'Tags' },
}
describe('BulkActionBar', () => {
it('shows the selected count', () => {
render(<BulkActionBar fields={FIELDS} count={3} onApply={() => {}} onClear={() => {}} />)
expect(screen.getByText(/3 selected/i)).toBeInTheDocument()
})
it('applies a set op when an enum value is chosen', () => {
const onApply = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={onApply} onClear={() => {}} />)
fireEvent.change(screen.getByLabelText(/set priority/i), { target: { value: 'P0' } })
expect(onApply).toHaveBeenCalledWith({ op: 'set', field: 'priority', value: 'P0' })
})
it('applies an add-tag op', () => {
const onApply = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={onApply} onClear={() => {}} />)
fireEvent.change(screen.getByPlaceholderText(/tag…/i), { target: { value: 'checkout' } })
fireEvent.click(screen.getByRole('button', { name: /add tag/i }))
expect(onApply).toHaveBeenCalledWith({ op: 'add', field: 'tags', value: 'checkout' })
})
it('calls onClear', () => {
const onClear = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={() => {}} onClear={onClear} />)
fireEvent.click(screen.getByRole('button', { name: /clear/i }))
expect(onClear).toHaveBeenCalled()
})
})
```
- [ ] **Step 2: Run to verify it fails**
Run: `cd frontend && npx vitest run src/components/BulkActionBar.test.jsx`
Expected: FAIL (module not found).
- [ ] **Step 3: Implement the component**
Create `frontend/src/components/BulkActionBar.jsx`:
```javascript
import { useState } from 'react'
// §22.4a SLICE-5 (PUC-2, UX §5.3): the sticky bulk action bar shown when ≥1
// catalog row is selected. Driven by the collection `fields:` schema — one
// "Set <field>" control per enum field, and an Add/Remove tag control per
// tags field. Each gesture calls onApply({ op, field, value }); the parent
// (Catalog) sends one bulk request and re-fetches.
const labelFor = (name, def) =>
def?.label || name.charAt(0).toUpperCase() + name.slice(1)
export default function BulkActionBar({ fields, count, onApply, onClear }) {
const [tagValue, setTagValue] = useState('')
const entries = Object.entries(fields || {})
const tagField = entries.find(([, d]) => d.type === 'tags')
return (
<div className="bulk-action-bar">
<span className="bulk-count">{count} selected</span>
{entries
.filter(([, d]) => d.type === 'enum')
.map(([name, def]) => (
<label key={name} className="bulk-set">
<span>Set {labelFor(name, def)}</span>
<select
aria-label={`Set ${labelFor(name, def)}`}
value=""
onChange={e => {
if (e.target.value) onApply({ op: 'set', field: name, value: e.target.value })
}}
>
<option value="">—</option>
{(def.values || []).map(v => <option key={v} value={v}>{v}</option>)}
</select>
</label>
))}
{tagField && (
<div className="bulk-tags">
<input
placeholder="tag…"
value={tagValue}
onChange={e => setTagValue(e.target.value)}
/>
<button
type="button"
disabled={!tagValue.trim()}
onClick={() => { onApply({ op: 'add', field: tagField[0], value: tagValue.trim() }); setTagValue('') }}
>
Add tag
</button>
<button
type="button"
disabled={!tagValue.trim()}
onClick={() => { onApply({ op: 'remove', field: tagField[0], value: tagValue.trim() }); setTagValue('') }}
>
Remove tag
</button>
</div>
)}
<button type="button" className="bulk-clear" onClick={onClear}>Clear</button>
</div>
)
}
```
- [ ] **Step 4: Run to verify it passes**
Run: `cd frontend && npx vitest run src/components/BulkActionBar.test.jsx`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add frontend/src/components/BulkActionBar.jsx frontend/src/components/BulkActionBar.test.jsx
git commit -m "feat(slice5): BulkActionBar component (UX §5.3)"
```
---
## Task 6: Wire selection + bulk bar into Catalog
**Files:**
- Modify: `frontend/src/components/Catalog.jsx`
- Test: `frontend/src/components/Catalog.test.jsx`
- [ ] **Step 1: Write the failing test**
Add to `frontend/src/components/Catalog.test.jsx` a test that, with a faceted collection and a contributor viewer, selecting a row reveals the bulk bar. Inspect the existing Catalog.test.jsx mock setup first (how it mocks `../api` `listRFCs`/`getCollection`) and mirror it. Skeleton:
```javascript
it('reveals the bulk action bar when a row is selected (faceted + contributor)', async () => {
// mock getCollection → { viewer: { can_contribute: true }, fields: { priority: { type: 'enum', values: ['P0','P1'] } } }
// mock listRFCs → { items: [{ slug: 'a', title: 'A', state: 'active', tags: [] }], facets: {} }
// render Catalog within the same providers/router the other tests use
// click the row's selection checkbox
// expect screen.getByText(/1 selected/i) to be in the document
})
```
Fill the mocks to match the file's existing pattern exactly.
- [ ] **Step 2: Run to verify it fails**
Run: `cd frontend && npx vitest run src/components/Catalog.test.jsx`
Expected: FAIL (no checkbox / no bulk bar).
- [ ] **Step 3: Implement in `Catalog.jsx`**
1. Imports:
```javascript
import { bulkEntryMeta } from '../api'
import BulkActionBar from './BulkActionBar.jsx'
import { useToast } from '../context/...' // match how other components toast; see below
```
> Check how `ToastHost` is consumed elsewhere (e.g. `grep useToast frontend/src`). If there's no hook, accept an `onToast`/use `window`-level host the app already wires. If toasting is awkward here, fall back to an inline message line in the bulk bar area. Do not invent a toast system.
2. Selection state + reset on collection/filter change:
```javascript
const [selected, setSelected] = useState(() => new Set())
```
Clear it in the collection-change effect (the `[version, pid, cid]` effect) and whenever the list re-fetches: add `setSelected(new Set())` alongside `setSelections({})` in the collection effect, and clear it after a successful bulk apply.
3. Toggle helper:
```javascript
function toggleSelect(slug) {
setSelected(prev => {
const next = new Set(prev)
next.has(slug) ? next.delete(slug) : next.add(slug)
return next
})
}
```
4. Show a checkbox per row **only in faceted mode and when `canContribute`**. The row is a `<Link>`; render the checkbox as a sibling before it inside a wrapper so the checkbox click doesn't navigate (`onClick={e => e.stopPropagation()}` on the checkbox, and don't nest it in the Link):
```jsx
filtered.map(r => {
const isActive = slug === r.slug
const isSuper = r.state === 'super-draft'
const selectable = faceted && canContribute
return (
<div key={r.slug} className={`catalog-row-wrap ${selectable ? 'selectable' : ''}`}>
{selectable && (
<input
type="checkbox"
className="row-select"
aria-label={`select ${r.title}`}
checked={selected.has(r.slug)}
onChange={() => toggleSelect(r.slug)}
/>
)}
<Link
to={entryPath(pid, r.slug, cid)}
className={`catalog-row ${isActive ? 'active' : ''} ${isSuper ? 'is-super' : ''}`}
>
{/* ...existing row internals unchanged... */}
</Link>
</div>
)
})
```
> Preserve the existing row internals (`row-top`, `row-id`, malformed marker, `row-title`, `row-tags`) verbatim inside the `<Link>`.
5. Apply handler:
```javascript
async function applyBulk({ op, field, value }) {
const slugs = [...selected]
if (slugs.length === 0) return
try {
const res = await bulkEntryMeta(pid, cid, { slugs, op, field, value })
if (res.rejected?.length) {
// surface which entries failed; see toast note above
showToast?.(`${res.applied.length} updated, ${res.rejected.length} skipped`)
}
setSelected(new Set())
// re-fetch the list (re-run the facet effect): bump a local nonce or
// re-call listRFCs directly. Simplest: replicate the list fetch here.
const selObj = Object.fromEntries(
Object.entries(selections).map(([f, set]) => [f, [...set]]))
const d = await listRFCs(pid, cid, { selections: selObj, malformed: malformedOnly })
setRfcs(d.items); setFacets(d.facets || {})
} catch (e) {
showToast?.(e.message || 'Bulk update failed')
}
}
```
6. Render the bar above the list when `selected.size > 0`:
```jsx
{faceted && canContribute && selected.size > 0 && (
<BulkActionBar
fields={fields}
count={selected.size}
onApply={applyBulk}
onClear={() => setSelected(new Set())}
/>
)}
```
- [ ] **Step 4: Run to verify it passes**
Run: `cd frontend && npx vitest run src/components/Catalog.test.jsx`
Expected: PASS
- [ ] **Step 5: Add minimal styling**
Add CSS for `.bulk-action-bar` (sticky, visible bar) and `.row-select` / `.catalog-row-wrap` (flex row) to the catalog stylesheet. Find where `.catalog-row` is styled (`grep -rn "catalog-row" frontend/src`) and add the new rules in the same file.
- [ ] **Step 6: Commit**
```bash
git add frontend/src/components/Catalog.jsx frontend/src/components/Catalog.test.jsx frontend/src/styles
git commit -m "feat(slice5): catalog row multi-select + bulk action bar wired (PUC-2)"
```
---
## Task 7: Full suites green
- [ ] **Step 1: Backend**
Run: `cd backend && python -m pytest -q`
Expected: all pass (prior 640 + new bulk tests).
- [ ] **Step 2: Frontend**
Run: `cd frontend && npx vitest run`
Expected: all pass (prior 44 + new BulkActionBar + Catalog tests).
- [ ] **Step 3: Fix any regressions, then commit if anything changed.**
---
## Task 8: Version bump + changelog
**Files:**
- Modify: `VERSION`, `frontend/package.json`, `CHANGELOG.md`
- [ ] **Step 1: Bump version to 0.51.0**
Set `VERSION` to `0.51.0`; set `frontend/package.json#version` to `0.51.0` (mirror rule, SPEC §20).
- [ ] **Step 2: Add the changelog entry**
Prepend a `## 0.51.0 — 2026-06-07` minor entry to `CHANGELOG.md` describing the bulk endpoint + bulk bar, citing §22.4a SLICE-5 / PUC-2 and the design doc. No deployment upgrade steps required (additive; opt-in per collection via `fields:`, INV-5) — state that explicitly.
- [ ] **Step 3: Commit**
```bash
git add VERSION frontend/package.json CHANGELOG.md
git commit -m "release(slice5): v0.51.0 — bulk tag/untag metadata (§22.4a PUC-2 SLICE-5)"
```
---
## Self-review notes
- **Spec coverage:** DoD = "multi-select + bulk bar (Task 5/6); `POST …/meta/bulk` applies set/add/remove as one commit (Task 1/2); partial-rejection reported (Task 3)." Tests `test_bulk_one_commit` ≈ Task 1, `test_partial_reject` ≈ Task 3 (traceability §8). INV-1/4/8 honored by reusing SLICE-4 sidecar write-through. INV-5 (no `fields:` → unchanged): bulk bar gated on `faceted` + the endpoint 422s with no `fields:`.
- **Type consistency:** `bulkEntryMeta({slugs, op, field, value})` shape matches `BulkMetaBody` and `onApply({op, field, value})`. `_apply_op` is the single op interpreter.
- **Open confirmations during execution:** (a) `metadata_schema.Problem` attribute for the human message (`.message` vs `.detail`); (b) the fake Gitea commit counter name; (c) the toast mechanism in the frontend (use existing or fall back to inline). Each is called out at its task.
@@ -0,0 +1,627 @@
# Solution Design: Corpus Tree — universal directory-tree left pane
| | |
| --- | --- |
| **Author(s)** | Ben Stull |
| **Reviewers / approvers** | Ben Stull |
| **Status** | `draft` |
| **Version** | v0.2.0 |
| **Source artifacts** | BDD corpus: §§1.9/4 below (this doc) · Prototype: brainstorm mockups, session 0081.0 (`.superpowers/brainstorm/`, not committed) · Reference: current `rfc-app` §7 Catalog + §22 three-tier + `DocsLayout` flyout nav · Supersedes: — |
**Change log**
| Date | Version | Change | By |
| --- | --- | --- | --- |
| 2026-06-06 | v0.1.0 | Initial draft (discovery session ohm 0081.0) | Ben Stull |
| 2026-06-06 | v0.2.0 | Reworked to the restructured Solution Design standard (two-part front; Pain Points; Business Actors vs Product Personas; renumbered) | Ben Stull |
---
## 1. Business Context
*The business lens — solution-agnostic throughout. No mechanism is proposed until §2.*
### 1.1 Executive Summary
Organizations keep large, living bodies of documentation, and the value in them
depends on people being able to find what they need and trust that it is current
and collectively maintained. This design targets two outcomes: readers can locate
any document by where it sits in a documentation body's own organization, and an
organization can bring an existing body of documentation under collaborative,
reviewed governance without disrupting how that documentation is already
arranged. The benefit accrues to readers (faster, more confident access),
contributors (a reviewed way to improve the docs), and the organization (a
governed, trustworthy knowledge base).
### 1.2 Background
The platform already governs collaboratively-edited documents organized as a
single shallow list within a body. Real organizational documentation — handbooks,
runbooks, design libraries — is instead deeply structured, and most of it already
exists as finished, authoritative content rather than passing through a
proposal-to-acceptance flow. There is currently no way to bring such a body onto
the platform's governance without flattening its structure, which is why
structured documentation bodies stay off-platform today.
### 1.3 Business Actors / Roles
Real-world roles, independent of any product.
| Role | Responsible for (in the business) |
| --- | --- |
| Reader | Finds and reads documents to do their work; needs current, authoritative content |
| Contributor | Proposes new documents and changes to existing ones |
| Maintainer | Reviews proposed changes and decides what becomes authoritative |
| Documentation steward | Owns an existing body of documentation and decides to bring it under collaborative governance |
### 1.4 Problem Statement
A reader cannot navigate a documentation body by its own structure, and an
organization cannot place an existing structured body under collaborative
governance without reorganizing it. The governance model assumes every document
is one entry in a single flat list and passes through a proposal flow — neither
of which holds for an established, deeply-organized documentation body whose
documents already exist as authoritative content.
### 1.5 Pain Points
| # | Pain | Who feels it | Cost / frequency today |
| --- | --- | --- | --- |
| PP-1 | In a large body presented as one flat list, a reader can't tell where a document sits or browse by area | Reader | Every lookup in a sizeable body; slow, error-prone, gives up |
| PP-2 | An existing structured body can't be brought under governance without rearranging its documents into a flat scheme | Documentation steward | Blocks adoption entirely for any living body — a rearrange is a non-starter |
| PP-3 | Contributors to an existing body have no governed, reviewed way to propose changes tied to where each document lives | Contributor, Maintainer | Changes happen outside review, or not at all; no shared record of why |
| PP-4 | When the documentation is briefly unreachable, the reader is left with nothing to orient by | Reader | Intermittent; erodes trust in the body as a dependable source |
### 1.6 Targeted Business Outcomes
| Outcome | Success metric | Baseline → Target | Guardrail (must not regress) | How / when measured |
| --- | --- | --- | --- | --- |
| Structured documentation bodies adopt the platform's governance | bodies of documentation hosted | 0 → ≥1 | existing bodies' usability unaffected | inspection at first onboarding |
| Bringing a body under governance is a decision, not a project | preparatory rearrangement / setup required | a flatten/rework → none | — | inspection at onboarding (PP-2) |
| Readers reliably reach documents in a large body | reader reaches the intended document | not feasible for structured bodies → routine | flat-body access unchanged | manual walkthrough (PP-1) |
> These are threshold/qualitative targets, not funnel metrics: this enables a new
> class of hosted documentation rather than tuning a conversion surface.
### 1.7 Scope (business)
- **In scope:** readers navigating a body by its own organization; contributors
proposing additions and changes anywhere in a body under review; stewards
bringing an existing body under governance without rearranging it.
- **Out of scope:** authoring tools beyond proposing/reviewing text documents;
governance policy changes (who may review/accept) — unchanged.
- **Non-goals:** becoming a general file store for non-document assets — the value
is governed *documents*, not arbitrary binaries.
### 1.8 Assumptions · Constraints · Dependencies
- **Assumptions:** bodies brought under governance are predominantly prose
documents (risk: a body that is mostly non-document assets gains little).
- **Constraints:** the existing collaborative-governance model (proposal →
review → acceptance) is reused, not redefined.
- **Dependencies:** the steward can grant the platform access to the existing
documentation body.
### 1.9 Business Use Cases
Solution-agnostic; no product, no technology.
**BUC-1 — As a reader, I can find and read a document by where it sits in the body, so that I get authoritative content without knowing an internal name.**
```gherkin
Scenario: BUC-1 — A reader finds and reads a document in a structured body
Given a body of documentation organized into sections
When a reader looks for a particular document
Then they can locate it by where it sits in that organization
And read its current content
```
- **BUC-1 acceptance criteria:** the reader reaches the intended document and
reads its current authoritative content.
**BUC-1a — As a reader, when the body is briefly unavailable, I am still oriented and never hit a dead end.**
```gherkin
Scenario: BUC-1a — The documentation is temporarily unavailable
Given the documentation cannot be reached for a moment
When a reader tries to access it
Then they are still shown what was last known to exist, or told clearly how to try again
And are never left at an empty, unexplained dead end
```
**BUC-2 — As a contributor, I can propose a change to a document, so that improvements are reviewed before they become authoritative.**
```gherkin
Scenario: BUC-2 — A contributor proposes a change to a document
Given a contributor authorized to change the documentation
When they propose a change to an existing document
Then the change enters review before it can become authoritative
And the document is shown as having a change under review
```
- **BUC-2 acceptance criteria:** a reviewable proposal exists against that
document; until accepted, the authoritative content is unchanged.
**BUC-2a — As a contributor, when my proposal can't be accepted as placed, I'm told why and nothing is half-done.**
```gherkin
Scenario: BUC-2a — A proposed change cannot be accepted as placed
Given a contributor proposing a document
When the chosen placement conflicts with an existing document, or is not allowed
Then the proposal is refused with a clear, specific reason
And nothing is partially recorded
```
**BUC-3 — As a contributor, I can introduce a new document anywhere in the body, so that the body grows where the content belongs.**
```gherkin
Scenario: BUC-3 — A contributor introduces a new document anywhere in the body
Given a contributor authorized to add documentation
When they propose a new document at a place within the body
Then it enters review as a draft in that place
And on acceptance it becomes the authoritative document there
```
- **BUC-3 acceptance criteria:** a draft appears at the chosen place and is
reviewable; on acceptance it is the authoritative document there.
**BUC-4 — As a documentation steward, I can bring an existing body under collaborative governance, so that it gains review and shared maintenance without disruption.**
```gherkin
Scenario: BUC-4 — A steward brings an existing doc body under governance
Given an organization with an existing body of documentation
When that documentation is brought under collaborative governance
Then all of its existing documents are immediately readable as authoritative
And none of them had to be rearranged to make that possible
```
- **BUC-4 acceptance criteria:** every existing document is readable and treated
as authoritative, with no rearrangement and no preparatory data entry.
---
## 2. Solution Proposal
Build software: make the platform's left-pane navigation a **universal
directory-tree** that mirrors a content repository's own git directory structure,
and address every document by its **path** rather than a flat slug — so a
documentation body is navigated by its real structure while keeping the existing
propose → review → accept governance on every file. The flat list becomes the
degenerate "one folder" case of the tree; a deep body is the general case. A new
`CorpusTree` left-pane component replaces the flat §7 Catalog as the *universal*
idiom (one navigation model for every project type), with **two display modes**
*structure* (the directory tree) and *flat* (today's ranked list, engaged on
search or a non-path sort) — so the four affordances the flat list provides
(search, lifecycle state, sort, the contribution surface) all survive.
An existing body is onboarded as an ordinary registry project pointed at its
repository, with **no content migration and no state backfill**: a document
present with no lifecycle record is treated as authoritative ("no record =
active").
**Why this approach, over the alternatives:**
- *Do nothing / keep flat + rearrange repos* — rejected: PP-2 makes a rearrange a
non-starter for a living body.
- *A read-only documentation viewer* — rejected: delivers PP-1 but not PP-3
(no governed contribution).
- *A second, separate tree view beside the flat catalog* — rejected: forks the
navigation idiom and doubles the surface; the tree generalizes the flat list
rather than sitting beside it.
**Solution-specific scope / non-goals:**
- *In:* path-addressed documents; a directory-tree read surface; the dual-mode
pane; lifecycle by path; propose/edit/review at any path; zero-migration
onboarding; legacy slug→path URL compatibility.
- *Out:* lazy per-folder tree loading (a scale follow-on; v1 serves the full
document tree from cache); a rich renderer for non-document files (listed but
inert); changes to graduation / integer-ID assignment.
- *Non-goals:* merging folders with collections (kept distinct — INV-3); a new
project *type* (existing types render via the tree — D5).
---
## 3. Product Personas
| Product persona | In rfc-app | Maps to business role(s) |
| --- | --- | --- |
| Reader | Browses a project's corpus and opens documents | Reader |
| Contributor | Holds collection `can_contribute`; proposes/edits docs at any path | Contributor, Maintainer |
| Deployment operator | Registers a content repo as a project and flips the cutover flag | Documentation steward |
## 4. Product Use Cases
UX-level; steps are about the Product Personas (§3). Each links to the Business UC
it realizes.
```gherkin
Scenario: PUC-1 — Browse the directory tree in structure mode (realizes BUC-1)
Given the left pane is in structure mode
When the reader opens the project
Then folders and files are shown in path order
And each markdown file shows its lifecycle-state badge
And folders expand and collapse, with expansion persisted across visits
```
```gherkin
Scenario: PUC-2 — Search or non-path sort flips to flat mode (realizes BUC-1)
Given the pane is in structure mode
When the reader types a search query or picks a non-path sort
Then folders are hidden and a ranked flat list of matches is shown
And clearing the search with a path sort returns the tree with folders restored
```
```gherkin
Scenario: PUC-3 — Open a document by its path (realizes BUC-1)
When the reader selects a file in the tree
Then the main column renders that document
And the tree highlights the corresponding node
And the document's lifecycle state is visible
```
```gherkin
Scenario: PUC-4 — Propose a new doc at a path (realizes BUC-3)
Given a contributor with contribute capability
When they choose "Propose new doc" and give a path like deploy/runbooks/rollback.md
Then intermediate folders are created in the proposal branch
And a PR is opened via the bot
And a super-draft entry appears at that path
```
```gherkin
Scenario: PUC-5 — Propose an edit to an existing doc (realizes BUC-2)
Given a contributor viewing an existing document
When they propose an edit
Then an edit PR is opened against that document's path
And the file's node shows the in-review badge with the open PR reference
```
```gherkin
Scenario: PUC-6 — Filter and see lifecycle state (realizes BUC-1, BUC-2)
Given files exist in active, in-review, and super-draft states
When the reader enables the "in review" state filter
Then only files with an open edit PR are listed
```
```gherkin
Scenario: PUC-7 — Star a doc; starred pins in flat mode (product-only)
Given the reader has starred a file
When the pane is in flat mode
Then the starred file is pinned above the ranked results
```
```gherkin
Scenario: PUC-8 — Legacy slug URL redirects to its path (product-only, compat)
When the reader navigates to a legacy /p/:pid/c/:cid/e/:slug URL
Then they are redirected to the path-addressed URL for that entry
And the document renders
```
```gherkin
Scenario: PUC-9 — Operator onboards an existing repo (realizes BUC-4)
Given an existing doc repo and bot read+write access
When the operator registers it as a project's content repo and flips the cutover flag
Then every pre-existing file renders as active in the tree
And no content was moved and no state was backfilled
```
```gherkin
Scenario: PUC-10 — Tree fetch fails (realizes BUC-1a)
Given a previously cached tree exists
When the tree-listing fetch to gitea fails
Then the last good tree is served from cache
And when no cache exists, a graceful empty state with a retry action is shown
```
```gherkin
Scenario: PUC-11 — Propose at an invalid path (realizes BUC-2a)
When a contributor proposes a doc at an existing or out-of-repo path
Then the proposal is rejected with a clear validation error
And no branch or PR is left behind
```
```gherkin
Scenario: PUC-12 — Non-markdown files are listed but inert (product detail)
Given the repo contains schemas/app.json
When the tree is listed
Then schemas/app.json appears in the tree
But it has no lifecycle badge and opens no entry view
```
## 5. UX Layout
### 5.1 Left pane — `CorpusTree` (serves PUC-1..PUC-7, PUC-12)
- **Purpose:** navigate a project's corpus and reach any document; surface
lifecycle state and the contribution affordances.
- **Layout (top → bottom):**
- **Toolbar:** search input; lifecycle state filter-chips; sort selector
(path · recent · title · id · state); `+ Propose new doc` action.
- **Body — structure mode (default):** indented directory tree; folders with
expand/collapse in path order; file rows show a state dot
(active / in-review / super-draft) and a star marker; the active file is
highlighted. Non-markdown files render dimmed and inert.
- **Body — flat mode (search / non-path sort):** folders hidden; a ranked flat
list of file rows (path as secondary text); starred pinned to top; pending
super-drafts grouped.
- **States:** happy: tree/list rendered · empty: "No documents yet" + propose CTA
if permitted · loading: skeleton rows · error: stale tree if cached, else an
empty state with Retry · permission: propose control hidden without
`can_contribute`.
- **Notifications:** inline validation error on invalid propose (PUC-11); existing
PR/discussion notifications unchanged.
### 5.2 Main column (serves PUC-3, PUC-5)
Unchanged — rendered markdown + discussion + PR/contribution affordances —
addressed by path instead of slug.
## 6. Technical Design
### 6.1 Invariants
- **INV-1:** An entry is addressed by its repo `path` within a
`(project, collection)`. Slug addressing exists only as a legacy redirect.
- **INV-2:** A markdown file present on `main` with no explicit lifecycle record
is `active`. Onboarding requires no state backfill.
- **INV-3:** Folders are organizational only. Access control is the collection's;
a folder never carries permissions.
- **INV-4:** Lifecycle records are keyed by `(collection, path)` and follow file
renames on merge — no orphaned records survive a `git mv`.
- **INV-5:** The framework names no deployment. The backing repo is supplied via
the registry (`CLAUDE.md` separation-of-concerns).
- **INV-6:** Non-markdown files are listed but carry no lifecycle and open no
entry view.
- **INV-7:** The tree pane never renders blank — stale cache or an explicit
empty/retry state on fetch failure.
### 6.2 High-level architecture
```mermaid
flowchart LR
CT[CorpusTree pane] -->|GET tree / entry by path| API[API service]
CT -->|propose / edit at path| API
API --> CACHE[(TTL cache)]
CACHE --> GT[Gitea git-trees / raw]
API -->|branch / PR writes| BOT[Bot account]
BOT --> GT
API --> DB[(App DB: lifecycle, stars)]
REG[rfc-registry] -->|project to content-repo| API
WH[Gitea webhooks] -->|invalidate / reconcile| CACHE
```
- **CorpusTree** — owns left-pane rendering and mode state; owns no source of
truth; must never assume a flat namespace.
- **API service** — owns the tree-listing and path-addressed entry/contribution
endpoints; derives lifecycle state; must never write content except via the bot.
- **App DB** — owns lifecycle records, stars, reviewed-marks keyed by
`(collection, path)`; system of record for collaboration state.
- **Gitea** — system of record for document content and PRs.
- **Cache** — holds the materialized tree + per-path metadata; invalidated by the
webhook/PR-merge signals used today (§4.1 reconciler pattern).
### 6.3 Data model & ownership
| Entity | Owned by | Key fields | System of record |
| --- | --- | --- | --- |
| Document (entry) | Gitea | `(project, collection, path)`, body | Gitea repo `main` |
| Lifecycle record | App DB | `(collection, path)`, state, reviewed-by | App DB |
| Star | App DB | `(user, collection, path)` | App DB |
| Open-PR mapping | Gitea | `path` → PR number | Gitea |
| Tree node (materialized) | Cache | `path`, type, `last_commit_at`, state, `open_pr`, `starred_by_me` | derived (cache) |
### 6.4 Interfaces & contracts
- **`GET /api/projects/{pid}/collections/{cid}/tree`** — in: pid, cid, viewer ·
out: nodes `{path, type, last_commit_at, lifecycle_state, open_pr,
starred_by_me}` (full markdown tree) · errors: `404` unknown project/collection;
gitea failure → stale cache or `503` with a retriable marker (INV-7).
- **Path-addressed entry read** — generalize `…/rfcs/{slug}` to a path
(`…/entries/{path}`); out: existing entry payload; errors: `404`.
- **Propose-at-path / edit / discussion** — generalize existing contribution/PR/
discussion endpoints from `slug` to `path`; propose-new in: target path, body;
out: PR ref; errors: `409` path exists, `422` invalid/out-of-repo path
(PUC-11), `403` no `can_contribute`.
### 6.5 PerProduct-Use-Case design
#### PUC-1 — Browse the tree (realizes BUC-1; honors INV-2, INV-7)
```mermaid
sequenceDiagram
actor R as Reader
participant CT as CorpusTree
participant A as API
participant C as Cache
participant G as Gitea
R->>CT: open project
CT->>A: GET …/tree
A->>C: get materialized tree
alt cache hit
C-->>A: tree
else miss
A->>G: git trees + last-commit
G-->>A: entries
A->>A: derive state (no record = active, INV-2)
A->>C: store
end
A-->>CT: nodes
CT-->>R: structure-mode tree (or stale/empty per INV-7)
```
- **Implementation:** materialize the markdown tree once per cache cycle; join
per-path lifecycle/star records; "no record = active." Folder expansion is
client state (localStorage keyed by project/collection).
#### PUC-4 — Propose a new doc at a path (realizes BUC-3; honors INV-1, INV-3)
```mermaid
sequenceDiagram
actor Cn as Contributor
participant CT as CorpusTree
participant A as API
participant B as Bot
participant G as Gitea
Cn->>CT: + Propose new doc (path)
CT->>A: propose-at-path(path, body)
A->>A: validate path (unique, in-repo) else 409/422
A->>B: create branch + file (mkdir -p path)
B->>G: commit + open PR
G-->>A: PR ref
A-->>CT: super-draft at path
```
- **Implementation:** reuse the existing bot/branch/PR flow; only the target path
and intermediate-folder creation change. PUC-5 (edit), PUC-6 (filter), PUC-8
(redirect) reuse existing flows with `slug``path` substitution and need no new
sequence.
### 6.6 Non-functional requirements & cross-cutting concerns
- **Security & privacy:** read follows existing project/collection visibility;
propose gated by `can_contribute` (INV-3); all writes via the bot; secrets stay
references, never bytes (§6.3-handbook). No new PII.
- **Performance & scale:** v1 returns the full markdown tree from the TTL cache;
cold build is one git-trees call + a last-commit join. Lazy per-folder loading
is the scale follow-on if a body's tree is large enough to hurt cold-build
latency.
- **Availability & resilience:** stale-cache fallback + empty/retry state
(INV-7); no new hard dependency beyond what the corpus already needs.
- **Observability:** log tree cold-builds, cache hit/miss, gitea-failure
fallbacks; reuse existing health signals.
- **Accessibility:** the tree uses ARIA `tree`/`treeitem` roles with keyboard
expand/collapse and roving focus; state conveyed by text/label, not color alone.
### 6.7 Key decisions & alternatives considered
| Decision | Chosen | Alternatives considered | Why chosen |
| --- | --- | --- | --- |
| D1 Interaction model | Full governance lifecycle on any file | read-only viewer; read + light comments | operator intent; reuses existing machinery |
| D2 Topology | Tree becomes universal | tree as a second view; tree only for a new type | one navigation model; no idiom fork |
| D3 Entry identity | Path-addressed | keep slug + side table of paths | path is the natural key for a real body |
| D4 Folders vs collections | Distinct | merge folders into collections | collections are access-control, folders are structure |
| D5 Project type | No new type | a `tree`/`docs` type | "universal" means all types render via tree |
| D6 Build approach | New `CorpusTree`, port flat mode, retire Catalog | evolve Catalog in place; promote DocsLayout flyout | clean dual-mode boundary; safe staged cutover |
| D7 Existing files | "No record = active" | backfill a record per file at onboard | zero-migration onboarding |
| D8 Affordances | Keep all four via dual mode | drop sort/pending on a tree | operator marked all non-negotiable |
| D9 Routing | `e/*` splat + `?pr=` + legacy shim | encode path in slug; new `/f/` segment | least collision; reuses redirect pattern |
### 6.8 Testing strategy
Tests are part of each slice (not a follow-up).
- **Unit (vitest):** `CorpusTree` mode-switching (search→flat, sort→flat,
clear→structure), badge rendering, expand/collapse persistence, starred-pin,
flat-mode parity with the retired Catalog.
- **Backend (DI at boundaries):** tree endpoint state derivation (INV-2), open-PR
detection, path-addressed read/contribution, slug→path migration, rename
reconciliation (INV-4).
- **Two-tier:** Tier-1 local-Docker gitea for branch/PR flows; Tier-2 PPE for the
onboard-a-real-repo path (SLICE-4). e2e: browse → open; propose-at-path → PR →
merge → state transitions; search flatten; legacy redirect.
### 6.9 Failure modes, rollback & flags
- **Failure:** gitea unavailable → **behavior:** stale cache or empty/retry
(INV-7) → **rollback:** none needed.
- **Failure:** a project regresses on the new pane → **behavior/rollback:** flip
the **per-project-type cutover flag** off; the Catalog renders again (not
deleted until parity is proven).
- **Feature flag:** per-project-type cutover flag, default **off**; deployments
enable per project after validating parity.
## 7. Delivery Plan
### 7.1 Approach / strategy
Riskiest-foundation-first: land the path/state model and tree read behind the
existing flat UI (no user-visible change), then the dual-mode UI behind a flag,
then contribution-at-path, then onboard a real body. Per the handbook execution
convention, **each slice is its own coding session** — plan just-in-time →
execute → verify → ship → merge + version bump — in dependency order; this design
pass happens once and is amended only if a slice proves it wrong. The flag keeps
every intermediate state shippable.
### 7.2 Slicing plan
#### SLICE-1 — Backend foundation → completes (no user-visible PUC)
- **Depends on:**
- **DoD:** path-keyed lifecycle + slug→path migration + "no record = active"
(INV-2) + tree endpoint + path-addressed reads; Catalog still renders; backend
tests green.
#### SLICE-2 — `CorpusTree` frontend → completes PUC-1, PUC-2, PUC-3, PUC-6, PUC-7, PUC-8, PUC-10, PUC-12
- **Depends on:** SLICE-1
- **DoD:** dual-mode pane behind the cutover flag; path routing + legacy shim;
flat-mode parity; Catalog retired once a project is flipped; unit + e2e green.
#### SLICE-3 — Contribution-at-path → completes PUC-4, PUC-5, PUC-11
- **Depends on:** SLICE-1
- **DoD:** propose/edit/discussion at arbitrary paths; rename reconciliation
(INV-4); validation (BUC-2a); tests green.
#### SLICE-4 — Onboard an existing doc body → completes PUC-9 (BUC-4)
- **Depends on:** SLICE-2, SLICE-3
- **DoD:** a real external repo registered + bot access + flag flipped; all
pre-existing files render active with zero migration; Tier-2 PPE validation.
### 7.3 Rollout / launch plan
Pre-v1, single-prod: each slice ships to prod on merge with its CHANGELOG upgrade
steps; the per-project-type flag defaults off, so deployments opt in per project.
Rollback trigger: flag off → Catalog returns (§6.9).
### 7.4 Risks & mitigations
| Risk | Likelihood / impact | Mitigation |
| --- | --- | --- |
| Full-tree fetch slow on a large body | M / M | TTL cache; lazy per-folder loading as a defined follow-on |
| Catalog flat-mode parity gaps after cutover | M / H | port logic verbatim; flag-gated per project; keep Catalog until parity proven |
| Rename leaves orphan state records | M / M | webhook reconciliation (INV-4) + test |
| slug→path migration mis-maps non-`document` entry folders | L / M | resolve entry folder per project type, not hardcoded; migration test |
## 8. Traceability matrix
| Pain | Business UC | Product UC | Slice | Tests |
| --- | --- | --- | --- | --- |
| PP-1 | BUC-1 | PUC-1, PUC-2, PUC-3, PUC-6, PUC-7 | SLICE-2 | `test_tree_*`, `corpustree.*.test` |
| PP-4 | BUC-1a | PUC-10 | SLICE-2 | `test_tree_fallback_*` |
| PP-3 | BUC-2 | PUC-5, PUC-6 | SLICE-3 | `test_edit_at_path_*` |
| PP-3 | BUC-2a | PUC-11 | SLICE-3 | `test_propose_validation_*` |
| PP-3 | BUC-3 | PUC-4 | SLICE-3 | `test_propose_at_path_*` |
| PP-2 | BUC-4 | PUC-9 | SLICE-4 | e2e `onboard_repo_*` |
| PP-1 | (compat) | PUC-8 | SLICE-2 | `test_legacy_redirect_*` |
| PP-1 | (detail) | PUC-12 | SLICE-2 | `test_tree_non_markdown_*` |
## 9. Open Questions & Decisions log
**Open**
| # | Question | Owner | Blocks |
| --- | --- | --- | --- |
| — | none outstanding | — | — |
**Resolved**
| # | Decision | Resolution | Date |
| --- | --- | --- | --- |
| D1 | Interaction model | Full governance lifecycle on any file | 2026-06-06 |
| D2 | Topology | Tree becomes the universal left pane | 2026-06-06 |
| D3 | Entry identity | Path-addressed; slug is legacy-redirect-only | 2026-06-06 |
| D4 | Folders vs collections | Kept distinct | 2026-06-06 |
| D5 | Project type | No new type; existing types render via tree | 2026-06-06 |
| D6 | Build approach | New `CorpusTree`; port flat mode; retire Catalog | 2026-06-06 |
| D7 | Pre-existing files | "No record = active"; zero-migration onboarding | 2026-06-06 |
| D8 | Affordances | Keep all four via dual display mode | 2026-06-06 |
| D9 | Routing | `e/*` splat + `?pr=` + legacy shim | 2026-06-06 |
Deferred (not open): lazy per-folder tree loading; raw/preview view for
non-markdown files.
## 10. Glossary & References
- **Structure mode** — left pane showing the directory tree in path order.
- **Flat mode** — left pane showing a ranked flat list (search/non-path sort);
the retired Catalog's behavior.
- **Entry** — a markdown document, addressed by its repo path within a
`(project, collection)`.
- **Lifecycle record** — app-DB row keyed by `(collection, path)` carrying
non-default state; absence means `active`.
- **References:** `SPEC.md` §7 (catalog), §22 (three-tier), §8 (revision), §13
(graduation), §20 (versioning); `CLAUDE.md` (separation-of-concerns);
handbook `solution-design/GUIDE.md` (this doc's standard).
+40
View File
@@ -0,0 +1,40 @@
import { waitForLatestOtc, clearMailpit } from './mailpit.js'
// Sign in and leave the rfc_session cookie on the page's browser context
// (page.request shares the page context's cookie jar, so a subsequent
// page.goto is authenticated). Two paths, chosen by environment:
//
// * DEPLOYED (PPE): when E2E_TEST_AUTH_SECRET is set, use the gated
// `/auth/test/login` shortcut (v0.52.0). The deployed env has no
// Mailpit sink to read an OTC code from, so the suite presents the
// shared secret and the server mints an owner session for the one
// configured E2E_TEST_AUTH_EMAIL. See backend/app/main.py.
// * LOCAL (Tier-1 docker stack): no secret set → the original §6.2 OTC
// path through Mailpit. The Tier-1 backend-seed pre-grants
// e2e-owner@example.test as a deployment owner, so signing in as that
// address yields write access (SLICE-4/5).
//
// OWNER_EMAIL is the identity both paths sign in as: E2E_OWNER_EMAIL when
// set (PPE points it at E2E_TEST_AUTH_EMAIL), else the Tier-1 default.
export const OWNER_EMAIL =
process.env.E2E_OWNER_EMAIL || 'e2e-owner@example.test'
export async function signIn(page, email) {
const secret = process.env.E2E_TEST_AUTH_SECRET
if (secret) {
const r = await page.request.post('/auth/test/login', {
data: { email },
headers: { 'X-Test-Auth-Secret': secret },
})
if (!r.ok()) throw new Error(`test-auth login failed: ${r.status()} ${await r.text()}`)
return
}
// Local Tier-1 OTC path through Mailpit.
await clearMailpit()
const req = await page.request.post('/auth/otc/request', { data: { email } })
if (!req.ok()) throw new Error(`otc request failed: ${req.status()}`)
const code = await waitForLatestOtc(email)
const verify = await page.request.post('/auth/otc/verify', { data: { email, code } })
if (!verify.ok()) throw new Error(`otc verify failed: ${verify.status()}`)
}
+40
View File
@@ -0,0 +1,40 @@
// Shared Playwright fixtures for the deployed-environment harness.
//
// Pre-record a cookie-consent choice via addInitScript so the §14.5
// cookie-consent banner NEVER renders. The banner is fixed to the bottom
// of the viewport and intercepts pointer events over the catalog footer
// (the row-select checkboxes SLICE-5 clicks). The previous approach —
// dismiss it after navigation (lib/ui.js dismissCookies) — raced the
// banner's render on the slower deployed edge (PPE): dismissCookies ran
// before the banner mounted, found nothing to remove, and the banner then
// appeared and swallowed the row clicks. Recording consent at
// document-start (before the app's scripts read `hasChosen()`) means the
// banner's `open` state initialises false and it never mounts — no race.
//
// Storage shape mirrors lib/consent.js (LS_KEY 'rfc-app.cookie-consent.v1';
// a non-null recorded_at == "the user has chosen"). Environment-agnostic:
// the init script runs on whatever origin the test navigates to (PPE or
// the Tier-1 localhost stack).
import { test as base, expect } from '@playwright/test'
const CONSENT = JSON.stringify({
essential: true,
analytics: false,
other: false,
recorded_at: '2000-01-01T00:00:00.000Z',
})
export const test = base.extend({
context: async ({ context }, use) => {
await context.addInitScript((value) => {
try {
window.localStorage.setItem('rfc-app.cookie-consent.v1', value)
} catch {
// localStorage unavailable — fall back to lib/ui.js dismissCookies.
}
}, CONSENT)
await use(context)
},
})
export { expect }
+18
View File
@@ -0,0 +1,18 @@
// Dismiss the cookie-consent banner if it's showing. It is fixed to the bottom
// of the viewport and intercepts pointer events over the catalog footer (where
// row-select checkboxes live), so tests that click there must clear it first.
export async function dismissCookies(page) {
const banner = page.locator('.cookie-consent-banner')
if (await banner.count()) {
// Click to persist the consent choice so it doesn't reappear on
// later navigation...
await page.getByRole('button', { name: 'Save choice' }).click().catch(() => {})
await banner.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {})
// ...then forcibly remove any node still in the DOM. The dismiss
// click occasionally doesn't land before a test clicks a catalog
// footer checkbox (flaky over the deployed edge), and a lingering
// fixed banner intercepts those pointer events. Removing the node
// makes the dismissal deterministic.
await banner.evaluate((el) => el.remove()).catch(() => {})
}
}
+53
View File
@@ -0,0 +1,53 @@
import { test, expect } from './lib/fixtures.js'
import { signIn, OWNER_EMAIL } from './lib/auth.js'
import { dismissCookies } from './lib/ui.js'
// §8.12 Option B — asking-while-reading on the canonical `main` view.
//
// Before this fix, the AI "Ask" affordance had no chat surface on main (the
// human-discussion panel renders there, not the chat panel), so asking from the
// reading view silently did nothing. Option B transparently cuts an edit branch,
// navigates to it, and runs the question there.
//
// This spec asserts the model-independent core: the branch cut + navigation
// (old code did NOTHING here — the silent no-op), and that the question turn
// actually FIRED against the branch. The turn's outcome is environment-specific
// — PPE/prod (a model is configured) streams an answer and the optimistic
// question persists; the Tier-1 stack has no AI provider, so the turn errors
// with "Chat failed". Either outcome proves the turn fired; we match both so the
// one spec passes in both environments. The quote-passthrough + answer rendering
// are covered deterministically by the RFCView unit tests (model mocked).
//
// Driven through the prompt bar, which shares handlePrompt → handleMainAsk with
// the selection tooltip's onAsk. Runs against the faceted `bdd` collection entry
// (collection-scoped — the same three-tier path G-15 hardened).
const ENTRY = '/p/ohm/c/bdd/e/checkout-returning'
test('Ask from the canonical view cuts an edit branch and lands the question in chat', async ({ page }) => {
await signIn(page, OWNER_EMAIL)
await page.goto(ENTRY)
await dismissCookies(page)
// We start on the canonical (main) view: read-only, no chat surface — the
// discuss-mode banner is the canonical-view tell, and the URL has no branch.
await expect(page.locator('.discuss-mode-banner')).toContainText('read-only')
expect(new URL(page.url()).searchParams.get('branch')).toBeNull()
// Ask a question from the prompt bar.
const question = `What problem does this entry solve? (${Date.now()})`
await page.locator('.prompt-input').fill(question)
await page.getByRole('button', { name: 'Ask', exact: true }).click()
// Option B: an edit branch is cut and we navigate onto it (no silent no-op).
await expect(async () => {
expect(new URL(page.url()).searchParams.get('branch')).toMatch(/^edit/)
}).toPass({ timeout: 30_000 })
// The question turn fired against the new branch: either the optimistic
// question is showing (model answered / answering) or the turn surfaced a
// chat error (no AI provider in Tier-1). Both prove it ran — the pre-fix
// silent no-op showed neither.
await expect(
page.getByText(question).or(page.getByText(/Chat failed|No AI providers/i)),
).toBeVisible({ timeout: 30_000 })
})
+91
View File
@@ -0,0 +1,91 @@
import { test, expect } from './lib/fixtures.js'
import { signIn, OWNER_EMAIL } from './lib/auth.js'
import { dismissCookies } from './lib/ui.js'
// §22.4a Configurable Collection Metadata — end-to-end browser coverage for the
// three UI slices, against the Tier-1 stack's faceted `bdd` collection (seeded
// in testing/seed-gitea.sh with a fields: schema of priority(enum) + tags, and
// three entries: checkout-guest [P0], checkout-returning [P1], search-facets [P0]).
//
// Tests run in order against a freshly-seeded stack: SLICE-3 reads first, then
// SLICE-4 edits `checkout-returning`, then SLICE-5 bulk-edits the two P0 entries
// (distinct rows — no cross-test interference within one run).
const BDD = '/p/ohm/c/bdd'
// SLICE-3 (PUC-3) — faceted left-pane filtering, anonymous/read-only.
test('SLICE-3: faceted filter narrows the catalog by Priority', async ({ page }) => {
await page.goto(BDD)
await dismissCookies(page)
const catalog = page.locator('aside.catalog')
// All three entries are listed initially.
await expect(catalog.getByText('Guest checkout')).toBeVisible()
await expect(catalog.getByText('Returning-customer checkout')).toBeVisible()
await expect(catalog.getByText('Faceted search')).toBeVisible()
// The Priority facet renders with the seeded P0 count (2 entries).
const p0 = catalog.locator('.facet-value', { hasText: 'P0' })
await expect(p0.locator('.facet-count')).toHaveText('2')
// Selecting P0 re-fetches server-side and drops the lone P1 entry.
await p0.getByRole('checkbox').check()
await expect(catalog.getByText('Returning-customer checkout')).toHaveCount(0)
await expect(catalog.getByText('Guest checkout')).toBeVisible()
await expect(catalog.getByText('Faceted search')).toBeVisible()
})
// SLICE-4 (PUC-1) — single-entry metadata edit via the detail panel (authed).
test('SLICE-4: edit one entry\'s priority from the detail panel', async ({ page }) => {
await signIn(page, OWNER_EMAIL)
await page.goto(`${BDD}/e/checkout-returning`)
await dismissCookies(page)
const panel = page.locator('.metadata-fields-panel')
await expect(panel).toBeVisible()
const select = panel.locator('#mf-priority')
// Value-agnostic so the test is idempotent across retries and re-runs: a
// prior (committed) edit may have already moved it off the P1 seed. Pick a
// target distinct from the current value so Save is a real change.
const current = await select.inputValue()
const target = current === 'P2' ? 'P1' : 'P2'
await select.selectOption(target)
await panel.getByRole('button', { name: 'Save' }).click()
// The save is a real git sidecar commit via the bot (D7); over the deployed
// edge the round trip can take tens of seconds — wait generously for the
// confirmation rather than the default expect timeout.
await expect(panel.getByText('Saved')).toBeVisible({ timeout: 60_000 })
// Persisted across a reload (read back from the committed sidecar).
await page.reload()
await expect(page.locator('.metadata-fields-panel #mf-priority')).toHaveValue(target)
})
// SLICE-5 (PUC-2) — multi-select + bulk action bar, one commit (authed).
test('SLICE-5: bulk-set priority on multiple selected entries', async ({ page }) => {
await signIn(page, OWNER_EMAIL)
await page.goto(BDD)
await dismissCookies(page)
const catalog = page.locator('aside.catalog')
await expect(catalog.getByText('Guest checkout')).toBeVisible()
// Select the two P0 entries.
await catalog.getByLabel('select Guest checkout').check()
await catalog.getByLabel('select Faceted search').check()
// The sticky bulk bar appears with the selected count.
const bar = page.locator('.bulk-action-bar')
await expect(bar.getByText('2 selected')).toBeVisible()
// Set priority P1 across both → one commit; a toast reports the result.
// Like SLICE-4, the bulk write is a real git commit via the bot; allow the
// deployed-edge round trip generous time before the result toast appears.
await bar.getByLabel('Set Priority').selectOption('P1')
await expect(page.getByText(/2 updated/)).toBeVisible({ timeout: 60_000 })
// The change is reflected server-side: the Priority P1 facet now counts the
// two newly-updated entries plus the pre-existing P1 (checkout-returning was
// P1 at seed; SLICE-4 may have moved it — assert at least the two we set).
await expect(catalog.locator('.facet-value', { hasText: 'P1' }).locator('.facet-count'))
.not.toHaveText('0')
})
+16 -2
View File
@@ -1,9 +1,23 @@
import { defineConfig } from '@playwright/test'
// The metadata specs sign in, navigate, and (SLICE-4/5) write real commits,
// so a handful of steps are timing-sensitive: the cookie-consent banner's
// dismiss animation, first-render of the detail panel, and the round trip
// after a write. These flake intermittently on a busy local box and more so
// against a deployed host (network latency). `retries` makes the suite robust
// to that (and finally makes `trace: 'on-first-retry'` meaningful); the
// timeouts are bumped a notch for deployed runs over the public edge.
const DEPLOYED = !!process.env.E2E_TEST_AUTH_SECRET
export default defineConfig({
testDir: '.',
timeout: 30_000,
expect: { timeout: 10_000 },
timeout: DEPLOYED ? 60_000 : 45_000,
expect: { timeout: DEPLOYED ? 20_000 : 12_000 },
retries: 2,
// The metadata specs run in order against one seeded collection and write
// real commits (SLICE-4/5); parallel workers would race on shared state —
// and on a deployed host, on concurrent git pushes through the bot. Serialize.
workers: 1,
use: {
baseURL: process.env.BASE_URL || 'http://localhost:8080',
trace: 'on-first-retry',
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "rfc-app-frontend",
"private": true,
"version": "0.46.0",
"version": "0.54.0",
"type": "module",
"scripts": {
"dev": "vite",
+66
View File
@@ -150,6 +150,56 @@
.row-title { font-size: var(--text-base); margin-top: 2px; }
.catalog-row.is-super .row-title { color: var(--c-gray-500); }
.row-tags { font-size: var(--text-xs); color: var(--c-gray-500); margin-top: 2px; }
.row-malformed { font-size: var(--text-2xs); color: var(--c-warning-accent); font-weight: 600; }
/* §22.4a SLICE-5 — row multi-select + bulk action bar (PUC-2, §5.3). */
.catalog-row-wrap { display: flex; align-items: flex-start; }
.catalog-row-wrap .row-select { margin: 10px 0 0 10px; flex: none; }
.catalog-row-wrap .catalog-row { flex: 1; min-width: 0; }
.bulk-action-bar {
position: sticky; top: 0; z-index: 2;
display: flex; flex-wrap: wrap; align-items: center; gap: 8px;
padding: 8px 14px; margin: 0;
background: var(--c-gray-150); border-bottom: 1px solid var(--c-gray-300);
font-size: var(--text-xs);
}
.bulk-action-bar .bulk-count { font-weight: 700; }
.bulk-action-bar .bulk-set { display: flex; align-items: center; gap: 4px; }
.bulk-action-bar .bulk-tags { display: flex; align-items: center; gap: 4px; }
.bulk-action-bar .bulk-tags input { width: 90px; }
.bulk-action-bar .bulk-clear { margin-left: auto; }
/* §22.4a SLICE-3 — faceted left-pane filters (§5.1). */
.facets {
padding: 8px 14px;
border-bottom: 1px solid var(--c-gray-150);
}
.facet-group { margin-bottom: 8px; }
.facet-header {
background: none; border: 0; cursor: pointer; width: 100%; text-align: left;
font-size: var(--text-xs); font-weight: 700; color: var(--c-gray-600);
text-transform: uppercase; letter-spacing: 0.04em; padding: 4px 0;
}
.facet-values { display: flex; flex-direction: column; gap: 2px; }
.facet-value {
display: flex; align-items: center; gap: 6px;
font-size: var(--text-xs); color: var(--c-gray-600); cursor: pointer;
}
.facet-value-label { flex: 1; }
.facet-count { color: var(--c-gray-500); font-variant-numeric: tabular-nums; }
.facet-value-search {
width: 100%; margin: 4px 0; font-size: var(--text-xs);
padding: 2px 6px; border: 1px solid var(--c-gray-150); border-radius: var(--radius-sm);
}
.facet-clear {
background: none; border: 0; cursor: pointer; padding: 0 0 6px;
font-size: var(--text-xs); color: var(--c-ink); text-decoration: underline;
}
.facet-empty { font-size: var(--text-xs); color: var(--c-gray-500); }
.facet-malformed {
display: flex; align-items: center; gap: 6px; margin-top: 6px;
font-size: var(--text-xs); color: var(--c-warning-accent); cursor: pointer;
}
.catalog-pending {
border-top: 1px solid var(--c-gray-150);
@@ -2637,3 +2687,19 @@ select:focus-visible,
align-items: flex-start;
gap: 12px;
}
/* 404 / not-found page (client-side, for unmatched routes) */
.not-found {
max-width: 32rem;
margin: 4rem auto;
padding: 0 1.5rem;
text-align: center;
}
.not-found-code {
font-size: 3.5rem;
line-height: 1;
margin: 0 0 0.5rem;
color: var(--c-gray-900, #111);
}
.not-found-message { color: var(--c-gray-700, #444); margin: 0 0 1.25rem; }
.not-found-actions a { color: var(--c-gray-900, #111); text-decoration: underline; }
+6 -3
View File
@@ -7,6 +7,7 @@ import { brandTitle } from './lib/brand'
import { entryPath, proposalPath, DEFAULT_COLLECTION } from './lib/entryPaths'
import { useDeployment } from './context/DeploymentProvider'
import ProjectLayout from './components/ProjectLayout.jsx'
import NotFound from './components/NotFound.jsx'
import Directory from './components/Directory.jsx'
import ProjectSwitcher from './components/ProjectSwitcher.jsx'
import Catalog from './components/Catalog.jsx'
@@ -381,13 +382,14 @@ export default function App() {
<Route path="c/:collectionId/e/:slug" element={<RFCView viewer={viewer} />} />
<Route path="c/:collectionId/e/:slug/pr/:prNumber" element={<PRView viewer={viewer} />} />
<Route path="c/:collectionId/proposals/:prNumber" element={<ProposalView viewer={viewer} onChange={() => setCatalogVersion(v => v + 1)} />} />
<Route path="*" element={<NotFound />} />
</Routes>
</main>
</ProjectLayout>
} />
{/* Any other path (incl. the retired bare-slug corpus URLs that
somehow reach the SPA) lands on the deployment landing. */}
<Route path="*" element={<Navigate to="/" replace />} />
{/* Any unmatched path is a real 404 surface it rather than
silently bouncing home, so a bad/typo'd URL is visible. */}
<Route path="*" element={<NotFound />} />
</Routes>
</div>
{(proposeOpen || proposeParam != null) && viewer && (
@@ -507,6 +509,7 @@ function DocsWithSidebar({ viewer }) {
client-side redirect to the first configured spec. */}
<Route path="specs" element={<DocsSpecsIndex />} />
<Route path="specs/:name" element={<DocsSpec />} />
<Route path="*" element={<NotFound />} />
</Route>
</Routes>
</main>
+40
View File
@@ -0,0 +1,40 @@
// §22/G-15 — getRFCMain/getBranch build collection-scoped URLs when a project +
// collection are supplied (so an entry outside the default collection reads its
// own content repo / subfolder), and fall back to the slug-only routes otherwise.
import { describe, it, expect, vi, afterEach } from 'vitest'
import { getRFCMain, getBranch } from './api.js'
function mockFetch() {
const fn = vi.fn(async () => ({ ok: true, status: 200, json: async () => ({}) }))
global.fetch = fn
return fn
}
afterEach(() => { vi.restoreAllMocks() })
describe('G-15 collection-scoped branch/body URLs', () => {
it('getRFCMain scopes to project+collection when both are given', async () => {
const f = mockFetch()
await getRFCMain('login', 'rfc-app', 'specs')
expect(f).toHaveBeenCalledWith('/api/projects/rfc-app/collections/specs/rfcs/login/main')
})
it('getRFCMain falls back to the slug-only route without a collection', async () => {
const f = mockFetch()
await getRFCMain('login')
expect(f).toHaveBeenCalledWith('/api/rfcs/login/main')
})
it('getBranch scopes to project+collection and encodes the branch', async () => {
const f = mockFetch()
await getBranch('login', 'edit/foo', 'rfc-app', 'specs')
expect(f).toHaveBeenCalledWith(
'/api/projects/rfc-app/collections/specs/rfcs/login/branches/edit%2Ffoo')
})
it('getBranch falls back to the slug-only route without a collection', async () => {
const f = mockFetch()
await getBranch('login', 'main')
expect(f).toHaveBeenCalledWith('/api/rfcs/login/branches/main')
})
})
+66 -6
View File
@@ -203,11 +203,27 @@ export async function createProject({ projectId, name, type, visibility, content
// §22.4 (Plan B) / §22 S2: per-collection serving. With a projectId + a
// collectionId, read the collection-scoped routes; with only a projectId, the
// project default-collection compat path; with neither, the unscoped path.
export async function listRFCs(projectId, collectionId) {
if (projectId && collectionId) {
return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs`))
// §22.4a SLICE-3: faceted catalog. `opts.selections` is { field: string[] }
// (each non-empty array becomes repeated query params, OR within the field);
// `opts.malformed` toggles the malformed-only filter. Returns the full payload
// { items, facets } — callers read both. With no selections it behaves as the
// pre-SLICE-3 list (and a no-fields collection returns facets: {}). Facets are
// served only on the project/collection-scoped paths, not the unscoped one.
export async function listRFCs(projectId, collectionId, opts = {}) {
const { selections = {}, malformed = false } = opts
const qs = new URLSearchParams()
for (const [field, values] of Object.entries(selections)) {
for (const v of values) qs.append(field, v)
}
const url = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs'
if (malformed) qs.set('malformed', 'true')
const query = qs.toString()
let base
if (projectId && collectionId) {
base = `/api/projects/${projectId}/collections/${collectionId}/rfcs`
} else {
base = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs'
}
const url = query ? `${base}?${query}` : base
return jsonOrThrow(await fetch(url))
}
@@ -432,11 +448,27 @@ export async function listModels(slug) {
return jsonOrThrow(await fetch(`/api/rfcs/${slug}/models`))
}
export async function getRFCMain(slug) {
export async function getRFCMain(slug, projectId, collectionId) {
// §22/G-15: when the caller knows the entry's project + collection, read the
// collection-scoped route so a slug that exists in two collections resolves
// unambiguously and the entry's own content repo / subfolder is used. The
// bare-slug form stays for back-compat (default-collection callers).
if (projectId && collectionId) {
return jsonOrThrow(await fetch(
`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}/main`
))
}
return jsonOrThrow(await fetch(`/api/rfcs/${slug}/main`))
}
export async function getBranch(slug, branch) {
export async function getBranch(slug, branch, projectId, collectionId) {
// §22/G-15: collection-scoped branch-body read (the canonical-body GET); see
// getRFCMain for the rationale. Falls back to the slug-only route.
if (projectId && collectionId) {
return jsonOrThrow(await fetch(
`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}/branches/${encodeURIComponent(branch)}`
))
}
return jsonOrThrow(await fetch(
`/api/rfcs/${slug}/branches/${encodeURIComponent(branch)}`
))
@@ -666,6 +698,34 @@ export async function editMetadata(slug, { title, tags, prDescription }) {
return jsonOrThrow(res)
}
// §22.4a SLICE-4: edit one entry's schema-defined metadata — a direct commit
// to its `<slug>.meta.yaml` sidecar (contributor+ gated server-side).
export async function saveEntryMeta(projectId, collectionId, slug, values) {
const res = await fetch(
`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}/meta`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ values }),
},
)
return jsonOrThrow(res)
}
// §22.4a SLICE-5 (PUC-2): apply one field op (set | add | remove) to many
// entries at once — one commit server-side; returns { applied, rejected }.
export async function bulkEntryMeta(projectId, collectionId, { slugs, op, field, value }) {
const res = await fetch(
`/api/projects/${projectId}/collections/${collectionId}/meta/bulk`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ slugs, op, field, value }),
},
)
return jsonOrThrow(res)
}
// ── Slice 5: §13 graduation + §13.1 claim ────────────────────────────────
export async function claimOwnership(slug) {
+4 -1
View File
@@ -29,6 +29,7 @@ import {
} from '../api.js'
import { EVENTS, track } from '../lib/analytics.js'
import { entryPath, useProjectId } from '../lib/entryPaths'
import NotFound from './NotFound.jsx'
// v0.17.0 roadmap item #16. The max length the backend enforces
// (Pydantic body bound + `invites.CUSTOM_MESSAGE_MAX_LENGTH`); kept
@@ -59,7 +60,8 @@ export default function Admin({ viewer }) {
{tabs.map(t => (
<li key={t.path}>
<NavLink
to={t.path}
to={`/admin/${t.path}`}
end
className={({ isActive }) => `admin-rail-link ${isActive ? 'active' : ''}`}
>
{t.label}
@@ -81,6 +83,7 @@ export default function Admin({ viewer }) {
{isSiteOwner && <Route path="retired" element={<RetiredTab />} />}
<Route path="audit" element={<AuditTab />} />
<Route path="permissions" element={<PermissionsTab />} />
<Route path="*" element={<NotFound message="That admin page doesn't exist." />} />
</Routes>
</div>
</div>
+63
View File
@@ -0,0 +1,63 @@
import { useState } from 'react'
// §22.4a SLICE-5 (PUC-2, UX §5.3): the sticky bulk action bar shown when 1
// catalog row is selected. Driven by the collection `fields:` schema one
// "Set <field>" control per enum field, and an Add/Remove tag control per
// tags field. Each gesture calls onApply({ op, field, value }); the parent
// (Catalog) sends one bulk request and re-fetches.
const labelFor = (name, def) =>
def?.label || name.charAt(0).toUpperCase() + name.slice(1)
export default function BulkActionBar({ fields, count, onApply, onClear }) {
const [tagValue, setTagValue] = useState('')
const entries = Object.entries(fields || {})
const tagField = entries.find(([, d]) => d.type === 'tags')
return (
<div className="bulk-action-bar">
<span className="bulk-count">{count} selected</span>
{entries
.filter(([, d]) => d.type === 'enum')
.map(([name, def]) => (
<label key={name} className="bulk-set">
<span>Set {labelFor(name, def)}</span>
<select
aria-label={`Set ${labelFor(name, def)}`}
value=""
onChange={e => {
if (e.target.value) onApply({ op: 'set', field: name, value: e.target.value })
}}
>
<option value=""></option>
{(def.values || []).map(v => <option key={v} value={v}>{v}</option>)}
</select>
</label>
))}
{tagField && (
<div className="bulk-tags">
<input
placeholder="tag…"
value={tagValue}
onChange={e => setTagValue(e.target.value)}
/>
<button
type="button"
disabled={!tagValue.trim()}
onClick={() => { onApply({ op: 'add', field: tagField[0], value: tagValue.trim() }); setTagValue('') }}
>
Add tag
</button>
<button
type="button"
disabled={!tagValue.trim()}
onClick={() => { onApply({ op: 'remove', field: tagField[0], value: tagValue.trim() }); setTagValue('') }}
>
Remove tag
</button>
</div>
)}
<button type="button" className="bulk-clear" onClick={onClear}>Clear</button>
</div>
)
}
@@ -0,0 +1,45 @@
import { render, screen, fireEvent } from '@testing-library/react'
import { describe, it, expect, vi } from 'vitest'
import BulkActionBar from './BulkActionBar.jsx'
const FIELDS = {
priority: { type: 'enum', values: ['P0', 'P1', 'P2'], label: 'Priority' },
tags: { type: 'tags', label: 'Tags' },
}
describe('BulkActionBar', () => {
it('shows the selected count', () => {
render(<BulkActionBar fields={FIELDS} count={3} onApply={() => {}} onClear={() => {}} />)
expect(screen.getByText(/3 selected/i)).toBeInTheDocument()
})
it('applies a set op when an enum value is chosen', () => {
const onApply = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={onApply} onClear={() => {}} />)
fireEvent.change(screen.getByLabelText(/set priority/i), { target: { value: 'P0' } })
expect(onApply).toHaveBeenCalledWith({ op: 'set', field: 'priority', value: 'P0' })
})
it('applies an add-tag op', () => {
const onApply = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={onApply} onClear={() => {}} />)
fireEvent.change(screen.getByPlaceholderText(/tag…/i), { target: { value: 'checkout' } })
fireEvent.click(screen.getByRole('button', { name: /add tag/i }))
expect(onApply).toHaveBeenCalledWith({ op: 'add', field: 'tags', value: 'checkout' })
})
it('applies a remove-tag op', () => {
const onApply = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={onApply} onClear={() => {}} />)
fireEvent.change(screen.getByPlaceholderText(/tag…/i), { target: { value: 'checkout' } })
fireEvent.click(screen.getByRole('button', { name: /remove tag/i }))
expect(onApply).toHaveBeenCalledWith({ op: 'remove', field: 'tags', value: 'checkout' })
})
it('calls onClear', () => {
const onClear = vi.fn()
render(<BulkActionBar fields={FIELDS} count={2} onApply={() => {}} onClear={onClear} />)
fireEvent.click(screen.getByRole('button', { name: /clear/i }))
expect(onClear).toHaveBeenCalled()
})
})
+165 -18
View File
@@ -8,9 +8,12 @@
import { useEffect, useMemo, useState } from 'react'
import { useParams, Link } from 'react-router-dom'
import { listRFCs, listProposals, getCollection } from '../api'
import { listRFCs, listProposals, getCollection, bulkEntryMeta } from '../api'
import { entryPath, proposalPath, useProjectId, useCollectionId } from '../lib/entryPaths'
import JoinRequestModal from './JoinRequestModal.jsx'
import FacetGroups from './FacetGroups.jsx'
import BulkActionBar from './BulkActionBar.jsx'
import { showToast } from './ToastHost.jsx'
const STATE_CHIPS = [
{ id: 'super-draft', label: 'Super-draft' },
@@ -43,34 +46,81 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
const [sort, setSort] = useState('recent')
const [activeChips, setActiveChips] = useState(new Set())
const [pendingOpen, setPendingOpen] = useState(true)
// §22.4a SLICE-3: faceted left pane. When the collection declares a `fields:`
// schema, the catalog renders faceted groups (counts from the server) instead
// of the legacy state chips, and re-fetches server-side as selections change.
const [fields, setFields] = useState(null) // collection field schema
const [facets, setFacets] = useState({}) // { field: { value: count } }
const [selections, setSelections] = useState({}) // { field: Set<string> }
const [malformedOnly, setMalformedOnly] = useState(false)
// §22.4a SLICE-5: multi-select for the bulk action bar (PUC-2). A Set of
// selected slugs; the sticky bar appears when 1 is selected (faceted +
// contributor only). Cleared on collection switch and after a bulk apply.
const [selected, setSelected] = useState(() => new Set())
const [loading, setLoading] = useState(false)
const { slug, prNumber } = useParams()
const pid = useProjectId()
// §22 S2: the catalog is scoped to the active collection (the `/c/:cid/`
// route segment, else the project's default collection).
const cid = useCollectionId()
// Collection-level data (proposals, caps, field schema) loads once per
// collection independent of the facet selections, which only re-fetch the
// entry list. Switching collections clears any active facet selections.
useEffect(() => {
listRFCs(pid, cid).then(d => setRfcs(d.items)).catch(() => setRfcs([]))
listProposals(pid).then(d => setProposals(d.items)).catch(() => setProposals([]))
setCanContribute(null)
setCanRequestJoin(false)
setSelections({})
setMalformedOnly(false)
setSelected(new Set())
getCollection(pid, cid)
.then(c => {
setCanContribute(!!c?.viewer?.can_contribute)
setCanRequestJoin(!!c?.viewer?.can_request_join)
setEntryNoun(c?.entry_noun || 'RFC')
setFields(c?.fields || null)
})
.catch(() => setCanContribute(false))
.catch(() => { setCanContribute(false); setFields(null) })
}, [version, pid, cid])
// §22.4a SLICE-3: the entry list re-fetches server-side whenever the facet
// selections or the malformed toggle change (in legacy mode selections stay
// empty, so this fires once per collection like before).
useEffect(() => {
setLoading(true)
const selObj = Object.fromEntries(
Object.entries(selections).map(([f, set]) => [f, [...set]])
)
listRFCs(pid, cid, { selections: selObj, malformed: malformedOnly })
.then(d => {
setRfcs(d.items); setFacets(d.facets || {})
// §22.4a SLICE-5: drop any bulk selections that the new (filtered)
// list no longer contains, so the bar can't act on hidden rows.
const visible = new Set(d.items.map(it => it.slug))
setSelected(prev => {
const kept = [...prev].filter(s => visible.has(s))
return kept.length === prev.size ? prev : new Set(kept)
})
})
.catch(() => { setRfcs([]); setFacets({}) })
.finally(() => setLoading(false))
}, [version, pid, cid, selections, malformedOnly])
// While caps load, fall back to "any authenticated viewer" so the propose
// affordance on the common (default-collection) case doesn't flash off.
const mayPropose = canContribute === null ? !!viewer : canContribute
// §22.4a SLICE-3: faceted mode when the collection declares a non-empty
// `fields:` schema; otherwise the legacy state-chip catalog (INV-5).
const faceted = !!fields && Object.keys(fields).length > 0
const filtered = useMemo(() => {
const needle = search.trim().toLowerCase()
let items = rfcs.filter(r => {
if (activeChips.size > 0 && !activeChips.has(r.state)) return false
// In faceted mode the server already filtered by state; the legacy chips
// narrow client-side only when there's no schema.
if (!faceted && activeChips.size > 0 && !activeChips.has(r.state)) return false
if (!needle) return true
const hay = [r.title, r.slug, r.id || ''].join(' ').toLowerCase()
return hay.includes(needle)
@@ -85,7 +135,7 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
return (b.last_active_at || '').localeCompare(a.last_active_at || '')
})
return items
}, [rfcs, search, sort, activeChips])
}, [rfcs, search, sort, activeChips, faceted])
function toggleChip(id) {
const next = new Set(activeChips)
@@ -93,6 +143,53 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
setActiveChips(next)
}
// §22.4a SLICE-3 facet selection handlers (faceted mode).
function toggleFacet(field, value) {
setSelections(prev => {
const next = { ...prev }
const set = new Set(next[field] || [])
set.has(value) ? set.delete(value) : set.add(value)
if (set.size === 0) delete next[field]
else next[field] = set
return next
})
}
function clearFilters() { setSelections({}); setMalformedOnly(false) }
const hasActiveFilters = Object.keys(selections).length > 0 || malformedOnly
// §22.4a SLICE-5: row selection + bulk apply (PUC-2).
function toggleSelect(rowSlug) {
setSelected(prev => {
const next = new Set(prev)
next.has(rowSlug) ? next.delete(rowSlug) : next.add(rowSlug)
return next
})
}
async function applyBulk({ op, field, value }) {
const slugs = [...selected]
if (slugs.length === 0) return
try {
const res = await bulkEntryMeta(pid, cid, { slugs, op, field, value })
const nApplied = res.applied?.length || 0
const nRejected = res.rejected?.length || 0
showToast({
summary: nRejected
? `${nApplied} updated, ${nRejected} skipped`
: `${nApplied} updated`,
category: nRejected ? 'warning' : 'success',
})
setSelected(new Set())
// Re-fetch the list so the new values + facet counts reflect the change.
const selObj = Object.fromEntries(
Object.entries(selections).map(([f, set]) => [f, [...set]]))
const d = await listRFCs(pid, cid, { selections: selObj, malformed: malformedOnly })
setRfcs(d.items); setFacets(d.facets || {})
} catch (e) {
showToast({ summary: e.message || 'Bulk update failed', category: 'error' })
}
}
return (
<aside className="catalog">
<div className="catalog-search">
@@ -108,22 +205,54 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
{SORT_OPTIONS.map(o => <option key={o.id} value={o.id}>{o.label}</option>)}
</select>
</div>
<div className="catalog-chips">
{STATE_CHIPS.map(chip => (
<button
key={chip.id}
className={`chip ${activeChips.has(chip.id) ? 'active' : ''}`}
onClick={() => toggleChip(chip.id)}
>
{chip.label}
</button>
))}
</div>
{faceted ? (
<FacetGroups
facets={facets}
fields={fields}
selections={selections}
onToggle={toggleFacet}
malformedOnly={malformedOnly}
onToggleMalformed={() => setMalformedOnly(m => !m)}
onClear={clearFilters}
hasActiveFilters={hasActiveFilters}
/>
) : (
<div className="catalog-chips">
{STATE_CHIPS.map(chip => (
<button
key={chip.id}
className={`chip ${activeChips.has(chip.id) ? 'active' : ''}`}
onClick={() => toggleChip(chip.id)}
>
{chip.label}
</button>
))}
</div>
)}
{faceted && canContribute && selected.size > 0 && (
<BulkActionBar
fields={fields}
count={selected.size}
onApply={applyBulk}
onClear={() => setSelected(new Set())}
/>
)}
<div className="catalog-list">
{filtered.length === 0 ? (
<div style={{ padding: '24px 14px', color: '#999', fontSize: 13 }}>
{rfcs.length === 0
{loading
? 'Loading…'
// §22.4a SLICE-3: faceted mode with active filters an
// explicitly-clearable "no entries match" state (§5.1).
: faceted && hasActiveFilters ? (
<>
No entries match.{' '}
<button className="facet-clear" onClick={clearFilters}>Clear filters</button>
</>
)
: rfcs.length === 0
// C3.5: a contributor sees a propose-first call to action; a
// granted viewer without contribute rights sees a bare empty
// state; an anonymous reader sees the read-only note (the footer
@@ -137,7 +266,9 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
filtered.map(r => {
const isActive = slug === r.slug
const isSuper = r.state === 'super-draft'
return (
// §22.4a SLICE-5: selectable rows in faceted mode for contributors.
const selectable = faceted && canContribute
const row = (
<Link
key={r.slug}
to={entryPath(pid, r.slug, cid)}
@@ -147,6 +278,9 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
<span className={`row-id ${isSuper ? 'super' : ''}`}>
{isSuper ? 'super-draft' : (r.id || '—')}
</span>
{r.metadata_malformed && (
<span className="row-malformed" title="Metadata fails this collection's schema"> malformed</span>
)}
</div>
<span className="row-title">{r.title}</span>
{r.tags.length > 0 && (
@@ -154,6 +288,19 @@ export default function Catalog({ viewer, onProposeRFC, version }) {
)}
</Link>
)
if (!selectable) return row
return (
<div key={r.slug} className="catalog-row-wrap selectable">
<input
type="checkbox"
className="row-select"
aria-label={`select ${r.title}`}
checked={selected.has(r.slug)}
onChange={() => toggleSelect(r.slug)}
/>
{row}
</div>
)
})
)}
</div>
+138
View File
@@ -0,0 +1,138 @@
import React from 'react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import { MemoryRouter } from 'react-router-dom'
// §22.4a SLICE-3 the faceted catalog pane (PUC-3). Mock the api so
// getCollection drives the field schema and listRFCs returns { items, facets }.
const listRFCs = vi.fn()
const getCollection = vi.fn()
const bulkEntryMeta = vi.fn()
vi.mock('../api', () => ({
listRFCs: (...a) => listRFCs(...a),
listProposals: () => Promise.resolve({ items: [] }),
getCollection: (...a) => getCollection(...a),
bulkEntryMeta: (...a) => bulkEntryMeta(...a),
}))
vi.mock('./ToastHost.jsx', () => ({ showToast: vi.fn() }))
vi.mock('../lib/entryPaths', () => ({
entryPath: () => '/x', proposalPath: () => '/p',
useProjectId: () => 'ohm', useCollectionId: () => 'default',
}))
import Catalog from './Catalog.jsx'
const renderCatalog = () =>
render(<MemoryRouter><Catalog viewer={null} onProposeRFC={() => {}} version={0} /></MemoryRouter>)
describe('Catalog faceted pane', () => {
beforeEach(() => { listRFCs.mockReset(); getCollection.mockReset(); bulkEntryMeta.mockReset() })
it('renders facet groups with counts when the collection declares fields', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0', 'P1'] } },
viewer: { can_contribute: false }, entry_noun: 'RFC',
})
listRFCs.mockResolvedValue({
items: [{ slug: 'a', title: 'A', state: 'active', tags: [], starred_by_me: false }],
facets: { priority: { P0: 3, P1: 1 }, state: { active: 4 } },
})
renderCatalog()
expect(await screen.findByText('P0')).toBeInTheDocument()
expect(screen.getByText('3')).toBeInTheDocument()
})
it('re-fetches with the selection when a facet checkbox is toggled', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0'] } },
viewer: {}, entry_noun: 'RFC',
})
listRFCs.mockResolvedValue({ items: [], facets: { priority: { P0: 2 }, state: {} } })
renderCatalog()
const box = await screen.findByText('P0')
fireEvent.click(box.closest('label').querySelector('input'))
await waitFor(() => {
const last = listRFCs.mock.calls.at(-1)
expect(last[2].selections.priority).toContain('P0')
})
})
it('reveals the bulk action bar when a row is selected (faceted + contributor)', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0', 'P1'] } },
viewer: { can_contribute: true }, entry_noun: 'RFC',
})
listRFCs.mockResolvedValue({
items: [{ slug: 'a', title: 'A', state: 'active', tags: [], starred_by_me: false }],
facets: { priority: { P0: 1 }, state: { active: 1 } },
})
renderCatalog()
const checkbox = await screen.findByLabelText(/select A/i)
fireEvent.click(checkbox)
expect(await screen.findByText(/1 selected/i)).toBeInTheDocument()
})
it('sends a bulk request and re-fetches when a set op is applied', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0', 'P1'] } },
viewer: { can_contribute: true }, entry_noun: 'RFC',
})
listRFCs.mockResolvedValue({
items: [{ slug: 'a', title: 'A', state: 'active', tags: [], starred_by_me: false }],
facets: { priority: { P0: 1 }, state: { active: 1 } },
})
bulkEntryMeta.mockResolvedValue({ applied: ['a'], rejected: [], committed: true })
renderCatalog()
fireEvent.click(await screen.findByLabelText(/select A/i))
fireEvent.change(await screen.findByLabelText(/set priority/i), { target: { value: 'P1' } })
await waitFor(() => {
expect(bulkEntryMeta).toHaveBeenCalledWith('ohm', 'default',
{ slugs: ['a'], op: 'set', field: 'priority', value: 'P1' })
})
})
it('drops a selection when a filter change hides the row', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0', 'P1'] } },
viewer: { can_contribute: true }, entry_noun: 'RFC',
})
// Content-driven: unfiltered row "a"; once the P0 facet is selected the
// server response omits "a" (drives the prune). Robust to extra mount-time
// fetches the collection effect triggers.
listRFCs.mockImplementation((p, c, opts) => Promise.resolve({
items: opts?.selections?.priority?.includes('P0')
? [{ slug: 'b', title: 'B', state: 'active', tags: [], starred_by_me: false }]
: [{ slug: 'a', title: 'A', state: 'active', tags: [], starred_by_me: false }],
facets: { priority: { P0: 1 }, state: { active: 1 } },
}))
const { container } = renderCatalog()
fireEvent.click(await screen.findByLabelText(/select A/i))
expect(await screen.findByText(/1 selected/i)).toBeInTheDocument()
// Toggle the P0 facet checkbox re-fetch returns a list without "a".
fireEvent.click(container.querySelector('.facet-group input[type="checkbox"]'))
await waitFor(() => {
expect(screen.queryByText(/1 selected/i)).not.toBeInTheDocument()
})
})
it('does not offer row selection for a non-contributor', async () => {
getCollection.mockResolvedValue({
fields: { priority: { type: 'enum', values: ['P0'] } },
viewer: { can_contribute: false }, entry_noun: 'RFC',
})
listRFCs.mockResolvedValue({
items: [{ slug: 'a', title: 'A', state: 'active', tags: [], starred_by_me: false }],
facets: { priority: { P0: 1 }, state: { active: 1 } },
})
renderCatalog()
await screen.findByText('A')
expect(screen.queryByLabelText(/select A/i)).not.toBeInTheDocument()
})
it('renders legacy state chips when the collection declares no fields', async () => {
getCollection.mockResolvedValue({ fields: null, viewer: {}, entry_noun: 'RFC' })
listRFCs.mockResolvedValue({ items: [], facets: {} })
renderCatalog()
expect(await screen.findByText('Super-draft')).toBeInTheDocument()
expect(screen.getByText('Active')).toBeInTheDocument()
})
})
+97
View File
@@ -0,0 +1,97 @@
// FacetGroups.jsx §22.4a SLICE-3 faceted left-pane filters (§5.1).
//
// Renders one collapsible group per facet field returned by the API, each with
// per-value result counts and multi-select checkboxes. A `tags`-type field gets
// a "filter values" search box so it stays usable at 30+ values. Selection
// state + the malformed toggle are owned by the parent (Catalog), which
// re-fetches server-side on change.
import { useState } from 'react'
// Human label for the built-in state facet; declared fields use their own name
// (or the schema `label` when the API supplies it on `fields`).
function groupLabel(field, fields) {
if (field === 'state') return 'State'
const def = fields?.[field]
return def?.label || field.charAt(0).toUpperCase() + field.slice(1)
}
function FacetGroup({ field, fields, counts, selected, onToggle, isTags }) {
const [open, setOpen] = useState(true)
const [valueSearch, setValueSearch] = useState('')
const entries = Object.entries(counts).sort((a, b) => b[1] - a[1])
const needle = valueSearch.trim().toLowerCase()
const shown = isTags && needle
? entries.filter(([v]) => v.toLowerCase().includes(needle))
: entries
return (
<div className="facet-group">
<button className="facet-header" onClick={() => setOpen(o => !o)}>
<span>{open ? '▾' : '▸'} {groupLabel(field, fields)}</span>
</button>
{open && (
<div className="facet-values">
{isTags && entries.length > 8 && (
<input
className="facet-value-search"
placeholder="filter values…"
value={valueSearch}
onChange={e => setValueSearch(e.target.value)}
/>
)}
{shown.length === 0 && (
<div className="facet-empty">No values.</div>
)}
{shown.map(([value, count]) => (
<label key={value} className="facet-value">
<input
type="checkbox"
checked={selected.has(value)}
onChange={() => onToggle(field, value)}
/>
<span className="facet-value-label">{value}</span>
<span className="facet-count">{count}</span>
</label>
))}
</div>
)}
</div>
)
}
export default function FacetGroups({
facets, fields, selections, onToggle, malformedOnly, onToggleMalformed,
onClear, hasActiveFilters,
}) {
// The API returns facets in field order with `state` last; render in that
// order. `selections` is { field: Set<string> }.
const fieldOrder = Object.keys(facets || {})
// A `tags`-type field renders the value-search box.
const isTags = (field) => fields?.[field]?.type === 'tags'
return (
<div className="facets">
{hasActiveFilters && (
<button className="facet-clear" onClick={onClear}>Clear filters</button>
)}
{fieldOrder.map(field => (
<FacetGroup
key={field}
field={field}
fields={fields}
counts={facets[field]}
selected={selections[field] || new Set()}
onToggle={onToggle}
isTags={isTags(field)}
/>
))}
<label className="facet-malformed">
<input
type="checkbox"
checked={malformedOnly}
onChange={onToggleMalformed}
/>
<span>Malformed metadata only</span>
</label>
</div>
)
}
@@ -0,0 +1,111 @@
import React, { useState } from 'react'
import { saveEntryMeta } from '../api'
// §22.4a SLICE-4 (PUC-1, UX §5.2): a schema-driven view/edit panel for one
// entry's metadata. One control per declared field enum single-select,
// tags removable chips + add-input, text text input. Authorized users
// (canEdit) save changed values via a direct sidecar commit; the document body
// renders separately as pure prose. A collection with no `fields` renders
// nothing (INV-5: today's behavior unchanged).
const labelFor = (name, def) =>
def?.label || name.charAt(0).toUpperCase() + name.slice(1)
const sameValue = (a, b) => JSON.stringify(a ?? null) === JSON.stringify(b ?? null)
export default function MetadataFieldsPanel({
projectId, collectionId, slug, fields, meta, canEdit,
}) {
const [draft, setDraft] = useState(() => ({ ...(meta || {}) }))
const [saving, setSaving] = useState(false)
const [error, setError] = useState(null)
const [saved, setSaved] = useState(false)
const [tagInput, setTagInput] = useState('')
if (!fields || Object.keys(fields).length === 0) return null
const setField = (name, value) => {
setSaved(false)
setDraft(d => ({ ...d, [name]: value }))
}
const changedFields = Object.keys(fields).filter(
n => !sameValue(draft[n], (meta || {})[n]))
const changed = changedFields.length > 0
const onSave = async () => {
setSaving(true); setError(null); setSaved(false)
const values = {}
for (const n of changedFields) values[n] = draft[n]
try {
await saveEntryMeta(projectId, collectionId, slug, values)
setSaved(true)
} catch (e) {
setError(e.message || 'Could not save metadata')
} finally {
setSaving(false)
}
}
return (
<div className="metadata-fields-panel">
{Object.entries(fields).map(([name, def]) => (
<div key={name} className="metadata-field">
<label htmlFor={`mf-${name}`}>{labelFor(name, def)}</label>
{!canEdit ? (
<ReadOnlyValue type={def.type} value={draft[name]} />
) : def.type === 'enum' ? (
<select id={`mf-${name}`} value={draft[name] ?? ''}
onChange={e => setField(name, e.target.value || null)}>
<option value=""></option>
{(def.values || []).map(v => <option key={v} value={v}>{v}</option>)}
</select>
) : def.type === 'tags' ? (
<div className="tags-edit">
{(draft[name] || []).map(t => (
<span key={t} className="tag-chip">
{t}
<button type="button" aria-label={`remove ${t}`}
onClick={() => setField(name, (draft[name] || []).filter(x => x !== t))}>×</button>
</span>
))}
<input id={`mf-${name}`} value={tagInput}
onChange={e => setTagInput(e.target.value)}
onKeyDown={e => {
if (e.key === 'Enter' && tagInput.trim()) {
e.preventDefault()
const cur = draft[name] || []
if (!cur.includes(tagInput.trim())) setField(name, [...cur, tagInput.trim()])
setTagInput('')
}
}} placeholder="add tag…" />
</div>
) : (
<input id={`mf-${name}`} type="text" value={draft[name] ?? ''}
onChange={e => setField(name, e.target.value || null)} />
)}
</div>
))}
{canEdit && (
<div className="metadata-fields-actions">
<button type="button" onClick={onSave} disabled={saving || !changed}>
{saving ? 'Saving…' : 'Save'}
</button>
{saved && !error && <span className="field-saved">Saved</span>}
{error && <span className="field-error">{error}</span>}
</div>
)}
</div>
)
}
function ReadOnlyValue({ type, value }) {
if (type === 'tags') {
return (
<span>
{(value || []).map(t => <span key={t} className="tag-chip">{t}</span>)}
</span>
)
}
return <span>{value ?? '—'}</span>
}
@@ -0,0 +1,73 @@
import React from 'react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
vi.mock('../api', () => ({ saveEntryMeta: vi.fn() }))
import { saveEntryMeta } from '../api'
import MetadataFieldsPanel from './MetadataFieldsPanel.jsx'
const fields = {
priority: { type: 'enum', values: ['P0', 'P1', 'P2'], label: 'Priority' },
tags: { type: 'tags', label: 'Tags' },
owner: { type: 'text', label: 'Owner' },
}
describe('MetadataFieldsPanel', () => {
beforeEach(() => saveEntryMeta.mockReset())
it('renders nothing when the collection declares no fields', () => {
const { container } = render(
<MetadataFieldsPanel projectId="ohm" collectionId="bdd" slug="a"
fields={null} meta={{}} canEdit />)
expect(container.firstChild).toBeNull()
})
it('renders one control per schema field with current values (read-only)', () => {
render(<MetadataFieldsPanel projectId="ohm" collectionId="bdd" slug="a"
fields={fields} meta={{ priority: 'P1', tags: ['x'], owner: 'sam' }}
canEdit={false} />)
expect(screen.getByText('Priority')).toBeInTheDocument()
expect(screen.getByText('P1')).toBeInTheDocument()
expect(screen.getByText('x')).toBeInTheDocument()
expect(screen.getByText('sam')).toBeInTheDocument()
// read-only: no select control
expect(screen.queryByRole('combobox')).toBeNull()
})
it('edits an enum and saves only the changed field', async () => {
saveEntryMeta.mockResolvedValue({ ok: true, meta: { priority: 'P0' } })
render(<MetadataFieldsPanel projectId="ohm" collectionId="bdd" slug="a"
fields={fields} meta={{ priority: 'P1' }} canEdit />)
fireEvent.change(screen.getByLabelText('Priority'), { target: { value: 'P0' } })
fireEvent.click(screen.getByText('Save'))
await waitFor(() => expect(saveEntryMeta).toHaveBeenCalledWith(
'ohm', 'bdd', 'a', { priority: 'P0' }))
await screen.findByText('Saved')
})
it('disables Save until something changes', () => {
render(<MetadataFieldsPanel projectId="ohm" collectionId="bdd" slug="a"
fields={fields} meta={{ priority: 'P1' }} canEdit />)
expect(screen.getByText('Save')).toBeDisabled()
})
it('adds and removes a tag', async () => {
saveEntryMeta.mockResolvedValue({ ok: true, meta: {} })
render(<MetadataFieldsPanel projectId="ohm" collectionId="bdd" slug="a"
fields={fields} meta={{ tags: ['keep'] }} canEdit />)
const tagInput = screen.getByPlaceholderText('add tag…')
fireEvent.change(tagInput, { target: { value: 'checkout' } })
fireEvent.keyDown(tagInput, { key: 'Enter' })
expect(screen.getByText('checkout')).toBeInTheDocument()
fireEvent.click(screen.getByText('Save'))
await waitFor(() => expect(saveEntryMeta).toHaveBeenCalledWith(
'ohm', 'bdd', 'a', { tags: ['keep', 'checkout'] }))
})
// NOTE: the rejected-save error-display branch is intentionally not unit-tested
// here. A caught rejection inside an event-handler-invoked async function trips
// vitest's per-test unhandled-rejection detector (the happy path proves the
// component+harness wiring works); the server-side 422 validation it surfaces
// is covered by backend test_metadata_edit_endpoint.py. The display itself is a
// trivial `{error && <span>}` branch.
})
+21
View File
@@ -0,0 +1,21 @@
// Client-side 404 for routes that don't match. An SPA serves index.html
// for every non-API path (HTTP 200), so an unknown URL would otherwise
// render blank or, worse, silently bounce home. This surfaces a clear
// "page not found" instead. Used by the top-level router and by the
// nested /admin, /p/:projectId, and /docs route groups so an invalid
// subpath (e.g. /admin/users/allowlist/graduation) lands here too.
import { Link } from 'react-router-dom'
export default function NotFound({ message }) {
return (
<div className="not-found" role="alert">
<h1 className="not-found-code">404</h1>
<p className="not-found-message">
{message || "We couldn't find that page."}
</p>
<p className="not-found-actions">
<Link to="/">Return to the catalog</Link>
</p>
</div>
)
}
+111 -25
View File
@@ -46,7 +46,9 @@ import GraduateDialog from './GraduateDialog.jsx'
import InvitationsModal from './InvitationsModal.jsx'
import { claimOwnership, retireRFC, unretireRFC } from '../api'
import { EVENTS, track } from '../lib/analytics'
import { entryPath, entryPrPath, useProjectId } from '../lib/entryPaths'
import { entryPath, entryPrPath, useProjectId, useCollectionId } from '../lib/entryPaths'
import { getCollection } from '../api'
import MetadataFieldsPanel from './MetadataFieldsPanel.jsx'
const MANUAL_IDLE_MS = 5 * 60 * 1000 // §8.6 idle window; exact value is impl detail.
const MANUAL_DEBOUNCE_MS = 800
@@ -70,10 +72,14 @@ export default function RFCView({ viewer }) {
const [searchParams, setSearchParams] = useSearchParams()
const navigate = useNavigate()
const pid = useProjectId()
const cid = useCollectionId()
const branchParam = searchParams.get('branch') || 'main'
const [entry, setEntry] = useState(null)
// §22.4a SLICE-4: the collection's metadata field schema (null when the
// collection declares none the panel then renders nothing, INV-5).
const [collectionFields, setCollectionFields] = useState(null)
const [mainView, setMainView] = useState(null)
const [branchView, setBranchView] = useState(null)
const [error, setError] = useState(null)
@@ -87,6 +93,15 @@ export default function RFCView({ viewer }) {
// surface to expose.
const editorRef = useRef(null)
const originalSourceLinesRef = useRef([])
// §8.12 Option B asking-while-reading on `main`. When Ask is invoked from
// the canonical view we transparently cut an edit branch, navigate to it, and
// run the question there once its view (and main_thread_id) loads. The pending
// question is stashed here and fired by the pending-ask effect below we
// can't push optimistic chat messages before branchView.main_thread_id exists.
const pendingAskRef = useRef(null) // { text, quote, branch } | null
// Live ref to submitChatTurn so the (intentionally narrow-dep) message-load
// effect can fire the stashed §8.12 main-view ask with the latest closure.
const submitChatTurnRef = useRef(null)
const [editorContent, setEditorContent] = useState('')
// Mirror of the live CM6 doc for the Contribute-mode preview pane,
// debounced so the preview doesn't re-render on every keystroke.
@@ -125,7 +140,14 @@ export default function RFCView({ viewer }) {
const [drawerOpen, setDrawerOpen] = useState(false)
useEffect(() => {
getRFC(pid, slug).then(entry => {
// §22.4a: an entry in a NAMED collection must be fetched collection-
// scoped. Omitting `cid` falls back to the project's DEFAULT-collection
// route (/api/projects/<pid>/rfcs/<slug>), which 404s for an entry that
// lives only in a named collection surfacing as "Error: Not found" and
// a missing metadata panel. (Tier-1 masked this when the same slug was
// also reachable via the default collection; PPE's isolated content
// exposed it.)
getRFC(pid, slug, cid).then(entry => {
setEntry(entry)
// v0.15.0 analytics: fire RFC Viewed once per slug load.
// We key on the slug param rather than the loaded entry so a
@@ -140,7 +162,16 @@ export default function RFCView({ viewer }) {
setSelectedModel(def || models?.[0]?.id || '')
})
.catch(() => {})
}, [slug, pid])
}, [slug, pid, cid])
// §22.4a SLICE-4: load the collection's metadata field schema for the
// detail panel. Independent of the entry load; the panel reads the entry's
// own `meta` values + `can_edit_meta` capability.
useEffect(() => {
getCollection(pid, cid)
.then(c => setCollectionFields(c?.fields || null))
.catch(() => setCollectionFields(null))
}, [pid, cid])
// Per §9.4 / §17's routing-collapse rule, super-drafts render through
// the same surface as active RFCs the bot, the chat, the change
@@ -179,12 +210,12 @@ export default function RFCView({ viewer }) {
setActionError(null)
try {
const res = await unretireRFC(slug)
getRFC(pid, slug).then(setEntry).catch(() => {})
getRFC(pid, slug, cid).then(setEntry).catch(() => {})
if (res?.state) navigate(entryPath(pid, slug))
} catch (err) {
setActionError(err.message)
}
}, [slug, navigate, pid])
}, [slug, navigate, pid, cid])
// Load main view + branch view whenever slug/branch changes.
useEffect(() => {
@@ -199,8 +230,8 @@ export default function RFCView({ viewer }) {
setSelection(null)
setMode('discuss')
getRFCMain(slug).then(setMainView).catch(err => setError(err.message))
getBranch(slug, branchParam)
getRFCMain(slug, pid, cid).then(setMainView).catch(err => setError(err.message))
getBranch(slug, branchParam, pid, cid)
.then(view => {
setBranchView(view)
setEditorContent(view.body || '')
@@ -209,13 +240,27 @@ export default function RFCView({ viewer }) {
setChanges(view.changes || [])
})
.catch(err => setError(err.message))
}, [slug, branchParam, entry])
// §22/G-15: pid+cid scope the body reads; re-run if the collection changes.
}, [slug, branchParam, entry, pid, cid])
// Load chat messages whenever the branch's main thread id resolves.
useEffect(() => {
if (!branchView?.main_thread_id) return
loadAllMessages(slug, branchParam, branchView.threads).then(setMessages)
}, [branchView?.main_thread_id, slug, branchParam])
loadAllMessages(slug, branchParam, branchView.threads).then(loaded => {
setMessages(loaded)
// §8.12 Option B run the stashed main-view question now that the freshly
// cut branch's messages have loaded. Firing here (rather than in a parallel
// effect) appends the turn AFTER the loaded set, so this late message load
// can't clobber the optimistic chat turn. Land in contribute mode to
// surface any edits the turn proposes (AI chat is an editing activity).
const pending = pendingAskRef.current
if (pending && pending.branch === branchView.branch_name && pending.branch === branchParam) {
pendingAskRef.current = null
setMode('contribute')
submitChatTurnRef.current?.(pending.text, pending.quote)
}
})
}, [branchView?.main_thread_id, branchView?.branch_name, slug, branchParam])
// Selection wiring (§8.12). MarkdownPreview reports {text, coords}
// sourced from window.getSelection(); the SelectionTooltip
@@ -244,7 +289,7 @@ export default function RFCView({ viewer }) {
paragraphCount: manualPending?.paragraphCount || 1,
})
if (!res.noop) {
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
setChanges(fresh.changes || [])
setPreviewContent(fresh.body || '')
@@ -312,8 +357,12 @@ export default function RFCView({ viewer }) {
}, [manualCountdown, flushManualBuffer])
// Start contributing
// Returns the branch the viewer should now be on (the freshly cut branch on
// main, or the current branch on a mode-flip), or null if no branch was cut
// (signed-out login redirect, or the server rejected the cut). §8.12's
// main-view Ask reuses this dispatch and consumes the returned branch name.
const handleStartContributing = useCallback(async () => {
if (!viewer) { window.location.href = '/auth/login'; return }
if (!viewer) { window.location.href = '/auth/login'; return null }
if (branchParam === 'main') {
try {
// §9.5 dispatch: super-drafts cut a meta-repo edit branch via
@@ -322,18 +371,34 @@ export default function RFCView({ viewer }) {
? await startEditBranch(slug)
: await promoteToBranch(slug)
setSearchParams({ branch: branch_name })
return branch_name
} catch (err) {
setError(err.message)
return null
}
return
}
// Non-main: pure mode flip per §8.14.
if (pendingDiscussChanges.length > 0) {
setPendingDiscussChanges([])
}
setMode('contribute')
return branchParam
}, [viewer, slug, branchParam, pendingDiscussChanges, setSearchParams, isSuperDraft])
// §8.12 Option B Ask invoked from the canonical `main` view
// AI chat is an editing activity and only runs on an edit branch, so asking
// while reading transparently cuts one (same dispatch as Start Contributing),
// navigates to it, and stashes the question; the pending-ask effect fires it
// once the new branch's view (and its main_thread_id) has resolved. Degrades
// gracefully: signed-out login; a viewer who can't contribute has the cut
// rejected server-side the error surfaces and we neither navigate nor chat.
const handleMainAsk = useCallback(async (text, quote) => {
if (!viewer) { window.location.href = '/auth/login'; return }
const branch = await handleStartContributing()
if (!branch || branch === 'main') return // cut failed setError already ran
pendingAskRef.current = { text, quote: quote || null, branch }
}, [viewer, handleStartContributing])
// Submit a chat turn (prompt bar or selection tooltip)
const submitChatTurn = useCallback(async (text, quote) => {
if (!branchView?.main_thread_id || isStreaming) return
@@ -386,7 +451,7 @@ export default function RFCView({ viewer }) {
))
}
// Re-pull authoritative state: changes have been materialized server-side.
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setChanges(fresh.changes || [])
// If we're in discuss mode and the new turn produced pending AI changes,
// surface them as discuss-mode buffered count.
@@ -404,16 +469,24 @@ export default function RFCView({ viewer }) {
}
}, [slug, branchParam, branchView?.main_thread_id, isStreaming, viewer, selectedModel, mode])
// Keep submitChatTurnRef pointed at the latest submitChatTurn so the
// message-load effect's §8.12 main-view ask uses the current branch closure.
useEffect(() => { submitChatTurnRef.current = submitChatTurn }, [submitChatTurn])
const handlePrompt = useCallback((text, sel) => {
const quote = sel?.text || null
// On the canonical view there is no chat surface; Ask cuts an edit branch
// and runs the question there (§8.12 Option B). On a branch, ask directly.
if (branchParam === 'main') { handleMainAsk(text, quote); return }
submitChatTurn(text, quote)
}, [submitChatTurn])
}, [branchParam, handleMainAsk, submitChatTurn])
const handleTooltipAsk = useCallback(async (textOrNull, quote) => {
if (textOrNull === null) { setSelection(null); return }
setSelection(null)
if (branchParam === 'main') { await handleMainAsk(textOrNull, quote); return }
await submitChatTurn(textOrNull, quote)
}, [submitChatTurn])
}, [branchParam, handleMainAsk, submitChatTurn])
const handleTooltipFlag = useCallback(async (label, quote) => {
if (!viewer) { window.location.href = '/auth/login'; return }
@@ -425,7 +498,7 @@ export default function RFCView({ viewer }) {
anchor_payload: { quote },
label,
})
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
loadAllMessages(slug, branchParam, fresh.threads).then(setMessages)
} catch (err) {
@@ -441,7 +514,7 @@ export default function RFCView({ viewer }) {
// body. The proper tracked-changes overlay lives in Phase 3's
// preview pane; with CM6 as the source editor there is no HTML
// surface to inject `<span class="tracked-*">` into.
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
setChanges(fresh.changes || [])
setEditorContent(fresh.body || '')
@@ -455,7 +528,7 @@ export default function RFCView({ viewer }) {
const handleDecline = useCallback(async (changeId) => {
try {
await apiDecline(slug, branchParam, changeId)
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setChanges(fresh.changes || [])
} catch (err) {
setError(err.message)
@@ -465,7 +538,7 @@ export default function RFCView({ viewer }) {
const handleReask = useCallback(async (changeId) => {
try {
await reaskChange(slug, branchParam, changeId)
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
setChanges(fresh.changes || [])
loadAllMessages(slug, branchParam, fresh.threads).then(setMessages)
@@ -477,7 +550,7 @@ export default function RFCView({ viewer }) {
const handleResolveThread = useCallback(async (threadId) => {
try {
await resolveThread(slug, branchParam, threadId)
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
loadAllMessages(slug, branchParam, fresh.threads).then(setMessages)
} catch (err) {
@@ -747,6 +820,19 @@ export default function RFCView({ viewer }) {
: <span style={{ color: '#999', fontStyle: 'italic' }}>Left blank by the proposer.</span>}
</div>
)}
{/* §22.4a SLICE-4 (PUC-1, UX §5.2): schema-driven metadata panel on
the canonical (main) view. Renders nothing when the collection
declares no `fields` (INV-5). */}
{branchParam === 'main' && collectionFields && (
<MetadataFieldsPanel
projectId={pid}
collectionId={cid}
slug={slug}
fields={collectionFields}
meta={entry.meta || {}}
canEdit={!!entry.can_edit_meta}
/>
)}
{inDiscuss && branchParam !== 'main' && (
<div className="discuss-mode-banner">
Discuss mode on <strong>{branchParam}</strong> chat freely;
@@ -942,7 +1028,7 @@ export default function RFCView({ viewer }) {
current={branchView.visibility}
onClose={() => setShowVisibility(false)}
onSaved={async () => {
const fresh = await getBranch(slug, branchParam)
const fresh = await getBranch(slug, branchParam, pid, cid)
setBranchView(fresh)
setShowVisibility(false)
}}
@@ -957,8 +1043,8 @@ export default function RFCView({ viewer }) {
onCompleted={() => {
setShowGraduateDialog(false)
// The catalog row and the RFC view now reflect `active`.
getRFC(pid, slug).then(setEntry).catch(() => {})
getRFCMain(slug).then(setMainView).catch(() => {})
getRFC(pid, slug, cid).then(setEntry).catch(() => {})
getRFCMain(slug, pid, cid).then(setMainView).catch(() => {})
}}
/>
)}
@@ -981,7 +1067,7 @@ export default function RFCView({ viewer }) {
setShowMetadataPane(false)
// Refresh main view so the new open PR surfaces in the
// breadcrumb meta count immediately.
getRFCMain(slug).then(setMainView).catch(() => {})
getRFCMain(slug, pid, cid).then(setMainView).catch(() => {})
navigate(entryPrPath(pid, slug, prNumber))
}}
/>
+220
View File
@@ -0,0 +1,220 @@
import React from 'react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import { MemoryRouter, Routes, Route } from 'react-router-dom'
// §8.12 Option B asking-while-reading on the canonical `main` view. The AI
// "Ask" affordance (selection tooltip + prompt bar) has no chat surface on
// main (RFCDiscussionPanel renders there, not ChatPanel). These tests cover the
// transparent branch-cut: Ask on main cuts an edit branch, navigates to it, and
// runs the question there; Ask on a branch is unchanged; a rejected cut degrades
// gracefully; Flag on main still opens a discussion thread.
const getRFC = vi.fn()
const getRFCMain = vi.fn()
const getBranch = vi.fn()
const listModels = vi.fn()
const getCollection = vi.fn()
const getThreadMessages = vi.fn()
const promoteToBranch = vi.fn()
const startEditBranch = vi.fn()
const streamChatTurn = vi.fn()
const createThread = vi.fn()
vi.mock('../api', () => ({
getRFC: (...a) => getRFC(...a),
getRFCMain: (...a) => getRFCMain(...a),
getBranch: (...a) => getBranch(...a),
listModels: (...a) => listModels(...a),
getCollection: (...a) => getCollection(...a),
getThreadMessages: (...a) => getThreadMessages(...a),
promoteToBranch: (...a) => promoteToBranch(...a),
startEditBranch: (...a) => startEditBranch(...a),
streamChatTurn: (...a) => streamChatTurn(...a),
createThread: (...a) => createThread(...a),
acceptChange: vi.fn(), declineChange: vi.fn(), editMetadata: vi.fn(),
manualFlush: vi.fn(), reaskChange: vi.fn(), resolveThread: vi.fn(),
setBranchVisibility: vi.fn(), claimOwnership: vi.fn(),
retireRFC: vi.fn(), unretireRFC: vi.fn(),
}))
vi.mock('../lib/entryPaths', () => ({
entryPath: () => '/p/ohm/c/rfc-app/e/the-entry',
entryPrPath: () => '/pr',
useProjectId: () => 'ohm',
useCollectionId: () => 'rfc-app',
}))
vi.mock('../lib/analytics', () => ({ EVENTS: { RFC_VIEWED: 'rfc_viewed' }, track: vi.fn() }))
// Stub the heavy children; expose just the callbacks/data the flow needs.
vi.mock('./MarkdownSourceEditor.jsx', () => ({ default: () => null }))
vi.mock('./MarkdownPreview.jsx', () => ({ default: () => <div data-testid="preview" /> }))
vi.mock('./RFCDiscussionPanel.jsx', () => ({ default: () => <div data-testid="discussion-panel" /> }))
vi.mock('./ChatPanel.jsx', () => ({
default: ({ messages }) => (
<div data-testid="chat-panel">
{(messages || []).map((m, i) => (
<div key={i} data-role={m.role}>{m.text}</div>
))}
</div>
),
}))
vi.mock('./SelectionTooltip.jsx', () => ({
default: ({ onAsk, onFlag, disabled }) => (
<div data-testid="tooltip">
<button data-testid="tooltip-ask" disabled={disabled}
onClick={() => onAsk('Why this approach?', 'the selected quote')}>ask</button>
<button data-testid="tooltip-flag"
onClick={() => onFlag('confusing', 'the selected quote')}>flag</button>
</div>
),
}))
vi.mock('./PromptBar.jsx', () => ({
default: ({ onSubmit, disabled }) => (
<button data-testid="prompt-submit" disabled={disabled}
onClick={() => onSubmit('Why this approach?', { text: 'the selected quote' })}>send</button>
),
}))
vi.mock('./ChangePanel.jsx', () => ({ default: () => null, diffWords: () => [] }))
vi.mock('./PRModal.jsx', () => ({ default: () => null }))
vi.mock('./GraduateDialog.jsx', () => ({ default: () => null }))
vi.mock('./InvitationsModal.jsx', () => ({ default: () => null }))
vi.mock('./MetadataFieldsPanel.jsx', () => ({ default: () => null }))
import RFCView from './RFCView.jsx'
const ACTIVE_ENTRY = {
id: 'RFC-0001', title: 'The Entry', state: 'active',
owners: [], meta: {}, proposed_use_case: '', tags: [],
}
const MAIN_VIEW = { slug: 'the-entry', branches: [], open_prs: [] }
function branchPayload(branch, { main_thread_id = 't-1' } = {}) {
return {
slug: 'the-entry', title: 'The Entry', branch_name: branch, body: '# Body\n',
body_sha: 'abc', main_thread_id, threads: [], changes: [],
visibility: {}, grants: [], creator: 'me',
capabilities: { can_contribute: true, can_change_branch_settings: true },
}
}
const VIEWER = { gitea_login: 'me', role: 'contributor' }
function renderView(viewer = VIEWER, initialBranch = null) {
const path = initialBranch ? `/e/the-entry?branch=${initialBranch}` : '/e/the-entry'
return render(
<MemoryRouter initialEntries={[path]}>
<Routes>
<Route path="/e/:slug" element={<RFCView viewer={viewer} />} />
</Routes>
</MemoryRouter>,
)
}
describe('RFCView — §8.12 main-view Ask (Option B)', () => {
beforeEach(() => {
vi.clearAllMocks()
getRFC.mockResolvedValue(ACTIVE_ENTRY)
getRFCMain.mockResolvedValue(MAIN_VIEW)
listModels.mockResolvedValue({ models: [{ id: 'claude' }], default: 'claude' })
getCollection.mockResolvedValue({ fields: null })
getThreadMessages.mockResolvedValue({ messages: [] })
getBranch.mockImplementation((_slug, branch) => Promise.resolve(branchPayload(branch)))
promoteToBranch.mockResolvedValue({ branch_name: 'edit-me-1' })
startEditBranch.mockResolvedValue({ branch_name: 'edit-the-entry' })
createThread.mockResolvedValue({ thread_id: 1, message_id: 1 })
streamChatTurn.mockImplementation(async (_slug, _branch, _threadId, { text, quote }, cb) => {
cb.onChunk?.('Here is the answer.')
cb.onChanges?.({ message_id: 42 })
cb.onDone?.()
return { assistantId: 42, userMsgId: 41 }
})
})
it('cuts an edit branch and runs the question (with the quote) there', async () => {
renderView()
await screen.findByTestId('discussion-panel') // we start on main
fireEvent.click(screen.getByTestId('prompt-submit'))
// The active-RFC dispatch is promote-to-branch.
await waitFor(() => expect(promoteToBranch).toHaveBeenCalledWith('the-entry'))
// Once the new branch view loads, the turn runs there not on main
// with the selected quote intact.
await waitFor(() => expect(streamChatTurn).toHaveBeenCalled())
const [slug, branch, threadId, payload] = streamChatTurn.mock.calls[0]
expect(slug).toBe('the-entry')
expect(branch).toBe('edit-me-1')
expect(threadId).toBe('t-1')
expect(payload).toMatchObject({ text: 'Why this approach?', quote: 'the selected quote' })
// The chat panel is now mounted (we left main) and shows the streamed answer.
const panel = await screen.findByTestId('chat-panel')
await waitFor(() => expect(panel).toHaveTextContent('Here is the answer.'))
})
it('uses start-edit-branch for a super-draft entry', async () => {
getRFC.mockResolvedValue({ ...ACTIVE_ENTRY, state: 'super-draft' })
getBranch.mockImplementation((_slug, branch) =>
Promise.resolve(branchPayload(branch === 'main' ? 'main' : 'edit-the-entry')))
renderView()
await screen.findByTestId('discussion-panel')
fireEvent.click(screen.getByTestId('tooltip-ask'))
await waitFor(() => expect(startEditBranch).toHaveBeenCalledWith('the-entry'))
expect(promoteToBranch).not.toHaveBeenCalled()
await waitFor(() => expect(streamChatTurn).toHaveBeenCalled())
expect(streamChatTurn.mock.calls[0][1]).toBe('edit-the-entry')
})
it('asking from an existing branch runs directly, without cutting a new one', async () => {
renderView(VIEWER, 'edit-me-1')
await screen.findByTestId('chat-panel')
fireEvent.click(screen.getByTestId('prompt-submit'))
await waitFor(() => expect(streamChatTurn).toHaveBeenCalled())
expect(promoteToBranch).not.toHaveBeenCalled()
expect(startEditBranch).not.toHaveBeenCalled()
expect(streamChatTurn.mock.calls[0][1]).toBe('edit-me-1')
})
it('surfaces an error and does not chat when the branch cut is rejected', async () => {
promoteToBranch.mockRejectedValue(new Error('not a contributor'))
renderView()
await screen.findByTestId('discussion-panel')
fireEvent.click(screen.getByTestId('prompt-submit'))
await waitFor(() => expect(promoteToBranch).toHaveBeenCalled())
// Explicit error path not a silent no-op and no spurious chat turn.
expect(await screen.findByText(/not a contributor/)).toBeInTheDocument()
expect(streamChatTurn).not.toHaveBeenCalled()
})
it('Flag on the canonical view still creates a discussion thread (no branch cut)', async () => {
renderView()
await screen.findByTestId('discussion-panel')
fireEvent.click(screen.getByTestId('tooltip-flag'))
await waitFor(() => expect(createThread).toHaveBeenCalled())
const [, branch, body] = createThread.mock.calls[0]
expect(branch).toBe('main')
expect(body).toMatchObject({ thread_kind: 'flag' })
expect(promoteToBranch).not.toHaveBeenCalled()
expect(streamChatTurn).not.toHaveBeenCalled()
})
it('signed-out viewer gets a read-only path on main — no prompt bar, Ask disabled, no cut', async () => {
renderView(null)
await screen.findByTestId('discussion-panel')
expect(screen.queryByTestId('prompt-submit')).toBeNull()
expect(screen.getByTestId('tooltip-ask')).toBeDisabled()
expect(promoteToBranch).not.toHaveBeenCalled()
expect(startEditBranch).not.toHaveBeenCalled()
})
})
+12 -2
View File
@@ -4,7 +4,7 @@
// project-scoped, so the builders emit the default collection segment; the
// collection-aware link layer (named collections) lands in S2. Components build
// links via these helpers so that flip happens in one place.
import { useParams } from 'react-router-dom'
import { useParams, useLocation } from 'react-router-dom'
import { useProject } from '../components/ProjectLayout.jsx'
import { useDeployment } from '../context/DeploymentProvider'
@@ -41,7 +41,17 @@ export function useProjectId() {
// §22 S2 — the collection id a component should scope to: the `/c/:collectionId/`
// route segment when present, else the project's default collection.
//
// The route param is only in scope for components rendered *under* the
// `c/:collectionId` route (e.g. RFCView). The Catalog renders one level up at
// `/p/:projectId/*` — a sibling of that route — so `useParams().collectionId`
// is undefined there and it would wrongly fall back to the default collection
// (the faceted/bulk catalog UI would then never scope to a named collection).
// Fall back to parsing the `/c/<id>/` segment from the pathname so the Catalog
// scopes correctly regardless of its position in the route tree.
export function useCollectionId() {
const { collectionId } = useParams()
return collectionId || DEFAULT_COLLECTION
const { pathname } = useLocation()
const fromPath = pathname.match(/\/c\/([^/]+)/)
return collectionId || (fromPath && fromPath[1]) || DEFAULT_COLLECTION
}
+42
View File
@@ -0,0 +1,42 @@
// Regression: useCollectionId must resolve the /c/<id>/ segment even when the
// component renders ABOVE the `c/:collectionId` route (the Catalog's position
// at `/p/:projectId/*`), where useParams() does not expose collectionId. A bug
// here makes the faceted/bulk catalog silently scope to the default collection.
import { describe, it, expect } from 'vitest'
import { renderHook } from '@testing-library/react'
import { MemoryRouter, Routes, Route } from 'react-router-dom'
import { useCollectionId } from './entryPaths.js'
function wrapperFor(initialPath, routePath) {
return ({ children }) => (
<MemoryRouter initialEntries={[initialPath]}>
<Routes>
<Route path={routePath} element={children} />
</Routes>
</MemoryRouter>
)
}
describe('useCollectionId', () => {
it('reads the /c/<id>/ segment from the path at the Catalog level (no param in scope)', () => {
// The Catalog matches `/p/:projectId/*` collectionId is NOT a param here.
const { result } = renderHook(() => useCollectionId(), {
wrapper: wrapperFor('/p/ohm/c/bdd/e/checkout', '/p/:projectId/*'),
})
expect(result.current).toBe('bdd')
})
it('honours the route param when in scope', () => {
const { result } = renderHook(() => useCollectionId(), {
wrapper: wrapperFor('/p/ohm/c/features/e/login', '/p/:projectId/c/:collectionId/*'),
})
expect(result.current).toBe('features')
})
it('falls back to the default collection when there is no /c/ segment', () => {
const { result } = renderHook(() => useCollectionId(), {
wrapper: wrapperFor('/p/ohm', '/p/:projectId/*'),
})
expect(result.current).toBe('default')
})
})
+10 -1
View File
@@ -3,7 +3,8 @@ GITEA_BOT_USER=rfc-bot
GITEA_BOT_TOKEN=tier1-bot-token-PLACEHOLDER
GITEA_ORG=wiggleverse
META_REPO=ohm-content
REGISTRY_REPO=
REGISTRY_REPO=rfc-registry
DEFAULT_PROJECT_ID=ohm
OAUTH_CLIENT_ID=tier1-oauth-client-PLACEHOLDER
OAUTH_CLIENT_SECRET=tier1-oauth-secret-PLACEHOLDER
APP_URL=http://localhost:8080
@@ -19,3 +20,11 @@ EMAIL_FROM=rfc@example.test
EMAIL_FROM_NAME=RFC Tier1
EMAIL_ENABLED=true
TURNSTILE_REQUIRED=false
# Tier-1/e2e: disable the per-email OTC request cooldown so a test can sign the
# same account in more than once across specs without 429s.
OTC_REQUEST_COOLDOWN_SECONDS=0
# Tier-1/e2e drives the auth endpoints repeatedly from one IP; lift the per-IP
# sliding-window budgets well above a single suite run (prod leaves these unset
# and keeps the secure defaults).
RATELIMIT_OTC_REQUEST_MAX=1000
RATELIMIT_VERIFY_MAX=1000
+25
View File
@@ -57,6 +57,7 @@ services:
restart: "no"
backend:
image: rfc-tier1-backend
build:
context: ..
dockerfile: testing/backend.Dockerfile
@@ -74,6 +75,30 @@ services:
timeout: 3s
retries: 30
# Insert a granted deployment-owner user keyed by a known e2e email, so the
# OTC sign-in path (which provisions only `pending` contributors) yields an
# owner who can exercise the metadata edit/bulk write paths (SLICE-4/5). The
# owner identity (gitea_login='owner') matches OWNER_GITEA_LOGIN. Idempotent.
backend-seed:
image: rfc-tier1-backend
depends_on:
backend:
condition: service_healthy
volumes:
- backend-data:/data
entrypoint: ["python", "-c"]
command:
- |
import sqlite3
c = sqlite3.connect("/data/rfc-app.db")
c.execute("""INSERT INTO users
(gitea_id, gitea_login, email, display_name, avatar_url, role, permission_state)
SELECT 9001, 'owner', 'e2e-owner@example.test', 'E2E Owner', '', 'owner', 'granted'
WHERE NOT EXISTS (SELECT 1 FROM users WHERE email='e2e-owner@example.test' COLLATE NOCASE)""")
c.commit(); c.close()
print("backend-seed: owner user ready")
restart: "no"
web:
build:
context: ..
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
set -euo pipefail
# One-shot resume for the §9 PPE deployed-environment E2E stage.
#
# PRECONDITION: the operator has run the interactive Workspace reauth:
# gcloud auth login && gcloud auth application-default login
# (Only they can — the gcloud CLI creds expire under the Workspace session
# policy even when ADC is valid.)
#
# This script then runs the whole pipeline non-interactively:
# 1. read the bot token from Secret Manager (never echoed) and use it to
# create + seed the dedicated PPE registry + content repos;
# 2. ensure the E2E test-auth shared secret exists (generates one if not);
# 3. deploy rfc-app-ppe via flotilla-core (pins .rfc-app-version.ppe=0.52.0);
# 4. wait for /api/health to report the expected version, then for the
# reconciler to sync the seeded bdd collection into the cache;
# 5. run metadata.spec.js (SLICE-3/4/5) against the deployed PPE host.
#
# Idempotent: re-running re-seeds (RESEED=1 restores SLICE-4/5 preconditions),
# reuses the existing E2E secret, and redeploys.
REPO_ROOT="$HOME/git/wiggleverse.org/ben.stull/rfc-app"
FLOTILLA="$HOME/git/wiggleverse.org/wiggleverse/flotilla-core/.venv/bin/flotilla-core"
PPE_HOST="https://rfc-ppe.wiggleverse.org"
EXPECT_VERSION="$(cat "$REPO_ROOT/VERSION")"
BOT_SECRET_PROJECT="wiggleverse-ohm"
BOT_SECRET_ID="ohm-rfc-app-gitea-bot-token"
E2E_SECRET_PROJECT="rfc-app-ppe"
E2E_SECRET_ID="rfc-app-ppe-e2e-test-auth-secret"
E2E_EMAIL="e2e-owner@example.test"
export CLOUDSDK_ACTIVE_CONFIG_NAME="rfc-app-ppe"
echo "== 0. precheck gcloud reauth =="
if ! gcloud secrets list --project="$E2E_SECRET_PROJECT" --limit=1 >/dev/null 2>&1; then
echo "gcloud is not reauthed. Run: gcloud auth login && gcloud auth application-default login" >&2
exit 1
fi
echo "gcloud OK"
echo "== 1. create + seed PPE repos (Keychain admin token; never echoed) =="
# Seeding CREATES the two org repos (rfc-registry-ppe, rfc-app-ppe-content),
# which needs a write:organization-scoped token. The SM bot token is
# write:repository only (org create → 403), so use the operator's Keychain
# admin PAT (wgl-gitea-token-<host>, legacy fallback ohm-gitea-token). The
# token stays in the env var — never echoed (§6.3).
SEED_TOKEN="$(security find-generic-password -s "wgl-gitea-token-git.wiggleverse.org" -w 2>/dev/null \
|| security find-generic-password -s "ohm-gitea-token" -w 2>/dev/null)"
[ -n "$SEED_TOKEN" ] || { echo "no Keychain Gitea token found" >&2; exit 1; }
GITEA_TOKEN="$SEED_TOKEN" \
RESEED="${RESEED:-1}" \
bash "$REPO_ROOT/testing/seed-ppe.sh"
unset SEED_TOKEN GITEA_TOKEN
echo "== 2. ensure E2E test-auth secret exists =="
if gcloud secrets describe "$E2E_SECRET_ID" --project="$E2E_SECRET_PROJECT" >/dev/null 2>&1; then
echo "E2E secret already exists; ensuring binding"
"$FLOTILLA" secret bind rfc-app-ppe E2E_TEST_AUTH_SECRET "$E2E_SECRET_PROJECT/$E2E_SECRET_ID@latest"
else
echo "creating E2E secret (random, via stdin — bytes never echoed)"
# `printf %s "$(...)"` stores EXACTLY 64 hex bytes with NO trailing newline.
# A bare `openssl rand -hex 32 | ...` stores 65 bytes (the trailing \n),
# which then rode into the VM .env and made the server's secret differ from
# the runner's command-substitution-stripped value → /auth/test/login 404
# (compare_digest mismatch). Keep it newline-free.
printf '%s' "$(openssl rand -hex 32)" | "$FLOTILLA" secret set rfc-app-ppe E2E_TEST_AUTH_SECRET
fi
echo "== 3. deploy rfc-app-ppe =="
"$FLOTILLA" deploy rfc-app-ppe
echo "== 4a. verify /api/health reports $EXPECT_VERSION =="
ok=0
for _ in $(seq 1 24); do
body="$(curl -s "$PPE_HOST/api/health" || true)"
echo " health: $body"
if printf '%s' "$body" | grep -q "\"version\":\"$EXPECT_VERSION\""; then ok=1; break; fi
sleep 5
done
[ "$ok" = 1 ] || { echo "health never reported $EXPECT_VERSION" >&2; exit 1; }
echo "== 4b. wait for the seeded bdd collection to sync into the cache =="
ok=0
for _ in $(seq 1 40); do
body="$(curl -s "$PPE_HOST/api/projects/ohm/collections/bdd/rfcs" || true)"
n="$(printf '%s' "$body" | grep -o 'checkout-guest\|checkout-returning\|search-facets' | sort -u | wc -l | tr -d ' ')"
echo " synced entries: $n/3"
if [ "$n" = 3 ]; then ok=1; break; fi
sleep 6
done
[ "$ok" = 1 ] || { echo "bdd collection never synced 3 entries" >&2; exit 1; }
echo "== 5. run metadata.spec.js against PPE =="
E2E_SECRET="$(gcloud secrets versions access latest --secret="$E2E_SECRET_ID" --project="$E2E_SECRET_PROJECT")"
cd "$REPO_ROOT/e2e"
BASE_URL="$PPE_HOST" \
E2E_TEST_AUTH_SECRET="$E2E_SECRET" \
E2E_OWNER_EMAIL="$E2E_EMAIL" \
npx playwright test metadata.spec.js
echo "== DONE: PPE E2E complete =="
+123 -27
View File
@@ -1,14 +1,21 @@
#!/usr/bin/env sh
set -eu
# Tier-1 seed (§22-current). Stands up Gitea content + registry so the current
# three-tier app boots, plus a faceted **named collection** (a `.collection.yaml`
# with a `fields:` schema + entries carrying metadata) so the §22.4a metadata
# UI — faceted filter (SLICE-3), edit panel (SLICE-4), bulk bar (SLICE-5) — can
# be exercised end-to-end in a real browser.
GITEA="${GITEA_URL:-http://gitea:3000}"
ADMIN_USER="${GITEA_ADMIN_USER:-giteaadmin}"
ADMIN_PASS="${GITEA_ADMIN_PASSWORD:-giteaadmin-pass}"
ADMIN_EMAIL="${GITEA_ADMIN_EMAIL:-admin@example.test}"
ORG="${GITEA_ORG:-wiggleverse}"
BOT_USER="${GITEA_BOT_USER:-rfc-bot}"
BOT_PASS="${GITEA_BOT_PASSWORD:-rfc-bot-pass}"
CONTENT_REPO="${META_REPO:-ohm-content}"
REGISTRY_REPO="${REGISTRY_REPO:-rfc-registry}"
DEFAULT_PROJECT_ID="${DEFAULT_PROJECT_ID:-ohm}"
APP_URL="${APP_URL:-http://localhost:8080}"
WEBHOOK_SECRET="${GITEA_WEBHOOK_SECRET:-tier1-webhook-secret}"
OUT="${SEED_OUT:-/seed/.env.tier1.generated}"
@@ -22,6 +29,21 @@ done
auth_admin() { curl -sf -u "$ADMIN_USER:$ADMIN_PASS" "$@"; }
# Idempotency guard: if a prior run already wrote a bot token that still works
# against THIS gitea (the registry is readable), the stack is already seeded —
# skip entirely. This makes a second invocation a true no-op, so a later
# dependency-triggered re-run can't delete/remint the token the backend is
# already using (that mismatch 401s the registry mirror). A fresh `down -v`
# brings up a new gitea where the stale token fails, so the seed re-runs.
if [ -f "$OUT" ]; then
EXIST_TOK=$(sed -n 's/^GITEA_BOT_TOKEN=//p' "$OUT")
if [ -n "$EXIST_TOK" ] && curl -sf -H "Authorization: token $EXIST_TOK" \
"$GITEA/api/v1/repos/$ORG/$REGISTRY_REPO/contents/projects.yaml?ref=main" >/dev/null 2>&1; then
echo "seed: existing token valid and registry present — already seeded, skipping"
exit 0
fi
fi
echo "seed: ensuring bot user"
auth_admin -X POST "$GITEA/api/v1/admin/users" \
-H 'Content-Type: application/json' \
@@ -34,55 +56,129 @@ auth_admin -X POST "$GITEA/api/v1/admin/users" \
-d "{\"username\":\"owner\",\"email\":\"owner@example.test\",\"password\":\"owner-pass\",\"must_change_password\":false}" \
|| echo "seed: owner exists, continuing"
echo "seed: minting bot access token"
echo "seed: minting bot access token (drop any prior 'tier1-bot' first — idempotent)"
curl -s -u "$BOT_USER:$BOT_PASS" -X DELETE "$GITEA/api/v1/users/$BOT_USER/tokens/tier1-bot" >/dev/null 2>&1 || true
TOKEN=$(curl -sf -u "$BOT_USER:$BOT_PASS" -X POST "$GITEA/api/v1/users/$BOT_USER/tokens" \
-H 'Content-Type: application/json' \
-d '{"name":"tier1-bot","scopes":["write:repository","write:organization","write:user","write:admin"]}' \
| sed -n 's/.*"sha1":"\([^"]*\)".*/\1/p')
[ -n "$TOKEN" ] || { echo "seed: failed to mint bot token" ; exit 1; }
api() { curl -s -H "Authorization: token $TOKEN" "$@"; }
echo "seed: ensuring org $ORG (owned by bot)"
curl -sf -H "Authorization: token $TOKEN" -X POST "$GITEA/api/v1/orgs" \
-H 'Content-Type: application/json' \
-d "{\"username\":\"$ORG\"}" || echo "seed: org exists, continuing"
api -X POST "$GITEA/api/v1/orgs" -H 'Content-Type: application/json' \
-d "{\"username\":\"$ORG\"}" >/dev/null || echo "seed: org exists, continuing"
echo "seed: ensuring content repo $ORG/$CONTENT_REPO"
curl -sf -H "Authorization: token $TOKEN" -X POST "$GITEA/api/v1/orgs/$ORG/repos" \
-H 'Content-Type: application/json' \
-d "{\"name\":\"$CONTENT_REPO\",\"auto_init\":true,\"default_branch\":\"main\"}" \
|| echo "seed: content repo exists, continuing"
ensure_repo() {
api -X POST "$GITEA/api/v1/orgs/$ORG/repos" -H 'Content-Type: application/json' \
-d "{\"name\":\"$1\",\"auto_init\":true,\"default_branch\":\"main\"}" >/dev/null \
|| echo "seed: repo $1 exists, continuing"
}
echo "seed: seeding one entry under rfcs/ so the catalog is non-empty"
B64=$(printf '%s' '---
# put_file <repo> <path> <plaintext>
put_file() {
_b64=$(printf '%s' "$3" | base64 | tr -d '\n')
api -X POST "$GITEA/api/v1/repos/$ORG/$1/contents/$2" \
-H 'Content-Type: application/json' \
-d "{\"message\":\"seed $2\",\"content\":\"$_b64\",\"branch\":\"main\"}" >/dev/null \
|| echo "seed: $1/$2 exists, continuing"
}
register_webhook() {
api -X POST "$GITEA/api/v1/repos/$ORG/$1/hooks" \
-H 'Content-Type: application/json' \
-d "{\"type\":\"gitea\",\"active\":true,\"events\":[\"push\",\"pull_request\"],\"config\":{\"url\":\"http://backend:8000/api/webhooks/gitea\",\"content_type\":\"json\",\"secret\":\"$WEBHOOK_SECRET\"}}" >/dev/null \
|| echo "seed: webhook on $1 exists, continuing"
}
echo "seed: ensuring content repo $ORG/$CONTENT_REPO and registry $ORG/$REGISTRY_REPO"
ensure_repo "$CONTENT_REPO"
ensure_repo "$REGISTRY_REPO"
echo "seed: registry projects.yaml (default project '$DEFAULT_PROJECT_ID')"
put_file "$REGISTRY_REPO" "projects.yaml" "deployment:
name: Tier1 RFC
tagline: Tier-1 end-to-end deployment
projects:
- id: $DEFAULT_PROJECT_ID
name: OHM
type: document
content_repo: $CONTENT_REPO
visibility: public
"
echo "seed: default-collection entry under rfcs/ (no fields — legacy/document path)"
put_file "$CONTENT_REPO" "rfcs/intro.md" "---
slug: intro
title: Intro
status: graduated
state: active
id: RFC-0001
owners: [owner]
---
# Intro
Seed entry for Tier-1 e2e.
' | base64 | tr -d '\n')
curl -s -H "Authorization: token $TOKEN" -X POST \
"$GITEA/api/v1/repos/$ORG/$CONTENT_REPO/contents/rfcs/intro.md" \
-H 'Content-Type: application/json' \
-d "{\"message\":\"seed intro\",\"content\":\"$B64\",\"branch\":\"main\"}" \
|| echo "seed: intro.md exists, continuing"
Seed entry for the default (document) collection.
"
echo "seed: faceted named collection 'bdd' with a fields: schema (§22.4a)"
put_file "$CONTENT_REPO" "bdd/.collection.yaml" "type: bdd
visibility: public
name: BDD Scenarios
fields:
priority:
type: enum
values: [P0, P1, P2]
label: Priority
tags:
type: tags
label: Tags
"
# Three entries with varied priority/tags so facets have counts and the bulk
# bar has multiple selectable rows.
put_file "$CONTENT_REPO" "bdd/rfcs/checkout-guest.md" "---
slug: checkout-guest
title: Guest checkout
state: active
priority: P0
tags: [checkout, payments]
---
Guest checkout scenario.
"
put_file "$CONTENT_REPO" "bdd/rfcs/checkout-returning.md" "---
slug: checkout-returning
title: Returning-customer checkout
state: active
priority: P1
tags: [checkout]
---
Returning-customer checkout scenario.
"
put_file "$CONTENT_REPO" "bdd/rfcs/search-facets.md" "---
slug: search-facets
title: Faceted search
state: active
priority: P0
tags: [search]
---
Faceted search scenario.
"
echo "seed: registering OAuth application"
OAUTH_JSON=$(curl -sf -u "$ADMIN_USER:$ADMIN_PASS" -X POST "$GITEA/api/v1/user/applications/oauth2" \
OAUTH_JSON=$(auth_admin -X POST "$GITEA/api/v1/user/applications/oauth2" \
-H 'Content-Type: application/json' \
-d "{\"name\":\"rfc-app-tier1\",\"redirect_uris\":[\"$APP_URL/auth/callback\"],\"confidential_client\":true}")
CLIENT_ID=$(printf '%s' "$OAUTH_JSON" | sed -n 's/.*"client_id":"\([^"]*\)".*/\1/p')
CLIENT_SECRET=$(printf '%s' "$OAUTH_JSON" | sed -n 's/.*"client_secret":"\([^"]*\)".*/\1/p')
echo "seed: registering webhook on content repo -> backend"
curl -s -H "Authorization: token $TOKEN" -X POST \
"$GITEA/api/v1/repos/$ORG/$CONTENT_REPO/hooks" \
-H 'Content-Type: application/json' \
-d "{\"type\":\"gitea\",\"active\":true,\"events\":[\"push\",\"pull_request\"],\"config\":{\"url\":\"http://backend:8000/api/webhooks/gitea\",\"content_type\":\"json\",\"secret\":\"$WEBHOOK_SECRET\"}}" \
|| echo "seed: webhook exists, continuing"
echo "seed: registering webhooks (content + registry) -> backend"
register_webhook "$CONTENT_REPO"
register_webhook "$REGISTRY_REPO"
echo "seed: writing generated env to $OUT"
cat > "$OUT" <<EOF
+209
View File
@@ -0,0 +1,209 @@
#!/usr/bin/env sh
set -eu
# PPE E2E content seed (§9 deployed-environment harness).
#
# The Tier-1 docker seed (seed-gitea.sh) stands up a throwaway Gitea. PPE
# instead runs against the REAL Gitea (git.wiggleverse.org) — which it
# shares with prod. To exercise the §22.4a metadata UI (faceted filter
# SLICE-3, edit panel SLICE-4, bulk bar SLICE-5) on PPE WITHOUT touching
# real OHM content, this script seeds a DEDICATED, PPE-only registry +
# content repo:
#
# wiggleverse/rfc-registry-ppe — PPE's own project registry (prod
# keeps using wiggleverse/rfc-registry,
# so prod is never affected).
# wiggleverse/rfc-app-ppe-content — content for the one project the PPE
# registry describes: a default
# (document) collection + a faceted
# `bdd` named collection with a
# fields: schema + three entries.
#
# Point PPE at the PPE registry with:
# flotilla-core overlay set rfc-app-ppe REGISTRY_REPO=rfc-registry-ppe
# The app's startup reconciler sweep loads this content into cached_rfcs
# on the next deploy/restart (no webhook needed for the initial load).
#
# Auth: pass a Gitea token with org repo-create + content-write scope via
# GITEA_TOKEN (the wiggleverse admin token — see the wgl-gitea-admin
# skill; never echo it). The collection path the E2E suite hits is
# /p/ohm/c/bdd, so the seeded project id is `ohm` (matching the Tier-1
# seed and DEFAULT_PROJECT_ID=ohm) and the collection is `bdd`.
#
# Idempotent: repos/files that already exist are left as-is. Pass
# RESEED=1 to force-overwrite the three entry files back to their seed
# values (so a re-run restores SLICE-4/5's expected preconditions).
GITEA="${GITEA_URL:-https://git.wiggleverse.org}"
ORG="${GITEA_ORG:-wiggleverse}"
REGISTRY_REPO="${REGISTRY_REPO:-rfc-registry-ppe}"
CONTENT_REPO="${CONTENT_REPO:-rfc-app-ppe-content}"
PROJECT_ID="${DEFAULT_PROJECT_ID:-ohm}"
TOKEN="${GITEA_TOKEN:?set GITEA_TOKEN to a wiggleverse-org admin/bot token (do not echo it)}"
RESEED="${RESEED:-0}"
api() { curl -s -H "Authorization: token $TOKEN" "$@"; }
# mutate <method> <url> <json> <ok_code> <label>
# Performs an authenticated write and FAILS LOUDLY on any non-<ok_code>
# response. `api` uses `curl -s` (no -f), so without this a 403/409/etc.
# returns exit 0 with an error JSON body — which once let a swallowed 403
# (org-repo create needs write:organization) sail past as "created…" and
# only surfaced as a 502 at deploy time. Never let an HTTP error be silent.
mutate() {
_m="$1"; _u="$2"; _d="$3"; _ok="$4"; _lbl="$5"
_resp=$(api -X "$_m" "$_u" -H 'Content-Type: application/json' -d "$_d" -w '\n%{http_code}')
_code=$(printf '%s' "$_resp" | tail -n1)
if [ "$_code" != "$_ok" ]; then
echo "seed-ppe: $_lbl FAILED (http $_code): $(printf '%s' "$_resp" | sed '$d' | head -c 300)" >&2
exit 1
fi
}
echo "seed-ppe: target $GITEA org=$ORG registry=$REGISTRY_REPO content=$CONTENT_REPO project=$PROJECT_ID"
ensure_repo() {
if api -o /dev/null -w '%{http_code}' "$GITEA/api/v1/repos/$ORG/$1" | grep -q '^200$'; then
echo "seed-ppe: repo $ORG/$1 exists"
return 0
fi
echo "seed-ppe: creating repo $ORG/$1 (private)"
mutate POST "$GITEA/api/v1/orgs/$ORG/repos" \
"{\"name\":\"$1\",\"auto_init\":true,\"default_branch\":\"main\",\"private\":true}" \
201 "create repo $ORG/$1 (org-repo create needs a write:organization token)"
}
# file_sha <repo> <path> -> prints the blob sha if the file exists, else empty
file_sha() {
api "$GITEA/api/v1/repos/$ORG/$1/contents/$2?ref=main" \
| sed -n 's/.*"sha":"\([0-9a-f]*\)".*/\1/p' | head -1
}
# put_file <repo> <path> <plaintext> [force]
# Creates the file if absent. If it exists: skipped, unless force=1, in
# which case it is updated in place (PUT with the current sha).
put_file() {
_repo="$1"; _path="$2"; _content="$3"; _force="${4:-0}"
_b64=$(printf '%s' "$_content" | base64 | tr -d '\n')
_sha=$(file_sha "$_repo" "$_path")
if [ -n "$_sha" ]; then
if [ "$_force" = "1" ]; then
echo "seed-ppe: updating $_repo/$_path"
mutate PUT "$GITEA/api/v1/repos/$ORG/$_repo/contents/$_path" \
"{\"message\":\"reseed $_path\",\"content\":\"$_b64\",\"sha\":\"$_sha\",\"branch\":\"main\"}" \
200 "update $_repo/$_path"
else
echo "seed-ppe: $_repo/$_path exists, leaving as-is"
fi
return 0
fi
echo "seed-ppe: creating $_repo/$_path"
mutate POST "$GITEA/api/v1/repos/$ORG/$_repo/contents/$_path" \
"{\"message\":\"seed $_path\",\"content\":\"$_b64\",\"branch\":\"main\"}" \
201 "create $_repo/$_path"
}
# delete_file <repo> <path> — remove the file if it exists (no-op if absent).
delete_file() {
_repo="$1"; _path="$2"
_sha=$(file_sha "$_repo" "$_path")
[ -n "$_sha" ] || { echo "seed-ppe: $_repo/$_path absent, nothing to delete"; return 0; }
echo "seed-ppe: deleting $_repo/$_path"
mutate DELETE "$GITEA/api/v1/repos/$ORG/$_repo/contents/$_path" \
"{\"message\":\"reseed: drop sidecar $_path\",\"sha\":\"$_sha\",\"branch\":\"main\"}" \
200 "delete $_repo/$_path"
}
ensure_repo "$REGISTRY_REPO"
ensure_repo "$CONTENT_REPO"
# The PPE registry describes a single project `ohm` whose content lives in
# the dedicated PPE content repo.
put_file "$REGISTRY_REPO" "projects.yaml" "deployment:
name: RFC PPE
tagline: rfc-app pre-prod (E2E fixtures)
projects:
- id: $PROJECT_ID
name: OHM
type: document
content_repo: $CONTENT_REPO
visibility: public
"
# Default (document) collection — one entry under rfcs/.
put_file "$CONTENT_REPO" "rfcs/intro.md" "---
slug: intro
title: Intro
state: active
id: RFC-0001
owners: [ben.stull]
---
# Intro
Seed entry for the default (document) collection on PPE.
"
# Faceted named collection 'bdd' with a fields: schema (§22.4a).
put_file "$CONTENT_REPO" "bdd/.collection.yaml" "type: bdd
visibility: public
name: BDD Scenarios
fields:
priority:
type: enum
values: [P0, P1, P2]
label: Priority
tags:
type: tags
label: Tags
"
# Three entries with varied priority/tags so facets have counts and the
# bulk bar has multiple selectable rows. Force-overwritten when RESEED=1
# so a re-run restores SLICE-4 (checkout-returning starts P1) and SLICE-5
# (two P0 entries) preconditions.
put_file "$CONTENT_REPO" "bdd/rfcs/checkout-guest.md" "---
slug: checkout-guest
title: Guest checkout
state: active
priority: P0
tags: [checkout, payments]
---
Guest checkout scenario.
" "$RESEED"
put_file "$CONTENT_REPO" "bdd/rfcs/checkout-returning.md" "---
slug: checkout-returning
title: Returning-customer checkout
state: active
priority: P1
tags: [checkout]
---
Returning-customer checkout scenario.
" "$RESEED"
put_file "$CONTENT_REPO" "bdd/rfcs/search-facets.md" "---
slug: search-facets
title: Faceted search
state: active
priority: P0
tags: [search]
---
Faceted search scenario.
" "$RESEED"
# RESEED also drops any metadata sidecars (<slug>.meta.yaml) left by prior
# SLICE-4/5 edits. A sidecar takes precedence over the .md frontmatter
# (dual-read, §22.4a), so without this a re-run inherits the edited
# priorities (everything ends up P1) and the faceted preconditions drift —
# SLICE-3 expects P0 count = 2. Dropping the sidecars restores the
# frontmatter as the source of truth: checkout-guest=P0, search-facets=P0,
# checkout-returning=P1.
if [ "$RESEED" = "1" ]; then
for _slug in checkout-guest checkout-returning search-facets; do
delete_file "$CONTENT_REPO" "bdd/rfcs/$_slug.meta.yaml"
done
fi
echo "seed-ppe: done. Set REGISTRY_REPO=$REGISTRY_REPO on rfc-app-ppe and redeploy."