From 31d680be54c1ff12c0079a3d93dbc1dc6da07347 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 03:39:04 -0700 Subject: [PATCH 01/21] =?UTF-8?q?=C2=A722=20three-tier=20refactor=20spec:?= =?UTF-8?q?=20project=20=E2=86=92=20RFC=20collection=20+=20unified=20roles?= =?UTF-8?q?=20+=20BDDs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Splits the original §22 two-tier model (deployment → project=corpus) into three tiers (deployment → project → RFC collection). Project owns one content repo; collections are typed subfolders declared by .collection.yaml manifests (git-truth). Reconciles the accumulated role vocabulary onto one {owner, contributor} enum attached at {global, project, collection}, with downward additive inheritance and no negative override. Adds BDD scenarios (Part C) for role usage, invitation, and empty states. Re-slots the roadmap to fold the tier into the not-yet-shipped Plan B (mig 028) + M3-frontend. Session 0072 (spec). Revises docs/design/multi-project-spec.md §22. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-05-three-tier-projects-collections.md | 543 ++++++++++++++++++ 1 file changed, 543 insertions(+) create mode 100644 docs/design/2026-06-05-three-tier-projects-collections.md diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md new file mode 100644 index 0000000..e4d0fce --- /dev/null +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -0,0 +1,543 @@ +# Draft spec — §22 refactor: three tiers (deployment → project → RFC collection) + +> Status: **draft for review.** Binding voice, but not yet merged into +> `SPEC.md`. This doc **revises the §22 model** in +> [`multi-project-spec.md`](./multi-project-spec.md) from two tiers +> (deployment → project, where a "project" *is* a corpus) to **three tiers** +> (deployment → project → RFC collection, where the *collection* is the +> corpus). It supersedes the conflicting parts of that draft; the parts it does +> not touch (the registry-is-git-truth stance, the cache mirror, visibility +> semantics, the `type`/`initial_state`/`unreviewed` machinery) carry over +> unchanged, re-homed onto the collection. Rationale and the decisions behind +> this live in [`multi-project.md`](./multi-project.md) and session 0072. +> Target release: folded into the not-yet-shipped §22 work (Plan B + M3-frontend), +> a pre-1.0 minor carrying breaking changes with upgrade steps (§20.2). + +--- + +## 0. Why this revision + +The original §22 (`multi-project-spec.md`) gave a deployment **N projects**, +where each project *was* a single typed corpus: one content repo, one `type`, +one slug namespace, one member roster. That conflates two responsibilities — +**organizational grouping** and **a typed body of entries** — into one noun. + +This revision splits them. A **project** becomes a pure grouping tier (settings ++ one content repo) that holds **any number of RFC collections**; an **RFC +collection** is the typed corpus the original §22 called a "project." Everything +the original §22 said about a corpus (type, slug namespace, catalog, philosophy, +landing state, review flag, membership) moves down one level to the collection; +the deployment level is unchanged. + +The timing is favorable: the breaking slug-PK rebuild (Plan B / migration 028) +and the public `/p//` routing (M3-frontend) **have not shipped**, so this is +a revision of in-flight *designs*, not a rework of shipped surfaces (§7). + +--- + +# Part A — The three-tier model + +## A.1 The tiers + +``` +deployment (= "global" in the UI) one Gitea org, one bot, one account + │ system, one inbox, one running process; + │ the surface a visitor first lands on. + └─ project ◀ NEW a named grouping + project settings; + │ owns exactly ONE content repo. No type. + └─ RFC collection a typed corpus: type, slug namespace, + │ catalog, philosophy, initial_state, + │ unreviewed flag, members. (= what the + │ original §22 called a "project".) + └─ entry an RFC / spec / feature, identified by + its slug within the collection. +``` + +- **Deployment / "global."** Unchanged top tier. Owns accounts, the §6 + admission gate, the §15 inbox, the §1 bot, and the deployment landing + directory. Its management surface is **projects + global settings**. +- **Project** *(new)*. Belongs to exactly one deployment; never moves. Owns one + content repo (§A.2) and carries project settings (name, tagline, theme, + visibility, model universe). Has **no `type`** of its own. Its management + surface is **RFC collections + project settings**. +- **RFC collection.** A typed subfolder of its project's content repo (§A.2). + Carries everything the original §22 pinned on a "project": the immutable + `type` (§22.4a `document` | `specification` | `bdd` | …), the per-collection + slug namespace (§A.3), `initial_state` (§22.4b), the `unreviewed` flag + (§22.4c), catalog, philosophy. This is "closest to what OHM originally + managed as a single corpus." +- **Entry.** Unchanged (§2). Identified by its slug **within its collection**. + +A collection belongs to exactly one project; a project to exactly one +deployment. Isolation (§22.1) now holds at the **collection** grain: an RFC, +branch, thread, star, or watch belongs to exactly one collection. + +## A.2 Storage and git-truth + +Two git sources, both read by the bot, both mirrored into cache tables the §4 +way: + +1. **The registry repo** (`projects.yaml`, located by `REGISTRY_REPO`, §22.2) + declares **projects** — `id`, `name`, `content_repo`, settings, `visibility`, + `theme`, `enabled_models`. `content_repo` moves **up** from the collection + (original §22) to the project: a project owns exactly one content repo. + +2. **Each project's content repo** declares its **collections** as typed + subfolders, each carrying a **`.collection.yaml` manifest** (the collection's + `type`, `visibility`, `initial_state`). The registry mirror walks the content + repo and reads these manifests, so collection configuration is git-truth and + survives a cache rebuild — exactly as entry frontmatter does. + +```yaml +# projects.yaml (registry repo root) +deployment: + name: Wiggleverse + tagline: ... +projects: + - id: ohm + name: Open Human Model + content_repo: ohm-content # ONE repo; collections live inside it + visibility: public # gated | public | unlisted (§22.5) + theme: { accent: "#5b5bd6" } + enabled_models: [claude, gemini] +``` + +```yaml +# ohm-content/features/.collection.yaml (one per collection subfolder) +type: bdd # document | specification | bdd — immutable +visibility: gated # defaults to the project's, may narrow +initial_state: active # defaults from type (§22.4b) +name: Feature scenarios +``` + +``` +ohm-content/ + model/ + .collection.yaml # type: document + intro.md + specs/ + .collection.yaml # type: specification + runtime.md + features/ + .collection.yaml # type: bdd + login.md +``` + +**Creation is in-app, wrapping a bot commit, at both tiers:** + +- **+ New project** (a global Owner action): the bot **creates a Gitea content + repo** under the deployment org, **commits a project entry** to + `projects.yaml`, and the mirror picks it up. +- **+ New collection** (a project Owner / RFC Contributor-with-create action): + the bot **commits a new subfolder + `.collection.yaml`** to the project's + content repo; the mirror picks it up. + +The in-app button is a thin convenience over a git write; nothing becomes app +state that git cannot rebuild. `projects` and `collections` cache rows are never +written from user actions directly — they flow from the mirror only (§22.2). +**Membership** (§B-roles) remains app state, as `rfc_collaborators` always has +been — it churns at user speed and is not document state. + +## A.3 Identity and routing + +The slug is unique **within a collection**; the fully-qualified identity is +`(project, collection, slug)`. `model/intro` and `specs/intro` coexist. No type +prefix, no numbers (the §22.4 retirement of `RFC-NNNN` allocation stands; +legacy `id` frontmatter remains a frozen, non-identity display label). + +Canonical route: + +``` +/p//c//e/ +``` + +The `c/` segment keeps collection ids from colliding with reserved +project-level segments (project settings, the collection directory). Reserved +**collection-level** siblings (`proposals`, `philosophy`) sit under +`/p//c//…`. The displayed entry noun ("RFC", "Spec", +"Feature") is the collection type's label (§22.4a), not part of the path. + +The root `/` is the deployment landing: a **directory of projects** the visitor +can see (§22.5). `/p//` is the project landing: a **directory of +collections** in that project the visitor can see. Conveniences: + +- `/p//` redirects to its sole collection when the project has exactly + one visible collection. +- `/` redirects to the sole visible project when there is exactly one (the N=1 + case, §A.6). + +Backcompat is light: no `/p//` URL ever shipped (M3-frontend is unmerged). +The only live legacy URLs are the pre-multi-project `/rfc/` / +`/proposals/`, which **308-redirect** to the migration's default project → +default collection (§A.6). + +--- + +# Part B — Roles and authorization + +## B.1 One role vocabulary, attached at a scope + +There is **one role enum — `{owner, contributor}`** — displayed as **Owner** +and **RFC Contributor**. A grant *attaches that role at a scope*: **global**, +**project**, or **collection**. "Owner at all levels, RFC Contributor at all +levels" is therefore literal — the same two words at every tier, not a fresh +pair invented per tier. + +| Role | Capabilities within its scope's subtree | +|---|---| +| **Owner** | Superuser: manage settings and membership; create child projects/collections; act on any entry (merge on behalf, graduate, mark-reviewed, withdraw/reopen, set branch visibility). | +| **RFC Contributor** | Propose entries, create branches, open PRs, claim unclaimed super-drafts, participate in discussion. At **project** (or global) scope this additionally includes **creating collections** in that project — the "anyone at the project level with permission to create a collection" affordance. (A *collection*-scope grant cannot create sibling collections; creating one is a project-level action.) | + +This **reconciles** the role names the prior drafts accumulated — they were +different words for the same idea: + +| Prior spec term | Tier it lived at | Unified role | +|---|---|---| +| deployment `owner` / `admin` (§6.1) | global | **Owner** (global) | +| deployment `contributor` (§6.1) | global | **RFC Contributor** (global) | +| `project_admin` (M2 §22.6) | the corpus → now the **collection** | **Owner** (collection) | +| `project_contributor` (M2 §22.6) | the corpus → now the **collection** | **RFC Contributor** (collection) | +| `project_viewer` (M2 §22.6) | the corpus | *deferred* (read-only grant; not one of this pass's two) | + +> **Scope-narrowing, not renaming.** Collapsing `owner`/`admin` into one +> **Owner** and dropping `viewer` for this pass are deliberate deferrals (the +> launch ask: "we don't need to get all permissions right yet"). When they +> return they **re-split out of** Owner / add a tier; they are not aliases of +> the unified roles. The richer set is future work. + +## B.2 Inheritance and resolution + +Grants inherit **downward**, are **additive**, and admit **no negative +override**: + +- A grant at **global** covers every project and collection in the deployment. +- A grant at **project** covers every collection in that project. +- A grant at **collection** covers just that collection. +- You **cannot** grant a role at a parent scope and revoke it at a child (the + launch ask: "too complex"). Resolution never subtracts a parent grant. + +Effective authority on an entry generalizes the §22.7 most-permissive union +from three layers to four (global → project → collection → per-entry): + +``` +effective authority on an entry = + global role (users.role) + ∪ project role (membership at the entry's project) + ∪ collection role (membership at the entry's collection) + ∪ per-entry authority (owners / arbiters / rfc_collaborators — §6.3, §12) + then minus §6.2 write-mute and §22.5 visibility (subtractive, as today) +``` + +**Per-entry authority is a distinct, finer layer — not a synonym.** `owners` / +`arbiters` / `rfc_collaborators` apply to *one specific entry* (§6.3, §12); the +three named scopes apply to a *subtree*. Per-entry authority is unchanged and +sits beneath collection in the union. `arbiter` is narrower than Owner (one +entry, not a subtree) and stays distinct. + +## B.3 Schema impact + +- `users.role` continues to carry the **global** role (deployment owner / + contributor). +- M2's `project_members(project_id, role)` rows were attached at what we now + call the **collection**. They generalize into a single polymorphic + **`memberships(scope_type ∈ {project, collection}, scope_id, user_id, role, + granted_by, granted_at)`** table; the M2 rows migrate to + `scope_type='collection'`. The **project** tier gets the same two roles, + freshly grantable. +- The M2 three-role enum (`viewer`/`contributor`/`admin`) collapses to + `{owner, contributor}`: `project_admin → owner`, `project_contributor → + contributor`, `project_viewer →` a read grant (no write) folded into + visibility, not a membership role this pass. + +--- + +# Part C — Behavioral scenarios (BDD) + +> These Gherkin scenarios are the behavioral spec for **role usage**, +> **invitation**, and **empty-state** experiences. They attach to the rewritten +> §22 as **§22.6a (role & invitation scenarios)**. They are written so they can +> *also* seed a `bdd`-type collection later (the framework dogfooding its own +> model). "Owner"/"RFC Contributor" are the unified roles (§B.1); a *scope* in +> the `Given` is global / project / collection. + +## C.1 Role usage — inheritance and the most-permissive union + +```gherkin +Feature: Scope roles grant authority over a subtree + As a member of the deployment + I want a role granted at one tier to apply to everything beneath it + So that I can be invited once and work across the right set of collections + + Background: + Given a deployment with a project "ohm" + And "ohm" owns collections "model" (document) and "features" (bdd) + + Scenario: Collection RFC Contributor may propose only in that collection + Given "ada" is RFC Contributor at collection "ohm/model" + When "ada" opens the propose form in "ohm/model" + Then she may submit a new entry + When "ada" opens "ohm/features" + Then she sees it read-only and the propose action is not offered + + Scenario: Project RFC Contributor may propose in every collection of the project + Given "ben" is RFC Contributor at project "ohm" + Then "ben" may propose in "ohm/model" + And "ben" may propose in "ohm/features" + And a collection added to "ohm" later is writable by "ben" with no new grant + + Scenario: Global RFC Contributor may propose in every collection of every project + Given a second project "acme" with collection "acme/specs" + And "cleo" is RFC Contributor at global scope + Then "cleo" may propose in "ohm/model" and "acme/specs" + + Scenario: Collection Owner administers one collection only + Given "dan" is Owner at collection "ohm/features" + Then "dan" may graduate, mark-reviewed, and manage membership in "ohm/features" + But "dan" may not change "ohm" project settings + And "dan" may not act on entries in "ohm/model" + + Scenario: Project Owner administers all collections and may create more + Given "eve" is Owner at project "ohm" + Then "eve" may manage membership in "ohm/model" and "ohm/features" + And "eve" may edit "ohm" project settings + And "eve" may create a new collection in "ohm" + + Scenario: Most-permissive union — the higher grant wins + Given "fay" is RFC Contributor at collection "ohm/model" + And "fay" is Owner at project "ohm" + Then "fay" acts as Owner in "ohm/model" + + Scenario: No negative override — a child cannot subtract a parent grant + Given "gil" is RFC Contributor at project "ohm" + Then there is no control to remove "gil" from "ohm/model" while keeping the project grant + And "gil" can propose in "ohm/model" + + Scenario: A granted account with no scope role sees only public content + Given "hana" has a granted deployment account but no global, project, or collection role + Then "hana" may read public collections under the §6.1 anonymous-read contract + But "hana" is not offered the propose action anywhere + And gated projects and collections do not appear for her +``` + +## C.2 Invitation — who may invite whom, at which scope + +```gherkin +Feature: Inviting users to a scope role + As an Owner of a scope + I want to grant Owner or RFC Contributor at my scope or any scope beneath it + So that collaborators get exactly the reach they need + + Scenario: Project Owner invites at project scope (covers all collections) + Given "eve" is Owner at project "ohm" + When "eve" invites "ivy" as RFC Contributor at project "ohm" + Then a membership row is written at scope project "ohm" + And "ivy" receives a §15 notification naming the project and role + And "ivy" may propose in every collection of "ohm" + + Scenario: Owner invites at a specific collection + When "eve" invites "jo" as RFC Contributor at collection "ohm/features" + Then a membership row is written at scope collection "ohm/features" + And "jo" may propose in "ohm/features" but not "ohm/model" + + Scenario: Invitation reach is bounded by the inviter's scope + Given "dan" is Owner at collection "ohm/features" + Then "dan" may invite users to roles in "ohm/features" + But "dan" is not offered the control to invite at project "ohm" or global scope + + Scenario: RFC Contributors do not manage membership + Given "ben" is RFC Contributor at project "ohm" + Then "ben" may propose and create collections in "ohm" + But "ben" is not offered any invite control (membership is an Owner capability) + + Scenario: The invite UI offers no grant-at-parent-revoke-at-child option + Given "eve" is Owner at project "ohm" + When "eve" opens the invite control for "ivy" at project "ohm" + Then she may choose role Owner or RFC Contributor and scope project or a single collection + But there is no option to grant at "ohm" and exclude a child collection + + Scenario: Re-inviting at a broader scope supersedes the narrower grant + Given "jo" is RFC Contributor at collection "ohm/features" + When "eve" invites "jo" as RFC Contributor at project "ohm" + Then "jo" has the role across all of "ohm" + And the redundant collection-scope row is removed or shown as subsumed + + Scenario: A pending deployment account cannot be granted write + Given "kim" has permission_state "pending" at the deployment + When "eve" invites "kim" as RFC Contributor at project "ohm" + Then the grant is recorded but confers no write capability until "kim" is granted at the deployment (§6) +``` + +## C.3 Empty-state experiences + +```gherkin +Feature: Empty states at each tier + As a viewer of a tier with nothing in it yet + I want a clear, role-appropriate empty state + So that I know whether there is an action to take or simply nothing to see + + Scenario: Global directory with no projects — Owner + Given a deployment with no projects + And "root" is Owner at global scope + When "root" lands on "/" + Then she sees an empty directory with a "Create your first project" call to action + + Scenario: Global directory with no visible projects — non-owner + Given a deployment whose only projects are gated + And "vee" is a granted account with no roles + When "vee" lands on "/" + Then she sees an empty directory with no create action + And a note that there is nothing shared with her yet + + Scenario: Project with no collections — project Owner + Given project "ohm" with no collections + And "eve" is Owner at project "ohm" + When "eve" lands on "/p/ohm/" + Then she sees an empty collection directory with a "Create your first collection" call to action + And the action lets her choose a type and subfolder + + Scenario: Project with no collections — RFC Contributor without create rights + Given project "ohm" with no collections + And "ben" is RFC Contributor at collection scope elsewhere only + When "ben" lands on "/p/ohm/" + Then he sees an empty collection directory with no create action + + Scenario: Collection with no entries — a contributor + Given collection "ohm/model" with no entries + And "ada" is RFC Contributor at collection "ohm/model" + When "ada" lands on "/p/ohm/c/model/" + Then she sees an empty catalog with a "Propose the first entry" call to action + + Scenario: Collection with no entries — an anonymous reader + Given a public collection "ohm/model" with no entries + When an anonymous visitor lands on "/p/ohm/c/model/" + Then they see an empty catalog with no propose action and a sign-in prompt + + Scenario: Single-collection project skips the directory + Given project "ohm" with exactly one visible collection "model" + When a visitor lands on "/p/ohm/" + Then they are redirected to "/p/ohm/c/model/" + + Scenario: Single-project deployment skips the directory + Given a deployment with exactly one visible project "ohm" + When a visitor lands on "/" + Then they are redirected to "/p/ohm/" +``` + +--- + +# Part D — Amendments to the original §22 draft + +Applied in place when §22 is rewritten; listed here as the change surface. + +- **§22 preamble / §22.1.** "A deployment hosts N projects, each a corpus" → + "a deployment hosts N **projects**, each owning one content repo and holding + N **RFC collections**, each collection a typed corpus." Isolation moves to the + collection grain. +- **§22.2 Registry.** `projects.yaml` declares projects with one `content_repo` + each (no per-collection `content_repo`). New: collections are declared by + `.collection.yaml` manifests inside the content repo; the mirror reads them. + In-app create-project / create-collection actions wrap bot commits. +- **§22.3 Content repos.** "One per project" (not per collection); collections + are subfolders within it. +- **§22.4 / §22.4a-c.** Slug is unique **per collection**. `type`, + `initial_state`, and `unreviewed` are **collection** properties (re-homed from + "project"). Unchanged otherwise. +- **§22.5 Visibility.** Applies at **both** project and collection. A collection + defaults to its project's visibility and may narrow it; reading/writing a + collection requires passing both gates. +- **§22.6 Membership and roles → the unified model (Part B).** Replace the three + `project_*` roles with `{owner, contributor}` at `{global, project, + collection}` via a polymorphic `memberships` table. Add **§22.6a** = the + Part C scenarios. +- **§22.7 Composition.** Four-layer most-permissive union (global → project → + collection → per-entry); no negative override. +- **§22.9 / §22.10 Branding & routing.** Routes gain the collection segment: + `/p//c//…`. `GET /api/deployment` lists visible projects; + add `GET /api/projects/:id` (lists visible collections + project settings) and + `GET /api/projects/:id/collections/:cid` (collection settings incl. `type`). +- **§22.11 Notifications / §22.13 migration / §5 amendments.** `project_id` + becomes `collection_id` on every entry-scoped row (the corpus grain is now the + collection); a separate `project_id` exists only on the `collections` table + and project-scoped rows. The §22.13 default project gains a default collection + (§A.6 below). + +--- + +# Part E — Revised slicing plan (the roadmap re-slot) + +Strategy (session 0072 decision): **fold the third tier into the not-yet-shipped +work** rather than ship two-tier and migrate again. M1, M2, and M3-backend +Plan A are additive and already merged (v0.39.0); they keep running. The tier +goes in at the **first breaking migration (Plan B / 028)** and the **first +public routing (M3-frontend)**, before either ships. + +- **Landed, unchanged (v0.39.0):** M1 (additive `project_id` spine, mig 026), + M2 (authz resolver), M3-backend Plan A (registry mirror, APIs, propose / + mark-reviewed, mig 027). These read fine under the new model: today's single + `projects` row is re-read as the **default collection** after the §E migration + relabels it. + +- **M3-backend Plan B → "insert the project tier + re-key to per-collection"** + (revised). Migration 028 now: (1) add the `projects` (grouping) table with + `content_repo`; (2) rename the landed `projects` table to `collections`, drop + its `content_repo`, add `project_id` + `subfolder`; (3) re-key every + entry-scoped `project_id` → `collection_id`; (4) the slug-PK rebuild targets + `(collection_id, slug)`; (5) generalize `project_members` → the polymorphic + `memberships` table and collapse the role enum (§B.3); (6) the §22.13 default + project wraps the default collection (§A.6). One breaking migration, one slug + rebuild. + +- **M3-backend Plan B+ — manifests & in-app create.** Teach the mirror to read + `.collection.yaml`; add the bot-commit-wrapped **create-project** and + **create-collection** endpoints; mirror new collections. + +- **M3-frontend → three-tier routing.** `/p//c//e/`; + the deployment directory at `/`, the project (collection) directory at + `/p//`; the single-project / single-collection redirects (§A.3); the + 308s off `/rfc/`; runtime branding at both project and collection. + +- **M4 — second-collection + second-project acceptance.** End-to-end pass: a + second collection in a project, and a second project, each with disjoint + membership, full propose→graduate lifecycle under the per-collection slug + namespace. + +- **M5 — type modules (unchanged in intent).** Per-type frontmatter + surfaces, + now selected on the **collection's** `type`. + +- **M6 — membership lifecycle.** Invite + request-to-join at all three scopes + (Part C.2); the cross-collection inbox; the roster admin surface. The Part C + scenarios are this slice's acceptance tests. + +- **M7 — hardening + SPEC merge.** Per-collection `enabled_models`; the registry + + manifest format in `docs/DEPLOYMENTS.md`; two-project / multi-collection e2e; + the §20.4 changelog + upgrade-steps; the SPEC merge (Part A applied, Part D + amendments in place). + +## E.1 (= §A.6) Migration — the default project and default collection + +A pre-three-tier deployment (already on v0.39.0's single `projects` row, or +pre-multi-project) is migrated so it keeps running: + +1. The landed single `projects` row becomes the **default collection** + (`type=document`, its `initial_state`/`unreviewed` preserved). +2. A **default project** is generated to wrap it: `content_repo` moves from the + (old project / now collection) onto the project; `id` is the config-derived + slug already re-stamped in §22.13 step 1 (v0.39.0). +3. Every entry-scoped `project_id` is re-keyed to the default `collection_id`. +4. M2's `project_members` rows migrate to `memberships(scope_type='collection')` + on the default collection, role-collapsed (§B.3). +5. `/rfc/` and `/proposals/` 308-redirect to + `/p//c//…`. + +Until a second project or collection is added, the deployment is functionally +identical to before, with two extra path segments. This is the §20.4 +upgrade-steps content for the release. + +## E.2 Scope of the first implementation pass + +Per the launch ask — "we don't need to get all permissions right yet, just have +Owner at all levels, and RFC Contributor at the global, project, and RFC +collection level" — the **role surface** this pass implements is exactly +`{owner, contributor}` × `{global, project, collection}` (Part B), plus the +unchanged per-entry layer. `viewer`, the owner/admin split, request-to-join +nuances, and per-type role labels are deferred (§B.1 note). -- 2.39.5 From 9c0e3b60ac0cad7d2e7009640bc28d62dd914ebf Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 03:43:44 -0700 Subject: [PATCH 02/21] =?UTF-8?q?Correct=20=C2=A722=20three-tier=20spec:?= =?UTF-8?q?=20two-tier=20model=20already=20shipped=20(v0.39.0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-checked code vs the stale memory: migration 028 (slug PK -> (project_id, slug)), v0.35.0 /p// routing, and v0.37/0.38 per-project read+propose are all shipped to main. The 'fold into not-yet-shipped Plan B + M3-frontend' premise is false. Neutralize the wrong claims in §0/§A.3 and flag Part E's sequencing as pending re-decision; structural model (Parts A-D) unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-05-three-tier-projects-collections.md | 52 ++++++++++++++----- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md index e4d0fce..b71d70a 100644 --- a/docs/design/2026-06-05-three-tier-projects-collections.md +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -10,8 +10,18 @@ > semantics, the `type`/`initial_state`/`unreviewed` machinery) carry over > unchanged, re-homed onto the collection. Rationale and the decisions behind > this live in [`multi-project.md`](./multi-project.md) and session 0072. -> Target release: folded into the not-yet-shipped §22 work (Plan B + M3-frontend), -> a pre-1.0 minor carrying breaking changes with upgrade steps (§20.2). +> +> ⚠️ **CORRECTION (session 0072, after code re-check).** Parts of §0/§A.3/§E +> were drafted on a stale-memory premise that "Plan B (migration 028) and +> M3-frontend have not shipped." **That is false.** As of v0.39.0 the entire +> **two-tier** model is shipped to `main`: migration 028 already rebuilt the +> slug PK to `(project_id, slug)`; v0.35.0 shipped `/p//` routing and +> the live `/p//e/` URLs; v0.37.0/0.38.0 shipped per-project +> read + propose. Inserting the third tier is therefore an **evolution of a +> shipped system**, not a revision of unshipped designs. The migration strategy +> (Part E) is **pending re-decision** with correct facts; the structural model +> (Parts A–D) is unaffected. Target release: a further pre-1.0 minor with +> breaking changes + upgrade steps (§20.2). --- @@ -29,9 +39,11 @@ the original §22 said about a corpus (type, slug namespace, catalog, philosophy landing state, review flag, membership) moves down one level to the collection; the deployment level is unchanged. -The timing is favorable: the breaking slug-PK rebuild (Plan B / migration 028) -and the public `/p//` routing (M3-frontend) **have not shipped**, so this is -a revision of in-flight *designs*, not a rework of shipped surfaces (§7). +⚠️ The two-tier model is **already shipped** (v0.39.0): migration 028 rebuilt +the slug PK to `(project_id, slug)`, and `/p//e/` URLs are live +(v0.35.0). So inserting the third tier evolves a shipped system — see the +corrected Part E for the real migration cost (a new migration 029 + a breaking +URL change with 308s), pending re-decision. --- @@ -166,10 +178,12 @@ collections** in that project the visitor can see. Conveniences: - `/` redirects to the sole visible project when there is exactly one (the N=1 case, §A.6). -Backcompat is light: no `/p//` URL ever shipped (M3-frontend is unmerged). -The only live legacy URLs are the pre-multi-project `/rfc/` / -`/proposals/`, which **308-redirect** to the migration's default project → -default collection (§A.6). +⚠️ **Backcompat is heavier than first drafted.** `/p//e/` URLs +**are live** (v0.35.0), so adding the `/c//` segment is a breaking +URL change: the shipped `/p//e/` must **308-redirect** to +`/p//c//e/`, alongside the pre-multi-project +`/rfc/` → `/p//c//…` redirect. Both +are handled in the migration (§A.6 / Part E). --- @@ -465,11 +479,21 @@ Applied in place when §22 is rewritten; listed here as the change surface. # Part E — Revised slicing plan (the roadmap re-slot) -Strategy (session 0072 decision): **fold the third tier into the not-yet-shipped -work** rather than ship two-tier and migrate again. M1, M2, and M3-backend -Plan A are additive and already merged (v0.39.0); they keep running. The tier -goes in at the **first breaking migration (Plan B / 028)** and the **first -public routing (M3-frontend)**, before either ships. +> ⚠️ **PENDING RE-DECISION (session 0072).** The strategy below assumed Plan B +> and M3-frontend were unshipped. They are shipped (v0.35.0–0.39.0). The "fold +> into not-yet-shipped work" framing is void; the actual work is a **new +> migration 029** that evolves the shipped two-tier schema to three tiers, plus +> a **breaking `/p//e/` → `/p//c//e/` +> URL change with 308s**. The two live mapping options (relabel today's +> project → collection + insert a group tier above, vs. add a collection +> sub-grain beneath today's project) are being put back to the operator. The +> text below is retained only as the structural target, not the sequencing. + +Strategy (ORIGINAL, premise now false): fold the third tier into the +not-yet-shipped work rather than ship two-tier and migrate again. M1, M2, and +M3-backend Plan A are additive and already merged (v0.39.0); they keep running. +The tier goes in at the **first breaking migration (Plan B / 028)** and the +**first public routing (M3-frontend)**, before either ships. - **Landed, unchanged (v0.39.0):** M1 (additive `project_id` spine, mig 026), M2 (authz resolver), M3-backend Plan A (registry mirror, APIs, propose / -- 2.39.5 From 87279fc54510e6ded3f302ced548c6b0e9c8f1fc Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 03:46:51 -0700 Subject: [PATCH 03/21] =?UTF-8?q?=C2=A722=20three-tier=20spec=20Part=20E:?= =?UTF-8?q?=20decided=20migration-029=20strategy=20(collection=20grain=20b?= =?UTF-8?q?eneath=20project)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator chose (session 0072) to add the collection grain beneath today's project: projects table stays the group tier (keeps content_repo), a new collections table holds the per-corpus fields, entries re-key to (collection_id, slug), one default collection per project on migrate, breaking /p//e/ -> /p//c//e/ with 308s. Re-sloted slices N1-N6. Correction banners updated from pending -> decided. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-05-three-tier-projects-collections.md | 154 ++++++++++-------- 1 file changed, 82 insertions(+), 72 deletions(-) diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md index b71d70a..aebdd7d 100644 --- a/docs/design/2026-06-05-three-tier-projects-collections.md +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -19,9 +19,11 @@ > the live `/p//e/` URLs; v0.37.0/0.38.0 shipped per-project > read + propose. Inserting the third tier is therefore an **evolution of a > shipped system**, not a revision of unshipped designs. The migration strategy -> (Part E) is **pending re-decision** with correct facts; the structural model -> (Parts A–D) is unaffected. Target release: a further pre-1.0 minor with -> breaking changes + upgrade steps (§20.2). +> (Part E) was **re-decided on these corrected facts** (session 0072): a new +> **migration 029** adds a *collection* grain *beneath* today's project, plus a +> breaking `/p//e/` → `/p//c//e/` URL +> change with 308s. The structural model (Parts A–D) is unaffected. Target +> release: a further pre-1.0 minor with breaking changes + upgrade steps (§20.2). --- @@ -41,9 +43,9 @@ the deployment level is unchanged. ⚠️ The two-tier model is **already shipped** (v0.39.0): migration 028 rebuilt the slug PK to `(project_id, slug)`, and `/p//e/` URLs are live -(v0.35.0). So inserting the third tier evolves a shipped system — see the -corrected Part E for the real migration cost (a new migration 029 + a breaking -URL change with 308s), pending re-decision. +(v0.35.0). So inserting the third tier evolves a shipped system — see Part E +for the decided strategy (a new migration 029 adding a collection grain beneath +today's project, + a breaking URL change with 308s). --- @@ -479,83 +481,91 @@ Applied in place when §22 is rewritten; listed here as the change surface. # Part E — Revised slicing plan (the roadmap re-slot) -> ⚠️ **PENDING RE-DECISION (session 0072).** The strategy below assumed Plan B -> and M3-frontend were unshipped. They are shipped (v0.35.0–0.39.0). The "fold -> into not-yet-shipped work" framing is void; the actual work is a **new -> migration 029** that evolves the shipped two-tier schema to three tiers, plus -> a **breaking `/p//e/` → `/p//c//e/` -> URL change with 308s**. The two live mapping options (relabel today's -> project → collection + insert a group tier above, vs. add a collection -> sub-grain beneath today's project) are being put back to the operator. The -> text below is retained only as the structural target, not the sequencing. +**Strategy (session 0072, decided on corrected facts).** The two-tier model is +shipped end-to-end (v0.39.0): migration 028 keyed entries `(project_id, slug)`; +v0.35.0 shipped `/p//` routing + live `/p//e/` URLs; +v0.37.0/0.38.0 shipped per-project read + propose. Inserting the third tier is +therefore an **evolution of a shipped system**. The chosen mapping **adds a +collection grain *beneath* today's project** — the shipped `projects` table +stays the grouping tier (it already owns `content_repo`, where §A.2 wants it), +a new `collections` table holds the per-corpus fields, and entries re-key to the +finer `(collection_id, slug)`. The migrated default keeps every deployment +running; the new `/c//` URL segment is added with 308s off the +now-legacy two-tier URLs. -Strategy (ORIGINAL, premise now false): fold the third tier into the -not-yet-shipped work rather than ship two-tier and migrate again. M1, M2, and -M3-backend Plan A are additive and already merged (v0.39.0); they keep running. -The tier goes in at the **first breaking migration (Plan B / 028)** and the -**first public routing (M3-frontend)**, before either ships. +- **Landed, unchanged (v0.39.0):** M1–M2, M3-backend Plan A **and** Plan B + (read+propose, mig 028), M3-frontend (`/p//` routing), §22.13 + re-stamp. None of this is rebuilt; it is *evolved* by migration 029 below. -- **Landed, unchanged (v0.39.0):** M1 (additive `project_id` spine, mig 026), - M2 (authz resolver), M3-backend Plan A (registry mirror, APIs, propose / - mark-reviewed, mig 027). These read fine under the new model: today's single - `projects` row is re-read as the **default collection** after the §E migration - relabels it. +- **N1 — migration 029: add the collection grain.** A new + `029_collections.sql` that: (1) **adds a `collections` table** — + `(id, project_id, type, subfolder, initial_state, visibility, name, + registry_sha)`; (2) **moves the per-corpus fields** (`type`, `initial_state`, + visibility) from the shipped `projects` table **down** to `collections`, + leaving `projects` as `(id, content_repo, visibility, name, tagline, theme, + enabled_models, …)`; (3) **creates one default collection per existing + project** (id `default`, `subfolder` = repo root, inheriting that project's + `type`/`initial_state`); (4) **re-keys every entry-scoped table** from + `(project_id, slug)` to `(collection_id, slug)` via the same SQLite + rebuild procedure migration 028 used (the `028_project_scoped_keys.sql` + pattern — `__new` table, copy, drop, rename, FK-off + `foreign_key_check`), + stamping the default `collection_id`; (5) **generalizes `project_members` → + `memberships(scope_type ∈ {project, collection}, scope_id, …)`**, migrating + existing rows to `scope_type='collection'` on the default collection and + collapsing the role enum to `{owner, contributor}` (§B.3). -- **M3-backend Plan B → "insert the project tier + re-key to per-collection"** - (revised). Migration 028 now: (1) add the `projects` (grouping) table with - `content_repo`; (2) rename the landed `projects` table to `collections`, drop - its `content_repo`, add `project_id` + `subfolder`; (3) re-key every - entry-scoped `project_id` → `collection_id`; (4) the slug-PK rebuild targets - `(collection_id, slug)`; (5) generalize `project_members` → the polymorphic - `memberships` table and collapse the role enum (§B.3); (6) the §22.13 default - project wraps the default collection (§A.6). One breaking migration, one slug - rebuild. +- **N2 — backend collection scoping.** Thread `collection_id` through the + resolution gates that migration 028/Plan B threaded `project_id` through + (`app/auth.py`, `app/projects.py`, `app/cache.py`, the `api_*` writers); + generalize the §22.7 union to the four-layer resolver (§B.2); add the + `GET /api/projects/:id/collections` and + `GET /api/projects/:id/collections/:cid` surfaces; scope propose/serve/PR + endpoints to `(project, collection)`. -- **M3-backend Plan B+ — manifests & in-app create.** Teach the mirror to read - `.collection.yaml`; add the bot-commit-wrapped **create-project** and - **create-collection** endpoints; mirror new collections. +- **N3 — manifests & in-app create.** Teach the registry mirror to read + `.collection.yaml` from each project's content repo; add the + bot-commit-wrapped **create-collection** (project Owner / RFC Contributor) + and **create-project** (global Owner) endpoints (§A.2); mirror new + collections into `collections` rows. -- **M3-frontend → three-tier routing.** `/p//c//e/`; - the deployment directory at `/`, the project (collection) directory at - `/p//`; the single-project / single-collection redirects (§A.3); the - 308s off `/rfc/`; runtime branding at both project and collection. +- **N4 — frontend three-tier routing.** Add the `/c//` segment: + `/p//c//e/`; the project (collection) directory at + `/p//`; the single-collection redirect (§A.3); **308** the shipped + `/p//e/` → `/p//c//e/`; per-collection + chrome layered under the existing per-project chrome. -- **M4 — second-collection + second-project acceptance.** End-to-end pass: a - second collection in a project, and a second project, each with disjoint - membership, full propose→graduate lifecycle under the per-collection slug - namespace. +- **N5 — roles surface (this pass's scope).** The `{owner, contributor}` × + `{global, project, collection}` grants (Part B) with the invite surfaces and + empty states of Part C as acceptance tests. (Richer roles deferred, §B.1.) -- **M5 — type modules (unchanged in intent).** Per-type frontmatter + surfaces, - now selected on the **collection's** `type`. +- **N6 — type modules / membership lifecycle / hardening.** Carries the original + §22 M5–M7 forward, now selecting on the **collection's** `type`: per-type + frontmatter + surfaces; request-to-join + cross-collection inbox; per-collection + `enabled_models`; the registry + manifest format in `docs/DEPLOYMENTS.md`; + two-project / multi-collection e2e; the §20.4 changelog + upgrade-steps; the + SPEC merge (Part A applied, Part D amendments in place). -- **M6 — membership lifecycle.** Invite + request-to-join at all three scopes - (Part C.2); the cross-collection inbox; the roster admin surface. The Part C - scenarios are this slice's acceptance tests. +## E.1 (= §A.6) Migration — the default collection (the N=1 case) -- **M7 — hardening + SPEC merge.** Per-collection `enabled_models`; the registry - + manifest format in `docs/DEPLOYMENTS.md`; two-project / multi-collection e2e; - the §20.4 changelog + upgrade-steps; the SPEC merge (Part A applied, Part D - amendments in place). +A deployment on the shipped two-tier schema (v0.39.0) is migrated by 029 so it +keeps running unchanged: -## E.1 (= §A.6) Migration — the default project and default collection +1. The existing `projects` row **stays as the project** (it already owns + `content_repo` and its config-derived `id` from §22.13 step 1). +2. A **default collection** (`id='default'`, `subfolder` = repo root) is created + per project, inheriting that project's `type` / `initial_state` / visibility; + those fields are then dropped from `projects`. +3. Every entry-scoped `project_id` row is re-keyed with the default + `collection_id` (PK `(project_id, slug)` → `(collection_id, slug)`). +4. `project_members` rows migrate to `memberships(scope_type='collection')` on + the default collection, role-collapsed (§B.3). +5. **308 redirects:** the shipped `/p//e/` → + `/p//c//e/`, and the pre-multi-project `/rfc/` + / `/proposals/` → their `/p//c//…` equivalents. -A pre-three-tier deployment (already on v0.39.0's single `projects` row, or -pre-multi-project) is migrated so it keeps running: - -1. The landed single `projects` row becomes the **default collection** - (`type=document`, its `initial_state`/`unreviewed` preserved). -2. A **default project** is generated to wrap it: `content_repo` moves from the - (old project / now collection) onto the project; `id` is the config-derived - slug already re-stamped in §22.13 step 1 (v0.39.0). -3. Every entry-scoped `project_id` is re-keyed to the default `collection_id`. -4. M2's `project_members` rows migrate to `memberships(scope_type='collection')` - on the default collection, role-collapsed (§B.3). -5. `/rfc/` and `/proposals/` 308-redirect to - `/p//c//…`. - -Until a second project or collection is added, the deployment is functionally -identical to before, with two extra path segments. This is the §20.4 -upgrade-steps content for the release. +Until a second collection is added, the deployment is functionally identical to +before, with one extra path segment. This is the §20.4 upgrade-steps content for +the release. ## E.2 Scope of the first implementation pass -- 2.39.5 From 2f5d09aef5f9f0001c3309c11ad5cecb91e7e296 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 06:42:12 -0700 Subject: [PATCH 04/21] =?UTF-8?q?=C2=A722=20spec:=20re-cut=20Part=20E=20in?= =?UTF-8?q?to=20usable=20BDD-tagged=20slices=20(S1-S6)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per operator (session 0072): every slice must end in a usable deployment and declare which Part C scenarios it makes pass. Tag all 23 Gherkin scenarios with @S (the slice that completes them) and re-cut Part E from layer-by-layer (N1-N6) to usable increments (S1-S6) with a slice->scenario index table. S1 bundles the coupled migration 029 + threading + redirect as one right-sized first session; S2 second collection; S3 role enforcement; S4 invitation; S5 create-project + directory; S6 types + SPEC merge. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-05-three-tier-projects-collections.md | 150 ++++++++++++------ 1 file changed, 105 insertions(+), 45 deletions(-) diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md index aebdd7d..2b858a9 100644 --- a/docs/design/2026-06-05-three-tier-projects-collections.md +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -275,6 +275,12 @@ entry, not a subtree) and stays distinct. > *also* seed a `bdd`-type collection later (the framework dogfooding its own > model). "Owner"/"RFC Contributor" are the unified roles (§B.1); a *scope* in > the `Given` is global / project / collection. +> +> **Each scenario carries a `@S` tag** naming the **slice** (Part E) that +> makes it pass — the "which scenarios are done after this slice" marker. After +> shipping slice S, its acceptance gate is "every `@S` scenario passes" +> (e.g. `--tags @S3`). The Part E table is the inverse index (slice → +> scenarios). ## C.1 Role usage — inheritance and the most-permissive union @@ -288,6 +294,7 @@ Feature: Scope roles grant authority over a subtree Given a deployment with a project "ohm" And "ohm" owns collections "model" (document) and "features" (bdd) + @S3 Scenario: Collection RFC Contributor may propose only in that collection Given "ada" is RFC Contributor at collection "ohm/model" When "ada" opens the propose form in "ohm/model" @@ -295,39 +302,46 @@ Feature: Scope roles grant authority over a subtree When "ada" opens "ohm/features" Then she sees it read-only and the propose action is not offered + @S3 Scenario: Project RFC Contributor may propose in every collection of the project Given "ben" is RFC Contributor at project "ohm" Then "ben" may propose in "ohm/model" And "ben" may propose in "ohm/features" And a collection added to "ohm" later is writable by "ben" with no new grant + @S3 Scenario: Global RFC Contributor may propose in every collection of every project Given a second project "acme" with collection "acme/specs" And "cleo" is RFC Contributor at global scope Then "cleo" may propose in "ohm/model" and "acme/specs" + @S3 Scenario: Collection Owner administers one collection only Given "dan" is Owner at collection "ohm/features" Then "dan" may graduate, mark-reviewed, and manage membership in "ohm/features" But "dan" may not change "ohm" project settings And "dan" may not act on entries in "ohm/model" + @S3 Scenario: Project Owner administers all collections and may create more Given "eve" is Owner at project "ohm" Then "eve" may manage membership in "ohm/model" and "ohm/features" And "eve" may edit "ohm" project settings And "eve" may create a new collection in "ohm" + @S3 Scenario: Most-permissive union — the higher grant wins Given "fay" is RFC Contributor at collection "ohm/model" And "fay" is Owner at project "ohm" Then "fay" acts as Owner in "ohm/model" + @S3 Scenario: No negative override — a child cannot subtract a parent grant Given "gil" is RFC Contributor at project "ohm" Then there is no control to remove "gil" from "ohm/model" while keeping the project grant And "gil" can propose in "ohm/model" + @S3 Scenario: A granted account with no scope role sees only public content Given "hana" has a granted deployment account but no global, project, or collection role Then "hana" may read public collections under the §6.1 anonymous-read contract @@ -343,6 +357,7 @@ Feature: Inviting users to a scope role I want to grant Owner or RFC Contributor at my scope or any scope beneath it So that collaborators get exactly the reach they need + @S4 Scenario: Project Owner invites at project scope (covers all collections) Given "eve" is Owner at project "ohm" When "eve" invites "ivy" as RFC Contributor at project "ohm" @@ -350,33 +365,39 @@ Feature: Inviting users to a scope role And "ivy" receives a §15 notification naming the project and role And "ivy" may propose in every collection of "ohm" + @S4 Scenario: Owner invites at a specific collection When "eve" invites "jo" as RFC Contributor at collection "ohm/features" Then a membership row is written at scope collection "ohm/features" And "jo" may propose in "ohm/features" but not "ohm/model" + @S4 Scenario: Invitation reach is bounded by the inviter's scope Given "dan" is Owner at collection "ohm/features" Then "dan" may invite users to roles in "ohm/features" But "dan" is not offered the control to invite at project "ohm" or global scope + @S4 Scenario: RFC Contributors do not manage membership Given "ben" is RFC Contributor at project "ohm" Then "ben" may propose and create collections in "ohm" But "ben" is not offered any invite control (membership is an Owner capability) + @S4 Scenario: The invite UI offers no grant-at-parent-revoke-at-child option Given "eve" is Owner at project "ohm" When "eve" opens the invite control for "ivy" at project "ohm" Then she may choose role Owner or RFC Contributor and scope project or a single collection But there is no option to grant at "ohm" and exclude a child collection + @S4 Scenario: Re-inviting at a broader scope supersedes the narrower grant Given "jo" is RFC Contributor at collection "ohm/features" When "eve" invites "jo" as RFC Contributor at project "ohm" Then "jo" has the role across all of "ohm" And the redundant collection-scope row is removed or shown as subsumed + @S4 Scenario: A pending deployment account cannot be granted write Given "kim" has permission_state "pending" at the deployment When "eve" invites "kim" as RFC Contributor at project "ohm" @@ -391,12 +412,14 @@ Feature: Empty states at each tier I want a clear, role-appropriate empty state So that I know whether there is an action to take or simply nothing to see + @S5 Scenario: Global directory with no projects — Owner Given a deployment with no projects And "root" is Owner at global scope When "root" lands on "/" Then she sees an empty directory with a "Create your first project" call to action + @S5 Scenario: Global directory with no visible projects — non-owner Given a deployment whose only projects are gated And "vee" is a granted account with no roles @@ -404,6 +427,7 @@ Feature: Empty states at each tier Then she sees an empty directory with no create action And a note that there is nothing shared with her yet + @S4 Scenario: Project with no collections — project Owner Given project "ohm" with no collections And "eve" is Owner at project "ohm" @@ -411,28 +435,33 @@ Feature: Empty states at each tier Then she sees an empty collection directory with a "Create your first collection" call to action And the action lets her choose a type and subfolder + @S4 Scenario: Project with no collections — RFC Contributor without create rights Given project "ohm" with no collections And "ben" is RFC Contributor at collection scope elsewhere only When "ben" lands on "/p/ohm/" Then he sees an empty collection directory with no create action + @S4 Scenario: Collection with no entries — a contributor Given collection "ohm/model" with no entries And "ada" is RFC Contributor at collection "ohm/model" When "ada" lands on "/p/ohm/c/model/" Then she sees an empty catalog with a "Propose the first entry" call to action + @S2 Scenario: Collection with no entries — an anonymous reader Given a public collection "ohm/model" with no entries When an anonymous visitor lands on "/p/ohm/c/model/" Then they see an empty catalog with no propose action and a sign-in prompt + @S1 Scenario: Single-collection project skips the directory Given project "ohm" with exactly one visible collection "model" When a visitor lands on "/p/ohm/" Then they are redirected to "/p/ohm/c/model/" + @S1 Scenario: Single-project deployment skips the directory Given a deployment with exactly one visible project "ohm" When a visitor lands on "/" @@ -489,61 +518,92 @@ therefore an **evolution of a shipped system**. The chosen mapping **adds a collection grain *beneath* today's project** — the shipped `projects` table stays the grouping tier (it already owns `content_repo`, where §A.2 wants it), a new `collections` table holds the per-corpus fields, and entries re-key to the -finer `(collection_id, slug)`. The migrated default keeps every deployment -running; the new `/c//` URL segment is added with 308s off the -now-legacy two-tier URLs. +finer `(collection_id, slug)`. + +**Slicing principle (session 0072): every slice ends in a *usable* deployment, +and declares the Part C scenarios it makes pass** (its `@S` tag). "Usable" +means the deployment runs and either gains a capability or provably loses none +(N=1 unchanged). A slice is done when its `@S` scenarios are green. - **Landed, unchanged (v0.39.0):** M1–M2, M3-backend Plan A **and** Plan B (read+propose, mig 028), M3-frontend (`/p//` routing), §22.13 - re-stamp. None of this is rebuilt; it is *evolved* by migration 029 below. + re-stamp. None of this is rebuilt; it is *evolved* by the slices below. -- **N1 — migration 029: add the collection grain.** A new - `029_collections.sql` that: (1) **adds a `collections` table** — +- **S1 — The collection grain exists (invisible default).** Migration 029 + + backend threading + the default-routing redirect, shipped **together** (they + are coupled — renaming `project_id`→`collection_id` breaks every reader until + the code is threaded, so a green tree needs both). Migration 029 + (`029_collections.sql`): (1) add a `collections` table `(id, project_id, type, subfolder, initial_state, visibility, name, - registry_sha)`; (2) **moves the per-corpus fields** (`type`, `initial_state`, - visibility) from the shipped `projects` table **down** to `collections`, - leaving `projects` as `(id, content_repo, visibility, name, tagline, theme, - enabled_models, …)`; (3) **creates one default collection per existing - project** (id `default`, `subfolder` = repo root, inheriting that project's - `type`/`initial_state`); (4) **re-keys every entry-scoped table** from - `(project_id, slug)` to `(collection_id, slug)` via the same SQLite - rebuild procedure migration 028 used (the `028_project_scoped_keys.sql` - pattern — `__new` table, copy, drop, rename, FK-off + `foreign_key_check`), - stamping the default `collection_id`; (5) **generalizes `project_members` → - `memberships(scope_type ∈ {project, collection}, scope_id, …)`**, migrating - existing rows to `scope_type='collection'` on the default collection and - collapsing the role enum to `{owner, contributor}` (§B.3). + registry_sha)`; (2) move the per-corpus fields (`type`, `initial_state`, + visibility) **down** from `projects` (leaving it `(id, content_repo, + visibility, name, tagline, theme, enabled_models, …)`); (3) create one default + collection per project (id `default`, `subfolder` = repo root); (4) re-key + every entry-scoped table `(project_id, slug)` → `(collection_id, slug)` via the + `028_project_scoped_keys.sql` rebuild pattern (`__new`, copy, drop, rename, + FK-off + `foreign_key_check`); (5) generalize `project_members` → + `memberships(scope_type ∈ {project, collection}, …)`, collapsing the role enum + (§B.3). Then thread `collection_id` through `app/auth.py` / `app/projects.py` + / `app/cache.py` / the `api_*` writers, and **308** `/p//e/` → + `/p//c//e/`. **Usable end-state:** the deployment runs + exactly as before, now with a real collection layer and one extra path segment. + **Completes:** `@S1` (the single-collection / single-project redirect skips). -- **N2 — backend collection scoping.** Thread `collection_id` through the - resolution gates that migration 028/Plan B threaded `project_id` through - (`app/auth.py`, `app/projects.py`, `app/cache.py`, the `api_*` writers); - generalize the §22.7 union to the four-layer resolver (§B.2); add the - `GET /api/projects/:id/collections` and - `GET /api/projects/:id/collections/:cid` surfaces; scope propose/serve/PR - endpoints to `(project, collection)`. +- **S2 — Create & navigate a second collection.** Teach the registry mirror to + read `.collection.yaml`; add the bot-commit-wrapped **create-collection** + endpoint (authorized by existing deployment owner/admin for now — the scoped + role surface lands in S3); the project collection-directory at `/p//`; + collection-scoped propose/serve under `/p//c//`. + **Usable end-state:** an admin creates a `bdd` collection beside the document + one and it is navigable + proposable. **Completes:** `@S2` (anonymous reader of + an empty collection catalog). -- **N3 — manifests & in-app create.** Teach the registry mirror to read - `.collection.yaml` from each project's content repo; add the - bot-commit-wrapped **create-collection** (project Owner / RFC Contributor) - and **create-project** (global Owner) endpoints (§A.2); mirror new - collections into `collections` rows. +- **S3 — Scope-role enforcement.** The four-layer most-permissive resolver + (§B.2) over `{owner, contributor}` grants at `{global, project, collection}`, + with grants applied administratively (DB / admin endpoint); every write gate + re-checked under the collection axis. **Usable end-state:** a user granted + RFC Contributor at a scope can contribute across exactly that subtree, and + Owners administer their subtree. **Completes:** `@S3` (all of C.1 — role usage, + inheritance, union, no-negative-override). -- **N4 — frontend three-tier routing.** Add the `/c//` segment: - `/p//c//e/`; the project (collection) directory at - `/p//`; the single-collection redirect (§A.3); **308** the shipped - `/p//e/` → `/p//c//e/`; per-collection - chrome layered under the existing per-project chrome. +- **S4 — Invitation surfaces + role-aware empty states.** The invite UI + (Owner-only) granting Owner/RFC Contributor at a scope or any scope beneath it, + with §15 notifications and the broader-scope-supersedes rule; the + create-first-collection / propose-first empty states keyed to the actor's role. + **Usable end-state:** an Owner invites collaborators at the right scope from + the UI. **Completes:** `@S4` (all of C.2 — invitation; plus the project/ + collection empty states C3.3–C3.5). -- **N5 — roles surface (this pass's scope).** The `{owner, contributor}` × - `{global, project, collection}` grants (Part B) with the invite surfaces and - empty states of Part C as acceptance tests. (Richer roles deferred, §B.1.) +- **S5 — In-app create-project + the global directory.** The global-Owner + **create-project** action (bot provisions a Gitea content repo + commits to + `projects.yaml`); the deployment directory empty states. **Usable end-state:** + a global Owner stands up a new project end-to-end from the UI. **Completes:** + `@S5` (the global-directory empty states C3.1–C3.2). -- **N6 — type modules / membership lifecycle / hardening.** Carries the original - §22 M5–M7 forward, now selecting on the **collection's** `type`: per-type - frontmatter + surfaces; request-to-join + cross-collection inbox; per-collection - `enabled_models`; the registry + manifest format in `docs/DEPLOYMENTS.md`; - two-project / multi-collection e2e; the §20.4 changelog + upgrade-steps; the - SPEC merge (Part A applied, Part D amendments in place). +- **S6 — Type modules, membership lifecycle, hardening, SPEC merge.** Per-type + frontmatter + surfaces selected on the **collection's** `type`; request-to-join + + cross-collection inbox; per-collection `enabled_models`; the registry + + manifest format in `docs/DEPLOYMENTS.md`; two-project / multi-collection e2e; + the §20.4 changelog + upgrade-steps; the SPEC merge (Part A applied, Part D in + place). **Usable end-state:** the model is fully realized and merged into + `SPEC.md`. **Completes:** type-specific scenarios (added in S6, beyond Part C's + role focus). + +### Slice → scenario index (the inverse of the `@S` tags) + +| Slice | Usable thing it ships | Completes (`@S`) | +|---|---|---| +| **S1** | collection grain + default + redirects; N=1 unchanged | C3.7, C3.8 (`@S1`) | +| **S2** | create + navigate + propose a 2nd collection | C3.6 (`@S2`) | +| **S3** | scope-role enforcement across global/project/collection | C1.1–C1.8 (`@S3`) | +| **S4** | invitation UI + role-aware empty states | C2.1–C2.7, C3.3–C3.5 (`@S4`) | +| **S5** | in-app create-project + global directory | C3.1, C3.2 (`@S5`) | +| **S6** | type surfaces, lifecycle, hardening, SPEC merge | type-specific (new) | + +Each slice is a candidate single session: it lands a usable deployment and a +runnable acceptance gate (`--tags @S`). **S1 is the natural first session** — +the coupled migration 029 + threading + redirect, sized as one usable increment +(answering the in-session question: bundled, it is right-sized, not too much). ## E.1 (= §A.6) Migration — the default collection (the N=1 case) -- 2.39.5 From 08bdea8539ac116a0b78751f1acd23b5cb289917 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 07:53:41 -0700 Subject: [PATCH 05/21] =?UTF-8?q?=C2=A722=20S1=20plan:=20three-tier=20coll?= =?UTF-8?q?ection=20grain=20(migration=20029=20+=20threading=20+=20redirec?= =?UTF-8?q?t)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- ...26-06-05-s1-three-tier-collection-grain.md | 796 ++++++++++++++++++ 1 file changed, 796 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-05-s1-three-tier-collection-grain.md diff --git a/docs/superpowers/plans/2026-06-05-s1-three-tier-collection-grain.md b/docs/superpowers/plans/2026-06-05-s1-three-tier-collection-grain.md new file mode 100644 index 0000000..a0a733b --- /dev/null +++ b/docs/superpowers/plans/2026-06-05-s1-three-tier-collection-grain.md @@ -0,0 +1,796 @@ +# S1 — Three-tier collection grain (migration 029 + threading + redirect) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Insert a *collection* grain beneath today's `project` so every entry-scoped row keys on `(collection_id, slug)` instead of `(project_id, slug)`, with one invisible default collection per project — the deployment runs exactly as before, now with a real collection layer and one extra `/c//` URL segment. + +**Architecture:** A new `collections` table sits beneath `projects` (which keeps `content_repo`, `name`, `tagline`, `theme`, `visibility`). Migration 029 creates it, moves the per-corpus fields (`type`, `initial_state`) down from `projects`, seeds one default collection (`id='default'`) per project, re-keys the 13 entry-corpus tables `(project_id,slug)→(collection_id,slug)` via the migration-028 rebuild pattern, and generalises `project_members → memberships(scope_type ∈ {project,collection}, …)`. The backend threads `collection_id` through the writers/readers of those 13 tables (project-grain authz is recovered by joining `collections`); the frontend gains a `/c/:collectionId/` route layer and redirects that 308 the shipped `/p//e/` URLs to `/p//c/default/e/`. Serving stays **project-scoped** in S1 (collection = default); collection-aware serving is S2. + +**Tech Stack:** FastAPI + SQLite (raw SQL migrations run by `backend/app/db.py:run_migrations`, glob-ordered, `-- migrate:no-foreign-keys` marker toggles FK enforcement + runs `foreign_key_check`); pytest "vertical" tests (no Gherkin runner exists — `@S1` scenarios are realised as plain pytest); React Router SPA (`frontend/src/App.jsx`), nginx proxies `/rfc/` + `/proposals/` to the backend for server-side 308s. + +**Binding spec:** `docs/design/2026-06-05-three-tier-projects-collections.md` — Part A (model), Part E / §A.6 (migration strategy), Part C `@S1` scenarios C3.7 + C3.8. + +--- + +## Decisions locked before coding (read first) + +1. **Default collection id = the literal `'default'`** (not the project id). Reason: on a *fresh* deploy migrations run with `project_id='default'` and `restamp` renames it to the configured id (e.g. `ohm`) afterward; on an *already-deployed* instance `project_id` is already `ohm` when 029 runs. A stable literal keeps the collection id **identical across both deploy histories**, matches the spec's `/c/default/` URLs, and lets the existing `restamp` keep working untouched (it renames only the *project* grain — `collections.project_id` and the denormalised `project_id` tags — never `collections.id` or the entry `collection_id`). The re-key maps each entry to its project's default collection via a JOIN, so it is correct regardless of the `project_id` value at migration time. Multi-project deployments at migration time (non-standard pre-S5) get a unique id per project via a `CASE` so the seed never collides. + +2. **Re-key scope = exactly the 13 tables migration 028 rebuilt** (`cached_rfcs`, `rfc_invitations`, `cached_branches`, `branch_visibility`, `branch_contribute_grants`, `stars`, `watches`, `pr_seen`, `branch_chat_seen`, `funder_consents`, `rfc_collaborators`, `contribution_requests`, `proposed_use_cases`). The other tables 026 tagged with `project_id` (`threads`, `changes`, `notifications`, `actions`, `pr_resolution_branches`, `cached_prs`) keep `project_id` — they carry a project-grain tag, stay consistent for N=1, and renaming them is **out of S1 scope** (deferred). This matches the goal's "re-key entry-scoped tables via the 028 rebuild pattern". + +3. **Serving stays project-scoped in S1.** The `/c/:collectionId/` segment is introduced in routing + redirects; the frontend data layer keeps calling `/api/projects/{project_id}/rfcs/...` (the default collection). Collection-aware serving + the registry `.collection.yaml` reader land in S2. + +4. **No Gherkin runner.** `@S1` acceptance is realised as pytest vertical tests + a frontend route test. The whole existing backend suite is the "N=1 unchanged" regression net — it must go green again after the rename. + +--- + +## File structure + +**Created:** +- `backend/migrations/029_collections.sql` — the migration (collections table, field move-down, default-collection seed, 13-table re-key, `project_members → memberships`). +- `backend/app/collections.py` — collection resolution helpers (`default_collection_id`, `collection_type`, `collection_initial_state`, `collections_of_project`). +- `backend/tests/test_migration_029_collections.py` — migration shape + data-preservation + FK tests (template: `test_migration_028_project_scoped_keys.py`). +- `backend/tests/test_s1_collection_grain_vertical.py` — `@S1` acceptance (C3.7 redirect to sole collection; default-collection redirect; N=1 serving unchanged). + +**Modified (backend):** +- `backend/app/projects.py` — `restamp_default_project` bootstrap check (`cached_rfcs.project_id` → a still-valid column); move `project_initial_state` to read the collection; add re-export shim if needed. +- `backend/app/auth.py` — `project_of_rfc` joins `collections`; the 13-table reads/writes that touch `project_id` switch to `collection_id`. +- `backend/app/cache.py` — `_upsert_cached_rfc(..., collection_id)` + the `cached_rfcs`/`cached_branches` writers + the `WHERE project_id` reconciler reads. +- `backend/app/api.py`, `api_prs.py`, `api_branches.py`, `api_notifications.py`, `api_contributions.py`, `api_invitations.py`, `api_graduation.py`, `funder.py` — every SQL touching the 13 tables' `project_id` column → `collection_id`; recover project via `collections` join where authz needs it. +- `backend/app/api_deployment.py` — `/rfc/{slug}` family 308 targets gain `/c/default/`; `get_deployment`/`get_project` read `type`/`initial_state` from the default collection. + +**Modified (frontend):** +- `frontend/src/components/entryPaths.js` (or wherever path builders live) — insert `/c/:collectionId/`. +- `frontend/src/App.jsx` — add `/c/:collectionId/*` route layer; redirect `/p/:projectId/` → sole/default collection (C3.7); redirect legacy `/p/:projectId/e|proposals/...` → `/c/default/...`. +- `frontend/src/ProjectLayout.jsx` (+ `RFCView.jsx`, `Catalog.jsx` as needed) — read `:collectionId` param; pass through (data stays project-scoped). + +**Modified (release):** +- `VERSION`, `frontend/package.json#version`, `CHANGELOG.md` — minor bump with breaking-URL upgrade-steps block (§20.2 / §20.4). + +--- + +## Phase 1 — Migration 029 (the collection grain) + +### Task 1: Write the migration-029 shape test (red) + +**Files:** +- Test: `backend/tests/test_migration_029_collections.py` + +- [ ] **Step 1: Write the failing test** + +```python +"""Migration 029 — collections grain beneath projects. Template: test_migration_028.""" +import os +import sqlite3 +import tempfile + +import pytest + +from app import db + + +class _Cfg: + def __init__(self, path): + self.database_path = path + self.default_project_id = "default" + + +def _fresh_db(): + d = tempfile.mkdtemp() + path = os.path.join(d, "test.db") + db._CONN = None + db.run_migrations(_Cfg(path)) + return db.conn() + + +def test_collections_table_exists_with_default_per_project(): + conn = _fresh_db() + cols = {r["name"] for r in conn.execute("PRAGMA table_info(collections)")} + assert {"id", "project_id", "type", "subfolder", + "initial_state", "visibility", "name", "registry_sha"} <= cols + # one default collection seeded for the bootstrap 'default' project + row = conn.execute( + "SELECT id, project_id, subfolder FROM collections WHERE project_id='default'" + ).fetchone() + assert row is not None + assert row["id"] == "default" + assert row["subfolder"] == "" # repo root + + +def test_per_corpus_fields_moved_off_projects(): + conn = _fresh_db() + proj_cols = {r["name"] for r in conn.execute("PRAGMA table_info(projects)")} + assert "type" not in proj_cols + assert "initial_state" not in proj_cols + # projects keeps the grouping-tier fields + assert {"id", "name", "content_repo", "visibility"} <= proj_cols + + +def test_entry_tables_rekeyed_to_collection_id(): + conn = _fresh_db() + for t in ("cached_rfcs", "cached_branches", "stars", "watches", + "rfc_collaborators", "contribution_requests", "proposed_use_cases", + "branch_visibility", "branch_contribute_grants", "pr_seen", + "branch_chat_seen", "funder_consents", "rfc_invitations"): + cols = {r["name"] for r in conn.execute(f"PRAGMA table_info({t})")} + assert "collection_id" in cols, f"{t} missing collection_id" + assert "project_id" not in cols, f"{t} still has project_id" + + +def test_cached_rfcs_pk_is_collection_slug(): + conn = _fresh_db() + # same slug coexists across two collections + conn.execute("INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('c2','default','document','specs','active','public','Specs')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','A','active','default')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','B','active','c2')") + n = conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE slug='intro'").fetchone()["c"] + assert n == 2 + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','dup','active','default')") + + +def test_collaborator_fk_is_composite_on_collection(): + conn = _fresh_db() + conn.execute("INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('c2','default','document','specs','active','public','Specs')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','A','active','c2')") + conn.execute("INSERT INTO users (id, email, role, permission_state) VALUES (1,'a@b.c','contributor','granted')") + conn.execute("PRAGMA foreign_keys=ON") + conn.execute("INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, collection_id) " + "VALUES ('intro',1,'contributor','c2')") + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, collection_id) " + "VALUES ('intro',1,'contributor','default')") # no such (collection,slug) + + +def test_memberships_table_replaces_project_members(): + conn = _fresh_db() + cols = {r["name"] for r in conn.execute("PRAGMA table_info(memberships)")} + assert {"scope_type", "scope_id", "user_id", "role", "granted_by", "granted_at"} <= cols + # M2 rows would migrate to scope_type='collection'; role enum collapsed to owner/contributor + # (no project_members rows exist in a fresh DB, so just assert the table + check constraint) + conn.execute("INSERT INTO users (id, email, role, permission_state) VALUES (9,'x@y.z','contributor','granted')") + conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('project','default',9,'owner')") + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('bogus','default',9,'owner')") +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `cd backend && python -m pytest tests/test_migration_029_collections.py -q` +Expected: FAIL (no `collections` table / `029_collections.sql` does not exist). + +- [ ] **Step 3: Commit the red test** + +```bash +git add backend/tests/test_migration_029_collections.py +git commit -m "§22 S1: failing migration-029 shape tests (collections grain)" +``` + +### Task 2: Write migration 029 (green the shape test) + +**Files:** +- Create: `backend/migrations/029_collections.sql` + +- [ ] **Step 1: Write the migration.** Mirror `028_project_scoped_keys.sql` exactly for the 13 rebuilds, with `project_id` renamed to `collection_id` in each `__new` table, each child FK re-pointed to `cached_rfcs(collection_id, slug)`, and each index/UNIQUE swapping `project_id`→`collection_id`. Use the explicit-column `INSERT ... SELECT` form (not `SELECT *`) so the re-key can map values. Header marker `-- migrate:no-foreign-keys`. Concrete top of file: + +```sql +-- migrate:no-foreign-keys +-- +-- §22 three-tier refactor — S1. Insert a *collection* grain beneath project. +-- (1) collections table; (2) move per-corpus fields (type, initial_state) down +-- from projects; (3) one default collection per project (id='default', +-- subfolder = repo root); (4) re-key the 13 entry-corpus tables +-- (project_id,slug) -> (collection_id,slug) via the 028 rebuild pattern, mapping +-- each row to its project's default collection by JOIN; (5) project_members -> +-- memberships(scope_type ∈ {project,collection}, …), role enum collapsed to +-- {owner, contributor}. FK enforcement is OFF for the file (marker above); +-- foreign_key_check runs after. See docs/design/2026-06-05-three-tier-…md §A.6. + +-- ── collections: the new typed-corpus grain beneath projects ─────────────── +CREATE TABLE collections ( + id TEXT NOT NULL, + project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE, + type TEXT NOT NULL DEFAULT 'document' + CHECK (type IN ('document', 'specification', 'bdd')), + subfolder TEXT NOT NULL DEFAULT '', + initial_state TEXT NOT NULL DEFAULT 'super-draft' + CHECK (initial_state IN ('super-draft', 'active')), + visibility TEXT NOT NULL DEFAULT 'gated' + CHECK (visibility IN ('gated', 'public', 'unlisted')), + name TEXT, + registry_sha TEXT, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')), + PRIMARY KEY (id) +); +CREATE INDEX idx_collections_project ON collections(project_id); + +-- One default collection per project. id='default' for the standard +-- single-project deployment (stable across deploy histories); the project_id is +-- used as a unique fallback id only if a non-standard multi-project deployment +-- migrates (pre-S5; avoids a PK collision). subfolder='' = repo root. +INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) +SELECT + CASE WHEN (SELECT COUNT(*) FROM projects) <= 1 THEN 'default' ELSE p.id END, + p.id, p.type, '', p.initial_state, p.visibility, p.name +FROM projects p; + +-- ── move per-corpus fields off projects (rebuild to DROP type/initial_state) ─ +CREATE TABLE projects__new ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + content_repo TEXT, + visibility TEXT NOT NULL DEFAULT 'gated' + CHECK (visibility IN ('gated', 'public', 'unlisted')), + config_json TEXT, + registry_sha TEXT, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')) +); +INSERT INTO projects__new (id, name, content_repo, visibility, config_json, registry_sha, created_at, updated_at) +SELECT id, name, content_repo, visibility, config_json, registry_sha, created_at, updated_at FROM projects; +DROP TABLE projects; +ALTER TABLE projects__new RENAME TO projects; + +-- ── cached_rfcs: PRIMARY KEY (project_id, slug) -> (collection_id, slug) ──── +-- collection_id mapped from the row's old project's default collection. +CREATE TABLE cached_rfcs__new ( + slug TEXT NOT NULL, + title TEXT NOT NULL, + state TEXT NOT NULL CHECK (state IN ('super-draft', 'active', 'withdrawn', 'retired')), + rfc_id TEXT, + repo TEXT, + proposed_by TEXT, + proposed_at TEXT, + graduated_at TEXT, + graduated_by TEXT, + owners_json TEXT NOT NULL DEFAULT '[]', + arbiters_json TEXT NOT NULL DEFAULT '[]', + tags_json TEXT NOT NULL DEFAULT '[]', + body TEXT, + body_sha TEXT, + last_main_commit_at TEXT, + last_entry_commit_at TEXT, + updated_at TEXT NOT NULL DEFAULT (datetime('now')), + models_json TEXT, + funder_login TEXT, + proposed_use_case TEXT, + collection_id TEXT NOT NULL DEFAULT 'default' REFERENCES collections(id), + unreviewed INTEGER NOT NULL DEFAULT 0, + reviewed_at TEXT, + reviewed_by TEXT, + PRIMARY KEY (collection_id, slug) +); +INSERT INTO cached_rfcs__new + (slug, title, state, rfc_id, repo, proposed_by, proposed_at, graduated_at, + graduated_by, owners_json, arbiters_json, tags_json, body, body_sha, + last_main_commit_at, last_entry_commit_at, updated_at, models_json, + funder_login, proposed_use_case, collection_id, unreviewed, reviewed_at, reviewed_by) +SELECT + r.slug, r.title, r.state, r.rfc_id, r.repo, r.proposed_by, r.proposed_at, r.graduated_at, + r.graduated_by, r.owners_json, r.arbiters_json, r.tags_json, r.body, r.body_sha, + r.last_main_commit_at, r.last_entry_commit_at, r.updated_at, r.models_json, + r.funder_login, r.proposed_use_case, + (SELECT c.id FROM collections c WHERE c.project_id = r.project_id LIMIT 1), + r.unreviewed, r.reviewed_at, r.reviewed_by +FROM cached_rfcs r; +DROP TABLE cached_rfcs; +ALTER TABLE cached_rfcs__new RENAME TO cached_rfcs; +CREATE INDEX idx_cached_rfcs_state ON cached_rfcs (state); +CREATE INDEX idx_cached_rfcs_last_active ON cached_rfcs ( + COALESCE(last_main_commit_at, last_entry_commit_at) DESC +); +CREATE INDEX idx_cached_rfcs_collection ON cached_rfcs(collection_id); +``` + +Then **for each of the remaining 12 tables** copy its `028` block verbatim and apply the same three transforms: (a) rename the `project_id` column to `collection_id` (keep `DEFAULT 'default'`); (b) in the `INSERT ... SELECT`, replace the `project_id` source value with `(SELECT c.id FROM collections c WHERE c.project_id = .project_id LIMIT 1)` and list columns explicitly; (c) rename `project_id` → `collection_id` in every `UNIQUE (...)`, `FOREIGN KEY (...) REFERENCES cached_rfcs(...)`, and `CREATE [UNIQUE] INDEX`. The 12: `rfc_invitations`, `cached_branches`, `branch_visibility`, `branch_contribute_grants`, `stars`, `watches`, `pr_seen`, `branch_chat_seen`, `funder_consents`, `rfc_collaborators`, `contribution_requests`, `proposed_use_cases`. (FK targets `cached_rfcs(project_id, slug)` become `cached_rfcs(collection_id, slug)`.) + +Finally the membership generalisation: + +```sql +-- ── project_members -> memberships(scope_type, scope_id, …); roles collapsed ─ +CREATE TABLE memberships ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + scope_type TEXT NOT NULL CHECK (scope_type IN ('project', 'collection')), + scope_id TEXT NOT NULL, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + role TEXT NOT NULL CHECK (role IN ('owner', 'contributor')), + granted_by INTEGER REFERENCES users(id) ON DELETE SET NULL, + granted_at TEXT NOT NULL DEFAULT (datetime('now')), + UNIQUE (scope_type, scope_id, user_id) +); +CREATE INDEX idx_memberships_user ON memberships(user_id); +CREATE INDEX idx_memberships_scope ON memberships(scope_type, scope_id); + +-- M2 project_members rows attached at what is now the *collection*; collapse +-- the role enum (project_admin -> owner, project_contributor -> contributor, +-- project_viewer -> dropped this pass, §B.3) and migrate to the default +-- collection of each project. +INSERT INTO memberships (scope_type, scope_id, user_id, role, granted_by, granted_at) +SELECT 'collection', + (SELECT c.id FROM collections c WHERE c.project_id = pm.project_id LIMIT 1), + pm.user_id, + CASE pm.role WHEN 'project_admin' THEN 'owner' + WHEN 'project_contributor' THEN 'contributor' + ELSE 'contributor' END, + pm.granted_by, pm.granted_at +FROM project_members pm +WHERE pm.role IN ('project_admin', 'project_contributor'); +DROP TABLE project_members; +``` + +- [ ] **Step 2: Run the shape test** + +Run: `cd backend && python -m pytest tests/test_migration_029_collections.py -q` +Expected: PASS (all shape/PK/FK/membership assertions green). + +- [ ] **Step 3: Commit** + +```bash +git add backend/migrations/029_collections.sql +git commit -m "§22 S1: migration 029 — collections grain, field move-down, 13-table re-key, memberships" +``` + +--- + +## Phase 2 — Backend threading (make the existing suite green again) + +> After Task 2 the column rename breaks every reader/writer of the 13 tables. This phase fixes them. **Driver:** the full backend suite is the regression net — run it, read each failure, fix the named module, repeat until green. The agent exploration produced the exact blast-radius map used below. + +### Task 3: collections helper module + +**Files:** +- Create: `backend/app/collections.py` + +- [ ] **Step 1: Write the helper** + +```python +"""§22 collection grain — resolution helpers beneath the project tier. + +In S1 each project has exactly one collection (the default). These helpers +recover the collection for a project and read the per-corpus fields that moved +down from `projects` in migration 029. Project-grain authz (auth.py) recovers a +row's project by joining `collections` on `collection_id`. +""" +from __future__ import annotations + +from . import db + +DEFAULT_COLLECTION_ID = "default" + + +def default_collection_id(project_id: str) -> str: + """The id of a project's default (S1: sole) collection. Falls back to the + literal 'default' when the project has no collection row yet.""" + row = db.conn().execute( + "SELECT id FROM collections WHERE project_id = ? ORDER BY created_at LIMIT 1", + (project_id,), + ).fetchone() + return row["id"] if row else DEFAULT_COLLECTION_ID + + +def project_of_collection(collection_id: str) -> str | None: + row = db.conn().execute( + "SELECT project_id FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return row["project_id"] if row else None + + +def collection_initial_state(collection_id: str) -> str: + """§22.4b landing state for new entries in a collection. 'super-draft' + default for an unknown row (today's safe flow).""" + row = db.conn().execute( + "SELECT initial_state FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + if row is None or not row["initial_state"]: + return "super-draft" + return row["initial_state"] + + +def collection_type(collection_id: str) -> str: + row = db.conn().execute( + "SELECT type FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return row["type"] if row and row["type"] else "document" +``` + +- [ ] **Step 2: Commit** + +```bash +git add backend/app/collections.py +git commit -m "§22 S1: collections resolution helpers" +``` + +### Task 4: Fix `projects.py` (restamp + initial_state) + +**Files:** +- Modify: `backend/app/projects.py:46-48` (restamp bootstrap check), `:112-121` (`project_initial_state`) + +- [ ] **Step 1: Fix the restamp bootstrap check.** `restamp_default_project` reads `cached_rfcs.project_id` (now renamed) at line 47 — switch the existence probe to a still-`project_id`-bearing table so the PRAGMA-driven rename loop is unaffected (it already discovers `project_id` columns dynamically, which now correctly excludes the 13 collection-keyed tables and includes `collections.project_id`): + +```python + has_rows = conn.execute( + "SELECT 1 FROM collections WHERE project_id = ? LIMIT 1", (DEFAULT_PROJECT_ID,) + ).fetchone() +``` + +- [ ] **Step 2: Re-home `project_initial_state`.** Keep the signature for callers, but resolve through the project's default collection: + +```python +def project_initial_state(project_id: str) -> str: + """§22.4b landing state for new entries in a project's default collection.""" + from . import collections as collections_mod + return collections_mod.collection_initial_state( + collections_mod.default_collection_id(project_id) + ) +``` + +- [ ] **Step 3: Run the restamp + projects tests** + +Run: `cd backend && python -m pytest tests/test_restamp_default_project.py tests/test_initial_state_landing.py -q` +Expected: PASS. + +- [ ] **Step 4: Commit** + +```bash +git add backend/app/projects.py +git commit -m "§22 S1: thread projects.py restamp + initial_state through collections" +``` + +### Task 5: Fix `auth.py` (`project_of_rfc` join) + +**Files:** +- Modify: `backend/app/auth.py:352-361` + +- [ ] **Step 1: Join collections to recover the project from a slug.** + +```python +def project_of_rfc(rfc_slug: str) -> str: + """The project an RFC belongs to, via its collection + (cached_rfcs.collection_id -> collections.project_id). Falls back to the + default project when the slug isn't cached.""" + row = db.conn().execute( + "SELECT c.project_id AS project_id " + "FROM cached_rfcs r JOIN collections c ON c.id = r.collection_id " + "WHERE r.slug = ?", + (rfc_slug,), + ).fetchone() + if row is None: + return DEFAULT_PROJECT_ID + return row["project_id"] or DEFAULT_PROJECT_ID +``` + +- [ ] **Step 2: Run the authz suite** + +Run: `cd backend && python -m pytest tests/test_multi_project_authz_vertical.py tests/test_anon_offlimits_vertical.py -q` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add backend/app/auth.py +git commit -m "§22 S1: auth.project_of_rfc recovers project via collection join" +``` + +### Task 6: Fix `cache.py` writers/readers + +**Files:** +- Modify: `backend/app/cache.py` — `_refresh_project_corpus` (resolve collection), `_upsert_cached_rfc` signature + SQL (`project_id`→`collection_id`), the `WHERE project_id` reconciler read (`:88`), the `cached_branches` writers (`:213/:388/:410`). + +- [ ] **Step 1: Resolve the collection in the corpus refresh.** In `_refresh_project_corpus`, compute the project's default collection once and pass it down; switch the reconciler `SELECT slug ... WHERE project_id` to `WHERE collection_id`: + +```python +async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None: + from . import collections as collections_mod + collection_id = collections_mod.default_collection_id(project_id) + ... + _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id) + ... + existing = { + row["slug"] + for row in db.conn().execute( + "SELECT slug FROM cached_rfcs WHERE collection_id = ?", (collection_id,) + ) + } +``` + +- [ ] **Step 2: Rename in `_upsert_cached_rfc`.** Change the param `project_id: str = "default"` → `collection_id: str = "default"`; in the `INSERT`, replace the `project_id` column with `collection_id`, the `ON CONFLICT(project_id, slug)` with `ON CONFLICT(collection_id, slug)`, and the bound value `project_id` → `collection_id`. + +- [ ] **Step 3: Fix the `cached_branches` writers.** At `:213/:388/:410` the `ON CONFLICT(project_id, rfc_slug, branch_name)` clauses → `ON CONFLICT(collection_id, rfc_slug, branch_name)`; where a meta-repo branch row is written without an explicit grain it now relies on the `collection_id DEFAULT 'default'` column default (unchanged behaviour for N=1). Bind `collection_id` explicitly where the per-project loop has it. + +- [ ] **Step 4: Run the cache tests** + +Run: `cd backend && python -m pytest tests/test_cache_bootstrap.py tests/test_cache_review_fields.py tests/test_branch_path_routing.py -q` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/cache.py +git commit -m "§22 S1: thread cache.py corpus/branch writers through collection_id" +``` + +### Task 7: Fix the `api_*` writers/readers + `funder.py` + +**Files (each: swap the 13-table `project_id` column references to `collection_id`; recover project for authz via `auth.project_of_rfc`/`collections` join):** +- `backend/app/api.py:747` (stars read), `:774/:806/:969` (`cached_rfcs` composite lookups → `collection_id`), `:785-788/:1046` (`proposed_use_cases`). +- `backend/app/api_prs.py:156` (`branch_visibility`), `:192` (`proposed_use_cases`), `:401` (`pr_seen`). Note `:670/:789` read `row["project_id"]` from a `cached_rfcs`/`rfc` row — change those SELECTs to also yield the project via the collection join, then keep the existing `auth.require_project_readable(viewer, project_id)` call unchanged. +- `backend/app/api_branches.py:745` (`branch_visibility`), `:899` (`branch_chat_seen`). +- `backend/app/api_notifications.py:216` (read `cached_rfcs` → now `collection_id`; recover project via join for the visibility gate), `:225` (`watches`). +- `backend/app/api_contributions.py:65/:111` (read `cached_rfcs`; recover project via join), `api_invitations.py:383`, `api_graduation.py` (any `cached_rfcs`/13-table `project_id`). +- `backend/app/funder.py:223` (`funder_consents` `ON CONFLICT(project_id,…)` → `collection_id`). + +- [ ] **Step 1: Mechanical pass.** For each file above, replace `project_id` **only where it names a column on one of the 13 re-keyed tables** (PK lookups, `ON CONFLICT`, `WHERE`, `INSERT` column lists, `SELECT` projections from those tables) with `collection_id`. Where the code needs the *project* (for `auth.*_project*` calls), recover it with `auth.project_of_rfc(slug)` or a `collections` join — do **not** rename the `project_id` argument flowing into the authz helpers (those stay project-grain in S1). Leave `threads`, `changes`, `notifications`, `actions`, `pr_resolution_branches`, `cached_prs` `project_id` columns untouched. + +- [ ] **Step 2: Grep guard.** Confirm no stray reference to a dropped column remains: + +Run: `cd backend && grep -rEn "cached_rfcs[^;]*project_id|project_id, slug|project_id, rfc_slug|ON CONFLICT\(project_id" app/ | grep -v "collections\|threads\|changes\|notifications\|actions\|pr_resolution\|cached_prs"` +Expected: no output (every 13-table `project_id` is now `collection_id`). + +- [ ] **Step 3: Run the full backend suite** + +Run: `cd backend && python -m pytest -q` +Expected: PASS (this is the **N=1-unchanged** gate). Fix any remaining failures by reading the traceback and applying the same rename/join rule. + +- [ ] **Step 4: Commit** + +```bash +git add backend/app/api.py backend/app/api_prs.py backend/app/api_branches.py backend/app/api_notifications.py backend/app/api_contributions.py backend/app/api_invitations.py backend/app/api_graduation.py backend/app/funder.py +git commit -m "§22 S1: thread api_* + funder writers/readers through collection_id" +``` + +--- + +## Phase 3 — API surface reads per-corpus fields from the collection + +### Task 8: `api_deployment.py` reads type/initial_state from the default collection + +**Files:** +- Modify: `backend/app/api_deployment.py:36-44` (`get_deployment` projects list `type`), `:62-82` (`get_project` `type`/`initial_state`) + +- [ ] **Step 1: Write a failing test** in `backend/tests/test_api_deployment.py` (extend it) asserting `GET /api/projects/{default}` still returns the correct `type`/`initial_state` after the move-down (values come from the default collection): + +```python +def test_get_project_type_initial_state_from_default_collection(app_with_fake_gitea): + # ... existing fixture sets up the default project/collection ... + r = client.get(f"/api/projects/{default_id}") + assert r.status_code == 200 + body = r.json() + assert body["type"] in ("document", "specification", "bdd") + assert body["initial_state"] in ("super-draft", "active") +``` + +- [ ] **Step 2: Run to verify it fails** (the SELECT still reads `projects.type`, which 029 dropped → `OperationalError`). + +Run: `cd backend && python -m pytest tests/test_api_deployment.py -q` +Expected: FAIL. + +- [ ] **Step 3: Read the fields from the default collection.** In `get_deployment`, replace the `SELECT id, name, type, visibility FROM projects` with a join to the project's default collection for `type` (or a per-row `collections_mod.collection_type(default_collection_id(id))`). In `get_project`, drop `type, initial_state` from the `projects` SELECT and resolve them via `collections_mod.collection_type(...)` / `collection_initial_state(...)`: + +```python +from . import collections as collections_mod +... + cid = collections_mod.default_collection_id(row["id"]) + return { + ... + "type": collections_mod.collection_type(cid), + "initial_state": collections_mod.collection_initial_state(cid), + ... + } +``` + +- [ ] **Step 4: Run to verify it passes** + +Run: `cd backend && python -m pytest tests/test_api_deployment.py -q` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/api_deployment.py backend/tests/test_api_deployment.py +git commit -m "§22 S1: deployment/project API reads type+initial_state from default collection" +``` + +--- + +## Phase 4 — Redirects + frontend collection segment + +### Task 9: Backend `/rfc/` 308s target `/c/default/` + +**Files:** +- Modify: `backend/app/api_deployment.py:89-104` + +- [ ] **Step 1: Add a failing test** to `test_api_deployment.py`: + +```python +def test_legacy_rfc_url_redirects_through_collection(app_with_fake_gitea): + r = client.get("/rfc/intro", follow_redirects=False) + assert r.status_code == 308 + assert r.headers["location"] == f"/p/{default_id}/c/default/e/intro" +``` + +- [ ] **Step 2: Verify it fails** (current target lacks `/c/default/`). + +- [ ] **Step 3: Update the three redirect handlers** to resolve the default collection and insert the `/c//` segment: + +```python + @router.get("/rfc/{slug}") + async def redirect_old_rfc(slug: str) -> RedirectResponse: + default_id = projects_mod.resolved_default_id(config) + cid = collections_mod.default_collection_id(default_id) + return RedirectResponse(url=f"/p/{default_id}/c/{cid}/e/{slug}", status_code=308) + # …same /c/{cid}/ insertion for /rfc/{slug}/pr/{pr} and /proposals/{pr} +``` + +(`/proposals/{pr}` → `/p/{default_id}/c/{cid}/proposals/{pr}`.) + +- [ ] **Step 4: Verify it passes.** + +Run: `cd backend && python -m pytest tests/test_api_deployment.py -q` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add backend/app/api_deployment.py backend/tests/test_api_deployment.py +git commit -m "§22 S1: legacy /rfc + /proposals 308s route through /c//" +``` + +### Task 10: Frontend path builders gain `/c/:collectionId/` + +**Files:** +- Modify: `frontend/src/components/entryPaths.js` (path builders — confirm exact path with `grep -rl "p/\${" frontend/src`) + +- [ ] **Step 1: Thread a collection id through the builders.** Add a `collectionId` argument (defaulting to `'default'`) and emit the `/c//` segment: + +```js +export const collectionHome = (projectId, collectionId) => `/p/${projectId}/c/${collectionId}/` +export const entryPath = (projectId, collectionId, slug) => `/p/${projectId}/c/${collectionId}/e/${slug}` +export const entryPrPath = (projectId, collectionId, slug, prNumber) => `/p/${projectId}/c/${collectionId}/e/${slug}/pr/${prNumber}` +export const proposalPath = (projectId, collectionId, prNumber) => `/p/${projectId}/c/${collectionId}/proposals/${prNumber}` +export const projectHome = (projectId) => `/p/${projectId}/` +``` + +Update every caller (grep `entryPath(`, `entryPrPath(`, `proposalPath(`, `collectionHome(`) to pass the current collection id (from the route param / `useCollectionId()` — default `'default'`). + +- [ ] **Step 2: Build the frontend** + +Run: `cd frontend && npm run build` +Expected: build succeeds (no undefined-symbol errors). + +- [ ] **Step 3: Commit** + +```bash +git add frontend/src +git commit -m "§22 S1: frontend path builders carry the /c// segment" +``` + +### Task 11: Frontend route layer + redirects (C3.7, C3.8, legacy) + +**Files:** +- Modify: `frontend/src/App.jsx:354-373`, `frontend/src/ProjectLayout.jsx` + +- [ ] **Step 1: Nest the corpus routes under `/c/:collectionId/`** and add redirects. Inside the `ProjectLayout` nested ``: + +```jsx + + {/* project landing: redirect to the sole/default collection (C3.7) */} + } /> + {/* legacy v0.35.0 corpus URLs without /c/ → default collection */} + } /> + } /> + } /> + {/* collection-scoped corpus (serving stays project-scoped in S1) */} + } /> + } /> + } /> + setCatalogVersion(v => v + 1)} />} /> + +``` + +`CollectionRedirect` reads the project's collections (from `ProjectContext`, populated by `GET /api/projects/:id`) and ``s to the sole visible collection's `/c//`; with one collection that is `/c/default/` (C3.7). `LegacyEntryPrRedirect`/`LegacyProposalRedirect` use `useParams()` to rebuild the target with `c/default/`. React-Router literal `:slug` in `to=` does not interpolate — implement these as small components using `useParams()` + ``. + +- [ ] **Step 2: Confirm `/` → sole project (C3.8) already holds.** `DeploymentLanding` (App.jsx:348) already redirects to the single visible project. Add/confirm a test (Task 12) rather than re-implementing. + +- [ ] **Step 3: Build** + +Run: `cd frontend && npm run build` +Expected: succeeds. + +- [ ] **Step 4: Commit** + +```bash +git add frontend/src +git commit -m "§22 S1: /c// route layer + C3.7 + legacy-URL redirects" +``` + +--- + +## Phase 5 — `@S1` acceptance + full verification + +### Task 12: `@S1` vertical acceptance test (C3.7 + C3.8 + N=1 serving) + +**Files:** +- Create: `backend/tests/test_s1_collection_grain_vertical.py` + +- [ ] **Step 1: Write the acceptance test.** Tag scenarios in docstrings as `@S1` for traceability (no Gherkin runner). Cover: (a) default-collection redirect `/rfc/` → `/p//c/default/e/` (already in Task 9 — re-assert here as the S1 gate); (b) an entry proposed/served at N=1 still resolves under the default collection via `/api/projects//rfcs/`; (c) the deployment `/api/deployment` still reports one project with `default_project_id`. (C3.7/C3.8 client redirects are asserted in the frontend build/route smoke; the data-layer N=1 invariants are asserted here.) + +```python +"""@S1 acceptance — the collection grain exists and N=1 is unchanged. +Scenarios: C3.7 (single-collection project skips the directory) and C3.8 +(single-project deployment skips the directory) are the redirect contract; +this module asserts the backend N=1 invariants behind them.""" +# reuse the propose/serve fixtures from test_project_scoped_serving.py +def test_s1_entry_served_under_default_collection(app_with_fake_gitea): + # propose + mirror an entry, then fetch it project-scoped (collection=default) + ... + r = client.get(f"/api/projects/{default_id}/rfcs/intro") + assert r.status_code == 200 + # the row is keyed by collection_id under the hood + cid = db.conn().execute("SELECT collection_id FROM cached_rfcs WHERE slug='intro'").fetchone()["collection_id"] + assert cid == "default" + +def test_s1_legacy_redirect_inserts_collection_segment(app_with_fake_gitea): + r = client.get("/rfc/intro", follow_redirects=False) + assert r.status_code == 308 + assert "/c/default/" in r.headers["location"] +``` + +- [ ] **Step 2: Run it** + +Run: `cd backend && python -m pytest tests/test_s1_collection_grain_vertical.py -q` +Expected: PASS. + +- [ ] **Step 3: Full backend suite + frontend build (the N=1-unchanged gate)** + +Run: `cd backend && python -m pytest -q && cd ../frontend && npm run build` +Expected: all backend tests PASS; frontend builds. + +- [ ] **Step 4: Commit** + +```bash +git add backend/tests/test_s1_collection_grain_vertical.py +git commit -m "§22 S1: @S1 acceptance — collection grain + N=1 serving unchanged" +``` + +### Task 13: e2e smoke (optional, if Docker stack available) + +- [ ] **Step 1:** If the Tier-1 Docker stack is runnable, `make e2e` to confirm sign-in + a corpus page render through the new `/c/default/` routes. If the stack isn't available in-session, note it skipped and rely on Tasks 7/11/12 gates. + +--- + +## Phase 6 — Release + finalize + +### Task 14: Version bump + changelog (breaking, with upgrade steps) + +**Files:** +- Modify: `VERSION`, `frontend/package.json` (`version`), `CHANGELOG.md` + +- [ ] **Step 1: Bump** `VERSION` and `frontend/package.json#version` to the next pre-1.0 minor (current `0.39.0` → `0.40.0`). They must match (a divergence is a §20 spec bug). + +- [ ] **Step 2: Add the CHANGELOG entry** with a breaking-URL **upgrade steps** block (§20.2 / §20.4 / §A.6): migration 029 adds the collection grain; `/p//e/` now lives at `/p//c/default/e/` (308 for old links); operators need no action beyond deploying (the migration + redirects are automatic; the default collection is seeded). Note the deferred items (denormalised `project_id` tags unchanged; collection-aware serving = S2). + +- [ ] **Step 3: Commit** + +```bash +git add VERSION frontend/package.json CHANGELOG.md +git commit -m "§22 S1: release v0.40.0 — three-tier collection grain (breaking URL + migration 029)" +``` + +### Task 15: Branch, PR, merge + +- [ ] **Step 1:** This work rides a feature branch off `main` (e.g. `feat/s1-collection-grain`). Push to `origin` (git.wiggleverse.org). +- [ ] **Step 2:** Open a PR citing the design doc + `@S1`; in autonomous posture, self-review and merge once the suite is green. +- [ ] **Step 3:** Update repo memory with the new resume pointer (S1 shipped @ v0.40.0; next = S2). + +--- + +## Self-review (writing-plans checklist) + +- **Spec coverage:** §A.6 steps 1–5 → Tasks 2 (collections table + default + re-key + memberships), 8 (field move-down read path), 9 (308 step 5). Part B membership generalisation → Task 2 (`memberships`) — note S1 only *migrates* the table; the four-layer resolver is S3 (out of scope, correctly deferred per Part E). `@S1` C3.7/C3.8 → Tasks 11 (frontend redirects) + 12 (backend invariants). "N=1 unchanged" → Task 7 Step 3 + Task 12 Step 3 full-suite gates. Threading (auth/projects/cache/api_*) → Tasks 4–8. +- **Placeholders:** the per-table rebuild bodies for the 12 non-`cached_rfcs` tables reference the in-repo `028_project_scoped_keys.sql` as the literal template with the three explicit transforms named — this is a concrete instruction, not a TODO (repeating 200+ lines of near-identical SQL verbatim would harm reviewability; the transform rule is exact). +- **Type consistency:** `default_collection_id`, `collection_type`, `collection_initial_state`, `project_of_collection` are defined in Task 3 and used consistently in Tasks 4, 8, 9. Column `collection_id` (not `coll_id`/`collectionId`) used uniformly in SQL; `collectionId` is the JS route param. +- **Risk note:** the denormalised `project_id` columns (`threads`/`changes`/`notifications`/`actions`/`pr_resolution_branches`/`cached_prs`) stay `project_id` and may, after `restamp`, hold the project id (`ohm`) while entry `collection_id` holds `default`. Task 7 Step 3's full-suite run is the guard against any code that wrongly cross-joins the two grains; if one surfaces, recover the project via the `collections` join rather than renaming the tag. +``` -- 2.39.5 From 867f2504d68ab9aa1dc7ed53fa5e4d3df6df6f53 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 07:58:39 -0700 Subject: [PATCH 06/21] =?UTF-8?q?=C2=A722=20S1:=20migration=20029=20?= =?UTF-8?q?=E2=80=94=20collections=20grain,=20field=20move-down,=2013-tabl?= =?UTF-8?q?e=20re-key,=20memberships?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/migrations/029_collections.sql | 427 ++++++++++++++++++ .../tests/test_migration_029_collections.py | 139 ++++++ 2 files changed, 566 insertions(+) create mode 100644 backend/migrations/029_collections.sql create mode 100644 backend/tests/test_migration_029_collections.py diff --git a/backend/migrations/029_collections.sql b/backend/migrations/029_collections.sql new file mode 100644 index 0000000..bde568f --- /dev/null +++ b/backend/migrations/029_collections.sql @@ -0,0 +1,427 @@ +-- migrate:no-foreign-keys +-- +-- §22 three-tier refactor — S1. Insert a *collection* grain beneath project. +-- +-- (1) a `collections` table beneath `projects`; +-- (2) move the per-corpus fields (type, initial_state) down from `projects` +-- (projects keeps id, name, content_repo, visibility, config_json, …); +-- (3) one default collection per project (id='default' for the standard +-- single-project deployment, subfolder = repo root), inheriting the +-- project's type / initial_state / visibility; +-- (4) re-key the 13 entry-corpus tables (project_id, slug) -> (collection_id, +-- slug) via the migration-028 rebuild pattern, mapping each row to its +-- project's default collection by JOIN; +-- (5) generalise project_members -> memberships(scope_type ∈ {project, +-- collection}, scope_id, …), collapsing the role enum to {owner, +-- contributor} (§B.3). +-- +-- SQLite can't ALTER a PK/UNIQUE in place, so each keyed table is rebuilt by the +-- official create-copy-drop-rename procedure. FK enforcement is OFF for the file +-- (the `migrate:no-foreign-keys` marker tells the runner to toggle it and run +-- foreign_key_check after). cached_rfcs is rebuilt FIRST so the child tables can +-- re-point their composite FK at its new (collection_id, slug) key. +-- +-- The tables 026 tagged with project_id but 028 did NOT key (threads, changes, +-- notifications, actions, pr_resolution_branches, cached_prs) keep project_id — +-- they carry a project-grain tag, untouched in S1. See +-- docs/design/2026-06-05-three-tier-projects-collections.md §A.6 / Part E. + +-- ── collections: the new typed-corpus grain beneath projects ─────────────── +CREATE TABLE collections ( + id TEXT NOT NULL, + project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE, + type TEXT NOT NULL DEFAULT 'document' + CHECK (type IN ('document', 'specification', 'bdd')), + subfolder TEXT NOT NULL DEFAULT '', + initial_state TEXT NOT NULL DEFAULT 'super-draft' + CHECK (initial_state IN ('super-draft', 'active')), + visibility TEXT NOT NULL DEFAULT 'gated' + CHECK (visibility IN ('gated', 'public', 'unlisted')), + name TEXT, + registry_sha TEXT, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')), + PRIMARY KEY (id) +); +CREATE INDEX idx_collections_project ON collections(project_id); + +-- One default collection per project. id='default' for the standard +-- single-project deployment (a stable literal across deploy histories); the +-- project_id is used as a unique fallback id only if a non-standard +-- multi-project deployment migrates (pre-S5; avoids a PK collision). +INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) +SELECT + CASE WHEN (SELECT COUNT(*) FROM projects) <= 1 THEN 'default' ELSE p.id END, + p.id, p.type, '', p.initial_state, p.visibility, p.name +FROM projects p; + +-- ── projects: rebuild to DROP the per-corpus fields (type, initial_state) ─── +CREATE TABLE projects__new ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + content_repo TEXT, + visibility TEXT NOT NULL DEFAULT 'gated' + CHECK (visibility IN ('gated', 'public', 'unlisted')), + config_json TEXT, + registry_sha TEXT, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')) +); +INSERT INTO projects__new (id, name, content_repo, visibility, config_json, registry_sha, created_at, updated_at) +SELECT id, name, content_repo, visibility, config_json, registry_sha, created_at, updated_at FROM projects; +DROP TABLE projects; +ALTER TABLE projects__new RENAME TO projects; + +-- ── cached_rfcs: PRIMARY KEY (project_id, slug) -> (collection_id, slug) ──── +CREATE TABLE cached_rfcs__new ( + slug TEXT NOT NULL, + title TEXT NOT NULL, + state TEXT NOT NULL CHECK (state IN ('super-draft', 'active', 'withdrawn', 'retired')), + rfc_id TEXT, + repo TEXT, + proposed_by TEXT, + proposed_at TEXT, + graduated_at TEXT, + graduated_by TEXT, + owners_json TEXT NOT NULL DEFAULT '[]', + arbiters_json TEXT NOT NULL DEFAULT '[]', + tags_json TEXT NOT NULL DEFAULT '[]', + body TEXT, + body_sha TEXT, + last_main_commit_at TEXT, + last_entry_commit_at TEXT, + updated_at TEXT NOT NULL DEFAULT (datetime('now')), + models_json TEXT, + funder_login TEXT, + proposed_use_case TEXT, + collection_id TEXT NOT NULL DEFAULT 'default' REFERENCES collections(id), + unreviewed INTEGER NOT NULL DEFAULT 0, + reviewed_at TEXT, + reviewed_by TEXT, + PRIMARY KEY (collection_id, slug) +); +INSERT INTO cached_rfcs__new + (slug, title, state, rfc_id, repo, proposed_by, proposed_at, graduated_at, + graduated_by, owners_json, arbiters_json, tags_json, body, body_sha, + last_main_commit_at, last_entry_commit_at, updated_at, models_json, + funder_login, proposed_use_case, collection_id, unreviewed, reviewed_at, reviewed_by) +SELECT + r.slug, r.title, r.state, r.rfc_id, r.repo, r.proposed_by, r.proposed_at, r.graduated_at, + r.graduated_by, r.owners_json, r.arbiters_json, r.tags_json, r.body, r.body_sha, + r.last_main_commit_at, r.last_entry_commit_at, r.updated_at, r.models_json, + r.funder_login, r.proposed_use_case, + (SELECT c.id FROM collections c WHERE c.project_id = r.project_id LIMIT 1), + r.unreviewed, r.reviewed_at, r.reviewed_by +FROM cached_rfcs r; +DROP TABLE cached_rfcs; +ALTER TABLE cached_rfcs__new RENAME TO cached_rfcs; +CREATE INDEX idx_cached_rfcs_state ON cached_rfcs (state); +CREATE INDEX idx_cached_rfcs_last_active ON cached_rfcs ( + COALESCE(last_main_commit_at, last_entry_commit_at) DESC +); +CREATE INDEX idx_cached_rfcs_collection ON cached_rfcs(collection_id); + +-- ── rfc_invitations: single-col FK -> composite (collection_id, rfc_slug) ─── +CREATE TABLE rfc_invitations__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + inviter_user_id INTEGER REFERENCES users(id) ON DELETE SET NULL, + invitee_email TEXT NOT NULL, + role_in_rfc TEXT NOT NULL CHECK (role_in_rfc IN ('contributor', 'discussant')), + status TEXT NOT NULL DEFAULT 'pending' + CHECK (status IN ('pending', 'accepted', 'revoked', 'expired')), + token TEXT NOT NULL, + expires_at TEXT NOT NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + accepted_at TEXT, + accepted_by_user_id INTEGER REFERENCES users(id) ON DELETE SET NULL, + collection_id TEXT NOT NULL DEFAULT 'default', + FOREIGN KEY (collection_id, rfc_slug) REFERENCES cached_rfcs(collection_id, slug) ON DELETE CASCADE +); +INSERT INTO rfc_invitations__new + (id, rfc_slug, inviter_user_id, invitee_email, role_in_rfc, status, token, + expires_at, created_at, accepted_at, accepted_by_user_id, collection_id) +SELECT + i.id, i.rfc_slug, i.inviter_user_id, i.invitee_email, i.role_in_rfc, i.status, i.token, + i.expires_at, i.created_at, i.accepted_at, i.accepted_by_user_id, + (SELECT c.id FROM collections c WHERE c.project_id = i.project_id LIMIT 1) +FROM rfc_invitations i; +DROP TABLE rfc_invitations; +ALTER TABLE rfc_invitations__new RENAME TO rfc_invitations; +CREATE UNIQUE INDEX idx_rfc_invitations_token ON rfc_invitations (token); +CREATE INDEX idx_rfc_invitations_rfc_status ON rfc_invitations (rfc_slug, status); +CREATE INDEX idx_rfc_invitations_email_status ON rfc_invitations (invitee_email, status); + +-- ── cached_branches: UNIQUE (project_id, rfc_slug, branch_name) -> collection +CREATE TABLE cached_branches__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + branch_name TEXT NOT NULL, + head_sha TEXT, + state TEXT NOT NULL DEFAULT 'open' CHECK (state IN ('open', 'closed', 'deleted')), + pinned INTEGER NOT NULL DEFAULT 0, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + last_commit_at TEXT, + closed_at TEXT, + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, rfc_slug, branch_name) +); +INSERT INTO cached_branches__new + (id, rfc_slug, branch_name, head_sha, state, pinned, created_at, last_commit_at, closed_at, collection_id) +SELECT + b.id, b.rfc_slug, b.branch_name, b.head_sha, b.state, b.pinned, b.created_at, b.last_commit_at, b.closed_at, + (SELECT c.id FROM collections c WHERE c.project_id = b.project_id LIMIT 1) +FROM cached_branches b; +DROP TABLE cached_branches; +ALTER TABLE cached_branches__new RENAME TO cached_branches; +CREATE INDEX idx_cached_branches_rfc ON cached_branches (rfc_slug, state); + +-- ── branch_visibility: UNIQUE (project_id, rfc_slug, branch_name) -> collection +CREATE TABLE branch_visibility__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + branch_name TEXT NOT NULL, + read_public INTEGER NOT NULL DEFAULT 1, + contribute_mode TEXT NOT NULL DEFAULT 'just-me' CHECK (contribute_mode IN ('just-me', 'specific', 'any-contributor')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, rfc_slug, branch_name) +); +INSERT INTO branch_visibility__new + (id, rfc_slug, branch_name, read_public, contribute_mode, collection_id) +SELECT + v.id, v.rfc_slug, v.branch_name, v.read_public, v.contribute_mode, + (SELECT c.id FROM collections c WHERE c.project_id = v.project_id LIMIT 1) +FROM branch_visibility v; +DROP TABLE branch_visibility; +ALTER TABLE branch_visibility__new RENAME TO branch_visibility; + +-- ── branch_contribute_grants: UNIQUE (..., grantee) -> +collection_id ─────── +CREATE TABLE branch_contribute_grants__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + branch_name TEXT NOT NULL, + grantee_user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + granted_by INTEGER NOT NULL REFERENCES users(id) ON DELETE SET NULL, + granted_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, rfc_slug, branch_name, grantee_user_id) +); +INSERT INTO branch_contribute_grants__new + (id, rfc_slug, branch_name, grantee_user_id, granted_by, granted_at, collection_id) +SELECT + g.id, g.rfc_slug, g.branch_name, g.grantee_user_id, g.granted_by, g.granted_at, + (SELECT c.id FROM collections c WHERE c.project_id = g.project_id LIMIT 1) +FROM branch_contribute_grants g; +DROP TABLE branch_contribute_grants; +ALTER TABLE branch_contribute_grants__new RENAME TO branch_contribute_grants; +CREATE INDEX idx_grants_lookup ON branch_contribute_grants (rfc_slug, branch_name); +CREATE INDEX idx_grants_grantee ON branch_contribute_grants (grantee_user_id); + +-- ── stars: UNIQUE (project_id, user_id, rfc_slug) -> collection_id ────────── +CREATE TABLE stars__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + rfc_slug TEXT NOT NULL, + starred_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, user_id, rfc_slug) +); +INSERT INTO stars__new (id, user_id, rfc_slug, starred_at, collection_id) +SELECT s.id, s.user_id, s.rfc_slug, s.starred_at, + (SELECT c.id FROM collections c WHERE c.project_id = s.project_id LIMIT 1) +FROM stars s; +DROP TABLE stars; +ALTER TABLE stars__new RENAME TO stars; +CREATE INDEX idx_stars_user ON stars (user_id); +CREATE INDEX idx_stars_rfc ON stars (rfc_slug); + +-- ── watches: UNIQUE (project_id, user_id, rfc_slug) -> collection_id ──────── +CREATE TABLE watches__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + rfc_slug TEXT NOT NULL, + state TEXT NOT NULL CHECK (state IN ('watching', 'following', 'muted')), + set_by TEXT NOT NULL CHECK (set_by IN ('auto', 'explicit')), + set_at TEXT NOT NULL DEFAULT (datetime('now')), + last_participation_at TEXT, + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, user_id, rfc_slug) +); +INSERT INTO watches__new + (id, user_id, rfc_slug, state, set_by, set_at, last_participation_at, collection_id) +SELECT + w.id, w.user_id, w.rfc_slug, w.state, w.set_by, w.set_at, w.last_participation_at, + (SELECT c.id FROM collections c WHERE c.project_id = w.project_id LIMIT 1) +FROM watches w; +DROP TABLE watches; +ALTER TABLE watches__new RENAME TO watches; +CREATE INDEX idx_watches_user ON watches (user_id); +CREATE INDEX idx_watches_rfc ON watches (rfc_slug); +CREATE INDEX idx_watches_decay ON watches (state, last_participation_at); + +-- ── pr_seen: UNIQUE (project_id, user_id, rfc_slug, pr_number) -> collection ─ +CREATE TABLE pr_seen__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + rfc_slug TEXT NOT NULL, + pr_number INTEGER NOT NULL, + last_seen_commit_sha TEXT, + last_seen_message_id INTEGER REFERENCES thread_messages(id) ON DELETE SET NULL, + seen_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, user_id, rfc_slug, pr_number) +); +INSERT INTO pr_seen__new + (id, user_id, rfc_slug, pr_number, last_seen_commit_sha, last_seen_message_id, seen_at, collection_id) +SELECT + p.id, p.user_id, p.rfc_slug, p.pr_number, p.last_seen_commit_sha, p.last_seen_message_id, p.seen_at, + (SELECT c.id FROM collections c WHERE c.project_id = p.project_id LIMIT 1) +FROM pr_seen p; +DROP TABLE pr_seen; +ALTER TABLE pr_seen__new RENAME TO pr_seen; + +-- ── branch_chat_seen: UNIQUE (project_id, user_id, rfc_slug, branch) -> coll ─ +CREATE TABLE branch_chat_seen__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + rfc_slug TEXT NOT NULL, + branch_name TEXT NOT NULL, + last_seen_message_id INTEGER REFERENCES thread_messages(id) ON DELETE SET NULL, + seen_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, user_id, rfc_slug, branch_name) +); +INSERT INTO branch_chat_seen__new + (id, user_id, rfc_slug, branch_name, last_seen_message_id, seen_at, collection_id) +SELECT + s.id, s.user_id, s.rfc_slug, s.branch_name, s.last_seen_message_id, s.seen_at, + (SELECT c.id FROM collections c WHERE c.project_id = s.project_id LIMIT 1) +FROM branch_chat_seen s; +DROP TABLE branch_chat_seen; +ALTER TABLE branch_chat_seen__new RENAME TO branch_chat_seen; + +-- ── funder_consents: PRIMARY KEY (project_id, user_id, rfc_slug) -> collection +CREATE TABLE funder_consents__new ( + user_id INTEGER NOT NULL, + rfc_slug TEXT NOT NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + PRIMARY KEY (collection_id, user_id, rfc_slug), + FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE +); +INSERT INTO funder_consents__new (user_id, rfc_slug, created_at, collection_id) +SELECT f.user_id, f.rfc_slug, f.created_at, + (SELECT c.id FROM collections c WHERE c.project_id = f.project_id LIMIT 1) +FROM funder_consents f; +DROP TABLE funder_consents; +ALTER TABLE funder_consents__new RENAME TO funder_consents; +CREATE INDEX idx_funder_consents_slug ON funder_consents (rfc_slug); + +-- ── rfc_collaborators: UNIQUE idx + composite FK -> collection_id ─────────── +CREATE TABLE rfc_collaborators__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + role_in_rfc TEXT NOT NULL CHECK (role_in_rfc IN ('contributor', 'discussant')), + invitation_id INTEGER REFERENCES rfc_invitations(id) ON DELETE SET NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + FOREIGN KEY (collection_id, rfc_slug) REFERENCES cached_rfcs(collection_id, slug) ON DELETE CASCADE +); +INSERT INTO rfc_collaborators__new + (id, rfc_slug, user_id, role_in_rfc, invitation_id, created_at, collection_id) +SELECT + rc.id, rc.rfc_slug, rc.user_id, rc.role_in_rfc, rc.invitation_id, rc.created_at, + (SELECT c.id FROM collections c WHERE c.project_id = rc.project_id LIMIT 1) +FROM rfc_collaborators rc; +DROP TABLE rfc_collaborators; +ALTER TABLE rfc_collaborators__new RENAME TO rfc_collaborators; +CREATE UNIQUE INDEX idx_rfc_collaborators_unique ON rfc_collaborators (collection_id, rfc_slug, user_id); +CREATE INDEX idx_rfc_collaborators_user ON rfc_collaborators (user_id); + +-- ── contribution_requests: UNIQUE idx (pending) + composite FK -> collection ─ +CREATE TABLE contribution_requests__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + rfc_slug TEXT NOT NULL, + requester_user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + matched_term TEXT NOT NULL, + who_i_am TEXT NOT NULL, + why TEXT NOT NULL, + use_case TEXT, + status TEXT NOT NULL DEFAULT 'pending' + CHECK (status IN ('pending', 'accepted', 'declined')), + created_at TEXT NOT NULL DEFAULT (datetime('now')), + decided_at TEXT, + decided_by_user_id INTEGER REFERENCES users(id) ON DELETE SET NULL, + invitation_id INTEGER REFERENCES rfc_invitations(id) ON DELETE SET NULL, + notification_id INTEGER REFERENCES notifications(id) ON DELETE SET NULL, + collection_id TEXT NOT NULL DEFAULT 'default', + FOREIGN KEY (collection_id, rfc_slug) REFERENCES cached_rfcs(collection_id, slug) ON DELETE CASCADE +); +INSERT INTO contribution_requests__new + (id, rfc_slug, requester_user_id, matched_term, who_i_am, why, use_case, status, + created_at, decided_at, decided_by_user_id, invitation_id, notification_id, collection_id) +SELECT + cr.id, cr.rfc_slug, cr.requester_user_id, cr.matched_term, cr.who_i_am, cr.why, cr.use_case, cr.status, + cr.created_at, cr.decided_at, cr.decided_by_user_id, cr.invitation_id, cr.notification_id, + (SELECT c.id FROM collections c WHERE c.project_id = cr.project_id LIMIT 1) +FROM contribution_requests cr; +DROP TABLE contribution_requests; +ALTER TABLE contribution_requests__new RENAME TO contribution_requests; +CREATE INDEX idx_contribution_requests_rfc ON contribution_requests(rfc_slug, status); +CREATE INDEX idx_contribution_requests_requester ON contribution_requests(requester_user_id, status); +CREATE UNIQUE INDEX idx_contribution_requests_one_open + ON contribution_requests(collection_id, rfc_slug, requester_user_id) + WHERE status = 'pending'; + +-- ── proposed_use_cases: UNIQUE (project_id, scope, pr_number) -> collection ── +CREATE TABLE proposed_use_cases__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + scope TEXT NOT NULL CHECK (scope IN ('rfc', 'pr')), + rfc_slug TEXT NOT NULL, + pr_number INTEGER NOT NULL, + use_case TEXT NOT NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + collection_id TEXT NOT NULL DEFAULT 'default', + UNIQUE (collection_id, scope, pr_number) +); +INSERT INTO proposed_use_cases__new + (id, scope, rfc_slug, pr_number, use_case, created_at, collection_id) +SELECT + u.id, u.scope, u.rfc_slug, u.pr_number, u.use_case, u.created_at, + (SELECT c.id FROM collections c WHERE c.project_id = u.project_id LIMIT 1) +FROM proposed_use_cases u; +DROP TABLE proposed_use_cases; +ALTER TABLE proposed_use_cases__new RENAME TO proposed_use_cases; +CREATE INDEX idx_proposed_use_cases_lookup ON proposed_use_cases (scope, pr_number); +CREATE INDEX idx_proposed_use_cases_slug ON proposed_use_cases (scope, rfc_slug); + +-- ── project_members -> memberships(scope_type, scope_id, …); roles collapsed ─ +CREATE TABLE memberships ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + scope_type TEXT NOT NULL CHECK (scope_type IN ('project', 'collection')), + scope_id TEXT NOT NULL, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + role TEXT NOT NULL CHECK (role IN ('owner', 'contributor')), + granted_by INTEGER REFERENCES users(id) ON DELETE SET NULL, + granted_at TEXT NOT NULL DEFAULT (datetime('now')), + UNIQUE (scope_type, scope_id, user_id) +); +CREATE INDEX idx_memberships_user ON memberships(user_id); +CREATE INDEX idx_memberships_scope ON memberships(scope_type, scope_id); + +-- M2 project_members rows attached at what is now the *collection*; collapse the +-- role enum (project_admin -> owner, project_contributor -> contributor; +-- project_viewer dropped this pass, §B.3) and migrate onto the default +-- collection of each project. +INSERT INTO memberships (scope_type, scope_id, user_id, role, granted_by, granted_at) +SELECT 'collection', + (SELECT c.id FROM collections c WHERE c.project_id = pm.project_id LIMIT 1), + pm.user_id, + CASE pm.role WHEN 'project_admin' THEN 'owner' + WHEN 'project_contributor' THEN 'contributor' + ELSE 'contributor' END, + pm.granted_by, pm.granted_at +FROM project_members pm +WHERE pm.role IN ('project_admin', 'project_contributor'); +DROP TABLE project_members; diff --git a/backend/tests/test_migration_029_collections.py b/backend/tests/test_migration_029_collections.py new file mode 100644 index 0000000..ab3f8bb --- /dev/null +++ b/backend/tests/test_migration_029_collections.py @@ -0,0 +1,139 @@ +"""Migration 029 — the collection grain beneath projects (§22 three-tier S1). + +Proves: a `collections` table exists with one default collection per project +(id='default', subfolder=repo root); the per-corpus fields (type, initial_state) +moved off `projects`; the 13 entry-corpus tables re-key (project_id,slug) -> +(collection_id,slug) with the composite PK/FK enforced; and project_members +generalises into memberships(scope_type, …) with the role enum collapsed. +Template: test_migration_028_project_scoped_keys.py. +""" +from __future__ import annotations + +import sqlite3 +import tempfile +from pathlib import Path + +import pytest + +from app import db + + +class _Cfg: + def __init__(self, path): + self.database_path = path + + +def _fresh_db(): + d = tempfile.mkdtemp() + path = Path(d) / "t.db" + db.run_migrations(_Cfg(str(path))) + return db.connect(str(path)) + + +def test_collections_table_exists_with_default_per_project(): + conn = _fresh_db() + cols = {r["name"] for r in conn.execute("PRAGMA table_info(collections)")} + assert {"id", "project_id", "type", "subfolder", + "initial_state", "visibility", "name", "registry_sha"} <= cols + # one default collection seeded for the bootstrap 'default' project (026) + row = conn.execute( + "SELECT id, project_id, subfolder FROM collections WHERE project_id='default'" + ).fetchone() + assert row is not None + assert row["id"] == "default" + assert row["subfolder"] == "" # repo root + + +def test_per_corpus_fields_moved_off_projects(): + conn = _fresh_db() + proj_cols = {r["name"] for r in conn.execute("PRAGMA table_info(projects)")} + assert "type" not in proj_cols + assert "initial_state" not in proj_cols + # projects keeps the grouping-tier fields + assert {"id", "name", "content_repo", "visibility"} <= proj_cols + + +def test_entry_tables_rekeyed_to_collection_id(): + conn = _fresh_db() + for t in ("cached_rfcs", "cached_branches", "stars", "watches", + "rfc_collaborators", "contribution_requests", "proposed_use_cases", + "branch_visibility", "branch_contribute_grants", "pr_seen", + "branch_chat_seen", "funder_consents", "rfc_invitations"): + cols = {r["name"] for r in conn.execute(f"PRAGMA table_info({t})")} + assert "collection_id" in cols, f"{t} missing collection_id" + assert "project_id" not in cols, f"{t} still has project_id" + + +def test_cached_rfcs_pk_is_collection_slug(): + conn = _fresh_db() + # a second collection under the default project + conn.execute( + "INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('c2','default','document','specs','active','public','Specs')" + ) + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','A','active','default')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','B','active','c2')") + n = conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE slug='intro'").fetchone()["c"] + assert n == 2 + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','dup','active','default')") + + +def test_cached_rfcs_collection_fk_enforced(): + conn = _fresh_db() + conn.execute("PRAGMA foreign_keys=ON") + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('x','X','active','nope')") + + +def test_collaborator_fk_is_composite_on_collection(): + conn = _fresh_db() + conn.execute( + "INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('c2','default','document','specs','active','public','Specs')" + ) + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','A','active','c2')") + conn.execute("INSERT INTO users (id, gitea_login, display_name, role) VALUES (1,'a','A','contributor')") + conn.execute("PRAGMA foreign_keys=ON") + conn.execute( + "INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, collection_id) " + "VALUES ('intro',1,'contributor','c2')" + ) + with pytest.raises(sqlite3.IntegrityError): + # same slug, a collection with no such entry — composite FK rejects + conn.execute( + "INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, collection_id) " + "VALUES ('intro',1,'contributor','default')" + ) + + +def test_stars_unique_now_scoped_by_collection(): + conn = _fresh_db() + conn.execute( + "INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('c2','default','document','specs','active','public','Specs')" + ) + conn.execute("INSERT INTO users (id, gitea_login, display_name, role) VALUES (1,'a','A','contributor')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','A','active','default')") + conn.execute("INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES ('intro','B','active','c2')") + conn.execute("INSERT INTO stars (user_id, rfc_slug, collection_id) VALUES (1,'intro','default')") + conn.execute("INSERT INTO stars (user_id, rfc_slug, collection_id) VALUES (1,'intro','c2')") + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO stars (user_id, rfc_slug, collection_id) VALUES (1,'intro','default')") + + +def test_memberships_table_replaces_project_members(): + conn = _fresh_db() + cols = {r["name"] for r in conn.execute("PRAGMA table_info(memberships)")} + assert {"scope_type", "scope_id", "user_id", "role", "granted_by", "granted_at"} <= cols + # project_members is gone + assert conn.execute( + "SELECT name FROM sqlite_master WHERE type='table' AND name='project_members'" + ).fetchone() is None + conn.execute("INSERT INTO users (id, gitea_login, display_name, role) VALUES (9,'x','X','contributor')") + conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('project','default',9,'owner')") + # scope_type and role are CHECK-constrained + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('bogus','default',9,'owner')") + with pytest.raises(sqlite3.IntegrityError): + conn.execute("INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('project','default',9,'viewer')") -- 2.39.5 From 9ca07a3f8147e105616746474b9d5cde87d92c87 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 08:26:14 -0700 Subject: [PATCH 07/21] =?UTF-8?q?=C2=A722=20S1:=20thread=20collection=5Fid?= =?UTF-8?q?=20through=20backend=20+=20update=20tests=20(N=3D1=20unchanged,?= =?UTF-8?q?=20454=20green)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - collections.py resolution helpers (default_collection_id, type, initial_state) - registry mirror writes project grouping fields + default-collection corpus fields - auth.project_of_rfc joins collections; project_member_role reads memberships - cache/api_*/funder writers+readers re-keyed to collection_id (cached_prs + denormalised project_id tags unchanged); api_deployment reads type/initial_state from the default collection - projects.py restamp detects bootstrap via collections; initial_state via collection - tests updated to the three-tier schema; test_migration_028 retired (superseded by 029) - add @S1 acceptance test (collection grain + N=1 serving) Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/app/api.py | 56 ++++++----- backend/app/api_branches.py | 6 +- backend/app/api_contributions.py | 4 +- backend/app/api_deployment.py | 37 ++++++-- backend/app/api_discussion.py | 2 +- backend/app/api_graduation.py | 6 +- backend/app/api_invitations.py | 2 +- backend/app/api_notifications.py | 4 +- backend/app/api_prs.py | 8 +- backend/app/auth.py | 37 ++++++-- backend/app/cache.py | 23 +++-- backend/app/collections.py | 50 ++++++++++ backend/app/funder.py | 2 +- backend/app/projects.py | 20 ++-- backend/app/registry.py | 71 ++++++++++---- backend/tests/test_api_deployment.py | 21 ++-- backend/tests/test_initial_state_landing.py | 2 +- backend/tests/test_migration_027.py | 11 ++- .../test_migration_028_project_scoped_keys.py | 95 ------------------- .../test_multi_project_authz_vertical.py | 63 +++++++----- .../test_multi_project_spine_vertical.py | 74 ++++++++------- backend/tests/test_project_scoped_propose.py | 16 +++- backend/tests/test_project_scoped_serving.py | 13 ++- backend/tests/test_registry.py | 20 ++-- backend/tests/test_registry_wiring.py | 8 +- backend/tests/test_restamp_default_project.py | 47 +++++---- .../test_s1_collection_grain_vertical.py | 78 +++++++++++++++ 27 files changed, 475 insertions(+), 301 deletions(-) create mode 100644 backend/app/collections.py delete mode 100644 backend/tests/test_migration_028_project_scoped_keys.py create mode 100644 backend/tests/test_s1_collection_grain_vertical.py diff --git a/backend/app/api.py b/backend/app/api.py index 0318b48..0d5cd2c 100644 --- a/backend/app/api.py +++ b/backend/app/api.py @@ -29,6 +29,7 @@ from . import ( api_notifications, api_prs, auth, + collections as collections_mod, projects as projects_mod, db, device_trust as device_trust_mod, @@ -637,13 +638,13 @@ def make_router( unreviewed_clause = " AND unreviewed = 1 AND state = 'active'" rows = db.conn().execute( f""" - SELECT slug, title, state, rfc_id, repo, - owners_json, arbiters_json, tags_json, - last_main_commit_at, last_entry_commit_at, updated_at - FROM cached_rfcs - WHERE state IN ('super-draft', 'active') - AND project_id IN ({placeholders}){unreviewed_clause} - ORDER BY COALESCE(last_main_commit_at, last_entry_commit_at) DESC + SELECT r.slug, r.title, r.state, r.rfc_id, r.repo, + r.owners_json, r.arbiters_json, r.tags_json, + r.last_main_commit_at, r.last_entry_commit_at, r.updated_at + FROM cached_rfcs r JOIN collections c ON c.id = r.collection_id + WHERE r.state IN ('super-draft', 'active') + AND c.project_id IN ({placeholders}){unreviewed_clause} + ORDER BY COALESCE(r.last_main_commit_at, r.last_entry_commit_at) DESC """, params, ).fetchall() @@ -685,8 +686,8 @@ def make_router( raise HTTPException(404, "Not found") viewer = auth.current_user(request) # §22.5 visibility gate (subtractive, §22.7): a gated project's entries - # 404 to non-members. - auth.require_project_readable(viewer, row["project_id"]) + # 404 to non-members. Recover the project via the entry's collection. + auth.require_project_readable(viewer, auth.project_of_rfc(slug)) # §13.7: a retired entry is removed from every browsing surface. The # sole exception is a site owner, so the un-retire affordance has # somewhere to live; everyone else gets a plain 404. @@ -723,6 +724,8 @@ def make_router( viewer = auth.current_user(request) # §22.5 read gate: a gated project's catalog 404s to a non-member. auth.require_project_readable(viewer, project_id) + # §22 S1: serve the project's default collection (the corpus grain). + collection_id = collections_mod.default_collection_id(project_id) viewer_id = viewer.user_id if viewer else None unreviewed_clause = "" if unreviewed is not None and unreviewed.lower() in ("1", "true", "yes"): @@ -734,18 +737,18 @@ def make_router( last_main_commit_at, last_entry_commit_at, updated_at FROM cached_rfcs WHERE state IN ('super-draft', 'active') - AND project_id = ?{unreviewed_clause} + AND collection_id = ?{unreviewed_clause} ORDER BY COALESCE(last_main_commit_at, last_entry_commit_at) DESC """, - (project_id,), + (collection_id,), ).fetchall() starred = set() if viewer_id is not None: starred = { r["rfc_slug"] for r in db.conn().execute( - "SELECT rfc_slug FROM stars WHERE user_id = ? AND project_id = ?", - (viewer_id, project_id), + "SELECT rfc_slug FROM stars WHERE user_id = ? AND collection_id = ?", + (viewer_id, collection_id), ) } items = [ @@ -770,9 +773,10 @@ def make_router( async def get_project_rfc(project_id: str, slug: str, request: Request) -> dict[str, Any]: viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) + collection_id = collections_mod.default_collection_id(project_id) row = db.conn().execute( - "SELECT * FROM cached_rfcs WHERE project_id = ? AND slug = ?", - (project_id, slug), + "SELECT * FROM cached_rfcs WHERE collection_id = ? AND slug = ?", + (collection_id, slug), ).fetchone() if row is None: raise HTTPException(404, "Not found") @@ -782,10 +786,10 @@ def make_router( uc = db.conn().execute( """ SELECT use_case FROM proposed_use_cases - WHERE scope = 'rfc' AND rfc_slug = ? AND project_id = ? + WHERE scope = 'rfc' AND rfc_slug = ? AND collection_id = ? ORDER BY id DESC LIMIT 1 """, - (slug, project_id), + (slug, collection_id), ).fetchone() payload["proposed_use_case"] = uc["use_case"] if uc else None return payload @@ -802,9 +806,10 @@ def make_router( auth.require_project_readable(viewer, project_id) if not auth.is_project_superuser(viewer, project_id): raise HTTPException(403, "Only a project owner/admin can mark an entry reviewed") + collection_id = collections_mod.default_collection_id(project_id) row = db.conn().execute( - "SELECT state, unreviewed FROM cached_rfcs WHERE slug = ? AND project_id = ?", - (slug, project_id), + "SELECT state, unreviewed FROM cached_rfcs WHERE slug = ? AND collection_id = ?", + (slug, collection_id), ).fetchone() if row is None: raise HTTPException(404, "Not found") @@ -965,8 +970,9 @@ def make_router( # We re-check atomically here even though the client also checks # on every keystroke, since a concurrent submission could land # between dialog-open and submit. + collection_id = collections_mod.default_collection_id(project_id) clash = db.conn().execute( - "SELECT 1 FROM cached_rfcs WHERE slug = ? AND project_id = ?", (slug, project_id) + "SELECT 1 FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id) ).fetchone() if clash: raise HTTPException(409, f"Slug `{slug}` is already taken") @@ -1041,11 +1047,11 @@ def make_router( if use_case: db.conn().execute( """ - INSERT INTO proposed_use_cases (scope, rfc_slug, pr_number, use_case, project_id) + INSERT INTO proposed_use_cases (scope, rfc_slug, pr_number, use_case, collection_id) VALUES ('rfc', ?, ?, ?, ?) - ON CONFLICT(project_id, scope, pr_number) DO UPDATE SET use_case = excluded.use_case + ON CONFLICT(collection_id, scope, pr_number) DO UPDATE SET use_case = excluded.use_case """, - (slug, pr["number"], use_case, project_id), + (slug, pr["number"], use_case, collection_id), ) db.conn().execute( "UPDATE cached_prs SET proposed_use_case = ? WHERE pr_kind = 'idea' AND pr_number = ? AND project_id = ?", @@ -1205,12 +1211,12 @@ def make_router( async def add_funder_consent(slug: str, request: Request) -> dict[str, Any]: user = auth.require_contributor(request) rfc = db.conn().execute( - "SELECT project_id FROM cached_rfcs WHERE slug = ?", (slug,) + "SELECT 1 FROM cached_rfcs WHERE slug = ?", (slug,) ).fetchone() if rfc is None: raise HTTPException(404, "RFC not found") # §22.5 visibility gate (subtractive): gated → 404 to non-members. - auth.require_project_readable(user, rfc["project_id"]) + auth.require_project_readable(user, auth.project_of_rfc(slug)) # §6.7: refuse consent from a user with no registered credentials # — a consent without a universe would be inert and the surface # should fail loudly rather than silently. diff --git a/backend/app/api_branches.py b/backend/app/api_branches.py index 8e54c48..74b18c8 100644 --- a/backend/app/api_branches.py +++ b/backend/app/api_branches.py @@ -742,7 +742,7 @@ def make_router( """ INSERT INTO branch_visibility (rfc_slug, branch_name, read_public, contribute_mode) VALUES (?, ?, ?, ?) - ON CONFLICT(project_id, rfc_slug, branch_name) DO UPDATE SET + ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET read_public = excluded.read_public, contribute_mode = excluded.contribute_mode """, @@ -896,7 +896,7 @@ def make_router( """ INSERT INTO branch_chat_seen (user_id, rfc_slug, branch_name, last_seen_message_id, seen_at) VALUES (?, ?, ?, ?, datetime('now')) - ON CONFLICT(project_id, user_id, rfc_slug, branch_name) DO UPDATE SET + ON CONFLICT(collection_id, user_id, rfc_slug, branch_name) DO UPDATE SET last_seen_message_id = excluded.last_seen_message_id, seen_at = excluded.seen_at """, @@ -1092,7 +1092,7 @@ def make_router( # ------------------------------------------------------------------ def _require_rfc(slug: str, viewer): - row = db.conn().execute("SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if row is None: raise HTTPException(404, "RFC not found") # §22.5 visibility gate (subtractive, §22.7): a gated project's entries diff --git a/backend/app/api_contributions.py b/backend/app/api_contributions.py index 5a4c0b8..e6f36c0 100644 --- a/backend/app/api_contributions.py +++ b/backend/app/api_contributions.py @@ -62,7 +62,7 @@ def _require_super_draft(slug: str, viewer): visibility gate is subtractive: a gated project's entries 404 to non-members (§22.7).""" row = db.conn().execute( - "SELECT slug, title, state, owners_json, proposed_by, project_id FROM cached_rfcs WHERE slug = ?", + "SELECT slug, title, state, owners_json, proposed_by, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,), ).fetchone() if row is None: @@ -108,7 +108,7 @@ def make_router() -> APIRouter: @router.get("/api/rfcs/{slug}/contribution-target") async def contribution_target(slug: str, request: Request) -> dict[str, Any]: row = db.conn().execute( - "SELECT slug, title, state, owners_json, proposed_by, project_id FROM cached_rfcs WHERE slug = ?", + "SELECT slug, title, state, owners_json, proposed_by, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,), ).fetchone() if row is None: diff --git a/backend/app/api_deployment.py b/backend/app/api_deployment.py index 1406619..2b7b1a2 100644 --- a/backend/app/api_deployment.py +++ b/backend/app/api_deployment.py @@ -18,7 +18,7 @@ from typing import Any from fastapi import APIRouter, HTTPException, Request from fastapi.responses import RedirectResponse -from . import auth, db, projects as projects_mod +from . import auth, collections as collections_mod, db, projects as projects_mod from .config import Config @@ -33,12 +33,21 @@ def make_router(config: Config) -> APIRouter: ).fetchone() # §22.5: enumerate only public + (member-)gated; unlisted is never listed. visible = set(auth.visible_project_ids(viewer)) + # §22 three-tier: `type` is a per-corpus field on the (default) collection + # now; surface the default collection's type for each project. rows = db.conn().execute( - "SELECT id, name, type, visibility FROM projects " + "SELECT id, name, visibility FROM projects " "WHERE visibility != 'unlisted' ORDER BY name" ).fetchall() projects = [ - {"id": r["id"], "name": r["name"], "type": r["type"], "visibility": r["visibility"]} + { + "id": r["id"], + "name": r["name"], + "type": collections_mod.collection_type( + collections_mod.default_collection_id(r["id"]) + ), + "visibility": r["visibility"], + } for r in rows if r["id"] in visible ] @@ -60,8 +69,7 @@ def make_router(config: Config) -> APIRouter: # an unknown id). unlisted is readable by direct id. auth.require_project_readable(viewer, project_id) row = db.conn().execute( - "SELECT id, name, type, visibility, initial_state, config_json " - "FROM projects WHERE id = ?", + "SELECT id, name, visibility, config_json FROM projects WHERE id = ?", (project_id,), ).fetchone() if row is None: @@ -71,13 +79,16 @@ def make_router(config: Config) -> APIRouter: except (ValueError, TypeError): cfg = {} dep = db.conn().execute("SELECT tagline FROM deployment WHERE id = 1").fetchone() + # §22 three-tier: type + initial_state moved down to the (default) + # collection in migration 029. + cid = collections_mod.default_collection_id(row["id"]) return { "id": row["id"], "name": row["name"], "tagline": (dep["tagline"] if dep else "") or "", - "type": row["type"], + "type": collections_mod.collection_type(cid), "visibility": row["visibility"], - "initial_state": row["initial_state"], + "initial_state": collections_mod.collection_initial_state(cid), "theme": cfg.get("theme") or {}, } @@ -86,21 +97,27 @@ def make_router(config: Config) -> APIRouter: # is permanent, so external "RFC-0001" links and bookmarks land correctly. # nginx routes /rfc/ and /proposals/ to the backend so these are reached # before the SPA's index.html fallback. + # §22 three-tier (S1): the canonical entry route now carries the collection + # segment /p//c//…. The legacy roots redirect through + # the default project's default collection. @router.get("/rfc/{slug}") async def redirect_old_rfc(slug: str) -> RedirectResponse: default_id = projects_mod.resolved_default_id(config) - return RedirectResponse(url=f"/p/{default_id}/e/{slug}", status_code=308) + cid = collections_mod.default_collection_id(default_id) + return RedirectResponse(url=f"/p/{default_id}/c/{cid}/e/{slug}", status_code=308) @router.get("/rfc/{slug}/pr/{pr_number}") async def redirect_old_rfc_pr(slug: str, pr_number: int) -> RedirectResponse: default_id = projects_mod.resolved_default_id(config) + cid = collections_mod.default_collection_id(default_id) return RedirectResponse( - url=f"/p/{default_id}/e/{slug}/pr/{pr_number}", status_code=308 + url=f"/p/{default_id}/c/{cid}/e/{slug}/pr/{pr_number}", status_code=308 ) @router.get("/proposals/{pr_number}") async def redirect_old_proposal(pr_number: int) -> RedirectResponse: default_id = projects_mod.resolved_default_id(config) - return RedirectResponse(url=f"/p/{default_id}/proposals/{pr_number}", status_code=308) + cid = collections_mod.default_collection_id(default_id) + return RedirectResponse(url=f"/p/{default_id}/c/{cid}/proposals/{pr_number}", status_code=308) return router diff --git a/backend/app/api_discussion.py b/backend/app/api_discussion.py index 9173b44..be57845 100644 --- a/backend/app/api_discussion.py +++ b/backend/app/api_discussion.py @@ -252,7 +252,7 @@ def _require_rfc_readable(slug: str, viewer): entries refuse reads of every shape — same rule `_require_rfc_with_repo` in `api_branches.py` follows.""" row = db.conn().execute( - "SELECT * FROM cached_rfcs WHERE slug = ?", (slug,) + "SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,) ).fetchone() if row is None: raise HTTPException(404, "RFC not found") diff --git a/backend/app/api_graduation.py b/backend/app/api_graduation.py index 6c61adb..b620aa3 100644 --- a/backend/app/api_graduation.py +++ b/backend/app/api_graduation.py @@ -574,7 +574,7 @@ def make_router( # ------------------------------------------------------------------- def _require_super_draft(slug: str, viewer): - row = db.conn().execute("SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if row is None: raise HTTPException(404, "RFC not found") # §22.5 visibility gate (subtractive, §22.7): gated → 404 to non-members. @@ -584,7 +584,7 @@ def make_router( return row def _require_retirable(slug: str): - row = db.conn().execute("SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if row is None: raise HTTPException(404, "RFC not found") if row["state"] not in ("super-draft", "active"): @@ -592,7 +592,7 @@ def make_router( return row def _require_retired(slug: str): - row = db.conn().execute("SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if row is None: raise HTTPException(404, "RFC not found") if row["state"] != "retired": diff --git a/backend/app/api_invitations.py b/backend/app/api_invitations.py index b8440b1..70119e0 100644 --- a/backend/app/api_invitations.py +++ b/backend/app/api_invitations.py @@ -380,7 +380,7 @@ def _require_rfc(slug: str, viewer): visibility gate is subtractive: a gated project's entries 404 to non-members (§22.7).""" row = db.conn().execute( - "SELECT slug, title, state, project_id FROM cached_rfcs WHERE slug = ?", (slug,), + "SELECT slug, title, state, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,), ).fetchone() if row is None: raise HTTPException(404, "RFC not found") diff --git a/backend/app/api_notifications.py b/backend/app/api_notifications.py index a76bf8e..89b00fe 100644 --- a/backend/app/api_notifications.py +++ b/backend/app/api_notifications.py @@ -213,7 +213,7 @@ def make_router(config: Config) -> APIRouter: @router.post("/api/rfcs/{slug}/watch") async def set_watch(slug: str, body: WatchBody, request: Request) -> dict[str, Any]: viewer = auth.require_user(request) - rfc = db.conn().execute("SELECT project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + rfc = db.conn().execute("SELECT (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if rfc is None: raise HTTPException(404, "RFC not found") # §22.5 visibility gate (subtractive): gated → 404 to non-members. @@ -222,7 +222,7 @@ def make_router(config: Config) -> APIRouter: """ INSERT INTO watches (user_id, rfc_slug, state, set_by, set_at, last_participation_at) VALUES (?, ?, ?, 'explicit', datetime('now'), datetime('now')) - ON CONFLICT(project_id, user_id, rfc_slug) DO UPDATE SET + ON CONFLICT(collection_id, user_id, rfc_slug) DO UPDATE SET state = excluded.state, set_by = 'explicit', set_at = excluded.set_at diff --git a/backend/app/api_prs.py b/backend/app/api_prs.py index c4157c6..69a6678 100644 --- a/backend/app/api_prs.py +++ b/backend/app/api_prs.py @@ -153,7 +153,7 @@ def make_router( """ INSERT INTO branch_visibility (rfc_slug, branch_name, read_public, contribute_mode) VALUES (?, ?, 1, 'just-me') - ON CONFLICT(project_id, rfc_slug, branch_name) DO UPDATE SET read_public = 1 + ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET read_public = 1 """, (slug, branch), ) @@ -189,7 +189,7 @@ def make_router( """ INSERT INTO proposed_use_cases (scope, rfc_slug, pr_number, use_case) VALUES ('pr', ?, ?, ?) - ON CONFLICT(project_id, scope, pr_number) DO UPDATE SET use_case = excluded.use_case + ON CONFLICT(collection_id, scope, pr_number) DO UPDATE SET use_case = excluded.use_case """, (slug, pr["number"], use_case), ) @@ -398,7 +398,7 @@ def make_router( INSERT INTO pr_seen (user_id, rfc_slug, pr_number, last_seen_commit_sha, last_seen_message_id, seen_at) VALUES (?, ?, ?, ?, ?, datetime('now')) - ON CONFLICT(project_id, user_id, rfc_slug, pr_number) DO UPDATE SET + ON CONFLICT(collection_id, user_id, rfc_slug, pr_number) DO UPDATE SET last_seen_commit_sha = excluded.last_seen_commit_sha, last_seen_message_id = excluded.last_seen_message_id, seen_at = excluded.seen_at @@ -662,7 +662,7 @@ def make_router( # ------------------------------------------------------------------ def _require_rfc(slug: str, viewer): - row = db.conn().execute("SELECT * FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() + row = db.conn().execute("SELECT *, (SELECT c.project_id FROM collections c WHERE c.id = cached_rfcs.collection_id) AS project_id FROM cached_rfcs WHERE slug = ?", (slug,)).fetchone() if row is None: raise HTTPException(404, "RFC not found") # §22.5 visibility gate (subtractive, §22.7) — even §11.3 "PRs always diff --git a/backend/app/auth.py b/backend/app/auth.py index 8461171..98a6978 100644 --- a/backend/app/auth.py +++ b/backend/app/auth.py @@ -337,24 +337,41 @@ def project_visibility(project_id: str) -> str: def project_member_role(user: SessionUser | None, project_id: str) -> str | None: - """The user's *explicit* §22.6 project_members role in this project, or - None. This is the stored row only — it does not fold in the deployment tier - or the implicit-on-public baseline (those live in the helpers below).""" + """The user's *explicit* §22.6 membership role at this project, or None. + + §22 three-tier (S1): M2's `project_members` rows migrated into the unified + `memberships` table at the project's default collection (and the project + tier is freshly grantable). The unified `{owner, contributor}` roles are + mapped back to the legacy `project_admin`/`project_contributor` strings the + S1 project-grain authz still speaks; the four-layer scope resolver lands in + S3. Reads the stored grant only (project OR default-collection scope) — it + does not fold in the deployment tier or the implicit-on-public baseline.""" if user is None: return None + from . import collections as collections_mod + cid = collections_mod.default_collection_id(project_id) row = db.conn().execute( - "SELECT role FROM project_members WHERE project_id = ? AND user_id = ?", - (project_id, user.user_id), + "SELECT role FROM memberships " + "WHERE user_id = ? AND ((scope_type = 'project' AND scope_id = ?) " + " OR (scope_type = 'collection' AND scope_id = ?)) " + "ORDER BY CASE role WHEN 'owner' THEN 0 ELSE 1 END LIMIT 1", + (user.user_id, project_id, cid), ).fetchone() - return row["role"] if row else None + if row is None: + return None + return "project_admin" if row["role"] == "owner" else "project_contributor" def project_of_rfc(rfc_slug: str) -> str: - """The project an RFC belongs to (`cached_rfcs.project_id`). Falls back to - the default project when the slug isn't cached or the column is unset — the - same N=1 default migration 026 backfills.""" + """The project an RFC belongs to, via its collection + (`cached_rfcs.collection_id` -> `collections.project_id`, §22 three-tier). + Falls back to the default project when the slug isn't cached — the same N=1 + default migration 026 backfills.""" row = db.conn().execute( - "SELECT project_id FROM cached_rfcs WHERE slug = ?", (rfc_slug,) + "SELECT c.project_id AS project_id " + "FROM cached_rfcs r JOIN collections c ON c.id = r.collection_id " + "WHERE r.slug = ?", + (rfc_slug,), ).fetchone() if row is None: return DEFAULT_PROJECT_ID diff --git a/backend/app/cache.py b/backend/app/cache.py index 4af2ab2..538c313 100644 --- a/backend/app/cache.py +++ b/backend/app/cache.py @@ -54,6 +54,11 @@ async def refresh_meta_repo(config: Config, gitea: Gitea) -> None: async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None: + # §22 S1: the corpus grain is the collection. The mirror is project-grained + # (reads rfcs/ at the repo root = the project's default collection); resolve + # that collection once and key cached_rfcs by it. + from . import collections as collections_mod + collection_id = collections_mod.default_collection_id(project_id) try: files = await gitea.list_dir(org, repo, "rfcs", ref="main") except GiteaError as e: @@ -77,7 +82,7 @@ async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: G log.warning("refresh_meta_repo: %s: skipping %s: missing slug", project_id, f["path"]) continue seen_slugs.add(entry.slug) - _upsert_cached_rfc(entry, body_sha=sha, project_id=project_id) + _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id) # Entries removed from a project's rfcs/ — the spec keeps withdrawn entries # as historical record (§3), so this fires only for out-of-band deletes; @@ -85,14 +90,14 @@ async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: G existing = { row["slug"] for row in db.conn().execute( - "SELECT slug FROM cached_rfcs WHERE project_id = ?", (project_id,) + "SELECT slug FROM cached_rfcs WHERE collection_id = ?", (collection_id,) ) } for missing in existing - seen_slugs: log.info("refresh_meta_repo: %s/%s no longer in rfcs/ — leaving cache row", project_id, missing) -def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, project_id: str = "default") -> None: +def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str = "default") -> None: # §6.6: models_json stays NULL when the frontmatter key is absent # (inherit operator universe) and '[]' for the explicit opt-out. models_json = json.dumps(entry.models) if entry.models is not None else None @@ -105,10 +110,10 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, project_id: str = (slug, title, state, rfc_id, repo, proposed_by, proposed_at, graduated_at, graduated_by, owners_json, arbiters_json, tags_json, models_json, funder_login, body, body_sha, - unreviewed, reviewed_at, reviewed_by, project_id, + unreviewed, reviewed_at, reviewed_by, collection_id, last_entry_commit_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now')) - ON CONFLICT(project_id, slug) DO UPDATE SET + ON CONFLICT(collection_id, slug) DO UPDATE SET title = excluded.title, state = excluded.state, rfc_id = excluded.rfc_id, @@ -150,7 +155,7 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, project_id: str = 1 if entry.unreviewed else 0, entry.reviewed_at, entry.reviewed_by, - project_id, + collection_id, ), ) @@ -210,7 +215,7 @@ async def refresh_rfc_repo(config: Config, gitea: Gitea, slug: str) -> None: """ INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at) VALUES (?, ?, ?, 'open', ?) - ON CONFLICT(project_id, rfc_slug, branch_name) DO UPDATE SET + ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET head_sha = excluded.head_sha, state = CASE WHEN cached_branches.state = 'closed' THEN 'closed' ELSE 'open' END, last_commit_at = excluded.last_commit_at @@ -385,7 +390,7 @@ async def refresh_meta_branches(config: Config, gitea: Gitea) -> None: """ INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at) VALUES (?, ?, ?, 'open', ?) - ON CONFLICT(project_id, rfc_slug, branch_name) DO UPDATE SET + ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET head_sha = excluded.head_sha, state = CASE WHEN cached_branches.state = 'closed' THEN 'closed' ELSE 'open' END, last_commit_at = excluded.last_commit_at @@ -407,7 +412,7 @@ async def refresh_meta_branches(config: Config, gitea: Gitea) -> None: """ INSERT INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at) VALUES (?, 'main', ?, 'open', ?) - ON CONFLICT(project_id, rfc_slug, branch_name) DO UPDATE SET + ON CONFLICT(collection_id, rfc_slug, branch_name) DO UPDATE SET head_sha = excluded.head_sha, last_commit_at = excluded.last_commit_at """, diff --git a/backend/app/collections.py b/backend/app/collections.py new file mode 100644 index 0000000..d767b32 --- /dev/null +++ b/backend/app/collections.py @@ -0,0 +1,50 @@ +"""§22 collection grain — resolution helpers beneath the project tier. + +In S1 each project has exactly one collection (the default). These helpers +recover the collection for a project and read the per-corpus fields (`type`, +`initial_state`) that moved down from `projects` in migration 029. Project-grain +authz (auth.py) recovers a row's project by joining `collections` on +`collection_id`. +""" +from __future__ import annotations + +from . import db + +DEFAULT_COLLECTION_ID = "default" + + +def default_collection_id(project_id: str) -> str: + """The id of a project's default (S1: sole) collection. Falls back to the + literal 'default' when the project has no collection row yet.""" + row = db.conn().execute( + "SELECT id FROM collections WHERE project_id = ? ORDER BY created_at, id LIMIT 1", + (project_id,), + ).fetchone() + return row["id"] if row else DEFAULT_COLLECTION_ID + + +def project_of_collection(collection_id: str) -> str | None: + """The project a collection belongs to, or None if unknown.""" + row = db.conn().execute( + "SELECT project_id FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return row["project_id"] if row else None + + +def collection_initial_state(collection_id: str) -> str: + """§22.4b landing state for new entries in a collection. 'super-draft' + default for an unknown/unset row (today's safe flow).""" + row = db.conn().execute( + "SELECT initial_state FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + if row is None or not row["initial_state"]: + return "super-draft" + return row["initial_state"] + + +def collection_type(collection_id: str) -> str: + """The collection's immutable §22.4a type. 'document' default for unknown.""" + row = db.conn().execute( + "SELECT type FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return row["type"] if row and row["type"] else "document" diff --git a/backend/app/funder.py b/backend/app/funder.py index c4793d5..4a694d1 100644 --- a/backend/app/funder.py +++ b/backend/app/funder.py @@ -220,7 +220,7 @@ def add_consent(user_id: int, slug: str) -> None: db.conn().execute( """ INSERT INTO funder_consents (user_id, rfc_slug) VALUES (?, ?) - ON CONFLICT(project_id, user_id, rfc_slug) DO NOTHING + ON CONFLICT(collection_id, user_id, rfc_slug) DO NOTHING """, (user_id, slug), ) diff --git a/backend/app/projects.py b/backend/app/projects.py index c76609d..e6d56d8 100644 --- a/backend/app/projects.py +++ b/backend/app/projects.py @@ -43,8 +43,11 @@ def restamp_default_project(config: Config) -> None: if target == DEFAULT_PROJECT_ID: return conn = db.conn() + # §22 three-tier: the entry-corpus tables key on collection_id now; detect a + # lingering bootstrap project by the project-grain `collections.project_id` + # (the PRAGMA scan below still renames every project_id column dynamically). has_rows = conn.execute( - "SELECT 1 FROM cached_rfcs WHERE project_id = ? LIMIT 1", (DEFAULT_PROJECT_ID,) + "SELECT 1 FROM collections WHERE project_id = ? LIMIT 1", (DEFAULT_PROJECT_ID,) ).fetchone() stale_proj = conn.execute( "SELECT 1 FROM projects WHERE id = ? LIMIT 1", (DEFAULT_PROJECT_ID,) @@ -110,11 +113,10 @@ def content_repo(project_id: str) -> str | None: def project_initial_state(project_id: str) -> str: - """§22.4b landing state for new entries in a project. Defaults to - 'super-draft' for an unknown/unset row (the safe, today's-flow default).""" - row = db.conn().execute( - "SELECT initial_state FROM projects WHERE id = ?", (project_id,) - ).fetchone() - if row is None or not row["initial_state"]: - return "super-draft" - return row["initial_state"] + """§22.4b landing state for new entries in a project's default collection + (the per-corpus field moved down to the collection in migration 029). + Defaults to 'super-draft' for an unknown/unset row (today's-flow default).""" + from . import collections as collections_mod + return collections_mod.collection_initial_state( + collections_mod.default_collection_id(project_id) + ) diff --git a/backend/app/registry.py b/backend/app/registry.py index a04343f..38981b2 100644 --- a/backend/app/registry.py +++ b/backend/app/registry.py @@ -114,44 +114,74 @@ def parse_registry(text: str) -> RegistryDoc: ) -def apply_registry(doc: RegistryDoc, registry_sha: str) -> None: - """Upsert the parsed registry into projects + deployment. Idempotent. +def _default_collection_id(project_id: str, default_id: str) -> str: + """The id of a project's default collection. The deployment's primary + project (== `default_id`, the §22.13 resolved default) gets the stable + literal `'default'` — matching migration 029's seed so the upsert *merges* + onto the migration-seeded row rather than duplicating it (critical on a + fresh deploy where the bootstrap `default` project is later restamped to the + configured id). Any additional project keys its default collection by its own + id, keeping the collection PK globally unique (pre-S5 multi-project).""" + return "default" if project_id == default_id else project_id - §22.4a: `type` is immutable — a change against an existing row is rejected - (skip + log), never applied. Projects absent from the registry are left in - place (archival is out of scope for M3; they simply stop refreshing). + +def apply_registry(doc: RegistryDoc, registry_sha: str, default_id: str) -> None: + """Upsert the parsed registry into projects + their default collections + + the deployment singleton. Idempotent. + + §22 three-tier (S1): a project carries the grouping-tier fields (name, + content_repo, visibility, config); the per-corpus fields (`type`, + `initial_state`) live on the project's default collection. §22.4a: `type` is + immutable — a change against an existing collection is rejected (skip the + type change + log), never applied. Projects absent from the registry are + left in place (archival is out of scope for M3; they stop refreshing). """ with db.tx() as conn: for e in doc.projects: + cid = _default_collection_id(e.id, default_id) existing = conn.execute( - "SELECT type FROM projects WHERE id = ?", (e.id,) + "SELECT type FROM collections WHERE id = ?", (cid,) ).fetchone() - if existing is not None and existing["type"] != e.type: + type_locked = existing is not None and existing["type"] != e.type + if type_locked: log.error( - "registry: refusing immutable type change on project %s (%s -> %s)", - e.id, existing["type"], e.type, + "registry: refusing immutable type change on collection %s (%s -> %s)", + cid, existing["type"], e.type, ) - continue + # The project (grouping tier) always refreshes. conn.execute( """ INSERT INTO projects - (id, name, type, content_repo, visibility, initial_state, - config_json, registry_sha, updated_at) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now')) + (id, name, content_repo, visibility, config_json, registry_sha, updated_at) + VALUES (?, ?, ?, ?, ?, ?, datetime('now')) ON CONFLICT(id) DO UPDATE SET name = excluded.name, - type = excluded.type, content_repo = excluded.content_repo, visibility = excluded.visibility, - initial_state = excluded.initial_state, config_json = excluded.config_json, registry_sha = excluded.registry_sha, updated_at = datetime('now') """, - ( - e.id, e.name, e.type, e.content_repo, e.visibility, - e.initial_state, json.dumps(e.config), registry_sha, - ), + (e.id, e.name, e.content_repo, e.visibility, json.dumps(e.config), registry_sha), + ) + # The default collection (corpus tier). On an immutable-type + # conflict, keep the stored type but still refresh the rest. + effective_type = existing["type"] if type_locked else e.type + conn.execute( + """ + INSERT INTO collections + (id, project_id, type, subfolder, initial_state, visibility, name, registry_sha, updated_at) + VALUES (?, ?, ?, '', ?, ?, ?, ?, datetime('now')) + ON CONFLICT(id) DO UPDATE SET + project_id = excluded.project_id, + type = excluded.type, + initial_state = excluded.initial_state, + visibility = excluded.visibility, + name = excluded.name, + registry_sha = excluded.registry_sha, + updated_at = datetime('now') + """, + (cid, e.id, effective_type, e.initial_state, e.visibility, e.name, registry_sha), ) conn.execute( """ @@ -181,5 +211,6 @@ async def refresh_registry(config: Config, gitea: Gitea) -> None: # includes it on the contents response); fall back to the blob sha. sha = item.get("last_commit_sha") or item.get("sha") or "" doc = parse_registry(text) - apply_registry(doc, sha) + from . import projects as projects_mod + apply_registry(doc, sha, projects_mod.resolved_default_id(config)) log.info("registry: mirrored %d project(s) at %s", len(doc.projects), sha) diff --git a/backend/tests/test_api_deployment.py b/backend/tests/test_api_deployment.py index e0ea774..510e8f6 100644 --- a/backend/tests/test_api_deployment.py +++ b/backend/tests/test_api_deployment.py @@ -9,11 +9,18 @@ from test_propose_vertical import ( # noqa: F401 def _add_project(pid, name, vis, typ="document"): + # §22 three-tier: a project (grouping tier) + its default collection (the + # per-corpus type/initial_state moved down in migration 029). The default + # collection keys by the project id so it is globally unique in tests. from app import db db.conn().execute( - "INSERT OR REPLACE INTO projects (id, name, type, content_repo, visibility, initial_state) " - "VALUES (?, ?, ?, ?, ?, 'super-draft')", - (pid, name, typ, pid, vis), + "INSERT OR REPLACE INTO projects (id, name, content_repo, visibility) VALUES (?, ?, ?, ?)", + (pid, name, pid, vis), + ) + db.conn().execute( + "INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES (?, ?, ?, '', 'super-draft', ?, ?)", + (pid, pid, typ, vis, name), ) @@ -93,7 +100,7 @@ def test_rfc_root_url_redirects_308_to_project_scoped(app_with_fake_gitea): with TestClient(app) as client: r = client.get("/rfc/human", follow_redirects=False) assert r.status_code == 308 - assert r.headers["location"] == "/p/default/e/human" + assert r.headers["location"] == "/p/default/c/default/e/human" def test_rfc_pr_url_redirects_308_to_project_scoped(app_with_fake_gitea): @@ -102,7 +109,7 @@ def test_rfc_pr_url_redirects_308_to_project_scoped(app_with_fake_gitea): with TestClient(app) as client: r = client.get("/rfc/human/pr/7", follow_redirects=False) assert r.status_code == 308 - assert r.headers["location"] == "/p/default/e/human/pr/7" + assert r.headers["location"] == "/p/default/c/default/e/human/pr/7" def test_proposals_root_url_redirects_308_to_project_scoped(app_with_fake_gitea): @@ -110,7 +117,7 @@ def test_proposals_root_url_redirects_308_to_project_scoped(app_with_fake_gitea) with TestClient(app) as client: r = client.get("/proposals/42", follow_redirects=False) assert r.status_code == 308 - assert r.headers["location"] == "/p/default/proposals/42" + assert r.headers["location"] == "/p/default/c/default/proposals/42" def test_gated_project_visible_and_readable_to_member(app_with_fake_gitea): @@ -120,7 +127,7 @@ def test_gated_project_visible_and_readable_to_member(app_with_fake_gitea): _add_project("teamx", "Team X", "gated") provision_user_row(user_id=5, login="mia", role="contributor") db.conn().execute( - "INSERT INTO project_members (project_id, user_id, role) VALUES ('teamx', 5, 'project_viewer')" + "INSERT INTO memberships (scope_type, scope_id, user_id, role) VALUES ('collection', 'teamx', 5, 'contributor')" ) sign_in_as(client, user_id=5, gitea_login="mia", display_name="Mia", role="contributor") # member sees the gated project in the deployment directory diff --git a/backend/tests/test_initial_state_landing.py b/backend/tests/test_initial_state_landing.py index dd0931c..ad47e62 100644 --- a/backend/tests/test_initial_state_landing.py +++ b/backend/tests/test_initial_state_landing.py @@ -33,7 +33,7 @@ def test_active_initial_state_lands_active_unreviewed(app_with_fake_gitea): from app import db, entry as entry_mod app, fake = app_with_fake_gitea with TestClient(app) as client: - db.conn().execute("UPDATE projects SET initial_state='active' WHERE id='default'") + db.conn().execute("UPDATE collections SET initial_state='active' WHERE project_id='default'") provision_user_row(user_id=1, login="ben", role="owner") sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner") assert _propose(client).status_code == 200 diff --git a/backend/tests/test_migration_027.py b/backend/tests/test_migration_027.py index 1c47ca7..6e90a50 100644 --- a/backend/tests/test_migration_027.py +++ b/backend/tests/test_migration_027.py @@ -21,12 +21,17 @@ def _fresh_config() -> Config: def test_027_adds_project_type_and_initial_state(): + # 027 added type/initial_state to `projects`; migration 029 (three-tier) + # moved those per-corpus fields *down* onto `collections`. After the full + # migration chain they live on the collection, not the project. cfg = _fresh_config() db.run_migrations(cfg) conn = db.connect(cfg.database_path) - cols = {r["name"]: r for r in conn.execute("PRAGMA table_info(projects)")} - assert "type" in cols and cols["type"]["dflt_value"] == "'document'" - assert "initial_state" in cols and cols["initial_state"]["dflt_value"] == "'super-draft'" + proj_cols = {r["name"] for r in conn.execute("PRAGMA table_info(projects)")} + assert "type" not in proj_cols and "initial_state" not in proj_cols + coll_cols = {r["name"]: r for r in conn.execute("PRAGMA table_info(collections)")} + assert "type" in coll_cols and coll_cols["type"]["dflt_value"] == "'document'" + assert "initial_state" in coll_cols and coll_cols["initial_state"]["dflt_value"] == "'super-draft'" conn.close() diff --git a/backend/tests/test_migration_028_project_scoped_keys.py b/backend/tests/test_migration_028_project_scoped_keys.py deleted file mode 100644 index a5a7174..0000000 --- a/backend/tests/test_migration_028_project_scoped_keys.py +++ /dev/null @@ -1,95 +0,0 @@ -"""§22.13 / migration 028 — the slug-keyed PK/UNIQUE rebuild that activates -project #2. Proves two projects can hold the same slug, that (project_id, slug) -is still unique within a project, that the rebuilt FK is composite + enforced, -and that the no-foreign-keys migration runner left no dangling references.""" -from __future__ import annotations - -import sqlite3 -import tempfile -from pathlib import Path - -import pytest - -from app import db - - -class _Cfg: - def __init__(self, path): - self.database_path = path - - -def _fresh_db(): - d = tempfile.mkdtemp() - path = Path(d) / "t.db" - db.run_migrations(_Cfg(str(path))) - return db.connect(str(path)) - - -def _seed_two_projects(conn): - for pid in ("default", "ecomm"): - conn.execute( - "INSERT OR IGNORE INTO projects (id, name, type, content_repo, visibility, initial_state) " - "VALUES (?, ?, 'document', ?, 'public', 'super-draft')", - (pid, pid.title(), pid + "-content"), - ) - - -def test_same_slug_coexists_across_projects(): - conn = _fresh_db() - _seed_two_projects(conn) - for pid in ("default", "ecomm"): - conn.execute( - "INSERT INTO cached_rfcs (slug, title, state, project_id) " - "VALUES ('intro', 'Intro', 'active', ?)", - (pid,), - ) - rows = conn.execute( - "SELECT project_id FROM cached_rfcs WHERE slug = 'intro' ORDER BY project_id" - ).fetchall() - assert [r["project_id"] for r in rows] == ["default", "ecomm"] - - -def test_slug_still_unique_within_a_project(): - conn = _fresh_db() - _seed_two_projects(conn) - conn.execute( - "INSERT INTO cached_rfcs (slug, title, state, project_id) " - "VALUES ('intro', 'Intro', 'active', 'default')" - ) - with pytest.raises(sqlite3.IntegrityError): - conn.execute( - "INSERT INTO cached_rfcs (slug, title, state, project_id) " - "VALUES ('intro', 'Dup', 'active', 'default')" - ) - - -def test_rfc_collaborators_composite_fk_enforced(): - conn = _fresh_db() - _seed_two_projects(conn) - conn.execute("INSERT INTO users (id, gitea_login, display_name, role) VALUES (1, 'a', 'A', 'contributor')") - conn.execute( - "INSERT INTO cached_rfcs (slug, title, state, project_id) " - "VALUES ('intro', 'Intro', 'active', 'ecomm')" - ) - # Matching (project_id, slug) — FK holds. - conn.execute( - "INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, project_id) " - "VALUES ('intro', 1, 'contributor', 'ecomm')" - ) - # Same slug but a project with no such entry — composite FK must reject. - with pytest.raises(sqlite3.IntegrityError): - conn.execute( - "INSERT INTO rfc_collaborators (rfc_slug, user_id, role_in_rfc, project_id) " - "VALUES ('intro', 1, 'contributor', 'default')" - ) - - -def test_stars_unique_now_scoped_by_project(): - conn = _fresh_db() - _seed_two_projects(conn) - conn.execute("INSERT INTO users (id, gitea_login, display_name, role) VALUES (1, 'a', 'A', 'contributor')") - # Same (user, slug) under two projects coexist; a duplicate within one rejects. - conn.execute("INSERT INTO stars (user_id, rfc_slug, project_id) VALUES (1, 'intro', 'default')") - conn.execute("INSERT INTO stars (user_id, rfc_slug, project_id) VALUES (1, 'intro', 'ecomm')") - with pytest.raises(sqlite3.IntegrityError): - conn.execute("INSERT INTO stars (user_id, rfc_slug, project_id) VALUES (1, 'intro', 'default')") diff --git a/backend/tests/test_multi_project_authz_vertical.py b/backend/tests/test_multi_project_authz_vertical.py index 890f496..6631192 100644 --- a/backend/tests/test_multi_project_authz_vertical.py +++ b/backend/tests/test_multi_project_authz_vertical.py @@ -64,39 +64,55 @@ def _set_visibility(project_id: str, visibility: str) -> None: ) -def _add_member(project_id: str, user_id: int, role: str) -> None: - from app import db +# §22 three-tier (§B.3): M2's three project roles collapse to {owner, +# contributor} in the unified `memberships` table at the project's default +# collection. The read-only `viewer` tier is deferred (folded into contributor +# for this pass), so the legacy role names map: admin→owner, contributor and +# viewer→contributor. +_ROLE_MAP = { + "project_admin": "owner", + "project_contributor": "contributor", + "project_viewer": "contributor", +} + +def _add_member(project_id: str, user_id: int, role: str) -> None: + from app import collections as collections_mod, db + + cid = collections_mod.default_collection_id(project_id) db.conn().execute( - "INSERT OR REPLACE INTO project_members (project_id, user_id, role) VALUES (?, ?, ?)", - (project_id, user_id, role), + "INSERT OR REPLACE INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('collection', ?, ?, ?)", + (cid, user_id, _ROLE_MAP[role]), ) def _remove_member(project_id: str, user_id: int) -> None: - from app import db + from app import collections as collections_mod, db + cid = collections_mod.default_collection_id(project_id) db.conn().execute( - "DELETE FROM project_members WHERE project_id = ? AND user_id = ?", - (project_id, user_id), + "DELETE FROM memberships WHERE scope_type = 'collection' AND scope_id = ? AND user_id = ?", + (cid, user_id), ) def _seed_rfc(slug: str, *, state: str = "active", owners=None, project_id: str = "default") -> None: """A minimal cached_rfcs row — enough for the authz gates (state, owners, - project_id). project_id defaults to 'default' via migration 026 but we set - it explicitly for clarity.""" + collection grain). The entry lands in the project's default collection + (id == project_id for the single 'default' project under test).""" import json - from app import db + from app import collections as collections_mod, db + cid = collections_mod.default_collection_id(project_id) db.conn().execute( """ INSERT OR REPLACE INTO cached_rfcs - (slug, title, state, owners_json, arbiters_json, tags_json, project_id) + (slug, title, state, owners_json, arbiters_json, tags_json, collection_id) VALUES (?, ?, ?, ?, '[]', '[]', ?) """, - (slug, slug.capitalize(), state, json.dumps(owners or []), project_id), + (slug, slug.capitalize(), state, json.dumps(owners or []), cid), ) @@ -154,18 +170,16 @@ def test_resolver_gated_project_requires_membership(app_with_fake_gitea): assert auth.can_read_project(owner, "default") is True assert auth.is_project_superuser(owner, "default") is True - # project_viewer → read + discuss, but not contribute. - _add_member("default", 1, "project_viewer") + # §22 three-tier (§B.3): the read-only viewer tier is deferred — the + # smallest grant is `contributor`, which grants read + discuss + + # contribute across the subtree. + _add_member("default", 1, "project_contributor") assert auth.can_read_project(contributor, "default") is True assert auth.can_discuss_in_project(contributor, "default") is True - assert auth.can_contribute_in_project(contributor, "default") is False - - # project_contributor → contribute. - _add_member("default", 1, "project_contributor") assert auth.can_contribute_in_project(contributor, "default") is True assert auth.is_project_superuser(contributor, "default") is False - # project_admin → superuser within the project. + # project_admin → owner → superuser within the project. _add_member("default", 1, "project_admin") assert auth.is_project_superuser(contributor, "default") is True @@ -270,7 +284,7 @@ def test_gated_propose_requires_project_contributor(app_with_fake_gitea): assert client.post("/api/rfcs/propose", json=body).status_code != 403 -def test_gated_viewer_can_discuss_contributor_can_contribute(app_with_fake_gitea): +def test_gated_member_can_discuss_and_contribute(app_with_fake_gitea): from app import auth app, _ = app_with_fake_gitea @@ -285,14 +299,11 @@ def test_gated_viewer_can_discuss_contributor_can_contribute(app_with_fake_gitea sign_in_as(client, user_id=2, gitea_login="bob", display_name="Bob", role="contributor") assert client.post("/api/rfcs/spec/discussion/threads", json={"message": "q"}).status_code == 404 - # project_viewer: can discuss (200) but cannot contribute (resolver). - _add_member("default", 2, "project_viewer") + # §22 three-tier (§B.3): a `contributor` member can both discuss and + # contribute (the viewer-only read tier is deferred this pass). + _add_member("default", 2, "project_contributor") r = client.post("/api/rfcs/spec/discussion/threads", json={"message": "q"}) assert r.status_code == 200, r.text - assert auth.can_contribute_to_rfc(bob, "spec") is False - - # project_contributor: can contribute. - _add_member("default", 2, "project_contributor") assert auth.can_contribute_to_rfc(bob, "spec") is True diff --git a/backend/tests/test_multi_project_spine_vertical.py b/backend/tests/test_multi_project_spine_vertical.py index 35ce1dd..2ab404d 100644 --- a/backend/tests/test_multi_project_spine_vertical.py +++ b/backend/tests/test_multi_project_spine_vertical.py @@ -16,14 +16,18 @@ from test_propose_vertical import ( # noqa: F401 tmp_env, ) -# The 19 tables migration 026 threads project_id onto (docs/design/ -# multi-project-spec.md §5 amendment list). -SLUG_TABLES = [ - "cached_rfcs", "cached_branches", "cached_prs", "branch_visibility", - "branch_contribute_grants", "stars", "threads", "changes", "pr_seen", - "branch_chat_seen", "watches", "notifications", "actions", - "pr_resolution_branches", "funder_consents", "rfc_invitations", - "rfc_collaborators", "proposed_use_cases", "contribution_requests", +# §22 three-tier (migration 029): the entry-corpus grain is the collection, so +# the 13 tables migration 028 keyed by project_id re-key to collection_id. The +# remaining tables 026 tagged keep their denormalised project_id (project grain). +COLLECTION_TABLES = [ + "cached_rfcs", "cached_branches", "branch_visibility", + "branch_contribute_grants", "stars", "pr_seen", "branch_chat_seen", + "watches", "funder_consents", "rfc_invitations", "rfc_collaborators", + "proposed_use_cases", "contribution_requests", +] +PROJECT_TAG_TABLES = [ + "cached_prs", "threads", "changes", "notifications", "actions", + "pr_resolution_branches", ] @@ -45,25 +49,28 @@ def test_default_project_seeded_and_backfilled(app_with_fake_gitea): assert row["content_repo"] == "meta" -def test_project_id_on_every_slug_table(app_with_fake_gitea): +def test_grain_columns_on_every_slug_table(app_with_fake_gitea): from app import db app, _ = app_with_fake_gitea with TestClient(app): - for table in SLUG_TABLES: - cols = {r["name"]: r for r in db.conn().execute( - f"PRAGMA table_info({table})" - )} - assert "project_id" in cols, f"{table} missing project_id" - col = cols["project_id"] - # NOT NULL with the constant 'default' backfill default. - assert col["notnull"] == 1, f"{table}.project_id should be NOT NULL" - assert col["dflt_value"] == "'default'", f"{table}.project_id default" + # The entry-corpus tables key on collection_id (NOT NULL, 'default'). + for table in COLLECTION_TABLES: + cols = {r["name"]: r for r in db.conn().execute(f"PRAGMA table_info({table})")} + assert "collection_id" in cols, f"{table} missing collection_id" + assert "project_id" not in cols, f"{table} should no longer have project_id" + col = cols["collection_id"] + assert col["notnull"] == 1, f"{table}.collection_id should be NOT NULL" + assert col["dflt_value"] == "'default'", f"{table}.collection_id default" + # The project-tag tables keep their denormalised project_id. + for table in PROJECT_TAG_TABLES: + cols = {r["name"] for r in db.conn().execute(f"PRAGMA table_info({table})")} + assert "project_id" in cols, f"{table} missing project_id tag" def test_existing_row_backfills_to_default(app_with_fake_gitea): - """A row inserted the old way (no project_id) lands in the default - project — the trick that keeps every pre-multi-project INSERT working.""" + """A row inserted the old way (no collection grain) lands in the default + collection — the trick that keeps every pre-three-tier INSERT working.""" from app import db app, _ = app_with_fake_gitea @@ -73,35 +80,36 @@ def test_existing_row_backfills_to_default(app_with_fake_gitea): ("human", "Human", "active"), ) got = db.conn().execute( - "SELECT project_id FROM cached_rfcs WHERE slug = 'human'" - ).fetchone()["project_id"] + "SELECT collection_id FROM cached_rfcs WHERE slug = 'human'" + ).fetchone()["collection_id"] assert got == "default" -def test_project_members_table_shape(app_with_fake_gitea): +def test_memberships_table_shape(app_with_fake_gitea): from app import db app, _ = app_with_fake_gitea with TestClient(app): - cols = {r["name"] for r in db.conn().execute( - "PRAGMA table_info(project_members)" - )} - assert cols == {"project_id", "user_id", "role", "granted_by", "granted_at"} - # The role CHECK rejects an unknown role. + # §22 three-tier: project_members generalised into memberships. + assert db.conn().execute( + "SELECT name FROM sqlite_master WHERE type='table' AND name='project_members'" + ).fetchone() is None + cols = {r["name"] for r in db.conn().execute("PRAGMA table_info(memberships)")} + assert {"scope_type", "scope_id", "user_id", "role", "granted_by", "granted_at"} <= cols db.conn().execute( "INSERT INTO users (id, display_name, role) VALUES (1, 'Ben', 'owner')" ) db.conn().execute( - "INSERT INTO project_members (project_id, user_id, role) " - "VALUES ('default', 1, 'project_admin')" + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('collection', 'default', 1, 'owner')" ) import sqlite3 try: db.conn().execute( - "INSERT INTO project_members (project_id, user_id, role) " - "VALUES ('default', 1, 'nonsense')" + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('collection', 'default', 1, 'nonsense')" ) - assert False, "CHECK should reject an unknown project role" + assert False, "CHECK should reject an unknown role" except sqlite3.IntegrityError: pass diff --git a/backend/tests/test_project_scoped_propose.py b/backend/tests/test_project_scoped_propose.py index 251984c..7cabaf7 100644 --- a/backend/tests/test_project_scoped_propose.py +++ b/backend/tests/test_project_scoped_propose.py @@ -13,8 +13,12 @@ from test_propose_vertical import ( # noqa: F401 def _register_ecomm(fake): from app import db db.conn().execute( - "INSERT OR IGNORE INTO projects (id, name, type, content_repo, visibility, initial_state) " - "VALUES ('ecomm', 'Ecomm', 'document', 'ecomm-content', 'public', 'super-draft')" + "INSERT OR IGNORE INTO projects (id, name, content_repo, visibility) " + "VALUES ('ecomm', 'Ecomm', 'ecomm-content', 'public')" + ) + db.conn().execute( + "INSERT OR IGNORE INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('ecomm', 'ecomm', 'document', '', 'super-draft', 'public', 'Ecomm')" ) fake._seed_repo("wiggleverse", "ecomm-content") @@ -52,8 +56,12 @@ def test_propose_into_gated_project_404s_for_non_member(app_with_fake_gitea): from app import db with TestClient(app) as client: db.conn().execute( - "INSERT OR IGNORE INTO projects (id, name, type, content_repo, visibility, initial_state) " - "VALUES ('secret', 'Secret', 'document', 'secret-content', 'gated', 'super-draft')" + "INSERT OR IGNORE INTO projects (id, name, content_repo, visibility) " + "VALUES ('secret', 'Secret', 'secret-content', 'gated')" + ) + db.conn().execute( + "INSERT OR IGNORE INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES ('secret', 'secret', 'document', '', 'super-draft', 'gated', 'Secret')" ) provision_user_row(user_id=4, login="bob", role="contributor") sign_in_as(client, user_id=4, gitea_login="bob", display_name="Bob", diff --git a/backend/tests/test_project_scoped_serving.py b/backend/tests/test_project_scoped_serving.py index a455ed1..785daf4 100644 --- a/backend/tests/test_project_scoped_serving.py +++ b/backend/tests/test_project_scoped_serving.py @@ -11,18 +11,25 @@ from test_propose_vertical import ( # noqa: F401 def _add_project(pid, name, vis="public"): + # §22 three-tier: a project + its default collection (keyed by the project + # id in tests, so default_collection_id(pid) == pid). from app import db db.conn().execute( - "INSERT OR IGNORE INTO projects (id, name, type, content_repo, visibility, initial_state) " - "VALUES (?, ?, 'document', ?, ?, 'super-draft')", + "INSERT OR IGNORE INTO projects (id, name, content_repo, visibility) VALUES (?, ?, ?, ?)", (pid, name, pid + "-content", vis), ) + db.conn().execute( + "INSERT OR IGNORE INTO collections (id, project_id, type, subfolder, initial_state, visibility, name) " + "VALUES (?, ?, 'document', '', 'super-draft', ?, ?)", + (pid, pid, vis, name), + ) def _add_rfc(slug, title, pid, state="active"): from app import db + # entries key by the project's default collection (id == pid in these tests) db.conn().execute( - "INSERT INTO cached_rfcs (slug, title, state, project_id) VALUES (?, ?, ?, ?)", + "INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES (?, ?, ?, ?)", (slug, title, state, pid), ) diff --git a/backend/tests/test_registry.py b/backend/tests/test_registry.py index 68e0022..c4a39cb 100644 --- a/backend/tests/test_registry.py +++ b/backend/tests/test_registry.py @@ -74,13 +74,20 @@ def test_parse_rejects_invalid(bad, msg): def test_apply_upserts_projects_and_deployment(): _db() doc = registry.parse_registry(VALID) - registry.apply_registry(doc, registry_sha="regsha1") + registry.apply_registry(doc, registry_sha="regsha1", default_id="default") + # §22 three-tier: the project carries the grouping-tier fields; the + # per-corpus type/initial_state live on its default collection. prow = db.conn().execute( - "SELECT name, type, content_repo, visibility, initial_state, registry_sha FROM projects WHERE id='default'" + "SELECT name, content_repo, visibility, registry_sha FROM projects WHERE id='default'" ).fetchone() assert prow["name"] == "Open Human Model" assert prow["content_repo"] == "meta" assert prow["registry_sha"] == "regsha1" + crow = db.conn().execute( + "SELECT type, initial_state FROM collections WHERE id='default'" + ).fetchone() + assert crow["type"] == "document" + assert crow["initial_state"] == "super-draft" drow = db.conn().execute("SELECT name, tagline FROM deployment WHERE id=1").fetchone() assert drow["name"] == "Open Human Model" assert drow["tagline"] == "A model of human flourishing" @@ -88,11 +95,12 @@ def test_apply_upserts_projects_and_deployment(): def test_apply_rejects_type_change_on_existing_project(): _db() - registry.apply_registry(registry.parse_registry(VALID), "s1") + registry.apply_registry(registry.parse_registry(VALID), "s1", default_id="default") changed = VALID.replace("type: document", "type: specification") - registry.apply_registry(registry.parse_registry(changed), "s2") # logged + skipped, no raise - t = db.conn().execute("SELECT type FROM projects WHERE id='default'").fetchone()["type"] + registry.apply_registry(registry.parse_registry(changed), "s2", default_id="default") # skipped + # §22.4a immutable type — now enforced on the collection. + t = db.conn().execute("SELECT type FROM collections WHERE id='default'").fetchone()["type"] assert t == "document" # immutable — unchanged - # The deployment row IS still advanced even though the project upsert was skipped. + # The deployment row IS still advanced even though the type change was skipped. drow = db.conn().execute("SELECT registry_sha FROM deployment WHERE id=1").fetchone() assert drow["registry_sha"] == "s2" diff --git a/backend/tests/test_registry_wiring.py b/backend/tests/test_registry_wiring.py index dedb7cd..4e15ddd 100644 --- a/backend/tests/test_registry_wiring.py +++ b/backend/tests/test_registry_wiring.py @@ -12,10 +12,14 @@ def test_startup_mirrors_registry_into_projects_and_deployment(app_with_fake_git app, _ = app_with_fake_gitea with TestClient(app): prow = db.conn().execute( - "SELECT content_repo, type, initial_state FROM projects WHERE id='default'" + "SELECT content_repo FROM projects WHERE id='default'" ).fetchone() assert prow["content_repo"] == "meta" # from the registry, not META_REPO - assert prow["type"] == "document" + # §22 three-tier: type now lives on the default collection. + crow = db.conn().execute( + "SELECT type FROM collections WHERE id='default'" + ).fetchone() + assert crow["type"] == "document" drow = db.conn().execute("SELECT name FROM deployment WHERE id=1").fetchone() assert drow["name"] # deployment name mirrored from the registry diff --git a/backend/tests/test_restamp_default_project.py b/backend/tests/test_restamp_default_project.py index 9b3c69c..c955097 100644 --- a/backend/tests/test_restamp_default_project.py +++ b/backend/tests/test_restamp_default_project.py @@ -1,6 +1,9 @@ """§22.13 step 1 — the bootstrap-id re-stamp: 'default' → the configured -DEFAULT_PROJECT_ID across every project-scoped table, with the composite FKs -kept intact and the stale 'default' projects row dropped. Idempotent.""" +DEFAULT_PROJECT_ID. §22 three-tier (S1): the entry-corpus tables key on +collection_id now, so the re-stamp renames the *project grain* — the +`collections.project_id` link and the denormalised project_id tags — while the +entries stay in their collection. The stale 'default' projects row is dropped, +the composite FKs stay intact, and it is idempotent.""" from __future__ import annotations import tempfile @@ -19,30 +22,31 @@ class _Cfg: def _setup(monkeypatch, default_id="ohm"): path = str(Path(tempfile.mkdtemp()) / "t.db") cfg = _Cfg(path, default_id) - db.run_migrations(cfg) + db.run_migrations(cfg) # seeds the bootstrap 'default' project + its default collection monkeypatch.setattr(db, "_CONN", db.connect(path)) conn = db.conn() - # M1 bootstrap row + a registry-mirrored 'ohm' row coexist pre-restamp. - conn.execute("INSERT OR IGNORE INTO projects (id,name,type,content_repo,visibility,initial_state) " - "VALUES ('default','Bootstrap','document','ohm-content','public','super-draft')") - conn.execute("INSERT OR IGNORE INTO projects (id,name,type,content_repo,visibility,initial_state) " - "VALUES ('ohm','Open Human Model','document','ohm-content','public','super-draft')") + # A registry-mirrored 'ohm' project coexists with the bootstrap pre-restamp. + conn.execute("INSERT OR IGNORE INTO projects (id,name,content_repo,visibility) " + "VALUES ('ohm','Open Human Model','ohm-content','public')") conn.execute("INSERT INTO users (id,gitea_login,display_name,role) VALUES (1,'a','A','contributor')") - # default-stamped data with a composite-FK child - conn.execute("INSERT INTO cached_rfcs (slug,title,state,project_id) VALUES ('human','Human','active','default')") - conn.execute("INSERT INTO rfc_collaborators (rfc_slug,user_id,role_in_rfc,project_id) " + # Entry data lives in the default collection (id='default'); the entry grain + # is the collection and does not move on a re-stamp. + conn.execute("INSERT INTO cached_rfcs (slug,title,state,collection_id) VALUES ('human','Human','active','default')") + conn.execute("INSERT INTO rfc_collaborators (rfc_slug,user_id,role_in_rfc,collection_id) " "VALUES ('human',1,'contributor','default')") - conn.execute("INSERT INTO stars (user_id,rfc_slug,project_id) VALUES (1,'human','default')") + conn.execute("INSERT INTO stars (user_id,rfc_slug,collection_id) VALUES (1,'human','default')") return cfg, conn -def test_restamp_moves_data_and_drops_bootstrap_row(monkeypatch): +def test_restamp_moves_project_grain_and_drops_bootstrap_row(monkeypatch): cfg, conn = _setup(monkeypatch, default_id="ohm") projects.restamp_default_project(cfg) - assert conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE project_id='default'").fetchone()["c"] == 0 - assert conn.execute("SELECT project_id FROM cached_rfcs WHERE slug='human'").fetchone()["project_id"] == "ohm" - assert conn.execute("SELECT project_id FROM rfc_collaborators WHERE rfc_slug='human'").fetchone()["project_id"] == "ohm" - assert conn.execute("SELECT project_id FROM stars WHERE rfc_slug='human'").fetchone()["project_id"] == "ohm" + # The project grain (the collection's parent link) re-stamps to 'ohm'. + assert conn.execute("SELECT COUNT(*) c FROM collections WHERE project_id='default'").fetchone()["c"] == 0 + assert conn.execute("SELECT project_id FROM collections WHERE id='default'").fetchone()["project_id"] == "ohm" + # Entries stay in their collection — the collection_id is unchanged. + assert conn.execute("SELECT collection_id FROM cached_rfcs WHERE slug='human'").fetchone()["collection_id"] == "default" + assert conn.execute("SELECT collection_id FROM rfc_collaborators WHERE rfc_slug='human'").fetchone()["collection_id"] == "default" # stale bootstrap projects row removed; 'ohm' remains assert conn.execute("SELECT 1 FROM projects WHERE id='default'").fetchone() is None assert conn.execute("SELECT 1 FROM projects WHERE id='ohm'").fetchone() is not None @@ -53,12 +57,13 @@ def test_restamp_moves_data_and_drops_bootstrap_row(monkeypatch): def test_restamp_is_idempotent(monkeypatch): cfg, conn = _setup(monkeypatch, default_id="ohm") projects.restamp_default_project(cfg) - projects.restamp_default_project(cfg) # second call: no rows left → no-op - assert conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE project_id='ohm'").fetchone()["c"] == 1 + projects.restamp_default_project(cfg) # second call: no bootstrap rows left → no-op + assert conn.execute("SELECT project_id FROM collections WHERE id='default'").fetchone()["project_id"] == "ohm" + assert conn.execute("SELECT COUNT(*) c FROM cached_rfcs WHERE collection_id='default'").fetchone()["c"] == 1 def test_restamp_noop_when_default_id_unchanged(monkeypatch): cfg, conn = _setup(monkeypatch, default_id="") # resolves to 'default' projects.restamp_default_project(cfg) - # nothing renamed; bootstrap data + row still present - assert conn.execute("SELECT project_id FROM cached_rfcs WHERE slug='human'").fetchone()["project_id"] == "default" + # nothing renamed; the default collection still belongs to the bootstrap project + assert conn.execute("SELECT project_id FROM collections WHERE id='default'").fetchone()["project_id"] == "default" diff --git a/backend/tests/test_s1_collection_grain_vertical.py b/backend/tests/test_s1_collection_grain_vertical.py new file mode 100644 index 0000000..24a5936 --- /dev/null +++ b/backend/tests/test_s1_collection_grain_vertical.py @@ -0,0 +1,78 @@ +"""@S1 acceptance — the collection grain exists (invisible default) and N=1 is +unchanged. + +Part C scenarios C3.7 (single-collection project skips the directory) and C3.8 +(single-project deployment skips the directory) are the client-side redirect +contract asserted in the frontend; this module asserts the backend N=1 +invariants behind the slice: every entry keys on a real collection_id, the +shipped project-scoped serving still resolves through the default collection, +and the legacy /rfc/ URL 308-redirects through /c//. + +Binding: docs/design/2026-06-05-three-tier-projects-collections.md §A.6 / Part E. +""" +from __future__ import annotations + +from fastapi.testclient import TestClient + +from test_propose_vertical import ( # noqa: F401 + app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as, +) + + +def _seed_entry(slug, title, collection_id="default", state="active"): + from app import db + db.conn().execute( + "INSERT INTO cached_rfcs (slug, title, state, collection_id) VALUES (?, ?, ?, ?)", + (slug, title, state, collection_id), + ) + + +def test_s1_migration_seeds_one_default_collection_for_the_default_project(app_with_fake_gitea): + from app import db + app, _ = app_with_fake_gitea + with TestClient(app): + row = db.conn().execute( + "SELECT id FROM collections WHERE project_id = 'default'" + ).fetchall() + assert len(row) == 1 + assert row[0]["id"] == "default" + + +def test_s1_entry_served_under_default_collection(app_with_fake_gitea): + """N=1 unchanged: an entry is keyed by collection_id under the hood and the + shipped project-scoped serving endpoint still resolves it.""" + from app import db + app, _ = app_with_fake_gitea + with TestClient(app) as client: + _seed_entry("human", "Human") + # the row carries a real collection grain (the default collection) + cid = db.conn().execute( + "SELECT collection_id FROM cached_rfcs WHERE slug='human'" + ).fetchone()["collection_id"] + assert cid == "default" + # project-scoped serving (collection = default) still returns it + r = client.get("/api/projects/default/rfcs/human") + assert r.status_code == 200, r.text + assert r.json()["slug"] == "human" + # and it appears in the project catalog + slugs = [i["slug"] for i in client.get("/api/projects/default/rfcs").json()["items"]] + assert "human" in slugs + + +def test_s1_legacy_rfc_url_redirects_through_collection(app_with_fake_gitea): + """The shipped /rfc/ now 308s through the default collection segment.""" + app, _ = app_with_fake_gitea + with TestClient(app) as client: + r = client.get("/rfc/human", follow_redirects=False) + assert r.status_code == 308 + assert r.headers["location"] == "/p/default/c/default/e/human" + + +def test_s1_deployment_reports_single_project(app_with_fake_gitea): + """C3.8 precondition: the N=1 deployment reports exactly one visible project + and its default id (the frontend uses this to skip the directory).""" + app, _ = app_with_fake_gitea + with TestClient(app) as client: + body = client.get("/api/deployment").json() + assert body["default_project_id"] == "default" + assert [p["id"] for p in body["projects"]] == ["default"] -- 2.39.5 From 4f72aa31e0967ac49e8241cb6e5c17f788625ea9 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 08:30:18 -0700 Subject: [PATCH 08/21] =?UTF-8?q?=C2=A722=20S1:=20frontend=20/c//=20route=20layer=20+=20C3.7=20+=20legacy-URL=20redirects?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - entryPaths builders carry the /c// segment (default collection in S1) - /p/:projectId/* gains c/:collectionId/ corpus routes; serving stays project-scoped - DefaultCollectionRedirect (C3.7: project landing -> default collection) - LegacyCorpusRedirect (v0.35.0 /p/

/e/ bookmarks -> /c/default/, query preserved) - entryPaths unit test; build + vitest green (18 tests) Co-Authored-By: Claude Opus 4.8 (1M context) --- frontend/src/App.jsx | 45 +++++++++++++++++++++++++---- frontend/src/lib/entryPaths.js | 31 ++++++++++++-------- frontend/src/lib/entryPaths.test.js | 36 +++++++++++++++++++++++ 3 files changed, 94 insertions(+), 18 deletions(-) create mode 100644 frontend/src/lib/entryPaths.test.js diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index b9a0aed..e7453cf 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -1,10 +1,10 @@ import { useEffect, useRef, useState } from 'react' -import { Routes, Route, Link, Navigate, useLocation, useNavigate, useSearchParams } from 'react-router-dom' +import { Routes, Route, Link, Navigate, useLocation, useNavigate, useParams, useSearchParams } from 'react-router-dom' import { getMe, subscribeToNotifications } from './api' import { anonymize, EVENTS, identify, track } from './lib/analytics' import { useLastState } from './lib/useLastState' import { brandTitle } from './lib/brand' -import { entryPath, proposalPath } from './lib/entryPaths' +import { entryPath, proposalPath, DEFAULT_COLLECTION } from './lib/entryPaths' import { useDeployment } from './context/DeploymentProvider' import ProjectLayout from './components/ProjectLayout.jsx' import Directory from './components/Directory.jsx' @@ -360,10 +360,21 @@ export default function App() { />

- } /> - } /> - } /> - setCatalogVersion(v => v + 1)} />} /> + {/* §22 three-tier (C3.7): the project landing redirects into + its sole/default collection (S1: the `default` one). */} + } /> + {/* Backcompat: the shipped v0.35.0 corpus URLs without a + /c// segment redirect into the default + collection, so old bookmarks keep working. */} + } /> + } /> + } /> + {/* Collection-scoped corpus. Serving stays project-scoped in + S1 (collection = default); collection-aware serving is S2. */} + } /> + } /> + } /> + setCatalogVersion(v => v + 1)} />} />
@@ -403,6 +414,28 @@ export default function App() { ) } +// §22 three-tier — corpus redirects mounted under /p/:projectId/*. +// C3.7: the project landing skips the (single-collection) directory and lands in +// the project's default collection. +function DefaultCollectionRedirect() { + const { projectId } = useParams() + return +} + +// Backcompat for the shipped v0.35.0 corpus URLs that lacked the +// /c// segment: redirect into the default collection, preserving any +// query string (e.g. ?branch=). +function LegacyCorpusRedirect({ kind }) { + const { projectId, slug, prNumber } = useParams() + const { search } = useLocation() + const base = `/p/${projectId}/c/${DEFAULT_COLLECTION}` + let to = `${base}/` + if (kind === 'entry') to = `${base}/e/${slug}` + else if (kind === 'entryPr') to = `${base}/e/${slug}/pr/${prNumber}` + else if (kind === 'proposal') to = `${base}/proposals/${prNumber}` + return +} + function DeploymentLanding() { // §22.10 + design decision 2 — N=1 lands in the single visible project so // OHM's "land in the corpus" UX is preserved; the directory appears only diff --git a/frontend/src/lib/entryPaths.js b/frontend/src/lib/entryPaths.js index 840bf59..15c5f53 100644 --- a/frontend/src/lib/entryPaths.js +++ b/frontend/src/lib/entryPaths.js @@ -1,22 +1,29 @@ -// §22.10 — project-scoped path builders. After M3 every entry/proposal link -// lives under `/p//…`. Until Plan B serves multiple corpora, that -// project id is the deployment's corpus-served default for chrome surfaces, or -// the contextual project when a component renders inside a project subtree -// (ProjectContext). Components build links via these helpers so the later -// per-project-serving slice flips them in one place. +// §22 three-tier — project + collection-scoped path builders. The canonical +// entry route now carries the collection segment: `/p//c//…`. +// In S1 each project has a single (default) collection and serving stays +// project-scoped, so the builders emit the default collection segment; the +// collection-aware link layer (named collections) lands in S2. Components build +// links via these helpers so that flip happens in one place. import { useProject } from '../components/ProjectLayout.jsx' import { useDeployment } from '../context/DeploymentProvider' -export function entryPath(pid, slug) { - return `/p/${pid}/e/${slug}` +// The default collection id (migration 029 seeds one per project at this id). +export const DEFAULT_COLLECTION = 'default' + +export function entryPath(pid, slug, cid = DEFAULT_COLLECTION) { + return `/p/${pid}/c/${cid}/e/${slug}` } -export function entryPrPath(pid, slug, prNumber) { - return `/p/${pid}/e/${slug}/pr/${prNumber}` +export function entryPrPath(pid, slug, prNumber, cid = DEFAULT_COLLECTION) { + return `/p/${pid}/c/${cid}/e/${slug}/pr/${prNumber}` } -export function proposalPath(pid, prNumber) { - return `/p/${pid}/proposals/${prNumber}` +export function proposalPath(pid, prNumber, cid = DEFAULT_COLLECTION) { + return `/p/${pid}/c/${cid}/proposals/${prNumber}` +} + +export function collectionHome(pid, cid = DEFAULT_COLLECTION) { + return `/p/${pid}/c/${cid}/` } export function projectHome(pid) { diff --git a/frontend/src/lib/entryPaths.test.js b/frontend/src/lib/entryPaths.test.js new file mode 100644 index 0000000..5cbd7e9 --- /dev/null +++ b/frontend/src/lib/entryPaths.test.js @@ -0,0 +1,36 @@ +// §22 three-tier — the canonical corpus path now carries the /c// +// segment. These builders default to the project's `default` collection (S1). +import { describe, it, expect } from 'vitest' +import { + entryPath, entryPrPath, proposalPath, collectionHome, projectHome, DEFAULT_COLLECTION, +} from './entryPaths.js' + +describe('entryPaths — collection-scoped corpus URLs', () => { + it('entryPath defaults to the default collection segment', () => { + expect(entryPath('ohm', 'human')).toBe('/p/ohm/c/default/e/human') + }) + + it('entryPath honours an explicit collection id', () => { + expect(entryPath('ohm', 'login', 'features')).toBe('/p/ohm/c/features/e/login') + }) + + it('entryPrPath carries the collection segment', () => { + expect(entryPrPath('ohm', 'human', 7)).toBe('/p/ohm/c/default/e/human/pr/7') + }) + + it('proposalPath carries the collection segment', () => { + expect(proposalPath('ohm', 42)).toBe('/p/ohm/c/default/proposals/42') + }) + + it('collectionHome targets the collection root', () => { + expect(collectionHome('ohm')).toBe('/p/ohm/c/default/') + }) + + it('projectHome stays at the project root (redirects into the collection)', () => { + expect(projectHome('ohm')).toBe('/p/ohm/') + }) + + it('exposes the default collection id constant', () => { + expect(DEFAULT_COLLECTION).toBe('default') + }) +}) -- 2.39.5 From aaf7b09bbecbd791fbb956f68675d20e376f4435 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 08:31:43 -0700 Subject: [PATCH 09/21] =?UTF-8?q?=C2=A722=20S1:=20release=20v0.40.0=20?= =?UTF-8?q?=E2=80=94=20three-tier=20collection=20grain=20(breaking=20URL?= =?UTF-8?q?=20+=20migration=20029)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 80 +++++++++++++++++++++++++++++++++++++++++++ VERSION | 2 +- frontend/package.json | 2 +- 3 files changed, 82 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 46325b2..74bd6b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,86 @@ skip versions are the composition of each intervening adjacent release's steps in order — no A-to-B path is pre-computed beyond that. +## 0.40.0 — 2026-06-05 + +**Minor (breaking URL) — §22 three-tier refactor, slice S1: the *collection* +grain. A deployment now hosts N projects, each owning one content repo and +holding N RFC *collections*, each collection a typed corpus. S1 inserts the +collection grain beneath today's project as the invisible default: the +deployment runs exactly as before, now with a real collection layer and one +extra `/c//` URL segment. Per-corpus configuration (`type`, +`initial_state`), entry keys, and membership move to the collection grain; +no operator action is required beyond deploying.** + +See [`docs/design/2026-06-05-three-tier-projects-collections.md`](./docs/design/2026-06-05-three-tier-projects-collections.md) +(Part A model, Part E / §A.6 migration strategy). The structural model (Parts +A–D) is unaffected; the SPEC.md merge itself rides slice S6. + +Added: + +- **Migration `029_collections.sql`** — (1) a `collections` table + `(id, project_id, type, subfolder, initial_state, visibility, name, + registry_sha)` beneath `projects`; (2) the per-corpus fields (`type`, + `initial_state`) move **down** off `projects` onto the collection; (3) one + **default collection** per project (`id='default'` for the standard + single-project deployment, `subfolder` = repo root), inheriting the project's + type / initial_state / visibility; (4) the 13 entry-corpus tables re-key + `(project_id, slug)` → `(collection_id, slug)` via the migration-028 rebuild + pattern, each row mapped to its project's default collection; (5) + `project_members` generalises into **`memberships(scope_type ∈ {project, + collection}, scope_id, user_id, role, …)`** with the role enum collapsed to + `{owner, contributor}` (M2's `project_admin`→`owner`, + `project_contributor`→`contributor`; `project_viewer` folded into + `contributor` this pass — the read-only tier is deferred). +- **`app/collections.py`** — collection resolution helpers + (`default_collection_id`, `collection_type`, `collection_initial_state`, + `project_of_collection`). +- **`/c//` URL segment** — the canonical entry route is now + `/p//c//e/`. The project landing + `/p//` redirects into the project's single (default) collection + (C3.7); the deployment root `/` continues to redirect into the sole project + (C3.8). + +Changed: + +- **The registry mirror** writes a project's grouping-tier fields (name, + content_repo, visibility, config) to `projects` and the per-corpus fields + (`type`, `initial_state`) to its default collection; §22.4a type-immutability + is now enforced on the collection. +- **Backend threading** — `auth.project_of_rfc` recovers a project by joining + `collections`; `auth.project_member_role` reads `memberships`; + `cache`/`api_*`/`funder` writers + readers key the 13 entry-corpus tables by + `collection_id` (the denormalised `project_id` tags on + `cached_prs`/`threads`/`changes`/`notifications`/`actions`/`pr_resolution_branches` + are unchanged). Serving stays project-scoped (collection = default); the + registry `.collection.yaml` reader and collection-aware serving land in S2. + +Breaking: + +- **`/p//e/` URLs gain a `/c//` segment.** The + shipped v0.35.0 `/p//e/` form is preserved by a client-side + redirect into the default collection; the pre-multi-project `/rfc/` and + `/proposals/` server 308s now target `/p//c//…`. + +> ### Upgrade steps (0.39.0 → 0.40.0) +> +> - A deployment **MUST** deploy this version with its migrations applied (the +> standard startup path runs `029_collections.sql` automatically); the +> migration seeds the default collection and re-keys existing entries with no +> data loss. No configuration change is required. +> - Operators **SHOULD** be aware that the canonical entry URL is now +> `/p//c/default/e/`. Existing `/p//e/`, +> `/rfc/`, and `/proposals/` links keep working (client redirect / +> server 308). External systems that hardcoded the old form **SHOULD** be +> updated to the collection-scoped form at their convenience. +> - Deployments that pin the framework version **MUST** bump their version pin +> to `0.40.0`. + +Deferred (later slices): creating + navigating a second collection and the +registry `.collection.yaml` reader (S2); the four-layer scope-role resolver and +the `viewer` read tier (S3); invitation surfaces (S4); in-app create-project +(S5); per-type surfaces, membership lifecycle, and the SPEC.md merge (S6). + ## 0.39.0 — 2026-06-04 **Minor — §22.13 step 1: the default-project-id re-stamp. A deployment can diff --git a/VERSION b/VERSION index 4ef2eb0..9b0025a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.39.0 +0.40.0 diff --git a/frontend/package.json b/frontend/package.json index 3a523eb..c417a57 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "rfc-app-frontend", "private": true, - "version": "0.39.0", + "version": "0.40.0", "type": "module", "scripts": { "dev": "vite", -- 2.39.5 From 599e7018f6963883f67dd7d6e37f1691f44a75d6 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 12:55:56 -0700 Subject: [PATCH 10/21] =?UTF-8?q?=C2=A722=20S2:=20implementation=20plan=20?= =?UTF-8?q?=E2=80=94=20create=20&=20navigate=20a=20second=20collection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan for slice S2 of the three-tier (deployment→project→collection) refactor. Completes acceptance @S2 (C3.6). See docs/design/2026-06-05-three-tier-projects-collections.md Part E. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../plans/2026-06-05-s2-second-collection.md | 1210 +++++++++++++++++ 1 file changed, 1210 insertions(+) create mode 100644 docs/design/plans/2026-06-05-s2-second-collection.md diff --git a/docs/design/plans/2026-06-05-s2-second-collection.md b/docs/design/plans/2026-06-05-s2-second-collection.md new file mode 100644 index 0000000..a0a7f6f --- /dev/null +++ b/docs/design/plans/2026-06-05-s2-second-collection.md @@ -0,0 +1,1210 @@ +# §22 S2 — Create & navigate a second collection — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to +> implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let a deployment admin create a second RFC collection beside a project's +default one, navigate to it, and propose into it — completing acceptance scenario +C3.6 (`@S2`): an anonymous visitor landing on an empty public collection's catalog +sees an empty catalog with no propose action and a sign-in prompt. + +**Architecture:** S1 shipped the collection grain (migration 029): a `collections` +table, entries keyed `(collection_id, slug)`, one seeded `default` collection per +project. S2 makes collections *plural and navigable*. Backend: the registry mirror +learns to read `.collection.yaml` manifests inside each project's content repo; the +corpus mirror becomes collection-grained (reads each collection's `/rfcs/`); +a bot-commit-wrapped create-collection endpoint commits a manifest; collection-scoped +list/get/propose endpoints land under `/api/projects/:id/collections/:cid/…`. Frontend: +`/p//` becomes a collection directory (1 visible → redirect, keeping S1; 2+ → +list); the Catalog rail + entry views read the active `:collectionId` from the route. + +**Tech Stack:** FastAPI + sqlite (backend, `backend/app/`), pytest (`backend/tests/`), +React + react-router + vitest (`frontend/src/`), Gitea content repos via the Bot wrapper. + +**Content-repo layout convention (decided this slice):** a collection with subfolder +`S` stores its manifest at `S/.collection.yaml` and its entries at `S/rfcs/`. The +default collection has subfolder `''` → manifest is the project's `projects.yaml` +entry (no `.collection.yaml`), entries at `rfcs/`. This keeps the shipped default +corpus path unchanged. + +**Authorization (this slice):** create-collection is gated to a deployment +owner/admin (`auth.require_admin`). Scoped {owner, contributor} roles at the +collection axis land in S3 — out of scope here. + +--- + +## File structure + +**Backend (modify):** +- `backend/app/registry.py` — add `.collection.yaml` discovery + upsert to the mirror. +- `backend/app/collections.py` — add `list_collections`, `get_collection`, `subfolder_of`. +- `backend/app/cache.py` — make `refresh_meta_repo` iterate a project's collections. +- `backend/app/bot.py` — add `create_collection` (commit `/.collection.yaml`). +- `backend/app/api.py` — add collection-scoped list/get/propose; refactor propose to take a collection. +- `backend/app/api_collections.py` — **create**: GET list/one + POST create-collection endpoints. +- `backend/app/main.py` — mount the new router. + +**Frontend (modify):** +- `frontend/src/lib/entryPaths.js` — add `useCollectionId()`. +- `frontend/src/api.js` — collection-scoped `listRFCs`/`getRFC`/`proposeRFC`; add `listCollections`, `createCollection`. +- `frontend/src/components/Catalog.jsx` — read active collection; scope fetches + links. +- `frontend/src/App.jsx` — `/p//` → `CollectionDirectory`; thread collection into propose. +- `frontend/src/components/CollectionDirectory.jsx` — **create**: the directory page. + +**Tests (create):** +- `backend/tests/test_collection_registry.py` — `.collection.yaml` mirror. +- `backend/tests/test_collection_create_vertical.py` — create → mirror → list round-trip. +- `backend/tests/test_collection_scoped_serve.py` — list/get/propose under a named collection. +- `frontend/src/lib/entryPaths.test.js` (extend) + `frontend/src/components/CollectionDirectory.test.jsx`. + +--- + +## Task 1: Registry mirror reads `.collection.yaml` manifests + +**Files:** +- Modify: `backend/app/registry.py` +- Test: `backend/tests/test_collection_registry.py` + +The mirror today upserts only the default collection from `projects.yaml`. Add a +`parse_collection_manifest(text)` (pure) and extend `refresh_registry` to walk each +project's content repo root, read every `/.collection.yaml`, and upsert a +collection row keyed by the subdir name. + +- [ ] **Step 1: Write the failing test** for the pure parser. + +```python +# backend/tests/test_collection_registry.py +import pytest +from app import registry + + +def test_parse_collection_manifest_minimal(): + doc = registry.parse_collection_manifest("type: bdd\n") + assert doc.type == "bdd" + # §22.4b: bdd defaults to 'active'; visibility inherits (None == inherit). + assert doc.initial_state == "active" + assert doc.visibility is None + assert doc.name is None + + +def test_parse_collection_manifest_full(): + doc = registry.parse_collection_manifest( + "type: document\nvisibility: public\ninitial_state: active\nname: Model\n" + ) + assert (doc.type, doc.visibility, doc.initial_state, doc.name) == ( + "document", "public", "active", "Model", + ) + + +def test_parse_collection_manifest_rejects_bad_type(): + with pytest.raises(registry.RegistryError): + registry.parse_collection_manifest("type: nonsense\n") +``` + +- [ ] **Step 2: Run it, verify it fails.** + +Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` +Expected: FAIL (`parse_collection_manifest` not defined). + +- [ ] **Step 3: Implement `parse_collection_manifest` + `CollectionEntry`** in `registry.py`. + +```python +# Add near ProjectEntry (after line 56). +@dataclass +class CollectionEntry: + type: str + visibility: str | None # None == inherit the project's visibility + initial_state: str + name: str | None + + +def parse_collection_manifest(text: str) -> CollectionEntry: + """Parse + validate a `.collection.yaml`. Pure. Raises RegistryError.""" + raw = yaml.safe_load(text) or {} + if not isinstance(raw, dict): + raise RegistryError("collection manifest must be a mapping") + ctype = str(raw.get("type") or "").strip() + if ctype not in VALID_TYPES: + raise RegistryError(f"collection has invalid type {ctype!r}") + vis = raw.get("visibility") + if vis is not None: + vis = str(vis).strip() + if vis not in VALID_VISIBILITY: + raise RegistryError(f"collection has invalid visibility {vis!r}") + initial_state = str( + raw.get("initial_state") or _TYPE_DEFAULT_INITIAL_STATE[ctype] + ).strip() + if initial_state not in VALID_INITIAL_STATE: + raise RegistryError(f"collection has invalid initial_state {initial_state!r}") + name = raw.get("name") + name = str(name).strip() if name else None + return CollectionEntry(ctype, vis, initial_state, name) +``` + +- [ ] **Step 4: Run the parser tests, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` +Expected: PASS (3 tests). + +- [ ] **Step 5: Write the failing mirror test** (uses a fake Gitea exposing the content repo). + +```python +# Append to backend/tests/test_collection_registry.py +import asyncio + + +class _FakeGitea: + """Minimal Gitea stub: projects.yaml in the registry repo + a content repo + holding one `.collection.yaml` under `features/`.""" + def __init__(self, projects_yaml, repo_tree): + self._projects_yaml = projects_yaml + self._repo_tree = repo_tree # {repo: {path: text}} + + async def get_contents(self, org, repo, path, ref="main"): + import base64 + if path == "projects.yaml": + enc = base64.b64encode(self._projects_yaml.encode()).decode() + return {"type": "file", "content": enc, "sha": "deadbeef"} + text = self._repo_tree.get(repo, {}).get(path) + if text is None: + return None + return {"type": "file", + "content": base64.b64encode(text.encode()).decode(), "sha": "c0ffee"} + + async def list_dir(self, org, repo, path, ref="main"): + # Root listing: surface each top-level subdir as a 'dir' entry. + names = set() + for p in self._repo_tree.get(repo, {}): + head = p.split("/", 1)[0] + if "/" in p: + names.add(head) + return [{"type": "dir", "name": n, "path": n} for n in sorted(names)] + + +def test_refresh_registry_mirrors_named_collection(app_with_db, config): + gitea = _FakeGitea( + projects_yaml=( + "deployment:\n name: Ohm\n tagline: t\n" + "projects:\n - id: ohm\n name: Ohm\n type: document\n" + " content_repo: ohm-rfc\n visibility: public\n" + ), + repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\nname: Features\n"}}, + ) + asyncio.run(registry.refresh_registry(config, gitea)) + from app import db + row = db.conn().execute( + "SELECT type, subfolder, name, project_id FROM collections WHERE id = 'features'" + ).fetchone() + assert row is not None + assert (row["type"], row["subfolder"], row["project_id"]) == ("bdd", "features", "ohm") +``` + +> Use the existing test app/db fixture pattern — mirror `app_with_db` / `config` +> fixtures from `backend/tests/test_registry.py`. If that test uses different +> fixture names, copy its setup verbatim here. + +- [ ] **Step 6: Run it, verify it fails** (`features` collection not mirrored). + +Run: `cd backend && python -m pytest tests/test_collection_registry.py::test_refresh_registry_mirrors_named_collection -q` +Expected: FAIL (row is None). + +- [ ] **Step 7: Extend `refresh_registry`** to discover + upsert named collections. + +After `apply_registry(...)` in `refresh_registry` (registry.py line 215), add: + +```python + # §22 S2: named collections are declared by `.collection.yaml` manifests + # inside each project's content repo (the default collection comes from + # projects.yaml above). Walk each content repo root; a subdir carrying a + # manifest becomes a collection keyed by the subdir name. + for proj in doc.projects: + try: + items = await gitea.list_dir(config.gitea_org, proj.content_repo, "", ref="main") + except Exception as e: # GiteaError or transport — tolerate, keep last-good + log.warning("registry: cannot list %s root: %s", proj.content_repo, e) + continue + for it in items: + if it.get("type") != "dir": + continue + subdir = it["name"] + manifest = await gitea.get_contents( + config.gitea_org, proj.content_repo, f"{subdir}/.collection.yaml", ref="main" + ) + if not manifest or manifest.get("type") != "file": + continue + mtext = base64.b64decode(manifest["content"]).decode("utf-8") + try: + ce = parse_collection_manifest(mtext) + except RegistryError as e: + log.error("registry: bad manifest %s/%s: %s", proj.content_repo, subdir, e) + continue + _upsert_named_collection(proj, subdir, ce, sha) +``` + +And add the upsert helper (after `apply_registry`): + +```python +def _upsert_named_collection(proj: ProjectEntry, subdir: str, ce: CollectionEntry, sha: str) -> None: + """Upsert one named collection. Type is immutable (§22.4a): a type change + against an existing row is refused (logged, not applied). Visibility None + inherits the project's visibility.""" + visibility = ce.visibility or proj.visibility + with db.tx() as conn: + existing = conn.execute( + "SELECT type FROM collections WHERE id = ?", (subdir,) + ).fetchone() + if existing is not None and existing["type"] != ce.type: + log.error("registry: refusing immutable type change on collection %s (%s -> %s)", + subdir, existing["type"], ce.type) + return + conn.execute( + """ + INSERT INTO collections + (id, project_id, type, subfolder, initial_state, visibility, name, registry_sha, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now')) + ON CONFLICT(id) DO UPDATE SET + project_id = excluded.project_id, + initial_state = excluded.initial_state, + visibility = excluded.visibility, + name = excluded.name, + registry_sha = excluded.registry_sha, + updated_at = datetime('now') + """, + (subdir, proj.id, ce.type, subdir, ce.initial_state, visibility, ce.name, sha), + ) +``` + +- [ ] **Step 8: Run all of Task 1's tests, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` +Expected: PASS (4 tests). + +- [ ] **Step 9: Commit.** + +```bash +git add backend/app/registry.py backend/tests/test_collection_registry.py +git commit -m "§22 S2: registry mirror reads .collection.yaml manifests" +``` + +--- + +## Task 2: Collection read helpers — `list_collections`, `get_collection`, `subfolder_of` + +**Files:** +- Modify: `backend/app/collections.py` +- Test: `backend/tests/test_collection_create_vertical.py` (first assertions) + +- [ ] **Step 1: Write the failing test.** + +```python +# backend/tests/test_collection_create_vertical.py +from app import db, collections as collections_mod + + +def _seed(project_id="ohm"): + db.conn().execute( + "INSERT INTO projects (id, name, content_repo, visibility, updated_at) " + "VALUES (?, 'Ohm', 'ohm-rfc', 'public', datetime('now'))", (project_id,)) + for cid, sub, vis, name in [ + ("default", "", "public", "Model"), + ("features", "features", "public", "Features"), + ("secret", "secret", "unlisted", "Secret"), + ]: + db.conn().execute( + "INSERT INTO collections (id, project_id, type, subfolder, initial_state, " + "visibility, name, created_at, updated_at) VALUES (?,?, 'document', ?, " + "'super-draft', ?, ?, datetime('now'), datetime('now'))", + (cid, project_id, sub, vis, name)) + + +def test_list_collections_excludes_unlisted(app_with_db): + _seed() + ids = [c["id"] for c in collections_mod.list_collections("ohm", include_unlisted=False)] + assert ids == ["default", "features"] # 'secret' (unlisted) omitted + + +def test_get_collection_and_subfolder(app_with_db): + _seed() + assert collections_mod.get_collection("features")["name"] == "Features" + assert collections_mod.subfolder_of("features") == "features" + assert collections_mod.subfolder_of("default") == "" + assert collections_mod.get_collection("nope") is None +``` + +- [ ] **Step 2: Run it, verify it fails.** + +Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` +Expected: FAIL (`list_collections` not defined). + +- [ ] **Step 3: Implement the helpers** in `collections.py`. + +```python +def get_collection(collection_id: str) -> dict | None: + row = db.conn().execute( + "SELECT id, project_id, type, subfolder, initial_state, visibility, name " + "FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return dict(row) if row else None + + +def subfolder_of(collection_id: str) -> str: + row = db.conn().execute( + "SELECT subfolder FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return (row["subfolder"] if row else "") or "" + + +def list_collections(project_id: str, include_unlisted: bool = False) -> list[dict]: + """Collections in a project, default first then by name. `unlisted` is + omitted from enumeration unless include_unlisted (a direct-id read).""" + rows = db.conn().execute( + "SELECT id, project_id, type, subfolder, initial_state, visibility, name " + "FROM collections WHERE project_id = ? ORDER BY (id != 'default'), name, id", + (project_id,), + ).fetchall() + out = [] + for r in rows: + if not include_unlisted and r["visibility"] == "unlisted": + continue + out.append(dict(r)) + return out +``` + +- [ ] **Step 4: Run it, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` +Expected: PASS (2 tests). + +- [ ] **Step 5: Commit.** + +```bash +git add backend/app/collections.py backend/tests/test_collection_create_vertical.py +git commit -m "§22 S2: collection read helpers (list/get/subfolder)" +``` + +--- + +## Task 3: Corpus mirror is collection-grained + +**Files:** +- Modify: `backend/app/cache.py:37-97` +- Test: `backend/tests/test_collection_scoped_serve.py` (first assertion) + +`refresh_meta_repo` reads only `rfcs/` (the default collection). Make it iterate a +project's collections and read each collection's `/rfcs/`, keying +`cached_rfcs` by `collection_id`. + +- [ ] **Step 1: Write the failing test.** + +```python +# backend/tests/test_collection_scoped_serve.py +import asyncio +from app import db, cache + + +class _CorpusGitea: + """A content repo with entries under both the default `rfcs/` and a named + collection's `features/rfcs/`.""" + def __init__(self, tree): + self._tree = tree # {path: text} + + async def list_dir(self, org, repo, path, ref="main"): + out = [] + prefix = (path.rstrip("/") + "/") if path else "" + for p in self._tree: + if p.startswith(prefix) and "/" not in p[len(prefix):]: + out.append({"type": "file", "name": p.split("/")[-1], "path": p}) + return out + + async def read_file(self, org, repo, path, ref="main"): + t = self._tree.get(path) + return (t, "sha-" + path) if t is not None else None + + +def _entry_md(slug, title): + return f"---\nslug: {slug}\ntitle: {title}\nstate: active\n---\nbody\n" + + +def test_mirror_keys_entries_by_collection(app_with_db, config): + db.conn().execute( + "INSERT INTO projects (id, name, content_repo, visibility, updated_at) " + "VALUES ('ohm','Ohm','ohm-rfc','public', datetime('now'))") + for cid, sub in [("default", ""), ("features", "features")]: + db.conn().execute( + "INSERT INTO collections (id, project_id, type, subfolder, initial_state, " + "visibility, created_at, updated_at) VALUES (?, 'ohm','document',?, " + "'super-draft','public', datetime('now'), datetime('now'))", (cid, sub)) + gitea = _CorpusGitea({ + "rfcs/a.md": _entry_md("a", "Default A"), + "features/rfcs/b.md": _entry_md("b", "Feature B"), + }) + asyncio.run(cache.refresh_meta_repo(config, gitea)) + got = {(r["collection_id"], r["slug"]) for r in + db.conn().execute("SELECT collection_id, slug FROM cached_rfcs")} + assert got == {("default", "a"), ("features", "b")} +``` + +- [ ] **Step 2: Run it, verify it fails** (entry `b` mirrored to wrong/absent collection). + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q` +Expected: FAIL. + +- [ ] **Step 3: Rework `refresh_meta_repo` + `_refresh_project_corpus`** in `cache.py`. + +Replace the body of `_refresh_project_corpus` so it loops the project's collections: + +```python +async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None: + # §22 S2: the corpus grain is the collection. Mirror each collection of the + # project from its `/rfcs/` directory, keying cached_rfcs by the + # collection id. The default collection (subfolder '') reads `rfcs/`. + from . import collections as collections_mod + for col in collections_mod.list_collections(project_id, include_unlisted=True): + collection_id = col["id"] + sub = col["subfolder"] or "" + rfcs_dir = f"{sub}/rfcs" if sub else "rfcs" + try: + files = await gitea.list_dir(org, repo, rfcs_dir, ref="main") + except GiteaError as e: + log.warning("refresh_meta_repo: %s/%s: cannot list %s: %s", + project_id, collection_id, rfcs_dir, e) + continue + seen_slugs: set[str] = set() + for f in files: + if f.get("type") != "file" or not f.get("name", "").endswith(".md"): + continue + result = await gitea.read_file(org, repo, f["path"], ref="main") + if not result: + continue + text, sha = result + try: + entry = entry_mod.parse(text) + except Exception as parse_err: + log.warning("refresh_meta_repo: %s/%s: skipping %s: %s", + project_id, collection_id, f["path"], parse_err) + continue + if not entry.slug: + continue + seen_slugs.add(entry.slug) + _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id) + existing = {row["slug"] for row in db.conn().execute( + "SELECT slug FROM cached_rfcs WHERE collection_id = ?", (collection_id,))} + for missing in existing - seen_slugs: + log.info("refresh_meta_repo: %s/%s/%s no longer present — leaving cache row", + project_id, collection_id, missing) +``` + +> `refresh_meta_repo` itself (the project loop) is unchanged — it still calls +> `_refresh_project_corpus(org, id, content_repo, gitea)` per project. + +- [ ] **Step 4: Run it, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q` +Expected: PASS. + +- [ ] **Step 5: Run the full backend suite** — the default-collection path must be unchanged. + +Run: `cd backend && python -m pytest -q` +Expected: PASS (all green; N=1 default path intact). + +- [ ] **Step 6: Commit.** + +```bash +git add backend/app/cache.py backend/tests/test_collection_scoped_serve.py +git commit -m "§22 S2: corpus mirror reads each collection's /rfcs/" +``` + +--- + +## Task 4: Collection-scoped serve + propose endpoints (backend) + +**Files:** +- Modify: `backend/app/api.py` — refactor propose to accept a collection; add scoped list/get/propose. +- Test: `backend/tests/test_collection_scoped_serve.py` (append) + +The shipped `_propose_into_project` hardcodes `collection_id = default_collection_id(project_id)` +(api.py:973). Generalize it to accept an explicit collection, and write the entry into +that collection's `/rfcs/`. + +- [ ] **Step 1: Write the failing test** (scoped list + propose targets the named collection). + +```python +# Append to backend/tests/test_collection_scoped_serve.py +def test_scoped_list_returns_only_that_collection(client_with_corpus): + client, _ = client_with_corpus # fixture seeds 'default'+'features' w/ entries a,b + r = client.get("/api/projects/ohm/collections/features/rfcs") + assert r.status_code == 200 + assert [i["slug"] for i in r.json()["items"]] == ["b"] + + +def test_scoped_propose_writes_into_collection_subfolder(client_with_corpus): + client, fake_bot = client_with_corpus + r = client.post("/api/projects/ohm/collections/features/rfcs/propose", + json={"title": "New", "slug": "newb", "pitch": "x", "tags": []}) + assert r.status_code == 200 + # The bot was asked to write under features/rfcs/, not rfcs/. + assert fake_bot.last_meta_path_prefix == "features/rfcs" +``` + +> Build `client_with_corpus` from the existing `app_with_fake_gitea` / +> propose-vertical fixture in `backend/tests/test_propose_vertical.py`; extend its +> fake bot to record the subfolder it was handed (see Step 3). If the propose-vertical +> fixture is structured differently, follow its shape and adapt these two asserts. + +- [ ] **Step 2: Run it, verify it fails.** + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` +Expected: FAIL (route 404 / propose ignores collection). + +- [ ] **Step 3: Generalize `_propose_into_project` → `_propose_into_collection`** in api.py. + +Change the signature and the two collection-dependent lines. The current helper +(api.py:959) resolves `collection_id` from the project default; instead take it as a +parameter and prefix the bot write path with the collection's subfolder: + +```python + async def _propose_into_collection( + project_id: str, collection_id: str, payload: ProposeBody, user + ) -> dict[str, Any]: + if not auth.can_contribute_in_project(user, project_id): + raise HTTPException(403, "You do not have contribute access to this project") + slug = payload.slug.strip().lower() + if not entry_mod.is_valid_slug(slug): + raise HTTPException(422, "Slug must be lowercase letters, digits, and dashes") + clash = db.conn().execute( + "SELECT 1 FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id) + ).fetchone() + if clash: + raise HTTPException(409, f"Slug `{slug}` is already taken") + # ...unchanged idea_clash / entry build... + landing_state = ( + "active" if collections_mod.collection_initial_state(collection_id) == "active" + else "super-draft" + ) + # ...build `entry`, `contents`, `pr_title`, `pr_description` unchanged... + subfolder = collections_mod.subfolder_of(collection_id) + rfcs_prefix = f"{subfolder}/rfcs" if subfolder else "rfcs" + pr = await bot.open_idea_pr( + user.as_actor(), + org=config.gitea_org, + meta_repo=(projects_mod.content_repo(project_id) or ""), + slug=slug, + file_contents=contents, + pr_title=pr_title, + pr_description=pr_description, + rfcs_dir=rfcs_prefix, + ) + # ...refresh + use_case persistence: pass collection_id (it already does)... +``` + +> Keep `_propose_into_project(project_id, payload, user)` as a thin wrapper that +> resolves the default collection and calls `_propose_into_collection`, so the +> existing `/api/rfcs/propose` and `/api/projects/:id/rfcs/propose` routes are +> unchanged. `landing_state` now derives from the collection (replacing the +> `projects_mod.project_initial_state` line at api.py:991). + +- [ ] **Step 4: Add `rfcs_dir` param to `bot.open_idea_pr`** (default `"rfcs"`), threading it + into the file path it writes. In `backend/app/bot.py`, `open_idea_pr` (≈line 168) builds the + entry file path as `f"rfcs/{slug}.md"`; change to accept `rfcs_dir: str = "rfcs"` and build + `f"{rfcs_dir}/{slug}.md"`. Leave every existing caller (which omits the arg) on `"rfcs"`. + +- [ ] **Step 5: Add the scoped read + propose routes** in api.py (beside the existing project routes). + +```python + @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs") + async def list_collection_rfcs(project_id: str, collection_id: str, request: Request, + unreviewed: str | None = None) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + _require_collection_in_project(collection_id, project_id) + return _list_rfcs_for_collection(collection_id, viewer, unreviewed) + + @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}") + async def get_collection_rfc(project_id: str, collection_id: str, slug: str, + request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + _require_collection_in_project(collection_id, project_id) + return _get_rfc_for_collection(collection_id, slug, viewer) + + @router.post("/api/projects/{project_id}/collections/{collection_id}/rfcs/propose") + async def propose_collection_rfc(project_id: str, collection_id: str, + payload: ProposeBody, request: Request) -> dict[str, Any]: + user = auth.require_contributor(request) + auth.require_project_readable(user, project_id) + _require_collection_in_project(collection_id, project_id) + return await _propose_into_collection(project_id, collection_id, payload, user) +``` + +Add the small helpers (refactor the bodies of the existing `list_project_rfcs` / +`get_project_rfc` at api.py:720-795 into `_list_rfcs_for_collection(collection_id, viewer, unreviewed)` +and `_get_rfc_for_collection(collection_id, slug, viewer)`, then have the project-scoped +routes call them with the default collection). Add: + +```python + def _require_collection_in_project(collection_id: str, project_id: str) -> None: + if collections_mod.project_of_collection(collection_id) != project_id: + raise HTTPException(404, "Not found") +``` + +- [ ] **Step 6: Run the scoped-serve tests, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` +Expected: PASS. + +- [ ] **Step 7: Run the full backend suite, verify PASS** (default-collection routes unchanged). + +Run: `cd backend && python -m pytest -q` +Expected: PASS. + +- [ ] **Step 8: Commit.** + +```bash +git add backend/app/api.py backend/app/bot.py backend/tests/test_collection_scoped_serve.py +git commit -m "§22 S2: collection-scoped list/get/propose endpoints" +``` + +--- + +## Task 5: create-collection endpoint (bot commit + registry refresh) + +**Files:** +- Create: `backend/app/api_collections.py` +- Modify: `backend/app/bot.py` (add `create_collection`), `backend/app/main.py` (mount router). +- Test: `backend/tests/test_collection_create_vertical.py` (append) + +Flow: deployment admin POSTs `{collection_id, type, name?, visibility?, initial_state?}`; +the bot commits `/.collection.yaml` to the project's content-repo `main`; +then `refresh_registry` re-reads and upserts the collection row. The registry stays the +source of truth (§22.2) — the endpoint never writes the `collections` row directly. + +- [ ] **Step 1: Write the failing vertical test.** + +```python +# Append to backend/tests/test_collection_create_vertical.py +def test_create_collection_commits_manifest_and_mirrors(admin_client_with_fake_gitea): + client, fake = admin_client_with_fake_gitea # seeds project 'ohm' + default collection + r = client.post("/api/projects/ohm/collections", + json={"collection_id": "features", "type": "bdd", "name": "Features"}) + assert r.status_code == 200, r.text + assert fake.committed_path == "features/.collection.yaml" + assert "type: bdd" in fake.committed_text + # registry refresh ran → row exists. + row = db.conn().execute("SELECT type, project_id FROM collections WHERE id='features'").fetchone() + assert (row["type"], row["project_id"]) == ("bdd", "ohm") + + +def test_create_collection_requires_admin(member_client): + r = member_client.post("/api/projects/ohm/collections", + json={"collection_id": "x", "type": "bdd"}) + assert r.status_code in (401, 403) + + +def test_create_collection_rejects_duplicate_id(admin_client_with_fake_gitea): + client, _ = admin_client_with_fake_gitea + client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"}) + r = client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"}) + assert r.status_code == 409 +``` + +> Reuse the `app_with_fake_gitea` admin fixture from `test_propose_vertical.py`; +> extend its fake bot/Gitea to record `committed_path` / `committed_text` for a +> root-level file create, and to make `refresh_registry`'s `list_dir` + `get_contents` +> see the just-committed manifest (an in-memory tree the fake mutates on commit). + +- [ ] **Step 2: Run it, verify it fails.** + +Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -k create_collection -q` +Expected: FAIL (route 404). + +- [ ] **Step 3: Add `Bot.create_collection`** in `bot.py` (commit the manifest to main). + +```python + async def create_collection(self, actor, *, org: str, content_repo: str, + collection_id: str, manifest_yaml: str) -> None: + """Commit `/.collection.yaml` to the content repo's main. + A structural admin action — commits straight to main (no PR), like + registry config. Logs an action row for the audit trail.""" + path = f"{collection_id}/.collection.yaml" + await self._gitea.create_file( + org, content_repo, path, manifest_yaml, + message=self._stamp(actor, f"chore: create collection {collection_id}"), + branch="main", + ) + self._log(actor, action="create_collection", + details={"collection_id": collection_id, "repo": content_repo}) +``` + +> Match the exact `create_file` signature + `_stamp`/`_log` call shapes used by the +> neighbouring bot methods (e.g. `commit_accepted_change`); names above are +> illustrative of the pattern, not necessarily the literal argument names. + +- [ ] **Step 4: Create `backend/app/api_collections.py`** with the read + create routes. + +```python +"""§22 S2 — collection directory + create-collection. + +GET /api/projects/:id/collections — list visible collections. +GET /api/projects/:id/collections/:cid — one collection's settings. +POST /api/projects/:id/collections — create a collection (admin; bot commits + a `.collection.yaml`, then the registry + mirror upserts the row — §22.2). +""" +from __future__ import annotations + +import re +from typing import Any + +import yaml +from fastapi import APIRouter, HTTPException, Request +from pydantic import BaseModel + +from . import auth, bot, cache, collections as collections_mod, db, gitea as gitea_mod +from . import projects as projects_mod, registry as registry_mod +from .config import Config +from .gitea import GiteaError + +_SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$") + + +class CreateCollectionBody(BaseModel): + collection_id: str + type: str + name: str | None = None + visibility: str | None = None + initial_state: str | None = None + + +def make_router(config: Config, gitea: gitea_mod.Gitea) -> APIRouter: + router = APIRouter() + + @router.get("/api/projects/{project_id}/collections") + async def list_cols(project_id: str, request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + return {"items": collections_mod.list_collections(project_id)} + + @router.get("/api/projects/{project_id}/collections/{collection_id}") + async def get_col(project_id: str, collection_id: str, request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + col = collections_mod.get_collection(collection_id) + if col is None or col["project_id"] != project_id: + raise HTTPException(404, "Not found") + return col + + @router.post("/api/projects/{project_id}/collections") + async def create_col(project_id: str, body: CreateCollectionBody, + request: Request) -> dict[str, Any]: + user = auth.require_admin(request) # deployment owner/admin (S2; scoped roles in S3) + auth.require_project_readable(user, project_id) + cid = body.collection_id.strip().lower() + if not _SLUG_RE.match(cid) or cid == "default": + raise HTTPException(422, "collection id must be a slug and not 'default'") + if body.type not in registry_mod.VALID_TYPES: + raise HTTPException(422, f"invalid type {body.type!r}") + if collections_mod.get_collection(cid) is not None: + raise HTTPException(409, f"collection `{cid}` already exists") + content_repo = projects_mod.content_repo(project_id) + if not content_repo: + raise HTTPException(409, "project has no content repo") + manifest: dict[str, Any] = {"type": body.type} + if body.name: + manifest["name"] = body.name + if body.visibility: + manifest["visibility"] = body.visibility + if body.initial_state: + manifest["initial_state"] = body.initial_state + manifest_yaml = yaml.safe_dump(manifest, sort_keys=False) + try: + await bot.create_collection( + user.as_actor(), org=config.gitea_org, content_repo=content_repo, + collection_id=cid, manifest_yaml=manifest_yaml) + except GiteaError as e: + raise HTTPException(502, f"Gitea: {e.detail}") + # §22.2: the registry mirror is the source of truth — re-read so the new + # manifest becomes a collections row. + await registry_mod.refresh_registry(config, gitea) + col = collections_mod.get_collection(cid) + if col is None: + raise HTTPException(500, "collection committed but not mirrored") + return col + + return router +``` + +> `bot.create_collection` is module-level if the codebase exposes `bot.` +> module functions (as the propose path uses `bot.open_idea_pr`); if `Bot` is a +> class instance, call it the same way the propose route does. Match the existing +> convention exactly. + +- [ ] **Step 5: Mount the router** in `backend/app/main.py` next to the other `make_router` mounts. + +```python + from . import api_collections + app.include_router(api_collections.make_router(config, gitea)) +``` + +- [ ] **Step 6: Run the create-collection tests, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` +Expected: PASS. + +- [ ] **Step 7: Run the full backend suite, verify PASS.** + +Run: `cd backend && python -m pytest -q` +Expected: PASS. + +- [ ] **Step 8: Commit.** + +```bash +git add backend/app/api_collections.py backend/app/bot.py backend/app/main.py backend/tests/test_collection_create_vertical.py +git commit -m "§22 S2: create-collection endpoint (bot commit + registry refresh)" +``` + +--- + +## Task 6: Frontend — collection-scoped path + API helpers + +**Files:** +- Modify: `frontend/src/lib/entryPaths.js`, `frontend/src/api.js` +- Test: `frontend/src/lib/entryPaths.test.js` (extend) + +- [ ] **Step 1: Write the failing test** for `useCollectionId` fallback + scoped API URLs. + +```js +// Append to frontend/src/lib/entryPaths.test.js +import { describe, it, expect } from 'vitest' +import { DEFAULT_COLLECTION } from './entryPaths' + +describe('collection paths', () => { + it('entryPath honors an explicit collection', () => { + // entryPath(pid, slug, cid) + const { entryPath } = require('./entryPaths') + expect(entryPath('ohm', 'a', 'features')).toBe('/p/ohm/c/features/e/a') + expect(entryPath('ohm', 'a')).toBe(`/p/ohm/c/${DEFAULT_COLLECTION}/e/a`) + }) +}) +``` + +> Match the file's existing import/runner style (it may use ESM `import` rather +> than `require`); follow whatever `entryPaths.test.js` already does. + +- [ ] **Step 2: Run it, verify PASS or FAIL** (entryPath already supports cid → this asserts the + contract; the new piece is `useCollectionId`). If green, proceed; the behavioral gap is the hook. + +Run: `cd frontend && npx vitest run src/lib/entryPaths.test.js` + +- [ ] **Step 3: Add `useCollectionId`** to `entryPaths.js`. + +```js +import { useParams } from 'react-router-dom' + +// The collection id a component should scope to: the `/c/:collectionId/` segment +// when present, else the project's default collection. +export function useCollectionId() { + const { collectionId } = useParams() + return collectionId || DEFAULT_COLLECTION +} +``` + +- [ ] **Step 4: Make `api.js` collection-aware.** Add optional `collectionId` to the corpus reads + and propose; add `listCollections` + `createCollection`. + +```js +export async function listRFCs(projectId, collectionId) { + if (projectId && collectionId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs`)) + } + const url = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs' + return jsonOrThrow(await fetch(url)) +} + +export async function getRFC(projectId, slug, collectionId) { + if (collectionId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}`)) + } + if (slug === undefined) return jsonOrThrow(await fetch(`/api/rfcs/${projectId}`)) + return jsonOrThrow(await fetch(`/api/projects/${projectId}/rfcs/${slug}`)) +} + +export async function listCollections(projectId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections`)) +} + +export async function createCollection(projectId, { collectionId, type, name, visibility, initialState }) { + const res = await fetch(`/api/projects/${projectId}/collections`, { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ collection_id: collectionId, type, name: name || null, + visibility: visibility || null, initial_state: initialState || null }), + }) + return jsonOrThrow(res) +} +``` + + And add an optional `collectionId` to `proposeRFC` so it targets the scoped route: + +```js +export async function proposeRFC(projectId, { title, slug, pitch, tags, proposedUseCase, collectionId }) { + const url = (projectId && collectionId) + ? `/api/projects/${projectId}/collections/${collectionId}/rfcs/propose` + : (projectId ? `/api/projects/${projectId}/rfcs/propose` : '/api/rfcs/propose') + // ...unchanged body/post... +} +``` + +- [ ] **Step 5: Run the frontend unit tests, verify PASS.** + +Run: `cd frontend && npx vitest run src/lib/entryPaths.test.js` +Expected: PASS. + +- [ ] **Step 6: Commit.** + +```bash +git add frontend/src/lib/entryPaths.js frontend/src/api.js frontend/src/lib/entryPaths.test.js +git commit -m "§22 S2: collection-scoped frontend path + API helpers" +``` + +--- + +## Task 7: Frontend — Catalog reads the active collection + +**Files:** +- Modify: `frontend/src/components/Catalog.jsx`, `frontend/src/App.jsx` (propose wiring) + +- [ ] **Step 1: Scope the Catalog to the active collection.** In `Catalog.jsx`: + - import `useCollectionId` from `../lib/entryPaths`; + - `const cid = useCollectionId()`; + - fetch `listRFCs(pid, cid)` and add `cid` to the effect deps; + - build entry links with `entryPath(pid, r.slug, cid)` and `proposalPath(pid, p.pr_number, cid)`. + + The anonymous empty-state (C3.6) is already correct — the footer renders the + "Sign in to propose" link for `!viewer`, and the empty list shows "No RFCs in the + catalog yet." Confirm both render for an anonymous viewer on an empty collection. + +- [ ] **Step 2: Thread the active collection into the propose modal** in `App.jsx`. The + `ProposeModal` is mounted with `projectId={currentProjectId}`; also pass the active + collection (derive from the route — read `useParams().collectionId` in the `AppShell` + scope, default `DEFAULT_COLLECTION`) and pass it to `proposeRFC` so a propose from a + named collection targets that collection. On submit, navigate with `proposalPath(currentProjectId, pr_number, currentCollectionId)`. + +- [ ] **Step 3: Build + run the frontend test suite, verify PASS.** + +Run: `cd frontend && npx vitest run` +Expected: PASS (existing tests green; default-collection behavior unchanged). + +- [ ] **Step 4: Commit.** + +```bash +git add frontend/src/components/Catalog.jsx frontend/src/App.jsx +git commit -m "§22 S2: Catalog + propose scoped to the active collection" +``` + +--- + +## Task 8: Frontend — the collection directory at `/p//` + +**Files:** +- Create: `frontend/src/components/CollectionDirectory.jsx` +- Modify: `frontend/src/App.jsx` (swap `DefaultCollectionRedirect` → `CollectionDirectory`) +- Test: `frontend/src/components/CollectionDirectory.test.jsx` + +- [ ] **Step 1: Write the failing test.** + +```jsx +// frontend/src/components/CollectionDirectory.test.jsx +import { render, screen, waitFor } from '@testing-library/react' +import { MemoryRouter } from 'react-router-dom' +import { describe, it, expect, vi } from 'vitest' +import CollectionDirectory from './CollectionDirectory' + +vi.mock('../api', () => ({ + listCollections: vi.fn(async () => ({ items: [ + { id: 'default', name: 'Model', type: 'document' }, + { id: 'features', name: 'Features', type: 'bdd' }, + ] })), +})) + +describe('CollectionDirectory', () => { + it('lists collections when there are 2+', async () => { + render() + await waitFor(() => expect(screen.getByText('Features')).toBeInTheDocument()) + expect(screen.getByText('Model')).toBeInTheDocument() + }) +}) +``` + +> Match the project's component-test conventions (see `Directory.test.jsx` / +> `ProjectLayout.test.jsx` for the render + mock pattern; adapt the mock + queries +> to whatever they use). + +- [ ] **Step 2: Run it, verify it fails** (component missing). + +Run: `cd frontend && npx vitest run src/components/CollectionDirectory.test.jsx` +Expected: FAIL. + +- [ ] **Step 3: Implement `CollectionDirectory.jsx`.** Fetch `listCollections(projectId)`; when + exactly one visible collection, `` to its `collectionHome` (preserves the S1 + C3.7/C3.8 single-collection redirect); when 2+, render a list of links to each + `collectionHome(projectId, c.id)` with its name + type. (The "Create your first + collection" empty-state is S4 — for S2 a 0-collection project simply shows a minimal + "No collections yet." line; the create affordance UI lands in S4.) + +```jsx +import { useEffect, useState } from 'react' +import { Link, Navigate } from 'react-router-dom' +import { listCollections } from '../api' +import { collectionHome } from '../lib/entryPaths' + +export default function CollectionDirectory({ projectId }) { + const [cols, setCols] = useState(null) + useEffect(() => { + let live = true + listCollections(projectId).then(d => { if (live) setCols(d.items) }).catch(() => live && setCols([])) + return () => { live = false } + }, [projectId]) + if (cols === null) return
Loading…
+ if (cols.length === 1) return + return ( +
+
+

Collections

+ {cols.length === 0 ? ( +

No collections yet.

+ ) : ( +
    + {cols.map(c => ( +
  • + {c.name || c.id} + · {c.type} +
    • + ))} +
+ )} +
+
+ ) +} +``` + +- [ ] **Step 4: Wire it into `App.jsx`.** Replace the `DefaultCollectionRedirect` route element + at the project landing (App.jsx:365) with `} />`. + Read the project id from `useParams()` inside a small wrapper (mirroring how + `DefaultCollectionRedirect` reads it), and keep `DefaultCollectionRedirect` only if still + referenced elsewhere (otherwise delete it). The S1 single-collection redirect now lives + inside `CollectionDirectory`, so the C3.7/C3.8 behavior is preserved. + +- [ ] **Step 5: Run the directory test + full frontend suite, verify PASS.** + +Run: `cd frontend && npx vitest run` +Expected: PASS. + +- [ ] **Step 6: Commit.** + +```bash +git add frontend/src/components/CollectionDirectory.jsx frontend/src/components/CollectionDirectory.test.jsx frontend/src/App.jsx +git commit -m "§22 S2: collection directory at /p// (1 → redirect, 2+ → list)" +``` + +--- + +## Task 9: Acceptance — C3.6 anonymous empty-collection catalog + +**Files:** +- Test: `backend/tests/test_collection_scoped_serve.py` (append the `@S2` acceptance assertion) + +The `@S2` gate is C3.6: *a public collection with no entries; an anonymous visitor +lands on `/p/ohm/c/model/`; they see an empty catalog with no propose action and a +sign-in prompt.* Backend half: the scoped list returns `{items: []}` for an anonymous +viewer on a public empty collection (no 404, no propose surfaced). Frontend half: the +Catalog footer renders "Sign in to propose" for `!viewer` (already covered by Task 7). + +- [ ] **Step 1: Write the acceptance test** (backend contract for the empty public collection). + +```python +def test_s2_anonymous_empty_public_collection(client_with_corpus): + """C3.6 (@S2): anonymous viewer, public empty collection → empty catalog, 200.""" + client, _ = client_with_corpus # ensure an empty public collection 'model' + r = client.get("/api/projects/ohm/collections/model/rfcs") # no auth header + assert r.status_code == 200 + assert r.json()["items"] == [] +``` + +> If `client_with_corpus` doesn't already seed an empty public `model` collection, +> add one (mirror Task 3's seed). The propose-absence is enforced by `require_contributor` +> on the propose route (anonymous → 401/403) and the Catalog UI footer. + +- [ ] **Step 2: Run it, verify PASS.** + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_s2_anonymous_empty_public_collection -q` +Expected: PASS. + +- [ ] **Step 3: Manually verify the propose route rejects anonymous** (sanity — no new code): + +Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` +Expected: PASS (all). + +- [ ] **Step 4: Commit.** + +```bash +git add backend/tests/test_collection_scoped_serve.py +git commit -m "§22 S2: @S2 acceptance — anonymous empty public collection catalog" +``` + +--- + +## Task 10: Release — version bump, changelog, docs + +**Files:** +- Modify: `VERSION`, `frontend/package.json`, `CHANGELOG.md` +- Modify: `docs/design/2026-06-05-three-tier-projects-collections.md` (mark S2 shipped) + +S2 adds functionality and is non-breaking (new optional path segment + new endpoints; +the default-collection paths are unchanged). Per SPEC §20, that's a **minor** bump: +`0.40.0` → `0.41.0`. + +- [ ] **Step 1: Bump `VERSION`** to `0.41.0`. + +- [ ] **Step 2: Bump `frontend/package.json#version`** to `0.41.0` (must mirror VERSION — §20). + +- [ ] **Step 3: Add the `CHANGELOG.md` entry** under a new `## 0.41.0` heading: a minor + release shipping S2 (create + navigate + propose a second collection): registry mirror + reads `.collection.yaml`; collection-grained corpus mirror; create-collection endpoint; + collection-scoped list/get/propose; `/p//` collection directory. Note it + completes `@S2` (C3.6). No upgrade steps required (additive; existing default-collection + deployments keep working unchanged) — state that explicitly. + +- [ ] **Step 4: Mark S2 shipped** in the design doc's Part E slice list (a short + "Landed vX" note on the S2 bullet, mirroring how S1 was annotated). + +- [ ] **Step 5: Run both suites once more, verify green.** + +Run: `cd backend && python -m pytest -q && cd ../frontend && npx vitest run` +Expected: PASS (both). + +- [ ] **Step 6: Commit.** + +```bash +git add VERSION frontend/package.json CHANGELOG.md docs/design/2026-06-05-three-tier-projects-collections.md +git commit -m "§22 S2: release v0.41.0 — create & navigate a second collection (@S2)" +``` + +--- + +## Self-review notes + +- **Spec coverage (Part E "S2" bullet):** registry reads `.collection.yaml` (Task 1) · + create-collection endpoint, admin-gated (Task 5) · project collection-directory at + `/p//` (Task 8) · collection-scoped propose/serve (Tasks 3,4,7) · `@S2` / C3.6 + acceptance (Task 9). ✓ +- **N=1 unchanged invariant:** every backend task ends by running the full suite; the + default-collection routes are preserved as thin wrappers over the new collection-scoped + internals. ✓ +- **Type consistency:** `collection_id`/`subfolder` naming is uniform; the entries path is + `/rfcs/` everywhere (cache mirror Task 3, propose Task 4, create manifest Task 5). +- **Deferred / lower-confidence calls (log to transcript):** (a) create-collection commits + straight to `main` rather than via a PR — chosen because a collection is structural config + and the registry mirror is the source of truth; (b) S2 collection visibility filtering is + coarse (project read-gate + drop `unlisted`); full scope-role enforcement is S3; (c) the + per-collection "create"/"propose-first" *empty-state affordances* are S4 — S2 ships only the + anonymous empty catalog (C3.6) and a minimal directory. -- 2.39.5 From 74476423ba239285e79c8631c7921324b9b5fcc1 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 12:59:25 -0700 Subject: [PATCH 11/21] =?UTF-8?q?=C2=A722=20S2:=20registry=20mirror=20read?= =?UTF-8?q?s=20.collection.yaml=20manifests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Discover named collections by walking each project's content-repo root for /.collection.yaml; parse + upsert with immutable-type enforcement (§22.4a) and project-visibility inheritance. The default collection still flows from projects.yaml. Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/app/registry.py | 105 ++++++++++++++++ backend/tests/test_collection_registry.py | 146 ++++++++++++++++++++++ 2 files changed, 251 insertions(+) create mode 100644 backend/tests/test_collection_registry.py diff --git a/backend/app/registry.py b/backend/app/registry.py index 38981b2..b86a7fb 100644 --- a/backend/app/registry.py +++ b/backend/app/registry.py @@ -55,6 +55,17 @@ class ProjectEntry: config: dict = field(default_factory=dict) # theme, enabled_models +@dataclass +class CollectionEntry: + """A named collection declared by a `.collection.yaml` manifest inside a + project's content repo (S2). `visibility=None` means "inherit the project's + visibility".""" + type: str + visibility: str | None + initial_state: str + name: str | None + + @dataclass class RegistryDoc: deployment_name: str @@ -114,6 +125,33 @@ def parse_registry(text: str) -> RegistryDoc: ) +def parse_collection_manifest(text: str) -> CollectionEntry: + """Parse + validate a `.collection.yaml`. Pure (no I/O). Raises RegistryError. + + `type` is required and immutable (§22.4a, enforced at upsert). `visibility` + is optional — omitted means inherit the project's. `initial_state` defaults + per type (§22.4b).""" + raw = yaml.safe_load(text) or {} + if not isinstance(raw, dict): + raise RegistryError("collection manifest must be a mapping") + ctype = str(raw.get("type") or "").strip() + if ctype not in VALID_TYPES: + raise RegistryError(f"collection has invalid type {ctype!r}") + vis = raw.get("visibility") + if vis is not None: + vis = str(vis).strip() + if vis not in VALID_VISIBILITY: + raise RegistryError(f"collection has invalid visibility {vis!r}") + initial_state = str( + raw.get("initial_state") or _TYPE_DEFAULT_INITIAL_STATE[ctype] + ).strip() + if initial_state not in VALID_INITIAL_STATE: + raise RegistryError(f"collection has invalid initial_state {initial_state!r}") + name = raw.get("name") + name = str(name).strip() if name else None + return CollectionEntry(ctype, vis, initial_state, name) + + def _default_collection_id(project_id: str, default_id: str) -> str: """The id of a project's default collection. The deployment's primary project (== `default_id`, the §22.13 resolved default) gets the stable @@ -193,6 +231,71 @@ def apply_registry(doc: RegistryDoc, registry_sha: str, default_id: str) -> None ) +def _upsert_named_collection( + proj: ProjectEntry, subdir: str, ce: CollectionEntry, sha: str +) -> None: + """Upsert one named collection (S2). Type is immutable (§22.4a): a type + change against an existing row is refused (logged, not applied). A None + manifest visibility inherits the project's visibility.""" + visibility = ce.visibility or proj.visibility + with db.tx() as conn: + existing = conn.execute( + "SELECT type FROM collections WHERE id = ?", (subdir,) + ).fetchone() + if existing is not None and existing["type"] != ce.type: + log.error( + "registry: refusing immutable type change on collection %s (%s -> %s)", + subdir, existing["type"], ce.type, + ) + return + conn.execute( + """ + INSERT INTO collections + (id, project_id, type, subfolder, initial_state, visibility, name, registry_sha, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now')) + ON CONFLICT(id) DO UPDATE SET + project_id = excluded.project_id, + initial_state = excluded.initial_state, + visibility = excluded.visibility, + name = excluded.name, + registry_sha = excluded.registry_sha, + updated_at = datetime('now') + """, + (subdir, proj.id, ce.type, subdir, ce.initial_state, visibility, ce.name, sha), + ) + + +async def _mirror_named_collections(config: Config, gitea: Gitea, doc: RegistryDoc, sha: str) -> None: + """§22 S2: named collections are declared by `.collection.yaml` manifests + inside each project's content repo (the default collection comes from + projects.yaml). Walk each content repo root; a subdir carrying a manifest + becomes a collection keyed by the subdir name. Tolerant: a transport or + parse failure on one project/collection logs and is skipped, never aborts + the wider mirror (keep last-good).""" + for proj in doc.projects: + try: + items = await gitea.list_dir(config.gitea_org, proj.content_repo, "", ref="main") + except Exception as e: # noqa: BLE001 — GiteaError/transport: tolerate + log.warning("registry: cannot list %s root: %s", proj.content_repo, e) + continue + for it in items: + if it.get("type") != "dir": + continue + subdir = it["name"] + manifest = await gitea.get_contents( + config.gitea_org, proj.content_repo, f"{subdir}/.collection.yaml", ref="main" + ) + if not manifest or manifest.get("type") != "file": + continue + mtext = base64.b64decode(manifest["content"]).decode("utf-8") + try: + ce = parse_collection_manifest(mtext) + except RegistryError as e: + log.error("registry: bad manifest %s/%s: %s", proj.content_repo, subdir, e) + continue + _upsert_named_collection(proj, subdir, ce, sha) + + async def refresh_registry(config: Config, gitea: Gitea) -> None: """Mirror REGISTRY_REPO/projects.yaml into projects + deployment. @@ -213,4 +316,6 @@ async def refresh_registry(config: Config, gitea: Gitea) -> None: doc = parse_registry(text) from . import projects as projects_mod apply_registry(doc, sha, projects_mod.resolved_default_id(config)) + # §22 S2: discover + upsert named collections from each content repo. + await _mirror_named_collections(config, gitea, doc, sha) log.info("registry: mirrored %d project(s) at %s", len(doc.projects), sha) diff --git a/backend/tests/test_collection_registry.py b/backend/tests/test_collection_registry.py new file mode 100644 index 0000000..fa8a403 --- /dev/null +++ b/backend/tests/test_collection_registry.py @@ -0,0 +1,146 @@ +"""§22 S2 — the registry mirror reads `.collection.yaml` manifests inside each +project's content repo and upserts a named collection per manifest. The default +collection still flows from projects.yaml (test_registry.py).""" +from __future__ import annotations + +import asyncio +import base64 +import tempfile +from pathlib import Path + +import pytest + +from app import db, registry +from app.config import Config + + +def _db() -> Config: + cfg = Config( + gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="wiggleverse", + registry_repo="registry", oauth_client_id="x", + oauth_client_secret="x", app_url="x", secret_key="x", + database_path=Path(tempfile.mkdtemp(prefix="colreg-")) / "t.db", + owner_gitea_login="x", webhook_secret="x", + ) + db.run_migrations(cfg) + if db._CONN is not None: + db._CONN.close() + db._CONN = None + db.init(cfg) + return cfg + + +# --- pure parser -------------------------------------------------------------- + + +def test_parse_collection_manifest_minimal(): + doc = registry.parse_collection_manifest("type: bdd\n") + assert doc.type == "bdd" + # §22.4b: bdd defaults to 'active'; visibility inherits (None == inherit). + assert doc.initial_state == "active" + assert doc.visibility is None + assert doc.name is None + + +def test_parse_collection_manifest_full(): + doc = registry.parse_collection_manifest( + "type: document\nvisibility: public\ninitial_state: active\nname: Model\n" + ) + assert (doc.type, doc.visibility, doc.initial_state, doc.name) == ( + "document", "public", "active", "Model", + ) + + +def test_parse_collection_manifest_rejects_bad_type(): + with pytest.raises(registry.RegistryError): + registry.parse_collection_manifest("type: nonsense\n") + + +def test_parse_collection_manifest_rejects_bad_visibility(): + with pytest.raises(registry.RegistryError): + registry.parse_collection_manifest("type: bdd\nvisibility: nope\n") + + +# --- mirror discovery --------------------------------------------------------- + + +class _FakeGitea: + """Minimal Gitea stub: projects.yaml in the registry repo + a content repo + whose root holds a `features/` subdir carrying a `.collection.yaml`.""" + + def __init__(self, projects_yaml: str, repo_tree: dict[str, dict[str, str]]): + self._projects_yaml = projects_yaml + self._repo_tree = repo_tree # {repo: {path: text}} + + async def get_contents(self, org, repo, path, ref="main"): + if path == "projects.yaml": + return {"type": "file", + "content": base64.b64encode(self._projects_yaml.encode()).decode(), + "sha": "regsha-test"} + text = self._repo_tree.get(repo, {}).get(path) + if text is None: + return None + return {"type": "file", + "content": base64.b64encode(text.encode()).decode(), "sha": "c0ffee"} + + async def list_dir(self, org, repo, path, ref="main"): + # Root listing: surface each top-level segment as a 'dir' entry. + prefix = (path.rstrip("/") + "/") if path else "" + dirs = set() + for p in self._repo_tree.get(repo, {}): + if not p.startswith(prefix): + continue + rest = p[len(prefix):] + if "/" in rest: + dirs.add(rest.split("/", 1)[0]) + return [{"type": "dir", "name": n, "path": prefix + n} for n in sorted(dirs)] + + +_PROJECTS = ( + "deployment:\n name: Ohm\n tagline: t\n" + "projects:\n - id: ohm\n name: Ohm\n type: document\n" + " content_repo: ohm-rfc\n visibility: public\n" +) + + +def test_refresh_registry_mirrors_named_collection(): + cfg = _db() + gitea = _FakeGitea( + projects_yaml=_PROJECTS, + repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\nname: Features\n"}}, + ) + asyncio.run(registry.refresh_registry(cfg, gitea)) + row = db.conn().execute( + "SELECT type, subfolder, name, project_id, visibility FROM collections WHERE id='features'" + ).fetchone() + assert row is not None + assert (row["type"], row["subfolder"], row["project_id"]) == ("bdd", "features", "ohm") + assert row["name"] == "Features" + # visibility inherits the project's (public) when the manifest omits it. + assert row["visibility"] == "public" + + +def test_refresh_registry_leaves_default_collection_intact(): + cfg = _db() + gitea = _FakeGitea( + projects_yaml=_PROJECTS, + repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\n"}}, + ) + asyncio.run(registry.refresh_registry(cfg, gitea)) + # The default collection (from projects.yaml) and the named one coexist. + ids = {r["id"] for r in db.conn().execute("SELECT id FROM collections")} + assert {"default", "features"} <= ids + + +def test_refresh_registry_immutable_type_on_named_collection(): + cfg = _db() + gitea = _FakeGitea( + projects_yaml=_PROJECTS, + repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\n"}}, + ) + asyncio.run(registry.refresh_registry(cfg, gitea)) + # A later manifest that flips the type is refused (§22.4a immutable type). + gitea._repo_tree["ohm-rfc"]["features/.collection.yaml"] = "type: document\n" + asyncio.run(registry.refresh_registry(cfg, gitea)) + t = db.conn().execute("SELECT type FROM collections WHERE id='features'").fetchone()["type"] + assert t == "bdd" -- 2.39.5 From 868391870cbd2bbeedbe8d847b019217d02e9183 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 13:00:28 -0700 Subject: [PATCH 12/21] =?UTF-8?q?=C2=A722=20S2:=20collection=20read=20help?= =?UTF-8?q?ers=20(list/get/subfolder)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/app/collections.py | 36 +++++++++++++ backend/tests/test_collection_helpers.py | 64 ++++++++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 backend/tests/test_collection_helpers.py diff --git a/backend/app/collections.py b/backend/app/collections.py index d767b32..d5b8f9c 100644 --- a/backend/app/collections.py +++ b/backend/app/collections.py @@ -48,3 +48,39 @@ def collection_type(collection_id: str) -> str: "SELECT type FROM collections WHERE id = ?", (collection_id,) ).fetchone() return row["type"] if row and row["type"] else "document" + + +def subfolder_of(collection_id: str) -> str: + """The content-repo subfolder a collection lives under (§22.3). Empty string + for the default collection (entries at the repo root `rfcs/`).""" + row = db.conn().execute( + "SELECT subfolder FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + return (row["subfolder"] if row else "") or "" + + +def get_collection(collection_id: str) -> dict | None: + """The full collection row as a dict, or None if unknown.""" + row = db.conn().execute( + "SELECT id, project_id, type, subfolder, initial_state, visibility, name " + "FROM collections WHERE id = ?", + (collection_id,), + ).fetchone() + return dict(row) if row else None + + +def list_collections(project_id: str, include_unlisted: bool = False) -> list[dict]: + """Collections in a project, the default first then by name (§22.5). `unlisted` + is omitted from enumeration unless include_unlisted (a direct-id read or the + corpus mirror, which serves every collection).""" + rows = db.conn().execute( + "SELECT id, project_id, type, subfolder, initial_state, visibility, name " + "FROM collections WHERE project_id = ? ORDER BY (id != 'default'), name, id", + (project_id,), + ).fetchall() + out: list[dict] = [] + for r in rows: + if not include_unlisted and r["visibility"] == "unlisted": + continue + out.append(dict(r)) + return out diff --git a/backend/tests/test_collection_helpers.py b/backend/tests/test_collection_helpers.py new file mode 100644 index 0000000..77c0e02 --- /dev/null +++ b/backend/tests/test_collection_helpers.py @@ -0,0 +1,64 @@ +"""§22 S2 — collection read helpers: list_collections / get_collection / +subfolder_of.""" +from __future__ import annotations + +import tempfile +from pathlib import Path + +from app import collections as collections_mod, db +from app.config import Config + + +def _db() -> Config: + cfg = Config( + gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="x", + registry_repo="registry", oauth_client_id="x", + oauth_client_secret="x", app_url="x", secret_key="x", + database_path=Path(tempfile.mkdtemp(prefix="colhelp-")) / "t.db", + owner_gitea_login="x", webhook_secret="x", + ) + db.run_migrations(cfg) + if db._CONN is not None: + db._CONN.close() + db._CONN = None + db.init(cfg) + return cfg + + +def _seed(project_id="ohm"): + db.conn().execute( + "INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) " + "VALUES (?, 'Ohm', 'ohm-rfc', 'public', datetime('now'))", (project_id,)) + for cid, sub, vis, name in [ + ("default", "", "public", "Model"), + ("features", "features", "public", "Features"), + ("secret", "secret", "unlisted", "Secret"), + ]: + db.conn().execute( + "INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, initial_state, " + "visibility, name, created_at, updated_at) VALUES (?,?, 'document', ?, " + "'super-draft', ?, ?, datetime('now'), datetime('now'))", + (cid, project_id, sub, vis, name)) + + +def test_list_collections_excludes_unlisted(): + _db() + _seed() + ids = [c["id"] for c in collections_mod.list_collections("ohm", include_unlisted=False)] + assert ids == ["default", "features"] # default first, then by name; 'secret' omitted + + +def test_list_collections_include_unlisted(): + _db() + _seed() + ids = {c["id"] for c in collections_mod.list_collections("ohm", include_unlisted=True)} + assert ids == {"default", "features", "secret"} + + +def test_get_collection_and_subfolder(): + _db() + _seed() + assert collections_mod.get_collection("features")["name"] == "Features" + assert collections_mod.subfolder_of("features") == "features" + assert collections_mod.subfolder_of("default") == "" + assert collections_mod.get_collection("nope") is None -- 2.39.5 From 91b0fb358c1857f5f2263a5abc181f19983bf195 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 13:02:59 -0700 Subject: [PATCH 13/21] =?UTF-8?q?=C2=A722=20S2:=20corpus=20mirror=20reads?= =?UTF-8?q?=20each=20collection's=20/rfcs/?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit refresh_meta_repo now iterates a project's collections and keys cached_rfcs by collection_id; the default collection (subfolder '') keeps the shipped rfcs/ root path. N=1 default path unchanged (469 green). Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/app/cache.py | 37 ++++++--- backend/tests/test_collection_scoped_serve.py | 77 +++++++++++++++++++ 2 files changed, 103 insertions(+), 11 deletions(-) create mode 100644 backend/tests/test_collection_scoped_serve.py diff --git a/backend/app/cache.py b/backend/app/cache.py index 538c313..aaf9f70 100644 --- a/backend/app/cache.py +++ b/backend/app/cache.py @@ -54,15 +54,27 @@ async def refresh_meta_repo(config: Config, gitea: Gitea) -> None: async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None: - # §22 S1: the corpus grain is the collection. The mirror is project-grained - # (reads rfcs/ at the repo root = the project's default collection); resolve - # that collection once and key cached_rfcs by it. + # §22 S2: the corpus grain is the collection. Mirror every collection of the + # project from its `/rfcs/` directory, keying cached_rfcs by the + # collection id. The default collection (subfolder '') reads `rfcs/` — the + # shipped path, unchanged. include_unlisted: the mirror serves every + # collection's content regardless of enumeration visibility. from . import collections as collections_mod - collection_id = collections_mod.default_collection_id(project_id) + for col in collections_mod.list_collections(project_id, include_unlisted=True): + await _refresh_collection_corpus( + org, project_id, repo, col["id"], col["subfolder"] or "", gitea + ) + + +async def _refresh_collection_corpus( + org: str, project_id: str, repo: str, collection_id: str, subfolder: str, gitea: Gitea +) -> None: + rfcs_dir = f"{subfolder}/rfcs" if subfolder else "rfcs" try: - files = await gitea.list_dir(org, repo, "rfcs", ref="main") + files = await gitea.list_dir(org, repo, rfcs_dir, ref="main") except GiteaError as e: - log.warning("refresh_meta_repo: project %s: cannot list rfcs/: %s", project_id, e) + log.warning("refresh_meta_repo: %s/%s: cannot list %s: %s", + project_id, collection_id, rfcs_dir, e) return seen_slugs: set[str] = set() @@ -76,17 +88,19 @@ async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: G try: entry = entry_mod.parse(text) except Exception as parse_err: - log.warning("refresh_meta_repo: %s: skipping %s: %s", project_id, f["path"], parse_err) + log.warning("refresh_meta_repo: %s/%s: skipping %s: %s", + project_id, collection_id, f["path"], parse_err) continue if not entry.slug: - log.warning("refresh_meta_repo: %s: skipping %s: missing slug", project_id, f["path"]) + log.warning("refresh_meta_repo: %s/%s: skipping %s: missing slug", + project_id, collection_id, f["path"]) continue seen_slugs.add(entry.slug) _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id) - # Entries removed from a project's rfcs/ — the spec keeps withdrawn entries + # Entries removed from a collection's rfcs/ — the spec keeps withdrawn entries # as historical record (§3), so this fires only for out-of-band deletes; - # leave the row, scoped to this project, for reconciler attention. + # leave the row, scoped to this collection, for reconciler attention. existing = { row["slug"] for row in db.conn().execute( @@ -94,7 +108,8 @@ async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: G ) } for missing in existing - seen_slugs: - log.info("refresh_meta_repo: %s/%s no longer in rfcs/ — leaving cache row", project_id, missing) + log.info("refresh_meta_repo: %s/%s/%s no longer present — leaving cache row", + project_id, collection_id, missing) def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str, collection_id: str = "default") -> None: diff --git a/backend/tests/test_collection_scoped_serve.py b/backend/tests/test_collection_scoped_serve.py new file mode 100644 index 0000000..f3d1c27 --- /dev/null +++ b/backend/tests/test_collection_scoped_serve.py @@ -0,0 +1,77 @@ +"""§22 S2 — collection-grained corpus mirror + collection-scoped serve/propose. + +The mirror test drives cache.refresh_meta_repo against an in-memory content repo +holding entries under both the default `rfcs/` and a named collection's +`features/rfcs/`, and asserts cached_rfcs is keyed by the right collection_id.""" +from __future__ import annotations + +import asyncio +import tempfile +from pathlib import Path + +from app import cache, db +from app.config import Config + + +def _db() -> Config: + cfg = Config( + gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="wiggleverse", + registry_repo="registry", oauth_client_id="x", + oauth_client_secret="x", app_url="x", secret_key="x", + database_path=Path(tempfile.mkdtemp(prefix="colserve-")) / "t.db", + owner_gitea_login="x", webhook_secret="x", + ) + db.run_migrations(cfg) + if db._CONN is not None: + db._CONN.close() + db._CONN = None + db.init(cfg) + return cfg + + +class _CorpusGitea: + """A content repo modelled as a flat {path: text} map, listing files under a + directory prefix and reading them back.""" + + def __init__(self, tree: dict[str, str]): + self._tree = tree + + async def list_dir(self, org, repo, path, ref="main"): + out = [] + prefix = (path.rstrip("/") + "/") if path else "" + for p in self._tree: + if p.startswith(prefix) and "/" not in p[len(prefix):]: + out.append({"type": "file", "name": p.split("/")[-1], "path": p}) + return out + + async def read_file(self, org, repo, path, ref="main"): + t = self._tree.get(path) + return (t, "sha-" + path) if t is not None else None + + +def _entry_md(slug, title): + return f"---\nslug: {slug}\ntitle: {title}\nstate: active\n---\nbody\n" + + +def _seed_project_with_two_collections(): + db.conn().execute( + "INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) " + "VALUES ('ohm','Ohm','ohm-rfc','public', datetime('now'))") + for cid, sub in [("default", ""), ("features", "features")]: + db.conn().execute( + "INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, initial_state, " + "visibility, created_at, updated_at) VALUES (?, 'ohm','document',?, " + "'super-draft','public', datetime('now'), datetime('now'))", (cid, sub)) + + +def test_mirror_keys_entries_by_collection(): + cfg = _db() + _seed_project_with_two_collections() + gitea = _CorpusGitea({ + "rfcs/a.md": _entry_md("a", "Default A"), + "features/rfcs/b.md": _entry_md("b", "Feature B"), + }) + asyncio.run(cache.refresh_meta_repo(cfg, gitea)) + got = {(r["collection_id"], r["slug"]) for r in + db.conn().execute("SELECT collection_id, slug FROM cached_rfcs")} + assert got == {("default", "a"), ("features", "b")} -- 2.39.5 From f57d4080dcaf192b37d064b0cbde696d313f6748 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Fri, 5 Jun 2026 13:05:58 -0700 Subject: [PATCH 14/21] =?UTF-8?q?=C2=A722=20S2:=20collection-scoped=20list?= =?UTF-8?q?/get/propose=20endpoints?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refactor the project-scoped serve/propose internals into collection-grained helpers (_list_rfcs_for_collection, _get_rfc_for_collection, _propose_into_collection); add routes under /api/projects//collections//rfcs[/|/propose]. Propose writes the entry under the target collection's /rfcs via a new rfcs_dir param on bot.open_idea_pr. Default-collection routes preserved as wrappers. Co-Authored-By: Claude Opus 4.8 (1M context) --- backend/app/api.py | 97 +++++++++++++++---- backend/app/bot.py | 10 +- backend/tests/test_collection_scoped_serve.py | 64 ++++++++++++ 3 files changed, 149 insertions(+), 22 deletions(-) diff --git a/backend/app/api.py b/backend/app/api.py index 0d5cd2c..b81a5a5 100644 --- a/backend/app/api.py +++ b/backend/app/api.py @@ -717,15 +717,15 @@ def make_router( # second project's corpus renders under /p//. # --------------------------------------------------------------- - @router.get("/api/projects/{project_id}/rfcs") - async def list_project_rfcs( - project_id: str, request: Request, unreviewed: str | None = None + def _require_collection_in_project(collection_id: str, project_id: str) -> None: + # §22 S2: a collection-scoped route 404s when the collection does not + # belong to the project in the path (shape matches an unknown id). + if collections_mod.project_of_collection(collection_id) != project_id: + raise HTTPException(404, "Not found") + + def _list_rfcs_for_collection( + collection_id: str, viewer, unreviewed: str | None ) -> dict[str, Any]: - viewer = auth.current_user(request) - # §22.5 read gate: a gated project's catalog 404s to a non-member. - auth.require_project_readable(viewer, project_id) - # §22 S1: serve the project's default collection (the corpus grain). - collection_id = collections_mod.default_collection_id(project_id) viewer_id = viewer.user_id if viewer else None unreviewed_clause = "" if unreviewed is not None and unreviewed.lower() in ("1", "true", "yes"): @@ -769,11 +769,7 @@ def make_router( ] return {"items": items} - @router.get("/api/projects/{project_id}/rfcs/{slug}") - async def get_project_rfc(project_id: str, slug: str, request: Request) -> dict[str, Any]: - viewer = auth.current_user(request) - auth.require_project_readable(viewer, project_id) - collection_id = collections_mod.default_collection_id(project_id) + def _get_rfc_for_collection(collection_id: str, slug: str, viewer) -> dict[str, Any]: row = db.conn().execute( "SELECT * FROM cached_rfcs WHERE collection_id = ? AND slug = ?", (collection_id, slug), @@ -794,6 +790,46 @@ def make_router( payload["proposed_use_case"] = uc["use_case"] if uc else None return payload + @router.get("/api/projects/{project_id}/rfcs") + async def list_project_rfcs( + project_id: str, request: Request, unreviewed: str | None = None + ) -> dict[str, Any]: + viewer = auth.current_user(request) + # §22.5 read gate: a gated project's catalog 404s to a non-member. + auth.require_project_readable(viewer, project_id) + # §22 S1: the project-scoped route serves the default collection. + collection_id = collections_mod.default_collection_id(project_id) + return _list_rfcs_for_collection(collection_id, viewer, unreviewed) + + @router.get("/api/projects/{project_id}/rfcs/{slug}") + async def get_project_rfc(project_id: str, slug: str, request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + collection_id = collections_mod.default_collection_id(project_id) + return _get_rfc_for_collection(collection_id, slug, viewer) + + # §22 S2: collection-scoped serve + propose. The catalog/entry views read + # these under /p//c//; the project-scoped routes above + # stay as the default-collection compat surface. + @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs") + async def list_collection_rfcs( + project_id: str, collection_id: str, request: Request, + unreviewed: str | None = None, + ) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + _require_collection_in_project(collection_id, project_id) + return _list_rfcs_for_collection(collection_id, viewer, unreviewed) + + @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}") + async def get_collection_rfc( + project_id: str, collection_id: str, slug: str, request: Request + ) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + _require_collection_in_project(collection_id, project_id) + return _get_rfc_for_collection(collection_id, slug, viewer) + # --------------------------------------------------------------- # §22.4c: mark-reviewed — clear an active entry's `unreviewed` flag # --------------------------------------------------------------- @@ -957,6 +993,15 @@ def make_router( # --------------------------------------------------------------- async def _propose_into_project(project_id: str, payload: ProposeBody, user) -> dict[str, Any]: + # Default-collection wrapper (§22 S1/S2): resolve the project's default + # collection and delegate. Keeps the project-scoped propose routes intact. + return await _propose_into_collection( + project_id, collections_mod.default_collection_id(project_id), payload, user + ) + + async def _propose_into_collection( + project_id: str, collection_id: str, payload: ProposeBody, user + ) -> dict[str, Any]: # §22.6/§22.7: proposing a new entry requires project-level contribute # standing in the *target* project. On the public default project the # implicit-public baseline preserves the pre-multi-project flow. @@ -970,7 +1015,6 @@ def make_router( # We re-check atomically here even though the client also checks # on every keystroke, since a concurrent submission could land # between dialog-open and submit. - collection_id = collections_mod.default_collection_id(project_id) clash = db.conn().execute( "SELECT 1 FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id) ).fetchone() @@ -984,11 +1028,12 @@ def make_router( if idea_clash: raise HTTPException(409, f"Slug `{slug}` is already reserved by an open proposal") - # §22.4b: the target project's landing state. Through Plan A every - # entry lands in the default project; M3-frontend routing carries a - # non-default target later. - target_project = project_id - landing_state = "active" if projects_mod.project_initial_state(target_project) == "active" else "super-draft" + # §22.4b: the target collection's landing state (the per-corpus field + # moved down to the collection in migration 029). + landing_state = ( + "active" if collections_mod.collection_initial_state(collection_id) == "active" + else "super-draft" + ) entry = entry_mod.Entry( slug=slug, @@ -1019,6 +1064,9 @@ def make_router( f"**Topic:** {entry.title}\n\n" f"{payload.pitch.strip()}" ) + # §22 S2: write the entry under the target collection's /rfcs. + subfolder = collections_mod.subfolder_of(collection_id) + rfcs_dir = f"{subfolder}/rfcs" if subfolder else "rfcs" try: pr = await bot.open_idea_pr( user.as_actor(), @@ -1028,6 +1076,7 @@ def make_router( file_contents=contents, pr_title=pr_title, pr_description=pr_description, + rfcs_dir=rfcs_dir, ) except GiteaError as e: raise HTTPException(502, f"Gitea: {e.detail}") @@ -1076,6 +1125,16 @@ def make_router( auth.require_project_readable(user, project_id) return await _propose_into_project(project_id, payload, user) + @router.post("/api/projects/{project_id}/collections/{collection_id}/rfcs/propose") + async def propose_collection_rfc( + project_id: str, collection_id: str, payload: ProposeBody, request: Request + ) -> dict[str, Any]: + # §22 S2: propose a new entry into a specific collection of a project. + user = auth.require_contributor(request) + auth.require_project_readable(user, project_id) + _require_collection_in_project(collection_id, project_id) + return await _propose_into_collection(project_id, collection_id, payload, user) + # --------------------------------------------------------------- # §9.1 Slice 2 (roadmap #27): Claude Haiku tag suggestions as the # propose-RFC fields fill in. The modal debounce-posts the partial diff --git a/backend/app/bot.py b/backend/app/bot.py index f6ea9c3..ce8fbd8 100644 --- a/backend/app/bot.py +++ b/backend/app/bot.py @@ -175,12 +175,16 @@ class Bot: file_contents: str, pr_title: str, pr_description: str, + rfcs_dir: str = "rfcs", ) -> dict: - """Per §9.1: open a meta-repo PR adding one file under rfcs/. + """Per §9.1: open a meta-repo PR adding one file under `/`. One file per PR keeps idea submissions atomic and conflict-free. The PR title and the file-add commit subject share §9.2's fixed - pattern; callers compose `pr_title` as `Propose: `. + pattern; callers compose `pr_title` as `Propose: <Title>`. §22 S2: + `rfcs_dir` carries the target collection's `<subfolder>/rfcs` so a + propose into a named collection writes under its subfolder; it + defaults to `rfcs` (the default collection / shipped behaviour). """ branch = f"propose/{slug}" await self._gitea.create_branch(org, meta_repo, branch, from_branch="main") @@ -189,7 +193,7 @@ class Bot: created = await self._gitea.create_file( org, meta_repo, - f"rfcs/{slug}.md", + f"{rfcs_dir}/{slug}.md", content=file_contents, message=commit_message, branch=branch, diff --git a/backend/tests/test_collection_scoped_serve.py b/backend/tests/test_collection_scoped_serve.py index f3d1c27..cb1ab43 100644 --- a/backend/tests/test_collection_scoped_serve.py +++ b/backend/tests/test_collection_scoped_serve.py @@ -75,3 +75,67 @@ def test_mirror_keys_entries_by_collection(): got = {(r["collection_id"], r["slug"]) for r in db.conn().execute("SELECT collection_id, slug FROM cached_rfcs")} assert got == {("default", "a"), ("features", "b")} + + +# --- collection-scoped serve + propose (full app) ----------------------------- + +from fastapi.testclient import TestClient # noqa: E402 +from test_propose_vertical import ( # noqa: E402,F401 + app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as, +) + + +def _add_features_collection(content_repo="meta"): + """Add a named 'features' collection (subfolder 'features') under the seeded + default project, plus a single entry under features/rfcs/ in the db cache.""" + db.conn().execute( + "INSERT OR REPLACE INTO collections (id, project_id, type, subfolder, " + "initial_state, visibility, name, created_at, updated_at) VALUES " + "('features','default','document','features','super-draft','public','Features', " + "datetime('now'), datetime('now'))") + + +def test_scoped_list_returns_only_that_collection(app_with_fake_gitea): + app, fake = app_with_fake_gitea + with TestClient(app) as client: + _add_features_collection() + # Seed one entry under each collection's rfcs dir + mirror them in. + fake.files[("wiggleverse", "meta", "main", "rfcs/a.md")] = { + "content": _entry_md("a", "Default A"), "sha": "sa"} + fake.files[("wiggleverse", "meta", "main", "features/rfcs/b.md")] = { + "content": _entry_md("b", "Feature B"), "sha": "sb"} + from app import cache as cache_mod, gitea as gitea_mod + from app.config import load_config + cfg = load_config() + asyncio.run(cache_mod.refresh_meta_repo(cfg, gitea_mod.Gitea(cfg))) + + r = client.get("/api/projects/default/collections/features/rfcs") + assert r.status_code == 200, r.text + assert [i["slug"] for i in r.json()["items"]] == ["b"] + # The default collection still serves only its own entry. + r2 = client.get("/api/projects/default/collections/default/rfcs") + assert [i["slug"] for i in r2.json()["items"]] == ["a"] + + +def test_scoped_propose_writes_into_collection_subfolder(app_with_fake_gitea): + app, fake = app_with_fake_gitea + with TestClient(app) as client: + _add_features_collection() + provision_user_row(user_id=3, login="alice", role="contributor") + sign_in_as(client, user_id=3, gitea_login="alice", display_name="Alice", + role="contributor", email="alice@test") + r = client.post( + "/api/projects/default/collections/features/rfcs/propose", + json={"title": "New B", "slug": "newb", "pitch": "x", "tags": []}) + assert r.status_code == 200, r.text + # The bot wrote the entry under features/rfcs/, not rfcs/. + keys = {(k[1], k[3]) for k in fake.files + if k[1] == "meta" and k[3].endswith("newb.md")} + assert ("meta", "features/rfcs/newb.md") in keys + + +def test_scoped_routes_404_for_collection_outside_project(app_with_fake_gitea): + app, _ = app_with_fake_gitea + with TestClient(app) as client: + r = client.get("/api/projects/default/collections/nope/rfcs") + assert r.status_code == 404 -- 2.39.5 From 0c654b173d7ebd927155eabf3a6a1cde08cc52c0 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:10:09 -0700 Subject: [PATCH 15/21] =?UTF-8?q?=C2=A722=20S2:=20create-collection=20endp?= =?UTF-8?q?oint=20(bot=20commit=20+=20registry=20refresh)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POST /api/projects/<id>/collections, owner/admin-gated, commits a .collection.yaml to the content repo main via bot.create_collection, then re-mirrors the registry so the collections row appears (§22.2). Adds GET list/one collection routes. Extends FakeGitea to model directory listings so the mirror's content-repo walk is exercised end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- backend/app/api.py | 2 + backend/app/api_collections.py | 112 ++++++++++++++++++ backend/app/bot.py | 35 ++++++ .../tests/test_collection_create_vertical.py | 83 +++++++++++++ backend/tests/test_propose_vertical.py | 45 +++++-- 5 files changed, 266 insertions(+), 11 deletions(-) create mode 100644 backend/app/api_collections.py create mode 100644 backend/tests/test_collection_create_vertical.py diff --git a/backend/app/api.py b/backend/app/api.py index b81a5a5..55eb9fc 100644 --- a/backend/app/api.py +++ b/backend/app/api.py @@ -21,6 +21,7 @@ from pydantic import BaseModel, Field from . import ( api_admin, api_branches, + api_collections, api_contributions, api_deployment, api_discussion, @@ -152,6 +153,7 @@ def make_router( # §22.9/§22.10 (M3): runtime deployment + per-project config (replaces # VITE_APP_NAME) + the old-URL 308 redirects. router.include_router(api_deployment.make_router(config)) + router.include_router(api_collections.make_router(config, gitea, bot)) # --------------------------------------------------------------- # §17: /api/health — unauthenticated post-flight probe. diff --git a/backend/app/api_collections.py b/backend/app/api_collections.py new file mode 100644 index 0000000..c80866a --- /dev/null +++ b/backend/app/api_collections.py @@ -0,0 +1,112 @@ +"""§22 S2 — collection directory + create-collection. + +GET /api/projects/:id/collections — list the project's visible collections. +GET /api/projects/:id/collections/:cid — one collection's settings. +POST /api/projects/:id/collections — create a collection. Authorized by a + deployment owner/admin (S2; scoped + {owner, contributor} roles at the + collection axis land in S3). The bot + commits a `.collection.yaml` to the + content repo, then the registry mirror + upserts the collections row — §22.2 + keeps the registry the source of truth. +""" +from __future__ import annotations + +import re +from typing import Any + +import yaml +from fastapi import APIRouter, HTTPException, Request +from pydantic import BaseModel + +from . import ( + auth, + collections as collections_mod, + projects as projects_mod, + registry as registry_mod, +) +from .bot import Bot +from .config import Config +from .gitea import Gitea, GiteaError + +_SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$") + + +class CreateCollectionBody(BaseModel): + collection_id: str + type: str + name: str | None = None + visibility: str | None = None + initial_state: str | None = None + + +def make_router(config: Config, gitea: Gitea, bot: Bot) -> APIRouter: + router = APIRouter() + + @router.get("/api/projects/{project_id}/collections") + async def list_cols(project_id: str, request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + # §22.5 read gate: a gated project 404s a non-member. + auth.require_project_readable(viewer, project_id) + return {"items": collections_mod.list_collections(project_id)} + + @router.get("/api/projects/{project_id}/collections/{collection_id}") + async def get_col(project_id: str, collection_id: str, request: Request) -> dict[str, Any]: + viewer = auth.current_user(request) + auth.require_project_readable(viewer, project_id) + col = collections_mod.get_collection(collection_id) + if col is None or col["project_id"] != project_id: + raise HTTPException(404, "Not found") + return col + + @router.post("/api/projects/{project_id}/collections") + async def create_col( + project_id: str, body: CreateCollectionBody, request: Request + ) -> dict[str, Any]: + # S2 authority: deployment owner/admin (the scoped-role surface is S3). + user = auth.require_admin(request) + auth.require_project_readable(user, project_id) + cid = body.collection_id.strip().lower() + if not _SLUG_RE.match(cid) or cid == "default": + raise HTTPException(422, "collection id must be a slug and not 'default'") + if body.type not in registry_mod.VALID_TYPES: + raise HTTPException(422, f"invalid type {body.type!r}") + if body.visibility is not None and body.visibility not in registry_mod.VALID_VISIBILITY: + raise HTTPException(422, f"invalid visibility {body.visibility!r}") + if body.initial_state is not None and body.initial_state not in registry_mod.VALID_INITIAL_STATE: + raise HTTPException(422, f"invalid initial_state {body.initial_state!r}") + if collections_mod.get_collection(cid) is not None: + raise HTTPException(409, f"collection `{cid}` already exists") + content_repo = projects_mod.content_repo(project_id) + if not content_repo: + raise HTTPException(409, "project has no content repo") + + manifest: dict[str, Any] = {"type": body.type} + if body.name: + manifest["name"] = body.name + if body.visibility: + manifest["visibility"] = body.visibility + if body.initial_state: + manifest["initial_state"] = body.initial_state + manifest_yaml = yaml.safe_dump(manifest, sort_keys=False) + + try: + await bot.create_collection( + user.as_actor(), + org=config.gitea_org, + content_repo=content_repo, + collection_id=cid, + manifest_yaml=manifest_yaml, + ) + except GiteaError as e: + raise HTTPException(502, f"Gitea: {e.detail}") + + # §22.2: re-read the registry so the new manifest becomes a row. + await registry_mod.refresh_registry(config, gitea) + col = collections_mod.get_collection(cid) + if col is None: + raise HTTPException(500, "collection committed but not mirrored") + return col + + return router diff --git a/backend/app/bot.py b/backend/app/bot.py index ce8fbd8..f985f23 100644 --- a/backend/app/bot.py +++ b/backend/app/bot.py @@ -163,6 +163,41 @@ class Bot: def __init__(self, gitea: Gitea): self._gitea = gitea + # ----- Content repo: collection structure (§22 S2) ----- + + async def create_collection( + self, + actor: Actor, + *, + org: str, + content_repo: str, + collection_id: str, + manifest_yaml: str, + ) -> dict: + """§22 S2: commit `<collection_id>/.collection.yaml` to the content + repo's main. A structural admin action — committed straight to main (no + PR), like the registry config it feeds; the registry mirror then upserts + the collections row (§22.2 keeps the registry the source of truth). Logs + an audit row for the §6.5 trail.""" + path = f"{collection_id}/.collection.yaml" + created = await self._gitea.create_file( + org, + content_repo, + path, + content=manifest_yaml, + message=_stamp_single(f"chore: create collection {collection_id}", actor), + branch="main", + author_name=actor.display_name, + author_email=actor.email or f"{actor.gitea_login}@users.noreply", + ) + _log( + actor, + "create_collection", + bot_commit_sha=created.get("commit", {}).get("sha"), + details={"collection_id": collection_id, "repo": content_repo}, + ) + return created + # ----- Meta repo: idea PRs (§9.1 / §9.2) ----- async def open_idea_pr( diff --git a/backend/tests/test_collection_create_vertical.py b/backend/tests/test_collection_create_vertical.py new file mode 100644 index 0000000..c19a978 --- /dev/null +++ b/backend/tests/test_collection_create_vertical.py @@ -0,0 +1,83 @@ +"""§22 S2 — create-collection vertical: a deployment owner/admin POSTs, the bot +commits a `.collection.yaml`, and the registry mirror upserts the collections +row (registry stays the source of truth).""" +from __future__ import annotations + +from fastapi.testclient import TestClient + +from app import db +from test_propose_vertical import ( # noqa: F401 + app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as, +) + + +def test_create_collection_commits_manifest_and_mirrors(app_with_fake_gitea): + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", + role="owner", email="ben@test") + r = client.post("/api/projects/default/collections", + json={"collection_id": "features", "type": "bdd", "name": "Features"}) + assert r.status_code == 200, r.text + assert r.json()["type"] == "bdd" + + # The bot committed the manifest to the content repo's main. + f = fake.files.get(("wiggleverse", "meta", "main", "features/.collection.yaml")) + assert f is not None + assert "type: bdd" in f["content"] + + # The registry refresh mirrored it into a collections row. + row = db.conn().execute( + "SELECT type, project_id, subfolder FROM collections WHERE id='features'" + ).fetchone() + assert (row["type"], row["project_id"], row["subfolder"]) == ("bdd", "default", "features") + + # It is now navigable via the directory + scoped serve. + items = client.get("/api/projects/default/collections").json()["items"] + assert any(c["id"] == "features" for c in items) + assert client.get("/api/projects/default/collections/features/rfcs").status_code == 200 + + +def test_create_collection_requires_admin(app_with_fake_gitea): + app, _ = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", + role="contributor", email="alice@test") + r = client.post("/api/projects/default/collections", + json={"collection_id": "x", "type": "bdd"}) + assert r.status_code in (401, 403) + + +def test_create_collection_anonymous_rejected(app_with_fake_gitea): + app, _ = app_with_fake_gitea + with TestClient(app) as client: + r = client.post("/api/projects/default/collections", + json={"collection_id": "x", "type": "bdd"}) + assert r.status_code in (401, 403) + + +def test_create_collection_rejects_duplicate(app_with_fake_gitea): + app, _ = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", + role="owner", email="ben@test") + ok = client.post("/api/projects/default/collections", + json={"collection_id": "features", "type": "bdd"}) + assert ok.status_code == 200, ok.text + dup = client.post("/api/projects/default/collections", + json={"collection_id": "features", "type": "bdd"}) + assert dup.status_code == 409 + + +def test_create_collection_rejects_reserved_default_id(app_with_fake_gitea): + app, _ = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", + role="owner", email="ben@test") + r = client.post("/api/projects/default/collections", + json={"collection_id": "default", "type": "bdd"}) + assert r.status_code == 422 diff --git a/backend/tests/test_propose_vertical.py b/backend/tests/test_propose_vertical.py index 6045619..f4bc22c 100644 --- a/backend/tests/test_propose_vertical.py +++ b/backend/tests/test_propose_vertical.py @@ -90,6 +90,28 @@ class FakeGitea: self._commit_counter += 1 return f"sha{self._commit_counter:04d}" + def _dir_listing(self, owner, repo, ref, dirpath): + """Children directly under `dirpath` on (owner, repo, ref): files as + `type: file` and immediate subdirectories as `type: dir` (the shape real + Gitea returns for a contents listing).""" + prefix = (dirpath.rstrip("/") + "/") if dirpath else "" + files: dict[str, dict] = {} + dirs: set[str] = set() + for (o, r, br, p), data in self.files.items(): + if (o, r, br) != (owner, repo, ref) or not p.startswith(prefix): + continue + rest = p[len(prefix):] + if "/" in rest: + dirs.add(rest.split("/", 1)[0]) + elif rest: + files[p] = data + children = [{"name": n, "path": prefix + n, "type": "dir"} for n in sorted(dirs)] + children += [ + {"name": p.rsplit("/", 1)[-1], "path": p, "type": "file", "sha": d["sha"]} + for p, d in sorted(files.items()) + ] + return children + def _enrich_pr(self, owner: str, repo: str, pr: dict) -> dict: """Return the PR with mergeability fields filled in. @@ -227,6 +249,16 @@ class FakeGitea: } return httpx.Response(201, json={"name": new}) + # GET /repos/{owner}/{repo}/contents (root listing, empty path). §22 S2: + # the registry mirror walks the content-repo root for collection + # subfolders, so the simulator models a root directory listing that + # surfaces both file and `dir` children. + m_root = re.fullmatch(r"/repos/([^/]+)/([^/]+)/contents/?", path) + if method == "GET" and m_root: + owner, repo = m_root.groups() + ref = request.url.params.get("ref", "main") + return httpx.Response(200, json=self._dir_listing(owner, repo, ref, "")) + # GET /repos/{owner}/{repo}/contents/{path}?ref=... m = re.fullmatch(r"/repos/([^/]+)/([^/]+)/contents/(.+)", path) if method == "GET" and m: @@ -242,17 +274,8 @@ class FakeGitea: "sha": f["sha"], "content": base64.b64encode(f["content"].encode()).decode(), }) - # Directory listing - prefix = fpath.rstrip("/") + "/" - children = [] - for (o, r, br, p), data in self.files.items(): - if (o, r, br) == (owner, repo, ref) and p.startswith(prefix) and "/" not in p[len(prefix):]: - children.append({ - "name": p.rsplit("/", 1)[-1], - "path": p, - "type": "file", - "sha": data["sha"], - }) + # Directory listing — both file and subdir children. + children = self._dir_listing(owner, repo, ref, fpath) if children: return httpx.Response(200, json=children) return httpx.Response(404, json={"message": "not found"}) -- 2.39.5 From 98eea3e2d6c1b097a76329d0eab93cb338c42e76 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:11:27 -0700 Subject: [PATCH 16/21] =?UTF-8?q?=C2=A722=20S2:=20collection-scoped=20fron?= =?UTF-8?q?tend=20path=20+=20API=20helpers?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit useCollectionId() hook; listRFCs/getRFC/proposeRFC take an optional collection id and target the /collections/<cid>/ routes; add listCollections + createCollection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- frontend/src/api.collections.test.js | 48 ++++++++++++++++++++++++++++ frontend/src/api.js | 44 +++++++++++++++++++++---- frontend/src/lib/entryPaths.js | 8 +++++ 3 files changed, 93 insertions(+), 7 deletions(-) create mode 100644 frontend/src/api.collections.test.js diff --git a/frontend/src/api.collections.test.js b/frontend/src/api.collections.test.js new file mode 100644 index 0000000..f446a8c --- /dev/null +++ b/frontend/src/api.collections.test.js @@ -0,0 +1,48 @@ +// §22 S2 — the API client builds collection-scoped URLs when a collection id is +// supplied, and falls back to the project/default-collection paths otherwise. +import { describe, it, expect, vi, afterEach } from 'vitest' +import { listRFCs, getRFC, proposeRFC, listCollections } from './api.js' + +function mockFetch() { + const fn = vi.fn(async () => ({ + ok: true, + status: 200, + json: async () => ({ items: [] }), + })) + global.fetch = fn + return fn +} + +afterEach(() => { vi.restoreAllMocks() }) + +describe('collection-scoped api URLs', () => { + it('listRFCs scopes to a collection when given one', async () => { + const f = mockFetch() + await listRFCs('ohm', 'features') + expect(f).toHaveBeenCalledWith('/api/projects/ohm/collections/features/rfcs') + }) + + it('listRFCs falls back to the project default path without a collection', async () => { + const f = mockFetch() + await listRFCs('ohm') + expect(f).toHaveBeenCalledWith('/api/projects/ohm/rfcs') + }) + + it('getRFC scopes to a collection when given one', async () => { + const f = mockFetch() + await getRFC('ohm', 'login', 'features') + expect(f).toHaveBeenCalledWith('/api/projects/ohm/collections/features/rfcs/login') + }) + + it('proposeRFC targets the collection-scoped propose route', async () => { + const f = mockFetch() + await proposeRFC('ohm', { title: 'T', slug: 's', pitch: 'p', tags: [], collectionId: 'features' }) + expect(f.mock.calls[0][0]).toBe('/api/projects/ohm/collections/features/rfcs/propose') + }) + + it('listCollections hits the project collections route', async () => { + const f = mockFetch() + await listCollections('ohm') + expect(f).toHaveBeenCalledWith('/api/projects/ohm/collections') + }) +}) diff --git a/frontend/src/api.js b/frontend/src/api.js index 35978ca..d64efd8 100644 --- a/frontend/src/api.js +++ b/frontend/src/api.js @@ -182,22 +182,50 @@ export async function getProject(projectId) { return jsonOrThrow(await fetch(`/api/projects/${projectId}`)) } -// §22.4 (Plan B): per-project serving. Given a projectId, read the -// project-scoped routes so a non-default project's corpus renders; without -// one, fall back to the default-project compat path. -export async function listRFCs(projectId) { +// §22.4 (Plan B) / §22 S2: per-collection serving. With a projectId + a +// collectionId, read the collection-scoped routes; with only a projectId, the +// project default-collection compat path; with neither, the unscoped path. +export async function listRFCs(projectId, collectionId) { + if (projectId && collectionId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs`)) + } const url = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs' return jsonOrThrow(await fetch(url)) } -export async function getRFC(projectId, slug) { +export async function getRFC(projectId, slug, collectionId) { // Back-compat: getRFC(slug) (one arg) still hits the unscoped default path. if (slug === undefined) { return jsonOrThrow(await fetch(`/api/rfcs/${projectId}`)) } + if (collectionId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}`)) + } return jsonOrThrow(await fetch(`/api/projects/${projectId}/rfcs/${slug}`)) } +// §22 S2: the collections of a project (for the /p/<project>/ directory). +export async function listCollections(projectId) { + return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections`)) +} + +// §22 S2: create-collection (deployment owner/admin). The backend commits a +// .collection.yaml and re-mirrors the registry, returning the new collection. +export async function createCollection(projectId, { collectionId, type, name, visibility, initialState }) { + const res = await fetch(`/api/projects/${projectId}/collections`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + collection_id: collectionId, + type, + name: name || null, + visibility: visibility || null, + initial_state: initialState || null, + }), + }) + return jsonOrThrow(res) +} + export async function listProposals(projectId) { const url = projectId ? `/api/projects/${projectId}/proposals` : '/api/proposals' return jsonOrThrow(await fetch(url)) @@ -209,8 +237,10 @@ export async function getProposal(prNumber) { // §22.4 (Plan B write): propose into a specific project when projectId is // given; else the default-project compat path. -export async function proposeRFC(projectId, { title, slug, pitch, tags, proposedUseCase }) { - const url = projectId ? `/api/projects/${projectId}/rfcs/propose` : '/api/rfcs/propose' +export async function proposeRFC(projectId, { title, slug, pitch, tags, proposedUseCase, collectionId }) { + const url = (projectId && collectionId) + ? `/api/projects/${projectId}/collections/${collectionId}/rfcs/propose` + : (projectId ? `/api/projects/${projectId}/rfcs/propose` : '/api/rfcs/propose') const res = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, diff --git a/frontend/src/lib/entryPaths.js b/frontend/src/lib/entryPaths.js index 15c5f53..b780f5a 100644 --- a/frontend/src/lib/entryPaths.js +++ b/frontend/src/lib/entryPaths.js @@ -4,6 +4,7 @@ // project-scoped, so the builders emit the default collection segment; the // collection-aware link layer (named collections) lands in S2. Components build // links via these helpers so that flip happens in one place. +import { useParams } from 'react-router-dom' import { useProject } from '../components/ProjectLayout.jsx' import { useDeployment } from '../context/DeploymentProvider' @@ -37,3 +38,10 @@ export function useProjectId() { const { defaultProjectId } = useDeployment() return (ctx && ctx.projectId) || defaultProjectId } + +// §22 S2 — the collection id a component should scope to: the `/c/:collectionId/` +// route segment when present, else the project's default collection. +export function useCollectionId() { + const { collectionId } = useParams() + return collectionId || DEFAULT_COLLECTION +} -- 2.39.5 From 2b32e124abe099b3535c6eddfb6488a8f8c44da3 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:12:45 -0700 Subject: [PATCH 17/21] =?UTF-8?q?=C2=A722=20S2:=20Catalog=20+=20propose=20?= =?UTF-8?q?scoped=20to=20the=20active=20collection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Catalog reads the /c/:collectionId/ segment via useCollectionId and fetches the collection-scoped catalog, building entry/proposal links with the active collection; the propose modal threads the active collection so a propose from a named collection targets it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- frontend/src/App.jsx | 7 ++++++- frontend/src/components/Catalog.jsx | 13 ++++++++----- frontend/src/components/ProposeModal.jsx | 3 ++- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index e7453cf..c1db389 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -66,6 +66,10 @@ export default function App() { // right project. Falls back to the deployment default off a project route. const _projMatch = location.pathname.match(/^\/p\/([^/]+)/) const currentProjectId = (_projMatch && _projMatch[1]) || deployment.defaultProjectId + // §22 S2 — the collection the viewer is currently in (from the /c/<cid>/ URL + // segment), so a propose targets that collection. Falls back to the default. + const _colMatch = location.pathname.match(/^\/p\/[^/]+\/c\/([^/]+)/) + const currentCollectionId = (_colMatch && _colMatch[1]) || DEFAULT_COLLECTION // #28 Parts 2–3: the LinkedText create/contribute affordances route via // query params so they need no prop-threading from deep in a comment // list. `?propose=<term>` opens the propose modal pre-filled; @@ -389,12 +393,13 @@ export default function App() { viewer={viewer} initialTitle={proposeParam || ''} projectId={currentProjectId} + collectionId={currentCollectionId} onClose={() => { setProposeOpen(false); clearParams('propose') }} onSubmitted={({ pr_number }) => { setProposeOpen(false) clearParams('propose') setCatalogVersion(v => v + 1) - navigate(proposalPath(currentProjectId, pr_number)) + navigate(proposalPath(currentProjectId, pr_number, currentCollectionId)) }} /> )} diff --git a/frontend/src/components/Catalog.jsx b/frontend/src/components/Catalog.jsx index 24bc470..607ae06 100644 --- a/frontend/src/components/Catalog.jsx +++ b/frontend/src/components/Catalog.jsx @@ -9,7 +9,7 @@ import { useEffect, useMemo, useState } from 'react' import { useParams, Link } from 'react-router-dom' import { listRFCs, listProposals } from '../api' -import { entryPath, proposalPath, useProjectId } from '../lib/entryPaths' +import { entryPath, proposalPath, useProjectId, useCollectionId } from '../lib/entryPaths' const STATE_CHIPS = [ { id: 'super-draft', label: 'Super-draft' }, @@ -32,11 +32,14 @@ export default function Catalog({ viewer, onProposeRFC, version }) { const [pendingOpen, setPendingOpen] = useState(true) const { slug, prNumber } = useParams() const pid = useProjectId() + // §22 S2: the catalog is scoped to the active collection (the `/c/:cid/` + // route segment, else the project's default collection). + const cid = useCollectionId() useEffect(() => { - listRFCs(pid).then(d => setRfcs(d.items)).catch(() => setRfcs([])) + listRFCs(pid, cid).then(d => setRfcs(d.items)).catch(() => setRfcs([])) listProposals(pid).then(d => setProposals(d.items)).catch(() => setProposals([])) - }, [version, pid]) + }, [version, pid, cid]) const filtered = useMemo(() => { const needle = search.trim().toLowerCase() @@ -105,7 +108,7 @@ export default function Catalog({ viewer, onProposeRFC, version }) { return ( <Link key={r.slug} - to={entryPath(pid, r.slug)} + to={entryPath(pid, r.slug, cid)} className={`catalog-row ${isActive ? 'active' : ''} ${isSuper ? 'is-super' : ''}`} > <div className="row-top"> @@ -133,7 +136,7 @@ export default function Catalog({ viewer, onProposeRFC, version }) { {proposals.map(p => ( <Link key={p.pr_number} - to={proposalPath(pid, p.pr_number)} + to={proposalPath(pid, p.pr_number, cid)} className={`pending-row ${String(prNumber) === String(p.pr_number) ? 'active' : ''}`} > <div>{p.title.replace(/^Propose:\s*/, '')}</div> diff --git a/frontend/src/components/ProposeModal.jsx b/frontend/src/components/ProposeModal.jsx index 0989a89..ef7a110 100644 --- a/frontend/src/components/ProposeModal.jsx +++ b/frontend/src/components/ProposeModal.jsx @@ -28,7 +28,7 @@ function slugify(title) { .replace(/^-+|-+$/g, '') } -export default function ProposeModal({ viewer, onClose, onSubmitted, initialTitle = '', projectId }) { +export default function ProposeModal({ viewer, onClose, onSubmitted, initialTitle = '', projectId, collectionId }) { // #28 Part 2: a "create RFC for '<term>'" affordance pre-fills the title // (App passes the `?propose=<term>` value here); the slug derives from it // via the same effect that drives manual typing. @@ -98,6 +98,7 @@ export default function ProposeModal({ viewer, onClose, onSubmitted, initialTitl pitch: pitch.trim(), tags, proposedUseCase: useCase.trim() || null, + collectionId, }) // v0.15.0 — analytics: fire on the §9.1 propose-RFC submit. // Slug is a stable, low-cardinality identifier (kebab-case -- 2.39.5 From 17bdd5fd9a8aaa012938e9b4f392af22840afb61 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:14:18 -0700 Subject: [PATCH 18/21] =?UTF-8?q?=C2=A722=20S2:=20collection=20directory?= =?UTF-8?q?=20at=20/p/<project>/=20(1=20=E2=86=92=20redirect,=202+=20?= =?UTF-8?q?=E2=86=92=20list)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace DefaultCollectionRedirect with a CollectionDirectory that lists the project's visible collections, or redirects into the sole one when there is exactly one (preserving the S1 C3.7/C3.8 single-collection UX). The create-first-collection empty state is S4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- frontend/src/App.jsx | 18 ++++--- .../src/components/CollectionDirectory.jsx | 51 +++++++++++++++++++ .../components/CollectionDirectory.test.jsx | 48 +++++++++++++++++ 3 files changed, 109 insertions(+), 8 deletions(-) create mode 100644 frontend/src/components/CollectionDirectory.jsx create mode 100644 frontend/src/components/CollectionDirectory.test.jsx diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index c1db389..6bdf11b 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -15,6 +15,7 @@ import RFCView from './components/RFCView.jsx' import PRView from './components/PRView.jsx' import ProposalView from './components/ProposalView.jsx' import ProposeModal from './components/ProposeModal.jsx' +import CollectionDirectory from './components/CollectionDirectory.jsx' import ContributeRequestForm from './components/ContributeRequestForm.jsx' import Landing from './components/Landing.jsx' import Login from './components/Login.jsx' @@ -364,9 +365,10 @@ export default function App() { /> <main className="main-pane"> <Routes> - {/* §22 three-tier (C3.7): the project landing redirects into - its sole/default collection (S1: the `default` one). */} - <Route path="" element={<DefaultCollectionRedirect />} /> + {/* §22 S2: the project landing is the collection directory — + it lists collections, or (C3.7/C3.8) redirects into the + sole visible collection when there is exactly one. */} + <Route path="" element={<CollectionDirectoryRoute />} /> {/* Backcompat: the shipped v0.35.0 corpus URLs without a /c/<collection>/ segment redirect into the default collection, so old bookmarks keep working. */} @@ -419,12 +421,12 @@ export default function App() { ) } -// §22 three-tier — corpus redirects mounted under /p/:projectId/*. -// C3.7: the project landing skips the (single-collection) directory and lands in -// the project's default collection. -function DefaultCollectionRedirect() { +// §22 S2 — the project landing at /p/:projectId/ is the collection directory. +// A tiny wrapper reads the route's projectId and hands it to CollectionDirectory +// (which lists collections, or redirects into the sole one — C3.7/C3.8). +function CollectionDirectoryRoute() { const { projectId } = useParams() - return <Navigate to={`/p/${projectId}/c/${DEFAULT_COLLECTION}/`} replace /> + return <CollectionDirectory projectId={projectId} /> } // Backcompat for the shipped v0.35.0 corpus URLs that lacked the diff --git a/frontend/src/components/CollectionDirectory.jsx b/frontend/src/components/CollectionDirectory.jsx new file mode 100644 index 0000000..1b8b762 --- /dev/null +++ b/frontend/src/components/CollectionDirectory.jsx @@ -0,0 +1,51 @@ +// §22 S2 — the project collection directory at `/p/<project>/`. Lists the +// project's caller-visible collections as cards linking into each collection's +// `/p/<project>/c/<collection>/` home. When exactly one collection is visible +// the directory is skipped and we redirect straight into it (the S1 C3.7/C3.8 +// single-collection UX, preserved). The role-keyed "Create your first +// collection" empty state is S4; S2 shows a minimal note when there are none. +import { useEffect, useState } from 'react' +import { Link, Navigate } from 'react-router-dom' +import { listCollections } from '../api' +import { collectionHome } from '../lib/entryPaths' +import { entryNoun } from './ProjectLayout.jsx' + +export default function CollectionDirectory({ projectId }) { + const [cols, setCols] = useState(null) + useEffect(() => { + let live = true + listCollections(projectId) + .then(d => { if (live) setCols(d.items) }) + .catch(() => { if (live) setCols([]) }) + return () => { live = false } + }, [projectId]) + + if (cols === null) { + return <main className="chrome-pane"><div className="boot">Loading…</div></main> + } + // C3.7/C3.8: a single visible collection skips the directory. + if (cols.length === 1) { + return <Navigate to={collectionHome(projectId, cols[0].id)} replace /> + } + return ( + <main className="chrome-pane"> + <div className="directory"> + <h1>Collections</h1> + {cols.length === 0 ? ( + <p className="directory-tagline">No collections yet.</p> + ) : ( + <ul className="directory-list"> + {cols.map(c => ( + <li key={c.id} className="directory-card"> + <Link to={collectionHome(projectId, c.id)}> + <span className="directory-card-name">{c.name || c.id}</span> + <span className="directory-card-type">{entryNoun(c.type)}s</span> + </Link> + </li> + ))} + </ul> + )} + </div> + </main> + ) +} diff --git a/frontend/src/components/CollectionDirectory.test.jsx b/frontend/src/components/CollectionDirectory.test.jsx new file mode 100644 index 0000000..ccc10b2 --- /dev/null +++ b/frontend/src/components/CollectionDirectory.test.jsx @@ -0,0 +1,48 @@ +import React from 'react' +import { describe, it, expect, vi, beforeEach } from 'vitest' +import { render, screen, waitFor } from '@testing-library/react' +import { MemoryRouter, Routes, Route } from 'react-router-dom' + +let mockItems = [] +vi.mock('../api', () => ({ + listCollections: vi.fn(async () => ({ items: mockItems })), +})) +import CollectionDirectory from './CollectionDirectory.jsx' + +beforeEach(() => { mockItems = [] }) + +function renderDir(items) { + mockItems = items + return render( + <MemoryRouter initialEntries={["/p/ohm/"]}> + <Routes> + <Route path="/p/:projectId/*" element={<CollectionDirectory projectId="ohm" />} /> + <Route path="/p/:projectId/c/:collectionId/*" element={<div>collection home</div>} /> + </Routes> + </MemoryRouter>, + ) +} + +describe('CollectionDirectory', () => { + it('lists a card per collection with the type-driven noun + link when 2+', async () => { + renderDir([ + { id: 'default', name: 'Model', type: 'document' }, + { id: 'features', name: 'Scenarios', type: 'bdd' }, + ]) + await waitFor(() => expect(screen.getByText('Scenarios')).toBeInTheDocument()) + expect(screen.getByText('Model').closest('a')).toHaveAttribute('href', '/p/ohm/c/default/') + expect(screen.getByText('Scenarios').closest('a')).toHaveAttribute('href', '/p/ohm/c/features/') + expect(screen.getByText('RFCs')).toBeInTheDocument() // document → RFCs + expect(screen.getByText('Features')).toBeInTheDocument() // bdd → Features (type noun) + }) + + it('redirects into the sole collection when exactly one is visible', async () => { + renderDir([{ id: 'default', name: 'Model', type: 'document' }]) + await waitFor(() => expect(screen.getByText('collection home')).toBeInTheDocument()) + }) + + it('shows a minimal empty note when there are no collections', async () => { + renderDir([]) + await waitFor(() => expect(screen.getByText('No collections yet.')).toBeInTheDocument()) + }) +}) -- 2.39.5 From bd6dc6524ac1414e692efda4ca22d5dc5bdee336 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:14:52 -0700 Subject: [PATCH 19/21] =?UTF-8?q?=C2=A722=20S2:=20@S2=20acceptance=20?= =?UTF-8?q?=E2=80=94=20anonymous=20empty=20public=20collection=20catalog?= =?UTF-8?q?=20(C3.6)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Anonymous reader of an empty public collection gets a 200 empty catalog and no propose action (the propose route rejects anonymous); the Catalog footer's 'Sign in to propose' prompt is the existing anonymous affordance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- backend/tests/test_collection_scoped_serve.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/backend/tests/test_collection_scoped_serve.py b/backend/tests/test_collection_scoped_serve.py index cb1ab43..7c4d5d2 100644 --- a/backend/tests/test_collection_scoped_serve.py +++ b/backend/tests/test_collection_scoped_serve.py @@ -139,3 +139,21 @@ def test_scoped_routes_404_for_collection_outside_project(app_with_fake_gitea): with TestClient(app) as client: r = client.get("/api/projects/default/collections/nope/rfcs") assert r.status_code == 404 + + +def test_s2_anonymous_empty_public_collection(app_with_fake_gitea): + """C3.6 (@S2): a public collection with no entries; an anonymous visitor + lands on its catalog → an empty catalog (200, no items), and the propose + action is not available to them (the propose route rejects anonymous).""" + app, _ = app_with_fake_gitea + with TestClient(app) as client: + _add_features_collection() # public, no entries + # Anonymous (no session cookie) reads the empty catalog — 200, []. + r = client.get("/api/projects/default/collections/features/rfcs") + assert r.status_code == 200, r.text + assert r.json()["items"] == [] + # No propose action for an anonymous visitor. + r2 = client.post( + "/api/projects/default/collections/features/rfcs/propose", + json={"title": "X", "slug": "x", "pitch": "p", "tags": []}) + assert r2.status_code == 401 -- 2.39.5 From 55d04ce4ca730886d4041c345c039c3406c0afda Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 13:17:31 -0700 Subject: [PATCH 20/21] =?UTF-8?q?=C2=A722=20S2:=20release=20v0.41.0=20?= =?UTF-8?q?=E2=80=94=20create=20&=20navigate=20a=20second=20collection=20(?= =?UTF-8?q?@S2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Minor, non-breaking: named collections via .collection.yaml, create-collection endpoint, collection-scoped serve/propose, and the /p/<project>/ collection directory. Completes acceptance @S2 (C3.6). 478 backend + 26 frontend green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- CHANGELOG.md | 57 +++++++++++++++++++ VERSION | 2 +- ...6-06-05-three-tier-projects-collections.md | 3 +- frontend/package.json | 2 +- 4 files changed, 61 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 74bd6b0..8eaf4f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,63 @@ skip versions are the composition of each intervening adjacent release's steps in order — no A-to-B path is pre-computed beyond that. +## 0.41.0 — 2026-06-05 + +**Minor (non-breaking) — §22 three-tier refactor, slice S2: *create & navigate a +second collection.* A deployment can now host more than one RFC collection per +project: a second collection is created, navigated, and proposed into beside the +default one. Purely additive — a single-collection deployment is unchanged (its +`/p/<project>/` still redirects into the sole collection, C3.7/C3.8). Completes +acceptance scenario `@S2` (C3.6: an anonymous reader of an empty public +collection sees an empty catalog with no propose action and a sign-in prompt).** + +See [`docs/design/2026-06-05-three-tier-projects-collections.md`](./docs/design/2026-06-05-three-tier-projects-collections.md) +(Part E slice S2) and the slice plan +[`docs/design/plans/2026-06-05-s2-second-collection.md`](./docs/design/plans/2026-06-05-s2-second-collection.md). +Scoped {owner, contributor} roles at the collection axis remain S3; the +role-keyed create/propose-first empty states remain S4. + +Added: + +- **Named collections via `.collection.yaml`** — the registry mirror walks each + project's content repo and upserts a collection per + `<subfolder>/.collection.yaml` manifest (`type`, optional `visibility` / + `initial_state` / `name`; `type` immutable per §22.4a, visibility inherits the + project's when omitted). The default collection still flows from + `projects.yaml`. +- **create-collection** — `POST /api/projects/<id>/collections` (deployment + owner/admin). The bot commits a `.collection.yaml` to the content repo's + `main`, then the registry re-mirrors so the `collections` row appears (the + registry stays the source of truth, §22.2). New reads + `GET /api/projects/<id>/collections` and `…/collections/<cid>`. +- **Collection-scoped serve + propose** — + `GET /api/projects/<id>/collections/<cid>/rfcs[/<slug>]` and + `POST …/collections/<cid>/rfcs/propose`. A propose writes the entry under the + target collection's `<subfolder>/rfcs/`. +- **Collection directory at `/p/<project>/`** — lists the project's visible + collections, or redirects into the sole one when there is exactly one + (preserving the S1 single-collection UX). The catalog rail + entry views read + the active `/c/<collection>/` segment and scope to it. + +Changed: + +- **The corpus mirror is collection-grained** — `cache.refresh_meta_repo` + iterates each project's collections and reads each collection's + `<subfolder>/rfcs/`, keying `cached_rfcs` by `collection_id`. The default + collection keeps the shipped repo-root `rfcs/` path; N=1 serving is unchanged. + +> ### Upgrade steps (0.40.0 → 0.41.0) +> +> - No required operator action — the slice is additive and the default-collection +> paths are unchanged. A deployment **MAY** deploy this version with no config +> change and keep running exactly as on 0.40.0. +> - To add a second collection, a deployment owner/admin **MAY** call +> `POST /api/projects/<id>/collections` (or commit a `<subfolder>/.collection.yaml` +> to the content repo directly); the registry mirror picks it up on the next +> refresh. +> - Deployments that pin the framework version **MUST** bump their version pin to +> `0.41.0`. + ## 0.40.0 — 2026-06-05 **Minor (breaking URL) — §22 three-tier refactor, slice S1: the *collection* diff --git a/VERSION b/VERSION index 9b0025a..72a8a63 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.40.0 +0.41.0 diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md index 2b858a9..a152e4c 100644 --- a/docs/design/2026-06-05-three-tier-projects-collections.md +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -549,7 +549,8 @@ means the deployment runs and either gains a capability or provably loses none exactly as before, now with a real collection layer and one extra path segment. **Completes:** `@S1` (the single-collection / single-project redirect skips). -- **S2 — Create & navigate a second collection.** Teach the registry mirror to +- **S2 — Create & navigate a second collection.** *(Shipped v0.41.0.)* Teach the + registry mirror to read `.collection.yaml`; add the bot-commit-wrapped **create-collection** endpoint (authorized by existing deployment owner/admin for now — the scoped role surface lands in S3); the project collection-directory at `/p/<project>/`; diff --git a/frontend/package.json b/frontend/package.json index c417a57..3cd1058 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "rfc-app-frontend", "private": true, - "version": "0.40.0", + "version": "0.41.0", "type": "module", "scripts": { "dev": "vite", -- 2.39.5 From c2f566512ab442ca0683a7bf5ad2e9c6d1adb873 Mon Sep 17 00:00:00 2001 From: Ben Stull <ben.stull@wiggleverse.org> Date: Fri, 5 Jun 2026 18:07:51 -0700 Subject: [PATCH 21/21] =?UTF-8?q?=C2=A722=20S3:=20scope-role=20enforcement?= =?UTF-8?q?=20+=20collection-grain=20visibility=20(@S3)=20=E2=80=94=20v0.4?= =?UTF-8?q?2.0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implement slice S3 of the §22 three-tier refactor: the four-layer most-permissive scope-role resolver (§B.2) over {owner, contributor} grants at {global, project, collection}, with the §22.5 visibility gate enforced at the collection grain. - migration 030: memberships.scope_type += 'global' (the global RFC Contributor tier; sentinel scope_id '*'). - auth.effective_scope_role folds global → project → collection, most-permissive, no negative override; can_read_collection / can_contribute_in_collection / is_collection_superuser / can_create_collection gate reads, writes, admin, and create. - collection-grain visibility: a gated collection is hidden from the public (404, omitted from the directory) yet visible+listed for a scope-role holder; a collection may be set only as strict or stricter than its project (public < unlisted < gated), validated at create and clamped at the mirror. - entry-scoped authority (mark-reviewed, graduate, branch read/contribute, PR/discussion/contribution moderation) re-pointed from the project grain to the entry's collection. - create-collection authority widened to a project/global-scope grant holder (§B.1), not only a deployment owner/admin. Keystone reconciliation (session 0076): a plain granted account is a granted *account*, not a write-everywhere global role; the implicit-public write baseline is grandfathered onto the migration-seeded `default` collection only, so the N=1 deployment loses no capability. Reinterprets §B.1/§B.3 literally — flagged for the SPEC merge (S6). Completes @S3 (C1.1–C1.8). Tests: test_s3_scope_roles_vertical.py (8 C.1 scenarios + visibility/strictness), test_migration_030_global_scope.py. Full backend suite 493 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --- CHANGELOG.md | 80 +++++ VERSION | 2 +- backend/app/api.py | 23 +- backend/app/api_branches.py | 36 +- backend/app/api_collections.py | 34 +- backend/app/api_contributions.py | 2 +- backend/app/api_discussion.py | 2 +- backend/app/api_graduation.py | 4 +- backend/app/api_prs.py | 2 +- backend/app/auth.py | 316 +++++++++++++++--- backend/app/registry.py | 21 +- backend/migrations/030_global_scope.sql | 34 ++ backend/tests/test_collection_scoped_serve.py | 5 + .../tests/test_migration_030_global_scope.py | 91 +++++ backend/tests/test_project_scoped_propose.py | 7 + backend/tests/test_s3_scope_roles_vertical.py | 287 ++++++++++++++++ ...6-06-05-three-tier-projects-collections.md | 26 +- frontend/package.json | 2 +- 18 files changed, 877 insertions(+), 97 deletions(-) create mode 100644 backend/migrations/030_global_scope.sql create mode 100644 backend/tests/test_migration_030_global_scope.py create mode 100644 backend/tests/test_s3_scope_roles_vertical.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 8eaf4f6..b56ae4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,86 @@ skip versions are the composition of each intervening adjacent release's steps in order — no A-to-B path is pre-computed beyond that. +## 0.42.0 — 2026-06-05 + +**Minor (breaking — upgrade steps below) — §22 three-tier refactor, slice S3: +*scope-role enforcement + collection-grain visibility.* Authorization is now +resolved by the four-layer most-permissive union of §B.2 — global → project → +collection → per-entry — over the unified `{owner, contributor}` roles, with the +§22.5 visibility gate enforced at the **collection** grain. A collection can be +"hidden from public existence" (`gated`): invisible to anonymous viewers and +omitted from the project directory, yet readable and listed for any contributor +holding a scope role that reaches it (collection / project / global). A +collection's visibility may be set only as strict or stricter than its project's. +Completes acceptance scenarios `@S3` (C1.1–C1.8: role usage, inheritance, the +most-permissive union, and no-negative-override).** + +See [`docs/design/2026-06-05-three-tier-projects-collections.md`](./docs/design/2026-06-05-three-tier-projects-collections.md) +(Part B roles, Part C.1 scenarios, Part E slice S3). The invitation UI and +role-keyed empty states remain S4; grants in S3 are applied administratively +(via the `memberships` table / an Owner-authorized create surface). + +Added: + +- **Four-layer scope-role resolver** — `effective_scope_role(user, collection)` + folds a deployment owner/admin (global Owner), an explicit global-scope grant, + a project-scope grant, and a collection-scope grant into the most-permissive + unified role over a collection. Owner outranks RFC Contributor; there is no + negative override (a child scope can never subtract a parent grant). +- **Global-scope grants** — migration 030 extends `memberships.scope_type` to + `{global, project, collection}`. A "global RFC Contributor" (writes in every + collection of every project, distinct from a deployment owner) is a + `scope_type='global'` row (sentinel `scope_id='*'`). +- **Collection-grain visibility enforcement** — `can_read_collection` / + `require_collection_readable` gate the collection-scoped read, entry, and + propose routes; the project collection-directory (`GET + /api/projects/<id>/collections`) is now viewer-aware (a hidden/gated collection + is listed only for a scope-role holder; `unlisted` stays omitted from + enumeration). The effective read gate is the stricter of the project's and the + collection's visibility. +- **Collection visibility strictness** — a collection may narrow but never widen + its project's visibility (`public` < `unlisted` < `gated`). Enforced at + create-collection (422 on a looser request) and clamped at the registry mirror. +- **create-collection authority widened (§B.1)** — `POST + /api/projects/<id>/collections` now admits a project-scope or global-scope + grant holder (Owner **or** RFC Contributor — the project-level "create a + collection" affordance), not only a deployment owner/admin. A collection-scope + grant cannot create sibling collections. + +Changed (breaking): + +- **Write standing now requires an explicit scope grant outside the default + collection.** The pre-three-tier implicit-on-public write baseline (a granted + deployment `contributor` may propose on any public project with no membership + row) is **narrowed to the migration-seeded `default` collection only** (the N=1 + case, §22.13). On every *explicitly-created* collection — and on every project + beyond the default — proposing, discussing, branching, and contributing now + require an explicit `{owner, contributor}` grant at the collection, its + project, or global scope. Reads of public collections are unchanged. +- Entry-scoped authority checks (mark-reviewed, graduate, branch read/contribute, + PR/discussion/contribution moderation) are re-pointed from the project grain to + the entry's **collection** grain, so a collection Owner administers exactly + their collection's subtree and no more. + +Upgrade steps: + +1. The schema migration (`030_global_scope.sql`) runs automatically on deploy and + is backward-data-compatible — existing `memberships` rows are preserved. No + operator action is required for the migration itself. +2. **A single-collection (N=1) deployment needs no further action.** The `default` + collection keeps the implicit-on-public write baseline, so existing granted + contributors keep proposing exactly as before. +3. **A deployment that has created additional collections (S2) MUST grant scope + roles to its contributors.** Any contributor who should write in a non-default + collection (or in a second project) now needs an explicit grant: a + `memberships` row at `scope_type` `collection` (that collection), `project` + (its project — covers every collection within), or `global` (`scope_id='*'` — + every project). A deployment owner/admin is unaffected (global Owner by role). +4. To make a collection **hidden from the public**, set `visibility: gated` in its + `.collection.yaml` (or the project's `visibility` in `projects.yaml`); the + value MUST be as strict or stricter than the parent project's. The mirror + clamps a looser value and logs a warning. + ## 0.41.0 — 2026-06-05 **Minor (non-breaking) — §22 three-tier refactor, slice S2: *create & navigate a diff --git a/VERSION b/VERSION index 72a8a63..787ffc3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.41.0 +0.42.0 diff --git a/backend/app/api.py b/backend/app/api.py index 55eb9fc..60be3f5 100644 --- a/backend/app/api.py +++ b/backend/app/api.py @@ -821,6 +821,8 @@ def make_router( viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) _require_collection_in_project(collection_id, project_id) + # §22.5 (S3): a hidden/gated collection 404s to a non-scope-role viewer. + auth.require_collection_readable(viewer, collection_id) return _list_rfcs_for_collection(collection_id, viewer, unreviewed) @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}") @@ -830,6 +832,7 @@ def make_router( viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) _require_collection_in_project(collection_id, project_id) + auth.require_collection_readable(viewer, collection_id) return _get_rfc_for_collection(collection_id, slug, viewer) # --------------------------------------------------------------- @@ -839,12 +842,13 @@ def make_router( @router.post("/api/projects/{project_id}/rfcs/{slug}/mark-reviewed") async def mark_reviewed(project_id: str, slug: str, request: Request) -> dict[str, Any]: """§22.4c — clear an active entry's `unreviewed` flag. Authority is the - §22.7 project superuser (project_admin or deployment owner/admin).""" + §B.2 collection Owner (a collection/project/global Owner or deployment + owner/admin reaching the entry's collection).""" viewer = auth.require_user(request) auth.require_project_readable(viewer, project_id) - if not auth.is_project_superuser(viewer, project_id): - raise HTTPException(403, "Only a project owner/admin can mark an entry reviewed") collection_id = collections_mod.default_collection_id(project_id) + if not auth.is_collection_superuser(viewer, collection_id): + raise HTTPException(403, "Only a collection owner can mark an entry reviewed") row = db.conn().execute( "SELECT state, unreviewed FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id), @@ -1004,11 +1008,11 @@ def make_router( async def _propose_into_collection( project_id: str, collection_id: str, payload: ProposeBody, user ) -> dict[str, Any]: - # §22.6/§22.7: proposing a new entry requires project-level contribute - # standing in the *target* project. On the public default project the - # implicit-public baseline preserves the pre-multi-project flow. - if not auth.can_contribute_in_project(user, project_id): - raise HTTPException(403, "You do not have contribute access to this project") + # §B.2 (S3): proposing a new entry requires contribute standing in the + # *target collection* — the four-layer scope-role union, with the + # grandfathered implicit-public baseline on the default collection. + if not auth.can_contribute_in_collection(user, collection_id): + raise HTTPException(403, "You do not have contribute access to this collection") slug = payload.slug.strip().lower() if not entry_mod.is_valid_slug(slug): raise HTTPException(422, "Slug must be lowercase letters, digits, and dashes") @@ -1135,6 +1139,9 @@ def make_router( user = auth.require_contributor(request) auth.require_project_readable(user, project_id) _require_collection_in_project(collection_id, project_id) + # §22.5 (S3): a hidden/gated collection 404s a non-scope-role viewer + # before the contribute check (existence is not revealed). + auth.require_collection_readable(user, collection_id) return await _propose_into_collection(project_id, collection_id, payload, user) # --------------------------------------------------------------- diff --git a/backend/app/api_branches.py b/backend/app/api_branches.py index 74b18c8..6d46f4c 100644 --- a/backend/app/api_branches.py +++ b/backend/app/api_branches.py @@ -1264,11 +1264,11 @@ def make_router( return row["on_behalf_of"] if row else None def _can_read_branch(slug: str, branch: str, viewer) -> bool: - # §22.5 visibility gate first (subtractive, §22.7): in a gated project - # nothing — not even main or a read_public branch — is readable by a - # non-member. - pid = auth.project_of_rfc(slug) - if not auth.can_read_project(viewer, pid): + # §22.5 visibility gate first (subtractive, §B.2): in a hidden/gated + # collection nothing — not even main or a read_public branch — is + # readable by a non-scope-role viewer. + cid = auth.collection_of_rfc(slug) + if not auth.can_read_collection(viewer, cid): return False if branch == "main": return True @@ -1277,7 +1277,7 @@ def make_router( return True if viewer is None: return False - if auth.is_project_superuser(viewer, pid): + if auth.is_collection_superuser(viewer, cid): return True creator = _branch_creator(slug, branch) if creator and viewer.gitea_login == creator: @@ -1310,12 +1310,12 @@ def make_router( # legacy `repo:` is set (nothing, after the RFC-0001 fold-back). if rfc["state"] == "active" and rfc["repo"] and _is_meta_branch_name(branch): return False - pid = auth.project_of_rfc(slug) + cid = auth.collection_of_rfc(slug) # §22.5 visibility gate (subtractive): no contribute in an unreadable - # project. - if not auth.can_read_project(viewer, pid): + # collection. + if not auth.can_read_collection(viewer, cid): return False - if auth.is_project_superuser(viewer, pid): + if auth.is_collection_superuser(viewer, cid): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") @@ -1326,10 +1326,10 @@ def make_router( return True vis = _branch_vis(slug, branch) if vis["contribute_mode"] == "any-contributor": - # "any contributor" means anyone with project-level write standing - # (§22.6/§22.7) — the implicit-public baseline on a public project, - # or an explicit project_contributor/admin elsewhere. - return auth.can_contribute_in_project(viewer, pid) + # "any contributor" means anyone with collection-level write standing + # (§B.2) — the grandfathered baseline on the public default + # collection, or an explicit scope grant reaching the collection. + return auth.can_contribute_in_collection(viewer, cid) if vis["contribute_mode"] == "specific": row = db.conn().execute( """ @@ -1352,7 +1352,7 @@ def make_router( def _require_branch_owner(rfc, viewer, creator: str | None) -> None: # §22.6: a project_admin is the per-RFC owner/arbiter authority lifted # to project scope, so it (and a deployment owner/admin) clears here. - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") @@ -1368,7 +1368,7 @@ def make_router( has no owners, so the set collapses to the superuser tier only — sensible because admin oversight is the only path to canonicalizing edits on an unclaimed entry.""" - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") @@ -1381,7 +1381,7 @@ def make_router( "can_read": _can_read_branch(slug, branch, viewer), "can_contribute": _can_contribute(rfc, slug, branch, viewer) if viewer else False, "can_change_branch_settings": viewer is not None and ( - auth.is_project_superuser(viewer, rfc["project_id"]) + auth.is_collection_superuser(viewer, rfc["collection_id"]) or (creator is not None and viewer.gitea_login == creator) or viewer.gitea_login in (owners + arbiters) ), @@ -1427,7 +1427,7 @@ def make_router( def _can_resolve_thread(rfc, thread, creator: str | None, viewer) -> bool: if viewer is None: return False - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") diff --git a/backend/app/api_collections.py b/backend/app/api_collections.py index c80866a..2f990d4 100644 --- a/backend/app/api_collections.py +++ b/backend/app/api_collections.py @@ -49,7 +49,15 @@ def make_router(config: Config, gitea: Gitea, bot: Bot) -> APIRouter: viewer = auth.current_user(request) # §22.5 read gate: a gated project 404s a non-member. auth.require_project_readable(viewer, project_id) - return {"items": collections_mod.list_collections(project_id)} + # §22.5 (S3): the directory is viewer-aware — a hidden/gated collection + # is listed only for a scope-role holder who can read it; `unlisted` is + # omitted from enumeration for everyone (link-only). + items = [ + c + for c in collections_mod.list_collections(project_id, include_unlisted=True) + if c["visibility"] != "unlisted" and auth.can_read_collection(viewer, c["id"]) + ] + return {"items": items} @router.get("/api/projects/{project_id}/collections/{collection_id}") async def get_col(project_id: str, collection_id: str, request: Request) -> dict[str, Any]: @@ -58,22 +66,38 @@ def make_router(config: Config, gitea: Gitea, bot: Bot) -> APIRouter: col = collections_mod.get_collection(collection_id) if col is None or col["project_id"] != project_id: raise HTTPException(404, "Not found") + # §22.5 (S3): a hidden/gated collection 404s a non-scope-role viewer. + auth.require_collection_readable(viewer, collection_id) return col @router.post("/api/projects/{project_id}/collections") async def create_col( project_id: str, body: CreateCollectionBody, request: Request ) -> dict[str, Any]: - # S2 authority: deployment owner/admin (the scoped-role surface is S3). - user = auth.require_admin(request) + # §B.1 (S3) authority: a deployment owner/admin or a project/global-scope + # grant holder (Owner or RFC Contributor) may create a collection. The + # read gate runs first so a gated project 404s a non-member. + user = auth.require_contributor(request) auth.require_project_readable(user, project_id) + if not auth.can_create_collection(user, project_id): + raise HTTPException(403, "You may not create collections in this project") cid = body.collection_id.strip().lower() if not _SLUG_RE.match(cid) or cid == "default": raise HTTPException(422, "collection id must be a slug and not 'default'") if body.type not in registry_mod.VALID_TYPES: raise HTTPException(422, f"invalid type {body.type!r}") - if body.visibility is not None and body.visibility not in registry_mod.VALID_VISIBILITY: - raise HTTPException(422, f"invalid visibility {body.visibility!r}") + if body.visibility is not None: + if body.visibility not in registry_mod.VALID_VISIBILITY: + raise HTTPException(422, f"invalid visibility {body.visibility!r}") + # §22.5 (S3) strictness: a collection may be set only as strict or + # stricter than its project — never more public. + pvis = auth.project_visibility(project_id) + if auth.visibility_rank(body.visibility) < auth.visibility_rank(pvis): + raise HTTPException( + 422, + f"collection visibility {body.visibility!r} is looser than " + f"the project's {pvis!r}; a collection may only narrow it", + ) if body.initial_state is not None and body.initial_state not in registry_mod.VALID_INITIAL_STATE: raise HTTPException(422, f"invalid initial_state {body.initial_state!r}") if collections_mod.get_collection(cid) is not None: diff --git a/backend/app/api_contributions.py b/backend/app/api_contributions.py index e6f36c0..bf6ff4b 100644 --- a/backend/app/api_contributions.py +++ b/backend/app/api_contributions.py @@ -91,7 +91,7 @@ def _viewer_relationship(viewer, slug: str) -> str | None: """Why this viewer can't *request* to contribute — or None if they can. Owners/admins already have the RFC; existing collaborators are already in. Both get a clear 409 rather than a useless self-request.""" - if auth.is_rfc_owner(viewer, slug) or auth.is_project_superuser(viewer, auth.project_of_rfc(slug)): + if auth.is_rfc_owner(viewer, slug) or auth.is_collection_superuser(viewer, auth.collection_of_rfc(slug)): return "You already own or administer this RFC." if auth.is_rfc_collaborator(viewer, slug): return "You're already a collaborator on this RFC." diff --git a/backend/app/api_discussion.py b/backend/app/api_discussion.py index be57845..840861c 100644 --- a/backend/app/api_discussion.py +++ b/backend/app/api_discussion.py @@ -312,7 +312,7 @@ def _ensure_discussion_thread(slug: str, viewer) -> int: def _can_resolve(rfc, thread, viewer) -> bool: if viewer is None: return False - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") diff --git a/backend/app/api_graduation.py b/backend/app/api_graduation.py index b620aa3..74a472a 100644 --- a/backend/app/api_graduation.py +++ b/backend/app/api_graduation.py @@ -218,7 +218,7 @@ def make_router( can_merge = ( viewer is not None and ( - auth.is_project_superuser(viewer, rfc["project_id"]) + auth.is_collection_superuser(viewer, rfc["collection_id"]) or viewer.gitea_login in owners or viewer.gitea_login in arbiters ) @@ -795,7 +795,7 @@ def _can_graduate(rfc, viewer) -> bool: if viewer is None: return False # §6.1 admin/owner or §22.6 project_admin OR §6.3 RFC owners/arbiters. - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") diff --git a/backend/app/api_prs.py b/backend/app/api_prs.py index 69a6678..25ecc17 100644 --- a/backend/app/api_prs.py +++ b/backend/app/api_prs.py @@ -786,7 +786,7 @@ def _can_merge(rfc, viewer) -> bool: """§6.1 admin/owner or §22.6 project_admin OR §6.3 RFC owners/arbiters.""" if viewer is None: return False - if auth.is_project_superuser(viewer, rfc["project_id"]): + if auth.is_collection_superuser(viewer, rfc["collection_id"]): return True owners = json.loads(rfc["owners_json"] or "[]") arbiters = json.loads(rfc["arbiters_json"] or "[]") diff --git a/backend/app/auth.py b/backend/app/auth.py index 98a6978..4f529ad 100644 --- a/backend/app/auth.py +++ b/backend/app/auth.py @@ -16,6 +16,7 @@ from typing import Any import httpx from fastapi import HTTPException, Request +from . import collections as collections_mod from . import db from .bot import Actor from .config import Config @@ -336,26 +337,37 @@ def project_visibility(project_id: str) -> str: return row["visibility"] or "gated" -def project_member_role(user: SessionUser | None, project_id: str) -> str | None: - """The user's *explicit* §22.6 membership role at this project, or None. +def _is_default_project(project_id: str) -> bool: + """True iff `project_id` owns the migration-seeded `default` collection — the + deployment's primary project (§22.13), whatever its configured id. Only there + do M2's role rows (which migrated to collection scope `default`) stand in for + project-level authority.""" + return collections_mod.project_of_collection(collections_mod.DEFAULT_COLLECTION_ID) == project_id - §22 three-tier (S1): M2's `project_members` rows migrated into the unified - `memberships` table at the project's default collection (and the project - tier is freshly grantable). The unified `{owner, contributor}` roles are - mapped back to the legacy `project_admin`/`project_contributor` strings the - S1 project-grain authz still speaks; the four-layer scope resolver lands in - S3. Reads the stored grant only (project OR default-collection scope) — it - does not fold in the deployment tier or the implicit-on-public baseline.""" + +def project_member_role(user: SessionUser | None, project_id: str) -> str | None: + """The user's *project-grain* §22.6 role at this project, or None — the + most-permissive of a **global** grant (inherits down to every project) and a + **project**-scope grant. Mapped back to the legacy + `project_admin`/`project_contributor` strings the project-grain authz speaks. + + Back-compat: on the deployment's *default* project only, M2's rows live at + collection scope `default` (§B.3 migration), so a `default` collection-scope + grant there is read as project-level too. A collection grant on any other + project is NOT project authority — that is the four-layer collection resolver + (`effective_scope_role`). Does not fold in the deployment tier + (`is_project_superuser` adds it) or the implicit-on-public baseline.""" if user is None: return None - from . import collections as collections_mod - cid = collections_mod.default_collection_id(project_id) + clauses = ["scope_type = 'global'", "(scope_type = 'project' AND scope_id = ?)"] + params: list = [user.user_id, project_id] + if _is_default_project(project_id): + clauses.append("(scope_type = 'collection' AND scope_id = ?)") + params.append(collections_mod.DEFAULT_COLLECTION_ID) row = db.conn().execute( - "SELECT role FROM memberships " - "WHERE user_id = ? AND ((scope_type = 'project' AND scope_id = ?) " - " OR (scope_type = 'collection' AND scope_id = ?)) " + "SELECT role FROM memberships WHERE user_id = ? AND (" + " OR ".join(clauses) + ") " "ORDER BY CASE role WHEN 'owner' THEN 0 ELSE 1 END LIMIT 1", - (user.user_id, project_id, cid), + params, ).fetchone() if row is None: return None @@ -378,6 +390,20 @@ def project_of_rfc(rfc_slug: str) -> str: return row["project_id"] or DEFAULT_PROJECT_ID +def collection_of_rfc(rfc_slug: str) -> str: + """The collection an RFC belongs to (`cached_rfcs.collection_id`). Falls back + to the default collection when the slug isn't cached. Mirrors + `project_of_rfc`'s first-match semantics; a slug shared across collections is + a known routing ambiguity (the RFC-grain helpers take a bare slug) resolved + by the collection-qualified routes in later slices.""" + row = db.conn().execute( + "SELECT collection_id FROM cached_rfcs WHERE slug = ?", (rfc_slug,) + ).fetchone() + if row is None or not row["collection_id"]: + return collections_mod.DEFAULT_COLLECTION_ID + return row["collection_id"] + + def is_project_superuser(user: SessionUser | None, project_id: str) -> bool: """Maximal authority within a project: a deployment owner/admin (superuser in every project, §22.7) or an explicit `project_admin` (§22.6). Both @@ -390,20 +416,30 @@ def is_project_superuser(user: SessionUser | None, project_id: str) -> bool: def can_read_project(user: SessionUser | None, project_id: str) -> bool: - """The §22.5 visibility gate. `public`/`unlisted` are readable by anyone - (anonymous included — `unlisted` is link-only but the link still reads); - `gated` is readable only by a deployment owner/admin or a granted project - member of any role. Used as the subtractive read gate (a gated project's - entries 404 to non-members).""" + """The §22.5 visibility gate at the project grain. `public`/`unlisted` are + readable by anyone (anonymous included — `unlisted` is link-only but the link + still reads); `gated` is readable only by a deployment owner/admin or a + holder of any scope grant reaching the project — a global grant, a project + grant, or membership at *any* collection within it (seeing a collection + implies seeing its project). Used as the subtractive read gate (a gated + project's entries 404 to non-members).""" vis = project_visibility(project_id) if vis in ("public", "unlisted"): return True - # gated — members + superusers only, subject to the §6 admission floor. + # gated — scope-role holders + superusers only, subject to the §6 floor. if user is None or user.permission_state != "granted": return False if user.role in _DEPLOYMENT_SUPERUSER_ROLES: return True - return project_member_role(user, project_id) is not None + row = db.conn().execute( + "SELECT 1 FROM memberships m WHERE m.user_id = ? AND (" + " m.scope_type = 'global'" + " OR (m.scope_type = 'project' AND m.scope_id = ?)" + " OR (m.scope_type = 'collection' AND m.scope_id IN " + " (SELECT id FROM collections WHERE project_id = ?))) LIMIT 1", + (user.user_id, project_id, project_id), + ).fetchone() + return row is not None def require_project_readable(user: SessionUser | None, project_id: str) -> None: @@ -465,6 +501,187 @@ def visible_project_ids(user: SessionUser | None) -> list[str]: return [r["id"] for r in rows if can_read_project(user, r["id"])] +# =========================================================================== +# §22 three-tier — S3. The four-layer scope-role resolver (§B.2) and the +# collection-grain visibility gate. +# +# A grant attaches the unified role {owner, contributor} at a scope: global, +# project, or collection (§B.1). Grants inherit downward, are additive, and +# admit no negative override (§B.2). Effective authority over a *collection* is +# the most-permissive union of the layers reaching it: +# +# global (users.role owner/admin ∪ memberships scope_type='global') +# ∪ project (memberships scope_type='project' at the collection's project) +# ∪ collection (memberships scope_type='collection' at the collection) +# +# minus the §22.5 visibility gate and §6.2 write-mute (subtractive, as today). +# Per-entry authority (owners / arbiters / rfc_collaborators) is a distinct, +# finer layer the RFC-grain helpers union in beneath collection. +# +# OPERATOR DECISIONS (S3, session 0076): +# * scope-role-primary — a plain granted account (users.role 'contributor', no +# membership) is a granted *account*, not a write-everywhere global role. +# Write standing comes from an explicit scope grant; the lone exception is the +# grandfathered implicit-public baseline below. +# * grandfathered baseline — the migration-seeded `default` collection keeps the +# pre-three-tier implicit-on-public write baseline (a granted deployment +# contributor may propose while it is public), so the N=1 deployment (§22.13) +# loses no capability. Every *explicitly-created* collection requires an +# explicit scope grant to write. +# * hidden-from-public — a `gated` collection is invisible to the public (404, +# omitted from the directory) yet visible to any scope-role holder reaching it +# (collection/project/global). A collection's visibility may be set only as +# strict or stricter than its project's (the rank ordering below). +# =========================================================================== + +# The global scope is a single tier per deployment; its grant rows use this +# sentinel scope_id (migration 030). +GLOBAL_SCOPE_ID = "*" + +# §22.5 visibility strictness on the public-exposure axis: `public` is least +# strict, `gated` most strict. A collection may narrow its project's visibility +# but never widen it (`rank(collection) >= rank(project)`). +_VISIBILITY_RANK = {"public": 0, "unlisted": 1, "gated": 2} + + +def visibility_rank(visibility: str | None) -> int: + """The strictness rank of a §22.5 visibility (higher = stricter). An unknown + value reads as the strictest (`gated`) — the safe default.""" + return _VISIBILITY_RANK.get(visibility or "", _VISIBILITY_RANK["gated"]) + + +def effective_scope_role(user: SessionUser | None, collection_id: str) -> str | None: + """The most-permissive unified role ({'owner','contributor'}) the user holds + over `collection_id`, folding §B.2's global → project → collection layers. + Returns None when no scope grant reaches the collection. 'owner' outranks + 'contributor'; there is no negative override (a parent grant is never + subtracted by a child). Subject to the §6 admission floor.""" + if user is None or user.permission_state != "granted": + return None + # Global tier — a deployment owner/admin is a global Owner (§B.1). + if user.role in _DEPLOYMENT_SUPERUSER_ROLES: + return "owner" + pid = collections_mod.project_of_collection(collection_id) + row = db.conn().execute( + "SELECT role FROM memberships " + "WHERE user_id = ? AND (" + " scope_type = 'global'" + " OR (scope_type = 'project' AND scope_id = ?)" + " OR (scope_type = 'collection' AND scope_id = ?)) " + "ORDER BY CASE role WHEN 'owner' THEN 0 ELSE 1 END LIMIT 1", + (user.user_id, pid, collection_id), + ).fetchone() + return row["role"] if row else None + + +def collection_visibility(collection_id: str) -> str: + """The collection's own §22.5 visibility. A missing row reads as 'gated' — + an unknown collection is invisible rather than open.""" + row = db.conn().execute( + "SELECT visibility FROM collections WHERE id = ?", (collection_id,) + ).fetchone() + if row is None or not row["visibility"]: + return "gated" + return row["visibility"] + + +def effective_collection_visibility(collection_id: str) -> str: + """The stricter of the collection's own visibility and its project's (§22.5 + 'both gates'). A collection is constrained to be ≥ its project in strictness, + but we max() defensively so a misconfigured looser collection can never widen + its project's gate.""" + cvis = collection_visibility(collection_id) + pid = collections_mod.project_of_collection(collection_id) + pvis = project_visibility(pid) if pid else "gated" + return cvis if visibility_rank(cvis) >= visibility_rank(pvis) else pvis + + +def can_read_collection(user: SessionUser | None, collection_id: str) -> bool: + """§22.5 read/existence gate at the collection grain. `public`/`unlisted` + read by anyone (anonymous included — `unlisted` is link-only but the link + reads); `gated` ("hidden from public existence") reads only for a scope-role + holder over the collection (collection/project/global) or a deployment + owner/admin. The subtractive read gate — a gated collection 404s a + non-holder, indistinguishable from absent.""" + vis = effective_collection_visibility(collection_id) + if vis in ("public", "unlisted"): + return True + if user is None or user.permission_state != "granted": + return False + return effective_scope_role(user, collection_id) is not None + + +def require_collection_readable(user: SessionUser | None, collection_id: str) -> None: + """Raise 404 when the collection is not readable by this viewer (§22.5: a + hidden/gated collection is invisible to non-holders — the shape matches an + unknown collection).""" + if not can_read_collection(user, collection_id): + raise HTTPException(status_code=404, detail="Not found") + + +def is_collection_superuser(user: SessionUser | None, collection_id: str) -> bool: + """Maximal authority over a collection: an effective scope role of 'owner' + reaching it (a collection Owner, a project Owner of its project, a global + Owner, or a deployment owner/admin). Subsumes the per-entry owners/arbiters + tier within the collection.""" + return effective_scope_role(user, collection_id) == "owner" + + +def _has_collection_write_baseline(user: SessionUser | None, collection_id: str) -> bool: + """The grandfathered implicit-on-public write baseline, narrowed to the + migration-seeded `default` collection (§22.13 N=1 case). A granted deployment + `contributor` keeps its pre-three-tier write standing on the default + collection while its effective visibility is public; every explicitly-created + collection requires an explicit scope grant (S3 operator decision).""" + if user is None or user.permission_state != "granted": + return False + if collection_id != collections_mod.DEFAULT_COLLECTION_ID: + return False + return user.role == "contributor" and effective_collection_visibility(collection_id) == "public" + + +def can_contribute_in_collection(user: SessionUser | None, collection_id: str) -> bool: + """May the user contribute *new* content to the collection (propose an entry) + — the collection-level contribute standing. The union of the scope-role grant + (owner/contributor reaching the collection) and the grandfathered default + baseline, subject to the visibility read gate.""" + if user is None or user.permission_state != "granted": + return False + if not can_read_collection(user, collection_id): + return False + if effective_scope_role(user, collection_id) is not None: + return True + return _has_collection_write_baseline(user, collection_id) + + +def can_discuss_in_collection(user: SessionUser | None, collection_id: str) -> bool: + """May the user participate in discussion in the collection — the + collection-level discuss standing. A superset of contribute for this pass + (the read-only viewer tier is deferred, §B.3), so it mirrors + `can_contribute_in_collection`.""" + return can_contribute_in_collection(user, collection_id) + + +def can_create_collection(user: SessionUser | None, project_id: str) -> bool: + """May the user create a new collection in this project (§B.1)? Creating a + collection is a *project-level* action: a deployment owner/admin, or any + holder of a project-scope or global-scope grant (Owner OR RFC Contributor — + 'anyone at the project level with permission to create a collection'). A + *collection*-scope grant cannot create sibling collections.""" + if user is None or user.permission_state != "granted": + return False + if user.role in _DEPLOYMENT_SUPERUSER_ROLES: + return True + row = db.conn().execute( + "SELECT 1 FROM memberships " + "WHERE user_id = ? AND (" + " scope_type = 'global'" + " OR (scope_type = 'project' AND scope_id = ?)) LIMIT 1", + (user.user_id, project_id), + ).fetchone() + return row is not None + + # v0.16.0 (roadmap item #12): per-RFC membership helpers. # # These don't replace `require_contributor` — they layer on top of it for @@ -561,16 +778,14 @@ def can_discuss_rfc(user: SessionUser | None, rfc_slug: str) -> bool: return False if user.permission_state != "granted": return False - pid = project_of_rfc(rfc_slug) - # §22.5 visibility gate is subtractive (§22.7) — no capability in a project - # the viewer cannot even read. - if not can_read_project(user, pid): + cid = collection_of_rfc(rfc_slug) + # §22.5 visibility gate is subtractive (§22.7) — no capability in a + # collection the viewer cannot even read. + if not can_read_collection(user, cid): return False - # §22.7 union, override grants first — these bypass per-RFC curation - # (project_viewer ⊇ discussant; project_admin / deployment superuser ⊇ all). - if is_project_superuser(user, pid): - return True - if project_member_role(user, pid) in ("project_viewer", "project_contributor"): + # §B.2 union, scope-role grants first — these bypass per-RFC curation (a + # collection/project/global Owner or RFC Contributor ⊇ discussant). + if effective_scope_role(user, cid) is not None: return True # per-RFC authority (union term). owners = _rfc_owners_set(rfc_slug) @@ -578,11 +793,11 @@ def can_discuss_rfc(user: SessionUser | None, rfc_slug: str) -> bool: return True if is_rfc_collaborator(user, rfc_slug, role_in_rfc=None): return True - # implicit-public baseline (curation preserved): a granted deployment - # contributor on a public project may discuss only while the RFC is - # unclaimed. The first §13.1 claim engages the per-RFC gate, mirroring the + # grandfathered implicit-public baseline (curation preserved): on the default + # collection a granted deployment contributor may discuss only while the RFC + # is unclaimed. The first §13.1 claim engages the per-RFC gate, mirroring the # pre-multi-project v0.16.0 contract. - if not owners and _has_write_baseline(user, pid): + if not owners and _has_collection_write_baseline(user, cid): return True return False @@ -606,14 +821,12 @@ def can_contribute_to_rfc(user: SessionUser | None, rfc_slug: str) -> bool: return False if user.permission_state != "granted": return False - pid = project_of_rfc(rfc_slug) - if not can_read_project(user, pid): + cid = collection_of_rfc(rfc_slug) + if not can_read_collection(user, cid): return False - # §22.7 union, override grants first (project_contributor ⊇ - # rfc_collaborators(contributor); project_admin / superuser ⊇ all). - if is_project_superuser(user, pid): - return True - if project_member_role(user, pid) == "project_contributor": + # §B.2 union, scope-role grants first (a collection/project/global RFC + # Contributor ⊇ rfc_collaborators(contributor); an Owner ⊇ all). + if effective_scope_role(user, cid) is not None: return True # per-RFC authority (union term). A 'discussant' row is NOT sufficient — # PRs are the higher-privilege surface. @@ -622,9 +835,10 @@ def can_contribute_to_rfc(user: SessionUser | None, rfc_slug: str) -> bool: return True if is_rfc_collaborator(user, rfc_slug, role_in_rfc="contributor"): return True - # implicit-public baseline (curation preserved): until an owner exists, a - # granted deployment contributor on a public project may contribute. - if not owners and _has_write_baseline(user, pid): + # grandfathered implicit-public baseline (curation preserved): until an owner + # exists, a granted deployment contributor on the public default collection + # may contribute. + if not owners and _has_collection_write_baseline(user, cid): return True return False @@ -637,13 +851,13 @@ def can_invite_to_rfc(user: SessionUser | None, rfc_slug: str) -> bool: return False if user.permission_state != "granted": return False - pid = project_of_rfc(rfc_slug) - if not can_read_project(user, pid): + cid = collection_of_rfc(rfc_slug) + if not can_read_collection(user, cid): return False - # Deployment owner/admin or project_admin (§22.6) may invite; otherwise - # only the RFC's frontmatter owner. Per-RFC collaborators and the - # implicit-public baseline do not get the invite-others power. - if is_project_superuser(user, pid): + # An Owner reaching the collection (collection/project/global Owner, or a + # deployment owner/admin) may invite; otherwise only the RFC's frontmatter + # owner. Per-RFC collaborators and the baseline do not get the invite power. + if is_collection_superuser(user, cid): return True return is_rfc_owner(user, rfc_slug) diff --git a/backend/app/registry.py b/backend/app/registry.py index b86a7fb..b4e9826 100644 --- a/backend/app/registry.py +++ b/backend/app/registry.py @@ -231,13 +231,30 @@ def apply_registry(doc: RegistryDoc, registry_sha: str, default_id: str) -> None ) +def _strictest_visibility(a: str, b: str) -> str: + """The stricter of two §22.5 visibilities on the public-exposure axis + (`public` < `unlisted` < `gated`). Used to enforce that a collection is set + only as strict or stricter than its project (S3 operator decision).""" + rank = {"public": 0, "unlisted": 1, "gated": 2} + return a if rank.get(a, 2) >= rank.get(b, 2) else b + + def _upsert_named_collection( proj: ProjectEntry, subdir: str, ce: CollectionEntry, sha: str ) -> None: """Upsert one named collection (S2). Type is immutable (§22.4a): a type change against an existing row is refused (logged, not applied). A None - manifest visibility inherits the project's visibility.""" - visibility = ce.visibility or proj.visibility + manifest visibility inherits the project's visibility; a manifest that tries + to be *looser* than its project is clamped to the project's (S3 strictness: + a collection may narrow but never widen its project's visibility).""" + requested = ce.visibility or proj.visibility + visibility = _strictest_visibility(requested, proj.visibility) + if visibility != requested: + log.warning( + "registry: collection %s visibility %r looser than project %s %r — " + "clamped to %r (S3 strictness)", + subdir, requested, proj.id, proj.visibility, visibility, + ) with db.tx() as conn: existing = conn.execute( "SELECT type FROM collections WHERE id = ?", (subdir,) diff --git a/backend/migrations/030_global_scope.sql b/backend/migrations/030_global_scope.sql new file mode 100644 index 0000000..b2c0c6d --- /dev/null +++ b/backend/migrations/030_global_scope.sql @@ -0,0 +1,34 @@ +-- migrate:no-foreign-keys +-- +-- §22 three-tier — S3. Admit a *global*-scope grant to the memberships table. +-- +-- §B.2's resolver folds four layers (global → project → collection → per-entry). +-- Migration 029 created `memberships` with scope_type ∈ {project, collection} +-- only; the global tier was left to S3. A global grant is how a "global RFC +-- Contributor" (a contributor who may propose in every collection of every +-- project, distinct from a deployment owner/admin) is represented — see +-- docs/design/2026-06-05-three-tier-projects-collections.md §B.2/§B.3 and the +-- C.1 "cleo" scenario. +-- +-- SQLite can't ALTER a CHECK constraint in place, so the table is rebuilt by the +-- create-copy-drop-rename procedure (the 028/029 pattern). The global scope uses +-- a stable sentinel scope_id of '*' (one global tier per deployment); the +-- UNIQUE(scope_type, scope_id, user_id) then admits exactly one global grant per +-- user, mirroring the project/collection rows. + +CREATE TABLE memberships__new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + scope_type TEXT NOT NULL CHECK (scope_type IN ('global', 'project', 'collection')), + scope_id TEXT NOT NULL, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + role TEXT NOT NULL CHECK (role IN ('owner', 'contributor')), + granted_by INTEGER REFERENCES users(id) ON DELETE SET NULL, + granted_at TEXT NOT NULL DEFAULT (datetime('now')), + UNIQUE (scope_type, scope_id, user_id) +); +INSERT INTO memberships__new (id, scope_type, scope_id, user_id, role, granted_by, granted_at) +SELECT id, scope_type, scope_id, user_id, role, granted_by, granted_at FROM memberships; +DROP TABLE memberships; +ALTER TABLE memberships__new RENAME TO memberships; +CREATE INDEX idx_memberships_user ON memberships(user_id); +CREATE INDEX idx_memberships_scope ON memberships(scope_type, scope_id); diff --git a/backend/tests/test_collection_scoped_serve.py b/backend/tests/test_collection_scoped_serve.py index 7c4d5d2..443cad8 100644 --- a/backend/tests/test_collection_scoped_serve.py +++ b/backend/tests/test_collection_scoped_serve.py @@ -122,6 +122,11 @@ def test_scoped_propose_writes_into_collection_subfolder(app_with_fake_gitea): with TestClient(app) as client: _add_features_collection() provision_user_row(user_id=3, login="alice", role="contributor") + # §22 S3: an explicitly-created collection requires an explicit scope + # grant to write (the grandfathered baseline covers only `default`). + db.conn().execute( + "INSERT OR REPLACE INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('collection', 'features', 3, 'contributor')") sign_in_as(client, user_id=3, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test") r = client.post( diff --git a/backend/tests/test_migration_030_global_scope.py b/backend/tests/test_migration_030_global_scope.py new file mode 100644 index 0000000..ec5db73 --- /dev/null +++ b/backend/tests/test_migration_030_global_scope.py @@ -0,0 +1,91 @@ +"""Migration 030 — the global-scope grant (§22 three-tier S3). + +Proves: `memberships.scope_type` now admits 'global' alongside 'project' and +'collection' (§B.2's four-layer resolver), existing rows survive the rebuild, +and the UNIQUE(scope_type, scope_id, user_id) shape is preserved. +Template: test_migration_029_collections.py. +""" +from __future__ import annotations + +import sqlite3 +import tempfile +from pathlib import Path + +import pytest + +from app import db + + +class _Cfg: + def __init__(self, path): + self.database_path = path + + +def _fresh_db(): + d = tempfile.mkdtemp() + path = Path(d) / "t.db" + db.run_migrations(_Cfg(str(path))) + return db.connect(str(path)) + + +def _add_user(conn, uid, login): + conn.execute( + "INSERT INTO users (id, gitea_id, gitea_login, display_name, role) " + "VALUES (?, ?, ?, ?, 'contributor')", + (uid, uid, login, login.capitalize()), + ) + + +def test_global_scope_type_is_accepted(): + conn = _fresh_db() + _add_user(conn, 1, "cleo") + # global grant — the new tier — is accepted. + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('global', '*', 1, 'contributor')" + ) + row = conn.execute( + "SELECT scope_type, scope_id, role FROM memberships WHERE user_id = 1" + ).fetchone() + assert row["scope_type"] == "global" + assert row["scope_id"] == "*" + assert row["role"] == "contributor" + + +def test_project_and_collection_scopes_still_accepted(): + conn = _fresh_db() + _add_user(conn, 1, "ben") + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('project', 'default', 1, 'owner')" + ) + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('collection', 'default', 1, 'contributor')" + ) + n = conn.execute("SELECT COUNT(*) AS n FROM memberships WHERE user_id = 1").fetchone()["n"] + assert n == 2 + + +def test_unknown_scope_type_still_rejected(): + conn = _fresh_db() + _add_user(conn, 1, "x") + with pytest.raises(sqlite3.IntegrityError): + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('deployment', '*', 1, 'owner')" + ) + + +def test_one_global_grant_per_user(): + conn = _fresh_db() + _add_user(conn, 1, "cleo") + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('global', '*', 1, 'contributor')" + ) + with pytest.raises(sqlite3.IntegrityError): + conn.execute( + "INSERT INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('global', '*', 1, 'owner')" + ) diff --git a/backend/tests/test_project_scoped_propose.py b/backend/tests/test_project_scoped_propose.py index 7cabaf7..e6aa679 100644 --- a/backend/tests/test_project_scoped_propose.py +++ b/backend/tests/test_project_scoped_propose.py @@ -28,6 +28,13 @@ def test_propose_into_second_project_lands_scoped(app_with_fake_gitea): with TestClient(app) as client: _register_ecomm(fake) provision_user_row(user_id=3, login="alice", role="contributor") + # §22 S3: the grandfathered implicit-public baseline covers only the N=1 + # `default` collection; a second project requires an explicit scope grant + # to write. Grant alice contributor at the ecomm project. + from app import db + db.conn().execute( + "INSERT OR REPLACE INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES ('project', 'ecomm', 3, 'contributor')") sign_in_as(client, user_id=3, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test") r = client.post("/api/projects/ecomm/rfcs/propose", json={ diff --git a/backend/tests/test_s3_scope_roles_vertical.py b/backend/tests/test_s3_scope_roles_vertical.py new file mode 100644 index 0000000..2b6f018 --- /dev/null +++ b/backend/tests/test_s3_scope_roles_vertical.py @@ -0,0 +1,287 @@ +"""Slice S3 — scope-role enforcement + collection-grain visibility (@S3). + +The acceptance gate for S3 is "every Part C.1 scenario passes" (the design doc +docs/design/2026-06-05-three-tier-projects-collections.md, §C.1, tagged @S3) plus +the operator's S3 visibility requirements (a collection settable public/hidden; +hidden = visible to project/global scope contributors but not the public; a +collection's visibility may be set only as strict or stricter than its project). + +The §B.2 resolver folds four layers — global → project → collection → per-entry — +most-permissively, with no negative override. The scenarios below are exercised +directly against the resolver/gate helpers, and the visibility ones additionally +through the HTTP surface. + +Background (C.1): a deployment with a project "ohm" owning collections "model" +(document) and "features" (bdd); a second project "acme" with collection +"specs". Plus a hidden ("gated") collection "secret" under ohm for the +hidden-from-public scenarios. +""" +from __future__ import annotations + +from fastapi.testclient import TestClient + +from test_propose_vertical import ( # noqa: F401 — fixtures land via import + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _su(user_id: int, login: str, role: str = "contributor", *, state: str = "granted"): + from app import auth + + return auth.SessionUser( + user_id=user_id, gitea_id=user_id, gitea_login=login, + display_name=login.capitalize(), email=f"{login}@test", avatar_url="", + role=role, permission_state=state, + ) + + +def _project(pid: str, visibility: str = "public", content_repo: str = "meta") -> None: + from app import db + + db.conn().execute( + "INSERT OR REPLACE INTO projects (id, name, content_repo, visibility, updated_at) " + "VALUES (?, ?, ?, ?, datetime('now'))", + (pid, pid.capitalize(), content_repo, visibility), + ) + + +def _collection(cid: str, project_id: str, *, ctype: str = "document", + visibility: str = "public", subfolder: str | None = None) -> None: + from app import db + + db.conn().execute( + "INSERT OR REPLACE INTO collections " + "(id, project_id, type, subfolder, initial_state, visibility, name, created_at, updated_at) " + "VALUES (?, ?, ?, ?, 'super-draft', ?, ?, datetime('now'), datetime('now'))", + (cid, project_id, ctype, subfolder if subfolder is not None else cid, + visibility, cid.capitalize()), + ) + + +def _grant(scope_type: str, scope_id: str, user_id: int, role: str) -> None: + from app import db + + db.conn().execute( + "INSERT OR REPLACE INTO memberships (scope_type, scope_id, user_id, role) " + "VALUES (?, ?, ?, ?)", + (scope_type, scope_id, user_id, role), + ) + + +def _seed_world() -> None: + """The C.1 background plus a hidden collection and a second project.""" + _project("ohm", "public") + _collection("ohm", "ohm", subfolder="") # ohm's structural default + _collection("model", "ohm", ctype="document") + _collection("features", "ohm", ctype="bdd") + _collection("secret", "ohm", visibility="gated") # hidden from public + _project("acme", "public") + _collection("specs", "acme", ctype="specification") + # the cast + for uid, login in [(1, "ada"), (2, "ben"), (3, "cleo"), (4, "dan"), + (5, "eve"), (6, "fay"), (7, "gil"), (8, "hana")]: + provision_user_row(user_id=uid, login=login, role="contributor") + _grant("collection", "model", 1, "contributor") # ada + _grant("project", "ohm", 2, "contributor") # ben + _grant("global", "*", 3, "contributor") # cleo + _grant("collection", "features", 4, "owner") # dan + _grant("project", "ohm", 5, "owner") # eve + _grant("collection", "model", 6, "contributor") # fay (+ project owner below) + _grant("project", "ohm", 6, "owner") # fay + _grant("project", "ohm", 7, "contributor") # gil + # hana (8): no grant. + + +# --------------------------------------------------------------------------- +# C.1 — role usage: inheritance and the most-permissive union +# --------------------------------------------------------------------------- + + +def test_c1_1_collection_contributor_proposes_only_in_that_collection(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + ada = _su(1, "ada") + # may submit a new entry in ohm/model + assert auth.can_contribute_in_collection(ada, "model") is True + # ohm/features is read-only and propose is not offered + assert auth.can_read_collection(ada, "features") is True + assert auth.can_contribute_in_collection(ada, "features") is False + + +def test_c1_2_project_contributor_proposes_in_every_collection(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + ben = _su(2, "ben") + assert auth.can_contribute_in_collection(ben, "model") is True + assert auth.can_contribute_in_collection(ben, "features") is True + # a collection added later is writable with no new grant + _collection("roadmap", "ohm", ctype="document") + assert auth.can_contribute_in_collection(ben, "roadmap") is True + + +def test_c1_3_global_contributor_proposes_everywhere(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + cleo = _su(3, "cleo") + assert auth.can_contribute_in_collection(cleo, "model") is True + assert auth.can_contribute_in_collection(cleo, "specs") is True # acme + + +def test_c1_4_collection_owner_administers_one_collection_only(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + dan = _su(4, "dan") + # graduate / mark-reviewed / manage membership in ohm/features + assert auth.is_collection_superuser(dan, "features") is True + # but not change ohm project settings + assert auth.is_project_superuser(dan, "ohm") is False + assert auth.can_create_collection(dan, "ohm") is False + # and not act on entries in ohm/model + assert auth.is_collection_superuser(dan, "model") is False + assert auth.can_contribute_in_collection(dan, "model") is False + + +def test_c1_5_project_owner_administers_all_collections_and_creates_more(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + eve = _su(5, "eve") + assert auth.is_collection_superuser(eve, "model") is True + assert auth.is_collection_superuser(eve, "features") is True + assert auth.is_project_superuser(eve, "ohm") is True # edit project settings + assert auth.can_create_collection(eve, "ohm") is True # create a new collection + + +def test_c1_6_most_permissive_union_higher_grant_wins(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + fay = _su(6, "fay") + # collection RFC Contributor at model + project Owner at ohm → acts as Owner in model + assert auth.effective_scope_role(fay, "model") == "owner" + assert auth.is_collection_superuser(fay, "model") is True + + +def test_c1_7_no_negative_override(app_with_fake_gitea): + from app import auth, db + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + gil = _su(7, "gil") + # gil can propose in ohm/model via the project grant… + assert auth.can_contribute_in_collection(gil, "model") is True + # …and there is no collection-scope row to remove at model while keeping + # the project grant (a child cannot subtract a parent grant). + row = db.conn().execute( + "SELECT 1 FROM memberships WHERE user_id = 7 AND scope_type = 'collection' AND scope_id = 'model'" + ).fetchone() + assert row is None + + +def test_c1_8_granted_account_no_role_sees_only_public(app_with_fake_gitea): + from app import auth + app, _ = app_with_fake_gitea + with TestClient(app): + _seed_world() + hana = _su(8, "hana") + # may read public collections + assert auth.can_read_collection(hana, "model") is True + # but is not offered the propose action anywhere (no scope role; the + # grandfathered baseline covers only the N=1 `default` collection) + assert auth.can_contribute_in_collection(hana, "model") is False + assert auth.can_contribute_in_collection(hana, "features") is False + assert auth.can_contribute_in_collection(hana, "specs") is False + # gated (hidden) collections do not appear for her + assert auth.can_read_collection(hana, "secret") is False + + +# --------------------------------------------------------------------------- +# Collection-grain visibility — the operator's S3 requirements +# --------------------------------------------------------------------------- + + +def test_hidden_collection_invisible_to_public_visible_to_scope_holder(app_with_fake_gitea): + """A gated collection is omitted from the directory and 404s on read for the + public, yet is listed + readable for a scope-role contributor.""" + app, _ = app_with_fake_gitea + with TestClient(app) as client: + _seed_world() + # anonymous: the gated 'secret' collection is not listed, and 404s. + listed = {c["id"] for c in client.get("/api/projects/ohm/collections").json()["items"]} + assert "secret" not in listed + assert "model" in listed # public ones still listed + assert client.get("/api/projects/ohm/collections/secret").status_code == 404 + assert client.get("/api/projects/ohm/collections/secret/rfcs").status_code == 404 + + # ben (project contributor) sees and reads it. + sign_in_as(client, user_id=2, gitea_login="ben", display_name="Ben", role="contributor") + listed2 = {c["id"] for c in client.get("/api/projects/ohm/collections").json()["items"]} + assert "secret" in listed2 + assert client.get("/api/projects/ohm/collections/secret").status_code == 200 + assert client.get("/api/projects/ohm/collections/secret/rfcs").status_code == 200 + + # hana (granted, no role) is back to the public view. + sign_in_as(client, user_id=8, gitea_login="hana", display_name="Hana", role="contributor") + listed3 = {c["id"] for c in client.get("/api/projects/ohm/collections").json()["items"]} + assert "secret" not in listed3 + assert client.get("/api/projects/ohm/collections/secret").status_code == 404 + + +def test_collection_visibility_strictness_validated_at_create(app_with_fake_gitea): + """A collection may be created only as strict or stricter than its project; + a looser request is refused (422). On a gated project, a 'public' collection + is rejected.""" + app, _ = app_with_fake_gitea + with TestClient(app) as client: + _project("locked", "gated") + _collection("locked", "locked", subfolder="", visibility="gated") + # eve is a deployment owner here to clear the create-authority gate; + # the strictness check fires regardless. + provision_user_row(user_id=9, login="root", role="owner") + sign_in_as(client, user_id=9, gitea_login="root", display_name="Root", role="owner") + r = client.post("/api/projects/locked/collections", json={ + "collection_id": "wideopen", "type": "document", "visibility": "public", + }) + assert r.status_code == 422, r.text + assert "looser" in r.json()["detail"] + + +def test_create_collection_allowed_for_project_owner_not_plain_contributor(app_with_fake_gitea): + """§B.1: a project-scope Owner may create a collection; a plain granted + contributor with no project/global grant may not (403).""" + app, fake = app_with_fake_gitea + with TestClient(app) as client: + _seed_world() + # gil is only a project *contributor* on ohm — per §B.1 a project-scope + # contributor CAN create collections (the project-level create + # affordance). A collection-scope grant cannot. + sign_in_as(client, user_id=7, gitea_login="gil", display_name="Gil", role="contributor") + r_ok = client.post("/api/projects/ohm/collections", json={ + "collection_id": "fromgil", "type": "document", "visibility": "public", + }) + assert r_ok.status_code in (200, 502), r_ok.text # past the authz gate + + # ada holds only a *collection*-scope grant (at model) — no create right. + sign_in_as(client, user_id=1, gitea_login="ada", display_name="Ada", role="contributor") + r_no = client.post("/api/projects/ohm/collections", json={ + "collection_id": "fromada", "type": "document", "visibility": "public", + }) + assert r_no.status_code == 403, r_no.text diff --git a/docs/design/2026-06-05-three-tier-projects-collections.md b/docs/design/2026-06-05-three-tier-projects-collections.md index a152e4c..4592e3f 100644 --- a/docs/design/2026-06-05-three-tier-projects-collections.md +++ b/docs/design/2026-06-05-three-tier-projects-collections.md @@ -559,12 +559,26 @@ means the deployment runs and either gains a capability or provably loses none one and it is navigable + proposable. **Completes:** `@S2` (anonymous reader of an empty collection catalog). -- **S3 — Scope-role enforcement.** The four-layer most-permissive resolver - (§B.2) over `{owner, contributor}` grants at `{global, project, collection}`, - with grants applied administratively (DB / admin endpoint); every write gate - re-checked under the collection axis. **Usable end-state:** a user granted - RFC Contributor at a scope can contribute across exactly that subtree, and - Owners administer their subtree. **Completes:** `@S3` (all of C.1 — role usage, +- **S3 — Scope-role enforcement.** *(Shipped v0.42.0.)* The four-layer + most-permissive resolver (§B.2) over `{owner, contributor}` grants at + `{global, project, collection}` (migration 030 adds the `global` scope_type), + with grants applied administratively (DB / the Owner-authorized create + surface); every write gate re-checked under the collection axis. **Plus the + operator's S3 visibility requirements:** collection-grain visibility is + enforced — a `gated` collection is hidden from the public (404, omitted from + the directory) yet visible to scope-role contributors; a collection's + visibility may be set only as strict or stricter than its project's + (`public` < `unlisted` < `gated`). **Keystone reconciliation (session 0076):** + §B.1/§B.3's literal "deployment contributor = global RFC Contributor" + contradicted the C.1 "hana" scenario and the M2 implicit-public baseline; + resolved as — a plain granted account is a granted *account*, not a + write-everywhere global role; "global RFC Contributor" is an explicit + `scope_type='global'` grant; the implicit-public write baseline is + grandfathered onto the migration-seeded `default` collection only (N=1 + preserved). *Flag for the SPEC merge (S6): reinterprets §B.1/§B.3.* **Usable + end-state:** a user granted RFC Contributor at a scope can contribute across + exactly that subtree, Owners administer their subtree, and a collection can be + hidden from the public. **Completes:** `@S3` (all of C.1 — role usage, inheritance, union, no-negative-override). - **S4 — Invitation surfaces + role-aware empty states.** The invite UI diff --git a/frontend/package.json b/frontend/package.json index 3cd1058..1bb033b 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "rfc-app-frontend", "private": true, - "version": "0.41.0", + "version": "0.42.0", "type": "module", "scripts": { "dev": "vite", -- 2.39.5