Files
rfc-app/docs/design/plans/2026-06-05-s2-second-collection.md
T
Ben Stull 599e7018f6 §22 S2: implementation plan — create & navigate a second collection
Plan for slice S2 of the three-tier (deployment→project→collection) refactor.
Completes acceptance @S2 (C3.6). See
docs/design/2026-06-05-three-tier-projects-collections.md Part E.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 12:55:56 -07:00

51 KiB

§22 S2 — Create & navigate a second collection — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Let a deployment admin create a second RFC collection beside a project's default one, navigate to it, and propose into it — completing acceptance scenario C3.6 (@S2): an anonymous visitor landing on an empty public collection's catalog sees an empty catalog with no propose action and a sign-in prompt.

Architecture: S1 shipped the collection grain (migration 029): a collections table, entries keyed (collection_id, slug), one seeded default collection per project. S2 makes collections plural and navigable. Backend: the registry mirror learns to read .collection.yaml manifests inside each project's content repo; the corpus mirror becomes collection-grained (reads each collection's <subfolder>/rfcs/); a bot-commit-wrapped create-collection endpoint commits a manifest; collection-scoped list/get/propose endpoints land under /api/projects/:id/collections/:cid/…. Frontend: /p/<project>/ becomes a collection directory (1 visible → redirect, keeping S1; 2+ → list); the Catalog rail + entry views read the active :collectionId from the route.

Tech Stack: FastAPI + sqlite (backend, backend/app/), pytest (backend/tests/), React + react-router + vitest (frontend/src/), Gitea content repos via the Bot wrapper.

Content-repo layout convention (decided this slice): a collection with subfolder S stores its manifest at S/.collection.yaml and its entries at S/rfcs/. The default collection has subfolder '' → manifest is the project's projects.yaml entry (no .collection.yaml), entries at rfcs/. This keeps the shipped default corpus path unchanged.

Authorization (this slice): create-collection is gated to a deployment owner/admin (auth.require_admin). Scoped {owner, contributor} roles at the collection axis land in S3 — out of scope here.


File structure

Backend (modify):

  • backend/app/registry.py — add .collection.yaml discovery + upsert to the mirror.
  • backend/app/collections.py — add list_collections, get_collection, subfolder_of.
  • backend/app/cache.py — make refresh_meta_repo iterate a project's collections.
  • backend/app/bot.py — add create_collection (commit <subfolder>/.collection.yaml).
  • backend/app/api.py — add collection-scoped list/get/propose; refactor propose to take a collection.
  • backend/app/api_collections.pycreate: GET list/one + POST create-collection endpoints.
  • backend/app/main.py — mount the new router.

Frontend (modify):

  • frontend/src/lib/entryPaths.js — add useCollectionId().
  • frontend/src/api.js — collection-scoped listRFCs/getRFC/proposeRFC; add listCollections, createCollection.
  • frontend/src/components/Catalog.jsx — read active collection; scope fetches + links.
  • frontend/src/App.jsx/p/<project>/CollectionDirectory; thread collection into propose.
  • frontend/src/components/CollectionDirectory.jsxcreate: the directory page.

Tests (create):

  • backend/tests/test_collection_registry.py.collection.yaml mirror.
  • backend/tests/test_collection_create_vertical.py — create → mirror → list round-trip.
  • backend/tests/test_collection_scoped_serve.py — list/get/propose under a named collection.
  • frontend/src/lib/entryPaths.test.js (extend) + frontend/src/components/CollectionDirectory.test.jsx.

Task 1: Registry mirror reads .collection.yaml manifests

Files:

  • Modify: backend/app/registry.py
  • Test: backend/tests/test_collection_registry.py

The mirror today upserts only the default collection from projects.yaml. Add a parse_collection_manifest(text) (pure) and extend refresh_registry to walk each project's content repo root, read every <subdir>/.collection.yaml, and upsert a collection row keyed by the subdir name.

  • Step 1: Write the failing test for the pure parser.
# backend/tests/test_collection_registry.py
import pytest
from app import registry


def test_parse_collection_manifest_minimal():
    doc = registry.parse_collection_manifest("type: bdd\n")
    assert doc.type == "bdd"
    # §22.4b: bdd defaults to 'active'; visibility inherits (None == inherit).
    assert doc.initial_state == "active"
    assert doc.visibility is None
    assert doc.name is None


def test_parse_collection_manifest_full():
    doc = registry.parse_collection_manifest(
        "type: document\nvisibility: public\ninitial_state: active\nname: Model\n"
    )
    assert (doc.type, doc.visibility, doc.initial_state, doc.name) == (
        "document", "public", "active", "Model",
    )


def test_parse_collection_manifest_rejects_bad_type():
    with pytest.raises(registry.RegistryError):
        registry.parse_collection_manifest("type: nonsense\n")
  • Step 2: Run it, verify it fails.

Run: cd backend && python -m pytest tests/test_collection_registry.py -q Expected: FAIL (parse_collection_manifest not defined).

  • Step 3: Implement parse_collection_manifest + CollectionEntry in registry.py.
# Add near ProjectEntry (after line 56).
@dataclass
class CollectionEntry:
    type: str
    visibility: str | None  # None == inherit the project's visibility
    initial_state: str
    name: str | None


def parse_collection_manifest(text: str) -> CollectionEntry:
    """Parse + validate a `.collection.yaml`. Pure. Raises RegistryError."""
    raw = yaml.safe_load(text) or {}
    if not isinstance(raw, dict):
        raise RegistryError("collection manifest must be a mapping")
    ctype = str(raw.get("type") or "").strip()
    if ctype not in VALID_TYPES:
        raise RegistryError(f"collection has invalid type {ctype!r}")
    vis = raw.get("visibility")
    if vis is not None:
        vis = str(vis).strip()
        if vis not in VALID_VISIBILITY:
            raise RegistryError(f"collection has invalid visibility {vis!r}")
    initial_state = str(
        raw.get("initial_state") or _TYPE_DEFAULT_INITIAL_STATE[ctype]
    ).strip()
    if initial_state not in VALID_INITIAL_STATE:
        raise RegistryError(f"collection has invalid initial_state {initial_state!r}")
    name = raw.get("name")
    name = str(name).strip() if name else None
    return CollectionEntry(ctype, vis, initial_state, name)
  • Step 4: Run the parser tests, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_registry.py -q Expected: PASS (3 tests).

  • Step 5: Write the failing mirror test (uses a fake Gitea exposing the content repo).
# Append to backend/tests/test_collection_registry.py
import asyncio


class _FakeGitea:
    """Minimal Gitea stub: projects.yaml in the registry repo + a content repo
    holding one `.collection.yaml` under `features/`."""
    def __init__(self, projects_yaml, repo_tree):
        self._projects_yaml = projects_yaml
        self._repo_tree = repo_tree  # {repo: {path: text}}

    async def get_contents(self, org, repo, path, ref="main"):
        import base64
        if path == "projects.yaml":
            enc = base64.b64encode(self._projects_yaml.encode()).decode()
            return {"type": "file", "content": enc, "sha": "deadbeef"}
        text = self._repo_tree.get(repo, {}).get(path)
        if text is None:
            return None
        return {"type": "file",
                "content": base64.b64encode(text.encode()).decode(), "sha": "c0ffee"}

    async def list_dir(self, org, repo, path, ref="main"):
        # Root listing: surface each top-level subdir as a 'dir' entry.
        names = set()
        for p in self._repo_tree.get(repo, {}):
            head = p.split("/", 1)[0]
            if "/" in p:
                names.add(head)
        return [{"type": "dir", "name": n, "path": n} for n in sorted(names)]


def test_refresh_registry_mirrors_named_collection(app_with_db, config):
    gitea = _FakeGitea(
        projects_yaml=(
            "deployment:\n  name: Ohm\n  tagline: t\n"
            "projects:\n  - id: ohm\n    name: Ohm\n    type: document\n"
            "    content_repo: ohm-rfc\n    visibility: public\n"
        ),
        repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\nname: Features\n"}},
    )
    asyncio.run(registry.refresh_registry(config, gitea))
    from app import db
    row = db.conn().execute(
        "SELECT type, subfolder, name, project_id FROM collections WHERE id = 'features'"
    ).fetchone()
    assert row is not None
    assert (row["type"], row["subfolder"], row["project_id"]) == ("bdd", "features", "ohm")

Use the existing test app/db fixture pattern — mirror app_with_db / config fixtures from backend/tests/test_registry.py. If that test uses different fixture names, copy its setup verbatim here.

  • Step 6: Run it, verify it fails (features collection not mirrored).

Run: cd backend && python -m pytest tests/test_collection_registry.py::test_refresh_registry_mirrors_named_collection -q Expected: FAIL (row is None).

  • Step 7: Extend refresh_registry to discover + upsert named collections.

After apply_registry(...) in refresh_registry (registry.py line 215), add:

    # §22 S2: named collections are declared by `.collection.yaml` manifests
    # inside each project's content repo (the default collection comes from
    # projects.yaml above). Walk each content repo root; a subdir carrying a
    # manifest becomes a collection keyed by the subdir name.
    for proj in doc.projects:
        try:
            items = await gitea.list_dir(config.gitea_org, proj.content_repo, "", ref="main")
        except Exception as e:  # GiteaError or transport — tolerate, keep last-good
            log.warning("registry: cannot list %s root: %s", proj.content_repo, e)
            continue
        for it in items:
            if it.get("type") != "dir":
                continue
            subdir = it["name"]
            manifest = await gitea.get_contents(
                config.gitea_org, proj.content_repo, f"{subdir}/.collection.yaml", ref="main"
            )
            if not manifest or manifest.get("type") != "file":
                continue
            mtext = base64.b64decode(manifest["content"]).decode("utf-8")
            try:
                ce = parse_collection_manifest(mtext)
            except RegistryError as e:
                log.error("registry: bad manifest %s/%s: %s", proj.content_repo, subdir, e)
                continue
            _upsert_named_collection(proj, subdir, ce, sha)

And add the upsert helper (after apply_registry):

def _upsert_named_collection(proj: ProjectEntry, subdir: str, ce: CollectionEntry, sha: str) -> None:
    """Upsert one named collection. Type is immutable (§22.4a): a type change
    against an existing row is refused (logged, not applied). Visibility None
    inherits the project's visibility."""
    visibility = ce.visibility or proj.visibility
    with db.tx() as conn:
        existing = conn.execute(
            "SELECT type FROM collections WHERE id = ?", (subdir,)
        ).fetchone()
        if existing is not None and existing["type"] != ce.type:
            log.error("registry: refusing immutable type change on collection %s (%s -> %s)",
                      subdir, existing["type"], ce.type)
            return
        conn.execute(
            """
            INSERT INTO collections
                (id, project_id, type, subfolder, initial_state, visibility, name, registry_sha, updated_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now'))
            ON CONFLICT(id) DO UPDATE SET
                project_id = excluded.project_id,
                initial_state = excluded.initial_state,
                visibility = excluded.visibility,
                name = excluded.name,
                registry_sha = excluded.registry_sha,
                updated_at = datetime('now')
            """,
            (subdir, proj.id, ce.type, subdir, ce.initial_state, visibility, ce.name, sha),
        )
  • Step 8: Run all of Task 1's tests, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_registry.py -q Expected: PASS (4 tests).

  • Step 9: Commit.
git add backend/app/registry.py backend/tests/test_collection_registry.py
git commit -m "§22 S2: registry mirror reads .collection.yaml manifests"

Task 2: Collection read helpers — list_collections, get_collection, subfolder_of

Files:

  • Modify: backend/app/collections.py

  • Test: backend/tests/test_collection_create_vertical.py (first assertions)

  • Step 1: Write the failing test.

# backend/tests/test_collection_create_vertical.py
from app import db, collections as collections_mod


def _seed(project_id="ohm"):
    db.conn().execute(
        "INSERT INTO projects (id, name, content_repo, visibility, updated_at) "
        "VALUES (?, 'Ohm', 'ohm-rfc', 'public', datetime('now'))", (project_id,))
    for cid, sub, vis, name in [
        ("default", "", "public", "Model"),
        ("features", "features", "public", "Features"),
        ("secret", "secret", "unlisted", "Secret"),
    ]:
        db.conn().execute(
            "INSERT INTO collections (id, project_id, type, subfolder, initial_state, "
            "visibility, name, created_at, updated_at) VALUES (?,?, 'document', ?, "
            "'super-draft', ?, ?, datetime('now'), datetime('now'))",
            (cid, project_id, sub, vis, name))


def test_list_collections_excludes_unlisted(app_with_db):
    _seed()
    ids = [c["id"] for c in collections_mod.list_collections("ohm", include_unlisted=False)]
    assert ids == ["default", "features"]  # 'secret' (unlisted) omitted


def test_get_collection_and_subfolder(app_with_db):
    _seed()
    assert collections_mod.get_collection("features")["name"] == "Features"
    assert collections_mod.subfolder_of("features") == "features"
    assert collections_mod.subfolder_of("default") == ""
    assert collections_mod.get_collection("nope") is None
  • Step 2: Run it, verify it fails.

Run: cd backend && python -m pytest tests/test_collection_create_vertical.py -q Expected: FAIL (list_collections not defined).

  • Step 3: Implement the helpers in collections.py.
def get_collection(collection_id: str) -> dict | None:
    row = db.conn().execute(
        "SELECT id, project_id, type, subfolder, initial_state, visibility, name "
        "FROM collections WHERE id = ?", (collection_id,)
    ).fetchone()
    return dict(row) if row else None


def subfolder_of(collection_id: str) -> str:
    row = db.conn().execute(
        "SELECT subfolder FROM collections WHERE id = ?", (collection_id,)
    ).fetchone()
    return (row["subfolder"] if row else "") or ""


def list_collections(project_id: str, include_unlisted: bool = False) -> list[dict]:
    """Collections in a project, default first then by name. `unlisted` is
    omitted from enumeration unless include_unlisted (a direct-id read)."""
    rows = db.conn().execute(
        "SELECT id, project_id, type, subfolder, initial_state, visibility, name "
        "FROM collections WHERE project_id = ? ORDER BY (id != 'default'), name, id",
        (project_id,),
    ).fetchall()
    out = []
    for r in rows:
        if not include_unlisted and r["visibility"] == "unlisted":
            continue
        out.append(dict(r))
    return out
  • Step 4: Run it, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_create_vertical.py -q Expected: PASS (2 tests).

  • Step 5: Commit.
git add backend/app/collections.py backend/tests/test_collection_create_vertical.py
git commit -m "§22 S2: collection read helpers (list/get/subfolder)"

Task 3: Corpus mirror is collection-grained

Files:

  • Modify: backend/app/cache.py:37-97
  • Test: backend/tests/test_collection_scoped_serve.py (first assertion)

refresh_meta_repo reads only rfcs/ (the default collection). Make it iterate a project's collections and read each collection's <subfolder>/rfcs/, keying cached_rfcs by collection_id.

  • Step 1: Write the failing test.
# backend/tests/test_collection_scoped_serve.py
import asyncio
from app import db, cache


class _CorpusGitea:
    """A content repo with entries under both the default `rfcs/` and a named
    collection's `features/rfcs/`."""
    def __init__(self, tree):
        self._tree = tree  # {path: text}

    async def list_dir(self, org, repo, path, ref="main"):
        out = []
        prefix = (path.rstrip("/") + "/") if path else ""
        for p in self._tree:
            if p.startswith(prefix) and "/" not in p[len(prefix):]:
                out.append({"type": "file", "name": p.split("/")[-1], "path": p})
        return out

    async def read_file(self, org, repo, path, ref="main"):
        t = self._tree.get(path)
        return (t, "sha-" + path) if t is not None else None


def _entry_md(slug, title):
    return f"---\nslug: {slug}\ntitle: {title}\nstate: active\n---\nbody\n"


def test_mirror_keys_entries_by_collection(app_with_db, config):
    db.conn().execute(
        "INSERT INTO projects (id, name, content_repo, visibility, updated_at) "
        "VALUES ('ohm','Ohm','ohm-rfc','public', datetime('now'))")
    for cid, sub in [("default", ""), ("features", "features")]:
        db.conn().execute(
            "INSERT INTO collections (id, project_id, type, subfolder, initial_state, "
            "visibility, created_at, updated_at) VALUES (?, 'ohm','document',?, "
            "'super-draft','public', datetime('now'), datetime('now'))", (cid, sub))
    gitea = _CorpusGitea({
        "rfcs/a.md": _entry_md("a", "Default A"),
        "features/rfcs/b.md": _entry_md("b", "Feature B"),
    })
    asyncio.run(cache.refresh_meta_repo(config, gitea))
    got = {(r["collection_id"], r["slug"]) for r in
           db.conn().execute("SELECT collection_id, slug FROM cached_rfcs")}
    assert got == {("default", "a"), ("features", "b")}
  • Step 2: Run it, verify it fails (entry b mirrored to wrong/absent collection).

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q Expected: FAIL.

  • Step 3: Rework refresh_meta_repo + _refresh_project_corpus in cache.py.

Replace the body of _refresh_project_corpus so it loops the project's collections:

async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None:
    # §22 S2: the corpus grain is the collection. Mirror each collection of the
    # project from its `<subfolder>/rfcs/` directory, keying cached_rfcs by the
    # collection id. The default collection (subfolder '') reads `rfcs/`.
    from . import collections as collections_mod
    for col in collections_mod.list_collections(project_id, include_unlisted=True):
        collection_id = col["id"]
        sub = col["subfolder"] or ""
        rfcs_dir = f"{sub}/rfcs" if sub else "rfcs"
        try:
            files = await gitea.list_dir(org, repo, rfcs_dir, ref="main")
        except GiteaError as e:
            log.warning("refresh_meta_repo: %s/%s: cannot list %s: %s",
                        project_id, collection_id, rfcs_dir, e)
            continue
        seen_slugs: set[str] = set()
        for f in files:
            if f.get("type") != "file" or not f.get("name", "").endswith(".md"):
                continue
            result = await gitea.read_file(org, repo, f["path"], ref="main")
            if not result:
                continue
            text, sha = result
            try:
                entry = entry_mod.parse(text)
            except Exception as parse_err:
                log.warning("refresh_meta_repo: %s/%s: skipping %s: %s",
                            project_id, collection_id, f["path"], parse_err)
                continue
            if not entry.slug:
                continue
            seen_slugs.add(entry.slug)
            _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id)
        existing = {row["slug"] for row in db.conn().execute(
            "SELECT slug FROM cached_rfcs WHERE collection_id = ?", (collection_id,))}
        for missing in existing - seen_slugs:
            log.info("refresh_meta_repo: %s/%s/%s no longer present — leaving cache row",
                     project_id, collection_id, missing)

refresh_meta_repo itself (the project loop) is unchanged — it still calls _refresh_project_corpus(org, id, content_repo, gitea) per project.

  • Step 4: Run it, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q Expected: PASS.

  • Step 5: Run the full backend suite — the default-collection path must be unchanged.

Run: cd backend && python -m pytest -q Expected: PASS (all green; N=1 default path intact).

  • Step 6: Commit.
git add backend/app/cache.py backend/tests/test_collection_scoped_serve.py
git commit -m "§22 S2: corpus mirror reads each collection's <subfolder>/rfcs/"

Task 4: Collection-scoped serve + propose endpoints (backend)

Files:

  • Modify: backend/app/api.py — refactor propose to accept a collection; add scoped list/get/propose.
  • Test: backend/tests/test_collection_scoped_serve.py (append)

The shipped _propose_into_project hardcodes collection_id = default_collection_id(project_id) (api.py:973). Generalize it to accept an explicit collection, and write the entry into that collection's <subfolder>/rfcs/.

  • Step 1: Write the failing test (scoped list + propose targets the named collection).
# Append to backend/tests/test_collection_scoped_serve.py
def test_scoped_list_returns_only_that_collection(client_with_corpus):
    client, _ = client_with_corpus  # fixture seeds 'default'+'features' w/ entries a,b
    r = client.get("/api/projects/ohm/collections/features/rfcs")
    assert r.status_code == 200
    assert [i["slug"] for i in r.json()["items"]] == ["b"]


def test_scoped_propose_writes_into_collection_subfolder(client_with_corpus):
    client, fake_bot = client_with_corpus
    r = client.post("/api/projects/ohm/collections/features/rfcs/propose",
                    json={"title": "New", "slug": "newb", "pitch": "x", "tags": []})
    assert r.status_code == 200
    # The bot was asked to write under features/rfcs/, not rfcs/.
    assert fake_bot.last_meta_path_prefix == "features/rfcs"

Build client_with_corpus from the existing app_with_fake_gitea / propose-vertical fixture in backend/tests/test_propose_vertical.py; extend its fake bot to record the subfolder it was handed (see Step 3). If the propose-vertical fixture is structured differently, follow its shape and adapt these two asserts.

  • Step 2: Run it, verify it fails.

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py -q Expected: FAIL (route 404 / propose ignores collection).

  • Step 3: Generalize _propose_into_project_propose_into_collection in api.py.

Change the signature and the two collection-dependent lines. The current helper (api.py:959) resolves collection_id from the project default; instead take it as a parameter and prefix the bot write path with the collection's subfolder:

    async def _propose_into_collection(
        project_id: str, collection_id: str, payload: ProposeBody, user
    ) -> dict[str, Any]:
        if not auth.can_contribute_in_project(user, project_id):
            raise HTTPException(403, "You do not have contribute access to this project")
        slug = payload.slug.strip().lower()
        if not entry_mod.is_valid_slug(slug):
            raise HTTPException(422, "Slug must be lowercase letters, digits, and dashes")
        clash = db.conn().execute(
            "SELECT 1 FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id)
        ).fetchone()
        if clash:
            raise HTTPException(409, f"Slug `{slug}` is already taken")
        # ...unchanged idea_clash / entry build...
        landing_state = (
            "active" if collections_mod.collection_initial_state(collection_id) == "active"
            else "super-draft"
        )
        # ...build `entry`, `contents`, `pr_title`, `pr_description` unchanged...
        subfolder = collections_mod.subfolder_of(collection_id)
        rfcs_prefix = f"{subfolder}/rfcs" if subfolder else "rfcs"
        pr = await bot.open_idea_pr(
            user.as_actor(),
            org=config.gitea_org,
            meta_repo=(projects_mod.content_repo(project_id) or ""),
            slug=slug,
            file_contents=contents,
            pr_title=pr_title,
            pr_description=pr_description,
            rfcs_dir=rfcs_prefix,
        )
        # ...refresh + use_case persistence: pass collection_id (it already does)...

Keep _propose_into_project(project_id, payload, user) as a thin wrapper that resolves the default collection and calls _propose_into_collection, so the existing /api/rfcs/propose and /api/projects/:id/rfcs/propose routes are unchanged. landing_state now derives from the collection (replacing the projects_mod.project_initial_state line at api.py:991).

  • Step 4: Add rfcs_dir param to bot.open_idea_pr (default "rfcs"), threading it into the file path it writes. In backend/app/bot.py, open_idea_pr (≈line 168) builds the entry file path as f"rfcs/{slug}.md"; change to accept rfcs_dir: str = "rfcs" and build f"{rfcs_dir}/{slug}.md". Leave every existing caller (which omits the arg) on "rfcs".

  • Step 5: Add the scoped read + propose routes in api.py (beside the existing project routes).

    @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs")
    async def list_collection_rfcs(project_id: str, collection_id: str, request: Request,
                                   unreviewed: str | None = None) -> dict[str, Any]:
        viewer = auth.current_user(request)
        auth.require_project_readable(viewer, project_id)
        _require_collection_in_project(collection_id, project_id)
        return _list_rfcs_for_collection(collection_id, viewer, unreviewed)

    @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}")
    async def get_collection_rfc(project_id: str, collection_id: str, slug: str,
                                 request: Request) -> dict[str, Any]:
        viewer = auth.current_user(request)
        auth.require_project_readable(viewer, project_id)
        _require_collection_in_project(collection_id, project_id)
        return _get_rfc_for_collection(collection_id, slug, viewer)

    @router.post("/api/projects/{project_id}/collections/{collection_id}/rfcs/propose")
    async def propose_collection_rfc(project_id: str, collection_id: str,
                                     payload: ProposeBody, request: Request) -> dict[str, Any]:
        user = auth.require_contributor(request)
        auth.require_project_readable(user, project_id)
        _require_collection_in_project(collection_id, project_id)
        return await _propose_into_collection(project_id, collection_id, payload, user)

Add the small helpers (refactor the bodies of the existing list_project_rfcs / get_project_rfc at api.py:720-795 into _list_rfcs_for_collection(collection_id, viewer, unreviewed) and _get_rfc_for_collection(collection_id, slug, viewer), then have the project-scoped routes call them with the default collection). Add:

    def _require_collection_in_project(collection_id: str, project_id: str) -> None:
        if collections_mod.project_of_collection(collection_id) != project_id:
            raise HTTPException(404, "Not found")
  • Step 6: Run the scoped-serve tests, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py -q Expected: PASS.

  • Step 7: Run the full backend suite, verify PASS (default-collection routes unchanged).

Run: cd backend && python -m pytest -q Expected: PASS.

  • Step 8: Commit.
git add backend/app/api.py backend/app/bot.py backend/tests/test_collection_scoped_serve.py
git commit -m "§22 S2: collection-scoped list/get/propose endpoints"

Task 5: create-collection endpoint (bot commit + registry refresh)

Files:

  • Create: backend/app/api_collections.py
  • Modify: backend/app/bot.py (add create_collection), backend/app/main.py (mount router).
  • Test: backend/tests/test_collection_create_vertical.py (append)

Flow: deployment admin POSTs {collection_id, type, name?, visibility?, initial_state?}; the bot commits <collection_id>/.collection.yaml to the project's content-repo main; then refresh_registry re-reads and upserts the collection row. The registry stays the source of truth (§22.2) — the endpoint never writes the collections row directly.

  • Step 1: Write the failing vertical test.
# Append to backend/tests/test_collection_create_vertical.py
def test_create_collection_commits_manifest_and_mirrors(admin_client_with_fake_gitea):
    client, fake = admin_client_with_fake_gitea  # seeds project 'ohm' + default collection
    r = client.post("/api/projects/ohm/collections",
                    json={"collection_id": "features", "type": "bdd", "name": "Features"})
    assert r.status_code == 200, r.text
    assert fake.committed_path == "features/.collection.yaml"
    assert "type: bdd" in fake.committed_text
    # registry refresh ran → row exists.
    row = db.conn().execute("SELECT type, project_id FROM collections WHERE id='features'").fetchone()
    assert (row["type"], row["project_id"]) == ("bdd", "ohm")


def test_create_collection_requires_admin(member_client):
    r = member_client.post("/api/projects/ohm/collections",
                           json={"collection_id": "x", "type": "bdd"})
    assert r.status_code in (401, 403)


def test_create_collection_rejects_duplicate_id(admin_client_with_fake_gitea):
    client, _ = admin_client_with_fake_gitea
    client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"})
    r = client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"})
    assert r.status_code == 409

Reuse the app_with_fake_gitea admin fixture from test_propose_vertical.py; extend its fake bot/Gitea to record committed_path / committed_text for a root-level file create, and to make refresh_registry's list_dir + get_contents see the just-committed manifest (an in-memory tree the fake mutates on commit).

  • Step 2: Run it, verify it fails.

Run: cd backend && python -m pytest tests/test_collection_create_vertical.py -k create_collection -q Expected: FAIL (route 404).

  • Step 3: Add Bot.create_collection in bot.py (commit the manifest to main).
    async def create_collection(self, actor, *, org: str, content_repo: str,
                                collection_id: str, manifest_yaml: str) -> None:
        """Commit `<collection_id>/.collection.yaml` to the content repo's main.
        A structural admin action — commits straight to main (no PR), like
        registry config. Logs an action row for the audit trail."""
        path = f"{collection_id}/.collection.yaml"
        await self._gitea.create_file(
            org, content_repo, path, manifest_yaml,
            message=self._stamp(actor, f"chore: create collection {collection_id}"),
            branch="main",
        )
        self._log(actor, action="create_collection",
                  details={"collection_id": collection_id, "repo": content_repo})

Match the exact create_file signature + _stamp/_log call shapes used by the neighbouring bot methods (e.g. commit_accepted_change); names above are illustrative of the pattern, not necessarily the literal argument names.

  • Step 4: Create backend/app/api_collections.py with the read + create routes.
"""§22 S2 — collection directory + create-collection.

GET  /api/projects/:id/collections        — list visible collections.
GET  /api/projects/:id/collections/:cid    — one collection's settings.
POST /api/projects/:id/collections         — create a collection (admin; bot commits
                                             a `.collection.yaml`, then the registry
                                             mirror upserts the row — §22.2).
"""
from __future__ import annotations

import re
from typing import Any

import yaml
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel

from . import auth, bot, cache, collections as collections_mod, db, gitea as gitea_mod
from . import projects as projects_mod, registry as registry_mod
from .config import Config
from .gitea import GiteaError

_SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$")


class CreateCollectionBody(BaseModel):
    collection_id: str
    type: str
    name: str | None = None
    visibility: str | None = None
    initial_state: str | None = None


def make_router(config: Config, gitea: gitea_mod.Gitea) -> APIRouter:
    router = APIRouter()

    @router.get("/api/projects/{project_id}/collections")
    async def list_cols(project_id: str, request: Request) -> dict[str, Any]:
        viewer = auth.current_user(request)
        auth.require_project_readable(viewer, project_id)
        return {"items": collections_mod.list_collections(project_id)}

    @router.get("/api/projects/{project_id}/collections/{collection_id}")
    async def get_col(project_id: str, collection_id: str, request: Request) -> dict[str, Any]:
        viewer = auth.current_user(request)
        auth.require_project_readable(viewer, project_id)
        col = collections_mod.get_collection(collection_id)
        if col is None or col["project_id"] != project_id:
            raise HTTPException(404, "Not found")
        return col

    @router.post("/api/projects/{project_id}/collections")
    async def create_col(project_id: str, body: CreateCollectionBody,
                         request: Request) -> dict[str, Any]:
        user = auth.require_admin(request)  # deployment owner/admin (S2; scoped roles in S3)
        auth.require_project_readable(user, project_id)
        cid = body.collection_id.strip().lower()
        if not _SLUG_RE.match(cid) or cid == "default":
            raise HTTPException(422, "collection id must be a slug and not 'default'")
        if body.type not in registry_mod.VALID_TYPES:
            raise HTTPException(422, f"invalid type {body.type!r}")
        if collections_mod.get_collection(cid) is not None:
            raise HTTPException(409, f"collection `{cid}` already exists")
        content_repo = projects_mod.content_repo(project_id)
        if not content_repo:
            raise HTTPException(409, "project has no content repo")
        manifest: dict[str, Any] = {"type": body.type}
        if body.name:
            manifest["name"] = body.name
        if body.visibility:
            manifest["visibility"] = body.visibility
        if body.initial_state:
            manifest["initial_state"] = body.initial_state
        manifest_yaml = yaml.safe_dump(manifest, sort_keys=False)
        try:
            await bot.create_collection(
                user.as_actor(), org=config.gitea_org, content_repo=content_repo,
                collection_id=cid, manifest_yaml=manifest_yaml)
        except GiteaError as e:
            raise HTTPException(502, f"Gitea: {e.detail}")
        # §22.2: the registry mirror is the source of truth — re-read so the new
        # manifest becomes a collections row.
        await registry_mod.refresh_registry(config, gitea)
        col = collections_mod.get_collection(cid)
        if col is None:
            raise HTTPException(500, "collection committed but not mirrored")
        return col

    return router

bot.create_collection is module-level if the codebase exposes bot.<verb> module functions (as the propose path uses bot.open_idea_pr); if Bot is a class instance, call it the same way the propose route does. Match the existing convention exactly.

  • Step 5: Mount the router in backend/app/main.py next to the other make_router mounts.
    from . import api_collections
    app.include_router(api_collections.make_router(config, gitea))
  • Step 6: Run the create-collection tests, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_create_vertical.py -q Expected: PASS.

  • Step 7: Run the full backend suite, verify PASS.

Run: cd backend && python -m pytest -q Expected: PASS.

  • Step 8: Commit.
git add backend/app/api_collections.py backend/app/bot.py backend/app/main.py backend/tests/test_collection_create_vertical.py
git commit -m "§22 S2: create-collection endpoint (bot commit + registry refresh)"

Task 6: Frontend — collection-scoped path + API helpers

Files:

  • Modify: frontend/src/lib/entryPaths.js, frontend/src/api.js

  • Test: frontend/src/lib/entryPaths.test.js (extend)

  • Step 1: Write the failing test for useCollectionId fallback + scoped API URLs.

// Append to frontend/src/lib/entryPaths.test.js
import { describe, it, expect } from 'vitest'
import { DEFAULT_COLLECTION } from './entryPaths'

describe('collection paths', () => {
  it('entryPath honors an explicit collection', () => {
    // entryPath(pid, slug, cid)
    const { entryPath } = require('./entryPaths')
    expect(entryPath('ohm', 'a', 'features')).toBe('/p/ohm/c/features/e/a')
    expect(entryPath('ohm', 'a')).toBe(`/p/ohm/c/${DEFAULT_COLLECTION}/e/a`)
  })
})

Match the file's existing import/runner style (it may use ESM import rather than require); follow whatever entryPaths.test.js already does.

  • Step 2: Run it, verify PASS or FAIL (entryPath already supports cid → this asserts the contract; the new piece is useCollectionId). If green, proceed; the behavioral gap is the hook.

Run: cd frontend && npx vitest run src/lib/entryPaths.test.js

  • Step 3: Add useCollectionId to entryPaths.js.
import { useParams } from 'react-router-dom'

// The collection id a component should scope to: the `/c/:collectionId/` segment
// when present, else the project's default collection.
export function useCollectionId() {
  const { collectionId } = useParams()
  return collectionId || DEFAULT_COLLECTION
}
  • Step 4: Make api.js collection-aware. Add optional collectionId to the corpus reads and propose; add listCollections + createCollection.
export async function listRFCs(projectId, collectionId) {
  if (projectId && collectionId) {
    return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs`))
  }
  const url = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs'
  return jsonOrThrow(await fetch(url))
}

export async function getRFC(projectId, slug, collectionId) {
  if (collectionId) {
    return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}`))
  }
  if (slug === undefined) return jsonOrThrow(await fetch(`/api/rfcs/${projectId}`))
  return jsonOrThrow(await fetch(`/api/projects/${projectId}/rfcs/${slug}`))
}

export async function listCollections(projectId) {
  return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections`))
}

export async function createCollection(projectId, { collectionId, type, name, visibility, initialState }) {
  const res = await fetch(`/api/projects/${projectId}/collections`, {
    method: 'POST', headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ collection_id: collectionId, type, name: name || null,
      visibility: visibility || null, initial_state: initialState || null }),
  })
  return jsonOrThrow(res)
}

And add an optional collectionId to proposeRFC so it targets the scoped route:

export async function proposeRFC(projectId, { title, slug, pitch, tags, proposedUseCase, collectionId }) {
  const url = (projectId && collectionId)
    ? `/api/projects/${projectId}/collections/${collectionId}/rfcs/propose`
    : (projectId ? `/api/projects/${projectId}/rfcs/propose` : '/api/rfcs/propose')
  // ...unchanged body/post...
}
  • Step 5: Run the frontend unit tests, verify PASS.

Run: cd frontend && npx vitest run src/lib/entryPaths.test.js Expected: PASS.

  • Step 6: Commit.
git add frontend/src/lib/entryPaths.js frontend/src/api.js frontend/src/lib/entryPaths.test.js
git commit -m "§22 S2: collection-scoped frontend path + API helpers"

Task 7: Frontend — Catalog reads the active collection

Files:

  • Modify: frontend/src/components/Catalog.jsx, frontend/src/App.jsx (propose wiring)

  • Step 1: Scope the Catalog to the active collection. In Catalog.jsx:

    • import useCollectionId from ../lib/entryPaths;
    • const cid = useCollectionId();
    • fetch listRFCs(pid, cid) and add cid to the effect deps;
    • build entry links with entryPath(pid, r.slug, cid) and proposalPath(pid, p.pr_number, cid).

    The anonymous empty-state (C3.6) is already correct — the footer renders the "Sign in to propose" link for !viewer, and the empty list shows "No RFCs in the catalog yet." Confirm both render for an anonymous viewer on an empty collection.

  • Step 2: Thread the active collection into the propose modal in App.jsx. The ProposeModal is mounted with projectId={currentProjectId}; also pass the active collection (derive from the route — read useParams().collectionId in the AppShell scope, default DEFAULT_COLLECTION) and pass it to proposeRFC so a propose from a named collection targets that collection. On submit, navigate with proposalPath(currentProjectId, pr_number, currentCollectionId).

  • Step 3: Build + run the frontend test suite, verify PASS.

Run: cd frontend && npx vitest run Expected: PASS (existing tests green; default-collection behavior unchanged).

  • Step 4: Commit.
git add frontend/src/components/Catalog.jsx frontend/src/App.jsx
git commit -m "§22 S2: Catalog + propose scoped to the active collection"

Task 8: Frontend — the collection directory at /p/<project>/

Files:

  • Create: frontend/src/components/CollectionDirectory.jsx

  • Modify: frontend/src/App.jsx (swap DefaultCollectionRedirectCollectionDirectory)

  • Test: frontend/src/components/CollectionDirectory.test.jsx

  • Step 1: Write the failing test.

// frontend/src/components/CollectionDirectory.test.jsx
import { render, screen, waitFor } from '@testing-library/react'
import { MemoryRouter } from 'react-router-dom'
import { describe, it, expect, vi } from 'vitest'
import CollectionDirectory from './CollectionDirectory'

vi.mock('../api', () => ({
  listCollections: vi.fn(async () => ({ items: [
    { id: 'default', name: 'Model', type: 'document' },
    { id: 'features', name: 'Features', type: 'bdd' },
  ] })),
}))

describe('CollectionDirectory', () => {
  it('lists collections when there are 2+', async () => {
    render(<MemoryRouter initialEntries={["/p/ohm/"]}><CollectionDirectory projectId="ohm" /></MemoryRouter>)
    await waitFor(() => expect(screen.getByText('Features')).toBeInTheDocument())
    expect(screen.getByText('Model')).toBeInTheDocument()
  })
})

Match the project's component-test conventions (see Directory.test.jsx / ProjectLayout.test.jsx for the render + mock pattern; adapt the mock + queries to whatever they use).

  • Step 2: Run it, verify it fails (component missing).

Run: cd frontend && npx vitest run src/components/CollectionDirectory.test.jsx Expected: FAIL.

  • Step 3: Implement CollectionDirectory.jsx. Fetch listCollections(projectId); when exactly one visible collection, <Navigate> to its collectionHome (preserves the S1 C3.7/C3.8 single-collection redirect); when 2+, render a list of links to each collectionHome(projectId, c.id) with its name + type. (The "Create your first collection" empty-state is S4 — for S2 a 0-collection project simply shows a minimal "No collections yet." line; the create affordance UI lands in S4.)
import { useEffect, useState } from 'react'
import { Link, Navigate } from 'react-router-dom'
import { listCollections } from '../api'
import { collectionHome } from '../lib/entryPaths'

export default function CollectionDirectory({ projectId }) {
  const [cols, setCols] = useState(null)
  useEffect(() => {
    let live = true
    listCollections(projectId).then(d => { if (live) setCols(d.items) }).catch(() => live && setCols([]))
    return () => { live = false }
  }, [projectId])
  if (cols === null) return <main className="chrome-pane"><div className="boot">Loading</div></main>
  if (cols.length === 1) return <Navigate to={collectionHome(projectId, cols[0].id)} replace />
  return (
    <main className="chrome-pane">
      <div className="collection-directory">
        <h1>Collections</h1>
        {cols.length === 0 ? (
          <p>No collections yet.</p>
        ) : (
          <ul>
            {cols.map(c => (
              <li key={c.id}>
                <Link to={collectionHome(projectId, c.id)}>{c.name || c.id}</Link>
                <span className="collection-type"> · {c.type}</span>
              </li>
            ))}
          </ul>
        )}
      </div>
    </main>
  )
}
  • Step 4: Wire it into App.jsx. Replace the DefaultCollectionRedirect route element at the project landing (App.jsx:365) with <CollectionDirectory projectId={<the route projectId>} />. Read the project id from useParams() inside a small wrapper (mirroring how DefaultCollectionRedirect reads it), and keep DefaultCollectionRedirect only if still referenced elsewhere (otherwise delete it). The S1 single-collection redirect now lives inside CollectionDirectory, so the C3.7/C3.8 behavior is preserved.

  • Step 5: Run the directory test + full frontend suite, verify PASS.

Run: cd frontend && npx vitest run Expected: PASS.

  • Step 6: Commit.
git add frontend/src/components/CollectionDirectory.jsx frontend/src/components/CollectionDirectory.test.jsx frontend/src/App.jsx
git commit -m "§22 S2: collection directory at /p/<project>/ (1 → redirect, 2+ → list)"

Task 9: Acceptance — C3.6 anonymous empty-collection catalog

Files:

  • Test: backend/tests/test_collection_scoped_serve.py (append the @S2 acceptance assertion)

The @S2 gate is C3.6: a public collection with no entries; an anonymous visitor lands on /p/ohm/c/model/; they see an empty catalog with no propose action and a sign-in prompt. Backend half: the scoped list returns {items: []} for an anonymous viewer on a public empty collection (no 404, no propose surfaced). Frontend half: the Catalog footer renders "Sign in to propose" for !viewer (already covered by Task 7).

  • Step 1: Write the acceptance test (backend contract for the empty public collection).
def test_s2_anonymous_empty_public_collection(client_with_corpus):
    """C3.6 (@S2): anonymous viewer, public empty collection → empty catalog, 200."""
    client, _ = client_with_corpus  # ensure an empty public collection 'model'
    r = client.get("/api/projects/ohm/collections/model/rfcs")  # no auth header
    assert r.status_code == 200
    assert r.json()["items"] == []

If client_with_corpus doesn't already seed an empty public model collection, add one (mirror Task 3's seed). The propose-absence is enforced by require_contributor on the propose route (anonymous → 401/403) and the Catalog UI footer.

  • Step 2: Run it, verify PASS.

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_s2_anonymous_empty_public_collection -q Expected: PASS.

  • Step 3: Manually verify the propose route rejects anonymous (sanity — no new code):

Run: cd backend && python -m pytest tests/test_collection_scoped_serve.py -q Expected: PASS (all).

  • Step 4: Commit.
git add backend/tests/test_collection_scoped_serve.py
git commit -m "§22 S2: @S2 acceptance — anonymous empty public collection catalog"

Task 10: Release — version bump, changelog, docs

Files:

  • Modify: VERSION, frontend/package.json, CHANGELOG.md
  • Modify: docs/design/2026-06-05-three-tier-projects-collections.md (mark S2 shipped)

S2 adds functionality and is non-breaking (new optional path segment + new endpoints; the default-collection paths are unchanged). Per SPEC §20, that's a minor bump: 0.40.00.41.0.

  • Step 1: Bump VERSION to 0.41.0.

  • Step 2: Bump frontend/package.json#version to 0.41.0 (must mirror VERSION — §20).

  • Step 3: Add the CHANGELOG.md entry under a new ## 0.41.0 heading: a minor release shipping S2 (create + navigate + propose a second collection): registry mirror reads .collection.yaml; collection-grained corpus mirror; create-collection endpoint; collection-scoped list/get/propose; /p/<project>/ collection directory. Note it completes @S2 (C3.6). No upgrade steps required (additive; existing default-collection deployments keep working unchanged) — state that explicitly.

  • Step 4: Mark S2 shipped in the design doc's Part E slice list (a short "Landed vX" note on the S2 bullet, mirroring how S1 was annotated).

  • Step 5: Run both suites once more, verify green.

Run: cd backend && python -m pytest -q && cd ../frontend && npx vitest run Expected: PASS (both).

  • Step 6: Commit.
git add VERSION frontend/package.json CHANGELOG.md docs/design/2026-06-05-three-tier-projects-collections.md
git commit -m "§22 S2: release v0.41.0 — create & navigate a second collection (@S2)"

Self-review notes

  • Spec coverage (Part E "S2" bullet): registry reads .collection.yaml (Task 1) · create-collection endpoint, admin-gated (Task 5) · project collection-directory at /p/<project>/ (Task 8) · collection-scoped propose/serve (Tasks 3,4,7) · @S2 / C3.6 acceptance (Task 9). ✓
  • N=1 unchanged invariant: every backend task ends by running the full suite; the default-collection routes are preserved as thin wrappers over the new collection-scoped internals. ✓
  • Type consistency: collection_id/subfolder naming is uniform; the entries path is <subfolder>/rfcs/ everywhere (cache mirror Task 3, propose Task 4, create manifest Task 5).
  • Deferred / lower-confidence calls (log to transcript): (a) create-collection commits straight to main rather than via a PR — chosen because a collection is structural config and the registry mirror is the source of truth; (b) S2 collection visibility filtering is coarse (project read-gate + drop unlisted); full scope-role enforcement is S3; (c) the per-collection "create"/"propose-first" empty-state affordances are S4 — S2 ships only the anonymous empty catalog (C3.6) and a minimal directory.