# §22 S2 — Create & navigate a second collection — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to > implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Let a deployment admin create a second RFC collection beside a project's default one, navigate to it, and propose into it — completing acceptance scenario C3.6 (`@S2`): an anonymous visitor landing on an empty public collection's catalog sees an empty catalog with no propose action and a sign-in prompt. **Architecture:** S1 shipped the collection grain (migration 029): a `collections` table, entries keyed `(collection_id, slug)`, one seeded `default` collection per project. S2 makes collections *plural and navigable*. Backend: the registry mirror learns to read `.collection.yaml` manifests inside each project's content repo; the corpus mirror becomes collection-grained (reads each collection's `/rfcs/`); a bot-commit-wrapped create-collection endpoint commits a manifest; collection-scoped list/get/propose endpoints land under `/api/projects/:id/collections/:cid/…`. Frontend: `/p//` becomes a collection directory (1 visible → redirect, keeping S1; 2+ → list); the Catalog rail + entry views read the active `:collectionId` from the route. **Tech Stack:** FastAPI + sqlite (backend, `backend/app/`), pytest (`backend/tests/`), React + react-router + vitest (`frontend/src/`), Gitea content repos via the Bot wrapper. **Content-repo layout convention (decided this slice):** a collection with subfolder `S` stores its manifest at `S/.collection.yaml` and its entries at `S/rfcs/`. The default collection has subfolder `''` → manifest is the project's `projects.yaml` entry (no `.collection.yaml`), entries at `rfcs/`. This keeps the shipped default corpus path unchanged. **Authorization (this slice):** create-collection is gated to a deployment owner/admin (`auth.require_admin`). Scoped {owner, contributor} roles at the collection axis land in S3 — out of scope here. --- ## File structure **Backend (modify):** - `backend/app/registry.py` — add `.collection.yaml` discovery + upsert to the mirror. - `backend/app/collections.py` — add `list_collections`, `get_collection`, `subfolder_of`. - `backend/app/cache.py` — make `refresh_meta_repo` iterate a project's collections. - `backend/app/bot.py` — add `create_collection` (commit `/.collection.yaml`). - `backend/app/api.py` — add collection-scoped list/get/propose; refactor propose to take a collection. - `backend/app/api_collections.py` — **create**: GET list/one + POST create-collection endpoints. - `backend/app/main.py` — mount the new router. **Frontend (modify):** - `frontend/src/lib/entryPaths.js` — add `useCollectionId()`. - `frontend/src/api.js` — collection-scoped `listRFCs`/`getRFC`/`proposeRFC`; add `listCollections`, `createCollection`. - `frontend/src/components/Catalog.jsx` — read active collection; scope fetches + links. - `frontend/src/App.jsx` — `/p//` → `CollectionDirectory`; thread collection into propose. - `frontend/src/components/CollectionDirectory.jsx` — **create**: the directory page. **Tests (create):** - `backend/tests/test_collection_registry.py` — `.collection.yaml` mirror. - `backend/tests/test_collection_create_vertical.py` — create → mirror → list round-trip. - `backend/tests/test_collection_scoped_serve.py` — list/get/propose under a named collection. - `frontend/src/lib/entryPaths.test.js` (extend) + `frontend/src/components/CollectionDirectory.test.jsx`. --- ## Task 1: Registry mirror reads `.collection.yaml` manifests **Files:** - Modify: `backend/app/registry.py` - Test: `backend/tests/test_collection_registry.py` The mirror today upserts only the default collection from `projects.yaml`. Add a `parse_collection_manifest(text)` (pure) and extend `refresh_registry` to walk each project's content repo root, read every `/.collection.yaml`, and upsert a collection row keyed by the subdir name. - [ ] **Step 1: Write the failing test** for the pure parser. ```python # backend/tests/test_collection_registry.py import pytest from app import registry def test_parse_collection_manifest_minimal(): doc = registry.parse_collection_manifest("type: bdd\n") assert doc.type == "bdd" # §22.4b: bdd defaults to 'active'; visibility inherits (None == inherit). assert doc.initial_state == "active" assert doc.visibility is None assert doc.name is None def test_parse_collection_manifest_full(): doc = registry.parse_collection_manifest( "type: document\nvisibility: public\ninitial_state: active\nname: Model\n" ) assert (doc.type, doc.visibility, doc.initial_state, doc.name) == ( "document", "public", "active", "Model", ) def test_parse_collection_manifest_rejects_bad_type(): with pytest.raises(registry.RegistryError): registry.parse_collection_manifest("type: nonsense\n") ``` - [ ] **Step 2: Run it, verify it fails.** Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` Expected: FAIL (`parse_collection_manifest` not defined). - [ ] **Step 3: Implement `parse_collection_manifest` + `CollectionEntry`** in `registry.py`. ```python # Add near ProjectEntry (after line 56). @dataclass class CollectionEntry: type: str visibility: str | None # None == inherit the project's visibility initial_state: str name: str | None def parse_collection_manifest(text: str) -> CollectionEntry: """Parse + validate a `.collection.yaml`. Pure. Raises RegistryError.""" raw = yaml.safe_load(text) or {} if not isinstance(raw, dict): raise RegistryError("collection manifest must be a mapping") ctype = str(raw.get("type") or "").strip() if ctype not in VALID_TYPES: raise RegistryError(f"collection has invalid type {ctype!r}") vis = raw.get("visibility") if vis is not None: vis = str(vis).strip() if vis not in VALID_VISIBILITY: raise RegistryError(f"collection has invalid visibility {vis!r}") initial_state = str( raw.get("initial_state") or _TYPE_DEFAULT_INITIAL_STATE[ctype] ).strip() if initial_state not in VALID_INITIAL_STATE: raise RegistryError(f"collection has invalid initial_state {initial_state!r}") name = raw.get("name") name = str(name).strip() if name else None return CollectionEntry(ctype, vis, initial_state, name) ``` - [ ] **Step 4: Run the parser tests, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` Expected: PASS (3 tests). - [ ] **Step 5: Write the failing mirror test** (uses a fake Gitea exposing the content repo). ```python # Append to backend/tests/test_collection_registry.py import asyncio class _FakeGitea: """Minimal Gitea stub: projects.yaml in the registry repo + a content repo holding one `.collection.yaml` under `features/`.""" def __init__(self, projects_yaml, repo_tree): self._projects_yaml = projects_yaml self._repo_tree = repo_tree # {repo: {path: text}} async def get_contents(self, org, repo, path, ref="main"): import base64 if path == "projects.yaml": enc = base64.b64encode(self._projects_yaml.encode()).decode() return {"type": "file", "content": enc, "sha": "deadbeef"} text = self._repo_tree.get(repo, {}).get(path) if text is None: return None return {"type": "file", "content": base64.b64encode(text.encode()).decode(), "sha": "c0ffee"} async def list_dir(self, org, repo, path, ref="main"): # Root listing: surface each top-level subdir as a 'dir' entry. names = set() for p in self._repo_tree.get(repo, {}): head = p.split("/", 1)[0] if "/" in p: names.add(head) return [{"type": "dir", "name": n, "path": n} for n in sorted(names)] def test_refresh_registry_mirrors_named_collection(app_with_db, config): gitea = _FakeGitea( projects_yaml=( "deployment:\n name: Ohm\n tagline: t\n" "projects:\n - id: ohm\n name: Ohm\n type: document\n" " content_repo: ohm-rfc\n visibility: public\n" ), repo_tree={"ohm-rfc": {"features/.collection.yaml": "type: bdd\nname: Features\n"}}, ) asyncio.run(registry.refresh_registry(config, gitea)) from app import db row = db.conn().execute( "SELECT type, subfolder, name, project_id FROM collections WHERE id = 'features'" ).fetchone() assert row is not None assert (row["type"], row["subfolder"], row["project_id"]) == ("bdd", "features", "ohm") ``` > Use the existing test app/db fixture pattern — mirror `app_with_db` / `config` > fixtures from `backend/tests/test_registry.py`. If that test uses different > fixture names, copy its setup verbatim here. - [ ] **Step 6: Run it, verify it fails** (`features` collection not mirrored). Run: `cd backend && python -m pytest tests/test_collection_registry.py::test_refresh_registry_mirrors_named_collection -q` Expected: FAIL (row is None). - [ ] **Step 7: Extend `refresh_registry`** to discover + upsert named collections. After `apply_registry(...)` in `refresh_registry` (registry.py line 215), add: ```python # §22 S2: named collections are declared by `.collection.yaml` manifests # inside each project's content repo (the default collection comes from # projects.yaml above). Walk each content repo root; a subdir carrying a # manifest becomes a collection keyed by the subdir name. for proj in doc.projects: try: items = await gitea.list_dir(config.gitea_org, proj.content_repo, "", ref="main") except Exception as e: # GiteaError or transport — tolerate, keep last-good log.warning("registry: cannot list %s root: %s", proj.content_repo, e) continue for it in items: if it.get("type") != "dir": continue subdir = it["name"] manifest = await gitea.get_contents( config.gitea_org, proj.content_repo, f"{subdir}/.collection.yaml", ref="main" ) if not manifest or manifest.get("type") != "file": continue mtext = base64.b64decode(manifest["content"]).decode("utf-8") try: ce = parse_collection_manifest(mtext) except RegistryError as e: log.error("registry: bad manifest %s/%s: %s", proj.content_repo, subdir, e) continue _upsert_named_collection(proj, subdir, ce, sha) ``` And add the upsert helper (after `apply_registry`): ```python def _upsert_named_collection(proj: ProjectEntry, subdir: str, ce: CollectionEntry, sha: str) -> None: """Upsert one named collection. Type is immutable (§22.4a): a type change against an existing row is refused (logged, not applied). Visibility None inherits the project's visibility.""" visibility = ce.visibility or proj.visibility with db.tx() as conn: existing = conn.execute( "SELECT type FROM collections WHERE id = ?", (subdir,) ).fetchone() if existing is not None and existing["type"] != ce.type: log.error("registry: refusing immutable type change on collection %s (%s -> %s)", subdir, existing["type"], ce.type) return conn.execute( """ INSERT INTO collections (id, project_id, type, subfolder, initial_state, visibility, name, registry_sha, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now')) ON CONFLICT(id) DO UPDATE SET project_id = excluded.project_id, initial_state = excluded.initial_state, visibility = excluded.visibility, name = excluded.name, registry_sha = excluded.registry_sha, updated_at = datetime('now') """, (subdir, proj.id, ce.type, subdir, ce.initial_state, visibility, ce.name, sha), ) ``` - [ ] **Step 8: Run all of Task 1's tests, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_registry.py -q` Expected: PASS (4 tests). - [ ] **Step 9: Commit.** ```bash git add backend/app/registry.py backend/tests/test_collection_registry.py git commit -m "§22 S2: registry mirror reads .collection.yaml manifests" ``` --- ## Task 2: Collection read helpers — `list_collections`, `get_collection`, `subfolder_of` **Files:** - Modify: `backend/app/collections.py` - Test: `backend/tests/test_collection_create_vertical.py` (first assertions) - [ ] **Step 1: Write the failing test.** ```python # backend/tests/test_collection_create_vertical.py from app import db, collections as collections_mod def _seed(project_id="ohm"): db.conn().execute( "INSERT INTO projects (id, name, content_repo, visibility, updated_at) " "VALUES (?, 'Ohm', 'ohm-rfc', 'public', datetime('now'))", (project_id,)) for cid, sub, vis, name in [ ("default", "", "public", "Model"), ("features", "features", "public", "Features"), ("secret", "secret", "unlisted", "Secret"), ]: db.conn().execute( "INSERT INTO collections (id, project_id, type, subfolder, initial_state, " "visibility, name, created_at, updated_at) VALUES (?,?, 'document', ?, " "'super-draft', ?, ?, datetime('now'), datetime('now'))", (cid, project_id, sub, vis, name)) def test_list_collections_excludes_unlisted(app_with_db): _seed() ids = [c["id"] for c in collections_mod.list_collections("ohm", include_unlisted=False)] assert ids == ["default", "features"] # 'secret' (unlisted) omitted def test_get_collection_and_subfolder(app_with_db): _seed() assert collections_mod.get_collection("features")["name"] == "Features" assert collections_mod.subfolder_of("features") == "features" assert collections_mod.subfolder_of("default") == "" assert collections_mod.get_collection("nope") is None ``` - [ ] **Step 2: Run it, verify it fails.** Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` Expected: FAIL (`list_collections` not defined). - [ ] **Step 3: Implement the helpers** in `collections.py`. ```python def get_collection(collection_id: str) -> dict | None: row = db.conn().execute( "SELECT id, project_id, type, subfolder, initial_state, visibility, name " "FROM collections WHERE id = ?", (collection_id,) ).fetchone() return dict(row) if row else None def subfolder_of(collection_id: str) -> str: row = db.conn().execute( "SELECT subfolder FROM collections WHERE id = ?", (collection_id,) ).fetchone() return (row["subfolder"] if row else "") or "" def list_collections(project_id: str, include_unlisted: bool = False) -> list[dict]: """Collections in a project, default first then by name. `unlisted` is omitted from enumeration unless include_unlisted (a direct-id read).""" rows = db.conn().execute( "SELECT id, project_id, type, subfolder, initial_state, visibility, name " "FROM collections WHERE project_id = ? ORDER BY (id != 'default'), name, id", (project_id,), ).fetchall() out = [] for r in rows: if not include_unlisted and r["visibility"] == "unlisted": continue out.append(dict(r)) return out ``` - [ ] **Step 4: Run it, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` Expected: PASS (2 tests). - [ ] **Step 5: Commit.** ```bash git add backend/app/collections.py backend/tests/test_collection_create_vertical.py git commit -m "§22 S2: collection read helpers (list/get/subfolder)" ``` --- ## Task 3: Corpus mirror is collection-grained **Files:** - Modify: `backend/app/cache.py:37-97` - Test: `backend/tests/test_collection_scoped_serve.py` (first assertion) `refresh_meta_repo` reads only `rfcs/` (the default collection). Make it iterate a project's collections and read each collection's `/rfcs/`, keying `cached_rfcs` by `collection_id`. - [ ] **Step 1: Write the failing test.** ```python # backend/tests/test_collection_scoped_serve.py import asyncio from app import db, cache class _CorpusGitea: """A content repo with entries under both the default `rfcs/` and a named collection's `features/rfcs/`.""" def __init__(self, tree): self._tree = tree # {path: text} async def list_dir(self, org, repo, path, ref="main"): out = [] prefix = (path.rstrip("/") + "/") if path else "" for p in self._tree: if p.startswith(prefix) and "/" not in p[len(prefix):]: out.append({"type": "file", "name": p.split("/")[-1], "path": p}) return out async def read_file(self, org, repo, path, ref="main"): t = self._tree.get(path) return (t, "sha-" + path) if t is not None else None def _entry_md(slug, title): return f"---\nslug: {slug}\ntitle: {title}\nstate: active\n---\nbody\n" def test_mirror_keys_entries_by_collection(app_with_db, config): db.conn().execute( "INSERT INTO projects (id, name, content_repo, visibility, updated_at) " "VALUES ('ohm','Ohm','ohm-rfc','public', datetime('now'))") for cid, sub in [("default", ""), ("features", "features")]: db.conn().execute( "INSERT INTO collections (id, project_id, type, subfolder, initial_state, " "visibility, created_at, updated_at) VALUES (?, 'ohm','document',?, " "'super-draft','public', datetime('now'), datetime('now'))", (cid, sub)) gitea = _CorpusGitea({ "rfcs/a.md": _entry_md("a", "Default A"), "features/rfcs/b.md": _entry_md("b", "Feature B"), }) asyncio.run(cache.refresh_meta_repo(config, gitea)) got = {(r["collection_id"], r["slug"]) for r in db.conn().execute("SELECT collection_id, slug FROM cached_rfcs")} assert got == {("default", "a"), ("features", "b")} ``` - [ ] **Step 2: Run it, verify it fails** (entry `b` mirrored to wrong/absent collection). Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q` Expected: FAIL. - [ ] **Step 3: Rework `refresh_meta_repo` + `_refresh_project_corpus`** in `cache.py`. Replace the body of `_refresh_project_corpus` so it loops the project's collections: ```python async def _refresh_project_corpus(org: str, project_id: str, repo: str, gitea: Gitea) -> None: # §22 S2: the corpus grain is the collection. Mirror each collection of the # project from its `/rfcs/` directory, keying cached_rfcs by the # collection id. The default collection (subfolder '') reads `rfcs/`. from . import collections as collections_mod for col in collections_mod.list_collections(project_id, include_unlisted=True): collection_id = col["id"] sub = col["subfolder"] or "" rfcs_dir = f"{sub}/rfcs" if sub else "rfcs" try: files = await gitea.list_dir(org, repo, rfcs_dir, ref="main") except GiteaError as e: log.warning("refresh_meta_repo: %s/%s: cannot list %s: %s", project_id, collection_id, rfcs_dir, e) continue seen_slugs: set[str] = set() for f in files: if f.get("type") != "file" or not f.get("name", "").endswith(".md"): continue result = await gitea.read_file(org, repo, f["path"], ref="main") if not result: continue text, sha = result try: entry = entry_mod.parse(text) except Exception as parse_err: log.warning("refresh_meta_repo: %s/%s: skipping %s: %s", project_id, collection_id, f["path"], parse_err) continue if not entry.slug: continue seen_slugs.add(entry.slug) _upsert_cached_rfc(entry, body_sha=sha, collection_id=collection_id) existing = {row["slug"] for row in db.conn().execute( "SELECT slug FROM cached_rfcs WHERE collection_id = ?", (collection_id,))} for missing in existing - seen_slugs: log.info("refresh_meta_repo: %s/%s/%s no longer present — leaving cache row", project_id, collection_id, missing) ``` > `refresh_meta_repo` itself (the project loop) is unchanged — it still calls > `_refresh_project_corpus(org, id, content_repo, gitea)` per project. - [ ] **Step 4: Run it, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_mirror_keys_entries_by_collection -q` Expected: PASS. - [ ] **Step 5: Run the full backend suite** — the default-collection path must be unchanged. Run: `cd backend && python -m pytest -q` Expected: PASS (all green; N=1 default path intact). - [ ] **Step 6: Commit.** ```bash git add backend/app/cache.py backend/tests/test_collection_scoped_serve.py git commit -m "§22 S2: corpus mirror reads each collection's /rfcs/" ``` --- ## Task 4: Collection-scoped serve + propose endpoints (backend) **Files:** - Modify: `backend/app/api.py` — refactor propose to accept a collection; add scoped list/get/propose. - Test: `backend/tests/test_collection_scoped_serve.py` (append) The shipped `_propose_into_project` hardcodes `collection_id = default_collection_id(project_id)` (api.py:973). Generalize it to accept an explicit collection, and write the entry into that collection's `/rfcs/`. - [ ] **Step 1: Write the failing test** (scoped list + propose targets the named collection). ```python # Append to backend/tests/test_collection_scoped_serve.py def test_scoped_list_returns_only_that_collection(client_with_corpus): client, _ = client_with_corpus # fixture seeds 'default'+'features' w/ entries a,b r = client.get("/api/projects/ohm/collections/features/rfcs") assert r.status_code == 200 assert [i["slug"] for i in r.json()["items"]] == ["b"] def test_scoped_propose_writes_into_collection_subfolder(client_with_corpus): client, fake_bot = client_with_corpus r = client.post("/api/projects/ohm/collections/features/rfcs/propose", json={"title": "New", "slug": "newb", "pitch": "x", "tags": []}) assert r.status_code == 200 # The bot was asked to write under features/rfcs/, not rfcs/. assert fake_bot.last_meta_path_prefix == "features/rfcs" ``` > Build `client_with_corpus` from the existing `app_with_fake_gitea` / > propose-vertical fixture in `backend/tests/test_propose_vertical.py`; extend its > fake bot to record the subfolder it was handed (see Step 3). If the propose-vertical > fixture is structured differently, follow its shape and adapt these two asserts. - [ ] **Step 2: Run it, verify it fails.** Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` Expected: FAIL (route 404 / propose ignores collection). - [ ] **Step 3: Generalize `_propose_into_project` → `_propose_into_collection`** in api.py. Change the signature and the two collection-dependent lines. The current helper (api.py:959) resolves `collection_id` from the project default; instead take it as a parameter and prefix the bot write path with the collection's subfolder: ```python async def _propose_into_collection( project_id: str, collection_id: str, payload: ProposeBody, user ) -> dict[str, Any]: if not auth.can_contribute_in_project(user, project_id): raise HTTPException(403, "You do not have contribute access to this project") slug = payload.slug.strip().lower() if not entry_mod.is_valid_slug(slug): raise HTTPException(422, "Slug must be lowercase letters, digits, and dashes") clash = db.conn().execute( "SELECT 1 FROM cached_rfcs WHERE slug = ? AND collection_id = ?", (slug, collection_id) ).fetchone() if clash: raise HTTPException(409, f"Slug `{slug}` is already taken") # ...unchanged idea_clash / entry build... landing_state = ( "active" if collections_mod.collection_initial_state(collection_id) == "active" else "super-draft" ) # ...build `entry`, `contents`, `pr_title`, `pr_description` unchanged... subfolder = collections_mod.subfolder_of(collection_id) rfcs_prefix = f"{subfolder}/rfcs" if subfolder else "rfcs" pr = await bot.open_idea_pr( user.as_actor(), org=config.gitea_org, meta_repo=(projects_mod.content_repo(project_id) or ""), slug=slug, file_contents=contents, pr_title=pr_title, pr_description=pr_description, rfcs_dir=rfcs_prefix, ) # ...refresh + use_case persistence: pass collection_id (it already does)... ``` > Keep `_propose_into_project(project_id, payload, user)` as a thin wrapper that > resolves the default collection and calls `_propose_into_collection`, so the > existing `/api/rfcs/propose` and `/api/projects/:id/rfcs/propose` routes are > unchanged. `landing_state` now derives from the collection (replacing the > `projects_mod.project_initial_state` line at api.py:991). - [ ] **Step 4: Add `rfcs_dir` param to `bot.open_idea_pr`** (default `"rfcs"`), threading it into the file path it writes. In `backend/app/bot.py`, `open_idea_pr` (≈line 168) builds the entry file path as `f"rfcs/{slug}.md"`; change to accept `rfcs_dir: str = "rfcs"` and build `f"{rfcs_dir}/{slug}.md"`. Leave every existing caller (which omits the arg) on `"rfcs"`. - [ ] **Step 5: Add the scoped read + propose routes** in api.py (beside the existing project routes). ```python @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs") async def list_collection_rfcs(project_id: str, collection_id: str, request: Request, unreviewed: str | None = None) -> dict[str, Any]: viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) _require_collection_in_project(collection_id, project_id) return _list_rfcs_for_collection(collection_id, viewer, unreviewed) @router.get("/api/projects/{project_id}/collections/{collection_id}/rfcs/{slug}") async def get_collection_rfc(project_id: str, collection_id: str, slug: str, request: Request) -> dict[str, Any]: viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) _require_collection_in_project(collection_id, project_id) return _get_rfc_for_collection(collection_id, slug, viewer) @router.post("/api/projects/{project_id}/collections/{collection_id}/rfcs/propose") async def propose_collection_rfc(project_id: str, collection_id: str, payload: ProposeBody, request: Request) -> dict[str, Any]: user = auth.require_contributor(request) auth.require_project_readable(user, project_id) _require_collection_in_project(collection_id, project_id) return await _propose_into_collection(project_id, collection_id, payload, user) ``` Add the small helpers (refactor the bodies of the existing `list_project_rfcs` / `get_project_rfc` at api.py:720-795 into `_list_rfcs_for_collection(collection_id, viewer, unreviewed)` and `_get_rfc_for_collection(collection_id, slug, viewer)`, then have the project-scoped routes call them with the default collection). Add: ```python def _require_collection_in_project(collection_id: str, project_id: str) -> None: if collections_mod.project_of_collection(collection_id) != project_id: raise HTTPException(404, "Not found") ``` - [ ] **Step 6: Run the scoped-serve tests, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` Expected: PASS. - [ ] **Step 7: Run the full backend suite, verify PASS** (default-collection routes unchanged). Run: `cd backend && python -m pytest -q` Expected: PASS. - [ ] **Step 8: Commit.** ```bash git add backend/app/api.py backend/app/bot.py backend/tests/test_collection_scoped_serve.py git commit -m "§22 S2: collection-scoped list/get/propose endpoints" ``` --- ## Task 5: create-collection endpoint (bot commit + registry refresh) **Files:** - Create: `backend/app/api_collections.py` - Modify: `backend/app/bot.py` (add `create_collection`), `backend/app/main.py` (mount router). - Test: `backend/tests/test_collection_create_vertical.py` (append) Flow: deployment admin POSTs `{collection_id, type, name?, visibility?, initial_state?}`; the bot commits `/.collection.yaml` to the project's content-repo `main`; then `refresh_registry` re-reads and upserts the collection row. The registry stays the source of truth (§22.2) — the endpoint never writes the `collections` row directly. - [ ] **Step 1: Write the failing vertical test.** ```python # Append to backend/tests/test_collection_create_vertical.py def test_create_collection_commits_manifest_and_mirrors(admin_client_with_fake_gitea): client, fake = admin_client_with_fake_gitea # seeds project 'ohm' + default collection r = client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd", "name": "Features"}) assert r.status_code == 200, r.text assert fake.committed_path == "features/.collection.yaml" assert "type: bdd" in fake.committed_text # registry refresh ran → row exists. row = db.conn().execute("SELECT type, project_id FROM collections WHERE id='features'").fetchone() assert (row["type"], row["project_id"]) == ("bdd", "ohm") def test_create_collection_requires_admin(member_client): r = member_client.post("/api/projects/ohm/collections", json={"collection_id": "x", "type": "bdd"}) assert r.status_code in (401, 403) def test_create_collection_rejects_duplicate_id(admin_client_with_fake_gitea): client, _ = admin_client_with_fake_gitea client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"}) r = client.post("/api/projects/ohm/collections", json={"collection_id": "features", "type": "bdd"}) assert r.status_code == 409 ``` > Reuse the `app_with_fake_gitea` admin fixture from `test_propose_vertical.py`; > extend its fake bot/Gitea to record `committed_path` / `committed_text` for a > root-level file create, and to make `refresh_registry`'s `list_dir` + `get_contents` > see the just-committed manifest (an in-memory tree the fake mutates on commit). - [ ] **Step 2: Run it, verify it fails.** Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -k create_collection -q` Expected: FAIL (route 404). - [ ] **Step 3: Add `Bot.create_collection`** in `bot.py` (commit the manifest to main). ```python async def create_collection(self, actor, *, org: str, content_repo: str, collection_id: str, manifest_yaml: str) -> None: """Commit `/.collection.yaml` to the content repo's main. A structural admin action — commits straight to main (no PR), like registry config. Logs an action row for the audit trail.""" path = f"{collection_id}/.collection.yaml" await self._gitea.create_file( org, content_repo, path, manifest_yaml, message=self._stamp(actor, f"chore: create collection {collection_id}"), branch="main", ) self._log(actor, action="create_collection", details={"collection_id": collection_id, "repo": content_repo}) ``` > Match the exact `create_file` signature + `_stamp`/`_log` call shapes used by the > neighbouring bot methods (e.g. `commit_accepted_change`); names above are > illustrative of the pattern, not necessarily the literal argument names. - [ ] **Step 4: Create `backend/app/api_collections.py`** with the read + create routes. ```python """§22 S2 — collection directory + create-collection. GET /api/projects/:id/collections — list visible collections. GET /api/projects/:id/collections/:cid — one collection's settings. POST /api/projects/:id/collections — create a collection (admin; bot commits a `.collection.yaml`, then the registry mirror upserts the row — §22.2). """ from __future__ import annotations import re from typing import Any import yaml from fastapi import APIRouter, HTTPException, Request from pydantic import BaseModel from . import auth, bot, cache, collections as collections_mod, db, gitea as gitea_mod from . import projects as projects_mod, registry as registry_mod from .config import Config from .gitea import GiteaError _SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$") class CreateCollectionBody(BaseModel): collection_id: str type: str name: str | None = None visibility: str | None = None initial_state: str | None = None def make_router(config: Config, gitea: gitea_mod.Gitea) -> APIRouter: router = APIRouter() @router.get("/api/projects/{project_id}/collections") async def list_cols(project_id: str, request: Request) -> dict[str, Any]: viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) return {"items": collections_mod.list_collections(project_id)} @router.get("/api/projects/{project_id}/collections/{collection_id}") async def get_col(project_id: str, collection_id: str, request: Request) -> dict[str, Any]: viewer = auth.current_user(request) auth.require_project_readable(viewer, project_id) col = collections_mod.get_collection(collection_id) if col is None or col["project_id"] != project_id: raise HTTPException(404, "Not found") return col @router.post("/api/projects/{project_id}/collections") async def create_col(project_id: str, body: CreateCollectionBody, request: Request) -> dict[str, Any]: user = auth.require_admin(request) # deployment owner/admin (S2; scoped roles in S3) auth.require_project_readable(user, project_id) cid = body.collection_id.strip().lower() if not _SLUG_RE.match(cid) or cid == "default": raise HTTPException(422, "collection id must be a slug and not 'default'") if body.type not in registry_mod.VALID_TYPES: raise HTTPException(422, f"invalid type {body.type!r}") if collections_mod.get_collection(cid) is not None: raise HTTPException(409, f"collection `{cid}` already exists") content_repo = projects_mod.content_repo(project_id) if not content_repo: raise HTTPException(409, "project has no content repo") manifest: dict[str, Any] = {"type": body.type} if body.name: manifest["name"] = body.name if body.visibility: manifest["visibility"] = body.visibility if body.initial_state: manifest["initial_state"] = body.initial_state manifest_yaml = yaml.safe_dump(manifest, sort_keys=False) try: await bot.create_collection( user.as_actor(), org=config.gitea_org, content_repo=content_repo, collection_id=cid, manifest_yaml=manifest_yaml) except GiteaError as e: raise HTTPException(502, f"Gitea: {e.detail}") # §22.2: the registry mirror is the source of truth — re-read so the new # manifest becomes a collections row. await registry_mod.refresh_registry(config, gitea) col = collections_mod.get_collection(cid) if col is None: raise HTTPException(500, "collection committed but not mirrored") return col return router ``` > `bot.create_collection` is module-level if the codebase exposes `bot.` > module functions (as the propose path uses `bot.open_idea_pr`); if `Bot` is a > class instance, call it the same way the propose route does. Match the existing > convention exactly. - [ ] **Step 5: Mount the router** in `backend/app/main.py` next to the other `make_router` mounts. ```python from . import api_collections app.include_router(api_collections.make_router(config, gitea)) ``` - [ ] **Step 6: Run the create-collection tests, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_create_vertical.py -q` Expected: PASS. - [ ] **Step 7: Run the full backend suite, verify PASS.** Run: `cd backend && python -m pytest -q` Expected: PASS. - [ ] **Step 8: Commit.** ```bash git add backend/app/api_collections.py backend/app/bot.py backend/app/main.py backend/tests/test_collection_create_vertical.py git commit -m "§22 S2: create-collection endpoint (bot commit + registry refresh)" ``` --- ## Task 6: Frontend — collection-scoped path + API helpers **Files:** - Modify: `frontend/src/lib/entryPaths.js`, `frontend/src/api.js` - Test: `frontend/src/lib/entryPaths.test.js` (extend) - [ ] **Step 1: Write the failing test** for `useCollectionId` fallback + scoped API URLs. ```js // Append to frontend/src/lib/entryPaths.test.js import { describe, it, expect } from 'vitest' import { DEFAULT_COLLECTION } from './entryPaths' describe('collection paths', () => { it('entryPath honors an explicit collection', () => { // entryPath(pid, slug, cid) const { entryPath } = require('./entryPaths') expect(entryPath('ohm', 'a', 'features')).toBe('/p/ohm/c/features/e/a') expect(entryPath('ohm', 'a')).toBe(`/p/ohm/c/${DEFAULT_COLLECTION}/e/a`) }) }) ``` > Match the file's existing import/runner style (it may use ESM `import` rather > than `require`); follow whatever `entryPaths.test.js` already does. - [ ] **Step 2: Run it, verify PASS or FAIL** (entryPath already supports cid → this asserts the contract; the new piece is `useCollectionId`). If green, proceed; the behavioral gap is the hook. Run: `cd frontend && npx vitest run src/lib/entryPaths.test.js` - [ ] **Step 3: Add `useCollectionId`** to `entryPaths.js`. ```js import { useParams } from 'react-router-dom' // The collection id a component should scope to: the `/c/:collectionId/` segment // when present, else the project's default collection. export function useCollectionId() { const { collectionId } = useParams() return collectionId || DEFAULT_COLLECTION } ``` - [ ] **Step 4: Make `api.js` collection-aware.** Add optional `collectionId` to the corpus reads and propose; add `listCollections` + `createCollection`. ```js export async function listRFCs(projectId, collectionId) { if (projectId && collectionId) { return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs`)) } const url = projectId ? `/api/projects/${projectId}/rfcs` : '/api/rfcs' return jsonOrThrow(await fetch(url)) } export async function getRFC(projectId, slug, collectionId) { if (collectionId) { return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections/${collectionId}/rfcs/${slug}`)) } if (slug === undefined) return jsonOrThrow(await fetch(`/api/rfcs/${projectId}`)) return jsonOrThrow(await fetch(`/api/projects/${projectId}/rfcs/${slug}`)) } export async function listCollections(projectId) { return jsonOrThrow(await fetch(`/api/projects/${projectId}/collections`)) } export async function createCollection(projectId, { collectionId, type, name, visibility, initialState }) { const res = await fetch(`/api/projects/${projectId}/collections`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ collection_id: collectionId, type, name: name || null, visibility: visibility || null, initial_state: initialState || null }), }) return jsonOrThrow(res) } ``` And add an optional `collectionId` to `proposeRFC` so it targets the scoped route: ```js export async function proposeRFC(projectId, { title, slug, pitch, tags, proposedUseCase, collectionId }) { const url = (projectId && collectionId) ? `/api/projects/${projectId}/collections/${collectionId}/rfcs/propose` : (projectId ? `/api/projects/${projectId}/rfcs/propose` : '/api/rfcs/propose') // ...unchanged body/post... } ``` - [ ] **Step 5: Run the frontend unit tests, verify PASS.** Run: `cd frontend && npx vitest run src/lib/entryPaths.test.js` Expected: PASS. - [ ] **Step 6: Commit.** ```bash git add frontend/src/lib/entryPaths.js frontend/src/api.js frontend/src/lib/entryPaths.test.js git commit -m "§22 S2: collection-scoped frontend path + API helpers" ``` --- ## Task 7: Frontend — Catalog reads the active collection **Files:** - Modify: `frontend/src/components/Catalog.jsx`, `frontend/src/App.jsx` (propose wiring) - [ ] **Step 1: Scope the Catalog to the active collection.** In `Catalog.jsx`: - import `useCollectionId` from `../lib/entryPaths`; - `const cid = useCollectionId()`; - fetch `listRFCs(pid, cid)` and add `cid` to the effect deps; - build entry links with `entryPath(pid, r.slug, cid)` and `proposalPath(pid, p.pr_number, cid)`. The anonymous empty-state (C3.6) is already correct — the footer renders the "Sign in to propose" link for `!viewer`, and the empty list shows "No RFCs in the catalog yet." Confirm both render for an anonymous viewer on an empty collection. - [ ] **Step 2: Thread the active collection into the propose modal** in `App.jsx`. The `ProposeModal` is mounted with `projectId={currentProjectId}`; also pass the active collection (derive from the route — read `useParams().collectionId` in the `AppShell` scope, default `DEFAULT_COLLECTION`) and pass it to `proposeRFC` so a propose from a named collection targets that collection. On submit, navigate with `proposalPath(currentProjectId, pr_number, currentCollectionId)`. - [ ] **Step 3: Build + run the frontend test suite, verify PASS.** Run: `cd frontend && npx vitest run` Expected: PASS (existing tests green; default-collection behavior unchanged). - [ ] **Step 4: Commit.** ```bash git add frontend/src/components/Catalog.jsx frontend/src/App.jsx git commit -m "§22 S2: Catalog + propose scoped to the active collection" ``` --- ## Task 8: Frontend — the collection directory at `/p//` **Files:** - Create: `frontend/src/components/CollectionDirectory.jsx` - Modify: `frontend/src/App.jsx` (swap `DefaultCollectionRedirect` → `CollectionDirectory`) - Test: `frontend/src/components/CollectionDirectory.test.jsx` - [ ] **Step 1: Write the failing test.** ```jsx // frontend/src/components/CollectionDirectory.test.jsx import { render, screen, waitFor } from '@testing-library/react' import { MemoryRouter } from 'react-router-dom' import { describe, it, expect, vi } from 'vitest' import CollectionDirectory from './CollectionDirectory' vi.mock('../api', () => ({ listCollections: vi.fn(async () => ({ items: [ { id: 'default', name: 'Model', type: 'document' }, { id: 'features', name: 'Features', type: 'bdd' }, ] })), })) describe('CollectionDirectory', () => { it('lists collections when there are 2+', async () => { render() await waitFor(() => expect(screen.getByText('Features')).toBeInTheDocument()) expect(screen.getByText('Model')).toBeInTheDocument() }) }) ``` > Match the project's component-test conventions (see `Directory.test.jsx` / > `ProjectLayout.test.jsx` for the render + mock pattern; adapt the mock + queries > to whatever they use). - [ ] **Step 2: Run it, verify it fails** (component missing). Run: `cd frontend && npx vitest run src/components/CollectionDirectory.test.jsx` Expected: FAIL. - [ ] **Step 3: Implement `CollectionDirectory.jsx`.** Fetch `listCollections(projectId)`; when exactly one visible collection, `` to its `collectionHome` (preserves the S1 C3.7/C3.8 single-collection redirect); when 2+, render a list of links to each `collectionHome(projectId, c.id)` with its name + type. (The "Create your first collection" empty-state is S4 — for S2 a 0-collection project simply shows a minimal "No collections yet." line; the create affordance UI lands in S4.) ```jsx import { useEffect, useState } from 'react' import { Link, Navigate } from 'react-router-dom' import { listCollections } from '../api' import { collectionHome } from '../lib/entryPaths' export default function CollectionDirectory({ projectId }) { const [cols, setCols] = useState(null) useEffect(() => { let live = true listCollections(projectId).then(d => { if (live) setCols(d.items) }).catch(() => live && setCols([])) return () => { live = false } }, [projectId]) if (cols === null) return
Loading…
if (cols.length === 1) return return (

Collections

{cols.length === 0 ? (

No collections yet.

) : (
    {cols.map(c => (
  • {c.name || c.id} · {c.type}
  • ))}
)}
) } ``` - [ ] **Step 4: Wire it into `App.jsx`.** Replace the `DefaultCollectionRedirect` route element at the project landing (App.jsx:365) with `} />`. Read the project id from `useParams()` inside a small wrapper (mirroring how `DefaultCollectionRedirect` reads it), and keep `DefaultCollectionRedirect` only if still referenced elsewhere (otherwise delete it). The S1 single-collection redirect now lives inside `CollectionDirectory`, so the C3.7/C3.8 behavior is preserved. - [ ] **Step 5: Run the directory test + full frontend suite, verify PASS.** Run: `cd frontend && npx vitest run` Expected: PASS. - [ ] **Step 6: Commit.** ```bash git add frontend/src/components/CollectionDirectory.jsx frontend/src/components/CollectionDirectory.test.jsx frontend/src/App.jsx git commit -m "§22 S2: collection directory at /p// (1 → redirect, 2+ → list)" ``` --- ## Task 9: Acceptance — C3.6 anonymous empty-collection catalog **Files:** - Test: `backend/tests/test_collection_scoped_serve.py` (append the `@S2` acceptance assertion) The `@S2` gate is C3.6: *a public collection with no entries; an anonymous visitor lands on `/p/ohm/c/model/`; they see an empty catalog with no propose action and a sign-in prompt.* Backend half: the scoped list returns `{items: []}` for an anonymous viewer on a public empty collection (no 404, no propose surfaced). Frontend half: the Catalog footer renders "Sign in to propose" for `!viewer` (already covered by Task 7). - [ ] **Step 1: Write the acceptance test** (backend contract for the empty public collection). ```python def test_s2_anonymous_empty_public_collection(client_with_corpus): """C3.6 (@S2): anonymous viewer, public empty collection → empty catalog, 200.""" client, _ = client_with_corpus # ensure an empty public collection 'model' r = client.get("/api/projects/ohm/collections/model/rfcs") # no auth header assert r.status_code == 200 assert r.json()["items"] == [] ``` > If `client_with_corpus` doesn't already seed an empty public `model` collection, > add one (mirror Task 3's seed). The propose-absence is enforced by `require_contributor` > on the propose route (anonymous → 401/403) and the Catalog UI footer. - [ ] **Step 2: Run it, verify PASS.** Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py::test_s2_anonymous_empty_public_collection -q` Expected: PASS. - [ ] **Step 3: Manually verify the propose route rejects anonymous** (sanity — no new code): Run: `cd backend && python -m pytest tests/test_collection_scoped_serve.py -q` Expected: PASS (all). - [ ] **Step 4: Commit.** ```bash git add backend/tests/test_collection_scoped_serve.py git commit -m "§22 S2: @S2 acceptance — anonymous empty public collection catalog" ``` --- ## Task 10: Release — version bump, changelog, docs **Files:** - Modify: `VERSION`, `frontend/package.json`, `CHANGELOG.md` - Modify: `docs/design/2026-06-05-three-tier-projects-collections.md` (mark S2 shipped) S2 adds functionality and is non-breaking (new optional path segment + new endpoints; the default-collection paths are unchanged). Per SPEC §20, that's a **minor** bump: `0.40.0` → `0.41.0`. - [ ] **Step 1: Bump `VERSION`** to `0.41.0`. - [ ] **Step 2: Bump `frontend/package.json#version`** to `0.41.0` (must mirror VERSION — §20). - [ ] **Step 3: Add the `CHANGELOG.md` entry** under a new `## 0.41.0` heading: a minor release shipping S2 (create + navigate + propose a second collection): registry mirror reads `.collection.yaml`; collection-grained corpus mirror; create-collection endpoint; collection-scoped list/get/propose; `/p//` collection directory. Note it completes `@S2` (C3.6). No upgrade steps required (additive; existing default-collection deployments keep working unchanged) — state that explicitly. - [ ] **Step 4: Mark S2 shipped** in the design doc's Part E slice list (a short "Landed vX" note on the S2 bullet, mirroring how S1 was annotated). - [ ] **Step 5: Run both suites once more, verify green.** Run: `cd backend && python -m pytest -q && cd ../frontend && npx vitest run` Expected: PASS (both). - [ ] **Step 6: Commit.** ```bash git add VERSION frontend/package.json CHANGELOG.md docs/design/2026-06-05-three-tier-projects-collections.md git commit -m "§22 S2: release v0.41.0 — create & navigate a second collection (@S2)" ``` --- ## Self-review notes - **Spec coverage (Part E "S2" bullet):** registry reads `.collection.yaml` (Task 1) · create-collection endpoint, admin-gated (Task 5) · project collection-directory at `/p//` (Task 8) · collection-scoped propose/serve (Tasks 3,4,7) · `@S2` / C3.6 acceptance (Task 9). ✓ - **N=1 unchanged invariant:** every backend task ends by running the full suite; the default-collection routes are preserved as thin wrappers over the new collection-scoped internals. ✓ - **Type consistency:** `collection_id`/`subfolder` naming is uniform; the entries path is `/rfcs/` everywhere (cache mirror Task 3, propose Task 4, create manifest Task 5). - **Deferred / lower-confidence calls (log to transcript):** (a) create-collection commits straight to `main` rather than via a PR — chosen because a collection is structural config and the registry mirror is the source of truth; (b) S2 collection visibility filtering is coarse (project read-gate + drop `unlisted`); full scope-role enforcement is S3; (c) the per-collection "create"/"propose-first" *empty-state affordances* are S4 — S2 ships only the anonymous empty catalog (C3.6) and a minimal directory.