§22 M3 backend Plan A: registry mirror + runtime config (v0.33.0) (#11)

Registry mirror, GET /api/deployment + /api/projects/:id, initial_state/unreviewed semantics, migration 027 (additive), hard-cut META_REPO -> REGISTRY_REPO. Full suite 434 green; version 0.33.0.
This commit is contained in:
Ben Stull
2026-06-04 07:34:47 +00:00
35 changed files with 3223 additions and 136 deletions
+68
View File
@@ -23,6 +23,74 @@ skip versions are the composition of each intervening adjacent
release's steps in order — no A-to-B path is pre-computed beyond
that.
## 0.33.0 — 2026-06-04
**Minor (breaking) — §22 M3 backend Plan A: project registry mirror +
runtime config. The framework now learns its projects from
`REGISTRY_REPO/projects.yaml`; `META_REPO` is retired. Migration `027`
runs automatically on deploy.**
Added:
- **Project registry mirror** (`app/registry.py`): the framework reads
`projects.yaml` from `REGISTRY_REPO` and mirrors its `deployment:`
block and `projects:` entries into the `projects` table + a new
`deployment` singleton. The mirror runs at startup (reconciler) and
on every §4 webhook push to the registry repo.
- **`GET /api/deployment`** — returns `name`, `tagline`, and the list
of visible projects (each item carries `id`, `name`, `type`,
`visibility`). Replaces the build-time `VITE_APP_NAME` as the
authoritative runtime config source (the frontend cut lands in
M3-frontend).
- **`GET /api/projects/:id`** — returns `id`, `name`, `tagline`, `type`,
`visibility`, `initial_state`, and `theme` for a single project.
- **§22.4b `initial_state`** honored at propose time: new RFCs enter
the state named by the project's `initial_state` field (default:
`super-draft`; `bdd` projects default to `active`).
- **§22.4c `unreviewed` flag**: newly proposed RFCs are flagged
`unreviewed = true`. Owners clear it via
`POST /api/projects/:id/rfcs/:slug/mark-reviewed`. The catalog
accepts `?unreviewed=true` to filter to the review queue.
- **Migration `027`** (additive): adds `projects.type` /
`projects.initial_state`, the `deployment` table, and
`cached_rfcs.unreviewed` / `reviewed_at` / `reviewed_by`.
Breaking:
- `META_REPO` is retired. The app **refuses to start** if `REGISTRY_REPO`
is unset. The corpus mirror now reads each project's `content_repo`
from `projects.yaml` rather than from the `META_REPO` env var.
Upgrade steps:
1. **MUST** create a registry repo under your deployment's Gitea org.
2. **MUST** author `projects.yaml` at its root with a `deployment:` block
(`name`, `tagline`) and one `projects:` entry for your existing corpus:
```yaml
deployment:
name: <your deployment name>
tagline: <your tagline>
projects:
- id: default
name: <your project name>
type: document
content_repo: <your old META_REPO value>
visibility: public
```
Keep `id: default` for this release; the pretty-slug re-stamp lands in
the next backend slice, before any `/p/` URL is public.
3. **MUST** set `REGISTRY_REPO=<your registry repo name>` and **MUST**
remove `META_REPO` (or leave it unset — it is ignored but its presence
may cause confusion).
4. **MUST** add a Gitea webhook on the registry repo pointing at
`/api/webhooks/gitea` (same secret as the corpus webhook).
5. **MUST** deploy. Migration `027` runs automatically at startup; the
registry mirror reconciles immediately after. Verify
`GET /api/deployment` returns your project and `/api/health` is green.
6. **SHOULD** rebuild the frontend (no new env var is required until
M3-frontend lands, but the frontend currently still reads
`VITE_APP_NAME` for the display name).
## 0.32.0 — 2026-06-01
**Minor — graduation's integer RFC number is now optional, and RFC
+2
View File
@@ -1,3 +1,5 @@
@~/.claude/wiggleverse.md
# Working in rfc-app
This is the framework — the software that hosts an RFC standardization
+1 -1
View File
@@ -1 +1 @@
0.32.0
0.33.0
+10 -11
View File
@@ -9,19 +9,18 @@ GITEA_URL=http://localhost:3000
GITEA_BOT_USER=rfc-bot
GITEA_BOT_TOKEN=
# The Gitea org or user that owns the meta repo and every RFC repo
# the bot will create on graduation.
# The Gitea org or user that owns every RFC repo the bot will create on
# graduation.
GITEA_ORG=wiggleverse
META_REPO=meta
# --- Multi-project registry (§22) ---
# The registry repo (under GITEA_ORG) the framework reads to learn which
# projects this deployment hosts. Optional through Slice M1: the registry
# mirror lands in M3, and until then the single default project's content
# repo is taken from META_REPO above. The repo's name is the deployment's
# choice. When the mirror lands, REGISTRY_REPO becomes required and
# supersedes META_REPO.
# REGISTRY_REPO=wiggleverse-registry
# §22.2 — the project registry repo (REQUIRED). The framework reads
# `projects.yaml` at its root to learn which projects exist. The repo name is
# the deployment's choice; the app fails to start if this is unset.
REGISTRY_REPO=registry
# §22.13 — optional id for the bootstrap/default project. Reserved for the
# Plan B re-stamp; leave unset in Plan A (the default project id stays
# `default`). When set, it must match an `id` in projects.yaml.
# DEFAULT_PROJECT_ID=ohm
# --- OAuth (Gitea) ---
# In Gitea: Site Administration → Applications → Add OAuth2 Application.
+61 -9
View File
@@ -22,12 +22,14 @@ from . import (
api_admin,
api_branches,
api_contributions,
api_deployment,
api_discussion,
api_graduation,
api_invitations,
api_notifications,
api_prs,
auth,
projects as projects_mod,
db,
device_trust as device_trust_mod,
docs as docs_mod,
@@ -146,6 +148,8 @@ def make_router(
# (super-draft) RFC. Reuses the #12 invite flow (api_invitations above)
# on accept; lands the request + owner notifications via §15 notify.
router.include_router(api_contributions.make_router())
# §22.9 (M3): runtime deployment + per-project config (replaces VITE_APP_NAME).
router.include_router(api_deployment.make_router())
# ---------------------------------------------------------------
# §17: /api/health — unauthenticated post-flight probe.
@@ -607,13 +611,16 @@ def make_router(
# ---------------------------------------------------------------
@router.get("/api/rfcs")
async def list_rfcs(request: Request) -> dict[str, Any]:
async def list_rfcs(request: Request, unreviewed: str | None = None) -> dict[str, Any]:
"""§7's left pane data.
The chip-filter / sort / search combinatorics live on the
client — the server returns the full set and lets the chips
narrow it. The set is small (hundreds, not thousands) for the
foreseeable future, so paginating here would buy nothing.
§22.4c: pass `?unreviewed=true` to narrow to active entries
still awaiting owner review.
"""
viewer = auth.current_user(request)
viewer_id = viewer.user_id if viewer else None
@@ -623,6 +630,10 @@ def make_router(
if not visible:
return {"items": []}
placeholders = ",".join("?" for _ in visible)
params = list(visible)
unreviewed_clause = ""
if unreviewed is not None and unreviewed.lower() in ("1", "true", "yes"):
unreviewed_clause = " AND unreviewed = 1 AND state = 'active'"
rows = db.conn().execute(
f"""
SELECT slug, title, state, rfc_id, repo,
@@ -630,10 +641,10 @@ def make_router(
last_main_commit_at, last_entry_commit_at, updated_at
FROM cached_rfcs
WHERE state IN ('super-draft', 'active')
AND project_id IN ({placeholders})
AND project_id IN ({placeholders}){unreviewed_clause}
ORDER BY COALESCE(last_main_commit_at, last_entry_commit_at) DESC
""",
visible,
params,
).fetchall()
starred = set()
@@ -696,6 +707,40 @@ def make_router(
payload["proposed_use_case"] = uc["use_case"] if uc else None
return payload
# ---------------------------------------------------------------
# §22.4c: mark-reviewed — clear an active entry's `unreviewed` flag
# ---------------------------------------------------------------
@router.post("/api/projects/{project_id}/rfcs/{slug}/mark-reviewed")
async def mark_reviewed(project_id: str, slug: str, request: Request) -> dict[str, Any]:
"""§22.4c — clear an active entry's `unreviewed` flag. Authority is the
§22.7 project superuser (project_admin or deployment owner/admin)."""
viewer = auth.require_user(request)
auth.require_project_readable(viewer, project_id)
if not auth.is_project_superuser(viewer, project_id):
raise HTTPException(403, "Only a project owner/admin can mark an entry reviewed")
row = db.conn().execute(
"SELECT state, unreviewed FROM cached_rfcs WHERE slug = ? AND project_id = ?",
(slug, project_id),
).fetchone()
if row is None:
raise HTTPException(404, "Not found")
if row["state"] != "active" or not row["unreviewed"]:
raise HTTPException(409, "Entry is not an unreviewed active entry")
try:
await bot.mark_entry_reviewed(
viewer.as_actor(),
org=config.gitea_org,
meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=slug,
reviewed_by=viewer.gitea_login,
reviewed_at=entry_mod.today(),
)
except GiteaError as e:
raise HTTPException(502, f"Gitea: {e.detail}")
await cache.refresh_meta_repo(config, gitea)
return {"ok": True}
# ---------------------------------------------------------------
# §7.3 / §9.3: pending ideas
# ---------------------------------------------------------------
@@ -765,7 +810,7 @@ def make_router(
# Read the proposed entry file from the head branch.
slug = row["rfc_slug"]
head = row["head_branch"]
result = await gitea.read_file(config.gitea_org, config.meta_repo, f"rfcs/{slug}.md", ref=head)
result = await gitea.read_file(config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref=head)
entry_payload: dict[str, Any] | None = None
if result:
text, _sha = result
@@ -823,10 +868,16 @@ def make_router(
if idea_clash:
raise HTTPException(409, f"Slug `{slug}` is already reserved by an open proposal")
# §22.4b: the target project's landing state. Through Plan A every
# entry lands in the default project; M3-frontend routing carries a
# non-default target later.
target_project = projects_mod.resolved_default_id(config)
landing_state = "active" if projects_mod.project_initial_state(target_project) == "active" else "super-draft"
entry = entry_mod.Entry(
slug=slug,
title=payload.title.strip(),
state="super-draft",
state=landing_state,
id=None,
repo=None,
proposed_by=user.email or user.gitea_login,
@@ -841,6 +892,7 @@ def make_router(
arbiters=[],
tags=[t.strip() for t in payload.tags if t.strip()],
body=payload.pitch.strip() + "\n",
unreviewed=(landing_state == "active"),
)
contents = entry_mod.serialize(entry)
pr_title = f"Propose: {entry.title}"
@@ -855,7 +907,7 @@ def make_router(
pr = await bot.open_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=config.meta_repo,
meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=slug,
file_contents=contents,
pr_title=pr_title,
@@ -931,7 +983,7 @@ def make_router(
await bot.merge_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=config.meta_repo,
meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
)
@@ -951,7 +1003,7 @@ def make_router(
await bot.decline_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=config.meta_repo,
meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
comment=body.comment,
@@ -975,7 +1027,7 @@ def make_router(
await bot.withdraw_idea_pr(
user.as_actor(),
org=config.gitea_org,
meta_repo=config.meta_repo,
meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=pr_number,
slug=row["rfc_slug"],
)
+2 -2
View File
@@ -29,7 +29,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, projects as projects_mod
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -1153,7 +1153,7 @@ def make_router(
def _repo_for(rfc, branch: str = "main") -> tuple[str, str]:
if _is_meta_target(rfc, branch):
return config.gitea_org, config.meta_repo
return config.gitea_org, (projects_mod.default_content_repo(config) or "")
owner, repo = rfc["repo"].split("/", 1)
return owner, repo
+72
View File
@@ -0,0 +1,72 @@
"""§22.9 runtime deployment/project config (replaces VITE_APP_NAME).
GET /api/deployment — the deployment name/tagline + the projects the caller can
see (§22.5: gated filtered by membership, unlisted omitted from enumeration).
GET /api/projects/:id — one project's runtime config + optional theme overlay,
gated behind the §22.5 read gate (404 for a non-member of a gated project).
"""
from __future__ import annotations
import json
from typing import Any
from fastapi import APIRouter, HTTPException, Request
from . import auth, db
def make_router() -> APIRouter:
router = APIRouter()
@router.get("/api/deployment")
async def get_deployment(request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
dep = db.conn().execute(
"SELECT name, tagline FROM deployment WHERE id = 1"
).fetchone()
# §22.5: enumerate only public + (member-)gated; unlisted is never listed.
visible = set(auth.visible_project_ids(viewer))
rows = db.conn().execute(
"SELECT id, name, type, visibility FROM projects "
"WHERE visibility != 'unlisted' ORDER BY name"
).fetchall()
projects = [
{"id": r["id"], "name": r["name"], "type": r["type"], "visibility": r["visibility"]}
for r in rows
if r["id"] in visible
]
return {
"name": (dep["name"] if dep else "") or "",
"tagline": (dep["tagline"] if dep else "") or "",
"projects": projects,
}
@router.get("/api/projects/{project_id}")
async def get_project(project_id: str, request: Request) -> dict[str, Any]:
viewer = auth.current_user(request)
# §22.5 read gate: a gated project 404s to a non-member (shape matches
# an unknown id). unlisted is readable by direct id.
auth.require_project_readable(viewer, project_id)
row = db.conn().execute(
"SELECT id, name, type, visibility, initial_state, config_json "
"FROM projects WHERE id = ?",
(project_id,),
).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="Not found")
try:
cfg = json.loads(row["config_json"] or "{}")
except (ValueError, TypeError):
cfg = {}
dep = db.conn().execute("SELECT tagline FROM deployment WHERE id = 1").fetchone()
return {
"id": row["id"],
"name": row["name"],
"tagline": (dep["tagline"] if dep else "") or "",
"type": row["type"],
"visibility": row["visibility"],
"initial_state": row["initial_state"],
"theme": cfg.get("theme") or {},
}
return router
+13 -13
View File
@@ -42,7 +42,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from . import auth, cache, db, entry as entry_mod
from . import auth, cache, db, entry as entry_mod, projects as projects_mod
from .bot import Actor, Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -345,7 +345,7 @@ def make_router(
# graduation PR's update_file call and the body to carry through
# unchanged (meta-only keeps the body in the entry, §13.3).
fetched = await gitea.read_file(
config.gitea_org, config.meta_repo, f"rfcs/{slug}.md", ref="main",
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
@@ -479,7 +479,7 @@ def make_router(
raise HTTPException(409, f"A claim PR is already open: #{already['pr_number']}")
fetched = await gitea.read_file(
config.gitea_org, config.meta_repo, f"rfcs/{slug}.md", ref="main",
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
@@ -495,7 +495,7 @@ def make_router(
try:
pr = await bot.open_claim_pr(
viewer.as_actor(),
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=slug,
new_file_contents=new_contents, prior_sha=meta_sha,
)
@@ -601,7 +601,7 @@ def make_router(
async def _read_meta_entry(slug: str) -> tuple[entry_mod.Entry, str]:
fetched = await gitea.read_file(
config.gitea_org, config.meta_repo, f"rfcs/{slug}.md", ref="main",
config.gitea_org, (projects_mod.default_content_repo(config) or ""), f"rfcs/{slug}.md", ref="main",
)
if fetched is None:
raise HTTPException(409, f"Meta entry rfcs/{slug}.md not found on main")
@@ -656,7 +656,7 @@ async def _orchestrate(
try:
pr = await bot.open_graduation_pr(
actor,
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=state.slug,
new_file_contents=graduated_contents,
prior_sha=meta_file_sha,
@@ -676,7 +676,7 @@ async def _orchestrate(
try:
await bot.merge_graduation_pr(
actor,
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=state.new_pr_number,
head_branch=state.graduation_branch or "",
slug=state.slug, rfc_id=state.rfc_id,
@@ -738,7 +738,7 @@ async def _cleanup_unmerged(
try:
await bot.close_graduation_pr(
actor,
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=state.new_pr_number,
head_branch=state.graduation_branch or "",
slug=state.slug, reason="graduation merge failed",
@@ -751,7 +751,7 @@ async def _cleanup_unmerged(
await bot.delete_branch(
actor,
owner=config.gitea_org,
repo=config.meta_repo,
repo=(projects_mod.default_content_repo(config) or ""),
branch=branch_name,
slug=state.slug,
action_kind="delete_post_merge_branch",
@@ -861,7 +861,7 @@ async def _run_state_flip(
try:
pr = await bot.open_retire_flip_pr(
actor,
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
slug=slug, new_file_contents=new_contents, prior_sha=prior_sha,
verb=verb, target_state=target_state,
)
@@ -872,7 +872,7 @@ async def _run_state_flip(
try:
await bot.merge_retire_flip_pr(
actor,
org=config.gitea_org, meta_repo=config.meta_repo,
org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=pr_number, head_branch=head_branch,
slug=slug, verb=verb,
)
@@ -881,12 +881,12 @@ async def _run_state_flip(
# accumulate (mirrors graduation's `_cleanup_unmerged`).
try:
await bot.close_graduation_pr(
actor, org=config.gitea_org, meta_repo=config.meta_repo,
actor, org=config.gitea_org, meta_repo=(projects_mod.default_content_repo(config) or ""),
pr_number=pr_number, head_branch=head_branch,
slug=slug, reason=f"{verb} merge failed",
)
await bot.delete_branch(
actor, owner=config.gitea_org, repo=config.meta_repo,
actor, owner=config.gitea_org, repo=(projects_mod.default_content_repo(config) or ""),
branch=head_branch, slug=slug,
action_kind="delete_post_merge_branch",
reason=f"{verb} merge failed",
+2 -2
View File
@@ -23,7 +23,7 @@ from typing import Any
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, rfc_links
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, projects as projects_mod, rfc_links
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -691,7 +691,7 @@ def make_router(
def _owner_repo(rfc) -> tuple[str, str]:
if _is_meta_resident(rfc):
return config.gitea_org, config.meta_repo
return config.gitea_org, (projects_mod.default_content_repo(config) or "")
owner, repo = rfc["repo"].split("/", 1)
return owner, repo
+49 -1
View File
@@ -27,7 +27,7 @@ import json
import logging
from dataclasses import dataclass
from . import db, notify
from . import db, entry as entry_mod, notify
from .gitea import Gitea, GiteaError
log = logging.getLogger(__name__)
@@ -1069,6 +1069,54 @@ class Bot:
)
return pr
# ----- §22.4c: mark-reviewed (direct main write) -----
async def mark_entry_reviewed(
self,
actor: Actor,
*,
org: str,
meta_repo: str,
slug: str,
reviewed_by: str,
reviewed_at: str,
) -> None:
"""Clear §22.4c unreviewed on an active entry by rewriting its
frontmatter on main. Stamps the commit with the §6.5 On-behalf-of
trailer and writes an actions-log row, mirroring the graduation
stamp's bot-write shape."""
path = f"rfcs/{slug}.md"
result = await self._gitea.read_file(org, meta_repo, path, ref="main")
if result is None:
raise GiteaError(404, f"{path} not found")
text, sha = result
e = entry_mod.parse(text)
e.unreviewed = False
e.reviewed_at = reviewed_at
e.reviewed_by = reviewed_by
commit_message = _stamp_single(f"Mark {slug} reviewed", actor)
result = await self._gitea.update_file(
org, meta_repo, path,
content=entry_mod.serialize(e),
sha=sha,
message=commit_message,
branch="main",
author_name=actor.display_name,
author_email=actor.email or f"{actor.gitea_login}@users.noreply",
)
commit_sha = (
result.get("commit", {}).get("sha")
or result.get("content", {}).get("sha")
or ""
)
_log(
actor,
"mark_reviewed",
rfc_slug=slug,
bot_commit_sha=commit_sha,
details={"reviewed_by": reviewed_by, "reviewed_at": reviewed_at},
)
# ----- Per-RFC repo: seeding (test/dev fixtures, future graduation) -----
async def ensure_rfc_repo_seed(
+30 -6
View File
@@ -27,7 +27,7 @@ import asyncio
import json
import logging
from . import db, entry as entry_mod
from . import db, entry as entry_mod, projects as projects_mod, registry as registry_mod
from .config import Config
from .gitea import Gitea, GiteaError
@@ -40,7 +40,11 @@ async def refresh_meta_repo(config: Config, gitea: Gitea) -> None:
Idempotent. Safe to call on every meta-repo webhook and on every
reconciler sweep.
"""
org, repo = config.gitea_org, config.meta_repo
org = config.gitea_org
repo = projects_mod.default_content_repo(config)
if not repo:
log.warning("refresh_meta_repo: default project has no content_repo yet; skipping")
return
try:
files = await gitea.list_dir(org, repo, "rfcs", ref="main")
except GiteaError as e:
@@ -87,8 +91,10 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None:
INSERT INTO cached_rfcs
(slug, title, state, rfc_id, repo, proposed_by, proposed_at,
graduated_at, graduated_by, owners_json, arbiters_json, tags_json,
models_json, funder_login, body, body_sha, last_entry_commit_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now'))
models_json, funder_login, body, body_sha,
unreviewed, reviewed_at, reviewed_by,
last_entry_commit_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now'))
ON CONFLICT(slug) DO UPDATE SET
title = excluded.title,
state = excluded.state,
@@ -105,6 +111,9 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None:
funder_login = excluded.funder_login,
body = excluded.body,
body_sha = excluded.body_sha,
unreviewed = excluded.unreviewed,
reviewed_at = excluded.reviewed_at,
reviewed_by = excluded.reviewed_by,
last_entry_commit_at = datetime('now'),
updated_at = datetime('now')
""",
@@ -125,6 +134,9 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None:
funder_login,
entry.body,
body_sha,
1 if entry.unreviewed else 0,
entry.reviewed_at,
entry.reviewed_by,
),
)
@@ -320,7 +332,11 @@ async def refresh_meta_branches(config: Config, gitea: Gitea) -> None:
structurally `edit/<slug>/<auto-name>` per §9.5, with dashes in place
of slashes per the §19.2 path-routing candidate.
"""
org, repo = config.gitea_org, config.meta_repo
org = config.gitea_org
repo = projects_mod.default_content_repo(config)
if not repo:
log.warning("refresh_meta_branches: default project has no content_repo yet; skipping")
return
try:
branches = await gitea.list_branches(org, repo)
except GiteaError as e:
@@ -437,7 +453,11 @@ async def refresh_meta_pulls(config: Config, gitea: Gitea) -> None:
`On-behalf-of:` trailer from the PR body, then to the raw Gitea
login as last resort.
"""
org, repo = config.gitea_org, config.meta_repo
org = config.gitea_org
repo = projects_mod.default_content_repo(config)
if not repo:
log.warning("refresh_meta_pulls: default project has no content_repo yet; skipping")
return
repo_full = f"{org}/{repo}"
try:
open_pulls = await gitea.list_pulls(org, repo, state="open")
@@ -663,6 +683,10 @@ class Reconciler:
async def sweep(self) -> None:
log.info("reconciler: starting sweep")
try:
try:
await registry_mod.refresh_registry(self._config, self._gitea)
except Exception:
log.exception("reconciler: registry refresh failed; keeping last-good projects")
await refresh_meta_repo(self._config, self._gitea)
await refresh_meta_branches(self._config, self._gitea)
await refresh_meta_pulls(self._config, self._gitea)
+5 -11
View File
@@ -32,7 +32,6 @@ class Config:
gitea_bot_user: str
gitea_bot_token: str
gitea_org: str
meta_repo: str
registry_repo: str
oauth_client_id: str
oauth_client_secret: str
@@ -45,14 +44,15 @@ class Config:
anthropic_api_key: str = ""
google_api_key: str = ""
openai_api_key: str = ""
default_project_id: str = ""
@property
def redirect_uri(self) -> str:
return f"{self.app_url}/auth/callback"
@property
def meta_repo_full(self) -> str:
return f"{self.gitea_org}/{self.meta_repo}"
def registry_repo_full(self) -> str:
return f"{self.gitea_org}/{self.registry_repo}"
def load_config() -> Config:
@@ -80,14 +80,7 @@ def load_config() -> Config:
gitea_bot_user=_required("GITEA_BOT_USER"),
gitea_bot_token=_required("GITEA_BOT_TOKEN"),
gitea_org=_required("GITEA_ORG"),
meta_repo=_optional("META_REPO", "meta"),
# §22.2: the multi-project registry repo the framework reads to learn
# which projects exist. Optional through Slice M1 — the registry
# *mirror* lands in M3; until then the default project (seeded by
# migration 026 + the §22.13 startup backfill) is the only project,
# and its content_repo comes from META_REPO. Becomes required when
# the mirror lands and META_REPO is retired.
registry_repo=_optional("REGISTRY_REPO"),
registry_repo=_required("REGISTRY_REPO"),
oauth_client_id=_required("OAUTH_CLIENT_ID"),
oauth_client_secret=_required("OAUTH_CLIENT_SECRET"),
app_url=_optional("APP_URL", "http://localhost:8000").rstrip("/"),
@@ -99,4 +92,5 @@ def load_config() -> Config:
anthropic_api_key=_optional("ANTHROPIC_API_KEY"),
google_api_key=_optional("GOOGLE_API_KEY"),
openai_api_key=_optional("OPENAI_API_KEY"),
default_project_id=_optional("DEFAULT_PROJECT_ID"),
)
+19
View File
@@ -50,6 +50,13 @@ class Entry:
# operator credentials per §18 are used. The binding is inert until
# the named user has a funder_consents row (the hybrid two-key rule).
funder: str | None = None
# §22.4c: an `active` entry that landed without a human review gate
# carries unreviewed=True until an owner clears it. Orthogonal to
# `state`; only meaningful for active entries. reviewed_at/reviewed_by
# are the provenance of the clear, paralleling graduated_at/by.
unreviewed: bool = False
reviewed_at: str | None = None
reviewed_by: str | None = None
body: str = ""
@@ -66,6 +73,7 @@ def parse(text: str) -> Entry:
models = [str(m) for m in raw_models]
raw_funder = fm.get("funder")
funder = str(raw_funder).strip() if raw_funder else None
unreviewed = bool(fm.get("unreviewed") or False)
return Entry(
slug=str(fm.get("slug") or ""),
title=str(fm.get("title") or ""),
@@ -81,6 +89,9 @@ def parse(text: str) -> Entry:
tags=list(fm.get("tags") or []),
models=models,
funder=funder,
unreviewed=unreviewed,
reviewed_at=fm.get("reviewed_at") or None,
reviewed_by=fm.get("reviewed_by") or None,
body=body,
)
@@ -110,6 +121,14 @@ def serialize(entry: Entry) -> str:
# second meaning here as with `models:`; one set of semantics.
if entry.funder:
fm["funder"] = entry.funder
# §22.4c: emit unreviewed only when True (a super-draft / reviewed
# active entry leaves the key absent → frontmatter stays minimal).
if entry.unreviewed:
fm["unreviewed"] = True
if entry.reviewed_at:
fm["reviewed_at"] = entry.reviewed_at
if entry.reviewed_by:
fm["reviewed_by"] = entry.reviewed_by
yaml_text = yaml.safe_dump(fm, sort_keys=False, default_flow_style=False).rstrip()
body = entry.body.lstrip("\n")
if body:
+9 -2
View File
@@ -30,7 +30,7 @@ import logging
import os
from datetime import datetime, timedelta, timezone
from . import db
from . import db, projects as projects_mod
from .bot import Bot
from .config import Config
@@ -299,7 +299,14 @@ async def _delete_branch_via_bot(
log.warning("hygiene: cannot delete %s/%s — slug missing from cache", slug, branch)
return False
if not rfc["repo"]:
owner, repo = config.gitea_org, config.meta_repo
repo = projects_mod.default_content_repo(config)
if not repo:
log.warning(
"hygiene: default project has no content_repo; skipping branch delete for %s/%s",
slug, branch,
)
return False
owner = config.gitea_org
elif "/" in rfc["repo"]:
owner, repo = rfc["repo"].split("/", 1)
else:
+18 -2
View File
@@ -31,6 +31,7 @@ from . import (
projects,
providers as providers_mod,
ratelimit,
registry as registry_mod,
turnstile,
webhooks,
)
@@ -100,8 +101,23 @@ async def lifespan(app: FastAPI):
config = load_config()
db.run_migrations(config)
db.init(config)
projects.seed_default_project(config) # §22.13 step 1
gitea = Gitea(config)
# §22.2: mirror the registry before anything reads projects/content_repo.
# First boot has no last-good rows, so a missing/invalid registry is fatal
# (loud-fail per separation-of-concerns); the reconciler sweep keeps it
# fresh thereafter and tolerates a later bad PR.
try:
await registry_mod.refresh_registry(config, gitea)
except Exception as e:
raise RuntimeError(
f"registry mirror failed at startup ({config.registry_repo_full}/projects.yaml): {e}"
) from e
if projects.default_content_repo(config) is None:
raise RuntimeError(
f"registry does not describe the default project "
f"{projects.resolved_default_id(config)!r} (no content_repo). "
f"Add it to {config.registry_repo_full}/projects.yaml."
)
bot = Bot(gitea)
reconciler = cache.Reconciler(config, gitea)
digest_sched = digest.DigestScheduler()
@@ -130,7 +146,7 @@ async def lifespan(app: FastAPI):
reconciler.start()
digest_sched.start()
hygiene_sched.start()
log.info("RFC app started — meta repo %s/%s", config.gitea_org, config.meta_repo)
log.info("RFC app started — registry %s", config.registry_repo_full)
try:
yield
finally:
+29 -27
View File
@@ -1,11 +1,9 @@
"""Project registry — the §22 multi-project layer.
A deployment hosts one or more projects (§22.1). Through Slice M1 the only
project is the migration-seeded `default` one (the N=1 case, §22.13); the git
registry mirror that lets a deployment declare more projects lands in M3 and
will grow this module. For now this holds just the §22.13 step-1 backfill:
completing the default project's row from deployment config, which pure-SQL
migration 026 could not read.
A deployment hosts one or more projects (§22.1). The git registry mirror
that lets a deployment declare projects lands in M3 and drives this module.
`seed_default_project` (the §22.13 META_REPO backfill) is retired in M3;
the registry mirror (`registry.refresh_registry`) is authoritative.
"""
from __future__ import annotations
@@ -15,26 +13,30 @@ from .config import Config
DEFAULT_PROJECT_ID = "default"
def seed_default_project(config: Config) -> None:
"""Fill the default project's content_repo from config.
def resolved_default_id(config: Config) -> str:
"""The id of the deployment's bootstrap/default project. Plan A: always
'default' (the re-stamp to a config slug rides Plan B). The config knob is
read here so Plan B can flip the resolution without touching call sites."""
return config.default_project_id.strip() or DEFAULT_PROJECT_ID
Migration 026 seeds `default` with content_repo NULL because SQL cannot
read the environment. This sets it from META_REPO (the pre-multi-project
single-corpus repo) the first time the configured app boots, so the row
accurately names the corpus the existing rows belong to. Idempotent: only
touches the row while content_repo is still unset, so a later registry
mirror (M3) that has populated it is never clobbered.
The display `name` is deliberately left as-is: the deployment's
user-visible name lives in the frontend build (VITE_APP_NAME) today and
moves to the registry + GET /api/deployment in M3 (§22.9); the backend has
no authoritative name to backfill from yet.
"""
db.conn().execute(
"""
UPDATE projects
SET content_repo = ?, updated_at = datetime('now')
WHERE id = ? AND content_repo IS NULL
""",
(config.meta_repo, DEFAULT_PROJECT_ID),
)
def default_content_repo(config: Config) -> str | None:
"""The content repo the single-corpus mirror reads, from the default
project's row (filled by the registry mirror). Replaces the retired
META_REPO. None until the registry mirror has run."""
row = db.conn().execute(
"SELECT content_repo FROM projects WHERE id = ?",
(resolved_default_id(config),),
).fetchone()
return row["content_repo"] if row and row["content_repo"] else None
def project_initial_state(project_id: str) -> str:
"""§22.4b landing state for new entries in a project. Defaults to
'super-draft' for an unknown/unset row (the safe, today's-flow default)."""
row = db.conn().execute(
"SELECT initial_state FROM projects WHERE id = ?", (project_id,)
).fetchone()
if row is None or not row["initial_state"]:
return "super-draft"
return row["initial_state"]
+185
View File
@@ -0,0 +1,185 @@
"""§22.2 project registry mirror — the config-side analogue of
cache.refresh_meta_repo.
A deployment declares its projects in a `projects.yaml` at the root of
REGISTRY_REPO. This module mirrors that file into the `projects` cache table
and the `deployment` singleton. Per §22.2, `projects` rows flow from the
registry only never from user actions. The mirror runs on the registry-repo
webhook and on every reconciler sweep (Option A wiring, Task 5).
"""
from __future__ import annotations
import base64
import json
import logging
import re
from dataclasses import dataclass, field
import yaml
from . import db
from .config import Config
from .gitea import Gitea
log = logging.getLogger(__name__)
VALID_TYPES = {"document", "specification", "bdd"}
VALID_VISIBILITY = {"gated", "public", "unlisted"}
VALID_INITIAL_STATE = {"super-draft", "active"}
# §22.4b: per-type default landing state.
_TYPE_DEFAULT_INITIAL_STATE = {
"document": "super-draft",
"specification": "super-draft",
"bdd": "active",
}
_SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$")
class RegistryError(Exception):
"""A registry document that fails validation, or a missing registry file.
Raised by parse_registry/refresh_registry. The caller decides severity:
fatal at startup (no last-good to serve), tolerated on a running deployment
(keep the last-good projects rows). See Task 5 wiring.
"""
@dataclass
class ProjectEntry:
id: str
name: str
type: str
content_repo: str
visibility: str
initial_state: str
config: dict = field(default_factory=dict) # theme, enabled_models
@dataclass
class RegistryDoc:
deployment_name: str
deployment_tagline: str
projects: list[ProjectEntry]
def parse_registry(text: str) -> RegistryDoc:
"""Parse + validate projects.yaml. Pure (no I/O). Raises RegistryError."""
raw = yaml.safe_load(text) or {}
dep = raw.get("deployment") or {}
projects_raw = raw.get("projects") or []
if not isinstance(projects_raw, list) or not projects_raw:
raise RegistryError("registry must declare at least one project")
seen: set[str] = set()
entries: list[ProjectEntry] = []
for p in projects_raw:
if not isinstance(p, dict):
raise RegistryError(f"each project entry must be a mapping, got {type(p).__name__}")
pid = str(p.get("id") or "").strip()
if not _SLUG_RE.match(pid):
raise RegistryError(f"project id {pid!r} is not a valid slug")
if pid in seen:
raise RegistryError(f"duplicate project id {pid!r}")
seen.add(pid)
name = str(p.get("name") or "").strip()
if not name:
raise RegistryError(f"project {pid!r} missing name")
ptype = str(p.get("type") or "").strip()
if ptype not in VALID_TYPES:
raise RegistryError(f"project {pid!r} has invalid type {ptype!r}")
content_repo = str(p.get("content_repo") or "").strip()
if not content_repo:
raise RegistryError(f"project {pid!r} missing content_repo")
vis = str(p.get("visibility") or "gated").strip()
if vis not in VALID_VISIBILITY:
raise RegistryError(f"project {pid!r} has invalid visibility {vis!r}")
initial_state = str(
p.get("initial_state") or _TYPE_DEFAULT_INITIAL_STATE[ptype]
).strip()
if initial_state not in VALID_INITIAL_STATE:
raise RegistryError(
f"project {pid!r} has invalid initial_state {initial_state!r}"
)
cfg: dict = {}
if p.get("theme") is not None:
cfg["theme"] = p["theme"]
if p.get("enabled_models") is not None:
cfg["enabled_models"] = [str(m) for m in p["enabled_models"]]
entries.append(
ProjectEntry(pid, name, ptype, content_repo, vis, initial_state, cfg)
)
return RegistryDoc(
deployment_name=str(dep.get("name") or "").strip(),
deployment_tagline=str(dep.get("tagline") or "").strip(),
projects=entries,
)
def apply_registry(doc: RegistryDoc, registry_sha: str) -> None:
"""Upsert the parsed registry into projects + deployment. Idempotent.
§22.4a: `type` is immutable a change against an existing row is rejected
(skip + log), never applied. Projects absent from the registry are left in
place (archival is out of scope for M3; they simply stop refreshing).
"""
with db.tx() as conn:
for e in doc.projects:
existing = conn.execute(
"SELECT type FROM projects WHERE id = ?", (e.id,)
).fetchone()
if existing is not None and existing["type"] != e.type:
log.error(
"registry: refusing immutable type change on project %s (%s -> %s)",
e.id, existing["type"], e.type,
)
continue
conn.execute(
"""
INSERT INTO projects
(id, name, type, content_repo, visibility, initial_state,
config_json, registry_sha, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now'))
ON CONFLICT(id) DO UPDATE SET
name = excluded.name,
type = excluded.type,
content_repo = excluded.content_repo,
visibility = excluded.visibility,
initial_state = excluded.initial_state,
config_json = excluded.config_json,
registry_sha = excluded.registry_sha,
updated_at = datetime('now')
""",
(
e.id, e.name, e.type, e.content_repo, e.visibility,
e.initial_state, json.dumps(e.config), registry_sha,
),
)
conn.execute(
"""
UPDATE deployment
SET name = ?, tagline = ?, registry_sha = ?, updated_at = datetime('now')
WHERE id = 1
""",
(doc.deployment_name, doc.deployment_tagline, registry_sha),
)
async def refresh_registry(config: Config, gitea: Gitea) -> None:
"""Mirror REGISTRY_REPO/projects.yaml into projects + deployment.
Idempotent. Raises RegistryError on a missing/invalid file and GiteaError
on transport failure; the caller chooses fatal-vs-tolerated.
"""
item = await gitea.get_contents(
config.gitea_org, config.registry_repo, "projects.yaml", ref="main"
)
if not item or item.get("type") != "file":
raise RegistryError(
f"{config.gitea_org}/{config.registry_repo}/projects.yaml not found"
)
text = base64.b64decode(item["content"]).decode("utf-8")
# Prefer the file's last commit sha for provenance (production Gitea
# includes it on the contents response); fall back to the blob sha.
sha = item.get("last_commit_sha") or item.get("sha") or ""
doc = parse_registry(text)
apply_registry(doc, sha)
log.info("registry: mirrored %d project(s) at %s", len(doc.projects), sha)
+16 -8
View File
@@ -16,7 +16,7 @@ import os
from fastapi import APIRouter, Header, HTTPException, Request
from . import cache, db
from . import cache, db, projects as projects_mod, registry as registry_mod
from .config import Config
from .gitea import Gitea
@@ -79,9 +79,22 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
except Exception:
payload = {}
repo_full = (payload.get("repository") or {}).get("full_name") or ""
meta_full = f"{config.gitea_org}/{config.meta_repo}"
registry_full = f"{config.gitea_org}/{config.registry_repo}"
content_repo = projects_mod.default_content_repo(config)
if not content_repo:
log.warning("webhook: default project content_repo is unknown; corpus refresh skipped")
content_full = f"{config.gitea_org}/{content_repo}" if content_repo else None
try:
if repo_full == meta_full or not repo_full:
if repo_full == registry_full:
# §22.2: a registry-repo push re-mirrors the projects table.
# Tolerate a malformed projects.yaml (keep last-good rows); let a
# transport error bubble to the outer 500 so an unreachable Gitea
# on a registry push is loud rather than silently dropped.
try:
await registry_mod.refresh_registry(config, gitea)
except registry_mod.RegistryError:
log.exception("registry webhook: invalid projects.yaml; keeping last-good")
elif content_full and (repo_full == content_full or not repo_full):
await cache.refresh_meta_repo(config, gitea)
await cache.refresh_meta_branches(config, gitea)
await cache.refresh_meta_pulls(config, gitea)
@@ -90,11 +103,6 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
if slug:
await cache.refresh_rfc_repo(config, gitea, slug)
else:
# v0.18.0: the proposal's "unknown-repo logging"
# gesture — a hook on a fork or a stale repo binding
# used to silently 200-OK here, hiding the
# misconfiguration. Now the operator sees it in
# the log.
log.info(
"webhook received for unknown repo: repo_full=%s event=%s "
"(no cached_rfcs row matched; hook may be on a fork or stale)",
@@ -0,0 +1,31 @@
-- §22 M3 (Plan A) — additive registry/runtime-config schema.
--
-- This is the additive half of M3's backend. It adds the project `type` and
-- `initial_state` columns (mirrored from the registry), the deployment
-- singleton (deployment name/tagline mirrored from the registry), and the
-- §22.4c review columns on cached_rfcs. NO table rebuilds: the §22.13 PK
-- rebuilds and the default->slug re-stamp ride a later migration (Plan B),
-- just before a second project can collide (see migration 026's DEFERRED
-- block and docs/superpowers/specs/2026-06-03-m3-backend-design.md §1/§6).
ALTER TABLE projects ADD COLUMN type TEXT NOT NULL DEFAULT 'document'
CHECK (type IN ('document', 'specification', 'bdd'));
ALTER TABLE projects ADD COLUMN initial_state TEXT NOT NULL DEFAULT 'super-draft'
CHECK (initial_state IN ('super-draft', 'active'));
-- Deployment-level identity (name, tagline) mirrored from the registry's
-- `deployment:` block. A singleton: the CHECK pins it to one row.
CREATE TABLE IF NOT EXISTS deployment (
id INTEGER PRIMARY KEY CHECK (id = 1),
name TEXT,
tagline TEXT,
registry_sha TEXT,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
INSERT OR IGNORE INTO deployment (id) VALUES (1);
-- §22.4c review flag + provenance. unreviewed is git-truth (mirrored from
-- entry frontmatter); it survives a cache rebuild like `state` does.
ALTER TABLE cached_rfcs ADD COLUMN unreviewed INTEGER NOT NULL DEFAULT 0;
ALTER TABLE cached_rfcs ADD COLUMN reviewed_at TEXT;
ALTER TABLE cached_rfcs ADD COLUMN reviewed_by TEXT;
+97
View File
@@ -0,0 +1,97 @@
"""§22.9/§22.5 — GET /api/deployment + GET /api/projects/:id with visibility."""
from __future__ import annotations
from fastapi.testclient import TestClient
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as,
)
def _add_project(pid, name, vis, typ="document"):
from app import db
db.conn().execute(
"INSERT OR REPLACE INTO projects (id, name, type, content_repo, visibility, initial_state) "
"VALUES (?, ?, ?, ?, ?, 'super-draft')",
(pid, name, typ, pid, vis),
)
def test_deployment_lists_public_omits_gated_and_unlisted_for_anon(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
_add_project("pub", "Public", "public")
_add_project("gat", "Gated", "gated")
_add_project("unl", "Unlisted", "unlisted")
r = client.get("/api/deployment")
assert r.status_code == 200
body = r.json()
assert body["name"] == "Test Deployment"
ids = {p["id"] for p in body["projects"]}
assert "pub" in ids and "default" in ids # both public
assert "gat" not in ids # gated, anon not a member
assert "unl" not in ids # unlisted never enumerated
def test_projects_id_404_for_gated_non_member(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
_add_project("gat", "Gated", "gated")
assert client.get("/api/projects/gat").status_code == 404
def test_projects_id_returns_config_for_public(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
from app import db
db.conn().execute(
"UPDATE projects SET config_json = ? WHERE id = 'default'",
('{"theme": {"accent": "#5b5bd6"}}',),
)
r = client.get("/api/projects/default")
assert r.status_code == 200
body = r.json()
assert body["id"] == "default"
assert body["type"] == "document"
assert body["visibility"] == "public"
assert body["theme"] == {"accent": "#5b5bd6"}
def test_projects_id_unlisted_readable_by_direct_id(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
_add_project("unl", "Unlisted", "unlisted")
assert client.get("/api/projects/unl").status_code == 200
def test_projects_id_unknown_returns_404_even_for_superuser(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
assert client.get("/api/projects/does-not-exist").status_code == 404
def test_projects_id_unknown_returns_404_for_anon(app_with_fake_gitea):
app, _ = app_with_fake_gitea
with TestClient(app) as client:
assert client.get("/api/projects/nope").status_code == 404
def test_gated_project_visible_and_readable_to_member(app_with_fake_gitea):
from app import db
app, _ = app_with_fake_gitea
with TestClient(app) as client:
_add_project("teamx", "Team X", "gated")
provision_user_row(user_id=5, login="mia", role="contributor")
db.conn().execute(
"INSERT INTO project_members (project_id, user_id, role) VALUES ('teamx', 5, 'project_viewer')"
)
sign_in_as(client, user_id=5, gitea_login="mia", display_name="Mia", role="contributor")
# member sees the gated project in the deployment directory
ids = {p["id"] for p in client.get("/api/deployment").json()["projects"]}
assert "teamx" in ids
# member can read it directly
r = client.get("/api/projects/teamx")
assert r.status_code == 200
assert r.json()["id"] == "teamx"
+24
View File
@@ -0,0 +1,24 @@
"""§22.4c — _upsert_cached_rfc mirrors the review fields into cached_rfcs."""
from __future__ import annotations
from fastapi.testclient import TestClient
from test_propose_vertical import app_with_fake_gitea, tmp_env # noqa: F401
def test_upsert_writes_review_fields(app_with_fake_gitea):
from app import cache, db, entry as entry_mod
app, _ = app_with_fake_gitea
with TestClient(app):
e = entry_mod.Entry(
slug="rev", title="Rev", state="active",
unreviewed=True, reviewed_at="2026-06-03", reviewed_by="ben",
)
cache._upsert_cached_rfc(e, body_sha="sha-rev")
row = db.conn().execute(
"SELECT unreviewed, reviewed_at, reviewed_by FROM cached_rfcs WHERE slug = 'rev'"
).fetchone()
assert row["unreviewed"] == 1
assert row["reviewed_at"] == "2026-06-03"
assert row["reviewed_by"] == "ben"
+18 -13
View File
@@ -70,27 +70,32 @@ class _UpstreamHandler:
@pytest.fixture
def patched_httpx(monkeypatch):
def patched_httpx(monkeypatch, app_with_fake_gitea): # noqa: F811
"""Provide a hook the test can call to install a MockTransport.
Returns a closure: `install(handler)` patches
`app.docs_sessions.httpx.AsyncClient` so every constructed client
uses the handler's transport.
Returns a closure: `install(handler)` patches `httpx.AsyncClient`
(via `app.docs_sessions.httpx.AsyncClient`) so every constructed
client uses the handler's transport.
NB: the upstream `app_with_fake_gitea` fixture also patches
`httpx.AsyncClient` (to route gitea calls to a FakeGitea handler),
and because `httpx` is a single shared module, that patch mutates
the *same* `AsyncClient` attribute we're about to overwrite. We
therefore import the unpatched class directly from the
`httpx._client` module so our install path can construct a fresh
real client around our MockTransport without going through the
FakeGitea wrapper.
M3 note: lifespan now calls `refresh_registry` which hits FakeGitea
via the gitea transport. Since `httpx` is a module singleton, installing
the docs transport would clobber the FakeGitea mock already installed
by `app_with_fake_gitea`. We use a COMPOSITE handler: Gitea API
requests (to `http://gitea.test/`) are delegated to FakeGitea; all
other requests go to the test-specific handler.
"""
from httpx._client import AsyncClient as RealAsyncClient
_fake = app_with_fake_gitea[1]
def install(handler):
def composite(request: httpx.Request) -> httpx.Response:
if "gitea.test" in str(request.url):
return _fake.handle(request)
return handler(request)
def patched(*args, **kwargs):
kwargs["transport"] = httpx.MockTransport(handler)
kwargs["transport"] = httpx.MockTransport(composite)
return RealAsyncClient(*args, **kwargs)
monkeypatch.setattr("app.docs_sessions.httpx.AsyncClient", patched)
+15 -6
View File
@@ -79,19 +79,28 @@ class _UpstreamHandler:
@pytest.fixture
def patched_httpx(monkeypatch):
def patched_httpx(monkeypatch, app_with_fake_gitea): # noqa: F811
"""Provide a hook the test can call to install a MockTransport.
Same shape as the docs_sessions fixture `app_with_fake_gitea`
monkeypatches `httpx.AsyncClient` for the gitea side, so we
construct from the unpatched class directly to avoid the
FakeGitea wrapper.
M3 note: lifespan now calls `refresh_registry` which hits FakeGitea
via the gitea transport. Since `httpx` is a module singleton, installing
the docs transport would clobber the FakeGitea mock already installed
by `app_with_fake_gitea`. We use a COMPOSITE handler: Gitea API
requests (to `http://gitea.test/`) are delegated to FakeGitea; all
other requests go to the test-specific handler.
"""
from httpx._client import AsyncClient as RealAsyncClient
_fake = app_with_fake_gitea[1]
def install(handler):
def composite(request: httpx.Request) -> httpx.Response:
if "gitea.test" in str(request.url):
return _fake.handle(request)
return handler(request)
def patched(*args, **kwargs):
kwargs["transport"] = httpx.MockTransport(handler)
kwargs["transport"] = httpx.MockTransport(composite)
return RealAsyncClient(*args, **kwargs)
monkeypatch.setattr("app.docs_specs.httpx.AsyncClient", patched)
+42
View File
@@ -0,0 +1,42 @@
"""§22.4c — the unreviewed/reviewed_at/reviewed_by entry frontmatter fields."""
from __future__ import annotations
from app import entry as entry_mod
def test_parse_defaults_unreviewed_false_when_absent():
text = "---\nslug: ohm\ntitle: OHM\nstate: active\n---\n\nBody.\n"
e = entry_mod.parse(text)
assert e.unreviewed is False
assert e.reviewed_at is None
assert e.reviewed_by is None
def test_parse_reads_review_fields():
text = (
"---\nslug: ohm\ntitle: OHM\nstate: active\n"
"unreviewed: true\nreviewed_at: '2026-06-03'\nreviewed_by: ben\n---\n\nBody.\n"
)
e = entry_mod.parse(text)
assert e.unreviewed is True
assert e.reviewed_at == "2026-06-03"
assert e.reviewed_by == "ben"
def test_serialize_emits_review_fields_only_when_meaningful():
e = entry_mod.Entry(slug="a", title="A", state="super-draft")
assert "unreviewed" not in entry_mod.serialize(e)
assert "reviewed_at" not in entry_mod.serialize(e)
e2 = entry_mod.Entry(slug="b", title="B", state="active", unreviewed=True)
assert "unreviewed: true" in entry_mod.serialize(e2)
def test_round_trip_preserves_review_fields():
e = entry_mod.Entry(
slug="b", title="B", state="active",
unreviewed=False, reviewed_at="2026-06-03", reviewed_by="ben",
)
back = entry_mod.parse(entry_mod.serialize(e))
assert back.reviewed_at == "2026-06-03"
assert back.reviewed_by == "ben"
assert back.unreviewed is False
@@ -0,0 +1,43 @@
"""§22.4b — a project with initial_state='active' lands new entries active +
unreviewed; the default 'super-draft' project is unchanged."""
from __future__ import annotations
from fastapi.testclient import TestClient
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as,
)
def _propose(client):
return client.post("/api/rfcs/propose", json={
"title": "Active Lander", "slug": "active-lander",
"pitch": "Lands active.", "tags": [],
})
def test_super_draft_default_unchanged(app_with_fake_gitea):
from app import entry as entry_mod
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
assert _propose(client).status_code == 200
f = fake.files[("wiggleverse", "meta", "propose/active-lander", "rfcs/active-lander.md")]
e = entry_mod.parse(f["content"])
assert e.state == "super-draft"
assert e.unreviewed is False
def test_active_initial_state_lands_active_unreviewed(app_with_fake_gitea):
from app import db, entry as entry_mod
app, fake = app_with_fake_gitea
with TestClient(app) as client:
db.conn().execute("UPDATE projects SET initial_state='active' WHERE id='default'")
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
assert _propose(client).status_code == 200
f = fake.files[("wiggleverse", "meta", "propose/active-lander", "rfcs/active-lander.md")]
e = entry_mod.parse(f["content"])
assert e.state == "active"
assert e.unreviewed is True
+65
View File
@@ -0,0 +1,65 @@
"""§22.4c — owner/admin mark-reviewed clears the flag; catalog unreviewed filter."""
from __future__ import annotations
from fastapi.testclient import TestClient
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea, tmp_env, provision_user_row, sign_in_as,
)
def _seed_unreviewed_active(fake, slug="feat"):
"""Put an active+unreviewed entry on the meta repo main + cache."""
from app import cache, entry as entry_mod
body = entry_mod.serialize(entry_mod.Entry(
slug=slug, title="Feat", state="active", unreviewed=True,
owners=["ben"], proposed_by="ben",
))
fake.files[("wiggleverse", "meta", "main", f"rfcs/{slug}.md")] = {"content": body, "sha": "s1"}
cache._upsert_cached_rfc(entry_mod.parse(body), body_sha="s1")
def test_catalog_unreviewed_filter(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_unreviewed_active(fake, "feat")
from app import cache, entry as entry_mod
ok = entry_mod.serialize(entry_mod.Entry(slug="ok", title="OK", state="active", owners=["ben"]))
fake.files[("wiggleverse", "meta", "main", "rfcs/ok.md")] = {"content": ok, "sha": "s2"}
cache._upsert_cached_rfc(entry_mod.parse(ok), body_sha="s2")
r = client.get("/api/rfcs", params={"unreviewed": "true"})
slugs = {i["slug"] for i in r.json()["items"]}
assert slugs == {"feat"}
def test_mark_reviewed_clears_flag(app_with_fake_gitea):
from app import db
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_unreviewed_active(fake, "feat")
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner")
r = client.post("/api/projects/default/rfcs/feat/mark-reviewed")
assert r.status_code == 200
row = db.conn().execute(
"SELECT unreviewed, reviewed_at, reviewed_by FROM cached_rfcs WHERE slug='feat'"
).fetchone()
assert row["unreviewed"] == 0
assert row["reviewed_by"] == "ben"
assert row["reviewed_at"] # provenance stamped
# git-side: the entry file on main was rewritten with the cleared flag.
from app import entry as entry_mod
written = fake.files[("wiggleverse", "meta", "main", "rfcs/feat.md")]["content"]
e = entry_mod.parse(written)
assert e.unreviewed is False
assert e.reviewed_by == "ben"
def test_mark_reviewed_forbidden_for_non_superuser(app_with_fake_gitea):
app, fake = app_with_fake_gitea
with TestClient(app) as client:
_seed_unreviewed_active(fake, "feat")
provision_user_row(user_id=2, login="carol", role="contributor")
sign_in_as(client, user_id=2, gitea_login="carol", display_name="Carol", role="contributor")
r = client.post("/api/projects/default/rfcs/feat/mark-reviewed")
assert r.status_code == 403
+54
View File
@@ -0,0 +1,54 @@
"""Migration 027 — additive §22 M3 schema (projects.type/initial_state, the
deployment singleton, cached_rfcs review columns). No table rebuilds in Plan A."""
from __future__ import annotations
import tempfile
from pathlib import Path
from app import db
from app.config import Config
def _fresh_config() -> Config:
tmp = Path(tempfile.mkdtemp(prefix="mig027-")) / "t.db"
return Config(
gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="x",
registry_repo="registry",
oauth_client_id="x", oauth_client_secret="x", app_url="x",
secret_key="x", database_path=tmp, owner_gitea_login="x",
webhook_secret="x",
)
def test_027_adds_project_type_and_initial_state():
cfg = _fresh_config()
db.run_migrations(cfg)
conn = db.connect(cfg.database_path)
cols = {r["name"]: r for r in conn.execute("PRAGMA table_info(projects)")}
assert "type" in cols and cols["type"]["dflt_value"] == "'document'"
assert "initial_state" in cols and cols["initial_state"]["dflt_value"] == "'super-draft'"
conn.close()
def test_027_creates_deployment_singleton():
cfg = _fresh_config()
db.run_migrations(cfg)
conn = db.connect(cfg.database_path)
rows = list(conn.execute("SELECT id FROM deployment"))
assert [r["id"] for r in rows] == [1]
try:
conn.execute("INSERT INTO deployment (id) VALUES (2)")
raised = False
except Exception:
raised = True
assert raised
conn.close()
def test_027_adds_review_columns_to_cached_rfcs():
cfg = _fresh_config()
db.run_migrations(cfg)
conn = db.connect(cfg.database_path)
cols = {r["name"] for r in conn.execute("PRAGMA table_info(cached_rfcs)")}
assert {"unreviewed", "reviewed_at", "reviewed_by"} <= cols
conn.close()
@@ -1,11 +1,11 @@
"""Slice M1 — the §22 multi-project spine.
"""Slice M1+M3 — the §22 multi-project spine.
Migration 026 introduces the `projects` and `project_members` tables, seeds
the single `default` project (the N=1 case, §22.13), and threads a
`project_id` column onto every slug-bearing table, backfilled to `default`.
The §22.13 startup backfill then fills the default project's content_repo
from META_REPO. These tests prove the spine lands without disturbing the
single-project app.
M3 retires the META_REPO startup backfill; content_repo now comes from the
registry mirror (projects.yaml in REGISTRY_REPO). These tests prove the
spine lands without disturbing the single-project app.
"""
from __future__ import annotations
@@ -40,7 +40,8 @@ def test_default_project_seeded_and_backfilled(app_with_fake_gitea):
assert row["id"] == "default"
# public preserves the pre-multi-project open-by-default posture.
assert row["visibility"] == "public"
# §22.13 startup backfill set content_repo from META_REPO (tmp_env).
# M3: content_repo now comes from the registry mirror (projects.yaml),
# not the retired META_REPO startup backfill.
assert row["content_repo"] == "meta"
@@ -105,22 +106,25 @@ def test_project_members_table_shape(app_with_fake_gitea):
pass
def test_seed_default_project_is_idempotent(app_with_fake_gitea):
"""Re-running the backfill never clobbers a content_repo already set —
so a later registry mirror (M3) wins over the META_REPO fallback."""
from app import db, projects
from app.config import load_config
def test_registry_mirror_is_idempotent(app_with_fake_gitea):
"""Re-running the registry mirror is safe — it upserts (overwrites) the
projects row from projects.yaml each time without raising. M3 retirement
of seed_default_project: the registry mirror is now the sole authority."""
import asyncio
from app import db, registry as registry_mod
app, _ = app_with_fake_gitea
with TestClient(app):
db.conn().execute(
"UPDATE projects SET content_repo = 'ohm-content' WHERE id = 'default'"
)
projects.seed_default_project(load_config())
got = db.conn().execute(
before = db.conn().execute("SELECT COUNT(*) AS n FROM projects").fetchone()["n"]
cfg = app.state.config
gitea = app.state.gitea
asyncio.run(registry_mod.refresh_registry(cfg, gitea))
after = db.conn().execute("SELECT COUNT(*) AS n FROM projects").fetchone()["n"]
assert after == before
row = db.conn().execute(
"SELECT content_repo FROM projects WHERE id='default'"
).fetchone()["content_repo"]
assert got == "ohm-content"
).fetchone()
assert row["content_repo"] == "meta"
def test_registry_repo_config_wired(app_with_fake_gitea, monkeypatch):
@@ -128,6 +132,8 @@ def test_registry_repo_config_wired(app_with_fake_gitea, monkeypatch):
monkeypatch.setenv("REGISTRY_REPO", "wiggleverse-registry")
assert load_config().registry_repo == "wiggleverse-registry"
# Optional through M1: absent resolves to empty, not a startup failure.
# M3: REGISTRY_REPO is now required — absent raises RuntimeError.
monkeypatch.delenv("REGISTRY_REPO", raising=False)
assert load_config().registry_repo == ""
import pytest
with pytest.raises(RuntimeError, match="REGISTRY_REPO"):
load_config()
+19 -1
View File
@@ -55,6 +55,24 @@ class FakeGitea:
self._pr_counter = 0
self._commit_counter = 0
self._seed_repo("wiggleverse", "meta")
# §22 M3: the deployment's project registry. Startup refresh_registry
# reads projects.yaml here; the single 'default' project's content_repo
# points back at the seeded meta repo so the corpus mirror is unchanged.
self._seed_repo("wiggleverse", "registry")
self.files[("wiggleverse", "registry", "main", "projects.yaml")] = {
"content": (
"deployment:\n"
" name: Test Deployment\n"
" tagline: A test deployment\n"
"projects:\n"
" - id: default\n"
" name: Test Deployment\n"
" type: document\n"
" content_repo: meta\n"
" visibility: public\n"
),
"sha": "regsha0001",
}
def _seed_repo(self, owner, repo):
self.branches[(owner, repo)] = {"main": {"sha": "initial", "ts": "2026-05-23T00:00:00Z"}}
@@ -431,7 +449,7 @@ def tmp_env(monkeypatch):
"GITEA_BOT_USER": "rfc-bot",
"GITEA_BOT_TOKEN": "bot-token",
"GITEA_ORG": "wiggleverse",
"META_REPO": "meta",
"REGISTRY_REPO": "registry",
"OAUTH_CLIENT_ID": "cid",
"OAUTH_CLIENT_SECRET": "csec",
"APP_URL": "http://localhost:8000",
+98
View File
@@ -0,0 +1,98 @@
"""§22.2 registry parse + apply: validation, type-immutability, upsert."""
from __future__ import annotations
import tempfile
from pathlib import Path
import pytest
from app import db, registry
from app.config import Config
def _db():
cfg = Config(
gitea_url="x", gitea_bot_user="x", gitea_bot_token="x", gitea_org="x",
registry_repo="registry", oauth_client_id="x",
oauth_client_secret="x", app_url="x", secret_key="x",
database_path=Path(tempfile.mkdtemp(prefix="reg-")) / "t.db",
owner_gitea_login="x", webhook_secret="x",
)
db.run_migrations(cfg)
if db._CONN is not None:
db._CONN.close()
db._CONN = None
db.init(cfg)
return cfg
VALID = """
deployment:
name: Open Human Model
tagline: A model of human flourishing
projects:
- id: default
name: Open Human Model
type: document
content_repo: meta
visibility: public
"""
def test_parse_valid_registry():
doc = registry.parse_registry(VALID)
assert doc.deployment_name == "Open Human Model"
assert doc.deployment_tagline == "A model of human flourishing"
assert len(doc.projects) == 1
p = doc.projects[0]
assert (p.id, p.type, p.content_repo, p.visibility) == ("default", "document", "meta", "public")
assert p.initial_state == "super-draft"
def test_parse_initial_state_defaults_per_type():
doc = registry.parse_registry(
"projects:\n - {id: a, name: A, type: bdd, content_repo: a}\n"
)
assert doc.projects[0].initial_state == "active" # bdd default
@pytest.mark.parametrize("bad,msg", [
("projects: []\n", "at least one"),
("projects:\n - just-a-string\n", "must be a mapping"),
("projects:\n - {id: 'Bad Slug', name: A, type: document, content_repo: a}\n", "valid slug"),
("projects:\n - {id: a, name: A, type: nope, content_repo: a}\n", "invalid type"),
("projects:\n - {id: a, name: A, type: document}\n", "content_repo"),
("projects:\n - {id: a, name: A, type: document, content_repo: a, visibility: x}\n", "visibility"),
("projects:\n - {id: a, name: A, type: document, content_repo: a}\n - {id: a, name: B, type: document, content_repo: b}\n", "duplicate"),
])
def test_parse_rejects_invalid(bad, msg):
with pytest.raises(registry.RegistryError) as e:
registry.parse_registry(bad)
assert msg in str(e.value)
def test_apply_upserts_projects_and_deployment():
_db()
doc = registry.parse_registry(VALID)
registry.apply_registry(doc, registry_sha="regsha1")
prow = db.conn().execute(
"SELECT name, type, content_repo, visibility, initial_state, registry_sha FROM projects WHERE id='default'"
).fetchone()
assert prow["name"] == "Open Human Model"
assert prow["content_repo"] == "meta"
assert prow["registry_sha"] == "regsha1"
drow = db.conn().execute("SELECT name, tagline FROM deployment WHERE id=1").fetchone()
assert drow["name"] == "Open Human Model"
assert drow["tagline"] == "A model of human flourishing"
def test_apply_rejects_type_change_on_existing_project():
_db()
registry.apply_registry(registry.parse_registry(VALID), "s1")
changed = VALID.replace("type: document", "type: specification")
registry.apply_registry(registry.parse_registry(changed), "s2") # logged + skipped, no raise
t = db.conn().execute("SELECT type FROM projects WHERE id='default'").fetchone()["type"]
assert t == "document" # immutable — unchanged
# The deployment row IS still advanced even though the project upsert was skipped.
drow = db.conn().execute("SELECT registry_sha FROM deployment WHERE id=1").fetchone()
assert drow["registry_sha"] == "s2"
+50
View File
@@ -0,0 +1,50 @@
"""Startup mirrors the registry; the registry webhook re-mirrors it."""
from __future__ import annotations
from fastapi.testclient import TestClient
from test_propose_vertical import app_with_fake_gitea, tmp_env # noqa: F401
def test_startup_mirrors_registry_into_projects_and_deployment(app_with_fake_gitea):
from app import db
app, _ = app_with_fake_gitea
with TestClient(app):
prow = db.conn().execute(
"SELECT content_repo, type, initial_state FROM projects WHERE id='default'"
).fetchone()
assert prow["content_repo"] == "meta" # from the registry, not META_REPO
assert prow["type"] == "document"
drow = db.conn().execute("SELECT name FROM deployment WHERE id=1").fetchone()
assert drow["name"] # deployment name mirrored from the registry
def test_registry_webhook_remirrors(app_with_fake_gitea):
import hashlib
import hmac
import json as _json
app, fake = app_with_fake_gitea
with TestClient(app) as client:
from app import db
new_yaml = (
"deployment:\n name: OHM\n tagline: Edited tagline\n"
"projects:\n - id: default\n name: OHM\n type: document\n"
" content_repo: meta\n visibility: public\n"
)
fake.files[("wiggleverse", "registry", "main", "projects.yaml")] = {
"content": new_yaml, "sha": "regsha2",
}
body = _json.dumps({"repository": {"full_name": "wiggleverse/registry"}}).encode()
secret = "test-webhook-secret-for-signature-verification"
sig = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
r = client.post(
"/api/webhooks/gitea",
content=body,
headers={"X-Gitea-Event": "push", "X-Gitea-Signature": sig,
"Content-Type": "application/json"},
)
assert r.status_code == 200
tagline = db.conn().execute("SELECT tagline FROM deployment WHERE id=1").fetchone()["tagline"]
assert tagline == "Edited tagline"
+2
View File
@@ -73,6 +73,7 @@ def test_config_loads_with_empty_secret_when_bypass_is_set(monkeypatch, tmp_path
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
monkeypatch.setenv("REGISTRY_REPO", "registry")
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
@@ -92,6 +93,7 @@ def test_config_loads_with_secret_set(monkeypatch, tmp_path):
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
monkeypatch.setenv("REGISTRY_REPO", "registry")
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,413 @@
# M3-backend — §22 multi-project: registry mirror + data spine + APIs
> Design spec for the backend half of §22 slice **M3** ("Registry mirror +
> routing + runtime branding"). The roadmap bundles M3 as one slice; this
> session splits it at the natural backend/frontend seam. **M3-backend** (this
> doc) ships the data spine, the registry mirror, the two runtime-config APIs,
> and the entry-state/review semantics. **M3-frontend** (a separate spec) ships
> `/p/<project>/` routing, the 308 redirects, the `VITE_APP_NAME`→runtime-config
> cut, the per-project theme overlay, the deployment directory at `/`, and the
> project switcher — all consuming the APIs defined here.
>
> Section references `§22.x` point at `docs/design/multi-project-spec.md` (the
> draft §22 + slicing plan). The SPEC.md §22 merge itself lands in M7.
## Status
- **Date:** 2026-06-03
- **Slice:** §22 M3 (backend half). M1 + M2 landed and merged to `main`.
- **Version impact:** minor bump, breaking (pre-1.0) — see §8.
## Goal
After M3-backend, the framework learns its projects from a git **registry**
(not from `META_REPO`), the `projects` cache table and all slug-bearing tables
are keyed by `(project_id, …)` so a second project can exist without collision,
the default project's identity is re-stamped to its real slug while no `/p/`
URL is yet public, and the runtime exposes deployment + project config over two
new endpoints. The entry-state/review semantics (`initial_state`, `unreviewed`)
ship complete even though OHM (a `document`/`super-draft` project) does not yet
exercise them.
Non-goals (M3-frontend, later): `/p/<project>/` routing, 308 redirects off
`/rfc/<slug>`, runtime branding in the UI, theme application, the deployment
directory, the project switcher, the catalog's unreviewed-filter **UI**.
## Decisions taken in brainstorming
1. **Scope split** — backend spine first; frontend surfaces are a separate
spec/session.
2. **Review machinery** — build the full `initial_state` / `unreviewed`
plumbing now (parse + columns + mirror + landing logic + mark-reviewed +
catalog filter query side), per the spec's M3 bundle, even though no live
project exercises it yet.
3. **Config cut** — hard cut. `REGISTRY_REPO` required (loud fail if unset),
`META_REPO` removed. Upgrade-steps document the manual registry creation.
4. **Re-stamp** — rewrite `project_id` everywhere: rename `projects.id`
`default` → the config slug and rewrite every child row, folded into the
same create-copy-drop-rename rebuild that adds the `project_id` FK. One
identifier; DB and URL agree.
5. **Mirror structure** — a self-contained `app/registry.py` module (not folded
into `cache.py`), driven by the existing webhook dispatcher + the existing
`Reconciler.sweep()`.
6. **Two execution plans (found during planning).** The 12-table PK rebuild is
not self-contained: folding `project_id` into keys + FK forces every
`ON CONFLICT` upsert target to gain `project_id` (~10 sites across 6 modules)
and — because the rebuilt tables can no longer default `project_id` to a live
value once the default is re-stamped — forces **every RFC/branch writer** to
be threaded to supply the real `project_id`. That activation is larger and
riskier than the rest of M3-backend combined, and it is only required *before
a second project can collide* (i.e. right before M4). So M3-backend is split
into two plans at that seam:
- **Plan A (ships first):** registry mirror + the two APIs + `initial_state`/
`unreviewed` semantics. Migration `027` is **additive only** (no rebuilds).
Operates entirely on the `default`-id project — no re-stamp, no rebuild, no
writer threading. `cached_rfcs` keeps its `slug` PK, so no upsert breakage.
- **Plan B (before M4 / before public `/p/` URLs):** the 12-table PK rebuild
(migration `028`), `project_id` threading through every writer, the
`default`→slug **re-stamp** (which rides here because it is only correct
once the rebuild's column-default fix + threading land), and two-project
isolation tests.
The §1 sections below describe the **full** backend (both plans); §1c (the
rebuilds) and §1d (the re-stamp) are **Plan B**. Everything else is Plan A.
---
## 1. Migration `027_projects_activate.sql`
Runs while `default` is still the sole project and no `/p/` URL exists — the
safe window for an identity rewrite. One transaction (SQLite DDL is
transactional): the deployment either fully advances to `027` or stays on `026`.
`DEFAULT_PROJECT_ID` is read at migration time, so it **must be set before the
upgrade deploy** (documented in §8). The slug it names is referred to below as
`<slug>`; absent the env var, `<slug>` stays `default`.
### 1a. Additive columns
- `projects`:
- `type TEXT NOT NULL DEFAULT 'document' CHECK (type IN ('document','specification','bdd'))`
- `initial_state TEXT NOT NULL DEFAULT 'super-draft' CHECK (initial_state IN ('super-draft','active'))`
- `cached_rfcs`:
- `unreviewed INTEGER NOT NULL DEFAULT 0`
- `reviewed_at TEXT`
- `reviewed_by TEXT`
### 1b. New `deployment` singleton table
Holds deployment-level identity mirrored from the registry's `deployment:`
block (rather than overloading `projects`):
```sql
CREATE TABLE IF NOT EXISTS deployment (
id INTEGER PRIMARY KEY CHECK (id = 1),
name TEXT,
tagline TEXT,
registry_sha TEXT,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
INSERT OR IGNORE INTO deployment (id) VALUES (1);
```
> **Re-stamp split (correctness, found during planning).** The migration
> runner executes pure-SQL files and **cannot read `DEFAULT_PROJECT_ID` from the
> environment** — the same constraint that forced M1's `seed_default_project`
> into Python. So the re-stamp is **not** in the `.sql` file. Migration `027`
> (pure SQL) does §1a–§1c with `project_id` copied **verbatim** (`default` stays
> `default`); the rebuilt FKs are declared `ON UPDATE CASCADE ON DELETE
> CASCADE`. The re-stamp (§1d) is a **Python startup step** in `app/projects.py`,
> run after migrations and before the registry mirror, that issues a single
> `UPDATE projects SET id = <slug>` (cascading to the 12 FK tables) plus a plain
> `UPDATE` of the 7 non-FK `project_id` tables. Idempotent: a no-op once no
> `default` row remains.
### 1c. PK / uniqueness rebuilds (the 12 deferred tables)
Per migration 026's deferred block, create-copy-drop-rename each table to fold
`project_id` into the key and add `project_id … REFERENCES projects(id) ON
UPDATE CASCADE ON DELETE CASCADE` (the `ON UPDATE CASCADE` is what lets the
§1d Python re-stamp move all child rows with one parent UPDATE):
| Table | Key change |
| --- | --- |
| `cached_rfcs` | PK `(slug)``(project_id, slug)` |
| `cached_branches` | UNIQUE `(rfc_slug, branch_name)``+project_id` |
| `branch_visibility` | UNIQUE `(rfc_slug, branch_name)``+project_id` |
| `branch_contribute_grants` | UNIQUE `(rfc_slug, branch_name, grantee_user_id)``+project_id` |
| `stars` | UNIQUE `(user_id, rfc_slug)``+project_id` |
| `watches` | UNIQUE `(user_id, rfc_slug)``+project_id` |
| `pr_seen` | UNIQUE `(user_id, rfc_slug, pr_number)``+project_id` |
| `branch_chat_seen` | UNIQUE `(user_id, rfc_slug, branch_name)``+project_id` |
| `funder_consents` | PK `(user_id, rfc_slug)``+project_id` |
| `rfc_collaborators` | UNIQUE INDEX `(rfc_slug, user_id)``+project_id` |
| `contribution_requests` | UNIQUE INDEX `(rfc_slug, requester_user_id) WHERE pending``+project_id` |
| `proposed_use_cases` | UNIQUE `(scope, pr_number)``+project_id` |
`cached_prs` is already globally unique (`repo` is the full `org/repo` string,
distinct per project) — **no rebuild**.
The copy step writes `<slug>` in place of `default` for `project_id`, so these
12 tables are re-stamped for free (§1d).
### 1d. Re-stamp `default``<slug>` (Python startup step)
In `app/projects.py`, run at startup after `db.init` and before
`refresh_registry`. When `DEFAULT_PROJECT_ID` is set and a `default` project row
still exists, in one `db.tx()`:
- `UPDATE projects SET id = <slug>, updated_at = datetime('now') WHERE id =
'default'` — cascades `project_id` across the 12 FK tables via `ON UPDATE
CASCADE`.
- For the 7 `project_id`-bearing tables with **no** FK (`threads`, `changes`,
`notifications`, `actions`, `pr_resolution_branches`, `rfc_invitations`,
`cached_prs`): `UPDATE <t> SET project_id = <slug> WHERE project_id =
'default'`.
End state: a single `project_id` value DB-wide. Idempotent — a no-op once no
`default` row remains, so it is safe on every boot.
### 1e. Notes
- Migration `016` is absent from the on-disk sequence (`015 → 017`); the runner
already tolerates the gap (the app runs today). **Do not renumber.** New file
is `027`.
- `PRAGMA foreign_keys` is honored going forward; the FK lands on the 12 rebuilt
tables. The other 7 keep app-layer integrity (matching today's posture for
non-rebuilt tables).
---
## 2. Registry format + `app/registry.py`
### 2a. `projects.yaml` (root of `REGISTRY_REPO`)
```yaml
deployment:
name: Open Human Model # replaces VITE_APP_NAME (M3-frontend consumes)
tagline: ...
projects:
- id: ohm # url-stable slug, unique in the deployment
name: Open Human Model
type: document # document | specification | bdd — immutable
content_repo: ohm # repo under the deployment's Gitea org
visibility: public # gated | public | unlisted
initial_state: super-draft # optional; defaults from type
enabled_models: [claude, gemini] # optional; falls back to ENABLED_MODELS
theme: { accent: "#5b5bd6" } # optional; M3-frontend consumes
```
### 2b. `refresh_registry(config, gitea) -> RegistryResult`
The config-side analogue of `cache.refresh_meta_repo`. Fetches `projects.yaml`
from `REGISTRY_REPO` at HEAD, parses, **validates**, and upserts:
- Each project → `projects` row: `id, name, type, content_repo, visibility,
initial_state`, plus `config_json` (JSON blob for `theme`, `enabled_models`),
plus `registry_sha` (commit SHA, provenance).
- The `deployment:` block → the `deployment` singleton (`name`, `tagline`,
`registry_sha`).
Projects present in the table but absent from the registry are **not** deleted
in M3-backend (archival semantics are out of scope; a removed entry simply
stops being refreshed). This is noted as a known limitation; revisit if/when
project archival is specced.
### 2c. Validation (loud)
Per the framework's separation-of-concerns rule, malformed config fails
visibly rather than shipping wrong content silently:
- Each project requires `id`, `name`, `type`, `content_repo`.
- `type` ∈ {`document`,`specification`,`bdd`}; `visibility`
{`gated`,`public`,`unlisted`}.
- `initial_state` defaults from `type` when omitted: `document`/`specification`
`super-draft`, `bdd``active`. When present it must be a valid §2.4
super-draft entry-state value.
- `id` values unique and slug-shaped (`^[a-z0-9][a-z0-9-]*$`).
- **`type` is immutable:** an incoming `type` differing from the stored row's is
rejected (the entry is skipped, the rest proceed; logged loudly).
- **Default-id consistency:** the re-stamped default id (`DEFAULT_PROJECT_ID`,
else `default`) MUST appear as an `id` in the registry, or the registry is
inconsistent with config → surfaced loudly.
### 2d. Wiring (Option A)
- **Webhook** (`app/webhooks.py`): add a branch — if the pushed repo
`full_name` matches `REGISTRY_REPO`, call `registry.refresh_registry(...)`.
Same HMAC-verified `/api/webhooks/gitea` dispatcher; no new endpoint. Add
`REGISTRY_REPO` to the set of repos the dispatcher recognizes.
- **Sweep** (`cache.Reconciler.sweep()`): add one `await
registry.refresh_registry(...)` at the top of each pass, so the safety-net
loop keeps `projects` in sync if a webhook is missed.
- **Startup** (`main.py` lifespan): run `refresh_registry` once after
migrations. This **replaces** M1's `seed_default_project` (which is removed).
### 2e. Failure posture
- **Startup / first boot:** if `REGISTRY_REPO` is unset/unreachable, or
`projects.yaml` is missing or fails validation, the app **fails loudly**
(refuses to start) — there is no last-known-good to serve.
- **Running deployment:** a malformed `projects.yaml` pushed in a later PR is
logged and **skipped**, leaving the last-good `projects` rows intact — a bad
config PR must not take the deployment down. This mirrors how the corpus
reconciler tolerates a bad content push today.
---
## 3. Config (`app/config.py`)
- `registry_repo`: **required** — construction fails loudly if unset (matching
the other required vars). Add `registry_repo_full``{gitea_org}/{registry_repo}`.
- `meta_repo` and `meta_repo_full`: **removed**.
- `default_project_id`: optional; consumed by migration `027` (re-stamp) and by
`refresh_registry` (the §2c consistency gate).
- `enabled_models`: unchanged — the deployment-level fallback for a project's
optional `enabled_models`.
- `app/projects.py`: `seed_default_project` retired (superseded by the mirror).
`DEFAULT_PROJECT_ID` constant and resolution helpers retained as needed.
---
## 4. APIs — `app/api_deployment.py`
A new sub-router mounted in `app/api.py`.
### `GET /api/deployment`
Returns `{ name, tagline, projects: [...] }`. The deployment `name`/`tagline`
come from the `deployment` singleton. The project list is filtered by caller
visibility (§22.5):
- `public` projects → visible to everyone (incl. anonymous).
- `gated` projects → only when the caller is a member (`visible_project_ids`
from M2's resolver).
- `unlisted` projects → **omitted entirely** (reachable only by direct id).
Each item: `{ id, name, type, visibility }` — enough for the M3-frontend
directory + switcher. (Theme is fetched per project.)
### `GET /api/projects/:id`
Returns `{ id, name, tagline, type, visibility, initial_state, theme }`.
Guarded by `require_project_readable(user, id)` — 404 for a non-member of a
gated project, reusing the M2 resolver. `unlisted` is readable here by direct
id (it is hidden only from enumeration).
---
## 5. Entry-state & review semantics
### 5a. Frontmatter (`app/entry.py`)
Parse three new fields, lenient (default `unreviewed=false`, nulls):
`unreviewed: bool`, `reviewed_at`, `reviewed_by`. Add to the `Entry`
dataclass. These are git-truth (§2 frontmatter) so they survive a cache
rebuild, exactly like `state`.
### 5b. Cache mirror (`app/cache.py`)
`_upsert_cached_rfc` writes the three fields into `cached_rfcs`
(`unreviewed`, `reviewed_at`, `reviewed_by`).
### 5c. Entry-landing path
When a creating idea-PR merges (§2.4), resolve the project's `initial_state`:
- `super-draft` → today's behavior unchanged (propose → super-draft → graduate).
- `active` → land the entry `active` with `unreviewed = true`, skipping the §13
graduate gate. The exact merge-handling site (in the PR-merge reconcile path)
is located during implementation.
### 5d. Mark-reviewed
`POST /api/projects/:pid/rfcs/:slug/mark-reviewed`. Authority:
`is_project_superuser` (project_admin or deployment owner/admin) — the same tier
that graduates an entry. Effect: the bot writes `unreviewed: false` +
`reviewed_at`/`reviewed_by` into the entry frontmatter (a git commit, paralleling
graduate), which the mirror then reflects into `cached_rfcs`. Stamps provenance
paralleling `graduated_at`/`graduated_by`.
### 5e. Catalog filter (query side)
`GET /api/rfcs` (and the project-scoped form) gains an `unreviewed=true` query
param that filters on the cached column — the owner's worklist. The UI for it
is M3-frontend; only the query side ships here. `unreviewed` applies to
`active` entries only.
---
## 6. Testing
- **Migration `027`:** seed a `026`-shaped DB with rows under
`project_id='default'`; run `027`; assert: new columns present; the 12 tables
rebuilt with composite keys + FK; all 19 `project_id`-bearing tables
re-stamped to `<slug>`; row counts preserved; FK integrity on; idempotent
re-run is a no-op.
- **`registry.py`:** valid `projects.yaml` upserts all fields + `registry_sha`;
each validation failure rejected (bad enum, missing field, dup id, `type`
mutation, missing default id); `initial_state` type-default applied;
startup-strict vs running-tolerant posture.
- **APIs:** `/api/deployment` visibility filtering across
gated/public/unlisted × member/non-member/anonymous; `/api/projects/:id` 404
gate for gated non-member, 200 for unlisted-by-id.
- **Review flow:** `initial_state: active` lands `unreviewed=true`;
mark-reviewed authority (allow superuser, deny others) + frontmatter write +
mirror reflection; catalog `unreviewed` filter returns the worklist.
- **Two-project isolation:** extend `test_multi_project_authz_vertical.py` with
a genuine **second** registry project sharing a slug with the first, proving
the PK rebuilds isolate them (the core point of M3's activation).
---
## 7. File-touch summary
**New**
- `backend/migrations/027_projects_activate.sql`
- `backend/app/registry.py`
- `backend/app/api_deployment.py`
- tests: `backend/tests/test_migration_027.py`, `test_registry.py`,
`test_api_deployment.py`, `test_review_flow.py`; extend
`test_multi_project_authz_vertical.py`
**Modified**
- `backend/app/config.py` (registry_repo required; meta_repo removed;
default_project_id)
- `backend/app/webhooks.py` (registry-repo branch)
- `backend/app/cache.py` (`Reconciler.sweep` registry call;
`_upsert_cached_rfc` review fields)
- `backend/app/entry.py` (frontmatter fields)
- `backend/app/api.py` (mount `api_deployment`; `unreviewed` filter on
`/api/rfcs`)
- `backend/app/main.py` (startup `refresh_registry`; drop `seed_default_project`)
- `backend/app/projects.py` (retire `seed_default_project`)
- `backend/.env.example` (`REGISTRY_REPO`, `DEFAULT_PROJECT_ID`; remove
`META_REPO`)
---
## 8. Versioning & upgrade
Minor bump, breaking (pre-1.0). `VERSION` + `frontend/package.json#version` move
together (§20). `CHANGELOG.md` gets a breaking entry with an **upgrade-steps**
block:
1. Create a registry repo under the deployment's Gitea org.
2. Author `projects.yaml`: a `deployment:` block (`name`, `tagline`) and one
`projects:` entry for the existing corpus — `id: <slug>`, `name`, `type:
document`, `content_repo: <old META_REPO value>`, `visibility: public`.
3. Set env: `REGISTRY_REPO=<registry repo name>`, `DEFAULT_PROJECT_ID=<slug>`
(must equal the entry's `id`); remove `META_REPO`.
4. Deploy. Migration `027` runs the rebuilds + re-stamp; `refresh_registry`
reconciles the registry into `projects`. Verify `/api/deployment` returns the
project and `/api/health` is green.
The SPEC.md §22 merge stays in M7 per the slicing plan; this slice references the
draft at `docs/design/multi-project-spec.md`.
## Known limitations / deferred
- **Project archival/deletion** from the registry is not handled (a removed
entry stops refreshing but its rows persist). Defer to a future archival spec.
- All routing, redirects, runtime branding, theme application, directory, and
switcher are **M3-frontend**.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "rfc-app-frontend",
"private": true,
"version": "0.32.0",
"version": "0.33.0",
"type": "module",
"scripts": {
"dev": "vite",