feat(projects): M1 — the multi-project schema spine (§22)
Lays the additive foundation for hosting N projects per deployment, with today's single corpus as the N=1 case. No behavior change: the app runs exactly as before, single project, with the spine underneath. - migration 025: `projects` + `project_members` tables; seed the `default` project (visibility=public, preserving open-by-default); thread `project_id NOT NULL DEFAULT 'default'` onto all 19 slug-bearing tables, backfilling existing rows. Additive — no table rebuilds; the slug-keyed uniqueness/PK rework is enumerated in the migration header and deferred to the slice that activates project #2. - app/projects.py: §22.13 startup backfill of the default project's content_repo from META_REPO (idempotent — never clobbers a value the future registry mirror sets). - config: REGISTRY_REPO wired (optional through M1; consumed by the M3 mirror), documented in .env.example as META_REPO's successor. - tests: 6 vertical assertions on the spine (seed, backfill, column shape, role CHECK, idempotency, config). Full suite 381 passed. - docs: align Part C M1/M3 boundaries with the landed code (registry mirror + redirect move to M3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ben Stull
@@ -14,6 +14,15 @@ GITEA_BOT_TOKEN=
|
||||
GITEA_ORG=wiggleverse
|
||||
META_REPO=meta
|
||||
|
||||
# --- Multi-project registry (§22) ---
|
||||
# The registry repo (under GITEA_ORG) the framework reads to learn which
|
||||
# projects this deployment hosts. Optional through Slice M1: the registry
|
||||
# mirror lands in M3, and until then the single default project's content
|
||||
# repo is taken from META_REPO above. The repo's name is the deployment's
|
||||
# choice. When the mirror lands, REGISTRY_REPO becomes required and
|
||||
# supersedes META_REPO.
|
||||
# REGISTRY_REPO=wiggleverse-registry
|
||||
|
||||
# --- OAuth (Gitea) ---
|
||||
# In Gitea: Site Administration → Applications → Add OAuth2 Application.
|
||||
# Redirect URI: {APP_URL}/auth/callback
|
||||
|
||||
@@ -33,6 +33,7 @@ class Config:
|
||||
gitea_bot_token: str
|
||||
gitea_org: str
|
||||
meta_repo: str
|
||||
registry_repo: str
|
||||
oauth_client_id: str
|
||||
oauth_client_secret: str
|
||||
app_url: str
|
||||
@@ -80,6 +81,13 @@ def load_config() -> Config:
|
||||
gitea_bot_token=_required("GITEA_BOT_TOKEN"),
|
||||
gitea_org=_required("GITEA_ORG"),
|
||||
meta_repo=_optional("META_REPO", "meta"),
|
||||
# §22.2: the multi-project registry repo the framework reads to learn
|
||||
# which projects exist. Optional through Slice M1 — the registry
|
||||
# *mirror* lands in M3; until then the default project (seeded by
|
||||
# migration 025 + the §22.13 startup backfill) is the only project,
|
||||
# and its content_repo comes from META_REPO. Becomes required when
|
||||
# the mirror lands and META_REPO is retired.
|
||||
registry_repo=_optional("REGISTRY_REPO"),
|
||||
oauth_client_id=_required("OAUTH_CLIENT_ID"),
|
||||
oauth_client_secret=_required("OAUTH_CLIENT_SECRET"),
|
||||
app_url=_optional("APP_URL", "http://localhost:8000").rstrip("/"),
|
||||
|
||||
@@ -28,6 +28,7 @@ from . import (
|
||||
invites as invites_mod,
|
||||
otc,
|
||||
passcode as passcode_mod,
|
||||
projects,
|
||||
providers as providers_mod,
|
||||
ratelimit,
|
||||
turnstile,
|
||||
@@ -99,6 +100,7 @@ async def lifespan(app: FastAPI):
|
||||
config = load_config()
|
||||
db.run_migrations(config)
|
||||
db.init(config)
|
||||
projects.seed_default_project(config) # §22.13 step 1
|
||||
gitea = Gitea(config)
|
||||
bot = Bot(gitea)
|
||||
reconciler = cache.Reconciler(config, gitea)
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Project registry — the §22 multi-project layer.
|
||||
|
||||
A deployment hosts one or more projects (§22.1). Through Slice M1 the only
|
||||
project is the migration-seeded `default` one (the N=1 case, §22.13); the git
|
||||
registry mirror that lets a deployment declare more projects lands in M3 and
|
||||
will grow this module. For now this holds just the §22.13 step-1 backfill:
|
||||
completing the default project's row from deployment config, which pure-SQL
|
||||
migration 025 could not read.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from . import db
|
||||
from .config import Config
|
||||
|
||||
DEFAULT_PROJECT_ID = "default"
|
||||
|
||||
|
||||
def seed_default_project(config: Config) -> None:
|
||||
"""Fill the default project's content_repo from config.
|
||||
|
||||
Migration 025 seeds `default` with content_repo NULL because SQL cannot
|
||||
read the environment. This sets it from META_REPO (the pre-multi-project
|
||||
single-corpus repo) the first time the configured app boots, so the row
|
||||
accurately names the corpus the existing rows belong to. Idempotent: only
|
||||
touches the row while content_repo is still unset, so a later registry
|
||||
mirror (M3) that has populated it is never clobbered.
|
||||
|
||||
The display `name` is deliberately left as-is: the deployment's
|
||||
user-visible name lives in the frontend build (VITE_APP_NAME) today and
|
||||
moves to the registry + GET /api/deployment in M3 (§22.9); the backend has
|
||||
no authoritative name to backfill from yet.
|
||||
"""
|
||||
db.conn().execute(
|
||||
"""
|
||||
UPDATE projects
|
||||
SET content_repo = ?, updated_at = datetime('now')
|
||||
WHERE id = ? AND content_repo IS NULL
|
||||
""",
|
||||
(config.meta_repo, DEFAULT_PROJECT_ID),
|
||||
)
|
||||
@@ -0,0 +1,105 @@
|
||||
-- §22 (multi-project deployments) — Slice M1: the project spine.
|
||||
--
|
||||
-- A deployment now hosts one or more *projects*, each a corpus with its own
|
||||
-- content repo, slug/RFC-NNNN namespace, catalog, roster, and branding. The
|
||||
-- pre-multi-project single-corpus deployment is the N=1 case: this migration
|
||||
-- generates one 'default' project and stamps every existing RFC-scoped row
|
||||
-- to it, so the app keeps running exactly as before with the spine
|
||||
-- underneath (see docs/design/multi-project-spec.md §22.13).
|
||||
--
|
||||
-- STRATEGY — additive, no table rebuilds. Every slug-bearing table gets a
|
||||
-- `project_id TEXT NOT NULL DEFAULT 'default'` column. The constant default
|
||||
-- means every existing INSERT in the codebase that does not yet mention
|
||||
-- project_id keeps working and lands rows in the default project; no query
|
||||
-- breaks because, with a single project, slugs remain globally unique. The
|
||||
-- column carries no inline REFERENCES clause: SQLite's ALTER TABLE ADD
|
||||
-- COLUMN forbids a FK column with a non-NULL default. project_id referential
|
||||
-- integrity is therefore enforced at the app layer for now; the FK lands
|
||||
-- with the table rebuilds below.
|
||||
--
|
||||
-- ============================================================================
|
||||
-- DEFERRED to the slice that activates project #2 (M3/M4). Until a second
|
||||
-- project exists these are correct as-is; the moment two projects can share a
|
||||
-- slug or a Gitea PR number they become cross-project collision bugs and MUST
|
||||
-- be rebuilt (SQLite needs a create-copy-drop-rename per table) to fold
|
||||
-- project_id into the key, and to add the project_id FK:
|
||||
-- * cached_rfcs PRIMARY KEY (slug) -> (project_id, slug)
|
||||
-- * cached_branches UNIQUE (rfc_slug, branch_name) -> +project_id
|
||||
-- * branch_visibility UNIQUE (rfc_slug, branch_name) -> +project_id
|
||||
-- * branch_contribute_grants UNIQUE (rfc_slug, branch_name, grantee_user_id) -> +project_id
|
||||
-- * stars UNIQUE (user_id, rfc_slug) -> +project_id
|
||||
-- * watches UNIQUE (user_id, rfc_slug) -> +project_id
|
||||
-- * pr_seen UNIQUE (user_id, rfc_slug, pr_number) -> +project_id
|
||||
-- * branch_chat_seen UNIQUE (user_id, rfc_slug, branch_name) -> +project_id
|
||||
-- * funder_consents PRIMARY KEY (user_id, rfc_slug) -> +project_id
|
||||
-- * rfc_collaborators UNIQUE INDEX (rfc_slug, user_id) -> +project_id
|
||||
-- * contribution_requests UNIQUE INDEX (rfc_slug, requester_user_id) WHERE pending -> +project_id
|
||||
-- * proposed_use_cases UNIQUE (scope, pr_number) -> +project_id
|
||||
-- (PR numbers are per-content-repo = per-project)
|
||||
-- cached_prs UNIQUE (repo, pr_number) is already globally unique (repo is the
|
||||
-- full 'org/repo' string, distinct per project) and needs no rebuild.
|
||||
-- ============================================================================
|
||||
|
||||
-- The project registry cache (mirrored from the git registry by the M3
|
||||
-- reconciler; rows are never written from user actions). content_repo is
|
||||
-- NULL until the mirror — or the M1 startup backfill (§22.13 step 1) — sets
|
||||
-- it from the deployment's configured repo.
|
||||
CREATE TABLE IF NOT EXISTS projects (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
content_repo TEXT,
|
||||
visibility TEXT NOT NULL DEFAULT 'gated'
|
||||
CHECK (visibility IN ('gated', 'public', 'unlisted')),
|
||||
config_json TEXT,
|
||||
registry_sha TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
-- The default project. visibility='public' preserves the pre-multi-project
|
||||
-- open-by-default posture (§22.5, §22.13). name is a placeholder the startup
|
||||
-- backfill / registry overwrites with the deployment's display name.
|
||||
INSERT OR IGNORE INTO projects (id, name, visibility)
|
||||
VALUES ('default', 'default', 'public');
|
||||
|
||||
-- Per-(user, project) membership and the §22.6 middle-tier role. This is the
|
||||
-- new tier between the §6.1 deployment role (users.role, now deployment-scope
|
||||
-- only) and the §6.3 per-RFC authority.
|
||||
CREATE TABLE IF NOT EXISTS project_members (
|
||||
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
role TEXT NOT NULL DEFAULT 'project_viewer'
|
||||
CHECK (role IN ('project_admin', 'project_contributor', 'project_viewer')),
|
||||
granted_by INTEGER REFERENCES users(id) ON DELETE SET NULL,
|
||||
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
PRIMARY KEY (project_id, user_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_project_members_user ON project_members(user_id);
|
||||
|
||||
-- The project_id spine across every slug-bearing table. Backfills existing
|
||||
-- rows to 'default' via the constant column default.
|
||||
ALTER TABLE cached_rfcs ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE cached_branches ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE cached_prs ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE branch_visibility ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE branch_contribute_grants ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE stars ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE threads ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE changes ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE pr_seen ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE branch_chat_seen ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE watches ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE notifications ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE actions ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE pr_resolution_branches ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE funder_consents ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE rfc_invitations ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE rfc_collaborators ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE proposed_use_cases ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
ALTER TABLE contribution_requests ADD COLUMN project_id TEXT NOT NULL DEFAULT 'default';
|
||||
|
||||
-- The catalog/directory lookup the M3 surfaces will make (RFCs in a project).
|
||||
-- The per-table composite-key rebuilds in the DEFERRED block above will add
|
||||
-- their own (project_id, …) indexes when they land.
|
||||
CREATE INDEX IF NOT EXISTS idx_cached_rfcs_project ON cached_rfcs(project_id);
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Slice M1 — the §22 multi-project spine.
|
||||
|
||||
Migration 025 introduces the `projects` and `project_members` tables, seeds
|
||||
the single `default` project (the N=1 case, §22.13), and threads a
|
||||
`project_id` column onto every slug-bearing table, backfilled to `default`.
|
||||
The §22.13 startup backfill then fills the default project's content_repo
|
||||
from META_REPO. These tests prove the spine lands without disturbing the
|
||||
single-project app.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from test_propose_vertical import ( # noqa: F401
|
||||
app_with_fake_gitea,
|
||||
tmp_env,
|
||||
)
|
||||
|
||||
# The 19 tables migration 025 threads project_id onto (docs/design/
|
||||
# multi-project-spec.md §5 amendment list).
|
||||
SLUG_TABLES = [
|
||||
"cached_rfcs", "cached_branches", "cached_prs", "branch_visibility",
|
||||
"branch_contribute_grants", "stars", "threads", "changes", "pr_seen",
|
||||
"branch_chat_seen", "watches", "notifications", "actions",
|
||||
"pr_resolution_branches", "funder_consents", "rfc_invitations",
|
||||
"rfc_collaborators", "proposed_use_cases", "contribution_requests",
|
||||
]
|
||||
|
||||
|
||||
def test_default_project_seeded_and_backfilled(app_with_fake_gitea):
|
||||
from app import db
|
||||
|
||||
app, _ = app_with_fake_gitea
|
||||
with TestClient(app):
|
||||
rows = list(db.conn().execute(
|
||||
"SELECT id, name, visibility, content_repo FROM projects"
|
||||
))
|
||||
assert len(rows) == 1
|
||||
row = rows[0]
|
||||
assert row["id"] == "default"
|
||||
# public preserves the pre-multi-project open-by-default posture.
|
||||
assert row["visibility"] == "public"
|
||||
# §22.13 startup backfill set content_repo from META_REPO (tmp_env).
|
||||
assert row["content_repo"] == "meta"
|
||||
|
||||
|
||||
def test_project_id_on_every_slug_table(app_with_fake_gitea):
|
||||
from app import db
|
||||
|
||||
app, _ = app_with_fake_gitea
|
||||
with TestClient(app):
|
||||
for table in SLUG_TABLES:
|
||||
cols = {r["name"]: r for r in db.conn().execute(
|
||||
f"PRAGMA table_info({table})"
|
||||
)}
|
||||
assert "project_id" in cols, f"{table} missing project_id"
|
||||
col = cols["project_id"]
|
||||
# NOT NULL with the constant 'default' backfill default.
|
||||
assert col["notnull"] == 1, f"{table}.project_id should be NOT NULL"
|
||||
assert col["dflt_value"] == "'default'", f"{table}.project_id default"
|
||||
|
||||
|
||||
def test_existing_row_backfills_to_default(app_with_fake_gitea):
|
||||
"""A row inserted the old way (no project_id) lands in the default
|
||||
project — the trick that keeps every pre-multi-project INSERT working."""
|
||||
from app import db
|
||||
|
||||
app, _ = app_with_fake_gitea
|
||||
with TestClient(app):
|
||||
db.conn().execute(
|
||||
"INSERT INTO cached_rfcs (slug, title, state) VALUES (?, ?, ?)",
|
||||
("human", "Human", "active"),
|
||||
)
|
||||
got = db.conn().execute(
|
||||
"SELECT project_id FROM cached_rfcs WHERE slug = 'human'"
|
||||
).fetchone()["project_id"]
|
||||
assert got == "default"
|
||||
|
||||
|
||||
def test_project_members_table_shape(app_with_fake_gitea):
|
||||
from app import db
|
||||
|
||||
app, _ = app_with_fake_gitea
|
||||
with TestClient(app):
|
||||
cols = {r["name"] for r in db.conn().execute(
|
||||
"PRAGMA table_info(project_members)"
|
||||
)}
|
||||
assert cols == {"project_id", "user_id", "role", "granted_by", "granted_at"}
|
||||
# The role CHECK rejects an unknown role.
|
||||
db.conn().execute(
|
||||
"INSERT INTO users (id, display_name, role) VALUES (1, 'Ben', 'owner')"
|
||||
)
|
||||
db.conn().execute(
|
||||
"INSERT INTO project_members (project_id, user_id, role) "
|
||||
"VALUES ('default', 1, 'project_admin')"
|
||||
)
|
||||
import sqlite3
|
||||
try:
|
||||
db.conn().execute(
|
||||
"INSERT INTO project_members (project_id, user_id, role) "
|
||||
"VALUES ('default', 1, 'nonsense')"
|
||||
)
|
||||
assert False, "CHECK should reject an unknown project role"
|
||||
except sqlite3.IntegrityError:
|
||||
pass
|
||||
|
||||
|
||||
def test_seed_default_project_is_idempotent(app_with_fake_gitea):
|
||||
"""Re-running the backfill never clobbers a content_repo already set —
|
||||
so a later registry mirror (M3) wins over the META_REPO fallback."""
|
||||
from app import db, projects
|
||||
from app.config import load_config
|
||||
|
||||
app, _ = app_with_fake_gitea
|
||||
with TestClient(app):
|
||||
db.conn().execute(
|
||||
"UPDATE projects SET content_repo = 'ohm-content' WHERE id = 'default'"
|
||||
)
|
||||
projects.seed_default_project(load_config())
|
||||
got = db.conn().execute(
|
||||
"SELECT content_repo FROM projects WHERE id = 'default'"
|
||||
).fetchone()["content_repo"]
|
||||
assert got == "ohm-content"
|
||||
|
||||
|
||||
def test_registry_repo_config_wired(app_with_fake_gitea, monkeypatch):
|
||||
from app.config import load_config
|
||||
|
||||
monkeypatch.setenv("REGISTRY_REPO", "wiggleverse-registry")
|
||||
assert load_config().registry_repo == "wiggleverse-registry"
|
||||
# Optional through M1: absent resolves to empty, not a startup failure.
|
||||
monkeypatch.delenv("REGISTRY_REPO", raising=False)
|
||||
assert load_config().registry_repo == ""
|
||||
@@ -315,13 +315,17 @@ first, the surfaces that consume them after, hardening last. Each slice is
|
||||
shippable: a deployment can stop at any slice boundary and still run (the
|
||||
default project keeps the N=1 case whole throughout).
|
||||
|
||||
**M1 — The project spine (schema + registry + default-project migration).**
|
||||
The `projects` and `project_members` tables; the §4 registry mirror (webhook
|
||||
+ reconciler over the registry repo); `project_id` threaded through every §5
|
||||
table per the Part B list; the §22.13 migration that generates the default
|
||||
project, backfills `project_id`, and stands up the redirect. No UI yet — the
|
||||
app runs exactly as before, single project, with the spine underneath. This
|
||||
is the slice that must land atomically; everything after is additive.
|
||||
**M1 — The project spine (schema + default-project migration).** *(landed)*
|
||||
The `projects` and `project_members` tables; `project_id` threaded
|
||||
additively onto every slug-bearing §5 table (migration 025); the §22.13
|
||||
default project generated and every existing row backfilled to it; the
|
||||
startup backfill that fills the default project's `content_repo` from
|
||||
`META_REPO`; `REGISTRY_REPO` wired into config (consumed in M3). No UI, no
|
||||
routing change, no registry mirror yet — the app runs exactly as before,
|
||||
single project, with the spine underneath. Additive only: no table rebuilds
|
||||
(the slug-keyed uniqueness/PK rework is deferred to the slice that activates
|
||||
project #2, enumerated in migration 025's header). This is the foundation
|
||||
everything after builds on.
|
||||
|
||||
**M2 — Project-scoped authorization + the §22.7 resolver.** The three-tier
|
||||
composition: `project_members` roles, the most-permissive union with
|
||||
@@ -331,12 +335,16 @@ re-checked under the project axis. Still single visible project; this slice
|
||||
is verifiable by granting/revoking roles on the default project and asserting
|
||||
the gates.
|
||||
|
||||
**M3 — Routing + runtime branding.** The `/p/<project>/` route prefix, the
|
||||
308 redirects, `GET /api/deployment` and `GET /api/projects/:id`, the
|
||||
frontend cut from `VITE_APP_NAME` to runtime config, per-project `theme`
|
||||
token overlay. The deployment directory at `/` and the project switcher in
|
||||
deployment chrome. After M3 a deployment with two registry projects is fully
|
||||
navigable.
|
||||
**M3 — Registry mirror + routing + runtime branding.** The §4 registry
|
||||
mirror (webhook + reconciler over the `REGISTRY_REPO`, populating `projects`
|
||||
rows beyond `default`); the `/p/<project>/` route prefix and the 308
|
||||
redirects off the old corpus-root URLs; `GET /api/deployment` and `GET
|
||||
/api/projects/:id`; the frontend cut from `VITE_APP_NAME` to runtime config,
|
||||
per-project `theme` token overlay. The deployment directory at `/` and the
|
||||
project switcher in deployment chrome. After M3 a deployment with two
|
||||
registry projects is fully navigable — which makes this the slice that must
|
||||
also land the deferred slug-keyed uniqueness/PK rebuilds (migration 025
|
||||
header) before a second project can collide with the first.
|
||||
|
||||
**M4 — Per-project corpus surfaces.** The §7 catalog, §8 RFC view, §9/§10
|
||||
proposal/PR flows, §13 graduation, and §14 philosophy all confirmed working
|
||||
|
||||
Reference in New Issue
Block a user