diff --git a/README.md b/README.md index a832f96..dcaf277 100644 --- a/README.md +++ b/README.md @@ -5,10 +5,12 @@ materializes the Wiggleverse RFC framework specified in [`SPEC.md`](./SPEC.md). The framework's mission lives in [`PHILOSOPHY.md`](./PHILOSOPHY.md); the spec is the binding contract; this README is how to bring the app up against a local Gitea instance -and exercise the slice the build session has shipped so far. +and exercise the surface the build has shipped. -The implementation is in progress. See [`docs/DEV.md`](./docs/DEV.md) -for the slicing plan and the current state. +The v1 build is complete. Subsequent sessions pick from §19.2 by user +choice per §19.3's working agreement. See +[`docs/DEV.md`](./docs/DEV.md) for the build history and the work mode +that follows v1. ## What the app expects to talk to @@ -107,6 +109,8 @@ Optional values, picked up at process start: | `EMAIL_ENABLED` | `1` (default) to dispatch email; `0` to suppress all sends without disabling the inbox. | | `EMAIL_BUNDLE_THRESHOLD` | Held-during-quiet-hours threshold for the "Activity while you were away" bundle (default 5, §15.4). | | `DIGEST_TICK_SECONDS` | Cadence of the §15.5 digest scheduler's loop (default 3600). Tests drive ticks synchronously via `digest.run_tick`. | +| `HYGIENE_TICK_SECONDS` | Cadence of the §12 hygiene scheduler's loop (default 3600). Tests drive ticks via `hygiene.run_tick(now=...)`. | +| `WEBHOOK_EMAIL_BOUNCE_SECRET` | When set, `/api/webhooks/email-bounce` requires the same value in the `X-Webhook-Secret` header. Unset leaves the webhook open for dev — the v1 contract. | ### 6. Install dependencies @@ -159,9 +163,10 @@ Open `http://localhost:5173`. Sign in with your owner-zero Gitea account. The catalog should appear empty; the **+ Propose New RFC** button at the bottom opens the propose modal. -## What the build lets you do so far +## What the build lets you do -Slices 1–6 are shipped. End-to-end paths the app supports today: +Slices 1–8 are shipped — v1 is complete. End-to-end paths the app +supports: - **Propose → idea PR → merge → super-draft** (Slice 1, §9.1–§9.3). - **Super-draft body editing** via meta-repo edit branches, with AI @@ -198,18 +203,33 @@ Slices 1–6 are shipped. End-to-end paths the app supports today: quiet hours hold email and digest while letting the inbox row still land, and the per-user mute suppresses inbox rows produced by a specific actor (Slice 6). +- **§14 chrome and the settings / admin neighbourhoods** — the + landing page with the three-item deck, the `/philosophy` route + reading `PHILOSOPHY.md` from disk, the persistent About header + link, `/settings/notifications` exposing the five §15-derived + sub-sections, and `/admin` as the four-tab home base for role + management, write-mute, audit-log, graduation-queue, and the §6.5 + permission-events read (Slice 7). +- **§12 branch hygiene + §10.7 post-merge deletion** — the + 30/90-day timers ride on a scheduler next to the digest, the bot + is the only Git writer per §1, and the audit-log rows surface as + "the app" actor per §15.9. The §19.2 candidates the hardening + pass folded in: cache bootstrap against a meta repo the bot did + not author, branch-name path routing via `{branch:path}`, in-app + merge for `meta_metadata` PRs, the graduation-rollback branch + cleanup, and the email-bounce webhook signing seam (Slice 8). This exercises the §4 cache (webhook + reconciler), the §6 permission model in full, the §1 bot wrapper (every Git write goes through it, every commit and PR carries the `On-behalf-of:` -trailer), and the §17 routing-collapse rule that lets active and -super-draft surfaces share their endpoints. +trailer), the §17 routing-collapse rule that lets active and +super-draft surfaces share their endpoints, and three scheduled +jobs in the same shape (reconciler, digest, hygiene). -Out of scope for the slices shipped so far: landing-page and -`/philosophy` chrome polish, the notification-settings UI surface, -and the admin neighbourhood (Slice 7, §14 + §19.2 candidates); the -§12 30/90 branch-hygiene timers (Slice 8). The full slicing plan -and the next slice's brief live in [`docs/DEV.md`](./docs/DEV.md). +Out of scope for v1: every item under §16 ("What is deliberately +deferred"), and the §19.2 candidates the hardening pass left +queued. Subsequent sessions pick from §19.2 by user choice per +§19.3's working agreement. ## Verifying it worked @@ -263,5 +283,6 @@ has something real to render. The spec's decisions answer to it. - [`docs/DEV.md`](./docs/DEV.md) — the build's slicing plan, the current state, and the next slice's brief. -- [`deploy/DEPLOY.md`](./deploy/DEPLOY.md) — single-host production - deployment behind nginx + Let's Encrypt. +- [`deploy/RUNBOOK.md`](./deploy/RUNBOOK.md) — single-host production + bring-up, day-2 operations (logs, database backup, secret rotation, + the §12 hygiene cadence), and rollback shape. diff --git a/SPEC.md b/SPEC.md index e34d4b0..9479851 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1391,10 +1391,12 @@ After merge, the PR page renders identically with a "Merged" banner in the header strip. The diff, the compressed conversation, and all review threads become read-only. The branch enters the closed state per §12's hygiene table (the 90-day deletion timer starts; owners and -arbiters can still pin to disable it). The branch chat persists per -§8.4 as historical record, with new posts disabled. The PR page remains -at its stable URL indefinitely — it is the canonical surface for "show -me how this definition came to be." +arbiters can still pin to disable it). The 90-day timer rides on §12's +hygiene sweep rather than its own schedule — the actuator is one extra +branch in the sweep that already handles meta-repo edit branches. The +branch chat persists per §8.4 as historical record, with new posts +disabled. The PR page remains at its stable URL indefinitely — it is +the canonical surface for "show me how this definition came to be." Post-merge questions about the merged work belong on main's chat (§8.4), not on the merged branch's. Main is the surface for ongoing argument @@ -1492,6 +1494,22 @@ work is paused but legitimately ongoing. | Deleted | 60 days after close (90 from last activity) | branch removed from Gitea, row remains | | Pinned | owner/arbiter pins | auto-close disabled | +The bot is the actuator on the deletion: per §1 the bot is the only +Git writer, and the §12 sweep fires through it. The audit row lands +with `actor_user_id = NULL` and `on_behalf_of = ` per §15.9 +— the timer is system-generated, "the app" in the noun slot. No +notification fires on a hygiene gesture; the affected population would +be churn-grade noise per §15.4. + +The per-user message-cursor preservation contract: chat history +survives the branch's deletion in Gitea because the chat tables +(`thread_messages`, `branch_chat_seen`) are app-canonical, not cached. +Only the Gitea branch and the `cached_branches` row are touched by +the sweep; every `branch_chat_seen` row that points at a message on +the deleted branch stays intact, so a returning contributor sees the +same cursor when the conversation surfaces in the §9.8 pre-graduation +history affordance (or wherever else the app renders archived chat). + Future story (not v1): out-of-band reopening of a deleted branch by email request to an owner. @@ -2435,15 +2453,32 @@ surface. With Topic 13 folded in, the structural surface is complete. What follows is no longer "topics that block specifying v1" but "topics to address during or shortly after the v1 build." -### 19.1 Next slice: hardening +### 19.1 v1 is complete -Slice 7 of the build has landed. The §14 chrome, the -`/settings/notifications` neighborhood, and the `/admin` home base -all run end-to-end against the local Gitea, and the next slice has -the v1 surface fully wrapped — what remains is the hardening pass -that lets a single-operator deployment actually run. +Slice 8 — the hardening pass — has landed. With it, every slice of the +v1 build is in: the bot wrapper, the §4 cache, Gitea OAuth + user +provisioning, the §5 schema, the §7 catalog, the §8 active-RFC view in +full, the §9.4–§9.7 super-draft vertical, the §10 PR flow against both +per-RFC repos and meta-repo edit branches, the §13 graduation flow +with rollback, the §15 notifications surface in full, the §14 chrome +plus the `/settings/notifications` and `/admin` neighborhoods, and the +§12 + §10.7 branch hygiene with the §19.2 candidates the hardening +pass clustered in. -The §14.1 landing page now carries the title, the subtitle, the +Subsequent sessions pick from §19.2 by user choice per §19.3's working +agreement. They need not be sequential — the user picks the next topic +based on what evidence the running app surfaces. The build itself is +the source-of-truth artifact; §19.2 is the queue of decisions to +settle when their turn comes. + +The Slice 8 entry below is preserved for the record. Future sessions +that fold a §19.2 candidate into the spec should mark that candidate +*settled* with a brief pointer at the section that resolved it — the +same shape Topic 13's resolution used. + +### 19.1.1 Slice 8 record + +**Slice 7 background.** The §14.1 landing page now carries the title, the subtitle, the short-form pitch from `PHILOSOPHY.md`, the sign-in affordance, the secondary "Read the full philosophy" link, and a three-item deck underneath the pitch that names what the framework is — one word per @@ -2507,42 +2542,60 @@ filter chips, the graduation-queue partition under both preconditions, and the permission-events listing. The full Slices 1–7 test suite is 75/75 green. -**Slice 8 is the hardening pass — the last slice of the v1 build.** -Three pieces hang together: +**Slice 8 — the hardening pass — completed the v1 build.** Three +pieces hang together: -The §12 30/90 branch-hygiene timers — the formalized policy that -closes the loop on §11.5's branch lifecycle (open → merged → 30d -read-only → 90d deleted-by-bot, with the per-user-message-cursor -preservation contract). The wiring is a scheduled task next to the -§15.5 digest scheduler; the §10.7 90-day deletion timer Slice 3 -left deferred lives here too. +The §12 30/90 branch-hygiene timers ride on `HygieneScheduler` in +`backend/app/hygiene.py`, modeled on `DigestScheduler`'s shape. The +sweep runs hourly by default, exposes a `run_tick(now=...)` test +seam, and orders its queries delete-first so a single sweep crossing +both the 30d and 90d boundaries (the cache-bootstrap and clock-jump +case) lands the delete rather than spending one tick at `closed` with +a fresh `closed_at` that would defer the delete by another 90 days. +The §10.7 90-day deletion timer Slice 3 deferred is one branch of the +same sweep. The bot grew a `delete_branch` method that takes +`actor: Actor | None`; the timer paths pass `None` so the audit row +lands `actor_user_id = NULL` and `on_behalf_of = ` per +§15.9. No notifications fire on hygiene gestures (the action kinds +are absent from `_AUTO_WATCH_ACTIONS` and `_ROUTING`). -An end-to-end smoke pass over the working surfaces — propose → -super-draft → branch → PR → merge → graduate → active-RFC PR → -notification fans out → inbox → email — to catch the integration -seams a per-slice test wouldn't. Plus the §19.2 candidates the -hardening pass is the natural place to fold in: cache bootstrap -from a meta repo (the audit-log-first attribution shape Slice 1 -chose, exercised against a meta repo with history the bot did not -author), branch-name path routing (converting every -`branches/` to `{branch:path}` with route-ordering -discipline), and the small Slice-2-onward follow-ons that are -deferred until the hardening pass demands them. +The §19.2 candidates folded in: -The dev/prod deployment shape — the `deploy/` directory already -has the nginx vhost, the systemd unit, and a runbook stub; Slice 8 -proves the bring-up against a fresh host, settles the secret- -material handling (the existing `.env.example` plus the §15.4 -SMTP wiring), and lands the README updates that let a new operator -get from `git clone` to a signed-in browser. +- **Cache bootstrap from a pre-existing meta repo** — the + `_resolve_actor` fallback chain now has explicit integration + coverage against history the bot did not author (audit-log row, + trailer-parse, raw-login last-resort). +- **Branch-name path routing** — every `branches/` route is + `{branch:path}` with the bare GET declared last among the + branch-scoped GETs so deeper routes match first. +- **In-app merge for metadata PRs** — `api_prs._require_pr` accepts + `pr_kind='meta_metadata'`; the existing merge endpoint handles them + uniformly. +- **Graduation rollback's branch cleanup** — + `api_graduation._undo_open_pr` deletes the + `graduate--<6hex>` branch after closing the PR. +- **Email bounce webhook authentication** — `WEBHOOK_EMAIL_BOUNCE_SECRET` + is the signing seam; when set, the webhook requires the same value + in `X-Webhook-Secret`. -The next build session should read `SPEC.md`, `README.md`, -`docs/DEV.md`, and this §19.1 entry and pick up Slice 8 cleanly -without re-briefing. The working agreement in §19.3 continues to -apply: implement the slice, correct the spec only where running -code reveals it was wrong at a structural level, accumulate new -candidate topics in §19.2, do not extend the spec beyond what the -slice requires. +The dev/prod deployment shape: `deploy/RUNBOOK.md` is rewritten from +the prior stub into a real runbook (prerequisites, first-time +bring-up, day-2 operations, rollback, troubleshooting); the README's +`.env` table grows the SMTP block, `HYGIENE_TICK_SECONDS`, and +`WEBHOOK_EMAIL_BOUNCE_SECRET`; the `.env.example` carries the same +fields with inline commentary. + +Slice 8 ships covered by `test_hygiene_vertical.py` (seven cases), +`test_branch_path_routing.py` (four cases), `test_metadata_pr_merge.py` +(three cases), `test_cache_bootstrap.py` (four cases), and +`test_e2e_smoke.py` (three cases including the full lifecycle walk). +The full Slices 1–8 test suite is 96/96 green. + +§12 grew an explicit note that the bot is the actuator and that the +per-user `branch_chat_seen` cursor survives branch deletion — the +§11.5 contract made implicit; running code asked for the load-bearing +line to live in §12 too. §10.7 grew a one-line pointer that the timer +rides on §12's sweep rather than its own schedule. ### 19.2 Candidate topics for sessions after the v1 build lands @@ -2634,7 +2687,13 @@ binding. null-system, and how the bot-vs-user distinction in §15.9 extends. Will surface during build if AI turn-times grow large enough to warrant. -- **Cache bootstrap from a pre-existing meta repo.** §4.1 covers +- **Cache bootstrap from a pre-existing meta repo.** *Settled in + Slice 8 — the `_resolve_actor` fallback chain Slice 1 chose + (audit log → `On-behalf-of:` trailer → raw Gitea login) now has + explicit integration coverage in `test_cache_bootstrap.py` + against history the bot did not author. The disaster-recovery + and transferred-meta-repo cases work without the cache thinking + the bot proposed everything pre-app.* §4.1 covers steady-state cache freshness — the webhook is the fast path, the reconciler the safety net — but assumes the cache grew up alongside the bot. If the cache is rebuilt from scratch against @@ -2648,7 +2707,14 @@ binding. topic once the cost of "the cache thinks the bot proposed everything pre-app" becomes concrete. Touches §4.1 (the reconciler's job description) and §15.9 (the attribution rule). -- **Branch-name path routing.** Slice 2's `branches/` +- **Branch-name path routing.** *Settled in Slice 8 — every + `branches/` route in `api_branches.py` and `api_prs.py` is + now `{branch:path}`. The bare GET is declared last among the + branch-scoped GETs so deeper routes (`threads`, + `threads/{thread_id}/messages`) match before the greedy path + matcher swallows their sub-paths. `test_branch_path_routing.py` + covers the slashed-branch read and the deeper-route ordering.* + Slice 2's `branches/` endpoints use FastAPI's default `{branch}` path-segment matcher, which refuses slashes. The Slice 2 auto-generated branch name steered around this with `-draft-`, but a user who @@ -2721,12 +2787,21 @@ binding. the app — but anyone reading the PR directly on Gitea sees the pre-edit text. A small follow-on that propagates the edit through the bot wrapper closes the loop. -- **The §10.7 90-day deletion timer wiring.** Slice 3 lands the +- **The §10.7 90-day deletion timer wiring.** *Settled in Slice 8 — + the timer rides on `HygieneScheduler`'s sweep alongside the §12 + 30/90 idle timers. See §10.7 and §12 for the spec text and + `backend/app/hygiene.py` for the implementation.* Slice 3 lands the PR-merged state and the read-only treatment but does not wire the §12 hygiene timer that fires the deletion. Slice 8 ("Hardening") owns the §12 30/90 timers as a whole; calling out here so the dependency is explicit. -- **In-app merge for metadata PRs.** Slice 4's metadata pane opens +- **In-app merge for metadata PRs.** *Settled in Slice 8 — + `api_prs._require_pr` now accepts `pr_kind='meta_metadata'`. The + existing `prs//merge` endpoint handles them uniformly; the + diff-rendered review surface degrades gracefully (a metadata PR + has no body diff to render). `test_metadata_pr_merge.py` + covers the merge, the contributor refusal, and the withdraw path.* + Slice 4's metadata pane opens a meta-repo PR per §9.5; the merge surface for those PRs is the Gitea web UI for now, because `api_prs.py`'s merge endpoint is scoped to body-changing PRs (`rfc_branch` and `meta_body_edit`). @@ -2759,6 +2834,10 @@ binding. hits cases where a single sequence runs long enough that the reload-during-graduation path matters. - **Graduation PR auto-close on rollback's `close_graduation_pr`.** + *Settled in Slice 8 — `api_graduation._undo_open_pr` now deletes + the `graduate--<6hex>` branch via `bot.delete_branch` after + closing the PR. `test_hygiene_vertical.test_graduation_rollback_deletes_dash_suffixed_branch` + covers it.* Slice 5's rollback closes the graduation PR via `gitea.close_pull` but leaves the dash-suffixed branch (`graduate--<6hex>`) in place. The branch is short-lived in steady-state — graduation @@ -2824,7 +2903,15 @@ binding. actor verb form so each row reads naturally without picking up an apparent personification. Defer-able until contributor feedback surfaces an irritating render. -- **Email bounce webhook authentication.** Slice 6's +- **Email bounce webhook authentication.** *Settled in Slice 8 — the + `WEBHOOK_EMAIL_BOUNCE_SECRET` env var is the shared-secret seam. + When set, the webhook requires the same value in the + `X-Webhook-Secret` header; when unset, the v1 unauthenticated + behavior holds for dev. The per-provider signature scheme + (Sendgrid signed events, SES SNS signatures, Postmark HMAC) maps + cleanly onto the same seam: the operator picks one when wiring a + real SMTP provider.* + Slice 6's `/api/webhooks/email-bounce` accepts unauthenticated POSTs for v1 — the SMTP provider's callback URL is the contract. When an actual provider is wired in, the webhook needs a shared secret diff --git a/backend/.env.example b/backend/.env.example index 58b7202..7b98a77 100644 --- a/backend/.env.example +++ b/backend/.env.example @@ -41,3 +41,43 @@ ENABLED_MODELS=claude ANTHROPIC_API_KEY= GOOGLE_API_KEY= OPENAI_API_KEY= + +# --- Email (§15.4) --- +# Leave SMTP_HOST unset to use the stdout fallback — the integration +# tests rely on it, and a dev environment without a real SMTP provider +# still sees envelope traces in the logs. Set the rest to wire a real +# provider (Postmark, Mailgun, SES, etc.). +SMTP_HOST= +SMTP_PORT=587 +SMTP_USER= +SMTP_PASSWORD= +SMTP_STARTTLS=1 + +# Single non-spoofing envelope identity per §15.9 — every notification +# email goes out from the same address regardless of which user's +# gesture produced it. Configure both the SPF and DKIM records for +# this address with the chosen SMTP provider. +EMAIL_FROM=notifications@wiggleverse.local +EMAIL_FROM_NAME=Wiggleverse + +# §15.4 bundle threshold: when a user's quiet-hours release queue is +# at least this big, the flush goes out as one bundled "Activity while +# you were away" email instead of individual sends. +EMAIL_BUNDLE_THRESHOLD=5 + +# Set to 0 to suppress every outbound email (the inbox and SSE still +# work — only the email channel turns off). +EMAIL_ENABLED=1 + +# --- Email-bounce webhook (§15.4 + §19.2-settled) --- +# When set, `/api/webhooks/email-bounce` requires the same value in +# the `X-Webhook-Secret` header. Pick a long random string and +# configure the SMTP provider's webhook to inject it. When unset, +# the webhook stays unauthenticated for dev — the v1 contract. +WEBHOOK_EMAIL_BOUNCE_SECRET= + +# --- §12 hygiene cadence (Slice 8) --- +# How often the hygiene scheduler sweeps for the 30/90-day boundaries. +# Production default is hourly; tests override to seconds via the same +# env var. +HYGIENE_TICK_SECONDS=3600 diff --git a/backend/tests/test_branch_path_routing.py b/backend/tests/test_branch_path_routing.py new file mode 100644 index 0000000..fe86c13 --- /dev/null +++ b/backend/tests/test_branch_path_routing.py @@ -0,0 +1,157 @@ +"""End-to-end integration test for the §19.2 "branch-name path routing" +candidate that Slice 8 settles. + +Slice 2's `branches/` endpoints used FastAPI's default +`{branch}` matcher, which refuses slashes. Slice 2 worked around it +with dash-separated branch names (`-draft-`); Slice 8 +converts every `branches/` to `{branch:path}` and reorders +the routes so the bare GET is declared last — the more-specific +`threads` and `threads/{thread_id}/messages` GETs match first. + +The tests prove: + + * A branch with a slash in the name reads correctly via + `GET /api/rfcs//branches/`. + * The deeper threads GET still works for unslashed branches. + * The deeper threads GET still works for slashed branches — + the ordering discipline holds. + * The POST routes for slashed branches still resolve (visibility, + chat-seen, threads). +""" +from __future__ import annotations + +import pytest + +from test_propose_vertical import ( # noqa: F401 + FakeGitea, + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) +from test_rfc_view_vertical import SEED_BODY, seed_active_rfc # noqa: F401 + + +def _seed_slashed_branch(fake: FakeGitea, *, slug: str, branch: str, body: str) -> None: + """Seed a branch with a slash in the name directly on FakeGitea + + cached_branches so the GET / threads / etc. endpoints can read it.""" + from app import db + + repo_full = f"wiggleverse/rfc-0001-{slug}" + owner, repo = repo_full.split("/", 1) + sha = fake._next_sha() + fake.branches[(owner, repo)][branch] = {"sha": sha, "ts": "2026-05-23T00:00:00Z"} + fake.files[(owner, repo, branch, "RFC.md")] = {"content": body, "sha": sha} + db.conn().execute( + """ + INSERT OR IGNORE INTO cached_branches (rfc_slug, branch_name, head_sha, state, last_commit_at) + VALUES (?, ?, ?, 'open', datetime('now')) + """, + (slug, branch, sha), + ) + + +def test_get_branch_view_reads_slashed_branch_name(app_with_fake_gitea): + """The §19.1 brief's headline assertion: a branch with a slash in + the name reads correctly via the `{branch:path}` route shape.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _seed_slashed_branch( + fake, slug="ohm", branch="alice/feature-rename", + body="# Slashed branch body\n", + ) + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + + r = client.get("/api/rfcs/ohm/branches/alice/feature-rename") + assert r.status_code == 200, r.text + view = r.json() + assert view["branch_name"] == "alice/feature-rename" + assert "Slashed branch body" in view["body"] + + +def test_deeper_threads_get_still_routes_for_unslashed_branch(app_with_fake_gitea): + """Ordering discipline: declaring the bare GET last means the + `threads` GET still wins for unslashed `branches/foo/threads`.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _seed_slashed_branch( + fake, slug="ohm", branch="plain-branch", + body="# plain\n", + ) + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + + r = client.get("/api/rfcs/ohm/branches/plain-branch/threads") + assert r.status_code == 200, r.text + # Empty until a thread is posted, but the route fired. + assert "items" in r.json() + + +def test_deeper_threads_get_still_routes_for_slashed_branch(app_with_fake_gitea): + """The branch name carries a slash; the threads GET must still + match (branch={branch:path} captures `alice/feature`, threads + is the literal anchor).""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _seed_slashed_branch( + fake, slug="ohm", branch="alice/feature", + body="# slashed\n", + ) + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + + r = client.get("/api/rfcs/ohm/branches/alice/feature/threads") + assert r.status_code == 200, r.text + assert "items" in r.json() + + +def test_post_visibility_resolves_for_slashed_branch(app_with_fake_gitea): + """The §11.1 visibility POST must also route correctly for a + slashed branch — proves the {branch:path} matcher applies to the + write paths too.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _seed_slashed_branch( + fake, slug="ohm", branch="alice/private-work", + body="# private\n", + ) + # Materialize the creator row so alice can flip her own branch. + from app import db + db.conn().execute( + """ + INSERT OR IGNORE INTO branch_visibility (rfc_slug, branch_name, read_public, contribute_mode) + VALUES ('ohm', 'alice/private-work', 1, 'just-me') + """ + ) + db.conn().execute( + """ + INSERT OR IGNORE INTO actions (actor_user_id, on_behalf_of, action_kind, rfc_slug, branch_name) + VALUES (2, 'alice', 'create_branch', 'ohm', 'alice/private-work') + """ + ) + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + + r = client.post( + "/api/rfcs/ohm/branches/alice/private-work/visibility", + json={"read_public": False, "contribute_mode": "just-me"}, + ) + assert r.status_code == 200, r.text + assert r.json()["ok"] is True diff --git a/backend/tests/test_cache_bootstrap.py b/backend/tests/test_cache_bootstrap.py new file mode 100644 index 0000000..eb00a57 --- /dev/null +++ b/backend/tests/test_cache_bootstrap.py @@ -0,0 +1,260 @@ +"""End-to-end integration test for the §19.2 "cache bootstrap from a +pre-existing meta repo" candidate that Slice 8 settles. + +Per §4.1, the cache is rebuildable from Gitea. Per §15.9, the cache +resolves the actor on a PR by joining against the §6.5 `actions` log, +falling back to the `On-behalf-of:` trailer in the PR body, then to +the raw Gitea login as last resort. Slice 1 chose this fallback chain +and Slice 8 exercises it against history the bot did not author — +the disaster-recovery / transferred-meta-repo case. + +The tests prove: + + * When `actions` carries a matching row, the audit log wins — + `opened_by` reads the on-behalf-of login regardless of the bot's + appearance as Gitea opener. + * When `actions` is empty but the PR body carries + `On-behalf-of: Name `, the trailer parses cleanly. + * When both are absent, the raw Gitea opener wins (and if even + that is the bot, the bot login is what surfaces — the v1 + fallback, honest about who Gitea sees). + * The §15.9 framing holds: the bot login surfaces only in the + Git log and the trailer, never inside the audit-log path. +""" +from __future__ import annotations + +import json +import pytest + +from test_propose_vertical import ( # noqa: F401 + FakeGitea, + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) + + +def _seed_meta_pr_directly_in_fake( + fake: FakeGitea, + *, + slug: str, + pr_number: int, + head_branch: str, + title: str, + body: str, + gitea_opener_login: str, +) -> None: + """Push a PR into FakeGitea as if it existed before the app cache + was bootstrapped — no propose endpoint, no audit log row, just a + pull on the meta repo with a chosen `user.login`. The reconciler + pulls this on its first sweep.""" + pr = { + "number": pr_number, + "title": title, + "body": body, + "head": {"ref": head_branch, "sha": "sha-bootstrap"}, + "base": {"ref": "main"}, + "state": "open", + "merged": False, + "merged_at": None, + "closed_at": None, + "created_at": "2026-05-23T00:00:00Z", + "user": {"login": gitea_opener_login}, + } + fake.pulls.setdefault(("wiggleverse", "meta"), []).append(pr) + # Create a corresponding branch so refresh_meta_branches sees it. + fake.branches[("wiggleverse", "meta")][head_branch] = { + "sha": "sha-bootstrap", "ts": "2026-05-23T00:00:00Z", + } + + +def test_audit_log_first_wins_over_bot_opener(app_with_fake_gitea): + """If `actions` carries a row for this PR, that's the authoritative + actor — even when Gitea reports the bot as the opener. This is the + common steady-state path (the bot opened on behalf of a human, the + audit log captured it).""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + # Seed a cached RFC and an `actions` row for the PR before + # the reconciler runs. + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('alpha', 'Alpha', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + db.conn().execute( + """ + INSERT INTO actions + (actor_user_id, on_behalf_of, action_kind, rfc_slug, pr_number) + VALUES (NULL, 'alice', 'propose_rfc', 'alpha', 99) + """ + ) + # Seed the fake's PR as if Gitea reports the bot as opener. + _seed_meta_pr_directly_in_fake( + fake, + slug="alpha", pr_number=99, + head_branch="propose/alpha", + title="Propose: Alpha", + body="Some idea\n", + gitea_opener_login="rfc-bot", + ) + + import asyncio + asyncio.run(cache_mod.refresh_meta_pulls(app.state.config, app.state.gitea)) + + row = db.conn().execute( + "SELECT opened_by FROM cached_prs WHERE pr_number = 99" + ).fetchone() + assert row is not None + # Audit-log row wins. + assert row["opened_by"] == "alice" + + +def test_trailer_parses_when_audit_log_missing(app_with_fake_gitea): + """The cache-bootstrap-against-history-the-bot-did-not-author case: + no `actions` rows, but the PR body carries the §6.5 + `On-behalf-of:` trailer. The trailer parses cleanly and the right + actor surfaces.""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('beta', 'Beta', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + # No actions row. PR body carries the trailer. + _seed_meta_pr_directly_in_fake( + fake, + slug="beta", pr_number=42, + head_branch="propose/beta", + title="Propose: Beta", + body="A new framing\n\nOn-behalf-of: Charlie ", + gitea_opener_login="rfc-bot", + ) + + import asyncio + asyncio.run(cache_mod.refresh_meta_pulls(app.state.config, app.state.gitea)) + + row = db.conn().execute( + "SELECT opened_by FROM cached_prs WHERE pr_number = 42" + ).fetchone() + assert row["opened_by"] == "charlie" + + +def test_raw_gitea_login_used_when_audit_and_trailer_both_absent(app_with_fake_gitea): + """When neither the audit log nor the trailer carries an actor, + the raw Gitea login is the last-resort fallback per §15.9 / Slice 1. + A non-bot login surfaces as the actor; a bot login surfaces as + the bot (honest about what Gitea sees — the v1 contract).""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('gamma', 'Gamma', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + # No audit row, no trailer — but Gitea reports a real user as opener. + _seed_meta_pr_directly_in_fake( + fake, + slug="gamma", pr_number=7, + head_branch="propose/gamma", + title="Propose: Gamma", + body="No trailer here\n", + gitea_opener_login="dana", # a real human, not the bot + ) + + import asyncio + asyncio.run(cache_mod.refresh_meta_pulls(app.state.config, app.state.gitea)) + + row = db.conn().execute( + "SELECT opened_by FROM cached_prs WHERE pr_number = 7" + ).fetchone() + assert row["opened_by"] == "dana" + + +def test_full_reconciler_sweep_resolves_actors_via_fallback_chain(app_with_fake_gitea): + """End-to-end: a clean `cached_*` set plus a meta repo that has + history with no audit-log rows. The reconciler's first sweep must + bring everything up correctly — every PR resolves an actor through + the fallback chain without crashing.""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + # Three PRs, three fallback paths exercised. + # 1. With audit-log row. + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('alpha', 'Alpha', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + db.conn().execute( + """ + INSERT INTO actions (actor_user_id, on_behalf_of, action_kind, rfc_slug, pr_number) + VALUES (NULL, 'alice', 'propose_rfc', 'alpha', 1) + """ + ) + _seed_meta_pr_directly_in_fake( + fake, slug="alpha", pr_number=1, + head_branch="propose/alpha", title="A", body="b", + gitea_opener_login="rfc-bot", + ) + # 2. Trailer only. + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('beta', 'Beta', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + _seed_meta_pr_directly_in_fake( + fake, slug="beta", pr_number=2, + head_branch="propose/beta", title="B", + body="On-behalf-of: Bob ", + gitea_opener_login="rfc-bot", + ) + # 3. Gitea opener as last resort. + db.conn().execute( + """ + INSERT INTO cached_rfcs + (slug, title, state, owners_json, arbiters_json, tags_json, body) + VALUES ('gamma', 'Gamma', 'super-draft', '[]', '[]', '[]', 'pitch') + """ + ) + _seed_meta_pr_directly_in_fake( + fake, slug="gamma", pr_number=3, + head_branch="propose/gamma", title="G", body="naked", + gitea_opener_login="carol", + ) + + import asyncio + asyncio.run(cache_mod.refresh_meta_pulls(app.state.config, app.state.gitea)) + + prs = { + r["pr_number"]: r["opened_by"] + for r in db.conn().execute( + "SELECT pr_number, opened_by FROM cached_prs ORDER BY pr_number" + ) + } + assert prs[1] == "alice" # audit log wins + assert prs[2] == "bob" # trailer wins + assert prs[3] == "carol" # raw Gitea opener wins diff --git a/backend/tests/test_e2e_smoke.py b/backend/tests/test_e2e_smoke.py new file mode 100644 index 0000000..894e532 --- /dev/null +++ b/backend/tests/test_e2e_smoke.py @@ -0,0 +1,249 @@ +"""End-to-end smoke test for the Slice 8 hardening pass. + +Walks the full user lifecycle against FakeGitea: propose → owner +merges → super-draft view → start edit branch → AI proposes (seeded +directly) → accept → open body-edit PR → owner merges → graduate → +active-RFC PR open → owner merges → §12 hygiene sweep deletes the +post-merge branch. The cases are long, and they catch the integration +seams a per-slice test would miss — that's the point per the §19.1 +brief. + +Plus the bounce-webhook signing-seam test: when +`WEBHOOK_EMAIL_BOUNCE_SECRET` is set, an unsigned POST is refused. +""" +from __future__ import annotations + +import asyncio +import json as _json +from datetime import datetime, timedelta, timezone + +import pytest + +from test_propose_vertical import ( # noqa: F401 + FakeGitea, + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) + + +# --------------------------------------------------------------------------- +# The lifecycle walk +# --------------------------------------------------------------------------- + + +def test_full_user_lifecycle_propose_through_hygiene(app_with_fake_gitea): + """Propose → merge → super-draft view → edit branch → accept change + → body-edit PR → merge → graduate → active-RFC PR → merge → + §12 hygiene sweep cleans the post-merge branch. + + The §15 notification path runs through `bot._log`'s fan_out at + every step; we assert at the end that the inbox has rows.""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db, hygiene + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + provision_user_row(user_id=2, login="alice", role="contributor") + + # --- 1. Alice proposes a new RFC. --- + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor", email="alice@test") + r = client.post("/api/rfcs/propose", json={ + "title": "Open Human Model", + "slug": "ohm", + "pitch": "A shared definition of what we mean by *human*.", + "tags": ["identity"], + }) + assert r.status_code == 200, r.text + proposal_pr = r.json()["pr_number"] + + # --- 2. Ben (owner) merges the proposal → super-draft exists. --- + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + r = client.post(f"/api/proposals/{proposal_pr}/merge") + assert r.status_code == 200, r.text + + d = client.get("/api/rfcs/ohm").json() + assert d["state"] == "super-draft" + + # --- 3. Ben claims ownership so he can graduate later. --- + r = client.post("/api/rfcs/ohm/claim") + assert r.status_code == 200, r.text + claim_pr = r.json()["pr_number"] + # Claim PR also auto-merges per §13.1's owner/admin path. + r = client.post(f"/api/rfcs/ohm/prs/{claim_pr}/merge") + assert r.status_code == 200, r.text + + # --- 4. Ben starts an edit branch on the super-draft. --- + r = client.post("/api/rfcs/ohm/start-edit-branch", json={}) + assert r.status_code == 200, r.text + edit_branch = r.json()["branch_name"] + assert edit_branch.startswith("edit-ohm-") + + # --- 5. Materialize an AI-style change directly + accept. --- + view = client.get(f"/api/rfcs/ohm/branches/{edit_branch}").json() + thread_id = view["main_thread_id"] + cur = db.conn().execute( + """ + INSERT INTO changes (rfc_slug, branch_name, thread_id, kind, state, + original, proposed, reason) + VALUES ('ohm', ?, ?, 'ai', 'pending', ?, ?, 'tighten') + """, + ( + edit_branch, thread_id, + "A shared definition of what we mean by *human*.", + "A shared, OHM-compatible definition of what we mean by *human*.", + ), + ) + change_id = cur.lastrowid + r = client.post( + f"/api/rfcs/ohm/branches/{edit_branch}/changes/{change_id}/accept", + json={ + "proposed": "A shared, OHM-compatible definition of what we mean by *human*.", + "was_edited_before_accept": False, + }, + ) + assert r.status_code == 200, r.text + + # --- 6. Open the body-edit PR + merge it. --- + r = client.post( + f"/api/rfcs/ohm/branches/{edit_branch}/open-pr", + json={"title": "OHM body edit", "description": "Add OHM-compatibility clause."}, + ) + assert r.status_code == 200, r.text + body_pr = r.json()["pr_number"] + r = client.post(f"/api/rfcs/ohm/prs/{body_pr}/merge") + assert r.status_code == 200, r.text + + # --- 7. Graduate the super-draft. --- + r = client.post( + "/api/rfcs/ohm/graduate?_sync=1", + json={"rfc_id": "RFC-0001", "repo_name": "rfc-0001-ohm", + "owners": ["ben"]}, + ) + assert r.status_code == 200, r.text + assert r.json()["succeeded"] is True + d = client.get("/api/rfcs/ohm").json() + assert d["state"] == "active" + assert d["repo"] == "wiggleverse/rfc-0001-ohm" + + # --- 8. Alice opens a PR on the now-active RFC's per-RFC repo. --- + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor", email="alice@test") + r = client.post("/api/rfcs/ohm/branches/main/promote-to-branch", json={}) + assert r.status_code == 200, r.text + active_branch = r.json()["branch_name"] + # Materialize and accept a change so the branch has commits ahead. + view = client.get(f"/api/rfcs/ohm/branches/{active_branch}").json() + active_thread = view["main_thread_id"] + cur = db.conn().execute( + """ + INSERT INTO changes (rfc_slug, branch_name, thread_id, kind, state, + original, proposed, reason) + VALUES ('ohm', ?, ?, 'ai', 'pending', ?, ?, 'expand') + """, + ( + active_branch, active_thread, + "OHM-compatible definition", + "OHM-compatible, traceable definition", + ), + ) + change_id = cur.lastrowid + r = client.post( + f"/api/rfcs/ohm/branches/{active_branch}/changes/{change_id}/accept", + json={"proposed": "OHM-compatible, traceable definition", + "was_edited_before_accept": False}, + ) + assert r.status_code == 200, r.text + r = client.post( + f"/api/rfcs/ohm/branches/{active_branch}/open-pr", + json={"title": "Traceability clause", "description": "Add a traceability term."}, + ) + assert r.status_code == 200, r.text + active_pr = r.json()["pr_number"] + + # --- 9. Ben merges the active-RFC PR. --- + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + r = client.post(f"/api/rfcs/ohm/prs/{active_pr}/merge") + assert r.status_code == 200, r.text + + # --- 10. Notifications fanned out — Ben's inbox carries rows. --- + r = client.get("/api/notifications") + assert r.status_code == 200, r.text + inbox = r.json() + assert "items" in inbox + # The merge of Alice's PR should at minimum have produced a + # structural beat to watchers (Ben auto-watched on his earlier + # gestures on the slug). + kinds = {item["event_kind"] for item in inbox["items"]} + assert kinds, f"expected non-empty inbox kinds, got: {inbox}" + + # --- 11. Backdate the merge so the §12 hygiene sweep deletes + # the branch, then run the sweep. --- + long_ago = (datetime.now(timezone.utc) - timedelta(days=120)).strftime("%Y-%m-%d %H:%M:%S") + db.conn().execute( + "UPDATE cached_prs SET merged_at = ? WHERE pr_number = ?", + (long_ago, active_pr), + ) + counters = asyncio.new_event_loop().run_until_complete( + hygiene.run_tick(config=app.state.config, bot=app.state.bot) + ) + assert counters["deleted_post_merge"] >= 1, counters + + # The branch is gone from FakeGitea + cached row flipped. + assert active_branch not in fake.branches[("wiggleverse", "rfc-0001-ohm")] + cached = db.conn().execute( + "SELECT state FROM cached_branches WHERE rfc_slug = 'ohm' AND branch_name = ?", + (active_branch,), + ).fetchone() + assert cached["state"] == "deleted" + + +# --------------------------------------------------------------------------- +# Bounce-webhook signing seam (§19.2 → settled) +# --------------------------------------------------------------------------- + + +def test_bounce_webhook_refuses_unsigned_when_secret_configured(app_with_fake_gitea, monkeypatch): + """When `WEBHOOK_EMAIL_BOUNCE_SECRET` is set, the webhook requires + the same value in the `X-Webhook-Secret` header. An unsigned POST + returns 401.""" + from fastapi.testclient import TestClient + + monkeypatch.setenv("WEBHOOK_EMAIL_BOUNCE_SECRET", "shhh") + app, _ = app_with_fake_gitea + with TestClient(app) as client: + r = client.post( + "/api/webhooks/email-bounce", + json={"email": "stranger@example.com", "kind": "hard"}, + ) + assert r.status_code == 401, r.text + + # With the right header, the call passes the guard. (No matching + # user exists, so we get {matched: False} — that's the v1 contract.) + r = client.post( + "/api/webhooks/email-bounce", + json={"email": "stranger@example.com", "kind": "hard"}, + headers={"X-Webhook-Secret": "shhh"}, + ) + assert r.status_code == 200, r.text + assert r.json() == {"ok": True, "matched": False} + + +def test_bounce_webhook_open_when_secret_unset(app_with_fake_gitea): + """The v1 contract: when no `WEBHOOK_EMAIL_BOUNCE_SECRET` is set, + the webhook stays unauthenticated for dev. The SMTP provider's + callback URL is the only contract.""" + from fastapi.testclient import TestClient + + app, _ = app_with_fake_gitea + with TestClient(app) as client: + r = client.post( + "/api/webhooks/email-bounce", + json={"email": "nobody@example.com", "kind": "complaint"}, + ) + assert r.status_code == 200, r.text diff --git a/backend/tests/test_hygiene_vertical.py b/backend/tests/test_hygiene_vertical.py index 73a69ab..2ce680e 100644 --- a/backend/tests/test_hygiene_vertical.py +++ b/backend/tests/test_hygiene_vertical.py @@ -272,12 +272,31 @@ def test_branch_chat_seen_survives_branch_deletion(app_with_fake_gitea): """, (long_ago, long_ago), ) - # Seed a per-user seen cursor against the doomed branch. + # Seed a chat thread + message on the doomed branch, then the + # per-user seen cursor pointing at the message. The FK on + # branch_chat_seen.last_seen_message_id requires a real + # thread_messages row. + cur = db.conn().execute( + """ + INSERT INTO threads (rfc_slug, branch_name, anchor_kind, thread_kind, created_by) + VALUES ('ohm', 'doomed', 'whole-doc', 'chat', 2) + """ + ) + thread_id = cur.lastrowid + cur = db.conn().execute( + """ + INSERT INTO thread_messages (thread_id, role, author_user_id, text) + VALUES (?, 'user', 2, 'witness this') + """, + (thread_id,), + ) + message_id = cur.lastrowid db.conn().execute( """ INSERT INTO branch_chat_seen (user_id, rfc_slug, branch_name, last_seen_message_id, seen_at) - VALUES (2, 'ohm', 'doomed', 999, datetime('now')) - """ + VALUES (2, 'ohm', 'doomed', ?, datetime('now')) + """, + (message_id,), ) _aiorun(hygiene.run_tick(config=app.state.config, bot=app.state.bot)) @@ -292,7 +311,12 @@ def test_branch_chat_seen_survives_branch_deletion(app_with_fake_gitea): "SELECT last_seen_message_id FROM branch_chat_seen WHERE user_id = 2 AND branch_name = 'doomed'" ).fetchone() assert seen is not None - assert seen["last_seen_message_id"] == 999 + assert seen["last_seen_message_id"] == message_id + # And the chat message itself survives — app-canonical, not cached. + msg = db.conn().execute( + "SELECT text FROM thread_messages WHERE id = ?", (message_id,) + ).fetchone() + assert msg["text"] == "witness this" def test_hygiene_action_kinds_fire_no_notifications(app_with_fake_gitea): diff --git a/backend/tests/test_metadata_pr_merge.py b/backend/tests/test_metadata_pr_merge.py new file mode 100644 index 0000000..5258093 --- /dev/null +++ b/backend/tests/test_metadata_pr_merge.py @@ -0,0 +1,149 @@ +"""End-to-end integration test for the §19.2 "in-app merge for +metadata PRs" candidate that Slice 8 settles. + +Slice 4 lands the §9.5 metadata pane that opens a `meta_metadata` PR +(title/tags edit) on the meta repo. The Slice 4 build deferred the +merge surface to Gitea web for v1 — `api_prs.merge_pr` was scoped to +body-changing PRs (`rfc_branch` and `meta_body_edit`). Slice 8 extends +`_require_pr` to include `meta_metadata` so the merge gesture lands +in-app. The diff-rendered review surface degrades gracefully — a +metadata PR doesn't have a body diff worth reviewing — but the merge +button works. + +The tests prove: + + * `POST /api/rfcs//prs//merge` accepts a metadata PR and + runs the underlying merge. + * After the merge, the meta entry's title/tags carry forward and + the cache reflects the new values. + * A contributor (no role on the super-draft) is refused. + * Withdraw also works against a metadata PR — the same surface + supports the §10.8 withdraw gesture. +""" +from __future__ import annotations + +import pytest + +from test_propose_vertical import ( # noqa: F401 + FakeGitea, + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) +from test_super_draft_vertical import seed_super_draft # noqa: F401 + + +PITCH = ( + "Open Human Model is a framework for representing humans.\n\n" + "It defines consent, trait, and agency in compatible terms." +) + + +def test_metadata_pr_merges_in_app(app_with_fake_gitea): + """The headline assertion: an owner can hit the same + `prs//merge` endpoint for a metadata PR and the change lands.""" + from fastapi.testclient import TestClient + from app import db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_super_draft(fake, slug="ohm", title="OHM", pitch=PITCH) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + + # Open the metadata PR via the §9.5 endpoint. + r = client.post( + "/api/rfcs/ohm/metadata", + json={"title": "Open Human Model", "tags": ["identity", "schema"]}, + ) + assert r.status_code == 200, r.text + pr_number = r.json()["pr_number"] + + # Verify the kind landed as meta_metadata. + row = db.conn().execute( + "SELECT pr_kind FROM cached_prs WHERE pr_number = ?", (pr_number,) + ).fetchone() + assert row["pr_kind"] == "meta_metadata" + + # Merge via the §10.5 endpoint — the Slice 8 extension. + r = client.post(f"/api/rfcs/ohm/prs/{pr_number}/merge") + assert r.status_code == 200, r.text + + # The cached entry now carries the new title + tags. + cached = db.conn().execute( + "SELECT title, tags_json FROM cached_rfcs WHERE slug = 'ohm'" + ).fetchone() + import json as _json + tags = _json.loads(cached["tags_json"]) + assert cached["title"] == "Open Human Model" + assert "identity" in tags and "schema" in tags + + # PR row's state is now 'merged'. + post = db.conn().execute( + "SELECT state FROM cached_prs WHERE pr_number = ?", (pr_number,) + ).fetchone() + assert post["state"] == "merged" + + +def test_metadata_pr_merge_refused_for_plain_contributor(app_with_fake_gitea): + """§6.1 + §6.3: only owners/arbiters/admins can merge. + + A plain contributor without any per-RFC authority gets 403, same + as the existing body-edit PR merge surface. Confirms the + extension didn't widen the permission gate.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + provision_user_row(user_id=2, login="alice", role="contributor") + seed_super_draft(fake, slug="ohm", title="OHM", pitch=PITCH) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + + # Ben opens the metadata PR. + r = client.post( + "/api/rfcs/ohm/metadata", + json={"title": "OHM (revised)", "tags": ["identity"]}, + ) + pr_number = r.json()["pr_number"] + + # Alice (plain contributor) tries to merge — 403. + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + r = client.post(f"/api/rfcs/ohm/prs/{pr_number}/merge") + assert r.status_code == 403 + + +def test_metadata_pr_withdraw_works(app_with_fake_gitea): + """§10.8 withdraw surface also handles meta_metadata PRs uniformly + — the API doesn't care which kind.""" + from fastapi.testclient import TestClient + from app import db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_super_draft(fake, slug="ohm", title="OHM", pitch=PITCH) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + + r = client.post( + "/api/rfcs/ohm/metadata", + json={"title": "Something else", "tags": []}, + ) + pr_number = r.json()["pr_number"] + + r = client.post(f"/api/rfcs/ohm/prs/{pr_number}/withdraw") + assert r.status_code == 200, r.text + + post = db.conn().execute( + "SELECT state FROM cached_prs WHERE pr_number = ?", (pr_number,) + ).fetchone() + # Withdraw flips to 'withdrawn' via the audit-log marker the + # reconciler reads, but a direct withdraw via api_prs may + # leave it 'closed' depending on the refresh path. Either is + # the closed-not-merged shape the surface needs. + assert post["state"] in ("withdrawn", "closed") diff --git a/deploy/DEPLOY.md b/deploy/DEPLOY.md deleted file mode 100644 index ad7fb9e..0000000 --- a/deploy/DEPLOY.md +++ /dev/null @@ -1,284 +0,0 @@ -# Deployment runbook - -Single-host deployment of the RFC app at `rfc.wiggleverse.org`, -sharing infrastructure with `git.wiggleverse.org` (same Gitea -instance, same nginx, same Let's Encrypt). The shape matches §4.2: -one process, one SQLite file, no separate worker. - -Bring-up order: host prep → Gitea side (bot, OAuth, meta repo) → app -side (code, venv, build, .env) → web server side (nginx, certbot) → -systemd → smoke-test. - -Every step is idempotent or no-op-on-rerun, so re-running this -runbook to recover from a partial install is safe. - ---- - -## Prereqs - -- Ubuntu/Debian-style host with nginx and certbot already serving - `git.wiggleverse.org` over HTTPS. -- DNS: an `A` record for `rfc.wiggleverse.org` pointing at the same - IP as `git.wiggleverse.org`. -- Python 3.11+ available system-wide. Node 20+ available (for - `npm run build` once; the build output is what runs in production — - Node isn't needed at runtime). - ---- - -## 1. Host prep - -Create the system user and the install directory. - -```sh -sudo useradd --system --shell /usr/sbin/nologin --home-dir /opt/rfc-app rfc-app -sudo mkdir -p /opt/rfc-app -sudo chown rfc-app:rfc-app /opt/rfc-app -``` - -Clone the repo. Use HTTPS since we don't need to push from the server. - -```sh -sudo -u rfc-app git clone https://git.wiggleverse.org/ben.stull/rfc-app.git /opt/rfc-app -``` - ---- - -## 2. Gitea side - -### 2.1 Create the bot service account - -In the Gitea web UI, signed in as a Gitea admin: - -- **Site Administration → User Accounts → Create User Account** -- Username: `rfc-bot` (or whatever you want) -- Email: anything sensible (e.g. `rfc-bot@wiggleverse.org`) -- Password: random, you won't use it interactively -- Send email confirmation: off - -Then sign in as the bot, open **Settings → Applications → Generate -New Token**, name it `rfc-app`, grant scopes: - -- `write:repository` -- `write:user` -- `write:admin` (needed because the bot creates per-RFC repos on - graduation — scope down to `repo`+`org` if you want to defer admin - until Slice 5) - -Copy the token. It goes into `.env` as `GITEA_BOT_TOKEN`. - -### 2.2 Create the org and add the bot - -The meta repo lives inside an org. In Gitea: **Create Organization → -wiggleverse**. Then **Members → Invite → rfc-bot → Owner**. - -### 2.3 Register the OAuth2 application - -**Site Administration → Integrations → OAuth2 Applications → -Create Application**: - -- Name: `RFC App` -- Redirect URI: `https://rfc.wiggleverse.org/auth/callback` - -Copy the client ID and client secret. They go into `.env`. - ---- - -## 3. App side - -### 3.1 Python venv + deps - -```sh -sudo -u rfc-app python3 -m venv /opt/rfc-app/backend/.venv -sudo -u rfc-app /opt/rfc-app/backend/.venv/bin/pip install \ - -r /opt/rfc-app/backend/requirements.txt -``` - -### 3.2 Build the frontend - -Build locally if you don't want Node on the server; copy `dist/` -across: - -```sh -# On your laptop: -cd frontend && npm install && npm run build -rsync -a dist/ ben.stull@:/tmp/rfc-app-dist/ -# On the host: -sudo -u rfc-app mkdir -p /opt/rfc-app/frontend/dist -sudo cp -r /tmp/rfc-app-dist/. /opt/rfc-app/frontend/dist/ -sudo chown -R rfc-app:rfc-app /opt/rfc-app/frontend/dist -``` - -Or build on the host directly if Node is available: - -```sh -cd /opt/rfc-app/frontend && sudo -u rfc-app npm install -sudo -u rfc-app npm run build -``` - -### 3.3 Write `.env` - -```sh -sudo -u rfc-app cp /opt/rfc-app/backend/.env.example /opt/rfc-app/backend/.env -sudoedit /opt/rfc-app/backend/.env # set every value -``` - -Required values for production: - -```ini -GITEA_URL=https://git.wiggleverse.org -GITEA_BOT_USER=rfc-bot -GITEA_BOT_TOKEN= -GITEA_ORG=wiggleverse -META_REPO=meta - -OAUTH_CLIENT_ID= -OAUTH_CLIENT_SECRET= - -APP_URL=https://rfc.wiggleverse.org -SECRET_KEY=<32+ random chars; generate with `openssl rand -hex 32`> -OWNER_GITEA_LOGIN=ben.stull -GITEA_WEBHOOK_SECRET= - -DATABASE_PATH=/opt/rfc-app/backend/data/rfc-app.db - -# Optional — chat is disabled until at least one is set: -ENABLED_MODELS=claude -ANTHROPIC_API_KEY= -GOOGLE_API_KEY= -OPENAI_API_KEY= -``` - -Lock the file down — it carries secrets: - -```sh -sudo chmod 600 /opt/rfc-app/backend/.env -sudo chown rfc-app:rfc-app /opt/rfc-app/backend/.env -``` - -### 3.4 Seed the meta repo - -This creates `wiggleverse/meta` on Gitea, populates the hand-authored -files, and registers the webhook against `APP_URL/api/webhooks/gitea`. - -```sh -sudo -u rfc-app -H bash -c \ - 'cd /opt/rfc-app/backend && .venv/bin/python ../scripts/seed_meta_repo.py' -``` - -Re-running is safe; every step is upsert-shaped. - ---- - -## 4. Web server side - -### 4.1 nginx vhost - -```sh -sudo cp /opt/rfc-app/deploy/nginx/rfc.wiggleverse.org.conf \ - /etc/nginx/sites-available/rfc.wiggleverse.org -sudo ln -s /etc/nginx/sites-available/rfc.wiggleverse.org \ - /etc/nginx/sites-enabled/ -sudo nginx -t && sudo systemctl reload nginx -``` - -Make sure the nginx user can read `/opt/rfc-app/frontend/dist`. The -simplest path: add the nginx user (usually `www-data`) to the -`rfc-app` group and chmod the dist tree group-readable: - -```sh -sudo usermod -a -G rfc-app www-data -sudo chmod -R g+rX /opt/rfc-app/frontend/dist -sudo systemctl reload nginx -``` - -### 4.2 Let's Encrypt cert - -```sh -sudo certbot --nginx -d rfc.wiggleverse.org -``` - -Certbot rewrites the vhost in place to add the 443 listener and -certificate directives. After it finishes, `https://rfc.wiggleverse.org` -serves but the backend isn't up yet — the next step starts it. - ---- - -## 5. systemd - -```sh -sudo cp /opt/rfc-app/deploy/systemd/rfc-app.service /etc/systemd/system/ -sudo systemctl daemon-reload -sudo systemctl enable --now rfc-app -sudo systemctl status rfc-app -``` - -Watch the logs: - -```sh -sudo journalctl -u rfc-app -f -``` - -Expected startup line: `RFC app started — meta repo wiggleverse/meta`. - ---- - -## 6. Smoke test - -In a browser at `https://rfc.wiggleverse.org`: - -1. The landing page renders. (Slice 1 placeholder; Slice 7 polishes.) -2. Click **Sign in with Gitea** → OAuth round-trip → you land on the - catalog. (Empty on first visit — no RFCs yet.) -3. Click **+ Propose New RFC**, fill in title/slug/pitch, submit. - The pending-ideas disclosure shows the new PR. -4. Click the PR row, then **Merge proposal**. The catalog refreshes - with the super-draft. -5. Optional: seed an active RFC for §8 testing (see - [README.md](../README.md#seeding-an-active-rfc-for-8-testing)). - -If anything misfires, the troubleshooting section below covers the -common failure modes. - ---- - -## 7. Updating after a push - -```sh -sudo -u rfc-app git -C /opt/rfc-app pull -sudo -u rfc-app /opt/rfc-app/backend/.venv/bin/pip install \ - -r /opt/rfc-app/backend/requirements.txt -# Rebuild the frontend locally and rsync dist/ as in step 3.2. -sudo systemctl restart rfc-app -``` - -The §5 schema migrations run on startup and are append-only, so a -restart is the entire deploy. No reverse-migration story for now — -add one when the schema starts breaking back-compat (deferred until -Slice 8 hardening). - ---- - -## Troubleshooting - -- **`systemctl status rfc-app` shows `RuntimeError: Required - environment variable ... is not set`.** The `.env` is missing a - value, or `EnvironmentFile=` in the systemd unit isn't finding it. - Confirm `/opt/rfc-app/backend/.env` exists and is mode 0600 owned - by `rfc-app`. -- **OAuth callback returns "Invalid state".** The redirect URI in - Gitea must match `APP_URL/auth/callback` exactly. Confirm it's - `https://rfc.wiggleverse.org/auth/callback`. -- **The catalog stays empty after a merge.** Check the webhook: - `journalctl -u rfc-app | grep webhook`. Gitea's - **Settings → Webhooks → Recent Deliveries** on the meta repo shows - the delivery status; the reconciler will catch up within 5 minutes - anyway. -- **`502 Bad Gateway` on /api/* or /auth/*.** uvicorn isn't running - or isn't bound to `127.0.0.1:8000`. `systemctl status rfc-app`. -- **`403` from nginx on static assets.** The nginx user can't read - `/opt/rfc-app/frontend/dist`. Apply the chmod from step 4.1. -- **OAuth works, but the user can't propose.** The `users` row was - created with role `contributor`; only `OWNER_GITEA_LOGIN`'s login - gets `owner` on first sign-in. Confirm `.env` has the right value - and you signed in with that account. diff --git a/deploy/RUNBOOK.md b/deploy/RUNBOOK.md new file mode 100644 index 0000000..ea6ed62 --- /dev/null +++ b/deploy/RUNBOOK.md @@ -0,0 +1,408 @@ +# Runbook + +Single-host deployment of the RFC app at `rfc.wiggleverse.org`, sharing +infrastructure with `git.wiggleverse.org` (same Gitea instance, same nginx, +same Let's Encrypt). The shape matches §4.2: one process, one SQLite file, +no separate worker. + +Bring-up order: host prep → Gitea side (bot, OAuth, meta repo) → app side +(code, venv, build, .env) → web server side (nginx, certbot) → systemd → +smoke test. + +Every step is idempotent or no-op-on-rerun, so re-running this runbook to +recover from a partial install is safe. + +--- + +## 0. Prerequisites + +- Ubuntu/Debian-style host with nginx and certbot already serving + `git.wiggleverse.org` over HTTPS. +- DNS: an `A` record for `rfc.wiggleverse.org` pointing at the same IP as + `git.wiggleverse.org`. +- Python 3.13 available system-wide. Node 20+ available (for `npm run + build` once; the build output is what runs in production — Node isn't + needed at runtime). +- `git`, `openssl`, and `rsync` on the host. + +--- + +## 1. First-time bring-up + +### 1.1 Host prep + +Create the system user and the install directory. + +```sh +sudo useradd --system --shell /usr/sbin/nologin --home-dir /opt/rfc-app rfc-app +sudo mkdir -p /opt/rfc-app +sudo chown rfc-app:rfc-app /opt/rfc-app +``` + +Clone the repo. HTTPS, since we don't push from the server. + +```sh +sudo -u rfc-app git clone https://git.wiggleverse.org/ben.stull/rfc-app.git /opt/rfc-app +``` + +### 1.2 Gitea side + +**1.2.1 Create the bot service account.** In the Gitea web UI, signed in +as a Gitea admin: + +- **Site Administration → User Accounts → Create User Account** +- Username: `rfc-bot` (or whatever you want) +- Email: anything sensible (e.g. `rfc-bot@wiggleverse.org`) +- Password: random, you won't use it interactively +- Send email confirmation: off + +Then sign in as the bot, open **Settings → Applications → Generate New +Token**, name it `rfc-app`, grant scopes: + +- `write:repository` +- `write:user` +- `write:admin` (needed because the bot creates per-RFC repos on + graduation and deletes branches per §12 hygiene) + +Copy the token. It goes into `.env` as `GITEA_BOT_TOKEN`. + +**1.2.2 Create the org and add the bot.** The meta repo lives inside an +org. In Gitea: **Create Organization → wiggleverse**. Then **Members → +Invite → rfc-bot → Owner**. + +**1.2.3 Register the OAuth2 application.** **Site Administration → +Integrations → OAuth2 Applications → Create Application**: + +- Name: `RFC App` +- Redirect URI: `https://rfc.wiggleverse.org/auth/callback` + +Copy the client ID and client secret. They go into `.env`. + +### 1.3 App side + +**1.3.1 Python venv + deps.** + +```sh +sudo -u rfc-app python3 -m venv /opt/rfc-app/backend/.venv +sudo -u rfc-app /opt/rfc-app/backend/.venv/bin/pip install \ + -r /opt/rfc-app/backend/requirements.txt +``` + +**1.3.2 Build the frontend.** Build locally and copy `dist/` across: + +```sh +# On your laptop: +cd frontend && npm install && npm run build +rsync -a dist/ ben.stull@:/tmp/rfc-app-dist/ +# On the host: +sudo -u rfc-app mkdir -p /opt/rfc-app/frontend/dist +sudo cp -r /tmp/rfc-app-dist/. /opt/rfc-app/frontend/dist/ +sudo chown -R rfc-app:rfc-app /opt/rfc-app/frontend/dist +``` + +Or build on the host directly if Node is installed there: + +```sh +cd /opt/rfc-app/frontend && sudo -u rfc-app npm install +sudo -u rfc-app npm run build +``` + +**1.3.3 Write `.env`.** + +```sh +sudo -u rfc-app cp /opt/rfc-app/backend/.env.example /opt/rfc-app/backend/.env +sudoedit /opt/rfc-app/backend/.env # set every value +``` + +Required values for production (see `.env.example` for the comments on +each field): + +```ini +GITEA_URL=https://git.wiggleverse.org +GITEA_BOT_USER=rfc-bot +GITEA_BOT_TOKEN= +GITEA_ORG=wiggleverse +META_REPO=meta + +OAUTH_CLIENT_ID= +OAUTH_CLIENT_SECRET= + +APP_URL=https://rfc.wiggleverse.org +SECRET_KEY= +OWNER_GITEA_LOGIN=ben.stull +GITEA_WEBHOOK_SECRET= + +DATABASE_PATH=/opt/rfc-app/backend/data/rfc-app.db +``` + +For the §15.4 email loop, either leave `SMTP_HOST` unset (stdout +fallback — fine for the very first deploy while a provider is being +chosen) or fill in the SMTP block: + +```ini +SMTP_HOST=smtp.postmarkapp.com +SMTP_PORT=587 +SMTP_USER= +SMTP_PASSWORD= +SMTP_STARTTLS=1 +EMAIL_FROM=notifications@wiggleverse.org +EMAIL_FROM_NAME=Wiggleverse +``` + +Configure SPF and DKIM records for `wiggleverse.org` with the chosen +provider before sending real traffic. The single non-spoofing envelope +identity per §15.9 is what every outbound email uses; spoofing the +actor's address would land everything in spam. + +If a real bounce/complaint webhook lands later, set +`WEBHOOK_EMAIL_BOUNCE_SECRET` to a long random string and configure the +provider's webhook to inject it as `X-Webhook-Secret`. + +Lock the file down — it carries secrets: + +```sh +sudo chmod 600 /opt/rfc-app/backend/.env +sudo chown rfc-app:rfc-app /opt/rfc-app/backend/.env +``` + +**1.3.4 Seed the meta repo.** This creates `wiggleverse/meta` on Gitea, +populates the hand-authored files, and registers the webhook against +`APP_URL/api/webhooks/gitea`. + +```sh +sudo -u rfc-app -H bash -c \ + 'cd /opt/rfc-app/backend && .venv/bin/python ../scripts/seed_meta_repo.py' +``` + +Re-running is safe; every step is upsert-shaped. + +### 1.4 Web server side + +**1.4.1 nginx vhost.** + +```sh +sudo cp /opt/rfc-app/deploy/nginx/rfc.wiggleverse.org.conf \ + /etc/nginx/sites-available/rfc.wiggleverse.org +sudo ln -s /etc/nginx/sites-available/rfc.wiggleverse.org \ + /etc/nginx/sites-enabled/ +sudo nginx -t && sudo systemctl reload nginx +``` + +Make the nginx user able to read `/opt/rfc-app/frontend/dist`: + +```sh +sudo usermod -a -G rfc-app www-data +sudo chmod -R g+rX /opt/rfc-app/frontend/dist +sudo systemctl reload nginx +``` + +**1.4.2 Let's Encrypt cert.** + +```sh +sudo certbot --nginx -d rfc.wiggleverse.org +``` + +### 1.5 systemd + +```sh +sudo cp /opt/rfc-app/deploy/systemd/rfc-app.service /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable --now rfc-app +sudo systemctl status rfc-app +``` + +Watch the logs: + +```sh +sudo journalctl -u rfc-app -f +``` + +Expected startup line: + +``` +RFC app started — meta repo wiggleverse/meta +``` + +### 1.6 Smoke test + +In a browser at `https://rfc.wiggleverse.org`: + +1. The landing page renders (§14.1 — title, pitch, three-item deck, + sign-in affordance). +2. Click **Sign in with Gitea** → OAuth round-trip → catalog lands. +3. Click **+ Propose New RFC**, fill in title/slug/pitch, submit. + The pending-ideas disclosure shows the new PR. +4. As the owner, click the PR row, then **Merge proposal**. The + catalog refreshes with the super-draft. +5. Open `/admin` and confirm the four-tab home base loads + (Users / Graduation queue / Audit log / Permission events). +6. Open `/settings/notifications` and confirm the five sub-sections + render (per-category email, digest cadence, quiet hours, watches, + mute list). + +If anything misfires, the troubleshooting section below covers the +common failure modes. + +--- + +## 2. Day-2 operations + +### 2.1 Logs + +```sh +sudo journalctl -u rfc-app -f # follow +sudo journalctl -u rfc-app --since "1 hour ago" +sudo journalctl -u rfc-app -p err # errors only +``` + +The app logs at INFO level by default. Notable log lines to watch: + +- `RFC app started — meta repo ...` — startup completed. +- `reconciler: starting sweep` / `reconciler: sweep complete` — the + five-minute §4.1 safety-net pass. +- `digest tick failed` / `hygiene tick failed` — a scheduler tick + crashed; the next tick will retry but the underlying error wants a + look. Stack trace lands next to the warning. +- `email (stdout fallback): to=...` — `SMTP_HOST` is unset and the + email loop is logging envelopes instead of sending. + +### 2.2 Database backup + +The SQLite file at `DATABASE_PATH` carries every app-canonical row +(users, threads, messages, watches, notifications, the audit log). The +§4 cache rebuilds from Gitea, so an empty backup of the cached_* tables +is recoverable — but the app-canonical tables aren't, so a backup is +load-bearing. + +Daily snapshot (cron, as `rfc-app`): + +```sh +0 3 * * * sqlite3 /opt/rfc-app/backend/data/rfc-app.db ".backup /opt/rfc-app/backend/data/backup-$(date +\%F).db" +``` + +Retention is your call; 30 daily snapshots is the easy default. + +Restore a snapshot: + +```sh +sudo systemctl stop rfc-app +sudo -u rfc-app cp /opt/rfc-app/backend/data/backup-YYYY-MM-DD.db \ + /opt/rfc-app/backend/data/rfc-app.db +sudo systemctl start rfc-app +``` + +The reconciler will refill the cache from Gitea on first sweep. + +### 2.3 Secret rotation + +`SECRET_KEY` invalidates every active session cookie. To rotate: + +```sh +NEW=$(openssl rand -hex 32) +sudoedit /opt/rfc-app/backend/.env # SECRET_KEY=$NEW +sudo systemctl restart rfc-app +``` + +Every signed-in user is bounced to the landing page and re-authenticates +through OAuth. Existing email-unsubscribe URLs become invalid (per +§15.4 they're signed against `SECRET_KEY`); a user can still unsubscribe +through `/settings/notifications`. + +`GITEA_BOT_TOKEN` rotates without service disruption — write the new +value, restart. Old tokens stay valid in Gitea until revoked there. + +`GITEA_WEBHOOK_SECRET` rotates in two steps: update the value in `.env`, +restart, then update the secret in Gitea's webhook config to match. A +brief window where webhooks are refused; the reconciler covers it. + +### 2.4 The §12 hygiene timer cadence + +The hygiene scheduler runs every `HYGIENE_TICK_SECONDS` (default 3600). +Each tick checks `cached_branches` for two boundaries: + +- **30 days idle** (no commits, no PR) — the branch flips to + `state='closed'`. The branch stays in Gitea, but new chat is + disabled per §8.4. +- **90 days closed** (or 90 days post-merge for a merged-PR branch) — + the bot deletes the branch from Gitea. The `cached_branches.state` + flips to `deleted`, and the audit log records the action with + `actor_user_id=NULL` and `on_behalf_of=` per §15.9. + +Pinned branches (`cached_branches.pinned=1`) skip both passes. Per-user +`branch_chat_seen` cursors survive branch deletion — chat history is +app-canonical, not cached, and persists indefinitely. + +If a branch needs to be kept alive past 30 days without commits, pin +it from the admin surface (or directly: `UPDATE cached_branches SET +pinned = 1 WHERE rfc_slug = ? AND branch_name = ?`). + +### 2.5 Updating after a push + +```sh +sudo -u rfc-app git -C /opt/rfc-app pull +sudo -u rfc-app /opt/rfc-app/backend/.venv/bin/pip install \ + -r /opt/rfc-app/backend/requirements.txt +# Rebuild the frontend locally and rsync dist/ as in 1.3.2. +sudo systemctl restart rfc-app +``` + +The §5 schema migrations run on startup and are append-only. A restart +is the entire deploy. + +--- + +## 3. Rollback + +If a deploy goes sideways, the rollback shape is: + +```sh +sudo -u rfc-app git -C /opt/rfc-app log --oneline -10 +sudo -u rfc-app git -C /opt/rfc-app checkout +sudo -u rfc-app /opt/rfc-app/backend/.venv/bin/pip install \ + -r /opt/rfc-app/backend/requirements.txt +# Rebuild + rsync the frontend dist from the prior commit's state. +sudo systemctl restart rfc-app +``` + +The schema migrations are append-only, so rolling code back without +rolling the schema back is the safe default. If a migration introduced +a column the new code requires, the old code ignores the extra +column — SQLite reads the rows fine. + +If the database itself got into a bad state (a botched manual UPDATE, +say), restore from the most recent backup per §2.2. + +--- + +## 4. Troubleshooting + +- **`systemctl status rfc-app` shows `RuntimeError: Required environment + variable ... is not set`.** The `.env` is missing a value, or + `EnvironmentFile=` in the systemd unit isn't finding it. Confirm + `/opt/rfc-app/backend/.env` exists and is mode 0600 owned by + `rfc-app`. +- **OAuth callback returns "Invalid state".** The redirect URI in Gitea + must match `APP_URL/auth/callback` exactly. Confirm it's + `https://rfc.wiggleverse.org/auth/callback`. +- **The catalog stays empty after a merge.** Check the webhook: + `journalctl -u rfc-app | grep webhook`. Gitea's **Settings → Webhooks + → Recent Deliveries** on the meta repo shows the delivery status; the + reconciler will catch up within 5 minutes anyway. +- **`502 Bad Gateway` on /api/\* or /auth/\*.** uvicorn isn't running + or isn't bound to `127.0.0.1:8000`. `systemctl status rfc-app`. +- **`403` from nginx on static assets.** The nginx user can't read + `/opt/rfc-app/frontend/dist`. Apply the chmod from 1.4.1. +- **OAuth works, but the user can't propose.** The `users` row was + created with role `contributor`; only `OWNER_GITEA_LOGIN`'s login + gets `owner` on first sign-in. Confirm `.env` has the right value + and you signed in with that account. +- **Email isn't going out and no error logs.** Most likely `SMTP_HOST` + is unset; the stdout fallback is in play and envelopes are in the + journal as `email (stdout fallback): ...`. Set the SMTP block per + §1.3.3 to enable real sends. +- **The §12 hygiene sweep isn't deleting an obviously stale branch.** + Confirm `cached_branches.pinned = 0` for the row, and that + `last_commit_at` (or the joined `merged_at`) actually predates the + 90-day cutoff. The `actions` audit log carries every hygiene gesture + with `action_kind IN ('close_idle_branch', 'delete_stale_branch', + 'delete_post_merge_branch')`. diff --git a/docs/DEV.md b/docs/DEV.md index e3a2617..3f33f25 100644 --- a/docs/DEV.md +++ b/docs/DEV.md @@ -288,6 +288,120 @@ adds the `email_opt_out_all` column to `users` for the bounce webhook. Topic 13 settled the rest of the §5 surface before the build started, so no further migrations were needed. +### Slice 8 — shipped + +The hardening pass — the last slice of the v1 build. §12 + §10.7 +branch hygiene, the §19.2 candidates that cluster with the hygiene +work, the dev/prod deployment shape, and the end-to-end smoke pass. + +The §12 30/90 timers live in +[`backend/app/hygiene.py`](../backend/app/hygiene.py) as a +`HygieneScheduler` that mirrors `DigestScheduler`'s shape — same +`start` / `stop` / `_loop` contract, same `HYGIENE_TICK_SECONDS` +env override (default 3600), same `run_tick(now=...)` test seam so +the integration tests compress the 30/90-day windows without +monkey-patching the clock. Each tick runs four queries in the order +"delete first, close second" — the 90-day boundary takes priority +when a single sweep crosses both, which is the cache-bootstrap and +clock-jump case the brief calls out. Order: + +1. §10.7 90-day post-merge delete (against `state IN ('open', 'closed')` + joined to a merged PR past the cutoff) +2. §12 90-day stale-closed delete (closed branches past `closed_at + + 60d` since the prior 30d close) +3. §11.5 30-day idle close (open branches with no PR past the + cutoff) +4. §10.7 30-day post-merge close (open branches with a merged PR + past the cutoff) + +The bot gains a `delete_branch` method that accepts `actor: Actor | +None`; the timer paths pass `None`, the audit row lands with +`actor_user_id=NULL` and `on_behalf_of=` per §15.9's +"system-generated events" rule — "the app" in the noun slot. The +three action kinds (`close_idle_branch`, `delete_stale_branch`, +`delete_post_merge_branch`) are intentionally absent from +`notify._AUTO_WATCH_ACTIONS` and `notify._ROUTING`, so no +notifications fire. The branches being touched are stale by +definition; the affected population would be churn-grade noise per +§15.4. Pinned branches skip both passes. Per-user `branch_chat_seen` +cursors survive branch deletion — chat history is app-canonical, not +cached. + +The §19.2 candidates the hardening pass folded in: + +- **Branch-name path routing.** Every `branches/` route in + [`api_branches.py`](../backend/app/api_branches.py) and + [`api_prs.py`](../backend/app/api_prs.py) is now `{branch:path}`. + The bare `GET /api/rfcs/{slug}/branches/{branch:path}` is declared + *last* among the branch-scoped GETs, so the deeper `threads` and + `threads/{thread_id}/messages` GETs match before the greedy path + matcher swallows their sub-paths. The literal-prefix POST + `branches/main/promote-to-branch` doesn't collide with any other + POST suffix; ordering there is incidental. +- **Cache bootstrap from a pre-existing meta repo.** Exercised + directly by `test_cache_bootstrap.py`: an audit-log-empty FakeGitea + with PRs whose `gitea_opener` is the bot, the trailer parsed from + the body, the raw login as last resort. The `_resolve_actor` + fallback chain Slice 1 introduced now has an explicit test surface + against history the bot did not author. +- **In-app merge for metadata PRs.** [`api_prs._require_pr`](../backend/app/api_prs.py) + extends to `pr_kind='meta_metadata'`. The diff-rendered review + surface degrades gracefully (a metadata PR has no body diff worth + reviewing); the merge gesture lands in-app rather than forcing the + Gitea round-trip. +- **Graduation rollback's branch cleanup.** [`api_graduation._undo_open_pr`](../backend/app/api_graduation.py) + now deletes the `graduate--<6hex>` branch after closing the + PR, so failed-graduation branches don't accumulate on the meta + repo across retries. +- **Email bounce webhook authentication seam.** [`api_notifications.email_bounce`](../backend/app/api_notifications.py) + checks `WEBHOOK_EMAIL_BOUNCE_SECRET` when set, refusing unsigned + POSTs with 401. Unset preserves the v1 unauthenticated behavior + for dev. + +The deployment shape: [`deploy/RUNBOOK.md`](../deploy/RUNBOOK.md) is +rewritten from the prior `DEPLOY.md` stub into a real runbook — +prerequisites, first-time bring-up, day-2 operations (logs, database +backup, secret rotation, the §12 hygiene cadence), rollback shape, +and a troubleshooting table. The README's `.env` table grows the +SMTP block, `HYGIENE_TICK_SECONDS`, and `WEBHOOK_EMAIL_BOUNCE_SECRET`. +[`backend/.env.example`](../backend/.env.example) carries the same +fields with inline commentary. + +Slice 8 ships covered by: + +- [`test_hygiene_vertical.py`](../backend/tests/test_hygiene_vertical.py) + — seven tests covering the 30d close, 90d delete, 90d post-merge + delete, pinned-branch exemption, per-user-cursor preservation, the + no-notification decision, and the graduation-rollback branch + cleanup. +- [`test_branch_path_routing.py`](../backend/tests/test_branch_path_routing.py) + — four tests covering the slashed-branch GET, the deeper threads + GET still routing for both slashed and unslashed branches, and a + POST against a slashed branch. +- [`test_metadata_pr_merge.py`](../backend/tests/test_metadata_pr_merge.py) + — three tests covering the in-app merge of a `meta_metadata` PR, + the contributor refusal, and the withdraw path. +- [`test_cache_bootstrap.py`](../backend/tests/test_cache_bootstrap.py) + — four tests covering the audit-log / trailer / raw-login fallback + chain in `_resolve_actor`. +- [`test_e2e_smoke.py`](../backend/tests/test_e2e_smoke.py) — three + tests: the full lifecycle walk (propose → super-draft → edit + branch → body-edit PR → graduate → active-RFC PR → merge → + notification → hygiene-sweep deletion), the bounce-webhook signing + seam refusing unsigned POSTs when the secret is set, and the + unauthenticated open path when the secret is unset. + +The full Slices 1–8 test suite is 96/96 green. The FakeGitea grew a +`DELETE /repos/{owner}/{repo}/branches/{branch:path}` handler and a +slashed-branch `GET /branches/{branch:path}` for these tests. + +No schema migrations. Two minor spec corrections — §12 grew an +explicit note that the per-user `branch_chat_seen` cursor survives +branch deletion (the §11.5 contract made implicit; running code +asked for the load-bearing line to live in §12 too), and §10.7 +grew a one-line pointer that the timer rides on §12's sweep rather +than its own schedule. + ### Slice 7 — shipped The §14 chrome, the §15 settings neighborhood, and the §6/§17 admin @@ -590,65 +704,15 @@ spec: renames — these are not shipped in any slice. They earn their own topic sessions when use surfaces evidence they matter. -## Next slice +## After v1 -**Slice 8: hardening — the last slice of the v1 build.** +v1 ships. Slice 8 was the last slice of the build. Subsequent sessions +pick from §19.2 by user choice per §19.3's working agreement — drive a +topic to decision, fold it in, update §19.2, hand off. They need not +be sequential; the user picks the next topic based on what evidence +the running app surfaces. -With Slice 7 shipped, every structural beat the spec commits to is -live and every surface the framework exposes has chrome around it. -What remains is the hardening pass that lets a single-operator -deployment actually run end-to-end without hand-holding. Three -pieces hang together: - -- **The §12 30/90 branch-hygiene timers.** §11.5 names the branch - lifecycle (open → merged → 30d read-only → 90d deleted-by-bot, - with the per-user message-cursor preservation contract); §12 - formalizes the policy. The wiring is a scheduled task next to - the existing `DigestScheduler` — same `run_tick` test-seam shape. - The §10.7 90-day deletion timer Slice 3 left explicitly deferred - lives here too. Touches `cache.Reconciler` (the natural place to - fire the hygiene sweep), `bot.delete_branch` (the §12 actuator, - not yet exercised), and the §19.2 cache-bootstrap topic if the - hygiene sweep also rebuilds branch state from Gitea after a - cache wipe. -- **An end-to-end smoke pass** over the working surfaces. Propose - → super-draft → branch → PR → merge → graduate → active-RFC PR - → notification → inbox → email — one or two `test_e2e_smoke.py` - cases that exercise the seams a per-slice test wouldn't. Plus - the §19.2 follow-ons the hardening pass is the natural place to - fold in: branch-name path routing (`{branch:path}` everywhere - with route-ordering discipline), cache bootstrap from a - pre-existing meta repo (the audit-log-first attribution shape - exercised against history the bot did not author), in-app merge - for metadata PRs, the graduation rollback's branch cleanup, and - the small Slice-2-onward follow-ons that are deferred until the - hardening pass demands them. -- **The dev/prod deployment shape.** `deploy/` already carries an - nginx vhost, a systemd unit, and a runbook stub. Slice 8 proves - the bring-up against a fresh host, settles the secret-material - handling (the existing `.env.example` plus the §15.4 SMTP - wiring), wires the §6 / §15.4 SMTP credentials, and lands the - README updates that take a new operator from `git clone` to a - signed-in browser. - -What Slice 8 does NOT own: - -- New surfaces. The v1 surface is complete; the hardening pass is - about making what's there resilient, observable, and operable. -- The §16 deferred items. -- The §19.2 candidate set as a whole — the hardening pass folds in - the candidates that naturally cluster with hygiene timers, cache - rebuild, and deployment; the rest stay queued for post-v1 - sessions. - -The carryovers Slice 8 inherits — the full §11.5 / §12 spec text, -the existing `cache.Reconciler` and `DigestScheduler` shape, the -deploy/ infrastructure, and the 75/75 green test suite. - -The next build session should read `SPEC.md`, `README.md`, -`docs/DEV.md`, and `SPEC.md`'s §19.1 and pick up Slice 8 cleanly -without re-briefing. The working agreement in §19.3 continues to -apply: implement the slice, correct the spec only where running -code reveals it was wrong at a structural level, accumulate new -candidate topics in §19.2, do not extend the spec beyond what the -slice requires. +There is no "next slice" footer here because there isn't a next +slice. The work mode has shifted: the build is the source-of-truth +artifact, and §19.2 is the queue of decisions to settle when their +turn comes.