From a255429e57e804f791b0292272896a52adcf8c51 Mon Sep 17 00:00:00 2001 From: Ben Stull Date: Mon, 25 May 2026 05:42:15 -0700 Subject: [PATCH] =?UTF-8?q?Post-v1:=20per-RFC=20model=20availability=20(UX?= =?UTF-8?q?=20half)=20folded=20into=20=C2=A76.6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First §19.2 candidate settled after v1. The heavier per-RFC-model topic subdivided into UX (this) and credential delegation + funder role (still §19.2). New §6.6 carries the rule: an optional `models:` frontmatter field on the meta-repo RFC entry; absent inherits the operator universe, populated narrows the picker to the intersection with provisioned providers, `[]` opts the RFC out of AI entirely. The first resolved entry is the RFC default. §18's ENABLED_MODELS is reframed as the operator universe. Code: migration 009 adds nullable cached_rfcs.models_json (NULL ≠ [] is load-bearing); entry.py grows the optional field with absent-vs- empty round-tripping in parse/serialize; new models_resolver module holds the rule; api_branches replaces /api/models with the slug-aware /api/rfcs/{slug}/models and threads the chat + reask paths through the resolver; api_prs §10.2 uses the resolver and extends the stub fallback to the opt-out case; frontend passes slug to listModels. Tests 106/106 green (96 prior + 10 in test_per_rfc_models.py). No behavioral change for entries without `models:` — operator universe preserved as default. Co-Authored-By: Claude Opus 4.7 (1M context) --- SPEC.md | 150 ++++++-- backend/app/api_branches.py | 42 ++- backend/app/api_prs.py | 19 +- backend/app/cache.py | 9 +- backend/app/entry.py | 17 + backend/app/models_resolver.py | 68 ++++ backend/migrations/009_per_rfc_models.sql | 7 + backend/tests/test_per_rfc_models.py | 408 ++++++++++++++++++++++ docs/DEV.md | 86 +++++ frontend/src/api.js | 4 +- frontend/src/components/RFCView.jsx | 2 +- 11 files changed, 755 insertions(+), 57 deletions(-) create mode 100644 backend/app/models_resolver.py create mode 100644 backend/migrations/009_per_rfc_models.sql create mode 100644 backend/tests/test_per_rfc_models.py diff --git a/SPEC.md b/SPEC.md index 9479851..2f794a5 100644 --- a/SPEC.md +++ b/SPEC.md @@ -77,6 +77,9 @@ graduated_by: null owners: [] # contributors elevated for this RFC arbiters: [ben] # contributors with merge authority for this RFC tags: [identity, schema] +# models: # optional per-§6.6 — absent inherits the +# - claude # operator universe; [] opts the RFC out of AI +# - gemini # entirely; populated narrows the picker --- ## Why this RFC is needed @@ -395,6 +398,67 @@ the acting user. Gitea's commit log is for code archaeology; the `actions` and `permission_events` tables are the real accountability record. +### 6.6 Per-RFC model availability + +Which AI models contributors can pick from is configurable per RFC. +The configuration lives in the meta-repo entry's frontmatter as an +optional `models:` list of model identifiers (see §2.1), in the same +shape as `owners:` and `arbiters:` — frontmatter-native, edited via +the meta-repo PR flow that already governs the rest of the entry's +canonical state, mirrored into the §4 cache for read-without-roundtrip. +The field is structurally +*optional* and the absent/empty distinction is load-bearing: + +- **Absent** (`models:` key omitted) — the RFC inherits the operator's + universe. The operator's universe is the set of models the + deployment is provisioned to run, configured at the process level + per §18. A freshly proposed super-draft and any RFC whose + contributors haven't expressed a preference fall here. +- **Present and non-empty** — the RFC opts into a specific subset. + The picker's option list, at every AI surface scoped to this RFC, + is the intersection of this list with the operator's currently + provisioned models. Models the RFC names but the operator no + longer provisions are silently hidden from the picker; the + frontmatter list is preserved so a later operator change can + restore them. +- **Present and empty** (`models: []`) — the RFC opts out of AI + participation entirely. Every AI surface on the RFC honors the + refusal honestly: the chat composer's AI affordances are absent, + flag-resolution's "Ask Claude to propose a fix" is absent, the + §10.2 PR-draft falls back to its deterministic stub, the §9.1 + AI-suggested tags surface is absent. A contributor who tries to + invoke the AI sees the refusal in the surface, not a mid-turn + failure. + +The **RFC default model** is the first entry in the resolved list +(intersection with the operator's provisioned set; or the operator +universe when the field is absent). Where a deterministic single +model must be chosen and the contributor has not picked one — the +§10.2 PR-draft, the §9.1 tag suggestions, the §8.13 "Ask Claude to +propose a fix" invocation from an empty thread — that's the model +used. Per-message picker grain inside a chat thread (the §18 +prototype carryover) is preserved: each message can name a different +model from the resolved list, and the picker's currently-selected +entry persists across messages within a session. + +Editing the field — adding a model, removing one, switching to the +empty-list opt-out — follows the same authority rules as editing +`owners:` or `arbiters:`. Super-draft entries: the claimed owners +and app-wide admins. Active RFCs: the RFC's owners and arbiters per +§6.3, plus app-wide admins. The dedicated chrome for the edit (the +metadata pane equivalent that the §19.2 metadata-pane UX topic will +settle for super-drafts, and whatever surface admins use for active- +RFC frontmatter edits) is a downstream concern — the structural +commitment here is the field, the resolution rule, and the +configuration capability, not the click path. + +This section names *which* models are permitted on a given RFC. It +does not name *whose API resources pay for them* — credentials +remain operator-supplied per §18. Credential delegation and the +funder role are a separate topic (§19.2) whose settlement will +attach a parallel "credential set" notion alongside this one +without changing the availability surface. + --- ## 7. The left pane @@ -783,6 +847,15 @@ the scroll-to-editor binding. Resolved and stale threads stay in the data; the chat feed's filter affordances surface or hide them on demand. +The chat's model picker (the §18 prototype carryover) draws its +option list from the resolved per-RFC set per §6.6 — the +intersection of the entry's optional `models:` frontmatter (or the +operator universe when absent) with the operator's currently +provisioned providers. An RFC whose resolved list is empty surfaces +the AI affordances as absent rather than disabled-but-present, so +the refusal reads as a property of the RFC, not as a transient +error. + ### 8.13 Flags A flag is the lightweight "I'm pointing at this, it's a problem" @@ -1290,8 +1363,13 @@ branch chat, both editable inline before submit: to consider. The model is told the audience is *an arbiter*, not Ben specifically — -the framework has to scale past one person. Title and description remain -editable post-open by the contributor or any of the RFC's arbiters. +the framework has to scale past one person. The model used for the +draft is the RFC's default per §6.6 — the first entry in the +resolved per-RFC list. When that list is empty (the RFC opts out of +AI per §6.6), the draft falls back to a deterministic stub naming +the RFC title; the contributor edits the prefilled text as usual. +Title and description remain editable post-open by the contributor +or any of the RFC's arbiters. There is no reviewer picker. The RFC's arbiters (§6.3) are the implicit reviewer set; surfacing a per-PR picker would either duplicate that or @@ -2202,7 +2280,9 @@ specified* and what is intentionally out of scope for v1. rich markdown, headings, links, code blocks. - **The per-RFC and per-branch chat UX.** Threading model, AI participation, the discuss-vs-contribute mode distinction from the - prototype, the selection tooltip, the prompt bar, model picker. + prototype, the selection tooltip, the prompt bar, the model picker + chrome (its option-list source is settled in §6.6 / §8.12; the + visual treatment and per-thread persistence remain). - **The revision flow.** How proposed changes from AI or contributors appear, the change-card panel, accept/decline/edit, tracked insertions/deletions in the editor. @@ -2434,7 +2514,11 @@ them: - The structured `` / `` / `` / `` protocol for AI-proposed edits. - Multi-provider LLM support: Anthropic Claude, Google Gemini, OpenAI / - GitHub Copilot. `ENABLED_MODELS` and per-provider API keys via env. + GitHub Copilot. `ENABLED_MODELS` and per-provider API keys via env — + these now define the operator's *universe* of available models, the + set the deployment is provisioned to run. Per-RFC selection from + that universe is settled in §6.6; credential delegation and the + funder role remain a §19.2 candidate. - The discuss-vs-contribute distinction inside an RFC view (to be fully specified in the follow-up session). - Gitea OAuth for user authentication. The OAuth identity is the basis @@ -2604,24 +2688,34 @@ build surfaces evidence for which one matters next. Topics are listed roughly in order of expected weight; the order is not binding. -- **Per-RFC model availability and credential delegation.** Which - AI models contributors can pick from when chatting on a given - RFC, and who supplies the API resources for those models. - Replaces §18's app-level `ENABLED_MODELS` and env-supplied keys - with per-RFC-scoped configuration. Touches every AI surface — - every chat thread (§8.12), every change-proposal turn (§8.9, - §8.11), every flag-resolution invocation (§8.13), the AI-drafted - PR title and description (§10.2), and the propose modal's - AI-suggested tags (§9.1). Touches §5 (schema for model config - and credentials), §6 (possibly a `funder` role, or a per-RFC - capability extension along the lines of §6.2's per-user - overrides), and §18 (carryover supersession). Subdividable into - "model availability" (lighter, UX-shaped) and "credential - delegation and the funder role" (heavier — security, billing, - abuse mitigation, rotation, mid-conversation key failure) if the - session driver judges the combined scope too large. Load-bearing - once the framework runs past single-operator deployment; - defer-able until then. +- **Per-RFC model availability — UX half.** *Settled in the + post-v1 session that picked it. The meta-repo entry frontmatter + now carries an optional `models:` list per §6.6; the resolution + rule (absent inherits the operator universe, populated narrows + the picker by intersection, `[]` opts the RFC out of AI + entirely) is uniform across every AI surface — §8.12 chat + picker, §8.9 / §8.11 change-proposal turns, §8.13 flag + resolution, §10.2 PR draft, §9.1 tag suggestions. The §18 + `ENABLED_MODELS` env contract is reframed as the operator's + universe of provisioned models; the per-RFC list picks from + within it. The dedicated chrome for editing the field is + downstream — clustered with the metadata-pane UX candidate + below for super-drafts, and with whatever surface admins use + for active-RFC frontmatter edits.* +- **Per-RFC credential delegation and the funder role.** The + heavier half of the original per-RFC-model topic — who supplies + the API resources behind the models a given RFC permits. The + availability half (§6.6) names *which* models are allowed; + credential delegation names *whose keys pay for the calls*. + Touches security (delegation grant shape, revocation), billing + (per-RFC spend, attribution), abuse mitigation (rate limits per + funder, quota exhaustion), key rotation (mid-conversation + failure handling, retry-with-fallback), and a possible + `funder` role distinct from owner/arbiter that scopes credential + authority without conferring frontmatter authority. Load-bearing + once the framework runs past single-operator deployment; the v1 + shape — operator-supplied keys per §18 — is the right default + until a real multi-operator scenario surfaces. - **Admin surfaces.** Where role management, muting, audit-log views, the graduation-readiness queue, and Topic 13's notification-preferences settings (email categories per §15.4, @@ -2763,12 +2857,12 @@ binding. - **The §10.2 modal's AI-drafted text when no provider is configured.** Slice 3 falls back to a deterministic stub (`Edits to ` plus a character-count line) when the - app has no LLM provider. The fallback is functional but does - not produce spec-voice text. Per-RFC model availability (the - first §19.2 candidate, on the funder-role topic) will need to - settle the credential-delegation shape before this earns its - own topic; until then, the stub is the right shape for the - no-credential-available case. + app has no LLM provider. The §6.6 settlement extends the same + fallback to the case where the RFC's resolved model list is + empty — the RFC has opted out of AI entirely. The fallback is + functional but does not produce spec-voice text; improving the + no-credential-available render remains its own future topic, + defer-able until evidence shows the stub bites. - **§10.9 replay AI participation.** Slice 3 implements the structural §10.9 path — fresh resolution branch off main, replay the accepted changes whose `original` text still locates exactly diff --git a/backend/app/api_branches.py b/backend/app/api_branches.py index 788046b..c9a9310 100644 --- a/backend/app/api_branches.py +++ b/backend/app/api_branches.py @@ -29,7 +29,7 @@ from fastapi import APIRouter, HTTPException, Request from fastapi.responses import StreamingResponse from pydantic import BaseModel, Field -from . import auth, cache, chat as chat_layer, db, entry as entry_mod +from . import auth, cache, chat as chat_layer, db, entry as entry_mod, models_resolver from .bot import Bot from .config import Config from .gitea import Gitea, GiteaError @@ -111,21 +111,23 @@ def make_router( ) -> APIRouter: router = APIRouter() - default_model = next(iter(providers)) if providers else "" - # ------------------------------------------------------------------- - # §17: model picker (the prototype carryover, scoped here since - # Slice 2 is where chat lights up). + # §6.6 / §17: per-RFC model picker. The option list and default are + # resolved per RFC by intersecting the meta-repo entry's optional + # `models:` frontmatter with the operator's provisioned universe. + # An empty resolved list means the RFC has opted out of AI per + # §6.6 and the picker surfaces no options. # ------------------------------------------------------------------- - @router.get("/api/models") - async def list_models() -> dict[str, Any]: + @router.get("/api/rfcs/{slug}/models") + async def list_models_for_rfc(slug: str) -> dict[str, Any]: + resolved = models_resolver.resolve_models_for_rfc(slug, providers) return { "models": [ - {"id": key, "name": p.display_name} - for key, p in providers.items() + {"id": key, "name": providers[key].display_name} + for key in resolved ], - "default": default_model, + "default": resolved[0] if resolved else "", } # ------------------------------------------------------------------- @@ -560,15 +562,21 @@ def make_router( thread_id = row["thread_id"] if thread_id is None: raise HTTPException(409, "Change has no originating thread") - if not providers: + # §6.6: refuse cleanly if the RFC's resolved model list is empty — + # either the operator has no providers, or the RFC opted out + # (`models: []`), or the entry names only models the operator + # no longer provisions. Same shape; same honest refusal. + resolved = models_resolver.resolve_models_for_rfc(slug, providers) + if not resolved: raise HTTPException(503, "No AI providers configured") + reask_model = resolved[0] owner, repo = _repo_for(rfc, branch) path = _file_path_for(rfc, branch) fetched = await gitea.read_file(owner, repo, path, ref=branch) body_text = _extract_body(rfc, fetched[0], branch) if fetched else "" - provider = next(iter(providers.values())) + provider = providers[reask_model] system = chat_layer.build_system_prompt(title=rfc["title"], body=body_text) history = chat_layer.build_history(thread_id) reask_prompt = ( @@ -581,7 +589,7 @@ def make_router( thread_id=thread_id, author_user_id=viewer.user_id, text=reask_prompt, quote=None ) assistant_id = chat_layer.append_assistant_placeholder( - thread_id=thread_id, model_id=default_model + thread_id=thread_id, model_id=reask_model ) text = provider.send(system, history + [{"role": "user", "content": reask_prompt}]) @@ -892,9 +900,13 @@ def make_router( thread = _require_thread(slug, branch, thread_id) if not _can_read_branch(slug, branch, viewer): raise HTTPException(403, "Branch is private") - if not providers: + # §6.6: option list and default come from the per-RFC resolved set, + # not the operator universe. body.model is honored only if it sits + # inside the resolved set; otherwise we fall to the RFC default. + resolved = models_resolver.resolve_models_for_rfc(slug, providers) + if not resolved: raise HTTPException(503, "No AI providers configured") - model_key = body.model if body.model in providers else default_model + model_key = body.model if body.model in resolved else resolved[0] provider = providers[model_key] # Fetch the live branch body so the prompt is anchored to diff --git a/backend/app/api_prs.py b/backend/app/api_prs.py index 41728b2..f640e98 100644 --- a/backend/app/api_prs.py +++ b/backend/app/api_prs.py @@ -23,7 +23,7 @@ from typing import Any from fastapi import APIRouter, HTTPException, Request from pydantic import BaseModel, Field -from . import auth, cache, chat as chat_layer, db, entry as entry_mod +from . import auth, cache, chat as chat_layer, db, entry as entry_mod, models_resolver from .bot import Bot from .config import Config from .gitea import Gitea, GiteaError @@ -68,13 +68,13 @@ def make_router( ) -> APIRouter: router = APIRouter() - default_model = next(iter(providers)) if providers else "" - # ------------------------------------------------------------------- # §10.2: AI-drafted PR title and description. # Returned ahead of submit so the modal renders with prefilled values # the contributor can edit. The contributor's gesture is what # produces the open-pr call; the draft is just a starting point. + # Per §6.6 the model used is the RFC's resolved default; an empty + # resolved list falls back to the deterministic stub. # ------------------------------------------------------------------- @router.post("/api/rfcs/{slug}/branches/{branch:path}/pr-draft") @@ -90,9 +90,10 @@ def make_router( if not branch_fetched: raise HTTPException(404, f"Branch {path} not found") chat_messages = _branch_chat_excerpt(slug, branch) + rfc_default_model = models_resolver.default_model_for_rfc(slug, providers) title, description = _draft_with_provider( providers=providers, - default_model=default_model, + default_model=rfc_default_model, rfc_title=rfc["title"], main_body=_extract_body(rfc, (main_fetched or ("", ""))[0]), branch_body=_extract_body(rfc, branch_fetched[0]), @@ -795,12 +796,12 @@ def _draft_with_provider( """Per §10.2: AI-drafted title (spec voice) and description (2–4 sentences pulling from chat). - When no provider is configured we fall back to a deterministic - stub — the surface still works; the contributor just edits the - text. The fallback also matches the test seam where Slice 3 - integration tests don't always inject a fake provider. + When no provider is configured — or per §6.6 the RFC's resolved + list is empty (operator universe empty, frontmatter opt-out, or + intersection empty) — we fall back to a deterministic stub. The + surface still works; the contributor edits the text. """ - if not providers: + if not providers or not default_model: return _stub_draft(rfc_title=rfc_title, main_body=main_body, branch_body=branch_body) provider = providers.get(default_model) or next(iter(providers.values())) system = ( diff --git a/backend/app/cache.py b/backend/app/cache.py index 2e63a90..2807ec6 100644 --- a/backend/app/cache.py +++ b/backend/app/cache.py @@ -76,13 +76,16 @@ async def refresh_meta_repo(config: Config, gitea: Gitea) -> None: def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None: + # §6.6: models_json stays NULL when the frontmatter key is absent + # (inherit operator universe) and '[]' for the explicit opt-out. + models_json = json.dumps(entry.models) if entry.models is not None else None db.conn().execute( """ INSERT INTO cached_rfcs (slug, title, state, rfc_id, repo, proposed_by, proposed_at, graduated_at, graduated_by, owners_json, arbiters_json, tags_json, - body, body_sha, last_entry_commit_at, updated_at) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now')) + models_json, body, body_sha, last_entry_commit_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now'), datetime('now')) ON CONFLICT(slug) DO UPDATE SET title = excluded.title, state = excluded.state, @@ -95,6 +98,7 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None: owners_json = excluded.owners_json, arbiters_json = excluded.arbiters_json, tags_json = excluded.tags_json, + models_json = excluded.models_json, body = excluded.body, body_sha = excluded.body_sha, last_entry_commit_at = datetime('now'), @@ -113,6 +117,7 @@ def _upsert_cached_rfc(entry: entry_mod.Entry, body_sha: str) -> None: json.dumps(entry.owners), json.dumps(entry.arbiters), json.dumps(entry.tags), + models_json, entry.body, body_sha, ), diff --git a/backend/app/entry.py b/backend/app/entry.py index 568c0cf..d9b236b 100644 --- a/backend/app/entry.py +++ b/backend/app/entry.py @@ -23,6 +23,8 @@ FRONTMATTER_RE = re.compile(r"^---\n(.*?)\n---\n?(.*)$", re.DOTALL) SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$") +_ABSENT = object() + @dataclass class Entry: @@ -38,6 +40,11 @@ class Entry: owners: list[str] = field(default_factory=list) arbiters: list[str] = field(default_factory=list) tags: list[str] = field(default_factory=list) + # §6.6: per-RFC model availability. None means the key is absent + # from frontmatter (inherit the operator universe). An empty list + # means an explicit opt-out from AI on this RFC. A populated list + # narrows the picker to its intersection with the operator universe. + models: list[str] | None = None body: str = "" @@ -47,6 +54,11 @@ def parse(text: str) -> Entry: raise ValueError("Entry file missing frontmatter") fm = yaml.safe_load(match.group(1)) or {} body = match.group(2).lstrip("\n") + raw_models = fm.get("models", _ABSENT) + if raw_models is _ABSENT or raw_models is None: + models: list[str] | None = None + else: + models = [str(m) for m in raw_models] return Entry( slug=str(fm.get("slug") or ""), title=str(fm.get("title") or ""), @@ -60,6 +72,7 @@ def parse(text: str) -> Entry: owners=list(fm.get("owners") or []), arbiters=list(fm.get("arbiters") or []), tags=list(fm.get("tags") or []), + models=models, body=body, ) @@ -80,6 +93,10 @@ def serialize(entry: Entry) -> str: "arbiters": entry.arbiters, "tags": entry.tags, } + # §6.6: emit `models:` only when set. Absent in the frontmatter + # is meaningfully different from `models: []` per §6.6. + if entry.models is not None: + fm["models"] = entry.models yaml_text = yaml.safe_dump(fm, sort_keys=False, default_flow_style=False).rstrip() body = entry.body.lstrip("\n") if body: diff --git a/backend/app/models_resolver.py b/backend/app/models_resolver.py new file mode 100644 index 0000000..4a97b5d --- /dev/null +++ b/backend/app/models_resolver.py @@ -0,0 +1,68 @@ +"""§6.6 per-RFC model availability — the resolver. + +The meta-repo entry's optional `models:` frontmatter and the operator's +provisioned providers (the §18 `ENABLED_MODELS` universe) combine into a +single resolved list per RFC. Every AI surface picks against that list: +the §8.12 chat picker, the §10.2 PR-draft, the §8.13 flag-resolution +invocation, the §9.1 tag suggestions when graduation is in scope. + +Rule, in one place: + +- Cache row's `models_json` is NULL → the field is absent on the entry. + Resolved list = the operator's provisioned universe. +- Cache row's `models_json` is a JSON array → the field is set on the + entry. Resolved list = intersection of the array with the operator's + provisioned universe, preserving the entry's stated order. + +The empty case folds in naturally: an entry with `models: []` yields an +empty intersection, and an entry whose listed models are no longer +provisioned by the operator also yields an empty intersection. Callers +treat both the same — surfaces refuse cleanly per §6.6. + +The function is slug-aware and provider-aware; it does not depend on +the FastAPI request lifecycle, which keeps it cheap to call inside any +endpoint that knows the slug and has the providers dict in scope. +""" +from __future__ import annotations + +import json + +from . import db +from .providers import BaseProvider + + +def resolve_models_for_rfc( + slug: str, providers: dict[str, BaseProvider] +) -> list[str]: + """Return the per-RFC resolved model keys per §6.6. + + The first entry is the RFC's default model. An empty list means + AI is unavailable on this RFC and callers refuse the AI surface. + """ + universe = list(providers.keys()) + row = db.conn().execute( + "SELECT models_json FROM cached_rfcs WHERE slug = ?", + (slug,), + ).fetchone() + if row is None or row["models_json"] is None: + return universe + try: + listed = [str(m) for m in json.loads(row["models_json"])] + except (json.JSONDecodeError, TypeError): + return universe + if not listed: + return [] + return [m for m in listed if m in providers] + + +def default_model_for_rfc( + slug: str, providers: dict[str, BaseProvider] +) -> str: + """The first entry in the resolved list, or empty string if none. + + Callers that need a deterministic single choice — §10.2's draft, + §9.1's tag suggestions — use this. An empty return signals the + refusal path per §6.6. + """ + resolved = resolve_models_for_rfc(slug, providers) + return resolved[0] if resolved else "" diff --git a/backend/migrations/009_per_rfc_models.sql b/backend/migrations/009_per_rfc_models.sql new file mode 100644 index 0000000..220b2b6 --- /dev/null +++ b/backend/migrations/009_per_rfc_models.sql @@ -0,0 +1,7 @@ +-- §6.6: per-RFC model availability. The frontmatter field is +-- optional; NULL in this column means absent (inherit the operator +-- universe), '[]' means explicit opt-out, '[...]' is the populated +-- list. The distinction is load-bearing — we cannot collapse to a +-- NOT NULL DEFAULT '[]' the way the other *_json columns do. + +ALTER TABLE cached_rfcs ADD COLUMN models_json TEXT; diff --git a/backend/tests/test_per_rfc_models.py b/backend/tests/test_per_rfc_models.py new file mode 100644 index 0000000..a3e6dcb --- /dev/null +++ b/backend/tests/test_per_rfc_models.py @@ -0,0 +1,408 @@ +"""Integration coverage for the §19.2 "per-RFC model availability — UX +half" candidate folded into §6.6. + +The meta-repo entry's optional `models:` frontmatter narrows what AI +models contributors can pick from on a given RFC. The resolution rule +is uniform across every AI surface: + + - Absent (`models:` key omitted) → inherit the operator universe. + - Populated list → intersection with the operator's provisioned + providers, preserving the entry's order. First entry is the RFC + default. + - Empty list (`models: []`) → AI surfaces refuse honestly. + +The tests below prove each branch through the API surface and verify +the cache round-trip from meta-repo frontmatter to picker option list. +""" +from __future__ import annotations + +import asyncio +import json + +from test_propose_vertical import ( # noqa: F401 + FakeGitea, + app_with_fake_gitea, + provision_user_row, + sign_in_as, + tmp_env, +) +from test_rfc_view_vertical import FakeProvider, seed_active_rfc # noqa: F401 +from test_super_draft_vertical import seed_super_draft # noqa: F401 + + +SEED_BODY = ( + "Open Human Model is a framework for representing humans.\n\n" + "It defines consent, trait, and agency in compatible terms." +) + + +def _set_models_json(slug: str, value: str | None) -> None: + """Write the resolved cached_rfcs.models_json directly. Mirrors what + `_upsert_cached_rfc` would do on the next reconciler sweep, without + requiring a full meta-repo refresh in tests focused on the resolver.""" + from app import db + db.conn().execute( + "UPDATE cached_rfcs SET models_json = ? WHERE slug = ?", + (value, slug), + ) + + +def _install_two_providers(app) -> None: + """Swap two fake providers — `claude` and `gemini` — into the + shared providers dict. The dict is held by reference inside the + router closures (see Slice 2's pattern in test_rfc_view_vertical), + so the mutation propagates.""" + app.state.providers.clear() + app.state.providers["claude"] = FakeProvider("TITLE: A\nDESCRIPTION: B") + app.state.providers["gemini"] = FakeProvider("TITLE: G\nDESCRIPTION: H") + + +# --------------------------------------------------------------------------- +# /api/rfcs//models — the picker option-list surface +# --------------------------------------------------------------------------- + + +def test_absent_frontmatter_inherits_operator_universe(app_with_fake_gitea): + """When the meta-repo entry has no `models:` field, the picker's + option list is the operator's full provisioned universe.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + _install_two_providers(app) + # models_json is NULL — set_models_json not called. + + r = client.get("/api/rfcs/ohm/models") + assert r.status_code == 200, r.text + body = r.json() + ids = [m["id"] for m in body["models"]] + assert ids == ["claude", "gemini"] + assert body["default"] == "claude" + + +def test_populated_frontmatter_narrows_picker_to_intersection(app_with_fake_gitea): + """A populated `models:` list narrows the picker. Operator order + does not matter — the entry's stated order is preserved.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + _install_two_providers(app) + _set_models_json("ohm", json.dumps(["gemini"])) + + r = client.get("/api/rfcs/ohm/models") + body = r.json() + assert [m["id"] for m in body["models"]] == ["gemini"] + assert body["default"] == "gemini" + + +def test_empty_frontmatter_disables_ai_surfaces(app_with_fake_gitea): + """`models: []` — the RFC opts out of AI per §6.6. The picker + surface returns an empty list and an empty default.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + _install_two_providers(app) + _set_models_json("ohm", json.dumps([])) + + r = client.get("/api/rfcs/ohm/models") + body = r.json() + assert body["models"] == [] + assert body["default"] == "" + + +def test_intersection_with_provisioned_providers(app_with_fake_gitea): + """An entry that lists models the operator no longer provisions + — they're silently hidden from the picker. The frontmatter list + is preserved so a later operator change can restore them; the + cache row is untouched by this test.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + _install_two_providers(app) + # Frontmatter names a model the operator doesn't have plus one + # it does — only the intersection comes out. + _set_models_json("ohm", json.dumps(["llama-3", "claude"])) + + r = client.get("/api/rfcs/ohm/models") + body = r.json() + assert [m["id"] for m in body["models"]] == ["claude"] + assert body["default"] == "claude" + + +def test_intersection_empty_same_as_opt_out(app_with_fake_gitea): + """When the frontmatter names only models the operator doesn't + provision, the resolved list is empty and the surface refuses + cleanly — same shape as the explicit `models: []` opt-out.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + sign_in_as(client, user_id=1, gitea_login="ben", + display_name="Ben", role="owner", email="ben@test") + _install_two_providers(app) + _set_models_json("ohm", json.dumps(["llama-3", "gpt-5"])) + + r = client.get("/api/rfcs/ohm/models") + assert r.json()["models"] == [] + + +# --------------------------------------------------------------------------- +# §10.2 PR draft: model selection follows the RFC default +# --------------------------------------------------------------------------- + + +def test_pr_draft_uses_rfc_default_model(app_with_fake_gitea): + """The §10.2 draft picks the RFC's default — the first entry of + the resolved list. We rig the providers so `claude` returns one + title and `gemini` returns a different one, then prove the entry's + pinned model is the one we get back.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + + # Two fakes, distinct outputs. + app.state.providers.clear() + app.state.providers["claude"] = FakeProvider( + "TITLE: from claude\nDESCRIPTION: claude wrote this." + ) + app.state.providers["gemini"] = FakeProvider( + "TITLE: from gemini\nDESCRIPTION: gemini wrote this." + ) + _set_models_json("ohm", json.dumps(["gemini"])) + + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + r = client.post("/api/rfcs/ohm/branches/main/promote-to-branch", json={}) + branch = r.json()["branch_name"] + + # Make the branch differ from main so the draft endpoint passes + # its "commits ahead" precondition. + from app import db + from test_rfc_view_vertical import FakeGitea # noqa: F401 + # Promote-to-branch starts a branch off main with main's content; + # the simplest way to give it a commit-ahead is via a manual edit + # flush. But for this test, an inline commit on the fake is + # cleaner. + repo_full = "wiggleverse/rfc-0001-ohm" + owner, repo = repo_full.split("/", 1) + new_body = SEED_BODY + "\n\nA further sentence." + fake.files[(owner, repo, branch, "RFC.md")] = { + "content": new_body, "sha": fake._next_sha(), + } + fake.branches[(owner, repo)][branch]["sha"] = fake.files[ + (owner, repo, branch, "RFC.md") + ]["sha"] + db.conn().execute( + """ + UPDATE cached_branches SET head_sha = ? + WHERE rfc_slug = 'ohm' AND branch_name = ? + """, + (fake.branches[(owner, repo)][branch]["sha"], branch), + ) + + r = client.post(f"/api/rfcs/ohm/branches/{branch}/pr-draft") + assert r.status_code == 200, r.text + # `gemini` is the resolved default — its title comes back. + assert r.json()["title"] == "from gemini" + + +def test_pr_draft_falls_back_to_stub_on_opt_out(app_with_fake_gitea): + """When the RFC opts out of AI (`models: []`), §10.2 falls back to + its deterministic stub — the title is the stub format, not an + LLM-generated string.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _install_two_providers(app) + _set_models_json("ohm", json.dumps([])) + + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + r = client.post("/api/rfcs/ohm/branches/main/promote-to-branch", json={}) + branch = r.json()["branch_name"] + + from app import db + repo_full = "wiggleverse/rfc-0001-ohm" + owner, repo = repo_full.split("/", 1) + new_body = SEED_BODY + "\n\nA further sentence." + fake.files[(owner, repo, branch, "RFC.md")] = { + "content": new_body, "sha": fake._next_sha(), + } + fake.branches[(owner, repo)][branch]["sha"] = fake.files[ + (owner, repo, branch, "RFC.md") + ]["sha"] + db.conn().execute( + """ + UPDATE cached_branches SET head_sha = ? + WHERE rfc_slug = 'ohm' AND branch_name = ? + """, + (fake.branches[(owner, repo)][branch]["sha"], branch), + ) + + r = client.post(f"/api/rfcs/ohm/branches/{branch}/pr-draft") + assert r.status_code == 200, r.text + body = r.json() + # The stub uses "Edits to " as the title (per Slice 3). + assert body["title"] == "Edits to OHM" + + +# --------------------------------------------------------------------------- +# Chat surface refuses cleanly when AI is unavailable for the RFC +# --------------------------------------------------------------------------- + + +def test_chat_refuses_when_rfc_opted_out_of_ai(app_with_fake_gitea): + """A chat turn on a branch of an RFC with `models: []` refuses + with 503 — same shape as "no providers configured." The refusal + surfaces as a property of the RFC, not a server failure.""" + from fastapi.testclient import TestClient + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=2, login="alice", role="contributor") + seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY) + _install_two_providers(app) + _set_models_json("ohm", json.dumps([])) + + sign_in_as(client, user_id=2, gitea_login="alice", + display_name="Alice", role="contributor") + r = client.post("/api/rfcs/ohm/branches/main/promote-to-branch", json={}) + branch = r.json()["branch_name"] + view = client.get(f"/api/rfcs/ohm/branches/{branch}").json() + thread_id = view["main_thread_id"] + + r = client.post( + f"/api/rfcs/ohm/branches/{branch}/threads/{thread_id}/chat", + json={"text": "tighten this", "model": "claude"}, + ) + assert r.status_code == 503 + + +# --------------------------------------------------------------------------- +# Cache round-trip: frontmatter `models:` flows into models_json +# --------------------------------------------------------------------------- + + +def test_meta_repo_frontmatter_models_round_trips_through_cache(app_with_fake_gitea): + """The production path: a meta-repo entry whose frontmatter carries + `models: [claude]` lands in `cached_rfcs.models_json` after the + next `refresh_meta_repo` sweep.""" + from fastapi.testclient import TestClient + from app import cache as cache_mod, db + + app, fake = app_with_fake_gitea + with TestClient(app) as client: + provision_user_row(user_id=1, login="ben", role="owner") + # Seed a super-draft entry directly on meta-main with a + # `models:` frontmatter field. seed_super_draft doesn't take + # this field, so we write the meta-repo file ourselves and + # let the cache reconciler parse it. + entry_text = ( + "---\n" + "slug: ohm\n" + "title: OHM\n" + "state: super-draft\n" + "id: null\n" + "repo: null\n" + "proposed_by: ben\n" + "proposed_at: 2026-05-23\n" + "graduated_at: null\n" + "graduated_by: null\n" + "owners: []\n" + "arbiters: []\n" + "tags: []\n" + "models:\n" + " - claude\n" + "---\n\n" + "The pitch goes here.\n" + ) + sha = fake._next_sha() + fake.files[("wiggleverse", "meta", "main", "rfcs/ohm.md")] = { + "content": entry_text, "sha": sha, + } + fake.branches[("wiggleverse", "meta")]["main"]["sha"] = sha + + asyncio.run(cache_mod.refresh_meta_repo(app.state.config, app.state.gitea)) + + row = db.conn().execute( + "SELECT models_json FROM cached_rfcs WHERE slug = 'ohm'" + ).fetchone() + assert row is not None + assert row["models_json"] is not None + assert json.loads(row["models_json"]) == ["claude"] + + # And an entry without `models:` lands NULL. + no_models_text = entry_text.replace("models:\n - claude\n", "") + sha2 = fake._next_sha() + fake.files[("wiggleverse", "meta", "main", "rfcs/beta.md")] = { + "content": no_models_text.replace("slug: ohm", "slug: beta") + .replace("title: OHM", "title: Beta"), + "sha": sha2, + } + fake.branches[("wiggleverse", "meta")]["main"]["sha"] = sha2 + + asyncio.run(cache_mod.refresh_meta_repo(app.state.config, app.state.gitea)) + + row = db.conn().execute( + "SELECT models_json FROM cached_rfcs WHERE slug = 'beta'" + ).fetchone() + assert row is not None + assert row["models_json"] is None + + +# --------------------------------------------------------------------------- +# entry.py: absent / empty / populated round-trip the parser+serializer +# --------------------------------------------------------------------------- + + +def test_entry_models_field_absent_vs_empty_distinguished(): + """The absent vs. empty distinction is load-bearing per §6.6. + parse(serialize(x)) preserves None as None and [] as [].""" + from app import entry as entry_mod + + absent = entry_mod.Entry(slug="x", title="X") + assert absent.models is None + text_absent = entry_mod.serialize(absent) + assert "models:" not in text_absent + round_absent = entry_mod.parse(text_absent) + assert round_absent.models is None + + opted_out = entry_mod.Entry(slug="y", title="Y", models=[]) + text_empty = entry_mod.serialize(opted_out) + assert "models: []" in text_empty + round_empty = entry_mod.parse(text_empty) + assert round_empty.models == [] + + populated = entry_mod.Entry(slug="z", title="Z", models=["claude", "gemini"]) + text_full = entry_mod.serialize(populated) + round_full = entry_mod.parse(text_full) + assert round_full.models == ["claude", "gemini"] diff --git a/docs/DEV.md b/docs/DEV.md index 3f33f25..4903b6c 100644 --- a/docs/DEV.md +++ b/docs/DEV.md @@ -44,6 +44,92 @@ rare and surgical and live in the appropriate numbered section per ## State of the codebase +### Post-v1: §6.6 per-RFC model availability (UX half) + +The first §19.2 candidate settled after v1 shipped. The heavier +per-RFC-model topic explicitly subdivided into "model availability" +(UX-shaped) and "credential delegation + funder role" (security, +billing, rotation); this session folded in the lighter half and +left the heavier one as its own §19.2 entry. + +The settlement: a new optional `models:` frontmatter field on the +meta-repo RFC entry, in the same shape as `owners:` / `arbiters:`. +Absent inherits the operator's universe (the §18 `ENABLED_MODELS` +env contract, now reframed as the operator's provisioned set). +Populated lists narrow the picker to the intersection with that +universe, preserving the entry's stated order. An empty list +(`models: []`) opts the RFC out of AI participation entirely — +every AI surface honors the refusal honestly. The first entry of +the resolved list is the RFC's default model; the §10.2 PR-draft +uses it when present and falls back to its existing deterministic +stub when the resolved list is empty. + +§6.6 carries the structural rule. §2.1's frontmatter example shows +the field. §8.12 names the picker's option-list source. §10.2 +names the model used for drafting. §16 narrows its "model picker" +deferral to the chrome only. §18 reframes `ENABLED_MODELS` as the +operator universe. §19.2's per-RFC-model entry is split — the UX +half is marked *settled* with a pointer to §6.6; the credential +half lives on as its own entry. + +Code changes: + +- [`backend/migrations/009_per_rfc_models.sql`](../backend/migrations/009_per_rfc_models.sql) + adds `cached_rfcs.models_json` as a nullable column — NULL means + the frontmatter field is absent (inherit the universe), `'[]'` + means the explicit opt-out, `'[...]'` is the populated list. The + absent-vs-empty distinction is load-bearing per §6.6, so this + column cannot collapse to a `NOT NULL DEFAULT '[]'` the way the + other `*_json` columns do. +- [`backend/app/entry.py`](../backend/app/entry.py) grows an + optional `models: list[str] | None` field on the `Entry` + dataclass. The parser uses a sentinel to distinguish "key not in + YAML" from "key set to null" so both round-trip cleanly. The + serializer emits `models:` only when set, preserving the absent + case in canonical entry text. +- [`backend/app/cache.py`](../backend/app/cache.py)'s + `_upsert_cached_rfc` writes `NULL` or `json.dumps(entry.models)` + on the round-trip from frontmatter through the cache. +- [`backend/app/models_resolver.py`](../backend/app/models_resolver.py) + is the new small module that holds the rule. Two functions: + `resolve_models_for_rfc(slug, providers)` returns the resolved + list, and `default_model_for_rfc(slug, providers)` returns the + first entry or empty string. Every AI surface calls into one of + these; the rule lives in one place. +- [`backend/app/api_branches.py`](../backend/app/api_branches.py) + replaces the slug-less `GET /api/models` with the per-RFC + `GET /api/rfcs/{slug}/models` per §6.6. The chat-stream and + reask-change paths now resolve through the per-RFC list and + refuse with 503 when it is empty. Both `default_model` references + inside the module are removed; the resolver replaces them. +- [`backend/app/api_prs.py`](../backend/app/api_prs.py)'s §10.2 + `pr-draft` endpoint resolves the RFC's default via the new + helper. The internal `_draft_with_provider` falls back to the + deterministic stub when `default_model` is empty (the §6.6 + opt-out case), extending the prior "no providers configured" + fallback symmetrically. +- [`frontend/src/api.js`](../frontend/src/api.js) and + [`frontend/src/components/RFCView.jsx`](../frontend/src/components/RFCView.jsx) + pass the slug to `listModels(slug)`, hitting the new per-RFC + endpoint. + +Settlement ships covered by +[`test_per_rfc_models.py`](../backend/tests/test_per_rfc_models.py) +— ten integration tests: absent-frontmatter inheriting the +operator universe, populated-frontmatter narrowing, the +empty-list opt-out, intersection with the operator's provisioned +set, intersection-empty matching the opt-out shape, the §10.2 +draft using the RFC default, the §10.2 stub fallback on opt-out, +the chat surface's 503 refusal on opt-out, the cache round-trip +from meta-repo frontmatter through `refresh_meta_repo`, and the +entry parser/serializer round-trip preserving the absent / +empty / populated distinction. + +The full test suite is 106/106 green (96 prior + 10 new). No +behavioral change for RFCs without `models:` in frontmatter — the +operator universe is preserved as the default, so existing +deployments see no surface change until a frontmatter edit lands. + ### Slice 1 — shipped The repository scaffolding (`backend/`, `frontend/`, `scripts/`, diff --git a/frontend/src/api.js b/frontend/src/api.js index c77b69b..0cb5f8f 100644 --- a/frontend/src/api.js +++ b/frontend/src/api.js @@ -71,8 +71,8 @@ export async function withdrawProposal(prNumber) { // ── Slice 2: active-RFC view (§8) ───────────────────────────────────────── -export async function listModels() { - return jsonOrThrow(await fetch('/api/models')) +export async function listModels(slug) { + return jsonOrThrow(await fetch(`/api/rfcs/${slug}/models`)) } export async function getRFCMain(slug) { diff --git a/frontend/src/components/RFCView.jsx b/frontend/src/components/RFCView.jsx index 7db9a1a..9391051 100644 --- a/frontend/src/components/RFCView.jsx +++ b/frontend/src/components/RFCView.jsx @@ -95,7 +95,7 @@ export default function RFCView({ viewer }) { useEffect(() => { getRFC(slug).then(setEntry).catch(err => setError(err.message)) - listModels() + listModels(slug) .then(({ models, default: def }) => { setModels(models || []) setSelectedModel(def || models?.[0]?.id || '')