Compare commits

...

39 Commits

Author SHA1 Message Date
Ben Stull 019c8a9185 Merge pull request 'Release v0.28.0: security-audit-0026 I3 + I4 (HTML-email guard + async Turnstile siteverify)' (#2) from feature/v0.28.0-email-turnstile-async into main 2026-05-29 02:42:05 +00:00
Ben Stull 79a447c77b Release v0.28.0: security-audit-0026 I3 + I4 (HTML-email guard + async Turnstile)
Two informational findings from the Session-0026 audit, both
framework-internal defense-in-depth. No operator action: no migration,
no schema/config/overlay change, no deployment-facing surface.

- I3: guard the dead text/html branch in email_envelope.build_envelope.
  No send path passes body_html; the unused branch would emit HTML built
  from possibly-unescaped user content (C1 stored-XSS class in the mail
  channel). Passing body_html now raises NotImplementedError; the arg is
  kept for documented future symmetry, enabling HTML mail becomes a
  deliberate escape-then-unguard change.

- I4: make turnstile.verify_token async. The sync httpx.post ran inside
  the async /auth/otc/request handler, blocking the event loop up to the
  10s timeout on a slow CloudFlare call. It now awaits httpx.AsyncClient
  via a narrow _siteverify_post seam (tests patch the seam, not the
  shared AsyncClient). The sole caller (main.py) now awaits it.

Tests: full backend suite 365 passed. Added a coroutine-contract unit
test for verify_token and flipped the email_envelope HTML test to assert
the guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 19:12:16 -07:00
Ben Stull fe044ed3db Merge pull request 'Release v0.27.0: security hardening (audit 0026)' (#1) from feature/v0.27.0-security-hardening into main 2026-05-29 01:28:42 +00:00
Ben Stull bd3ef269d4 Release v0.27.0: security hardening (audit 0026)
Remediates the rfc-app application + deploy-config findings from the
Session 0026 security audit. Cut as the "v0.25.0-security-hardening"
branch (from v0.24.0); reversioned to 0.27.0 on rebase onto main since
v0.26.0 (#28) shipped while this was in flight.

- C1 (Critical): single sanitizeHtml.js chokepoint (DOMPurify) for every
  marked→innerHTML / dangerouslySetInnerHTML sink (MarkdownPreview,
  ProposalView x2, Editor); rel=noopener hook on target=_blank links.
- H1: per-account OTC-verify lockout (migration 023, auto-applied) +
  per-IP throttle via new ratelimit.py; wired on otc verify/request +
  passcode check/verify.
- M1: device_trust.lookup() single indexed-row read — cookie value is now
  "<row_id>.<raw_token>"; bcrypt-checks one row, not a global table scan.
  (Behavior change: existing device-trust cookies re-prompt once.)
- M2: HTTP security headers (CSP/HSTS/XFO/XCTO/Referrer-Policy) at nginx.
- M4: session cookie Secure-by-default (SESSION_COOKIE_SECURE opt-out).
- M5: bounce webhook fails CLOSED (503) when secret unset, instead of open;
  RFC_APP_INSECURE_BOUNCE_WEBHOOK=1 dev opt-in.
- L2/L3: per-IP cooldown + check-endpoint throttle.
- L4: systemd sandbox knobs. L8/I1: nginx server_tokens off + TLS1.0/1.1 out.

VERSION + frontend/package.json → 0.27.0; CHANGELOG documents the upgrade
steps (incl. the out-of-band nginx + systemd apply, which the flotilla
deploy gesture does not perform).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 18:28:10 -07:00
Ben Stull 698821f065 Merge feature/v0.26.0-pr-rfc-links (#28 Part 1: auto-link accepted RFCs in PR text + comments) 2026-05-28 16:21:26 -07:00
Ben Stull e794523079 v0.26.0: auto-link referenced RFCs in PR text + comments (roadmap #28 Part 1)
Part 1 of item #28: references to existing accepted RFCs inside PR
descriptions and comment text now render as inline links to the
referenced RFC. Parts 2 (offer-to-create) and 3 (offer-to-contribute-
to-pending) are deliberately deferred — Part 1 ships first as the easy
win, per the roadmap row.

Shipped in parallel with the v0.25.0 security-hardening session; this
took the next free version slot (0.26.0) per the roadmap's
"claims the next available version number" rule. Expect a top-of-file
CHANGELOG/VERSION merge with 0.25.0 — distinct concerns, trivial to
resolve.

Backend:
- rfc_links.py (new): builds a term index from the live accepted
  (state='active') RFC corpus and segments plain text into text /
  rfc-link segments. Conservative matching — links only rfc_id tokens
  (RFC-0001), multi-word titles (Open Human Model), and hyphenated
  slugs (open-human-model); a single common-word title/slug is NOT
  linked (would turn every prose "human" into a link). Case-insensitive,
  word-boundary-anchored, longest-match-wins, self-reference suppressed.
- api_prs.py get_pr(): enriches the PR description (description_segments)
  and every PR comment (text_segments).
- api_discussion.py: enriches PR-less discussion comments (text_segments).

Read-time, not submit-time: the roadmap says "at submit time" but the
intent it names is "not as live compose preview", which read-time
honors. Chosen for correctness (links track the live active set —
newly-accepted RFCs start linking, withdrawn ones stop), zero migration,
and cheapness (small cache-resident corpus). Recorded as a §19.3-rule-2
note in the session transcript.

Frontend:
- LinkedText.jsx (new): maps backend segments onto React text nodes +
  anchors. No dangerouslySetInnerHTML — XSS-safe by construction,
  independent of any HTML-sanitization layer. Falls back to raw text
  when segments are absent.
- PRView.jsx: description + PR conversation comment bodies render via
  LinkedText.
- RFCDiscussionPanel.jsx: discussion comment bodies render via LinkedText.
- App.css: .rfc-autolink (subtle accent + dotted underline, tokenized).

Tests: 12 new (test_rfc_links_vertical.py) — 9 scanner units + 3
end-to-end (PR description / review comment / discussion comment all
surface *_segments; self-reference suppression). Full suite 363 green;
frontend builds clean.

No upgrade steps: additive, no migration, no secret, no config.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 16:17:01 -07:00
Ben Stull 28015ed1a2 Release v0.24.0: Claude Haiku tag suggestions on propose-RFC (#27)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 13:28:12 -07:00
Ben Stull 376a6daddc Merge feature/v0.24.0-haiku-tags (#27: Claude Haiku tag suggestions on propose-RFC) 2026-05-28 13:25:50 -07:00
Ben Stull fb9b4fa422 v0.24.0: Claude Haiku tag suggestions on propose-RFC (roadmap #27)
The §9.1 Slice-2 AI-suggested tag chips, deferred since the propose
modal landed, now wired up. As the propose-RFC draft fills in, the
backend asks Claude Haiku for tags drawn ONLY from the corpus's
existing tag set (the de-facto taxonomy — v1 tags are free-form chip
input, there is no curated list), surfaced as clickable suggestion
chips. Nothing auto-applies; the user clicks to add.

Backend:
- tag_suggest.py: universe gather (distinct corpus tags, most-common
  first), Haiku prompt + tolerant reply parser (drops invented tags,
  dedupes, clamps confidence), in-process per-user rate limit.
- providers.construct_haiku(): dedicated Haiku provider from the
  operator key, independent of ENABLED_MODELS — tag suggestion always
  uses the cheap+fast model. No RFC slug at propose time, so the §6.7
  funder path does not apply.
- POST /api/rfcs/suggest-tags: contributor-gated, rate-limited.
  Degrades to an empty list (never an error) when no Anthropic key is
  bound, the corpus has no tags, or the draft is empty.

Frontend:
- ProposeModal: debounced suggestion fetch (700ms, stale-response
  guarded), clickable suggestion chips, and the required inline
  disclosure that the draft text is sent to Anthropic.
- api.suggestTags(): forgiving — any non-OK resolves to [].

Tests: 11 new (vertical contributor-gating / filtering / no-key /
empty-corpus / 429, plus units for gather, parser tolerance, max,
short-circuit, provider-failure). Full suite 351 green.

New secret on deploy: ANTHROPIC_API_KEY (see CHANGELOG Upgrade steps).
Disclosure copy wants a counsel pass before deploy, per #22 discipline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 13:25:39 -07:00
Ben Stull daebb54f47 Release v0.23.0: server-side sign-in state resume (#29)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 12:37:21 -07:00
Ben Stull a598221812 Merge feature/v0.23.0-signin-resume (#29: server-side sign-in state resume) 2026-05-28 12:35:46 -07:00
Ben Stull bada72f87e v0.23.0: server-side sign-in state resume (roadmap #29)
Track each authenticated user's last-viewed route + light view state
server-side, and on next sign-in redirect them to that state, falling
back to the empty-state home only when there's no recorded state.

Backend:
- migration 022_user_session_state.sql: one row per user
  (user_id PK/FK, last_route, last_route_state JSON-as-TEXT,
  resume_enabled default 1, last_updated_at).
- PUT /api/me/last-state (require_user): upserts route + light state;
  no-ops when resume_enabled=0. Localized in the /me region.
- /api/auth/me payload now carries resume_enabled + last_route +
  decoded last_route_state (no extra round-trip).
- test_session_resume_vertical.py: auth-required, upsert/read-back,
  per-user isolation, resume_enabled=0 disable.

Frontend:
- lib/useLastState.js: debounced (~1s) route-change PUT for
  authenticated users; one-time resume redirect on sign-in, gated on
  identify having fired (preserves #21 Part C identify-then-track).
- api.js: putLastState() client call.
- App.jsx: import + call the hook; set identifyReady after identify.
  Header region untouched.

Per-user (not per-device); profile-settings opt-out toggle UI
deferred (column + default-on behavior ship now). Stored state is
route + light view state ONLY, never draft buffers — documented in
SPEC §6.8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 12:34:19 -07:00
Ben Stull 7d8371dea1 Release v0.22.0: optional propose use-case field (#26)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 12:32:34 -07:00
Ben Stull 3a51425ec7 Merge feature/v0.22.0-usecase-field (#26: optional propose use-case field) 2026-05-28 12:30:36 -07:00
Ben Stull 7c6c906db2 #26: optional proposed-use-case field on propose-RFC + propose-PR
Adds an optional "What will you be using this for?" capture as a sibling
to the required justification on both propose surfaces, per roadmap #26.

- propose-RFC modal (ProposeModal): optional textarea below the required
  "Why is this RFC needed?" pitch, labeled "What will you be using this
  RFC for? (optional)".
- propose-PR modal (PRModal): optional textarea below the required
  description, labeled "What will you be using this change for?".
- Backend: ProposeBody / OpenPRBody gain an optional `proposed_use_case`
  (NULL/omitted accepted, no min, 8000-char cap matching the existing
  free-text bound). Persisted to a new canonical side table
  `proposed_use_cases` keyed by PR number, mirrored onto the cache
  columns added by migration 021. Returned on the proposal list/detail,
  RFC detail (by slug), and PR detail endpoints.
- Display: ProposalView, RFCView (main only), and PRView render the
  captured use case with a muted "left blank" treatment when NULL.
- migration 021: nullable `proposed_use_case` on cached_rfcs/cached_prs
  plus the reconcile-proof `proposed_use_cases` truth table.
- New vertical test_proposed_use_case_vertical: persists+returns when
  supplied, accepted as NULL/omitted, for both surfaces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 12:28:52 -07:00
Ben Stull 2ac20b1621 Merge feature/v0.21.0-ux-polish (UX-polish wave: #31 + #24 + #25 + #32) 2026-05-28 11:52:05 -07:00
Ben Stull 493d6b6eee Release v0.21.0: UX-polish wave (#31 foundation + #24 + #25 + #32)
Token foundation + App.css sweep, header Philosophy rename, inbox
inline-SVG icon + light UX pass, session/transcript page collapse +
metadata header. Pure frontend; 332 backend tests green; build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 11:36:55 -07:00
Ben Stull 959fc906de Merge feature/v0.21.0-docs-32 into v0.21.0 polish wave 2026-05-28 11:34:28 -07:00
Ben Stull b648b3ed45 Merge feature/v0.21.0-header-inbox into v0.21.0 polish wave 2026-05-28 11:34:28 -07:00
Ben Stull 54736de91c Merge feature/v0.21.0-appcss-sweep into v0.21.0 polish wave 2026-05-28 11:34:28 -07:00
Ben Stull adb5d25715 docs(#32): inline-collapse session roots + transcript metadata header
Roadmap item #32 (session/transcript page polish) plus the /docs
surfaces' share of #31, for rfc-app v0.21.0.

- DocsSessionIndex: the session root (/docs/sessions/:nnnn) no longer
  renders a dead-end "N transcript(s) — select from the nav"
  placeholder. It now renders a transcript INLINE: the lone transcript
  for single-transcript sessions, or the `.0` driver transcript (falling
  back to first-by-sort) for multi-transcript sessions, with the
  remaining siblings listed/linked above the body. URL stays stable to
  the session number — inline render, no 301.

- DocsSessionTranscript: adds a compact metadata header above the body
  (title; started/ended parsed from the filename's ISO segments,
  human-readable; derived duration; optional TL;DR from the manifest's
  `tldr` string field, graceful-degrade when absent; external
  "View source on git.wiggleverse.org" link). The parse/header helpers
  are exported so the inline-collapse view reuses identical rendering.

- Docs.css (new): token-based styling for the new metadata-header +
  sibling-list elements only; existing docs classes stay owned by
  App.css to avoid racing the #31 sweep.

- DocsUserGuide: loading/error states brought onto the shared
  .docs-empty/.docs-error convention with a retry button.

No backend change: /api/docs/sessions/manifest already passes the full
sessions.json entry through, so a `tldr` field on an entry reaches the
frontend with no docs_sessions.py change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 11:33:03 -07:00
Ben Stull cbf02d5507 v0.21.0: sweep App.css + index.css to design tokens (#31)
Replace hardcoded colors, font-sizes, and radii in App.css/index.css
with the tokens.css design-token system. Consolidate ~80 distinct
hex values onto the neutral ramp + semantic/status families, map
font-size literals to the --text-* scale and border-radius literals
to the --radius-* scale, route the on-dark translucent-white pattern
and header band through their semantic tokens, and point the base
rules at --color-bg/--color-text/--font-sans.

Add an appended interaction-polish layer: a coherent transition
vocabulary (var(--motion-base) var(--ease-out)) on surfaces that
already react to hover, plus one consistent :focus-visible ring using
var(--color-focus-ring). No existing selector renamed or removed;
only property values changed and additive rules appended.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 11:32:17 -07:00
Ben Stull 317738ed79 v0.21.0 header+inbox UX (#24,#25,#31): rename About→Philosophy, inline-SVG inbox icon, light inbox pass
- #24: header link "About" → "Philosophy" (route/title unchanged).
- #25 icon: replace 📮 emoji with a dependency-free inline-SVG envelope
  in .inbox-trigger; aria-label/title reframed to "Inbox". Badge intact.
- #25 inbox UX (light, no redesign): sharper unread/read distinction
  (accent dot + left bar + tint via tokens), per-row "mark as read"
  affordance that marks-without-navigating, clearer "Mark all read"
  label, and a real empty/caught-up state. New Inbox.css is tokenized
  and written one notch more specific than App.css where it overrides
  (Inbox.css injects before App.css under ESM eval order). Data flow,
  API calls, routing-on-click, and badge behavior preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 11:31:34 -07:00
Ben Stull 5be2c48afe v0.21.0 foundation: design-token module (#31)
Establish src/styles/tokens.css — the single source of truth for color,
type, spacing, radius, elevation, and motion. Imported first in main.jsx.
Sweep subagents map literal values to these tokens; no appearance change
is intended beyond consolidating near-duplicate grays (operator reviews
before deploy).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 11:26:56 -07:00
Ben Stull cbc9949972 Merge feature/v0.20.0-docs-specs-nav-hierarchy 2026-05-28 10:48:38 -07:00
Ben Stull e0d9ed7c5a Release v0.20.0: /docs/specs surface + nested flyout nav + session body-list removal
Wave 9 follow-up to roadmap item #30 (Session 0017.0 shipped #30 as v0.19.0;
v0.20.0 lands the operator-feedback follow-ups on top).

1. Specs on /docs/specs/<name> (backend docs_specs.py + frontend DocsSpec.jsx
   + DocsSpecsIndex.jsx). Configured via OHM_DOCS_SPECS; framework default
   carries OHM's two specs (rfc-app/SPEC.md + flotilla SPEC.md). Runtime
   fetch from gitea raw with 5-min TTL cache, mirroring docs_sessions.py.

2. Nested flyout nav hierarchy (DocsLayout.jsx). Sessions render as a tree
   with transcripts nested under each session row (labeled by .N ordinal).
   New Specs section between User Guide and Sessions.

3. /docs/sessions/<NNNN> body-list removed (DocsSessionIndex.jsx). Body
   becomes a session-overview card; navigation lives in the left nav.

19 new pytest cases for docs_specs (332 backend total green). Frontend
build clean. Sync frontend/package-lock.json version drift (0.15.0 → 0.20.0)
alongside the VERSION + package.json bump.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 10:48:31 -07:00
Ben Stull 69a166a6f2 Merge feature/v0.19.0-docs-sessions-browser 2026-05-28 09:14:56 -07:00
Ben Stull bb5137f176 CHANGELOG: normalize 0.19.0 header (drop 'v' prefix to match surrounding style) 2026-05-28 09:11:58 -07:00
Ben Stull 477f496cbf Release v0.19.0: /docs nav + on-site sessions browser
VERSION + CHANGELOG bump for roadmap item #30. The frontend
package.json was bumped alongside the frontend slice; this commit
finalizes the canonical VERSION at 0.19.0 and prepends the v0.19.0
CHANGELOG entry with the operator upgrade-steps block
(OHM_SESSION_HISTORY_RAW_BASE + the two TTL knobs; all MAY) and the
note about graceful degradation when the session-history repo is
still flat at deploy time (subsession 0017.2 ships the restructure
in parallel).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:07:27 -07:00
Ben Stull 822f4266f6 v0.19.0 frontend: /docs/* route tree + flyout nav + sessions browser
Reorganizes the /docs surface from a single DOCS.md route into a hub
with a left-side flyout nav and three new sub-routes for the on-site
sessions browser (roadmap item #30):

  /docs                      → redirect to /docs/user-guide
  /docs/user-guide           → existing DOCS.md content
  /docs/sessions             → redirect to /docs/sessions/about
  /docs/sessions/about       → README.md of ohm-session-history
  /docs/sessions/<NNNN>      → per-session transcript index
  /docs/sessions/<NNNN>/<f>  → per-transcript view

The flyout is a persistent left sidebar on desktop and a slide-out
drawer on mobile (toggled by a ☰ button in the docs header). Its
session list is driven by the /api/docs/sessions/manifest fetch —
loading shows a skeleton; 502 shows an inline retry; empty manifest
shows only the "About" row.

Each sub-route owns its own empty-state / error handling:
  - 404 transcripts render "This transcript isn't published yet"
    with a link back to the parent session index, no JS crash.
  - 502 (gitea unreachable) renders a retry button.
  - Manifest 404 is mapped to {} server-side so the flyout renders
    cleanly with no error banner when no sessions are published yet.

Analytics (per SPEC §21):
  - new EVENTS.DOC_VIEWED ("Doc Viewed") fires on each sub-route
    mount with `section`: 'user-guide' | 'sessions/about' |
    'sessions/<NNNN>' | 'sessions/<NNNN>/<filename>'.
  - every interactive nav element carries aria-label +
    data-amp-track-name so autocapture rows are readable.

The existing v0.14.0 Docs.jsx component is dropped — its content
moved verbatim into DocsUserGuide.jsx; the new layout subsumes the
back-button + signed-out home affordances it used to carry. All
four new sub-routes reuse the existing MarkdownPreview renderer
(marked + mermaid lazy-load) so no second markdown library lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:07:20 -07:00
Ben Stull 39e57706d9 v0.19.0 backend: /api/docs/sessions/* endpoints + TTL cache
Adds the four read endpoints the v0.19.0 /docs/sessions/* surface
mounts on top of:

- GET /api/docs/sessions/manifest      → sessions.json (title map)
- GET /api/docs/sessions/about         → README.md
- GET /api/docs/sessions/<NNNN>/index  → per-session transcript list
- GET /api/docs/sessions/<NNNN>/<file> → transcript body

The framework mediates the gitea fetch so the rendered surface
inherits the same chrome as the v0.14.0 /docs route and the browser
makes no cross-origin call. Reads are aggressively cached in-process
(60s for the manifest, 5min for content) so the framework doesn't
hammer git.wiggleverse.org under normal traffic. Negative results
(gitea 404) are also cached at the content TTL to absorb the
expected empty-state at deploy-time (the parallel
ohm-session-history repo restructure ships in driver subsession
0017.2). All four endpoints are anonymous-reachable, sibling to
/api/philosophy and /api/docs.

Path validation gates the network: only /^\d{4}$/ session dirs and
the full SESSION-NNNN.M-TRANSCRIPT-...md filename shape pass to the
upstream. Legacy flat-root names (e.g. SESSION-A-TRANSCRIPT.md) and
path-traversal attempts are rejected 400 before any fetch.

18 new tests cover the happy paths, 404 empty-states, 502 upstream
errors, path-validation rejections, and the cache-hit-within-TTL
contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:07:01 -07:00
Ben Stull ac3513a686 Merge: CONTRIBUTING.md + SPEC.md §21 analytics chapter (Session 0013.1)
Docs-only merge from feature/contributing-and-spec-analytics @ 213f686:
- CONTRIBUTING.md (407 lines, new) — how to contribute to rfc-app:
  branch naming, CHANGELOG strict-descending, RFC 2119 upgrade-steps,
  SPEC.md §19.2 candidate hygiene, test-coverage expectations,
  analytics instrumentation checklist (rides #21 Part B), operator-
  only gestures (including the 'never ask for secret bytes' rule).
  Cites Sessions E/I/K/L (= 0005.0/0009.0/0011.0/0012.0) as worked
  examples.
- SPEC.md §21 (469 lines, new) — Analytics instrumentation and
  identity. Ten subsections covering event taxonomy, required prop
  families, autocapture-friendly DOM patterns, replay masking,
  consent-gate contract, identity lifecycle (Part C), set vs
  setOnce taxonomy, cohort implications, overlay-binding rule for
  VITE_AMPLITUDE_API_KEY, §19.2 candidates from this chapter.

No code change, no version bump. Authored by subsession M.1
(= 0013.1) of Session M (= 0013.0); reviewed in Session 0014.0;
merged in Session 0014.0 per operator delegation.
2026-05-28 08:29:41 -07:00
Ben Stull 31913b1e53 Merge feature/v0.18.0-email-webhook-hygiene
v0.18.0 — email + webhook hygiene per the proposal at
~/git/ohm-infra/RFC-APP-EMAIL-HYGIENE-PROPOSAL.md. All five
slices ship; 295 tests passing.
2026-05-28 07:39:39 -07:00
Ben Stull 0562d53f86 Release 0.18.0: email + webhook hygiene (roadmap items #18 + #20)
VERSION + frontend/package.json -> 0.18.0. CHANGELOG entry with
the binding Upgrade-steps block per SPEC.md §20.4.

This release lands all five slices of the v0.18.0 proposal at
~/git/ohm-infra/RFC-APP-EMAIL-HYGIENE-PROPOSAL.md:

  Slice 1: build_envelope helper + unit tests.
  Slice 2: migrate send paths; add POST /api/email/unsubscribe
           for RFC 8058 one-click.
  Slice 3: mandatory GITEA_WEBHOOK_SECRET (+ dev-bypass) +
           unknown-repo logging.
  Slice 4: outbound_emails audit table + admin endpoint.
  Slice 5: bounce correlation via Message-ID.

Full suite: 295 passed (was 252 pre-release).

Upgrade-steps (RFC 2119) block in CHANGELOG covers:
  * MUST: GITEA_WEBHOOK_SECRET non-empty at startup.
  * MAY: RFC_APP_INSECURE_WEBHOOKS=1 for local dev only.
  * MAY: EMAIL_UNSUBSCRIBE_MAILTO to route opt-out courtesy mail
         to a different mailbox than EMAIL_FROM.
  * MUST: apply migration 020_outbound_emails.sql (auto-applied
          on next start; no operator action).
  * MUST: rebuild frontend + restart backend.
  * SHOULD: run mail-tester.com probe post-upgrade.
2026-05-28 07:39:33 -07:00
Ben Stull 4666c4abe7 v0.18.0 Slice 5: bounce correlation hook
The `POST /api/webhooks/email-bounce` body accepts a new optional
`message_id` field. When supplied, the handler looks up the
matching row in `outbound_emails` and stamps status='bounced' +
appends "bounce (<kind>)" to the error column. The response
gains a `correlated_id` field (nullable: null when no
message_id was provided or no row matched).

Hard-bounce -> `email_opt_out_all = 1` still fires regardless
(the v1 contract from api_notifications.py:486-498); Slice 5
just adds the per-message audit trail on top.

Providers that don't surface Message-ID get the legacy v1
behavior — match by email, flip the global opt-out, leave
correlated_id null. Providers that replay an old bounce with a
message_id the framework no longer has log an INFO line but
still 200 (the bounce path can't refuse just because a row was
pruned).

Test update: `test_bounce_webhook_refuses_unsigned_when_secret_configured`
in test_e2e_smoke.py was asserting on the exact response shape
{ok, matched}; v0.18.0 adds `correlated_id`. The test now asserts
on the new shape; documented in CHANGELOG.

4 new tests covering: bounce with message_id stamps the row;
unknown message_id is logged + does not crash; bounce without
message_id still flips opt-out (backward compat); bounced rows
surface in the admin endpoint with `?status=bounced`.

Full suite: 295 passed.
2026-05-28 07:36:02 -07:00
Ben Stull 281a844513 v0.18.0 Slice 4: outbound_emails audit table + admin endpoint
Per the v0.18.0 email + webhook hygiene proposal §3:

  * `backend/migrations/020_outbound_emails.sql` — new table
    capturing every send attempt: to_address, from_address,
    subject, kind, sent_at, status ('sent' | 'failed' | 'deferred'
    | 'bounced'), error, notification_id (FK), message_id (for
    Slice 5 bounce correlation).
  * `email.record_outbound()` — best-effort write helper every
    send path calls. Status='sent' on SMTP success, 'failed' on
    exception (with class + message in `error`), 'deferred' on
    the dev-fallback path where SMTP_HOST is unset (the send
    didn't happen but the row records the attempt).
  * `email._deliver` (watcher notifications + bundles), `digest.py`,
    `email_otc.py`, `email_invite.py` — every send path now records.
  * `GET /api/admin/outbound-emails` — admin-only listing,
    filterable by kind / status / to_address. Answers "did this
    person ever get their invite?" without grepping VM logs. No
    admin UI in v0.18.0; operator queries via curl + jq for now.

7 new integration tests covering: OTC / invite / notification all
write rows with matching Message-ID; admin endpoint lists,
filters by kind, filters by to_address (case-insensitive),
refuses non-admins.

Full suite: 291 passed.
2026-05-28 07:32:31 -07:00
Ben Stull d3daa97264 v0.18.0 Slice 3: webhook tightening — mandatory secret + dev-bypass
`GITEA_WEBHOOK_SECRET` is now mandatory at startup. The framework
refuses to load_config() when the env var is empty unless the
operator opts into the dev-bypass with `RFC_APP_INSECURE_WEBHOOKS=1`.
This is the v0.18.0 startup-loud-failure shape — the pre-v0.18.0
"silently accept unsigned POSTs when the secret is empty" path is
the bug the proposal targets.

`webhooks.receive`:
  * Defense in depth: refuses 500 if the secret is empty at request
    time and the dev-bypass is not set (catches the case where
    something mutates env after startup).
  * Logs a loud warning every time a webhook lands under the
    bypass — so a misconfigured production deployment shows up in
    the logs even if the operator missed the warning at boot.
  * Adds an INFO log at the unknown-repo branch (previously
    silently 200-OK'd a hook on a fork or a stale Gitea binding).

`tmp_env` fixture binds a fake secret so the existing 277 tests
boot cleanly; the new test_webhooks_vertical.py exercises both
the production-secret path (valid signature, invalid signature,
missing signature) and the dev-bypass path (config loads with
empty secret when bypass set, refuses without it).

7 new tests; full suite: 284 passed.
2026-05-28 07:27:23 -07:00
Ben Stull e9fdc478f6 v0.18.0 Slice 2: migrate send paths to build_envelope + POST unsubscribe
Five send sites now construct their envelopes through
`email_envelope.build_envelope` instead of building `EmailMessage`
ad hoc:

  * `email_otc.py` — OTC mail (no List-Unsubscribe per the
    proposal's tradeoff: the recipient explicitly requested the
    code, so a list semantic would be wrong).
  * `email_invite.py` — admin/per-RFC invite mail (mailto:-only
    List-Unsubscribe — the invitee isn't a user yet, so no
    per-user opt-out URL exists).
  * `email._deliver` — watcher notifications (full one-click
    unsubscribe: mailto + signed URL + List-Unsubscribe-Post per
    RFC 8058, required by Gmail/Yahoo).
  * `email._send_bundle` — the "while you were away" bundle,
    one-click to the new `all` synthetic category (which lands
    `email_opt_out_all = 1` because the bundle spans multiple
    per-category flags).
  * `digest.py` — same as the bundle: bulk-adjacent, one-click to
    `all`.

`api_notifications.py` gains the POST `/api/email/unsubscribe`
endpoint (the matching receiver for `List-Unsubscribe-Post:
List-Unsubscribe=One-Click`) and accepts the `all` category in
both GET and POST handlers.

`EmailConfig` adds `unsubscribe_mailto` (env: `EMAIL_UNSUBSCRIBE_MAILTO`,
default = `EMAIL_FROM`) so deployments can route unsubscribe
courtesy mail to a humans-monitored mailbox distinct from the
no-reply sender.

The `_SENT` test buffer now also carries `envelope["message"]`
(the `EmailMessage` itself) so new tests can assert on headers
directly. Legacy `to`/`from`/`subject`/`body` keys remain for
backward compatibility.

10 new integration tests across test_otc_vertical /
test_admin_create_user_invite_vertical / test_notifications_vertical
covering: OTC has no List-Unsubscribe; invite has mailto: only;
notification has full one-click; POST one-click flips the
category; `all` category sets global opt-out via both GET and
POST.

Full suite: 277 passed.
2026-05-28 07:24:03 -07:00
Ben Stull 92059f319e v0.18.0 Slice 1: build_envelope helper + unit tests
Per the v0.18.0 email + webhook hygiene proposal
(~/git/ohm-infra/RFC-APP-EMAIL-HYGIENE-PROPOSAL.md §1), this is the
shared envelope builder every send path will call in Slice 2. No
call-site changes yet — Slice 2 migrates email_otc / email_invite /
email._deliver / email._send_bundle to use it.

The helper centralizes the deliverability-critical headers (Date,
Message-ID, Auto-Submitted) and exposes per-kind unsubscribe
semantics (none for OTC, mailto: for invites, full one-click for
watcher notifications) as explicit kwargs rather than buried in
each call site.

15 new unit tests covering: always-present headers, Auto-Submitted
toggle, the three List-Unsubscribe shapes, plain-only vs
multipart/alternative body. Full suite: 267 passed (was 252).
2026-05-28 07:15:06 -07:00
79 changed files with 9349 additions and 881 deletions
+600
View File
@@ -23,6 +23,606 @@ skip versions are the composition of each intervening adjacent
release's steps in order — no A-to-B path is pre-computed beyond
that.
## 0.28.0 — 2026-05-28
**Minor — security-hardening follow-up (Session-0026 audit, informational
findings I3 + I4). No operator action required: no migration, no schema
change, no config/overlay change, no API/behavior change for any caller.
A plain code deploy picks it up. Shipped from driver session 0032.0.**
Two informational findings from the Session-0026 audit, both
framework-internal defense-in-depth:
- **I3 — dead HTML-email branch guarded.** `email_envelope.build_envelope`
accepted a `body_html=` argument that built a `multipart/alternative`
body, but no send path ever passed it — every rfc-app mail is plain
text. An unused branch that would emit HTML built from (potentially
unescaped) user content is the C1 stored-XSS class waiting in the mail
channel. The branch is now a loud guard: passing `body_html` raises
`NotImplementedError`. The argument is kept in the signature for
documented future symmetry; enabling HTML mail becomes a deliberate
change that MUST HTML-escape user content at the call site and remove
the guard in the same commit.
- **I4 — Turnstile siteverify no longer blocks the event loop.**
`turnstile.verify_token` was a synchronous function issuing a blocking
`httpx.post` from inside the async `/auth/otc/request` handler, so a
slow CloudFlare response stalled the single worker for up to the 10s
timeout. It is now `async` and awaits the call on an
`httpx.AsyncClient` (matching the codebase's existing async-httpx
pattern), isolated behind a narrow `_siteverify_post` seam. The sole
caller (`main.py`) now `await`s it.
Upgrade steps: **none.** Both changes are internal. The `verify_token`
signature changed from sync to `async` (callers must `await`), but the
only caller is in-tree (`main.py`) and is updated in this release; no
deployment-facing surface, config key, or migration is affected.
## 0.27.0 — 2026-05-28
**Minor — security-hardening release (Session-0026 audit remediation).
One auto-applied migration (023); one behavior change that re-prompts
device-trust; deployments MUST re-apply the nginx + systemd files.**
This is the work cut as the "v0.25.0 security-hardening" branch; it
reversioned to 0.27.0 because v0.26.0 (#28) took the next slot while it
was in flight. Shipped from driver session 0030.0.
- **C1 (Critical) — stored-XSS closed.** Every markdown→HTML sink now
routes through one chokepoint, `frontend/src/lib/sanitizeHtml.js`
(DOMPurify), before any `innerHTML` / `dangerouslySetInnerHTML` write:
`MarkdownPreview`, both `ProposalView` sinks (entry body +
`proposed_use_case`), and `Editor`. A hook adds
`rel="noopener noreferrer"` to `target=_blank` links. `marked` no
longer passes raw HTML / `javascript:` URIs to the DOM, so a
contributor can no longer plant a payload that runs in an admin/owner
session during review.
- **H1 — OTC verify is rate-limited.** New `backend/app/ratelimit.py`
(per-IP token buckets) gates `/auth/otc/verify`, `/auth/otc/request`,
and the passcode check/verify paths; a per-account OTC-verify lockout
(migration `023_otc_verify_lockout.sql`) mirrors the passcode lockout.
- **M1 — device-trust lookup no longer table-scans.** The device-trust
cookie value is now `"<row_id>.<raw_token>"`; `device_trust.lookup`
reads the one indexed row and bcrypt-checks only it, instead of
bcrypt-checking every row in the table on each unauthenticated
`/auth/device-trust/start`.
- **M2 — HTTP security headers** (CSP, HSTS, X-Frame-Options,
X-Content-Type-Options, Referrer-Policy) added to the nginx server
block. **L8/I1**: `server_tokens off` + legacy TLS1.0/1.1 removed.
- **M4 — session cookie `Secure` by default** (`SESSION_COOKIE_SECURE`,
defaults on; a dev box on plain http sets it `false`).
- **M5 — bounce webhook fails closed.** An unset
`WEBHOOK_EMAIL_BOUNCE_SECRET` now **disables** `/api/webhooks/email-bounce`
(503) instead of leaving it open; a dev opts back in with
`RFC_APP_INSECURE_BOUNCE_WEBHOOK=1`.
- **L2/L3** per-IP cooldown + check-endpoint throttle. **L4** systemd
sandbox knobs (`CapabilityBoundingSet=`, `ProtectKernel*`,
`RestrictAddressFamilies`, `SystemCallFilter`, …).
Upgrade steps:
1. **Migration** — none manual; `023_otc_verify_lockout.sql` auto-applies
at startup via `db.run_migrations`.
2. **Device trust (MUST expect re-prompt)** — the cookie format changed,
so existing "trusted device" cookies no longer match; affected users
are re-prompted for device verification once. No data migration; old
rows are simply never matched and age out.
3. **nginx + systemd (MUST apply out-of-band)** — the deploy gesture does
**not** install `deploy/nginx/ohm.wiggleverse.org.conf` or
`deploy/systemd/rfc-app.service`. After deploying the code, copy both
to their system locations, then `nginx -t && systemctl reload nginx`
and `systemctl daemon-reload && systemctl restart <unit>`. (M2 headers
and L4 sandboxing do not take effect until this is done.)
4. **Bounce webhook (SHOULD)** — bind `WEBHOOK_EMAIL_BOUNCE_SECRET` (or
set `RFC_APP_INSECURE_BOUNCE_WEBHOOK=1` for dev). Unset → the endpoint
returns 503 (closed). No legitimate bounce source is wired today, so
503 is the safe default.
5. **Session cookie (SHOULD, dev only)** — a deployment served over plain
http MUST set `SESSION_COOKIE_SECURE=false` or the session cookie
won't be sent. Production over HTTPS leaves it unset (Secure on).
## 0.26.0 — 2026-05-28
**Minor — no schema migration, no new secret, no config, no upgrade
steps. Additive read-time enrichment; deployments inherit it on deploy
with nothing to set.** Roadmap item #28 **Part 1**: references to existing
**accepted** RFCs inside PR descriptions and comment text now render as
inline links to the referenced RFC. Shipped from driver session 0029.0,
in parallel with the v0.25.0 security-hardening session — hence the
version-slot gap (0.25.0 is that session's; this took the next free slot
per the roadmap's "claims the next available version number" rule).
Parts 2 (offer-to-create-RFC for strong-candidate terms) and 3
(offer-to-contribute-to-a-pending-RFC) of item #28 are deliberately **not**
in this release — Part 1 ships first as the easy win, exactly as the
roadmap row anticipates.
- **Where it links.** The PR review page's description and review
comments (`GET /api/rfcs/{slug}/prs/{n}`), and the PR-less per-RFC
discussion comments (`GET /api/rfcs/{slug}/discussion/threads/{id}/messages`).
Branch-chat (the AI-collaboration surface) is intentionally out of
scope for Part 1 — a noted follow-up.
- **Read-time, not submit-time.** The roadmap row phrases the scan as
happening "at submit/post time"; this ships it as **read-time**
enrichment instead. The intent the row actually names — "not as live
compose preview" — holds (drafts are never scanned, only submitted
content on read). Read-time was chosen because (a) links track the
*live* accepted-RFC set — a newly-accepted RFC starts linking in older
comments, a withdrawn one stops linking everywhere — rather than
freezing stale at submit; (b) it needs **no migration** (no derived
data to persist); (c) the active-RFC corpus is small and cache-resident,
so per-read scanning is cheap. (Recorded as a §19.3-rule-2 spec note in
the session transcript.)
- **XSS-safe by construction.** The backend returns structured *segments*
(a list of `{type:"text"}` / `{type:"rfc", slug, label, title}` items),
never HTML. The new `LinkedText` frontend component maps segments onto
React text nodes and anchors — no `dangerouslySetInnerHTML` — so a
comment author cannot inject markup through this path. Every enriched
field keeps its raw `text`/`description` alongside the `*_segments`, so
any non-segment-aware caller is unaffected.
- **Conservative matching (false-positive-averse).** A reference links
only when it is unlikely to be coincidental: an `rfc_id` token
(`RFC-0001`), a **multi-word** title (`Open Human Model`), or a
**hyphenated** slug (`open-human-model`). A single common-word title or
slug (a hypothetical RFC titled "Human") is **not** auto-linked — that
would turn every prose "human" into a link. Matching is case-insensitive,
word-boundary-anchored, longest-match-wins, and suppresses an RFC's
self-references inside its own PR/discussion. The roadmap's "curated
canonical-terms list" remains an explicit future opt-in rather than a
guessed-at default. New module: `backend/app/rfc_links.py`.
Tests: 12 new (`test_rfc_links_vertical.py`) — 9 scanner units (gating
rules, word boundaries, longest-match, case-insensitivity, casing
preservation, the rfc_id/multi-word/hyphenated gates) plus 3 end-to-end
(PR description + review comment + discussion comment all surface
`*_segments`; self-reference suppression). Full suite 363 green; frontend
builds clean.
Upgrade steps:
1. None. This release is purely additive: no migration, no new
environment variable, no secret, no overlay. A deployment picks up RFC
auto-linking the moment it deploys this version. (RFCs only link once
they are in the `active` state — proposed/super-draft and withdrawn
RFCs are never link targets, matching the §11.3 universal-public read
rule.)
## 0.24.0 — 2026-05-28
**Minor — one new secret required before this version serves tag
suggestions; no schema migration; no behavior change for deployments
that do not bind the key.** Roadmap item #27: as a contributor fills in
the propose-RFC form, the backend asks Claude Haiku to recommend tags
drawn from the collection's existing tag set, surfaced as clickable
suggestion chips. Shipped from driver session 0025.0. This is the §9.1
"Slice 2" AI-suggested chips that the propose modal has carried a
deferred placeholder for since Slice 1.
- **`POST /api/rfcs/suggest-tags`** (contributor-gated, per-user
rate-limited). Takes the partial draft (`title`, `pitch`,
`use_case`) and returns `{ "suggestions": [{ "tag", "confidence" },
…] }`. The model is constrained to the corpus's existing tags — v1
tags are free-form chip input, so "the taxonomy" is the de-facto set
of distinct tags the existing RFCs carry. The model MUST NOT invent
tags; taxonomy extension is a deliberate out-of-scope follow-up.
- **Always Claude Haiku, for cost.** Tag suggestion uses Haiku
regardless of the `ENABLED_MODELS` chat-picker universe, via the new
`providers.construct_haiku()` factory. There is no RFC slug at propose
time, so the §6.7 per-RFC funder credential path does not apply — the
surface runs on the operator's own `ANTHROPIC_API_KEY`.
- **Degrades to silence, never error.** No key bound, an empty corpus,
an empty draft, a rate-limited caller, or an unparseable model reply
all yield an empty list; the propose modal hides its suggestion row,
and the rest of the app is unaffected. So a deployment that does not
set the key sees no change at all.
- **Privacy disclosure.** The modal carries an inline note that the
draft text is sent to Anthropic to generate the suggestions. This is
required for honesty (the text leaves the deployment before the RFC
is submitted); cookie-consent does not gate it because the user has
actively typed into a draft surface. The exact wording wants a
counsel pass before OHM's deploy, per the #22 drafting discipline.
- **Frontend.** `ProposeModal` debounce-posts the draft (700 ms, with a
stale-response guard); suggestion chips are clickable and add to the
tag list (nothing auto-applies). `api.suggestTags()` is forgiving —
any non-OK response resolves to `[]`.
Tests: 11 new (`test_tag_suggest_vertical.py`) — contributor-gating,
universe-constrained filtering, no-key / empty-corpus / rate-limit
paths, plus units for the universe gather, the tolerant reply parser,
the max cap, the empty-draft short-circuit, and provider-failure
fallback. Full suite 351 green.
Upgrade steps:
1. You **MUST** bind `ANTHROPIC_API_KEY` for the suggestion surface to
work. On OHM this is an operator gesture — the key never touches the
conversation. Set it from your own terminal via stdin (so the bytes
never land in shell history):
```bash
printf '%s' "$(pbpaste)" | .venv/bin/ohm-rfc-app-flotilla \
secret set ohm-rfc-app ANTHROPIC_API_KEY
```
(The §18 chat stack already reads this same key, so a deployment
that has chat configured **MAY** already have it bound — confirm with
`flotilla secret list ohm-rfc-app`.) If the key is absent the app
still boots and serves normally; tag suggestions are simply
unavailable until it is set.
2. You **MAY** tune the per-user rate limit via `TAG_SUGGEST_RATE_MAX`
(default 30) and `TAG_SUGGEST_RATE_WINDOW_SECONDS` (default 60). The
defaults apply when unset.
3. You **SHOULD** have counsel review the modal's disclosure wording
before exposing the surface to users, per the #22 drafting
discipline — the copy is honest as written, but the legal review is
the right call for any "your text is sent to a third party" notice.
## 0.23.0 — 2026-05-28
Roadmap item #29: signing in lands the user on their most recently
viewed app state instead of the empty-state home view. Shipped from
driver session 0022.0. Per-user, server-side, on by default; first-ever
sign-ins still land on home.
1. **Server-side last-state.** A new `user_session_state` table holds one
row per user: `last_route TEXT`, `last_route_state TEXT` (light view
state encoded as JSON — SQLite has no native JSONB — never draft-buffer
contents), `resume_enabled INTEGER NOT NULL DEFAULT 1`, and
`last_updated_at`. Per-user (not per-device), matching the
"go to where I was last" intent.
2. **Route-change capture.** A new frontend hook (`frontend/src/lib/
useLastState.js`) debounce-posts the current route + light state to
`PUT /api/me/last-state` for authenticated users only; anonymous
sessions are a no-op. The handler upserts the user's row and no-ops
when `resume_enabled` is 0.
3. **Sign-in redirect.** The stored `last_route` (+ decoded state +
`resume_enabled`) is folded onto the existing `/api/auth/me` payload
(no new GET endpoint), so the frontend reads it on boot. On a hard
sign-in landing (app booted on `/`), the app redirects to the stored
route. The redirect is gated on the #21-Part-C Amplitude `identify`
having fired (`identifyReady`), preserving identify-then-track
ordering — the first event on the resumed route carries the user_id.
4. **Edge cases.** Stale routes (an RFC since withdrawn or now
unreadable) are a graceful no-op — existing routing falls through to
the catalog/empty-state. Opt-out ships at the column level
(`resume_enabled`); the profile-settings toggle UI is a follow-up.
Stored state is route + light view state only, never draft contents
(documented in `SPEC.md` §6.8).
Migration `022_user_session_state.sql` creates the table. No new
environment variables, overlay keys, or secrets.
Upgrade steps:
1. Deployments **MUST** apply database migrations; `022_user_session_state.sql`
runs automatically on next boot (the migration runner globs
`backend/migrations/*.sql` and applies any not yet recorded in
`schema_migrations`). The migration is additive — a new table — and
requires no data backfill.
2. No new environment variables, overlay keys, or secrets. No operator
action beyond the standard `flotilla deploy ohm-rfc-app`.
## 0.22.0 — 2026-05-28
Roadmap item #26: an optional **"What will you be using this for?"**
field on both propose surfaces. Shipped from driver session 0022.0.
Additive and backward-compatible — no behavior changes for anyone who
leaves the field blank.
1. **Propose-RFC modal.** Below the required "Why is this RFC needed?"
field (`pitch`) sits a new optional **"What will you be using this
RFC for?"** textarea. "Needed" is the abstract justification; "using
it for" is the concrete ground-truth use case — captured without
being forced.
2. **Propose-PR modal.** Below the required "Why is this change needed?"
field (`description`) sits a new optional **"What will you be using
this change for?"** textarea.
3. **Display.** The RFC view (`RFCView` / `ProposalView`) and PR review
view (`PRView`) render the captured use case alongside the existing
"why" text, with a muted "left blank" treatment when absent.
4. **Persistence (deployment note).** In this deployment the framework's
`rfcs` / PR surfaces are the Gitea-backed cache tables `cached_rfcs`
/ `cached_prs`, rebuilt by the reconciler. Because the propose write
path is endpoint → Gitea → reconcile (and the reconciler doesn't
carry the new field), the durable home is a canonical, reconcile-proof
side table `proposed_use_cases` (keyed by `(scope, pr_number)`) that
the propose/open endpoints write directly and the view endpoints read
back. The literal nullable `proposed_use_case` columns named by the
roadmap are also added to `cached_rfcs` / `cached_prs` for parity and
any future reconciler that learns to carry the field. NULL/blank use
cases simply never write a side-table row — absence is "left blank".
Migration `021_proposed_use_case.sql` adds the two cache columns
(`ALTER TABLE … ADD COLUMN proposed_use_case TEXT`), the
`proposed_use_cases` canonical table, and its lookup indexes. The
reconciler's upsert (`ON CONFLICT DO UPDATE`) sets only known columns,
so the added cache columns survive reconciles. Backend validators accept
the field as NULL/omitted (no required-validation) with an 8000-char cap
matching the existing PR `description` bound; blank/whitespace is treated
as absent.
Upgrade steps:
1. Deployments **MUST** apply database migrations; `021_proposed_use_case.sql`
runs automatically on next boot (the migration runner globs
`backend/migrations/*.sql` and applies any not yet recorded in
`schema_migrations`). The migration is additive — nullable cache
columns plus a new table — and requires no data backfill.
2. No new environment variables, overlay keys, or secrets. No operator
action beyond the standard `flotilla deploy ohm-rfc-app`.
## 0.21.0 — 2026-05-28
UX-polish wave. Roadmap items #31 (comprehensive UX polish — foundation
slice), #24 (header "About" → "Philosophy"), #25 (inbox icon + light UX),
and #32 (session/transcript page polish). Pure frontend; no schema, no
backend changes, no new secret. Shipped from one driver session (0019.0)
via three parallel subagents working on disjoint surfaces.
1. **Design-token foundation (#31).** New `frontend/src/styles/tokens.css`
establishes the app's first coherent design system — semantic color
palette, type scale, spacing scale, radius scale, elevation, and a
motion vocabulary (with `prefers-reduced-motion` honored) — as CSS
custom properties, imported first in `main.jsx`. Before this the app
carried ~98 distinct hardcoded hex colors, font sizes across 16
unscaled values, and radii across 13. `App.css` and `index.css` were
swept to the tokens (~630 color / 280 font-size / 128 radius
references), consolidating near-duplicate grays to the nearest ramp
step and rounding off-scale type to the nearest step. No CSS class was
renamed or removed; an additive `:focus-visible` ring and a subtle
hover/transition layer were added. 35 special-purpose hexes (true
blues/violets, status dots, deep diff-contrast shades) were
deliberately left as literals. This is the polish *foundation*; a
follow-up (#31b) covers the bespoke per-surface re-spacing that wants
operator review against screenshots.
2. **Header: "About" → "Philosophy" (#24).** The persistent header link
now reads "Philosophy" (the route `/philosophy` and its page already
existed; only the label changed).
3. **Inbox icon + light UX (#25).** The header inbox trigger's `📮`
emoji is replaced with a dependency-free inline-SVG envelope icon
(`aria-label="Inbox"`); no icon library was added. The inbox panel
got a light pass — clearer unread/read distinction, mark-all-read and
per-row affordances surfaced, better empty state, tokenized spacing in
a new component-scoped `Inbox.css`. Behavior, filters, deep-links, and
§15 notification data flow are unchanged. A full inbox redesign is
deferred to a #25 follow-up (the operator's reference screenshot did
not transmit).
4. **Session/transcript page polish (#32).** `/docs/sessions/<NNNN>` no
longer dead-ends on a "select a transcript" placeholder: a
single-transcript session renders that transcript inline at the
session root; a multi-transcript session renders its `.0` driver
transcript inline and lists the siblings. Each rendered transcript now
carries a metadata header — session title, Started/Ended (parsed from
the filename's ISO segments), derived Duration, an optional one-line
TL;DR, and a "View source on git.wiggleverse.org" external link to the
canonical raw transcript. The TL;DR reads an optional `tldr` string on
the per-session `sessions.json` manifest entry and degrades gracefully
when absent.
Upgrade steps:
MAY: add a `tldr` string to any per-session entry in
`wiggleverse/ohm-session-history`'s `sessions.json`
(e.g. `"0019": { "title": "…", "tldr": "one-line summary" }`) to surface
a summary in each transcript's metadata header. Absent `tldr` renders
nothing — no deployment action is required. This is a data edit in the
session-history repo, not a `flotilla` gesture.
## 0.20.0 — 2026-05-28
Wave 9 follow-up to roadmap item #30. Three changes bundled into one minor:
1. **Specs on `/docs/specs/<name>`** — a new public surface alongside the user guide that renders the framework's spec corpus at runtime. Configured via the `OHM_DOCS_SPECS` env var; the framework default carries OHM's two specs (`rfc-app/SPEC.md` and `ohm-rfc-app-flotilla/SPEC.md`) fetched from gitea raw URLs with a 5-minute TTL cache. Each spec page renders the current version only — git is the history surface; a "View source" link beside the title points at the upstream raw URL. Bare `/docs/specs` client-side redirects to the first configured spec (or renders a "no specs configured" empty state if the deployment cleared the list).
2. **Nested flyout nav hierarchy** — `/docs/*` nav now renders sessions as a tree: each session row has its transcripts nested under it as nav children, labeled by their `.N` ordinal (`0014.0`, `0014.1`, …). Each session's transcript index is fetched alongside the manifest on layout mount (Promise.all over the manifest's keys); the backend's 5-minute content TTL makes the repeat cost negligible. Always-expanded — at the current scale (≤20 sessions) lazy expansion isn't worth the click. A new "Specs" section sits between User Guide and Sessions, populated by the new manifest endpoint.
3. **`/docs/sessions/<NNNN>` body-list removed** — operator preference: navigation lives in the left nav, not in body content. The per-session page is now a session-overview card (title + transcript count + "select a transcript from the navigation" hint). Empty-state, not-found, and error paths preserved; only the inline transcript-link list is gone.
Upgrade steps:
MAY: `flotilla overlay set ohm-rfc-app OHM_DOCS_SPECS='<JSON array>'` to override the configured spec set. Each entry is `{"name": "<slug>", "title": "<human label>", "url": "<gitea raw URL>"}`. Malformed JSON, a non-array root, or an entry that fails validation (missing fields, non-slug `name`) logs a warning and falls back to the framework default; deployment startup is never crashed by a bad value.
MAY: `flotilla overlay set ohm-rfc-app OHM_DOCS_SPECS_CONTENT_TTL_SEC=300` to tune the per-spec content cache TTL.
Note on the `frontend/package-lock.json` version drift fix: the lockfile's top-level and `packages.""` version fields drifted to `0.15.0` somewhere in the v0.16.0v0.19.0 window and weren't caught. This release syncs them to `0.20.0` alongside `frontend/package.json` and `VERSION`. No dependency changes; only the version-string fields move.
## 0.19.0 — 2026-05-28
Roadmap item #30: docs nav with on-site sessions browser. Adds a left-side flyout nav on `/docs/*` and three new public surfaces — `/docs/sessions/about` (renders the session-history README), `/docs/sessions/<NNNN>` (per-session index), `/docs/sessions/<NNNN>/<filename>` (per-transcript view). Backend mediates the fetch from `wiggleverse/ohm-session-history` over gitea raw URLs with a small in-process TTL cache (60 s manifest, 5 min content; both env-tunable). Existing `/docs` content moves to `/docs/user-guide`; bare `/docs` redirects.
Upgrade steps:
MAY: `flotilla overlay set ohm-rfc-app OHM_SESSION_HISTORY_RAW_BASE=<url>` if the deployment points at a non-OHM transcript repo. Default in code matches OHM's `wiggleverse/ohm-session-history`.
MAY: `flotilla overlay set ohm-rfc-app OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC=60` and `OHM_DOCS_SESSIONS_CONTENT_TTL_SEC=300` to tune cache TTLs.
Note: this release depends on the parallel restructure of `wiggleverse/ohm-session-history` into per-session `NNNN/` folders + `README.md` + `sessions.json` (driver session 0017.0, subsession 0017.2). If the repo is still flat at deploy time, `/docs/sessions/about` and the per-session pages return 404 and the route tree degrades to "About not yet published" — no JS crashes; the User Guide remains fully functional.
## 0.18.0 — 2026-05-28
**Minor — schema migration required; one env var now mandatory; no
new secrets.** This release lands the framework-side half of OHM
roadmap items #18 (Secure the SMTP relay + Gitea webhook) and #20
(Email deliverability). It is the framework counterpart to the
operator-side SMTP / DNS hardening covered in the
`EMAIL-AND-WEBHOOK-HARDENING-RUNBOOK.md` companion doc.
The release is shipped in five atomic slices per the v0.18.0 proposal:
1. **`build_envelope` shared helper.** A single place where every
outbound `EmailMessage` is constructed. Lands the
deliverability-critical headers (`Date`, `Message-ID`,
`Auto-Submitted`) uniformly across OTC, invite, watcher
notification, bundle, and digest paths; exposes per-kind
unsubscribe semantics (none for OTC, mailto: for invites, full
one-click for bulk-adjacent paths) as explicit kwargs.
2. **Migrated send paths.** `email_otc.py`, `email_invite.py`,
`email._deliver`, `email._send_bundle`, and `digest.py` now
build their envelopes through the helper. Per-RFC invite
(v0.16.0) rides through `email_invite.py`'s helper and picks
up the change for free. The shared `_SENT` test buffer also
carries the constructed `EmailMessage` under `envelope["message"]`
so tests can assert on the header surface directly.
3. **Webhook handler tightening.** `GITEA_WEBHOOK_SECRET` is now
mandatory at startup — the framework refuses to load_config()
if it's empty, unless the operator explicitly opts into the
dev-bypass with `RFC_APP_INSECURE_WEBHOOKS=1`. The
`/api/webhooks/gitea` receiver carries defense-in-depth checks
that surface the misconfiguration loudly at the request layer
too. Mis-targeted webhooks (a hook on a fork or a stale Gitea
binding) now log at INFO instead of silently 200-OK'ing.
4. **`outbound_emails` audit table + admin endpoint.** Every send
helper writes one row to `outbound_emails` before returning,
capturing the send attempt regardless of outcome
(status='sent' / 'failed' / 'deferred'). The new admin endpoint
`GET /api/admin/outbound-emails` (filterable by kind / status /
to_address) lets the operator answer "did this person ever get
their invite?" without grepping VM logs. No admin UI ships
with v0.18.0; operator queries via curl + jq for now.
5. **Bounce correlation.** The `POST /api/webhooks/email-bounce`
body accepts a new optional `message_id` field; when supplied,
the handler stamps status='bounced' on the matching
`outbound_emails` row and returns the row id as
`correlated_id`. The pre-existing hard-bounce ->
`email_opt_out_all = 1` flow still fires.
The `POST /api/email/unsubscribe` endpoint also lands in Slice 2
as the matching receiver for the new `List-Unsubscribe-Post:
List-Unsubscribe=One-Click` header (Gmail and Yahoo POST that
payload on the user's one-click action per RFC 8058 — the GET
endpoint alone is no longer sufficient for senders at OHM's tier).
### Added
- **`backend/app/email_envelope.py`** — the `build_envelope` helper.
Single source of truth for every outbound `EmailMessage`'s
headers + body shape. Standalone module so tests can exercise it
without booting the FastAPI app.
- **`backend/migrations/020_outbound_emails.sql`** — the audit
table. Single new table with three indexes (to_address, sent_at,
message_id); no changes to existing tables.
- **`email.record_outbound(...)`** — best-effort write helper every
send path calls. Catches `RuntimeError` (so pure-helper unit
tests where `db.init()` was never called don't break) and any
other exception (so the audit write never breaks a send).
- **`GET /api/admin/outbound-emails`** in `api_admin.py` —
admin-only listing of `outbound_emails`. Newest-first, filterable
by kind, status, and to_address (case-insensitive). Returns
`{items: [...], has_more}` per the rest of the admin endpoints'
shape.
- **`POST /api/email/unsubscribe`** in `api_notifications.py` —
RFC 8058 one-click receiver. Accepts the same `?t=` token as the
GET handler; idempotent; returns 200 + `{ok, category}` on
success.
- **`all` synthetic category** for unsubscribe URLs. Used by the
bundle + digest paths (which can't honor per-category opt-outs
because they span multiple categories); flips
`email_opt_out_all = 1` rather than a per-category column.
### Changed
- **`backend/app/email.py`** — `EmailConfig` gains
`unsubscribe_mailto` (env: `EMAIL_UNSUBSCRIBE_MAILTO`, default
falls back to `EMAIL_FROM`). `_deliver` and `_send_bundle`
build envelopes through `build_envelope` and thread `kind` +
`notification_id` into the audit write.
- **`backend/app/email_otc.py`** + **`email_invite.py`** — both
call `build_envelope` and `record_outbound`. OTC carries no
`List-Unsubscribe` (recipient explicitly requested the code);
invite carries `List-Unsubscribe: <mailto:…>` only (no signed
URL — the invitee isn't a user yet, no per-user opt-out row
exists).
- **`backend/app/digest.py`** — calls `_deliver` with `kind='digest'`
+ the new `all`-category one-click unsubscribe.
- **`backend/app/webhooks.py`** — refuses 500 at request time if
the secret is empty + bypass isn't set; logs a loud warning
per-request when running under the bypass; logs INFO when a
hook targets a repo not in `cached_rfcs`.
- **`backend/app/config.py`** — `load_config()` raises
RuntimeError if `GITEA_WEBHOOK_SECRET` is empty unless
`RFC_APP_INSECURE_WEBHOOKS=1`.
- **`backend/app/api_notifications.py`** — GET unsubscribe handler
accepts the `all` category (sets `email_opt_out_all = 1`).
Bounce webhook body adds optional `message_id` field; response
shape adds `correlated_id` field. (Tests that read the exact
response shape — currently just
`test_bounce_webhook_refuses_unsigned_when_secret_configured`
in test_e2e_smoke.py — updated to assert on the new shape.)
- **`backend/tests/test_propose_vertical.py`** — the shared
`tmp_env` fixture binds a fake `GITEA_WEBHOOK_SECRET` so all
252 pre-v0.18.0 tests boot cleanly under the new mandatory
secret. Tests that want to exercise the dev-bypass path
monkeypatch `RFC_APP_INSECURE_WEBHOOKS=1` explicitly.
### Tests
- 15 new unit tests in `test_email_envelope.py` for the helper.
- 10 new integration tests across `test_otc_vertical`,
`test_admin_create_user_invite_vertical`, and
`test_notifications_vertical` covering: OTC has no
List-Unsubscribe; invite has mailto: only; notification has
full one-click; POST one-click flips per-category; `all` flips
global; respects `EMAIL_UNSUBSCRIBE_MAILTO` override.
- 7 new integration tests in `test_webhooks_vertical.py` covering
the startup-time mandatory-secret check, the dev-bypass, and
the request-time signature verification including the unknown-
repo log line.
- 11 new integration tests in `test_outbound_emails_vertical.py`
covering the audit table write path (OTC / invite / notification),
the admin endpoint (list, filter by kind, filter by to_address,
non-admin refusal), and the bounce correlation (matched
message_id stamps status='bounced'; unknown message_id is
logged; absent message_id falls back to legacy behavior; bounced
rows surface in admin endpoint with `?status=bounced`).
- One test updated for intentional response-shape change:
`test_e2e_smoke.test_bounce_webhook_refuses_unsigned_when_secret_configured`.
Full suite: 295 passed (was 252 pre-v0.18.0).
### Migration
- **`backend/migrations/020_outbound_emails.sql`** — auto-applied
on next backend start. Single new table with three indexes; no
changes to existing tables.
### Upgrade steps (from 0.17.0)
- Operators **MUST** ensure `GITEA_WEBHOOK_SECRET` is set in the
deployment's env. The framework now refuses to start if it's
empty. (For OHM-flotilla deployments,
`flotilla secret list <deployment>` confirms the binding; OHM
has carried this binding since v0.14.0, so the upgrade is
gesture-free for OHM specifically.)
- Operators **MAY** set `RFC_APP_INSECURE_WEBHOOKS=1` to bypass
the requirement in local-dev environments. Production
deployments **MUST NOT** set this; if they do, every webhook
POST logs a loud warning line per request.
- Operators **MAY** set `EMAIL_UNSUBSCRIBE_MAILTO` to route
`List-Unsubscribe: <mailto:…>` opt-out courtesy mail to a
humans-monitored mailbox distinct from the no-reply
`EMAIL_FROM` sender. Default falls back to `EMAIL_FROM`.
- You **MUST** apply schema migration
`020_outbound_emails.sql`. The migration creates a single new
table with three indexes; the framework runs migrations
automatically at process start, so no manual step is required
beyond restarting the backend so the migration runner picks
the file up. Existing deployments pick it up on first start
after upgrade with no operator action required.
- You **MUST** rebuild the frontend and restart the backend
after upgrading. `frontend/package.json#version` and `VERSION`
both move to `0.18.0`. No new secrets (the `outbound_emails`
table writes synchronously to the same SQLite file as every
other write).
- Operators **SHOULD** run a `mail-tester.com` probe against the
upgraded deployment to confirm the new envelope headers
(`Date`, `Message-ID`, `Auto-Submitted`, `List-Unsubscribe`,
`List-Unsubscribe-Post`) land cleanly with the upstream SMTP
relay's DKIM signing. The expected delta from pre-v0.18.0 is
+2-3 points on the spam-score axis (typical 5-6/10 baseline
→ 9+/10 post-upgrade).
## 0.17.0 — 2026-05-28
**Minor — schema migration required; no new env vars; no new secrets.**
+41
View File
@@ -714,6 +714,47 @@ The lighter half ships the structural shape — frontmatter, consent,
resolution, revocation. The heavier half ships the runtime
hardening.
### 6.8 Sign-in state resume (roadmap item #29, v0.23.0)
Signing in lands the user back on their most recently-viewed app
state instead of always on the empty-state home view. The model is
**per-user**, not per-device — the safe default the roadmap calls
for: a sign-in on any device resumes the most-recently-recorded
route. A per-device split and a profile-settings toggle UI are
follow-ups; v0.23.0 ships the column-level opt-out flag
(`resume_enabled`, default on) and the default-on behavior.
**Storage.** A single row per user in `user_session_state`
(migration 022): `user_id` (PK, FK → `users.id`, cascade-delete),
`last_route` (TEXT, the frontend pathname), `last_route_state`
(TEXT, JSON-encoded — SQLite has no native JSONB, so JSON-as-TEXT
matches how the app stores every other JSON blob), `resume_enabled`
(INTEGER, default 1), and `last_updated_at`.
**Wiring.** A debounced (~1s) frontend route-change hook posts the
current route to `PUT /api/me/last-state` for authenticated users
(anonymous: no-op). The stored `last_route` is folded onto the
existing `/api/auth/me` payload (no extra round-trip); on sign-in,
after the §21-Part-C Amplitude `identify` fires, the frontend
`navigate()`s to it. The identify-then-redirect ordering is
preserved: the redirect is gated on identify having fired.
**Stale state.** If the stored route is an RFC since withdrawn or
one the user lost rights to read, the redirect is a graceful no-op:
`navigate(last_route)` lands on whatever that route renders today,
and the existing routing already falls through to the
catalog/empty-state for a missing/unreadable RFC. No special-casing
on the server.
**Privacy (binding).** The stored state is **route + light view
state ONLY** — scroll anchors, open-tab selection, filter chips, and
the like. It **MUST NOT** carry draft-buffer contents, PR/comment
draft text, or any user-typed content. The frontend never sends such
content; the `last_route_state` column comment in migration 022 and
this paragraph are the contract. Resume state is purposely cheap to
discard: a deliberate "clear" (or `resume_enabled = 0`) drops the
user back to today's empty-state behavior.
---
## 7. The left pane
+1 -1
View File
@@ -1 +1 @@
0.17.0
0.28.0
+12
View File
@@ -38,10 +38,22 @@ GITEA_WEBHOOK_SECRET=change-me-to-a-shared-secret
# Comma-separated list of provider keys to enable. Per the §19.2
# per-RFC-model topic, this is app-wide until that topic lands.
ENABLED_MODELS=claude
# ANTHROPIC_API_KEY also powers the §9.1 propose-RFC tag suggestions
# (roadmap #27) — that surface always uses Claude Haiku for cost,
# independent of ENABLED_MODELS. With no key set, tag suggestions are
# simply unavailable (the modal hides the row); the rest of the app is
# unaffected.
ANTHROPIC_API_KEY=
GOOGLE_API_KEY=
OPENAI_API_KEY=
# --- Tag suggestions (§9.1 / roadmap #27) ---
# Per-user rate limit on the suggest-tags endpoint (cost backstop; the
# modal debounces and the endpoint is contributor-gated). Optional —
# these defaults apply when unset.
TAG_SUGGEST_RATE_MAX=30
TAG_SUGGEST_RATE_WINDOW_SECONDS=60
# --- Email (§15.4) ---
# Leave SMTP_HOST unset to use the stdout fallback — the integration
# tests rely on it, and a dev environment without a real SMTP provider
+349 -1
View File
@@ -15,6 +15,7 @@ import json
from typing import Any
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import PlainTextResponse, Response
from pydantic import BaseModel, Field
from . import (
@@ -29,6 +30,8 @@ from . import (
db,
device_trust as device_trust_mod,
docs as docs_mod,
docs_sessions,
docs_specs,
entry as entry_mod,
cache,
funder,
@@ -36,6 +39,7 @@ from . import (
notify,
philosophy,
providers as providers_mod,
tag_suggest,
)
from .bot import Bot
from .config import Config
@@ -48,6 +52,22 @@ class ProposeBody(BaseModel):
slug: str = Field(min_length=1, max_length=80)
pitch: str = Field(min_length=1)
tags: list[str] = Field(default_factory=list)
# Roadmap #26: optional "What will you be using this RFC for?" — the
# concrete ground-truth use case, distinct from the `pitch`'s abstract
# "why is this needed." Optional (NULL/omitted accepted), no minimum,
# generous cap matching the pitch's free-text body bound.
proposed_use_case: str | None = Field(default=None, max_length=8000)
class SuggestTagsBody(BaseModel):
# Roadmap #27: the partial propose-RFC draft, sent as the user types
# (debounced on the frontend). All fields optional — suggestions
# refine as the draft fills in. `pitch` is the "why is this needed"
# rationale; `use_case` is the #26 optional ground-truth field.
# Bounds mirror the propose body's free-text caps.
title: str = Field(default="", max_length=200)
pitch: str = Field(default="", max_length=8000)
use_case: str = Field(default="", max_length=8000)
class DeclineBody(BaseModel):
@@ -59,6 +79,18 @@ class FunderCredentialBody(BaseModel):
api_key: str = Field(min_length=1, max_length=2048)
class LastStateBody(BaseModel):
# v0.23.0 / roadmap item #29: server-side sign-in state resume.
# `route` is a frontend pathname the user was last on (bounded so a
# hostile client can't stuff arbitrary blobs through). `state` is an
# optional bag of *light* view state (scroll anchors, open tab,
# filter chips). PRIVACY: it MUST NOT carry draft-buffer contents —
# the frontend only ever sends ephemeral view state, and the column
# comment in migration 022 + SPEC §6.2 are the binding contract.
route: str = Field(min_length=1, max_length=2048)
state: dict[str, Any] | None = None
class BetaRequestBody(BaseModel):
# v0.8.0 — captured on the first OTC sign-in. All three fields are
# required so the admin queue has a coherent triage shape.
@@ -143,6 +175,177 @@ def make_router(
payload = docs_mod.load()
return {"body": payload["body"]}
# ---------------------------------------------------------------
# v0.19.0 / roadmap item #30 — /api/docs/sessions/*
#
# The framework mediates reads against the public
# `wiggleverse/ohm-session-history` gitea repo so the rendered
# `/docs/sessions/*` surface inherits the same chrome as the
# /docs/user-guide route and doesn't require a cross-origin
# gesture from the frontend. See backend/app/docs_sessions.py
# for the cache shape and env knobs.
#
# The route mapping for the three `status` values returned by
# the fetchers:
#
# "ok" → HTTP 200, payload as documented per endpoint
# "404" → HTTP 200/404 depending on the endpoint (the
# manifest's empty state is 200 + {} so the
# frontend can short-circuit without an error
# banner; transcripts/about return 404 so the
# frontend can render its own empty-state)
# "error" → HTTP 502, {"error": ..., "detail": ...} so the
# frontend retry surface reads as "couldn't reach
# the session-history repo" rather than as a
# generic 5xx.
# ---------------------------------------------------------------
@router.get("/api/docs/sessions/manifest")
async def get_sessions_manifest() -> dict[str, Any]:
result = await docs_sessions.fetch_manifest()
if result["status"] == "ok":
return result["manifest"]
if result["status"] == "404":
# Empty-state contract: render no session rows in the
# flyout but don't show an error banner. The frontend
# treats `{}` as "no sessions published yet".
return {}
raise HTTPException(
status_code=502,
detail={
"error": "session-history fetch failed",
"detail": result.get("detail", "unknown"),
},
)
@router.get("/api/docs/sessions/about")
async def get_sessions_about() -> Response:
result = await docs_sessions.fetch_about()
if result["status"] == "ok":
return PlainTextResponse(
content=result["body"],
media_type="text/markdown; charset=utf-8",
)
if result["status"] == "404":
raise HTTPException(
status_code=404,
detail="session-history README not yet published",
)
raise HTTPException(
status_code=502,
detail={
"error": "session-history fetch failed",
"detail": result.get("detail", "unknown"),
},
)
@router.get("/api/docs/sessions/{nnnn}/index")
async def get_sessions_index(nnnn: str) -> dict[str, Any]:
if not docs_sessions._is_valid_session_dir(nnnn):
# 400 over 404: the request itself is malformed (the
# session directory name doesn't match `^\d{4}$`),
# distinct from "no such session published yet".
raise HTTPException(status_code=400, detail="invalid session directory")
result = await docs_sessions.fetch_session_index(nnnn)
if result["status"] == "ok":
return {"files": result["files"]}
if result["status"] == "404":
raise HTTPException(
status_code=404,
detail="no transcripts published for this session",
)
raise HTTPException(
status_code=502,
detail={
"error": "session-history fetch failed",
"detail": result.get("detail", "unknown"),
},
)
@router.get("/api/docs/sessions/{nnnn}/{filename}")
async def get_sessions_transcript(nnnn: str, filename: str) -> Response:
# Path-shape validation before any network — refuses anything
# that would resolve outside the `NNNN/SESSION-...md` layout
# (e.g. legacy `SESSION-A-TRANSCRIPT.md` at the repo root,
# `../etc/passwd`, or any non-numeric session dir).
if not docs_sessions._is_valid_session_dir(nnnn):
raise HTTPException(status_code=400, detail="invalid session directory")
if not docs_sessions._is_valid_transcript_filename(filename):
raise HTTPException(status_code=400, detail="invalid transcript filename")
result = await docs_sessions.fetch_transcript(nnnn, filename)
if result["status"] == "ok":
return PlainTextResponse(
content=result["body"],
media_type="text/markdown; charset=utf-8",
)
if result["status"] == "404":
raise HTTPException(
status_code=404,
detail="transcript not found",
)
raise HTTPException(
status_code=502,
detail={
"error": "session-history fetch failed",
"detail": result.get("detail", "unknown"),
},
)
# ---------------------------------------------------------------
# v0.20.0 — /api/docs/specs/*
#
# Sibling of the v0.19.0 docs-sessions surface: the framework
# mediates a fetch against the public gitea raw URL for each
# configured spec so the rendered `/docs/specs/*` route inherits
# the same chrome (and the same auth-less reach) as the user
# guide and the session-history browser. See
# backend/app/docs_specs.py for the manifest shape, the env
# knobs, and the cache.
#
# Status-to-HTTP mapping mirrors docs_sessions:
# "ok" → HTTP 200, payload as documented per endpoint
# "404" → HTTP 200 / 404 (manifest 404 doesn't apply here —
# the manifest is derived from env, never 404s; spec
# 404 returns HTTP 404 so the frontend can render
# "spec not yet published / unknown name")
# "error" → HTTP 502
# ---------------------------------------------------------------
@router.get("/api/docs/specs/manifest")
async def get_specs_manifest() -> dict[str, Any]:
# The manifest is derived from env (`OHM_DOCS_SPECS`) and
# never fails — a malformed value falls back to the framework
# default at parse time. So this endpoint always returns 200
# + a list (the framework default is non-empty).
result = docs_specs.fetch_specs_manifest()
return {"specs": result["specs"]}
@router.get("/api/docs/specs/{name}")
async def get_spec(name: str) -> Response:
# Slug validation before any network — refuses `..`, `/`,
# uppercase, whitespace, etc. Same defense-in-depth posture
# as the docs-sessions transcript endpoint.
if not docs_specs._is_valid_name(name):
raise HTTPException(status_code=400, detail="invalid spec name")
result = await docs_specs.fetch_spec(name)
if result["status"] == "ok":
return PlainTextResponse(
content=result["body"],
media_type="text/markdown; charset=utf-8",
)
if result["status"] == "404":
raise HTTPException(
status_code=404,
detail="spec not found",
)
raise HTTPException(
status_code=502,
detail={
"error": "specs fetch failed",
"detail": result.get("detail", "unknown"),
},
)
# ---------------------------------------------------------------
# Auth surface — reads role from our users table per §6.
# ---------------------------------------------------------------
@@ -175,6 +378,28 @@ def make_router(
)
has_passcode = bool(row and row["passcode_hash"])
passcode_set_at = row["passcode_set_at"] if (row and has_passcode) else None
# v0.23.0 / item #29: fold the sign-in-resume state onto the
# same round-trip the frontend already makes on boot. When
# resume is disabled (resume_enabled = 0) we hand back a null
# route so the client never redirects; the stored row stays put
# so re-enabling later resumes the last-known route.
state_row = db.conn().execute(
"SELECT last_route, last_route_state, resume_enabled "
"FROM user_session_state WHERE user_id = ?",
(user.user_id,),
).fetchone()
resume_enabled = bool(state_row["resume_enabled"]) if state_row else True
last_route = (
state_row["last_route"]
if (state_row and resume_enabled)
else None
)
last_route_state = None
if state_row and resume_enabled and state_row["last_route_state"]:
try:
last_route_state = json.loads(state_row["last_route_state"])
except (ValueError, TypeError):
last_route_state = None
return {
"authenticated": True,
"user": {
@@ -191,6 +416,10 @@ def make_router(
"needs_profile": needs_profile,
"has_passcode": has_passcode,
"passcode_set_at": passcode_set_at,
# v0.23.0 / item #29 — sign-in state resume.
"resume_enabled": resume_enabled,
"last_route": last_route,
"last_route_state": last_route_state,
},
}
@@ -260,6 +489,52 @@ def make_router(
notify.fan_out_new_beta_request(requester_user_id=user.user_id)
return {"ok": True}
# ---------------------------------------------------------------
# v0.23.0 (§6.2, roadmap item #29): server-side sign-in state
# resume. The frontend debounce-posts the user's current route +
# a small bag of light view state here on every route change; the
# next sign-in reads `last_route` off `/api/auth/me` and redirects.
#
# Per-user (NOT per-device) — one row per user, keyed on user_id.
# `resume_enabled` is the opt-out flag (default on); when it's 0
# this endpoint no-ops so a user who turned resume off doesn't keep
# silently rewriting their stored route. PRIVACY: the body carries
# route + light state ONLY, never draft-buffer contents (migration
# 022 column comment + SPEC §6.2 are the binding contract).
#
# `require_user` (not `require_contributor`) — a pending/granted
# distinction is irrelevant for "remember where I was", and a
# pending user navigating read-only surfaces should still resume.
# Anonymous callers get the 401 `require_user` raises.
# ---------------------------------------------------------------
@router.put("/api/me/last-state")
async def put_last_state(body: LastStateBody, request: Request) -> dict[str, Any]:
user = auth.require_user(request)
# Respect the opt-out: if a row already exists with resume
# disabled, leave it untouched and report the no-op. A first-
# ever POST (no row yet) defaults to enabled and stores.
existing = db.conn().execute(
"SELECT resume_enabled FROM user_session_state WHERE user_id = ?",
(user.user_id,),
).fetchone()
if existing is not None and not existing["resume_enabled"]:
return {"ok": True, "stored": False}
state_json = json.dumps(body.state) if body.state is not None else None
db.conn().execute(
"""
INSERT INTO user_session_state
(user_id, last_route, last_route_state, last_updated_at)
VALUES (?, ?, ?, datetime('now'))
ON CONFLICT(user_id) DO UPDATE SET
last_route = excluded.last_route,
last_route_state = excluded.last_route_state,
last_updated_at = excluded.last_updated_at
""",
(user.user_id, body.route, state_json),
)
return {"ok": True, "stored": True}
# ---------------------------------------------------------------
# v0.11.0: trust device for 30 days (§6.2, roadmap item #9).
#
@@ -383,12 +658,36 @@ def make_router(
).fetchone()
if row is None:
raise HTTPException(404, "Not found")
return _serialize_rfc(row)
payload = _serialize_rfc(row)
# Roadmap #26: surface the optional propose-time use case on the
# RFC view. The idea PR closes on merge, but the canonical row in
# `proposed_use_cases` persists; look it up by slug (the latest
# 'rfc'-scope row for this slug). NULL == "left blank".
uc = db.conn().execute(
"""
SELECT use_case FROM proposed_use_cases
WHERE scope = 'rfc' AND rfc_slug = ?
ORDER BY id DESC LIMIT 1
""",
(slug,),
).fetchone()
payload["proposed_use_case"] = uc["use_case"] if uc else None
return payload
# ---------------------------------------------------------------
# §7.3 / §9.3: pending ideas
# ---------------------------------------------------------------
def _proposal_use_case(pr_number: int) -> str | None:
"""Roadmap #26: read the optional use case for an idea PR from the
canonical side table. Returns None when none was supplied (the
"left blank" sentinel the frontend renders tastefully)."""
row = db.conn().execute(
"SELECT use_case FROM proposed_use_cases WHERE scope = 'rfc' AND pr_number = ?",
(pr_number,),
).fetchone()
return row["use_case"] if row else None
@router.get("/api/proposals")
async def list_proposals() -> dict[str, Any]:
rows = db.conn().execute(
@@ -408,6 +707,7 @@ def make_router(
"description": r["description"],
"opened_by": r["opened_by"],
"opened_at": r["opened_at"],
"proposed_use_case": _proposal_use_case(r["pr_number"]),
}
for r in rows
]
@@ -456,6 +756,7 @@ def make_router(
"opened_at": row["opened_at"],
"entry": entry_payload,
"affordances": affordances,
"proposed_use_case": _proposal_use_case(pr_number),
}
# ---------------------------------------------------------------
@@ -532,8 +833,55 @@ def make_router(
# cache write is idempotent.)
await cache.refresh_meta_pulls(config, gitea)
# Roadmap #26: persist the optional use case to the canonical,
# reconcile-proof side table keyed by the idea PR number. NULL/
# blank simply writes no row (absence == "left blank"). Done after
# the refresh so the cache row exists; the mirror onto cached_prs
# keeps the cache column in parity for any read that uses it.
use_case = (payload.proposed_use_case or "").strip()
if use_case:
db.conn().execute(
"""
INSERT INTO proposed_use_cases (scope, rfc_slug, pr_number, use_case)
VALUES ('rfc', ?, ?, ?)
ON CONFLICT(scope, pr_number) DO UPDATE SET use_case = excluded.use_case
""",
(slug, pr["number"], use_case),
)
db.conn().execute(
"UPDATE cached_prs SET proposed_use_case = ? WHERE pr_kind = 'idea' AND pr_number = ?",
(use_case, pr["number"]),
)
return {"pr_number": pr["number"], "slug": slug}
# ---------------------------------------------------------------
# §9.1 Slice 2 (roadmap #27): Claude Haiku tag suggestions as the
# propose-RFC fields fill in. The modal debounce-posts the partial
# draft; we constrain Haiku to the corpus's existing tag set and
# return a short ranked list of clickable chips. Gated to
# contributors (same gate as propose) so the cost surface is bounded
# to people who can actually file an RFC; rate-limited per user as a
# backstop. Degrades to an empty list (no error) when no Anthropic
# key is bound, the corpus has no tags yet, or the draft is empty —
# so the modal simply shows nothing extra.
# ---------------------------------------------------------------
@router.post("/api/rfcs/suggest-tags")
async def suggest_rfc_tags(payload: SuggestTagsBody, request: Request) -> dict[str, Any]:
user = auth.require_contributor(request)
if not tag_suggest.rate_limit_ok(user.user_id):
raise HTTPException(429, "Too many tag-suggestion requests; please slow down.")
provider = tag_suggest.haiku_provider(config)
if provider is None:
return {"suggestions": []}
universe = tag_suggest.gather_tag_universe()
draft = tag_suggest.Draft(
title=payload.title, pitch=payload.pitch, use_case=payload.use_case
)
suggestions = tag_suggest.suggest(provider, draft, universe)
return {"suggestions": suggestions}
# ---------------------------------------------------------------
# §9.3: merge / decline / withdraw an idea PR
# ---------------------------------------------------------------
+67
View File
@@ -672,6 +672,73 @@ def make_router(config: Config) -> APIRouter:
"has_more": len(rows) == limit,
}
@router.get("/api/admin/outbound-emails")
async def list_outbound_emails(
request: Request,
kind: str | None = None,
status: str | None = None,
to_address: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
before_id: int | None = None,
) -> dict[str, Any]:
"""v0.18.0 Slice 4: read-only inspection of the
`outbound_emails` audit table.
Answers questions like "did this person ever get their
invite?" without grepping VM logs. Filterable by kind
('otc' | 'invite' | 'notification' | 'bundle' | 'digest'),
status ('sent' | 'failed' | 'deferred' | 'bounced'), and
to_address; the latter is exact-match because the audit
question is usually "the specific person who said they
didn't receive it." Per the proposal, no admin UI ships
with v0.18.0 operator queries via curl + jq for now.
"""
auth.require_admin(request)
clauses: list[str] = []
args: list[Any] = []
if kind:
clauses.append("kind = ?")
args.append(kind)
if status:
clauses.append("status = ?")
args.append(status)
if to_address:
clauses.append("LOWER(to_address) = LOWER(?)")
args.append(to_address)
if before_id is not None:
clauses.append("id < ?")
args.append(before_id)
where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
rows = db.conn().execute(
f"""
SELECT id, to_address, from_address, subject, kind, sent_at,
status, error, notification_id, message_id
FROM outbound_emails
{where}
ORDER BY id DESC
LIMIT ?
""",
(*args, limit),
).fetchall()
return {
"items": [
{
"id": r["id"],
"to_address": r["to_address"],
"from_address": r["from_address"],
"subject": r["subject"],
"kind": r["kind"],
"sent_at": r["sent_at"],
"status": r["status"],
"error": r["error"],
"notification_id": r["notification_id"],
"message_id": r["message_id"],
}
for r in rows
],
"has_more": len(rows) == limit,
}
@router.get("/api/admin/permission-events")
async def list_permission_events(
request: Request,
+10 -2
View File
@@ -40,7 +40,7 @@ from typing import Any
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from . import auth, chat as chat_layer, db
from . import auth, chat as chat_layer, db, rfc_links
log = logging.getLogger(__name__)
@@ -171,9 +171,17 @@ def make_router() -> APIRouter:
""",
(thread_id,),
).fetchall()
# Roadmap #28 Part 1: enrich each discussion comment with RFC
# auto-link segments scanned against the live accepted-RFC corpus
# (read-time; see rfc_links.py). exclude_slug suppresses self-links
# to this RFC inside its own discussion.
link_index = rfc_links.build_index(db.conn(), exclude_slug=slug)
messages = [_serialize_message(r) for r in rows]
for m in messages:
m["text_segments"] = link_index.segment(m["text"])
return {
"thread": _serialize_thread(thread),
"messages": [_serialize_message(r) for r in rows],
"messages": messages,
}
# -------------------------------------------------------------------
+135 -14
View File
@@ -73,6 +73,13 @@ class MarkReadBody(BaseModel):
class BounceBody(BaseModel):
email: str = Field(min_length=3, max_length=320)
kind: str = Field(default="hard") # 'hard' or 'complaint'
# v0.18.0 Slice 5: when the bounce provider includes the
# original Message-ID, the framework correlates it back to
# the matching `outbound_emails` row and stamps
# `status='bounced'`. Optional — providers that don't surface
# the Message-ID still flip the global opt-out via the email
# match, but lose the per-message attribution.
message_id: str | None = Field(default=None, max_length=1000)
class CookieConsentBody(BaseModel):
@@ -443,6 +450,40 @@ def make_router(config: Config) -> APIRouter:
# ----- Email: one-click unsubscribe + bounce webhook -----
# v0.18.0: the category → column map. The `all` synthetic
# category lands the bundle's one-click on the global opt-out
# flag (per `email._send_bundle` in v0.18.0 Slice 2 — a bundle
# spans multiple categories, so a per-category flip wouldn't
# honor the user's intent).
_CATEGORY_COLUMN: dict[str, str] = {
"personal-direct": "email_personal_direct",
"structural": "email_watched_structural",
"admin-actionable": "email_admin_actionable",
"all": "email_opt_out_all",
}
def _apply_unsubscribe(user_id: int, category: str) -> bool:
"""Flip the matching column. Returns True on success, False
if the category is unknown. Idempotent running twice on
the same (user, category) is harmless (it sets the column
to its current value)."""
column = _CATEGORY_COLUMN.get(category)
if column is None:
return False
# `all` sets the flag to 1 (opt out); per-category sets to 0
# (turn that category off). The column semantic is "1 means
# don't send"; the per-category booleans are "1 means do
# send". Different polarities, hence the case split.
if category == "all":
db.conn().execute(
f"UPDATE users SET {column} = 1 WHERE id = ?", (user_id,)
)
else:
db.conn().execute(
f"UPDATE users SET {column} = 0 WHERE id = ?", (user_id,)
)
return True
@router.get("/api/email/unsubscribe")
async def email_unsubscribe(t: str = Query(..., description="Signed token from the email footer")) -> HTMLResponse:
try:
@@ -453,20 +494,51 @@ def make_router(config: Config) -> APIRouter:
"<p>Open the app to manage your notification preferences directly.</p>",
status_code=400,
)
column = {
"personal-direct": "email_personal_direct",
"structural": "email_watched_structural",
"admin-actionable": "email_admin_actionable",
}.get(category)
if column is None:
if not _apply_unsubscribe(user_id, category):
return HTMLResponse(
f"<h1>Unknown category</h1><p>{category}</p>", status_code=400
)
db.conn().execute(f"UPDATE users SET {column} = 0 WHERE id = ?", (user_id,))
return HTMLResponse(
f"<h1>Unsubscribed</h1><p>You will no longer receive {category} emails. "
f"You can re-enable them in your notification preferences.</p>"
)
if category == "all":
body = (
"<h1>Unsubscribed</h1><p>You will no longer receive any email "
"from this app. You can re-enable individual categories from "
"your notification preferences after signing in.</p>"
)
else:
body = (
f"<h1>Unsubscribed</h1><p>You will no longer receive {category} emails. "
f"You can re-enable them in your notification preferences.</p>"
)
return HTMLResponse(body)
@router.post("/api/email/unsubscribe")
async def email_unsubscribe_post(
request: Request,
t: str = Query(..., description="Signed token from the List-Unsubscribe header"),
) -> dict[str, Any]:
"""v0.18.0: RFC 8058 one-click endpoint.
Gmail and Yahoo POST `List-Unsubscribe=One-Click` (as a
form-encoded body) to the URL in the `List-Unsubscribe`
header when the user clicks their MUA's "Unsubscribe"
button. The endpoint MUST accept POST (per the
`List-Unsubscribe-Post` header we advertise) and MUST be
idempotent.
The body content is checked loosely RFC 8058 says it
SHOULD be exactly `List-Unsubscribe=One-Click`, but some
intermediaries strip / re-encode the body, so the
framework accepts any POST to the URL once the token
verifies. The bar is that the token signature carries the
authority; the body is hint-only.
"""
try:
user_id, category = email_mod.verify_unsubscribe_token(t)
except BadSignature:
raise HTTPException(400, "Invalid or expired token")
if not _apply_unsubscribe(user_id, category):
raise HTTPException(400, f"Unknown category: {category}")
return {"ok": True, "category": category}
@router.post("/api/webhooks/email-bounce")
async def email_bounce(body: BounceBody, request: Request) -> dict[str, Any]:
@@ -485,21 +557,70 @@ def make_router(config: Config) -> APIRouter:
# stays unauthenticated for dev (the v1 contract).
import os as _os
expected = _os.environ.get("WEBHOOK_EMAIL_BOUNCE_SECRET", "").strip()
if expected:
# v0.25.0 (audit 0026 M5): fail closed. An unset secret used to
# leave this endpoint fully unauthenticated — anyone could suppress
# any user's mail by POSTing their address (email_opt_out_all flip
# below). Now an unset secret DISABLES the endpoint (503) instead
# of opening it. A dev that genuinely wants it open opts in
# explicitly with RFC_APP_INSECURE_BOUNCE_WEBHOOK=1, mirroring the
# RFC_APP_INSECURE_WEBHOOKS dev-bypass on the Gitea hook.
if not expected:
if _os.environ.get("RFC_APP_INSECURE_BOUNCE_WEBHOOK", "").strip() == "1":
log.warning(
"email-bounce webhook running UNAUTHENTICATED "
"(RFC_APP_INSECURE_BOUNCE_WEBHOOK=1) — never set this in production"
)
else:
log.error(
"email-bounce webhook refused: WEBHOOK_EMAIL_BOUNCE_SECRET is unset "
"(set the secret to enable, or RFC_APP_INSECURE_BOUNCE_WEBHOOK=1 for dev)"
)
raise HTTPException(503, "Bounce webhook not configured")
else:
received = request.headers.get("X-Webhook-Secret", "")
import hmac as _hmac
if not received or not _hmac.compare_digest(expected, received):
raise HTTPException(401, "Invalid webhook signature")
# v0.18.0 Slice 5: correlate the bounce back to the
# matching outbound_emails row if the provider supplied
# the Message-ID. The hard-bounce -> global-opt-out
# logic below still fires regardless; this is an
# additional audit signal.
correlated_row_id: int | None = None
if body.message_id:
correlated = db.conn().execute(
"SELECT id FROM outbound_emails WHERE message_id = ?",
(body.message_id,),
).fetchone()
if correlated is not None:
correlated_row_id = correlated["id"]
db.conn().execute(
"UPDATE outbound_emails SET status = 'bounced', "
"error = COALESCE(error, '') || ? WHERE id = ?",
(f"bounce ({body.kind})", correlated_row_id),
)
log.info(
"email-bounce: correlated message_id=%s -> outbound_emails.id=%s",
body.message_id, correlated_row_id,
)
else:
log.info(
"email-bounce: message_id=%s did not match any "
"outbound_emails row (provider may be replaying an old bounce, "
"or the row was pruned)",
body.message_id,
)
row = db.conn().execute(
"SELECT id FROM users WHERE LOWER(email) = LOWER(?)", (body.email,),
).fetchone()
if row is None:
return {"ok": True, "matched": False}
return {"ok": True, "matched": False, "correlated_id": correlated_row_id}
db.conn().execute(
"UPDATE users SET email_opt_out_all = 1 WHERE id = ?", (row["id"],),
)
log.info("email-bounce: opted out user %s (%s)", row["id"], body.kind)
return {"ok": True, "matched": True}
return {"ok": True, "matched": True, "correlated_id": correlated_row_id}
return router
+52 -1
View File
@@ -23,7 +23,7 @@ from typing import Any
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver
from . import auth, cache, chat as chat_layer, db, entry as entry_mod, funder, models_resolver, rfc_links
from .bot import Bot
from .config import Config
from .gitea import Gitea, GiteaError
@@ -42,6 +42,11 @@ RFC_FILE_PATH = "RFC.md"
class OpenPRBody(BaseModel):
title: str = Field(min_length=1, max_length=240)
description: str = Field(max_length=8000)
# Roadmap #26: optional "What will you be using this change for?" —
# the concrete ground-truth use case sibling to the required
# "why is this change needed" (the `description`). Optional, generous
# cap matching the description bound.
proposed_use_case: str | None = Field(default=None, max_length=8000)
class PRDescriptionBody(BaseModel):
@@ -173,6 +178,26 @@ def make_router(
raise HTTPException(502, f"Gitea: {e.detail}")
await _refresh_after_pr_write(rfc)
# Roadmap #26: persist the optional use case to the canonical,
# reconcile-proof side table keyed by the PR number. Blank/omitted
# writes no row (absence == "left blank"). The mirror onto
# cached_prs keeps the cache column in parity.
use_case = (body.proposed_use_case or "").strip()
if use_case:
db.conn().execute(
"""
INSERT INTO proposed_use_cases (scope, rfc_slug, pr_number, use_case)
VALUES ('pr', ?, ?, ?)
ON CONFLICT(scope, pr_number) DO UPDATE SET use_case = excluded.use_case
""",
(slug, pr["number"], use_case),
)
db.conn().execute(
"UPDATE cached_prs SET proposed_use_case = ? WHERE rfc_slug = ? AND pr_number = ?",
(use_case, slug, pr["number"]),
)
return {"pr_number": pr["number"], "slug": slug, "branch": branch}
# -------------------------------------------------------------------
@@ -188,6 +213,12 @@ def make_router(
path = _file_path_for(rfc)
head_branch = pr_row["head_branch"]
# Roadmap #28 Part 1: build the RFC auto-link index once for this
# PR view (read-time enrichment against the live accepted-RFC
# corpus; see rfc_links.py). exclude_slug suppresses self-links to
# this RFC inside its own PR.
link_index = rfc_links.build_index(db.conn(), exclude_slug=slug)
# §11.3: PRs are always public; no visibility check.
main_fetched = await gitea.read_file(owner, repo, path, ref="main")
main_body = _extract_body(rfc, (main_fetched or ("", ""))[0])
@@ -234,6 +265,13 @@ def make_router(
for r in msg_rows:
messages_by_thread.setdefault(r["thread_id"], []).append(_serialize_message(r))
# Roadmap #28 Part 1: enrich every comment with RFC auto-link
# segments (read-time; see rfc_links.py). The description is
# enriched alongside it in the return dict below.
for _msgs in messages_by_thread.values():
for _m in _msgs:
_m["text_segments"] = link_index.segment(_m["text"])
# Per-user seen cursor per §10.3. Anonymous viewers get no
# cursor — they always see "everything new" but cannot advance
# the cursor (no row to write to).
@@ -300,6 +338,8 @@ def make_router(
"pr_number": pr_number,
"title": pr_row["title"],
"description": pr_row["description"],
"description_segments": link_index.segment(pr_row["description"]),
"proposed_use_case": _pr_use_case(pr_number),
"state": pr_row["state"],
"opened_by": pr_row["opened_by"],
"opened_at": pr_row["opened_at"],
@@ -762,6 +802,17 @@ def _can_edit_pr_text(rfc, pr_row, viewer) -> bool:
return _can_withdraw(rfc, pr_row, viewer)
def _pr_use_case(pr_number: int) -> str | None:
"""Roadmap #26: the optional propose-PR use case from the canonical
side table, or None when the change was opened without one ("left
blank")."""
row = db.conn().execute(
"SELECT use_case FROM proposed_use_cases WHERE scope = 'pr' AND pr_number = ?",
(pr_number,),
).fetchone()
return row["use_case"] if row else None
def _pr_capabilities(rfc, pr_row, viewer) -> dict:
return {
"can_merge": _can_merge(rfc, viewer) and pr_row["state"] == "open",
+15 -1
View File
@@ -60,6 +60,20 @@ def load_config() -> Config:
enabled = [m.strip() for m in _optional("ENABLED_MODELS", "claude").split(",") if m.strip()]
# v0.18.0: `GITEA_WEBHOOK_SECRET` is now mandatory (per the
# email + webhook hygiene proposal). An empty value used to
# silently accept unsigned webhook POSTs — that was the
# invisible-failure shape the proposal targets. Now the
# framework refuses to start when the secret is empty unless
# the operator opts into the dev-bypass with
# `RFC_APP_INSECURE_WEBHOOKS=1`. Local-dev deployments without
# a wired Gitea hook set the bypass; production MUST NOT.
insecure_webhooks = os.environ.get("RFC_APP_INSECURE_WEBHOOKS", "").strip() == "1"
if insecure_webhooks:
webhook_secret = _optional("GITEA_WEBHOOK_SECRET")
else:
webhook_secret = _required("GITEA_WEBHOOK_SECRET")
return Config(
gitea_url=_required("GITEA_URL").rstrip("/"),
gitea_bot_user=_required("GITEA_BOT_USER"),
@@ -72,7 +86,7 @@ def load_config() -> Config:
secret_key=_required("SECRET_KEY"),
database_path=database_path,
owner_gitea_login=_optional("OWNER_GITEA_LOGIN"),
webhook_secret=_optional("GITEA_WEBHOOK_SECRET"),
webhook_secret=webhook_secret,
enabled_models=enabled,
anthropic_api_key=_optional("ANTHROPIC_API_KEY"),
google_api_key=_optional("GOOGLE_API_KEY"),
+32 -21
View File
@@ -97,6 +97,13 @@ class IssueOutcome:
raw_token: str
row_id: int
@property
def cookie_value(self) -> str:
"""The value to put in the `rfc_device_trust` cookie: the row-id
selector joined to the raw token (v0.25.0 / audit 0026 M1). The
selector lets `lookup` read one indexed row instead of scanning."""
return f"{self.row_id}.{self.raw_token}"
def _new_token() -> str:
return secrets.token_urlsafe(TOKEN_BYTES)
@@ -173,33 +180,37 @@ def lookup(raw_token: str) -> LookupOutcome:
if not raw:
return LookupOutcome(ok=False, user=None, reason="invalid")
# The unique index on `device_token_hash` would let us SELECT by
# hash if bcrypt were a stable hash, but bcrypt incorporates a
# per-row salt — equal tokens produce different hashes. We walk
# the candidate set instead. In practice the set is small (a
# human has a handful of trusted devices) and bcrypt is cheap on
# the order of milliseconds; the walk is bounded by the user's
# active device count.
# v0.25.0 (audit 0026 M1): the cookie is "<row_id>.<raw_token>". We
# parse the row-id selector and read exactly ONE row by its indexed
# primary key, then bcrypt-check the token against that single row.
#
# We don't pre-filter by `revoked_at IS NULL` here so that a
# token presented for a recently-revoked row produces a
# 'revoked' outcome (the endpoint surfaces a different shape).
# Same for expired: we let the walk hit and classify after.
rows = db.conn().execute(
# The previous shape read EVERY device_trust row (all users, including
# revoked/expired) and bcrypt-checked each — an unauthenticated
# CPU-amplification DoS reachable at /auth/device-trust/start that
# grew without bound as the table accumulated. bcrypt's per-row salt
# is why we can't SELECT by hash; carrying the row-id in the cookie is
# the standard fix (the id is not secret; the token still is).
selector, sep, token = raw.partition(".")
if not sep or not selector.isdigit() or not token:
# Legacy bare-token cookies (pre-v0.25.0) and malformed values land
# here. We refuse rather than fall back to a full-table scan, so
# the amplification path is fully closed; affected users simply
# re-authenticate once via OTC/passcode and get a new cookie.
return LookupOutcome(ok=False, user=None, reason="invalid")
matched = db.conn().execute(
"""
SELECT id, user_id, device_token_hash, expires_at, revoked_at
FROM device_trust
ORDER BY id DESC
WHERE id = ?
""",
).fetchall()
(int(selector),),
).fetchone()
matched = None
for row in rows:
if _check(raw, row["device_token_hash"]):
matched = row
break
if matched is None:
# One bcrypt check, against the selected row only. A wrong/forged token
# for a real id reads as 'unknown' (cookie cleared), same as a missing
# row — a probing client can't distinguish the two.
if matched is None or not _check(token, matched["device_token_hash"]):
return LookupOutcome(ok=False, user=None, reason="unknown")
if matched["revoked_at"] is not None:
+13 -1
View File
@@ -180,7 +180,19 @@ def assemble_for_user(
subject = _subject(eligible, cadence)
body = _body(eligible, cadence, cfg)
sent = email_mod._deliver(cfg, email, subject, body)
# v0.18.0: the digest is the bulk-adjacent surface par excellence
# (it can carry weeks of accumulated activity), so it gets the
# full one-click unsubscribe to the global opt-out. Per-category
# opt-outs are managed from the preferences page; this footer is
# the "stop sending me anything" escape hatch Gmail and Yahoo
# expect for senders at this tier.
unsubscribe_url = email_mod.make_unsubscribe_url(user_id, "all")
sent = email_mod._deliver(
cfg, email, subject, body,
unsubscribe_mailto=cfg.unsubscribe_mailto,
unsubscribe_url=unsubscribe_url,
kind="digest",
)
if not sent:
return False
ids = [r["id"] for r, _ in eligible]
+358
View File
@@ -0,0 +1,358 @@
"""§14 + roadmap item #30 — on-site sessions-history browser source.
Sibling of `docs.py` / `philosophy.py` but with a different read shape:
the bodies here live in the **public** `wiggleverse/ohm-session-history`
gitea repo (transcripts of every OHM build session, published per the
ohm-infra SESSION-PROTOCOL.md), not on disk. The framework mediates
the gitea fetch on behalf of the browser so the rendered `/docs/sessions/*`
surface inherits the same chrome as `/philosophy` and `/docs/user-guide`
and stays free of any cross-origin gestures from the frontend.
Three read endpoints, all anonymous-reachable:
GET /api/docs/sessions/manifest sessions.json (title manifest)
GET /api/docs/sessions/about README.md (the about page)
GET /api/docs/sessions/<NNNN>/<file> a transcript body
GET /api/docs/sessions/<NNNN>/index per-session file listing
All three sit behind a small in-process TTL cache (manifest TTL default
60 s, content TTL default 300 s). Negative results (404 from gitea) are
also cached at the content TTL to avoid hammering gitea when a
deployment hasn't yet been populated with transcripts. The cache key
is the URL path on the gitea raw base (or the contents API for the
per-session listing); the cache lives in-process, plain dict +
`time.monotonic()` check, no external dep.
Env knobs:
OHM_SESSION_HISTORY_RAW_BASE
Override the gitea raw base URL. Default points at OHM's canonical
transcript repo:
https://git.wiggleverse.org/wiggleverse/ohm-session-history/raw/branch/main
The framework-default value is OHM-flavored because OHM is the
only deployment to date a deployment running its own
transcript repo overrides this via flotilla's overlay.
OHM_SESSION_HISTORY_CONTENTS_BASE
Override the gitea contents-API base URL (for the per-session
listing endpoint, which enumerates files inside a `NNNN/` folder).
Default:
https://git.wiggleverse.org/api/v1/repos/wiggleverse/ohm-session-history/contents
OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC
Cache TTL for the manifest (default 60 s). The manifest is small
and changes when a new session is added; 60 s strikes a balance
between freshness and gitea load.
OHM_DOCS_SESSIONS_CONTENT_TTL_SEC
Cache TTL for transcript bodies + README + per-session listings
(default 300 s = 5 minutes). Transcripts are append-only once
published, so 5 minutes of staleness is harmless.
§3 invariant 1 is preserved: the framework holds no secret bytes; the
gitea repo is public, the fetch carries no auth header.
"""
from __future__ import annotations
import logging
import os
import re
import threading
import time
from typing import Any
import httpx
log = logging.getLogger(__name__)
_DEFAULT_RAW_BASE = (
"https://git.wiggleverse.org/wiggleverse/ohm-session-history/raw/branch/main"
)
_DEFAULT_CONTENTS_BASE = (
"https://git.wiggleverse.org/api/v1/repos/wiggleverse/ohm-session-history/contents"
)
_DEFAULT_MANIFEST_TTL_SEC = 60.0
_DEFAULT_CONTENT_TTL_SEC = 300.0
# The transcript filename shape per SESSION-PROTOCOL.md §1. The
# `<start>--<end>` suffix is optional so legacy renamed-letter
# transcripts (e.g. `SESSION-0009.0-TRANSCRIPT.md` without timestamps)
# remain reachable. The `\.\d+(\.\d+)*` after the session number
# accommodates `0017.0`, `0017.1`, `0017.1.1`, etc.
_TRANSCRIPT_FILENAME_RE = re.compile(
r"^SESSION-\d{4}\.\d+(\.\d+)*-TRANSCRIPT"
r"(-\d{4}-\d{2}-\d{2}T\d{2}-\d{2}--\d{4}-\d{2}-\d{2}T\d{2}-\d{2})?"
r"\.md$"
)
_SESSION_DIR_RE = re.compile(r"^\d{4}$")
_HTTP_TIMEOUT_SEC = 5.0
def _env_float(name: str, default: float) -> float:
raw = os.environ.get(name, "").strip()
if not raw:
return default
try:
return float(raw)
except ValueError:
log.warning("invalid %s=%r — falling back to %s", name, raw, default)
return default
def _raw_base() -> str:
return os.environ.get("OHM_SESSION_HISTORY_RAW_BASE", "").strip() or _DEFAULT_RAW_BASE
def _contents_base() -> str:
return (
os.environ.get("OHM_SESSION_HISTORY_CONTENTS_BASE", "").strip()
or _DEFAULT_CONTENTS_BASE
)
def _manifest_ttl() -> float:
return _env_float("OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC", _DEFAULT_MANIFEST_TTL_SEC)
def _content_ttl() -> float:
return _env_float("OHM_DOCS_SESSIONS_CONTENT_TTL_SEC", _DEFAULT_CONTENT_TTL_SEC)
# ---------------------------------------------------------------------------
# In-process TTL cache
# ---------------------------------------------------------------------------
#
# Plain dict + `time.monotonic()` check, no external dep. The cache
# value is a `(stored_at, payload)` tuple; `payload` may carry an
# error-shape sentinel for negative caching (404s). Lock guards
# read-modify-write across worker tasks; entries are immutable once
# stored so reads under the lock are fast.
_lock = threading.Lock()
_cache: dict[str, tuple[float, dict[str, Any]]] = {}
def _cache_get(key: str, ttl_sec: float) -> dict[str, Any] | None:
with _lock:
entry = _cache.get(key)
if entry is None:
return None
stored_at, payload = entry
if time.monotonic() - stored_at > ttl_sec:
# Don't evict here; let _cache_put overwrite on next fetch.
# The stale entry is gated by the TTL check, so it stays
# invisible to readers regardless.
return None
return payload
def _cache_put(key: str, payload: dict[str, Any]) -> None:
with _lock:
_cache[key] = (time.monotonic(), payload)
def reset_cache() -> None:
"""Drop every cached entry. Test seam — not called in production."""
with _lock:
_cache.clear()
# ---------------------------------------------------------------------------
# Public fetch surface
# ---------------------------------------------------------------------------
#
# Each fetcher returns a `{status, ...}` dict. `status` is one of:
# "ok" — payload field carries the body
# "404" — gitea returned 404 (or content was missing)
# "error" — gitea returned 5xx, timed out, or returned malformed data
#
# The route layer maps these onto HTTP responses; keeping the mapping
# out of this module makes the cache transparent to the test harness.
async def _http_get(url: str) -> tuple[int, str]:
"""Perform a single GET against `url`; return (status_code, body).
On timeout or network error, returns (599, error_message). The 599
pseudo-status maps to a 502 at the route layer the same way an
upstream 5xx does the caller doesn't care which leg of the
network broke.
"""
try:
async with httpx.AsyncClient(timeout=_HTTP_TIMEOUT_SEC) as client:
r = await client.get(url)
return r.status_code, r.text
except httpx.HTTPError as e:
log.warning("gitea fetch failed for %s: %s", url, e)
return 599, f"fetch error: {e}"
def _is_valid_session_dir(nnnn: str) -> bool:
return bool(_SESSION_DIR_RE.match(nnnn))
def _is_valid_transcript_filename(filename: str) -> bool:
return bool(_TRANSCRIPT_FILENAME_RE.match(filename))
async def fetch_manifest() -> dict[str, Any]:
"""Fetch and parse `sessions.json` from the public repo.
Returns one of:
{"status": "ok", "manifest": {...}} successful parse
{"status": "404"} gitea 404 (empty state)
{"status": "error", "detail": "..."} 5xx / timeout / bad JSON
"""
cache_key = "manifest"
cached = _cache_get(cache_key, _manifest_ttl())
if cached is not None:
return cached
url = f"{_raw_base()}/sessions.json"
status, body = await _http_get(url)
if status == 200:
try:
import json
data = json.loads(body)
except (json.JSONDecodeError, ValueError) as e:
payload: dict[str, Any] = {
"status": "error",
"detail": f"sessions.json malformed: {e}",
}
# Don't cache parse errors — give the upstream a chance to
# fix the file without waiting for TTL expiry.
return payload
if not isinstance(data, dict):
return {
"status": "error",
"detail": "sessions.json is not a JSON object",
}
payload = {"status": "ok", "manifest": data}
_cache_put(cache_key, payload)
return payload
if status == 404:
payload = {"status": "404"}
_cache_put(cache_key, payload)
return payload
return {"status": "error", "detail": f"upstream returned {status}"}
async def fetch_about() -> dict[str, Any]:
"""Fetch the repo's README.md (rendered as the /docs/sessions/about page).
Returns one of:
{"status": "ok", "body": "..."}
{"status": "404"}
{"status": "error", "detail": "..."}
"""
cache_key = "about:README.md"
cached = _cache_get(cache_key, _content_ttl())
if cached is not None:
return cached
url = f"{_raw_base()}/README.md"
status, body = await _http_get(url)
if status == 200:
payload: dict[str, Any] = {"status": "ok", "body": body}
_cache_put(cache_key, payload)
return payload
if status == 404:
payload = {"status": "404"}
_cache_put(cache_key, payload)
return payload
return {"status": "error", "detail": f"upstream returned {status}"}
async def fetch_transcript(nnnn: str, filename: str) -> dict[str, Any]:
"""Fetch a single transcript body from `{nnnn}/{filename}` in the repo.
The caller is expected to have validated `nnnn` and `filename`
against `_is_valid_session_dir` / `_is_valid_transcript_filename`
before calling this invalid paths shouldn't reach the network.
"""
cache_key = f"transcript:{nnnn}/{filename}"
cached = _cache_get(cache_key, _content_ttl())
if cached is not None:
return cached
url = f"{_raw_base()}/{nnnn}/{filename}"
status, body = await _http_get(url)
if status == 200:
payload: dict[str, Any] = {"status": "ok", "body": body}
_cache_put(cache_key, payload)
return payload
if status == 404:
payload = {"status": "404"}
_cache_put(cache_key, payload)
return payload
return {"status": "error", "detail": f"upstream returned {status}"}
async def fetch_session_index(nnnn: str) -> dict[str, Any]:
"""List the transcript filenames inside the `{nnnn}/` folder.
Uses gitea's contents API (one HTTP per session-index page-view per
cache-TTL) rather than the raw URL there's no flat way to list a
folder via the raw mount.
Returns one of:
{"status": "ok", "files": ["SESSION-...md", ...]}
{"status": "404"}
{"status": "error", "detail": "..."}
Only filenames that match `_is_valid_transcript_filename` are
surfaced sibling files (e.g. an attached `notes.md`) are ignored
so the /docs/sessions/<NNNN> page never lists a non-transcript
masquerading as one.
"""
cache_key = f"index:{nnnn}"
cached = _cache_get(cache_key, _content_ttl())
if cached is not None:
return cached
url = f"{_contents_base()}/{nnnn}"
status, body = await _http_get(url)
if status == 200:
try:
import json
data = json.loads(body)
except (json.JSONDecodeError, ValueError) as e:
return {
"status": "error",
"detail": f"contents API response malformed: {e}",
}
if not isinstance(data, list):
return {
"status": "error",
"detail": "contents API returned non-list",
}
files: list[str] = []
for entry in data:
if not isinstance(entry, dict):
continue
if entry.get("type") != "file":
continue
name = entry.get("name")
if not isinstance(name, str):
continue
if _is_valid_transcript_filename(name):
files.append(name)
files.sort()
payload: dict[str, Any] = {"status": "ok", "files": files}
_cache_put(cache_key, payload)
return payload
if status == 404:
payload = {"status": "404"}
_cache_put(cache_key, payload)
return payload
return {"status": "error", "detail": f"upstream returned {status}"}
+326
View File
@@ -0,0 +1,326 @@
"""v0.20.0 — on-site framework-specs surface source.
Sibling of `docs_sessions.py` (v0.19.0 / roadmap item #30): the
framework mediates a gitea fetch on behalf of the browser so the
rendered `/docs/specs/*` surface inherits the same chrome as
`/docs/user-guide` and `/docs/sessions/*` and stays free of any
cross-origin gestures from the frontend.
Two read endpoints, both anonymous-reachable:
GET /api/docs/specs/manifest the configured spec list
GET /api/docs/specs/<name> a single spec body (markdown)
The framework-default manifest is OHM-flavored (rfc-app's own SPEC.md
+ flotilla's SPEC.md on `git.wiggleverse.org`) for the same reason
`docs_sessions.py`'s defaults are: OHM is the only deployment to
date. A deployment running its own spec set overrides the manifest
via the `OHM_DOCS_SPECS` env var (set through flotilla's overlay).
History is intentionally not surfaced here the operator-stated
intent is "current version only; git is the history surface".
Per-spec entries carry three fields:
name URL-safe slug (`[a-z0-9-]+`) the path segment
title human-readable label shown in the nav and the page header
url the upstream raw URL the framework fetches
Validation:
- The configured list must be a JSON array of `{name, title, url}`
objects. A malformed `OHM_DOCS_SPECS` value (bad JSON, wrong
shape, invalid slug) logs a warning and falls back to the default
so a typo in the overlay doesn't crash startup.
- Each `name` is checked against `^[a-z0-9-]+$` before the manifest
is accepted. The route layer also validates the path-bound `name`
parameter before any network call, so a malformed URL never
reaches the cache or the upstream.
Cache shape mirrors `docs_sessions.py`: in-process `dict` + monotonic
TTL check, negative results (404) cached, no external dep. The
manifest is cheap (parsed from an env var, no network), so it has no
TTL every request re-derives it. Per-spec content has a 5-minute
default TTL (env-tunable via `OHM_DOCS_SPECS_CONTENT_TTL_SEC`).
§3 invariant 1 is preserved: the framework holds no secret bytes;
the upstream specs are public-repo raw URLs, the fetch carries no
auth header.
"""
from __future__ import annotations
import json
import logging
import os
import re
import threading
import time
from typing import Any
import httpx
log = logging.getLogger(__name__)
# The framework-default spec set. OHM-flavored per the same precedent
# `docs_sessions.py` set: the only live deployment is OHM, so the
# default points there. A deployment running its own specs overrides
# `OHM_DOCS_SPECS` via the overlay.
_DEFAULT_SPECS: list[dict[str, str]] = [
{
"name": "rfc-app",
"title": "rfc-app SPEC",
"url": (
"https://git.wiggleverse.org/ben.stull/rfc-app/"
"raw/branch/main/SPEC.md"
),
},
{
"name": "flotilla",
"title": "flotilla SPEC",
"url": (
"https://git.wiggleverse.org/wiggleverse/ohm-rfc-app-flotilla/"
"raw/branch/main/SPEC.md"
),
},
]
_DEFAULT_CONTENT_TTL_SEC = 300.0
# URL-safe slug. Matches `docs_sessions.py`'s `_SESSION_DIR_RE` spirit
# (rejecting anything that could resolve outside the intended layout)
# but with the lowercase-alphanumeric-plus-dash shape the manifest
# enforces. Path traversal (`..`), separators (`/`), tilde, uppercase,
# and whitespace all fail this regex; the route layer rejects 400
# before any cache or network call.
_NAME_RE = re.compile(r"^[a-z0-9-]+$")
_HTTP_TIMEOUT_SEC = 5.0
def _env_float(name: str, default: float) -> float:
raw = os.environ.get(name, "").strip()
if not raw:
return default
try:
return float(raw)
except ValueError:
log.warning("invalid %s=%r — falling back to %s", name, raw, default)
return default
def _content_ttl() -> float:
return _env_float("OHM_DOCS_SPECS_CONTENT_TTL_SEC", _DEFAULT_CONTENT_TTL_SEC)
def _is_valid_name(name: str) -> bool:
"""Slug guard for path-bound `name` parameters.
Mirrors `docs_sessions._is_valid_session_dir`'s contract: the
route layer calls this before any network or cache work, so a
malformed name never escapes the FastAPI surface.
"""
return bool(isinstance(name, str) and _NAME_RE.match(name))
def _parse_spec_entry(entry: Any) -> dict[str, str] | None:
"""Validate a single manifest entry; return None if invalid.
Required fields: `name`, `title`, `url`. All three must be
non-empty strings; `name` must match `_NAME_RE`. The validator is
strict: an entry that fails any check is dropped from the manifest
(and the caller logs at warning level).
"""
if not isinstance(entry, dict):
return None
name = entry.get("name")
title = entry.get("title")
url = entry.get("url")
if not isinstance(name, str) or not _is_valid_name(name):
return None
if not isinstance(title, str) or not title.strip():
return None
if not isinstance(url, str) or not url.strip():
return None
return {"name": name, "title": title.strip(), "url": url.strip()}
def _load_configured_specs() -> list[dict[str, str]]:
"""Parse `OHM_DOCS_SPECS` (if set) or return the default list.
Malformed JSON or wrong-shape values log a warning and fall back
to the default the deployment continues to render the spec
surface rather than crashing startup. The strict validation (each
entry's name slug, presence of all three fields) drops bad entries
one-by-one; if every entry is dropped, the default applies.
"""
raw = os.environ.get("OHM_DOCS_SPECS", "").strip()
if not raw:
return list(_DEFAULT_SPECS)
try:
parsed = json.loads(raw)
except (json.JSONDecodeError, ValueError) as e:
log.warning(
"OHM_DOCS_SPECS is not valid JSON (%s) — falling back to default", e
)
return list(_DEFAULT_SPECS)
if not isinstance(parsed, list):
log.warning(
"OHM_DOCS_SPECS must be a JSON array — falling back to default"
)
return list(_DEFAULT_SPECS)
out: list[dict[str, str]] = []
seen: set[str] = set()
for entry in parsed:
validated = _parse_spec_entry(entry)
if validated is None:
log.warning(
"OHM_DOCS_SPECS entry %r failed validation — dropped", entry
)
continue
if validated["name"] in seen:
log.warning(
"OHM_DOCS_SPECS has duplicate name %r — dropped", validated["name"]
)
continue
seen.add(validated["name"])
out.append(validated)
if not out:
log.warning(
"OHM_DOCS_SPECS yielded no valid entries — falling back to default"
)
return list(_DEFAULT_SPECS)
return out
# ---------------------------------------------------------------------------
# In-process TTL cache
# ---------------------------------------------------------------------------
#
# Same shape as `docs_sessions.py`: plain dict + `time.monotonic()` check,
# no external dep. The cache value is a `(stored_at, payload)` tuple;
# `payload` may carry an error-shape sentinel for negative caching (404s).
# Lock guards read-modify-write across worker tasks; entries are immutable
# once stored so reads under the lock are fast.
_lock = threading.Lock()
_cache: dict[str, tuple[float, dict[str, Any]]] = {}
def _cache_get(key: str, ttl_sec: float) -> dict[str, Any] | None:
with _lock:
entry = _cache.get(key)
if entry is None:
return None
stored_at, payload = entry
if time.monotonic() - stored_at > ttl_sec:
return None
return payload
def _cache_put(key: str, payload: dict[str, Any]) -> None:
with _lock:
_cache[key] = (time.monotonic(), payload)
def reset_cache() -> None:
"""Drop every cached entry. Test seam — not called in production."""
with _lock:
_cache.clear()
# ---------------------------------------------------------------------------
# Public fetch surface
# ---------------------------------------------------------------------------
#
# Each fetcher returns a `{status, ...}` dict, same convention as
# `docs_sessions.py`:
# "ok" — payload field carries the body / manifest
# "404" — gitea returned 404 (or the configured name doesn't exist)
# "error" — gitea returned 5xx, timed out, or returned malformed data
#
# The route layer maps these onto HTTP responses; keeping the mapping
# out of this module makes the cache transparent to the test harness.
async def _http_get(url: str) -> tuple[int, str]:
"""Perform a single GET against `url`; return (status_code, body).
On timeout or network error, returns (599, error_message). The 599
pseudo-status maps to a 502 at the route layer the same way an
upstream 5xx does the caller doesn't care which leg of the
network broke.
"""
try:
async with httpx.AsyncClient(timeout=_HTTP_TIMEOUT_SEC) as client:
r = await client.get(url)
return r.status_code, r.text
except httpx.HTTPError as e:
log.warning("specs fetch failed for %s: %s", url, e)
return 599, f"fetch error: {e}"
def fetch_specs_manifest() -> dict[str, Any]:
"""Return the configured spec manifest.
The manifest is derived from the `OHM_DOCS_SPECS` env var (or the
framework default if unset / malformed) and carries no network
work it's safe to call on every request. The return shape mirrors
the docs_sessions manifest endpoint for frontend consistency:
{"status": "ok", "specs": [{"name", "title", "url"}, ...]}
The "url" field is exposed in the manifest so the frontend can
offer a "view source on gitea" affordance alongside each rendered
spec (operator-stated intent: "include the history so you can see
it in git" — that gesture lives in the source link, not on the
rendered page).
"""
specs = _load_configured_specs()
return {"status": "ok", "specs": specs}
async def fetch_spec(name: str) -> dict[str, Any]:
"""Fetch a single spec body by its manifest `name`.
The caller is expected to have validated `name` against
`_is_valid_name` before calling this an invalid name shouldn't
reach the network. We re-check inside as defense-in-depth: a
bogus name here returns the same `{status: "404"}` shape so the
route layer's `404 → HTTP 404` mapping handles it uniformly.
Returns one of:
{"status": "ok", "body": "..."}
{"status": "404"} no such spec OR upstream 404
{"status": "error", "detail": "..."} upstream 5xx / timeout
"""
if not _is_valid_name(name):
return {"status": "404"}
cache_key = f"spec:{name}"
cached = _cache_get(cache_key, _content_ttl())
if cached is not None:
return cached
specs = _load_configured_specs()
match = next((s for s in specs if s["name"] == name), None)
if match is None:
# Cache the negative — a deployment with an unstable manifest
# would still benefit from the TTL window, and the cached 404
# is automatically displaced when the next request happens
# after TTL expiry.
payload: dict[str, Any] = {"status": "404"}
_cache_put(cache_key, payload)
return payload
url = match["url"]
status, body = await _http_get(url)
if status == 200:
payload = {"status": "ok", "body": body}
_cache_put(cache_key, payload)
return payload
if status == 404:
payload = {"status": "404"}
_cache_put(cache_key, payload)
return payload
return {"status": "error", "detail": f"upstream returned {status}"}
+165 -10
View File
@@ -24,7 +24,6 @@ import os
import smtplib
from dataclasses import dataclass
from datetime import datetime, time, timezone
from email.message import EmailMessage
from email.utils import formataddr
from itertools import groupby
from typing import Any
@@ -33,6 +32,7 @@ from urllib.parse import urlencode
from itsdangerous import BadSignature, URLSafeSerializer
from . import db
from .email_envelope import build_envelope
log = logging.getLogger(__name__)
@@ -69,6 +69,7 @@ class EmailConfig:
app_url: str
bundle_threshold: int
enabled: bool
unsubscribe_mailto: str
@classmethod
def from_env(cls) -> "EmailConfig":
@@ -84,6 +85,16 @@ class EmailConfig:
app_url=os.environ.get("APP_URL", "http://localhost:8000").rstrip("/"),
bundle_threshold=int(os.environ.get("EMAIL_BUNDLE_THRESHOLD", "5")),
enabled=os.environ.get("EMAIL_ENABLED", "1") not in ("0", "false", "False"),
# v0.18.0: the `List-Unsubscribe: <mailto:…>` target on
# invite + notification mail. Defaults to the From
# address when unset; a deployment can route opt-out
# mail to a separate mailbox (e.g., a humans-monitored
# account distinct from the no-reply notifications
# sender) by setting this explicitly.
unsubscribe_mailto=os.environ.get(
"EMAIL_UNSUBSCRIBE_MAILTO",
os.environ.get("EMAIL_FROM", "notifications@wiggleverse.local"),
).strip(),
)
@@ -98,6 +109,14 @@ def _signer() -> URLSafeSerializer:
def make_unsubscribe_url(user_id: int, category: str) -> str:
"""Build the §15.4 per-category one-click URL.
`category` is one of `personal-direct`, `structural`,
`admin-actionable` (the three per-category flags) or `all`
(v0.18.0: the bundle path, which sets `email_opt_out_all = 1`
because a bundle covers multiple categories and a per-category
opt-out wouldn't honor the user's intent).
"""
cfg = EmailConfig.from_env()
token = _signer().dumps({"u": user_id, "c": category})
qs = urlencode({"t": token})
@@ -250,7 +269,21 @@ def _send_one(user: Any, notif_id: int, payload: dict, category: str) -> None:
return
subject = _subject(payload)
body = _body(payload, user["id"], category, cfg)
sent = _deliver(cfg, user["email"], subject, body)
# v0.18.0: notification mail is bulk-adjacent (a watcher can
# accumulate dozens of structural events on a busy RFC), so it
# carries the full one-click unsubscribe — Gmail and Yahoo
# require this for senders at OHM's volume tier per RFC 8058.
unsubscribe_url = make_unsubscribe_url(user["id"], category)
sent = _deliver(
cfg,
user["email"],
subject,
body,
unsubscribe_mailto=cfg.unsubscribe_mailto,
unsubscribe_url=unsubscribe_url,
kind="notification",
notification_id=notif_id,
)
if not sent:
return
db.conn().execute(
@@ -305,23 +338,65 @@ def _deep_link(payload: dict, cfg: EmailConfig) -> str:
return cfg.app_url
def _deliver(cfg: EmailConfig, to_address: str, subject: str, body: str) -> bool:
def _deliver(
cfg: EmailConfig,
to_address: str,
subject: str,
body: str,
*,
unsubscribe_mailto: str | None = None,
unsubscribe_url: str | None = None,
kind: str = "notification",
notification_id: int | None = None,
) -> bool:
"""Build the envelope via the shared `build_envelope` helper and
hand it to SMTP.
The `_SENT` buffer carries the helper's `EmailMessage` under
`message` plus the legacy `to`/`from`/`subject`/`body` keys for
backward-compatibility with tests that read those directly.
Newer tests can assert on the header surface by inspecting
`envelope["message"]`.
v0.18.0 Slice 4: also writes one row to `outbound_emails`
capturing the attempt. status='sent' on success, 'failed' on
SMTP exception, 'deferred' on the dev-fallback path (no
SMTP_HOST configured the send didn't happen, but the row
records the attempt so the admin endpoint can answer "did the
framework try?").
"""
msg = build_envelope(
to_address=to_address,
from_address=cfg.from_address,
from_name=cfg.from_name,
subject=subject,
body_plain=body,
unsubscribe_mailto=unsubscribe_mailto,
unsubscribe_url=unsubscribe_url,
)
envelope = {
"to": to_address,
"from": formataddr((cfg.from_name, cfg.from_address)),
"subject": subject,
"body": body,
"message": msg,
"kind": kind,
}
_SENT.append(envelope)
message_id = msg["Message-ID"]
if not cfg.smtp_host:
log.info("email (stdout fallback): to=%s subject=%s", to_address, subject)
record_outbound(
to_address=to_address,
from_address=cfg.from_address,
subject=subject,
kind=kind,
status="deferred",
message_id=message_id,
notification_id=notification_id,
)
return True
try:
msg = EmailMessage()
msg["From"] = envelope["from"]
msg["To"] = to_address
msg["Subject"] = subject
msg.set_content(body)
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
try:
if cfg.smtp_starttls:
@@ -331,12 +406,78 @@ def _deliver(cfg: EmailConfig, to_address: str, subject: str, body: str) -> bool
smtp.send_message(msg)
finally:
smtp.quit()
record_outbound(
to_address=to_address,
from_address=cfg.from_address,
subject=subject,
kind=kind,
status="sent",
message_id=message_id,
notification_id=notification_id,
)
return True
except Exception:
except Exception as exc:
log.exception("email send failed: to=%s subject=%s", to_address, subject)
record_outbound(
to_address=to_address,
from_address=cfg.from_address,
subject=subject,
kind=kind,
status="failed",
error=f"{type(exc).__name__}: {exc}",
message_id=message_id,
notification_id=notification_id,
)
return False
def record_outbound(
*,
to_address: str,
from_address: str,
subject: str,
kind: str,
status: str,
error: str | None = None,
notification_id: int | None = None,
message_id: str | None = None,
) -> int | None:
"""v0.18.0 Slice 4: write one row to `outbound_emails`.
Returns the inserted row's id, or `None` if the DB connection
isn't initialized (which happens in unit tests that don't boot
the full app the write is best-effort and never raises).
"""
try:
cur = db.conn().execute(
"""
INSERT INTO outbound_emails
(to_address, from_address, subject, kind, sent_at, status,
error, notification_id, message_id)
VALUES (?, ?, ?, ?, datetime('now'), ?, ?, ?, ?)
""",
(
to_address,
from_address,
subject,
kind,
status,
error,
notification_id,
message_id,
),
)
return cur.lastrowid
except RuntimeError:
# db.conn() raises RuntimeError if init() hasn't been called.
# Pure-helper unit tests for build_envelope hit this path; the
# audit row is best-effort and not part of the contract.
return None
except Exception:
log.exception("outbound_emails write failed: to=%s subject=%s", to_address, subject)
return None
# ---------------------------------------------------------------------------
# Quiet-hours release pass — called from the digest job
# ---------------------------------------------------------------------------
@@ -440,13 +581,27 @@ def _send_bundle(cfg: EmailConfig, user: Any, emailable: list) -> int:
for r, _cat, extras in group_rows:
summary = _summary_for(r["event_kind"], r["actor_display"], r["rfc_title"], extras)
sections.append(f" · {summary}")
# v0.18.0: the bundle covers multiple categories, so a
# per-category opt-out can't honor the user's intent. The
# `all` category lands at the §15.4 endpoint and sets
# `email_opt_out_all = 1`.
unsubscribe_url = make_unsubscribe_url(user["id"], "all")
body = (
"Activity on RFCs you watch, accumulated during your quiet hours:\n"
+ "\n".join(sections)
+ f"\n\nOpen your inbox: {cfg.app_url}/inbox\n"
+ f"Manage all preferences: {cfg.app_url}/settings/notifications\n"
+ f"Unsubscribe from all email: {unsubscribe_url}\n"
)
sent = _deliver(
cfg,
user["email"],
subject,
body,
unsubscribe_mailto=cfg.unsubscribe_mailto,
unsubscribe_url=unsubscribe_url,
kind="bundle",
)
sent = _deliver(cfg, user["email"], subject, body)
if not sent:
return 0
ids = [r["id"] for r, _, _ in emailable]
+155
View File
@@ -0,0 +1,155 @@
"""v0.18.0 / roadmap items #18 + #20: a shared envelope builder.
Every outbound mail in rfc-app today (OTC, admin-invite, watcher
notification, "while you were away" bundle, per-RFC invite) constructs
its own `email.message.EmailMessage` ad-hoc. The four sites diverged
just enough to be a deliverability hazard: missing `Date`, missing
`Message-ID`, no `Auto-Submitted`, no `List-Unsubscribe` on the
bulk-adjacent paths, no `multipart/alternative` body.
This module is the one place an `EmailMessage` is constructed. Every
send path imports `build_envelope` and calls it; the headers that
matter for inbox placement (Date, Message-ID, Auto-Submitted) land
uniformly, and the per-kind variations (unsubscribe semantics,
HTML alternative) are explicit arguments rather than buried in
each call site.
Per the v0.18.0 proposal at `~/git/ohm-infra/RFC-APP-EMAIL-HYGIENE-PROPOSAL.md`,
the per-kind unsubscribe matrix is:
* OTC: no `List-Unsubscribe` (the recipient explicitly requested
the code; advertising an unsubscribe header would imply OHM has
them on a list, which it doesn't).
* Admin invite / per-RFC invite: `mailto:` form only (the
recipient isn't a user yet, so there's no per-user opt-out row
to flip; the operator handles ad-hoc opt-outs manually).
* Watcher notification / bundle: full `mailto:` + signed-URL
`List-Unsubscribe` plus `List-Unsubscribe-Post:
List-Unsubscribe=One-Click` per RFC 8058 (Gmail and Yahoo
enforce this for bulk-adjacent senders).
The `is_transactional` flag governs `Auto-Submitted: auto-generated`,
which prevents auto-responder loops on every kind of mail we send.
All five mail kinds today are transactional in the SMTP sense (no
human is at the From mailbox watching for replies), so the default
is True; the argument is exposed for future symmetry.
"""
from __future__ import annotations
from email.message import EmailMessage
from email.utils import formataddr, formatdate, make_msgid
def build_envelope(
*,
to_address: str,
from_address: str,
from_name: str,
subject: str,
body_plain: str,
body_html: str | None = None,
reply_to: str | None = None,
unsubscribe_mailto: str | None = None,
unsubscribe_url: str | None = None,
is_transactional: bool = True,
msgid_domain: str | None = None,
) -> EmailMessage:
"""Compose an `EmailMessage` with hardened headers.
`to_address` / `from_address` are bare RFC 5322 addresses;
`from_name` is the display label that goes through `formataddr`
so spaces / commas in the display string are encoded correctly.
`body_plain` is mandatory. `body_html` is **reserved and not yet
enabled** (security-audit-0026 I3): no send path supplies it today
every rfc-app mail is plain text and passing it raises
`NotImplementedError`. The parameter is kept in the signature for
documented future symmetry: when HTML mail is enabled it will land
as the second part of a `multipart/alternative` body (text/plain
first per RFC 2046, so a plain-text client picking the first part
still gets the readable text). Enabling it is a deliberate act the
caller MUST HTML-escape any user content into `body_html` first (cf.
the C1 stored-XSS class: a mail client renders the HTML) and remove
the guard below in the same change.
`reply_to`, when set, lets a send path point replies at a
different mailbox than the From line (e.g., a watcher
notification with From=notifications@... but Reply-To=
ohm@... so a confused recipient who hits Reply lands at a
monitored mailbox).
`unsubscribe_mailto` / `unsubscribe_url` populate
`List-Unsubscribe`. If `unsubscribe_url` is set, the helper also
emits `List-Unsubscribe-Post: List-Unsubscribe=One-Click` per
RFC 8058 Gmail and Yahoo POST that payload on the user's
one-click action. (Send paths that wire `unsubscribe_url`
therefore MUST also expose a matching POST endpoint that accepts
the same token; see `api_notifications.py:email_unsubscribe`.)
`msgid_domain` defaults to the @-domain of `from_address` so
Message-IDs are aligned with the sending domain by default. A
deployment that wants the Message-ID domain to track a different
surface (e.g., a tracking-domain that's separate from the From
domain) can override.
`Date` is RFC 5322 formatted via `email.utils.formatdate`; the
`localtime=True` setting picks the running process's local
timezone, which is what every popular MUA does too. (A
deployment running in UTC stamps UTC; that's correct, not a
bug.)
"""
msg = EmailMessage()
msg["From"] = formataddr((from_name, from_address))
msg["To"] = to_address
msg["Subject"] = subject
msg["Date"] = formatdate(localtime=True)
# If the caller didn't pin a Message-ID domain, derive it from the
# From address. `make_msgid` accepts None and falls back to the
# local hostname, which is the wrong shape for a deliverable
# message (the hostname might be `gke-pool-xxx`); a deployment
# without a configured From would surface that as a build-time
# config error elsewhere, so the fallback here is just defensive.
if msgid_domain is None:
if "@" in from_address:
msgid_domain = from_address.split("@", 1)[1]
else:
msgid_domain = "localhost"
msg["Message-ID"] = make_msgid(domain=msgid_domain)
if reply_to:
msg["Reply-To"] = reply_to
if is_transactional:
# RFC 3834: prevents auto-responders (vacation replies, etc.)
# from triggering on this message. Every kind of mail rfc-app
# sends today is transactional in this sense.
msg["Auto-Submitted"] = "auto-generated"
if unsubscribe_mailto or unsubscribe_url:
parts: list[str] = []
if unsubscribe_mailto:
parts.append(f"<mailto:{unsubscribe_mailto}>")
if unsubscribe_url:
parts.append(f"<{unsubscribe_url}>")
msg["List-Unsubscribe"] = ", ".join(parts)
if unsubscribe_url:
# RFC 8058 one-click. Gmail and Yahoo POST the payload
# `List-Unsubscribe=One-Click` to the URL on the user's
# one-click action; the matching POST endpoint must be
# idempotent and not require auth. See
# `api_notifications.py` for the receiver.
msg["List-Unsubscribe-Post"] = "List-Unsubscribe=One-Click"
if body_html is not None:
# I3 (security-audit-0026): the multipart/alternative HTML path
# is intentionally NOT enabled. No send path passes `body_html`
# today, and emitting an HTML body built from user-supplied
# content without escaping it first would reintroduce the C1
# stored-XSS class in the mail channel (the recipient's client
# renders the HTML). Fail loudly here rather than silently
# shipping HTML: enabling HTML mail is a deliberate change that
# MUST HTML-escape user content at the call site and remove this
# guard together. The text/plain path below is the only live one.
raise NotImplementedError(
"HTML email is not enabled (security-audit-0026 I3): do not "
"pass body_html until user content is HTML-escaped at the "
"call site and this guard is intentionally removed."
)
msg.set_content(body_plain)
return msg
+38 -8
View File
@@ -31,10 +31,10 @@ from __future__ import annotations
import logging
import smtplib
from email.message import EmailMessage
from email.utils import formataddr
from .email import EmailConfig, _SENT
from .email import EmailConfig, _SENT, record_outbound
from .email_envelope import build_envelope
log = logging.getLogger(__name__)
@@ -59,31 +59,52 @@ def send_invite_email(
cfg = EmailConfig.from_env()
subject = _subject(inviter_display, cfg)
body = _body(claim_url, inviter_display, inviter_email, custom_message, cfg)
# v0.18.0: invite mail carries a `List-Unsubscribe: <mailto:…>`
# only (no signed URL) — the invitee isn't a user yet, so there
# is no per-user opt-out row to flip. The operator handles
# ad-hoc opt-outs from the mailto: target. Per the proposal's
# "Tradeoff discussion": the invite was unsolicited from the
# recipient's perspective, so the courtesy header is right;
# but it can't be a one-click URL because the row doesn't
# exist yet.
msg = build_envelope(
to_address=to_address,
from_address=cfg.from_address,
from_name=cfg.from_name,
subject=subject,
body_plain=body,
unsubscribe_mailto=cfg.unsubscribe_mailto,
)
envelope = {
"to": to_address,
"from": formataddr((cfg.from_name, cfg.from_address)),
"subject": subject,
"body": body,
"kind": "invite",
"message": msg,
}
_SENT.append(envelope)
message_id = msg["Message-ID"]
if not cfg.enabled:
log.info("invite email disabled (EMAIL_ENABLED=0): to=%s", to_address)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="invite", status="deferred", message_id=message_id,
)
return True
if not cfg.smtp_host:
# Dev fallback: surface the claim URL at INFO so the operator can
# complete a claim flow without an SMTP relay. In production
# SMTP_HOST is always set per OHM's overlay.
log.info("invite email (stdout fallback): to=%s claim_url=%s", to_address, claim_url)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="invite", status="deferred", message_id=message_id,
)
return True
try:
msg = EmailMessage()
msg["From"] = envelope["from"]
msg["To"] = to_address
msg["Subject"] = subject
msg.set_content(body)
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
try:
if cfg.smtp_starttls:
@@ -93,9 +114,18 @@ def send_invite_email(
smtp.send_message(msg)
finally:
smtp.quit()
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="invite", status="sent", message_id=message_id,
)
return True
except Exception:
except Exception as exc:
log.exception("invite email send failed: to=%s", to_address)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="invite", status="failed",
error=f"{type(exc).__name__}: {exc}", message_id=message_id,
)
return False
+34 -8
View File
@@ -23,10 +23,10 @@ from __future__ import annotations
import logging
import smtplib
from email.message import EmailMessage
from email.utils import formataddr
from .email import EmailConfig, _SENT
from .email import EmailConfig, _SENT, record_outbound
from .email_envelope import build_envelope
log = logging.getLogger(__name__)
@@ -44,31 +44,48 @@ def send_otc_email(to_address: str, code: str) -> bool:
cfg = EmailConfig.from_env()
subject = f"Your sign-in code for {cfg.from_name}"
body = _body(code, cfg)
# v0.18.0: OTC mail carries NO List-Unsubscribe — the recipient
# explicitly requested the code; advertising an unsubscribe
# header would imply OHM has them on a list, which it doesn't.
# See the proposal's "Tradeoff discussion" for the binding
# rationale.
msg = build_envelope(
to_address=to_address,
from_address=cfg.from_address,
from_name=cfg.from_name,
subject=subject,
body_plain=body,
)
envelope = {
"to": to_address,
"from": formataddr((cfg.from_name, cfg.from_address)),
"subject": subject,
"body": body,
"kind": "otc",
"message": msg,
}
_SENT.append(envelope)
message_id = msg["Message-ID"]
if not cfg.enabled:
log.info("otc email disabled (EMAIL_ENABLED=0): to=%s", to_address)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="otc", status="deferred", message_id=message_id,
)
return True
if not cfg.smtp_host:
# Dev fallback: surface the code at INFO so the operator can
# complete a sign-in flow without an SMTP relay. In production
# SMTP_HOST is always set per OHM's overlay.
log.info("otc email (stdout fallback): to=%s code=%s", to_address, code)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="otc", status="deferred", message_id=message_id,
)
return True
try:
msg = EmailMessage()
msg["From"] = envelope["from"]
msg["To"] = to_address
msg["Subject"] = subject
msg.set_content(body)
smtp = smtplib.SMTP(cfg.smtp_host, cfg.smtp_port, timeout=30)
try:
if cfg.smtp_starttls:
@@ -78,9 +95,18 @@ def send_otc_email(to_address: str, code: str) -> bool:
smtp.send_message(msg)
finally:
smtp.quit()
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="otc", status="sent", message_id=message_id,
)
return True
except Exception:
except Exception as exc:
log.exception("otc email send failed: to=%s", to_address)
record_outbound(
to_address=to_address, from_address=cfg.from_address,
subject=subject, kind="otc", status="failed",
error=f"{type(exc).__name__}: {exc}", message_id=message_id,
)
return False
+61 -19
View File
@@ -7,6 +7,7 @@ no need for a separate worker.
from __future__ import annotations
import logging
import os
import secrets
from contextlib import asynccontextmanager
@@ -28,6 +29,7 @@ from . import (
otc,
passcode as passcode_mod,
providers as providers_mod,
ratelimit,
turnstile,
webhooks,
)
@@ -142,12 +144,20 @@ def create_app() -> FastAPI:
# eagerly via load_config(). Everything else waits for lifespan.
config = load_config()
app = FastAPI(lifespan=lifespan)
# v0.25.0 (audit 0026 M4): the session cookie is the primary 30-day
# auth credential and must carry `Secure` in production so it never
# travels cleartext. Default to Secure; a dev box serving over plain
# http opts out with SESSION_COOKIE_SECURE=false. Production (OHM is
# HTTPS-only with an HTTP->HTTPS 301) leaves this unset → Secure on.
session_secure = os.environ.get("SESSION_COOKIE_SECURE", "true").strip().lower() not in (
"0", "false", "no", "off",
)
app.add_middleware(
SessionMiddleware,
secret_key=config.secret_key,
session_cookie="rfc_session",
max_age=60 * 60 * 24 * 30,
https_only=False,
https_only=session_secure,
)
return app
@@ -155,24 +165,25 @@ def create_app() -> FastAPI:
app = create_app()
def _set_device_trust_cookie(response: Response, raw_token: str) -> None:
def _set_device_trust_cookie(response: Response, cookie_value: str) -> None:
"""Attach the v0.11.0 device-trust cookie to the response.
HttpOnly + Secure + SameSite=Lax + 30-day Max-Age + Path=/. The
cookie value is the raw token; server-side storage is the hash.
The cookie is "essential" per the v0.13.0 cookie-consent contract
(it is part of authentication), so we set it regardless of the
user's analytics / other-cookies choice.
HttpOnly + Secure + SameSite=Lax + 30-day Max-Age + Path=/. As of
v0.25.0 (audit 0026 M1) the value is `IssueOutcome.cookie_value`
"<row_id>.<raw_token>" so `device_trust.lookup` can read one indexed
row instead of scanning; server-side storage remains the bcrypt hash
of the token half only. The cookie is "essential" per the v0.13.0
cookie-consent contract (it is part of authentication), so we set it
regardless of the user's analytics / other-cookies choice.
Secure=True means the cookie is only ever sent over HTTPS. The
SessionMiddleware in `create_app` keeps `https_only=False` for
dev parity, but the device-trust cookie holds a 30-day credential
and must not travel cleartext production deployments serve over
HTTPS, so Secure on the device-trust cookie is non-negotiable.
Secure=True means the cookie is only ever sent over HTTPS the
device-trust cookie holds a 30-day credential and must never travel
cleartext. (The session cookie now also defaults to Secure; see M4 in
`create_app`.)
"""
response.set_cookie(
key=device_trust_mod.COOKIE_NAME,
value=raw_token,
value=cookie_value,
max_age=device_trust_mod.COOKIE_MAX_AGE_SECONDS,
path="/",
secure=True,
@@ -245,6 +256,10 @@ def _oauth_router(config) -> APIRouter:
@router.post("/auth/otc/request")
async def otc_request(body: OtcRequestBody, request: Request):
# v0.25.0 (audit 0026 H1/L2): per-IP brake at the cheapest point,
# before the Turnstile network call or any bcrypt/SMTP work.
if not ratelimit.otc_request_limiter.allow(ratelimit.client_key(request)):
raise HTTPException(429, "Too many requests; please wait a few minutes")
# v0.12.0 / roadmap item #10: gate the request on a successful
# Turnstile siteverify before the bcrypt hash + SMTP send. The
# check runs first so a failed challenge spends no rate budget
@@ -252,7 +267,7 @@ def _oauth_router(config) -> APIRouter:
# secret AND TURNSTILE_REQUIRED=false (the default), the gate
# opens — see `backend/app/turnstile.py` for the full matrix.
client_ip = request.client.host if request.client else None
ts = turnstile.verify_token(body.turnstile_token, client_ip=client_ip)
ts = await turnstile.verify_token(body.turnstile_token, client_ip=client_ip)
if not ts.ok:
if ts.reason == "misconfigured":
# TURNSTILE_REQUIRED=true but the secret is unset. This
@@ -278,9 +293,25 @@ def _oauth_router(config) -> APIRouter:
@router.post("/auth/otc/verify")
async def otc_verify(body: OtcVerifyBody, request: Request, response: Response):
# v0.25.0 (audit 0026 H1): per-IP brake against fan-out guessing,
# plus the per-email lockout enforced inside otc.verify_code.
ip = ratelimit.client_key(request)
if not ratelimit.verify_limiter.allow(ip):
raise HTTPException(429, "Too many attempts; please wait a few minutes")
result = otc.verify_code(body.email, body.code)
if result.reason == "locked":
raise HTTPException(
423,
{
"detail": "Too many failed attempts; wait a few minutes or request a new code",
"locked_until": result.locked_until,
},
)
if not result.ok or result.user is None:
raise HTTPException(400, "Invalid or expired code")
# Legit sign-in: clear this IP's window so a user who fat-fingered
# a couple of codes isn't left throttled.
ratelimit.verify_limiter.reset(ip)
auth.store_session(request, result.user)
# v0.8.0: surface `needs_profile` so the Login.jsx surface can
# decide whether to advance to the first/last/why capture step
@@ -313,7 +344,7 @@ def _oauth_router(config) -> APIRouter:
if body.trust_device:
ua = request.headers.get("user-agent", "")
outcome = device_trust_mod.issue(result.user.user_id, ua)
_set_device_trust_cookie(response, outcome.raw_token)
_set_device_trust_cookie(response, outcome.cookie_value)
return {
"ok": True,
"user": {
@@ -337,12 +368,17 @@ def _oauth_router(config) -> APIRouter:
# ---------------------------------------------------------------
@router.get("/auth/passcode/check")
async def passcode_check(email: str = ""):
async def passcode_check(request: Request, email: str = ""):
"""Does this email have a passcode set? Anonymous endpoint —
the Login.jsx flow calls this after the user types their email
to decide whether to render a passcode input or fall back to
OTC. We surface only the boolean; lockout state, the hash, and
the set-at stamp are not leaked here."""
the set-at stamp are not leaked here.
v0.25.0 (audit 0026 L3): per-IP rate limit so the has-passcode
boolean can't be bulk-harvested to enumerate accounts."""
if not ratelimit.check_limiter.allow(ratelimit.client_key(request)):
raise HTTPException(429, "Too many requests; please wait a few minutes")
status = passcode_mod.passcode_status(email)
return {"has_passcode": status.has_passcode}
@@ -376,6 +412,11 @@ def _oauth_router(config) -> APIRouter:
v0.11.0: the body's `trust_device` flag, if true, mints a
fresh device-trust row and sets the long-lived cookie. Same
opt-in contract as `/auth/otc/verify`."""
# v0.25.0 (audit 0026 H1): per-IP brake in front of the per-account
# passcode lockout, so fan-out across emails is throttled too.
ip = ratelimit.client_key(request)
if not ratelimit.verify_limiter.allow(ip):
raise HTTPException(429, "Too many attempts; please wait a few minutes")
result = passcode_mod.verify_passcode(body.email, body.passcode)
if result.reason == "locked":
raise HTTPException(
@@ -387,11 +428,12 @@ def _oauth_router(config) -> APIRouter:
)
if not result.ok or result.user is None:
raise HTTPException(400, "Invalid passcode")
ratelimit.verify_limiter.reset(ip)
auth.store_session(request, result.user)
if body.trust_device:
ua = request.headers.get("user-agent", "")
outcome = device_trust_mod.issue(result.user.user_id, ua)
_set_device_trust_cookie(response, outcome.raw_token)
_set_device_trust_cookie(response, outcome.cookie_value)
return {
"ok": True,
"user": {
@@ -459,7 +501,7 @@ def _oauth_router(config) -> APIRouter:
if body.trust_device:
ua = request.headers.get("user-agent", "")
outcome = device_trust_mod.issue(result.user.user_id, ua)
_set_device_trust_cookie(response, outcome.raw_token)
_set_device_trust_cookie(response, outcome.cookie_value)
# Has the user already set a passcode? (Could only happen via
# an admin pre-population path that doesn't exist yet, but
+76
View File
@@ -85,6 +85,15 @@ def _cooldown_seconds() -> int:
return 60
# v0.25.0 / security audit 0026 (H1): per-email OTC verify lockout,
# mirroring the passcode path (passcode.py). Five consecutive wrong codes
# for an email lock its OTC verify for 15 minutes. The per-IP limiter in
# ratelimit.py is the primary brute-force brake; this is the durable,
# passcode-parity layer.
LOCKOUT_AFTER_FAILED_ATTEMPTS = 5
LOCKOUT_DURATION_MINUTES = 15
# ---------------------------------------------------------------------------
# Code generation + hashing
# ---------------------------------------------------------------------------
@@ -206,6 +215,61 @@ class VerifyOutcome:
ok: bool
user: SessionUser | None
reason: str
# v0.25.0 (H1): ISO-8601 stamp when reason == 'locked'.
locked_until: str | None = None
def _verify_lockout_until(email: str) -> str | None:
"""Return the active lockout stamp for `email`, or None if not locked.
Clears an elapsed lockout (and resets the counter) as a side effect so
the next failure starts a fresh budget mirrors passcode.verify_passcode.
"""
row = db.conn().execute(
"SELECT failed_attempts, locked_until FROM otc_verify_state WHERE email = ?",
(email,),
).fetchone()
if row is None or not row["locked_until"]:
return None
still_locked = db.conn().execute(
"SELECT datetime(?) > datetime('now') AS locked", (row["locked_until"],),
).fetchone()["locked"]
if still_locked:
return row["locked_until"]
db.conn().execute(
"UPDATE otc_verify_state SET failed_attempts = 0, locked_until = NULL WHERE email = ?",
(email,),
)
return None
def _record_verify_failure(email: str) -> None:
"""Increment the per-email failure counter; stamp a lockout once it
crosses the threshold. Mirrors the passcode lockout shape."""
db.conn().execute(
"""
INSERT INTO otc_verify_state (email, failed_attempts)
VALUES (?, 1)
ON CONFLICT(email) DO UPDATE SET failed_attempts = failed_attempts + 1
""",
(email,),
)
count = db.conn().execute(
"SELECT failed_attempts FROM otc_verify_state WHERE email = ?", (email,),
).fetchone()["failed_attempts"]
if count >= LOCKOUT_AFTER_FAILED_ATTEMPTS:
db.conn().execute(
f"""
UPDATE otc_verify_state
SET locked_until = datetime('now', '+{LOCKOUT_DURATION_MINUTES} minutes')
WHERE email = ?
""",
(email,),
)
def _clear_verify_state(email: str) -> None:
db.conn().execute("DELETE FROM otc_verify_state WHERE email = ?", (email,))
def verify_code(email: str, code: str) -> VerifyOutcome:
@@ -214,6 +278,13 @@ def verify_code(email: str, code: str) -> VerifyOutcome:
if not email or not code:
return VerifyOutcome(ok=False, user=None, reason="invalid")
# v0.25.0 (H1): refuse before spending any bcrypt if this email is in
# its OTC-verify lockout window. The passcode path is unaffected — a
# locked-out OTC user can still set/use a passcode, and vice versa.
locked_until = _verify_lockout_until(email)
if locked_until:
return VerifyOutcome(ok=False, user=None, reason="locked", locked_until=locked_until)
rows = db.conn().execute(
"""
SELECT id, code_hash, expires_at, consumed_at
@@ -237,6 +308,9 @@ def verify_code(email: str, code: str) -> VerifyOutcome:
break
if matched is None:
# A genuine wrong guess against this email — the brute-force
# signal. Count it toward the lockout threshold (H1).
_record_verify_failure(email)
return VerifyOutcome(ok=False, user=None, reason="wrong")
if matched["consumed_at"] is not None:
@@ -255,6 +329,8 @@ def verify_code(email: str, code: str) -> VerifyOutcome:
"UPDATE otc_codes SET consumed_at = datetime('now') WHERE id = ?",
(matched["id"],),
)
# Success wipes the per-email failure counter (H1).
_clear_verify_state(email)
user = provision_or_link_user(email)
return VerifyOutcome(ok=True, user=user, reason="ok")
+15
View File
@@ -184,6 +184,21 @@ def load_providers(env: dict) -> dict[str, BaseProvider]:
return providers
def construct_haiku(api_key: str) -> AnthropicProvider:
"""A dedicated Claude Haiku provider, independent of the
`ENABLED_MODELS` chat-picker universe.
The §9.1 tag-suggestion surface (roadmap #27) always wants the
cheap + fast model regardless of which models the operator surfaces
in the §8.12 picker, so it constructs Haiku directly from the
operator's Anthropic key rather than going through `load_providers`.
The model id is sourced from the same `_CLAUDE_VARIANTS` table the
picker uses, so a model-string bump lands in one place.
"""
model, name = _CLAUDE_VARIANTS["claude-haiku"]
return AnthropicProvider(api_key=api_key, model=model, display_name=name)
def load_from_config(config) -> dict[str, BaseProvider]:
"""Convenience adapter so callers can pass our Config dataclass directly."""
env = {
+92
View File
@@ -0,0 +1,92 @@
"""In-process per-IP sliding-window rate limiter (security audit 0026, H1).
The auth verify endpoints (`/auth/otc/verify`, `/auth/passcode/verify`)
had no per-IP brake, so an attacker could fan out guesses against a
target identity bounded only by bcrypt cost. This module is the brake.
It is deliberately tiny: §4.2 says the app is a single process with a
colocated SQLite file, so an in-memory dict of `key -> deque[timestamps]`
is sufficient and needs no shared store. State resets on restart, which
fails *open* for a brief window acceptable because the per-email OTC
lockout (`otc_verify_state`) and the passcode lockout both persist in the
database and carry the durable guarantee; this limiter is the
anti-fan-out layer on top.
Chosen over a per-identity lockout *as the primary control* because a
per-IP window throttles the attacker without letting them grief a victim
by locking that victim's account (the known downside of identity
lockouts). Both layers run together.
"""
from __future__ import annotations
import threading
import time
from collections import defaultdict, deque
class SlidingWindowLimiter:
"""Allow at most `max_events` per `window_seconds` per key.
`allow(key)` records an event and returns True if the key is still
within budget, False if it has exceeded it. Timestamps use a
monotonic clock so the limiter is immune to wall-clock jumps.
"""
def __init__(self, max_events: int, window_seconds: float) -> None:
self.max_events = max_events
self.window_seconds = window_seconds
self._events: dict[str, deque[float]] = defaultdict(deque)
self._lock = threading.Lock()
def allow(self, key: str) -> bool:
now = time.monotonic()
cutoff = now - self.window_seconds
with self._lock:
q = self._events[key]
while q and q[0] < cutoff:
q.popleft()
if len(q) >= self.max_events:
return False
q.append(now)
# Opportunistic cleanup so idle keys don't accumulate forever.
if not q:
self._events.pop(key, None)
return True
def reset(self, key: str) -> None:
"""Drop a key's window — e.g. after a successful sign-in so a
legitimate user who fat-fingered a few times isn't throttled."""
with self._lock:
self._events.pop(key, None)
# Module-level limiters shared across requests (one process, so module
# state is the natural home). Tunables are intentionally generous enough
# not to bother a human retyping a code, tight enough to kill fan-out:
# * verify: 10 attempts / 5 min / IP across the auth verify surfaces.
# * otc request: 5 sends / 5 min / IP (Turnstile is the primary gate;
# this is defense in depth against a solved-challenge replay loop).
verify_limiter = SlidingWindowLimiter(max_events=10, window_seconds=300)
otc_request_limiter = SlidingWindowLimiter(max_events=5, window_seconds=300)
# /auth/passcode/check is an anonymous has-passcode oracle (audit 0026 L3).
# It's a legitimate Login-flow affordance, so the budget is generous —
# enough for a human typing emails, tight enough to stop bulk scraping.
check_limiter = SlidingWindowLimiter(max_events=30, window_seconds=300)
def _reset_all_for_tests() -> None:
"""Clear every module-level limiter's window. Test support only — the
limiters are process-global singletons, so without a per-test reset
one test's requests bleed into the next and later tests trip the
budget (429). Not called in production."""
for lim in (verify_limiter, otc_request_limiter, check_limiter):
with lim._lock:
lim._events.clear()
def client_key(request) -> str:
"""Best-effort client identity for limiting. Behind nginx the app is
started with `--forwarded-allow-ips 127.0.0.1`, so `request.client.host`
reflects the real client IP via Uvicorn's ProxyHeaders handling."""
client = getattr(request, "client", None)
return client.host if client and client.host else "unknown"
+158
View File
@@ -0,0 +1,158 @@
"""Roadmap #28 Part 1 — auto-link RFC references in submitted prose.
Scans plain-text PR descriptions and comment bodies for references to
existing **accepted** (state='active') RFCs and returns a structured list
of *segments* the frontend renders: plain-text runs interleaved with
``{"type": "rfc", ...}`` link segments. The backend never emits HTML
the frontend maps link segments onto React anchors so the surface is
XSS-safe by construction and independent of any HTML-sanitization layer.
**Read-time enrichment, not submit-time persistence.** The roadmap row
phrases the scan as happening "at submit/post time"; this module instead
enriches on read. The intent the roadmap actually names "not as live
compose preview" — is honored (drafts are never scanned, only submitted
content on the read paths). Read-time was chosen for three reasons:
1. Correctness links track the *live* active-RFC set. A newly-accepted
RFC starts linking in older comments; a withdrawn RFC stops linking
everywhere. Submit-time freezing would drift stale.
2. Zero migration no derived data to store. (A concurrent session
already holds migration 023; staying migration-free keeps this slice
conflict-free as well as simpler.)
3. Cost the active-RFC corpus is small and cache-resident, so building
the term index and scanning a 20k-char body per read is cheap.
**Matching is conservative by design.** Only references that are unlikely
to be coincidental link:
* ``rfc_id`` tokens (e.g. ``RFC-0001``) inherently specific.
* Multi-word titles (containing whitespace, e.g. ``Open Human Model``).
* Hyphenated slugs (containing ``-``, e.g. ``open-human-model``).
Single common-word titles or slugs (e.g. a hypothetical RFC titled
"Human") are deliberately NOT auto-linked they would turn every prose
"human" into a link. Surfacing those is the job of the roadmap's
"curated canonical-terms list", an explicit per-deployment opt-in left as
a future extension rather than guessed at here.
"""
from __future__ import annotations
from typing import Any, Iterable
def _is_word_char(c: str) -> bool:
"""Word-boundary test. Hyphen and underscore count as word chars so a
match can't begin or end in the middle of a kebab/snake token."""
return c.isalnum() or c in ("-", "_")
def segment_text(text: str | None, terms: list[tuple[str, str, str]]) -> list[dict[str, Any]]:
"""Split ``text`` into text / rfc-link segments against ``terms``.
``terms`` is a list of ``(key_lower, slug, title)`` tuples; callers
pass it pre-sorted longest-first so the longest match wins at any
position (so "Open Human Model" wins over a bare "Open"). Matching is
case-insensitive and respects word boundaries on both ends. The
returned ``label`` preserves the source casing.
Always returns at least one segment; for empty/None input that is a
single empty text segment, so callers can render uniformly.
"""
if not text:
return [{"type": "text", "text": text or ""}]
out: list[dict[str, Any]] = []
buf: list[str] = []
low = text.lower()
n = len(text)
i = 0
while i < n:
match: tuple[str, str, str, int] | None = None
for key, slug, title in terms:
klen = len(key)
if klen == 0 or not low.startswith(key, i):
continue
before = text[i - 1] if i > 0 else ""
after = text[i + klen] if i + klen < n else ""
if _is_word_char(before) or _is_word_char(after):
continue
match = (key, slug, title, klen)
break
if match is not None:
_key, slug, title, klen = match
if buf:
out.append({"type": "text", "text": "".join(buf)})
buf = []
out.append({
"type": "rfc",
"slug": slug,
"label": text[i:i + klen],
"title": title,
})
i += klen
else:
buf.append(text[i])
i += 1
if buf:
out.append({"type": "text", "text": "".join(buf)})
return out
def _keys_for(slug: str, title: str, rfc_id: str | None) -> Iterable[str]:
"""The match keys an active RFC contributes. See the module docstring
for why each gate exists (conservative, false-positive-averse)."""
if rfc_id:
rid = rfc_id.strip()
if len(rid) >= 2:
yield rid.lower()
if title:
t = title.strip()
# Multi-word titles only — a single common word is too noisy.
if len(t) >= 2 and (" " in t or "\t" in t):
yield t.lower()
if slug:
s = slug.strip()
# Hyphenated slugs only — a single-token slug is a bare word.
if len(s) >= 2 and "-" in s:
yield s.lower()
class LinkIndex:
"""A reusable term index built once per request and applied to many
bodies (a PR's description plus every comment on it)."""
def __init__(self, terms: list[tuple[str, str, str]]):
# Longest key first so the longest reference wins at each position.
self._terms = sorted(terms, key=lambda t: len(t[0]), reverse=True)
def __bool__(self) -> bool:
return bool(self._terms)
def segment(self, text: str | None) -> list[dict[str, Any]]:
return segment_text(text, self._terms)
def build_index(conn, *, exclude_slug: str | None = None) -> LinkIndex:
"""Build a :class:`LinkIndex` from the accepted (active) RFC corpus.
``exclude_slug`` drops the RFC the surrounding surface is itself scoped
to, so an RFC's own title/id/slug don't self-link inside its own PR or
discussion. ``ORDER BY slug`` makes key de-duplication deterministic
when two RFCs would contribute the same key (first slug wins)."""
rows = conn.execute(
"SELECT slug, title, rfc_id FROM cached_rfcs WHERE state = 'active' ORDER BY slug"
).fetchall()
terms: list[tuple[str, str, str]] = []
seen: set[str] = set()
for r in rows:
slug = r["slug"]
if exclude_slug is not None and slug == exclude_slug:
continue
title = r["title"] or ""
rfc_id = r["rfc_id"] if "rfc_id" in r.keys() else None
for key in _keys_for(slug, title, rfc_id):
if key in seen:
continue
seen.add(key)
terms.append((key, slug, title))
return LinkIndex(terms)
+244
View File
@@ -0,0 +1,244 @@
"""Roadmap #27 — Claude Haiku tag suggestions for the propose-RFC modal.
A cheap, fast assist: given the partial RFC draft a user is typing, ask
Claude Haiku to recommend tags drawn ONLY from the tags already in use
across the corpus. v1 has no curated tag list tags are free-form chip
input (§9.1) so "the taxonomy" is the de-facto set of distinct tags
the existing RFCs already carry. The model is constrained to that set
and MUST NOT invent new tags; taxonomy extension (letting the model
propose genuinely new tags) is a deferred follow-up per the roadmap.
Why Haiku specifically: cost. Picking a few reasonable tags from a known
set is well within Haiku's range, and the modal fires this repeatedly as
the user types, so the per-call price has to stay small.
The whole surface degrades to silence rather than error: no Anthropic
key, no provider, an empty corpus, a rate-limited caller, an empty
draft, or an unparseable model reply all yield an empty suggestion list.
The propose modal hides its suggestion row on an empty list, so the
fallback is simply the existing free-form chip input with nothing extra
shown.
"""
from __future__ import annotations
import json
import logging
import os
import time
from dataclasses import dataclass
from . import db
from .providers import BaseProvider, construct_haiku
log = logging.getLogger(__name__)
# Bound the universe handed to the model so a large corpus can't blow up
# the prompt size (and the cost). The most-common tags matter most.
_UNIVERSE_CAP = 200
# How many suggestions we ever return to the modal.
DEFAULT_MAX_SUGGESTIONS = 6
@dataclass
class Draft:
"""The partial propose-RFC draft. `pitch` is the "why is this needed"
rationale; `use_case` is the #26 optional ground-truth field."""
title: str = ""
pitch: str = ""
use_case: str = ""
def is_empty(self) -> bool:
return not (self.title.strip() or self.pitch.strip() or self.use_case.strip())
def haiku_provider(config) -> BaseProvider | None:
"""Construct a dedicated Claude Haiku provider from the operator's
Anthropic key, or return None when no key is configured.
None means "suggestions unavailable" the caller returns an empty
list and the modal shows nothing. This is the seam tests monkeypatch
to inject a fake provider without a real key. There is no RFC slug at
propose time, so the §6.7 per-RFC funder path deliberately does not
apply: tag suggestion always runs on the operator's own key.
"""
key = getattr(config, "anthropic_api_key", "") or ""
if not key:
return None
return construct_haiku(key)
def gather_tag_universe(cap: int = _UNIVERSE_CAP) -> list[str]:
"""Every distinct tag in use across the cached corpus, most-common
first (ties broken alphabetically for determinism), capped.
This is the universe the model is constrained to. An empty corpus
yields an empty list, which short-circuits suggestion to silence.
"""
rows = db.conn().execute("SELECT tags_json FROM cached_rfcs").fetchall()
counts: dict[str, int] = {}
for r in rows:
try:
tags = json.loads(r["tags_json"] or "[]")
except (ValueError, TypeError):
continue
if not isinstance(tags, list):
continue
for t in tags:
if not isinstance(t, str):
continue
tag = t.strip()
if tag:
counts[tag] = counts.get(tag, 0) + 1
ranked = sorted(counts, key=lambda t: (-counts[t], t.lower()))
return ranked[:cap]
# ---------------------------------------------------------------------------
# Rate limiting — in-process, per-user sliding window.
#
# Cost control, not security: the `require_contributor` gate already
# bounds callers to authenticated beta users, and the modal debounces.
# This is the backstop against a stuck/abusive client hammering the
# endpoint. In-memory is sufficient (single-process uvicorn on the VM)
# and resets on restart, which is fine for a cost guard.
# ---------------------------------------------------------------------------
_CALLS: dict[int, list[float]] = {}
def _rate_config() -> tuple[int, float]:
try:
max_calls = int(os.environ.get("TAG_SUGGEST_RATE_MAX", "30"))
except ValueError:
max_calls = 30
try:
window = float(os.environ.get("TAG_SUGGEST_RATE_WINDOW_SECONDS", "60"))
except ValueError:
window = 60.0
return max_calls, window
def rate_limit_ok(user_id: int, *, now: float | None = None) -> bool:
"""True if this call is within the per-user window; records the call.
`now` is injectable for tests (defaults to a monotonic clock)."""
max_calls, window = _rate_config()
t = time.monotonic() if now is None else now
calls = _CALLS.setdefault(user_id, [])
cutoff = t - window
calls[:] = [c for c in calls if c > cutoff]
if len(calls) >= max_calls:
return False
calls.append(t)
return True
def reset_rate_limits() -> None:
"""Test seam — clear the in-process window state."""
_CALLS.clear()
# ---------------------------------------------------------------------------
# Prompt + parse.
# ---------------------------------------------------------------------------
_SYSTEM = (
"You suggest topic tags for a draft RFC — a structured proposal "
"document in a collection. You are given the draft's title, its "
"rationale, an optional use case, and the exact set of tags already "
"in use across the collection. Choose the tags from that set that "
"best fit the draft.\n"
"Rules:\n"
"- Choose ONLY from the provided tag set. Never invent a tag.\n"
"- Order best-fit first. Omit weak fits rather than padding the list.\n"
"- Return at most {max} tags.\n"
"Respond with ONLY a JSON array, no prose, of the form:\n"
'[{{"tag": "<exact tag from the set>", "confidence": <number between 0 and 1>}}]\n'
"If no tag in the set fits the draft, return []."
)
def _build_messages(draft: Draft, universe: list[str], max_suggestions: int):
system = _SYSTEM.format(max=max_suggestions)
parts: list[str] = []
if draft.title.strip():
parts.append(f"Title: {draft.title.strip()[:300]}")
if draft.pitch.strip():
parts.append(f"Why this RFC is needed:\n{draft.pitch.strip()[:4000]}")
if draft.use_case.strip():
parts.append(f"What it will be used for:\n{draft.use_case.strip()[:4000]}")
parts.append(
"Tags already in use (choose only from these):\n" + ", ".join(universe)
)
return system, [{"role": "user", "content": "\n\n".join(parts)}]
def _extract_json_array(text: str):
start = text.find("[")
end = text.rfind("]")
if start == -1 or end == -1 or end < start:
return None
try:
return json.loads(text[start : end + 1])
except ValueError:
return None
def parse_reply(text: str, universe: list[str], max_suggestions: int) -> list[dict]:
"""Parse the model reply into a clean, universe-constrained list.
Tolerant of the model returning bare strings or objects, extra prose
around the JSON, unknown/invented tags (dropped), duplicate tags
(deduped), and missing/garbage confidences (defaulted + clamped).
"""
data = _extract_json_array(text or "")
if not isinstance(data, list):
return []
# Map back to the canonical spelling in the universe, case-insensitively,
# so a model that lowercases a tag still resolves to the real one.
canonical = {t.lower(): t for t in universe}
out: list[dict] = []
seen: set[str] = set()
for item in data:
if isinstance(item, dict):
raw = item.get("tag")
conf = item.get("confidence")
elif isinstance(item, str):
raw, conf = item, None
else:
continue
if not isinstance(raw, str):
continue
tag = canonical.get(raw.strip().lower())
if tag is None or tag in seen:
continue
try:
c = float(conf) if conf is not None else 0.5
except (ValueError, TypeError):
c = 0.5
c = max(0.0, min(1.0, c))
out.append({"tag": tag, "confidence": round(c, 3)})
seen.add(tag)
if len(out) >= max_suggestions:
break
return out
def suggest(
provider: BaseProvider,
draft: Draft,
universe: list[str],
max_suggestions: int = DEFAULT_MAX_SUGGESTIONS,
) -> list[dict]:
"""Orchestrate one suggestion call. Returns [] for an empty draft or
empty universe (no model call), and for any provider/parse failure."""
if draft.is_empty() or not universe:
return []
system, history = _build_messages(draft, universe, max_suggestions)
try:
text = provider.send(system, history)
except Exception as exc: # provider/network failure → silent empty
log.warning("tag-suggest provider failed: %s", exc)
return []
return parse_reply(text, universe, max_suggestions)
+23 -5
View File
@@ -73,6 +73,19 @@ def _siteverify_url() -> str:
return os.environ.get("TURNSTILE_SITEVERIFY_URL", "").strip() or SITEVERIFY_URL
async def _siteverify_post(url: str, data: dict) -> httpx.Response:
"""Perform the siteverify POST on an `httpx.AsyncClient`.
Isolated as a narrow seam (I4, security-audit-0026): the call is
awaited so a slow CloudFlare response can't block the event loop,
and tests patch *this function* rather than the shared
`httpx.AsyncClient` (which other modules gitea, docs also
construct, so a global patch would break app boot).
"""
async with httpx.AsyncClient(timeout=10.0) as client:
return await client.post(url, data=data)
@dataclass
class VerifyOutcome:
"""Result of a Turnstile siteverify call.
@@ -98,7 +111,7 @@ class VerifyOutcome:
reason: str
def verify_token(token: str | None, *, client_ip: str | None = None) -> VerifyOutcome:
async def verify_token(token: str | None, *, client_ip: str | None = None) -> VerifyOutcome:
"""Validate a Turnstile token against CloudFlare's siteverify endpoint.
Returns a VerifyOutcome describing whether the calling endpoint
@@ -108,9 +121,14 @@ def verify_token(token: str | None, *, client_ip: str | None = None) -> VerifyOu
* 'misconfigured' 500 "auth misconfigured"
* 'missing-token' / 'failed' / 'network' 400 "verification failed"
Tests monkeypatch `httpx.post` (or set `TURNSTILE_SITEVERIFY_URL`
+ a MockTransport client) to avoid touching the real CloudFlare
endpoint. No real keys are ever embedded in tests.
Async (I4, security-audit-0026): the siteverify call is awaited on an
`httpx.AsyncClient` so a slow CloudFlare response can't block the
event loop (the prior synchronous `httpx.post` stalled the single
worker for up to the 10s timeout). Callers must `await` it.
Tests monkeypatch `_siteverify_post` (the narrow async seam) to avoid
touching the real CloudFlare endpoint and to keep the patch off the
shared `httpx.AsyncClient`. No real keys are ever embedded in tests.
"""
secret = _secret()
required = _required()
@@ -133,7 +151,7 @@ def verify_token(token: str | None, *, client_ip: str | None = None) -> VerifyOu
data["remoteip"] = client_ip
try:
response = httpx.post(_siteverify_url(), data=data, timeout=10.0)
response = await _siteverify_post(_siteverify_url(), data)
payload = response.json()
except Exception as exc: # network, JSON parse, etc.
log.warning("Turnstile siteverify call failed: %s", exc)
+33 -1
View File
@@ -12,6 +12,7 @@ import hashlib
import hmac
import json
import logging
import os
from fastapi import APIRouter, Header, HTTPException, Request
@@ -40,7 +41,27 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
x_gitea_signature: str = Header(default=""),
):
body = await request.body()
if config.webhook_secret:
# v0.18.0: defense in depth. config.py refuses to start
# when the secret is empty unless `RFC_APP_INSECURE_WEBHOOKS=1`
# is set; this branch catches the dev-bypass case (the only
# path where `config.webhook_secret` can be empty) and surfaces
# it loudly to the client. A POST that lands here with an
# empty secret on a production deployment indicates a
# mis-configuration (somebody flipped the bypass in prod),
# and the loud 500 is the proposal's whole point.
insecure = os.environ.get("RFC_APP_INSECURE_WEBHOOKS", "").strip() == "1"
if not config.webhook_secret:
if not insecure:
log.error(
"webhook receiver misconfigured: GITEA_WEBHOOK_SECRET is empty "
"and RFC_APP_INSECURE_WEBHOOKS=1 is not set"
)
raise HTTPException(status_code=500, detail="Webhook receiver misconfigured")
log.warning(
"webhook receiver running with RFC_APP_INSECURE_WEBHOOKS=1 — "
"signature verification is DISABLED. Production deployments MUST NOT set this."
)
else:
if not _verify_signature(body, x_gitea_signature, config.webhook_secret):
raise HTTPException(status_code=401, detail="Invalid signature")
@@ -68,6 +89,17 @@ def make_router(config: Config, gitea: Gitea) -> APIRouter:
slug = _slug_for_repo(repo_full)
if slug:
await cache.refresh_rfc_repo(config, gitea, slug)
else:
# v0.18.0: the proposal's "unknown-repo logging"
# gesture — a hook on a fork or a stale repo binding
# used to silently 200-OK here, hiding the
# misconfiguration. Now the operator sees it in
# the log.
log.info(
"webhook received for unknown repo: repo_full=%s event=%s "
"(no cached_rfcs row matched; hook may be on a fork or stale)",
repo_full, event,
)
except Exception:
log.exception("webhook refresh failed")
raise HTTPException(status_code=500, detail="Refresh failed")
@@ -0,0 +1,30 @@
-- v0.18.0 Slice 4: outbound_emails audit table.
--
-- Per the v0.18.0 email + webhook hygiene proposal §3, every send
-- helper writes a row to this table before returning, regardless
-- of outcome. status='sent' on success, 'failed' on exception,
-- 'deferred' on the dev-fallback path (no SMTP_HOST configured).
--
-- The table is queried by `GET /api/admin/outbound-emails` to
-- answer "did this person ever get their invite?" without having
-- to grep VM logs, and by the v0.18.0 Slice 5 bounce-correlation
-- hook (which looks up message_id when a POST lands at
-- /api/webhooks/email-bounce and marks the matching row
-- status='bounced').
CREATE TABLE IF NOT EXISTS outbound_emails (
id INTEGER PRIMARY KEY,
to_address TEXT NOT NULL,
from_address TEXT NOT NULL,
subject TEXT NOT NULL,
kind TEXT NOT NULL, -- 'otc' | 'invite' | 'notification' | 'bundle' | 'digest' | 'rfc-invite'
sent_at TEXT NOT NULL, -- ISO 8601, time the send was attempted
status TEXT NOT NULL, -- 'sent' | 'failed' | 'deferred' | 'bounced'
error TEXT, -- exception class + message if status='failed'
notification_id INTEGER, -- nullable FK to notifications.id for the watcher path
message_id TEXT -- the Message-ID header value, for bounce correlation
);
CREATE INDEX IF NOT EXISTS idx_outbound_emails_to ON outbound_emails(to_address);
CREATE INDEX IF NOT EXISTS idx_outbound_emails_sent_at ON outbound_emails(sent_at);
CREATE INDEX IF NOT EXISTS idx_outbound_emails_message ON outbound_emails(message_id);
@@ -0,0 +1,47 @@
-- Roadmap #26 (rfc-app v0.22.0): the optional "What will you be using
-- this for?" capture on the two propose surfaces.
--
-- The roadmap's framing names "the rfcs table" and "the PR-metadata
-- table" for a `proposed_use_case TEXT NULL` column. In this deployment
-- those two surfaces are the cache tables `cached_rfcs` and `cached_prs`
-- (002_cache.sql). We add the nullable column to each, matching the
-- existing naming convention (no NOT NULL, no default — NULL is the
-- "left blank" sentinel the view surfaces render tastefully).
--
-- BUT: those tables are *cache*, rebuilt from Gitea by the §4.1
-- reconciler (cache.py). The reconciler's INSERT...ON CONFLICT DO UPDATE
-- sets only the columns it knows about, so an unlisted column is
-- preserved on the update path — yet a propose/open never *writes* the
-- column through the cache (the write path is endpoint -> Gitea ->
-- reconcile, and the reconciler does not carry this field). So the cache
-- column alone would always read NULL.
--
-- The durable home is therefore a dedicated app-truth table the propose
-- /open endpoints write directly (keyed by the PR number, which is the
-- stable identity for both idea PRs and rfc_branch PRs) and the view
-- endpoints read back. This is not cache — it is canonical and survives
-- any reconcile. The cache columns are added too for parity with the
-- roadmap's literal shape and for any future reconciler that learns to
-- carry the field, but the side table is the source of truth read at
-- view time.
ALTER TABLE cached_rfcs ADD COLUMN proposed_use_case TEXT;
ALTER TABLE cached_prs ADD COLUMN proposed_use_case TEXT;
-- Canonical, reconcile-proof store. One row per propose/open that
-- supplied a use case. `scope` distinguishes the propose-RFC surface
-- ('rfc') from the propose-PR-against-an-RFC surface ('pr'); `pr_number`
-- is the join key the endpoints already have in hand. NULL/omitted use
-- cases simply never write a row here, so absence == "left blank".
CREATE TABLE proposed_use_cases (
id INTEGER PRIMARY KEY AUTOINCREMENT,
scope TEXT NOT NULL CHECK (scope IN ('rfc', 'pr')),
rfc_slug TEXT NOT NULL,
pr_number INTEGER NOT NULL,
use_case TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (scope, pr_number)
);
CREATE INDEX idx_proposed_use_cases_lookup ON proposed_use_cases (scope, pr_number);
CREATE INDEX idx_proposed_use_cases_slug ON proposed_use_cases (scope, rfc_slug);
@@ -0,0 +1,54 @@
-- v0.23.0 / roadmap item #29: server-side sign-in state resume.
--
-- Track each authenticated user's last-viewed route + a small bag of
-- "light" component state so that the *next* sign-in can land the user
-- back where they left off, rather than always dropping them on the
-- empty-state home view.
--
-- Storage shape (one row per user — per-user, NOT per-device, per the
-- #29 "safe default"):
--
-- * `user_id` — PRIMARY KEY and FK into users(id) with cascade on
-- delete. INTEGER to match users.id (INTEGER PRIMARY KEY
-- AUTOINCREMENT). A deleted user automatically loses their stored
-- resume state. One row per user means a later sign-in on any
-- device resumes the most-recently-recorded route — the per-user
-- model the roadmap asks for.
--
-- * `last_route` — the frontend pathname the user was last on
-- (e.g. "/rfc/open-human-model"). TEXT, nullable until the first
-- route-change POST lands. NEVER contains draft-buffer contents —
-- it is a route only. See SPEC §6.2 "Sign-in state resume
-- (privacy)".
--
-- * `last_route_state` — a JSON-encoded bag of *light* component
-- state (scroll anchors, open-tab selection, filter chips, etc.).
-- SQLite has no native JSONB; we store JSON as TEXT exactly as the
-- rest of the app stores its JSON blobs (json.dumps / json.loads,
-- cf. permission_events.details, actions.details). Nullable.
-- PRIVACY INVARIANT: this column MUST NOT carry draft-buffer text,
-- PR bodies, comment drafts, or any user-typed content — only
-- ephemeral view state safe to replay. The PUT handler is the
-- enforcement point; the column comment is the contract.
--
-- * `resume_enabled` — the per-user opt-out flag. 1 (default) means
-- "resume me where I left off"; 0 means "always land on home". The
-- PUT handler no-ops the upsert when this is 0, and the read path
-- refuses to hand back a stored route when this is 0. A
-- profile-settings toggle UI to flip this is a follow-up (the
-- column + default-on behavior ship now); see CHANGELOG v0.23.0.
--
-- * `last_updated_at` — TEXT timestamp, app convention
-- `datetime('now')`, matching device_trust.last_seen_at /
-- users.last_seen_at. Refreshed on every successful upsert.
--
-- No new env vars. The debounce interval for the frontend route-change
-- POST is a frontend constant (~1s), not a server knob.
CREATE TABLE user_session_state (
user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
last_route TEXT,
last_route_state TEXT, -- JSON-encoded light state, nullable
resume_enabled INTEGER NOT NULL DEFAULT 1,
last_updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
@@ -0,0 +1,18 @@
-- v0.25.0 / security audit 0026, finding H1.
--
-- The OTC verify path had no attempt-limit or lockout, unlike the
-- passcode path (015_passcode.sql gave users.passcode_failed_attempts +
-- passcode_locked_until). This table gives the OTC verify endpoint the
-- same per-identity lockout shape. It is keyed by email rather than
-- user_id because an OTC sign-in may not have a users row yet — the row
-- is provisioned only on a *successful* verify, so the lockout state has
-- to survive independently of it.
--
-- The per-IP rate limiter (app/ratelimit.py) is the primary brute-force
-- defense; this table is the parity layer that mirrors the passcode
-- lockout and persists across restarts.
CREATE TABLE IF NOT EXISTS otc_verify_state (
email TEXT PRIMARY KEY,
failed_attempts INTEGER NOT NULL DEFAULT 0,
locked_until TEXT
);
+19
View File
@@ -0,0 +1,19 @@
"""Shared pytest fixtures for the backend suite.
Added in v0.27.0 (security audit 0026) alongside the new per-IP rate
limiter. The limiters in `app.ratelimit` are process-global singletons,
so their state survives across tests within a run; without a reset, the
accumulated requests from earlier tests exhaust the budget and later
tests see spurious 429s. This autouse fixture gives every test a clean
limiter window.
"""
import pytest
from app import ratelimit
@pytest.fixture(autouse=True)
def _reset_rate_limiters():
ratelimit._reset_all_for_tests()
yield
ratelimit._reset_all_for_tests()
@@ -647,3 +647,82 @@ def test_pending_invites_listing_admin_only(app_with_fake_gitea):
)
r = client.get("/api/admin/users/invites")
assert r.status_code == 403
# ---------------------------------------------------------------------------
# v0.18.0: invite-envelope header shape — Slice 2
#
# Invite mail goes through `build_envelope` and MUST land Date,
# Message-ID, Auto-Submitted, AND a `List-Unsubscribe: <mailto:…>`
# (no URL — the invitee isn't a user yet, so no per-user opt-out
# row exists). The mailto: target is the operator's `EMAIL_FROM`
# by default; the operator can override via `EMAIL_UNSUBSCRIBE_MAILTO`.
# ---------------------------------------------------------------------------
def _provision_admin_and_send_invite(client, app_with_fake_gitea_fixture, *, to: str = "headers@ex.co"):
provision_user_row(user_id=400, login="adminH", role="admin")
sign_in_as(
client, user_id=400, gitea_login="adminH",
display_name="Admin H", role="admin",
email="adminh@test",
)
_reset_outbound()
r = client.post(
"/api/admin/users",
json={
"email": to,
"first_name": "Header",
"last_name": "Test",
"role": "contributor",
"custom_message": "",
},
)
assert r.status_code == 200, r.text
return _outbound_invite_envelopes(to)[-1]
def test_invite_envelope_sets_date_messageid_autosubmitted(app_with_fake_gitea):
from fastapi.testclient import TestClient
from email.utils import parsedate_to_datetime
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
env = _provision_admin_and_send_invite(client, (app, _fake))
msg = env["message"]
assert parsedate_to_datetime(msg["Date"]) is not None
assert msg["Message-ID"].startswith("<") and msg["Message-ID"].endswith(">")
assert msg["Auto-Submitted"] == "auto-generated"
def test_invite_envelope_has_mailto_list_unsubscribe_only(app_with_fake_gitea):
"""The invitee isn't a user yet — no per-user opt-out URL is
available. The `List-Unsubscribe` MUST be a mailto: form, and
the `List-Unsubscribe-Post` header MUST be absent (the
one-click semantic requires a URL the MUA can POST to)."""
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
env = _provision_admin_and_send_invite(client, (app, _fake))
msg = env["message"]
lu = msg["List-Unsubscribe"]
assert lu is not None and lu.startswith("<mailto:")
# No URL part — invite is mailto-only.
assert "https://" not in lu and "http://" not in lu
assert msg["List-Unsubscribe-Post"] is None
def test_invite_envelope_respects_email_unsubscribe_mailto_override(app_with_fake_gitea, monkeypatch):
"""When `EMAIL_UNSUBSCRIBE_MAILTO` is set, the mailto: target on
`List-Unsubscribe` honors it (lets a deployment route opt-outs
to a humans-monitored mailbox distinct from the no-reply
sender)."""
from fastapi.testclient import TestClient
monkeypatch.setenv("EMAIL_UNSUBSCRIBE_MAILTO", "ohm@wiggleverse.org?subject=remove")
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
env = _provision_admin_and_send_invite(client, (app, _fake))
msg = env["message"]
assert "ohm@wiggleverse.org?subject=remove" in msg["List-Unsubscribe"]
@@ -0,0 +1,379 @@
"""v0.19.0 / roadmap item #30 — `/api/docs/sessions/*` endpoints.
The framework mediates reads against the public
`wiggleverse/ohm-session-history` gitea repo so the rendered
`/docs/sessions/*` surface inherits the same chrome as `/docs/user-guide`.
This test suite covers the four endpoints + the in-process TTL cache,
mocking the upstream HTTP via `httpx.MockTransport` (the same shape the
rest of the test suite uses for Gitea).
The tests do NOT spin up the full FakeGitea they only need to mock
the gitea raw URL surface (and the contents API for the session-index
endpoint). Each test owns its mock transport so we can dial in 200 /
404 / 5xx / timeout responses per case.
Path-validation tests intentionally bypass the network a malformed
`nnnn` or `filename` MUST be rejected at the route layer before any
upstream call is made.
"""
from __future__ import annotations
import json
import httpx
import pytest
from fastapi.testclient import TestClient
from app import docs_sessions
# Reuse the proven app-construction fixtures from the proposal vertical
# (same shape every test file in this repo uses).
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea,
tmp_env,
)
# ---------------------------------------------------------------------------
# Test scaffolding
# ---------------------------------------------------------------------------
class _UpstreamHandler:
"""Records every URL the docs_sessions module fetched and returns
canned responses keyed by URL substring. Allows the test to assert
on call count (for cache verification) without needing a full Gitea
simulator.
`calls` tracks only URLs that hit the session-history host (the
`OHM_SESSION_HISTORY_*` bases) so reconciler/Gitea-side calls which
also pass through this handler because `httpx.AsyncClient` is a
shared attribute the gitea-side fixture also monkeypatches don't
inflate the count we use for cache-hit assertions.
"""
_SESSION_HOST_MARKERS = ("ohm-session-history", "wiggleverse/ohm-session-history")
def __init__(self, responses: dict[str, tuple[int, str]]):
self.responses = responses
self.calls: list[str] = []
def __call__(self, request: httpx.Request) -> httpx.Response:
url = str(request.url)
if any(m in url for m in self._SESSION_HOST_MARKERS):
self.calls.append(url)
for key, (status, body) in self.responses.items():
if key in url:
return httpx.Response(status, text=body)
# Default: 404. Lets tests skip declaring "the rest is 404".
return httpx.Response(404, text="not found")
@pytest.fixture
def patched_httpx(monkeypatch):
"""Provide a hook the test can call to install a MockTransport.
Returns a closure: `install(handler)` patches
`app.docs_sessions.httpx.AsyncClient` so every constructed client
uses the handler's transport.
NB: the upstream `app_with_fake_gitea` fixture also patches
`httpx.AsyncClient` (to route gitea calls to a FakeGitea handler),
and because `httpx` is a single shared module, that patch mutates
the *same* `AsyncClient` attribute we're about to overwrite. We
therefore import the unpatched class directly from the
`httpx._client` module so our install path can construct a fresh
real client around our MockTransport without going through the
FakeGitea wrapper.
"""
from httpx._client import AsyncClient as RealAsyncClient
def install(handler):
def patched(*args, **kwargs):
kwargs["transport"] = httpx.MockTransport(handler)
return RealAsyncClient(*args, **kwargs)
monkeypatch.setattr("app.docs_sessions.httpx.AsyncClient", patched)
return handler
yield install
@pytest.fixture
def app(app_with_fake_gitea):
"""Wrap the shared app fixture, resetting the docs-sessions cache so
cross-test state can't leak. Returns just the FastAPI app — the
fake-Gitea handle is irrelevant for the docs-sessions surface.
"""
docs_sessions.reset_cache()
fastapi_app, _fake = app_with_fake_gitea
return fastapi_app
# ---------------------------------------------------------------------------
# Manifest endpoint
# ---------------------------------------------------------------------------
def test_manifest_happy_path(app, patched_httpx):
manifest_body = json.dumps(
{
"0001": {"title": "Bootstrap"},
"0014": {"title": "Wave 7 driver"},
}
)
patched_httpx(
_UpstreamHandler({"sessions.json": (200, manifest_body)})
)
with TestClient(app) as client:
r = client.get("/api/docs/sessions/manifest")
assert r.status_code == 200, r.text
payload = r.json()
assert payload == {
"0001": {"title": "Bootstrap"},
"0014": {"title": "Wave 7 driver"},
}
def test_manifest_empty_state(app, patched_httpx):
"""A 404 from gitea means the manifest hasn't been published yet.
The endpoint returns HTTP 200 + `{}` so the frontend can render the
no-sessions-yet state without an error banner.
"""
patched_httpx(_UpstreamHandler({"sessions.json": (404, "not found")}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/manifest")
assert r.status_code == 200, r.text
assert r.json() == {}
def test_manifest_upstream_5xx_returns_502(app, patched_httpx):
patched_httpx(_UpstreamHandler({"sessions.json": (500, "internal")}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/manifest")
assert r.status_code == 502, r.text
body = r.json()
assert body["detail"]["error"] == "session-history fetch failed"
# ---------------------------------------------------------------------------
# About endpoint
# ---------------------------------------------------------------------------
def test_about_happy_path(app, patched_httpx):
readme = "# OHM session history\n\nWelcome.\n"
patched_httpx(_UpstreamHandler({"README.md": (200, readme)}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/about")
assert r.status_code == 200, r.text
assert "text/markdown" in r.headers["content-type"]
assert r.text == readme
def test_about_404(app, patched_httpx):
patched_httpx(_UpstreamHandler({"README.md": (404, "")}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/about")
assert r.status_code == 404, r.text
def test_about_upstream_5xx_returns_502(app, patched_httpx):
patched_httpx(_UpstreamHandler({"README.md": (503, "down")}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/about")
assert r.status_code == 502, r.text
# ---------------------------------------------------------------------------
# Transcript endpoint
# ---------------------------------------------------------------------------
def test_transcript_happy_path(app, patched_httpx):
body = "# Session 0017.1 — Transcript\n\nbody.\n"
fname = "SESSION-0017.1-TRANSCRIPT-2026-05-28T08-50--2026-05-28T11-20.md"
patched_httpx(_UpstreamHandler({fname: (200, body)}))
with TestClient(app) as client:
r = client.get(f"/api/docs/sessions/0017/{fname}")
assert r.status_code == 200, r.text
assert "text/markdown" in r.headers["content-type"]
assert r.text == body
def test_transcript_404(app, patched_httpx):
fname = "SESSION-9999.0-TRANSCRIPT-2026-01-01T00-00--2026-01-01T00-01.md"
patched_httpx(_UpstreamHandler({})) # everything 404s
with TestClient(app) as client:
r = client.get(f"/api/docs/sessions/9999/{fname}")
assert r.status_code == 404, r.text
def test_transcript_rejects_invalid_session_dir(app, patched_httpx):
"""`nnnn` must be exactly 4 digits. `abcd` fails before any
network call.
"""
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
r = client.get(
"/api/docs/sessions/abcd/"
"SESSION-0001.0-TRANSCRIPT-2026-01-01T00-00--2026-01-01T00-01.md"
)
assert r.status_code == 400, r.text
assert handler.calls == [], "rejected path must not hit the network"
def test_transcript_rejects_path_traversal(app, patched_httpx):
"""A filename that doesn't match the SESSION-NNNN.M-TRANSCRIPT regex
is rejected. `../etc/passwd` doesn't match; neither does the legacy
flat-root `SESSION-A-TRANSCRIPT.md`.
"""
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
# Path traversal — but FastAPI normalizes `..` in the path before
# routing, so this resolves to /api/docs/sessions/0001/etc/passwd
# which routes to the same handler with filename=etc/passwd, and
# gets rejected as an invalid transcript filename. Even if the
# normalization didn't apply (some intermediary), the regex
# check rejects anything not matching the SESSION- prefix.
r = client.get(
"/api/docs/sessions/0001/etc%2Fpasswd"
)
# 400 (filename validation) or 404 (path didn't match the
# route); both reject before any network call. Either is
# acceptable — what matters is that we never fetched it.
assert r.status_code in (400, 404), r.text
assert handler.calls == [], "rejected path must not hit the network"
def test_transcript_rejects_legacy_flat_filename(app, patched_httpx):
"""Legacy `SESSION-A-TRANSCRIPT.md` (letter form) doesn't match the
numeric regex by design, since post-#23 transcripts live in
`NNNN/` folders with numeric names. Reject 400.
"""
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
r = client.get("/api/docs/sessions/0001/SESSION-A-TRANSCRIPT.md")
assert r.status_code == 400, r.text
assert handler.calls == [], "rejected path must not hit the network"
def test_transcript_upstream_5xx_returns_502(app, patched_httpx):
fname = "SESSION-0001.0-TRANSCRIPT-2026-01-01T00-00--2026-01-01T00-01.md"
patched_httpx(_UpstreamHandler({fname: (502, "bad gateway")}))
with TestClient(app) as client:
r = client.get(f"/api/docs/sessions/0001/{fname}")
assert r.status_code == 502, r.text
# ---------------------------------------------------------------------------
# Session-index endpoint
# ---------------------------------------------------------------------------
def test_session_index_happy_path(app, patched_httpx):
"""The contents API returns a JSON list of file entries. The
endpoint filters to entries that match the transcript regex and
sorts them.
"""
# Two transcripts (driver + subagent) + a non-transcript sibling
# that must be filtered out.
listing = json.dumps(
[
{
"name": "SESSION-0017.0-TRANSCRIPT-"
"2026-05-28T08-30--2026-05-28T12-00.md",
"type": "file",
},
{
"name": "SESSION-0017.1-TRANSCRIPT-"
"2026-05-28T08-50--2026-05-28T11-20.md",
"type": "file",
},
{"name": "notes.md", "type": "file"}, # not a transcript
{"name": "attached-dir", "type": "dir"}, # not a file
]
)
patched_httpx(_UpstreamHandler({"/contents/0017": (200, listing)}))
with TestClient(app) as client:
r = client.get("/api/docs/sessions/0017/index")
assert r.status_code == 200, r.text
files = r.json()["files"]
assert files == [
"SESSION-0017.0-TRANSCRIPT-2026-05-28T08-30--2026-05-28T12-00.md",
"SESSION-0017.1-TRANSCRIPT-2026-05-28T08-50--2026-05-28T11-20.md",
]
def test_session_index_404(app, patched_httpx):
patched_httpx(_UpstreamHandler({})) # everything 404s
with TestClient(app) as client:
r = client.get("/api/docs/sessions/9999/index")
assert r.status_code == 404, r.text
def test_session_index_rejects_invalid_session_dir(app, patched_httpx):
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
r = client.get("/api/docs/sessions/abc/index")
assert r.status_code == 400, r.text
assert handler.calls == [], "rejected path must not hit the network"
# ---------------------------------------------------------------------------
# Cache behavior
# ---------------------------------------------------------------------------
def test_manifest_cache_hits_within_ttl(app, patched_httpx, monkeypatch):
"""Two consecutive manifest calls within the TTL window should
issue exactly one HTTP request to gitea.
"""
# Generous TTL so the test never races.
monkeypatch.setenv("OHM_DOCS_SESSIONS_MANIFEST_TTL_SEC", "60")
handler = _UpstreamHandler(
{"sessions.json": (200, json.dumps({"0001": {"title": "x"}}))}
)
patched_httpx(handler)
with TestClient(app) as client:
r1 = client.get("/api/docs/sessions/manifest")
r2 = client.get("/api/docs/sessions/manifest")
assert r1.status_code == 200
assert r2.status_code == 200
assert len(handler.calls) == 1, (
f"expected one upstream call, got {handler.calls}"
)
def test_transcript_cache_hits_within_ttl(app, patched_httpx, monkeypatch):
monkeypatch.setenv("OHM_DOCS_SESSIONS_CONTENT_TTL_SEC", "300")
fname = "SESSION-0001.0-TRANSCRIPT-2026-01-01T00-00--2026-01-01T00-01.md"
handler = _UpstreamHandler({fname: (200, "# body\n")})
patched_httpx(handler)
with TestClient(app) as client:
r1 = client.get(f"/api/docs/sessions/0001/{fname}")
r2 = client.get(f"/api/docs/sessions/0001/{fname}")
assert r1.status_code == 200
assert r2.status_code == 200
assert len(handler.calls) == 1
def test_transcript_404_is_cached(app, patched_httpx, monkeypatch):
"""Negative caching: a 404 result is cached at the content TTL so a
deployment with no published transcripts doesn't hammer gitea on
every navigation. Documented in `docs_sessions.fetch_transcript`.
"""
monkeypatch.setenv("OHM_DOCS_SESSIONS_CONTENT_TTL_SEC", "300")
fname = "SESSION-9999.0-TRANSCRIPT-2026-01-01T00-00--2026-01-01T00-01.md"
handler = _UpstreamHandler({}) # everything 404s
patched_httpx(handler)
with TestClient(app) as client:
r1 = client.get(f"/api/docs/sessions/9999/{fname}")
r2 = client.get(f"/api/docs/sessions/9999/{fname}")
assert r1.status_code == 404
assert r2.status_code == 404
assert len(handler.calls) == 1, "negative caching should suppress the 2nd call"
+469
View File
@@ -0,0 +1,469 @@
"""v0.20.0 — `/api/docs/specs/*` endpoints.
Sibling of `test_docs_sessions_vertical.py`. The framework mediates
reads of the configured framework-spec URLs (default: rfc-app's own
SPEC.md + flotilla's SPEC.md on `git.wiggleverse.org`) so the
`/docs/specs/*` surface inherits the same chrome as
`/docs/user-guide` and `/docs/sessions/*`.
This file covers:
- The manifest endpoint with the framework default
- The manifest endpoint with an overridden `OHM_DOCS_SPECS` JSON value
- Slug validation at the route layer (rejects `..`, `/`, `~`,
uppercase, whitespace, path traversal attempts)
- Gitea 200 / 404 / 5xx response mapping
- Negative caching (404 is cached, not re-fetched within TTL)
- Malformed `OHM_DOCS_SPECS` fallback to the default + a logged
warning (asserted by caplog)
- A manifest entry that fails per-entry validation (bad slug,
missing field) is dropped, with the rest of the list retained
Mocking approach: same as docs_sessions `httpx.MockTransport`
substituted into `app.docs_specs.httpx.AsyncClient` via a fixture.
"""
from __future__ import annotations
import json
import logging
import httpx
import pytest
from fastapi.testclient import TestClient
from app import docs_specs
# Reuse the proven app-construction fixtures from the proposal vertical
# (same shape every test file in this repo uses).
from test_propose_vertical import ( # noqa: F401
app_with_fake_gitea,
tmp_env,
)
# ---------------------------------------------------------------------------
# Test scaffolding
# ---------------------------------------------------------------------------
class _UpstreamHandler:
"""Records every URL the docs_specs module fetched and returns
canned responses keyed by URL substring. Lets the test assert on
call count (for cache verification) without booting a full upstream
simulator.
`calls` tracks only URLs that hit a host configured in the spec
manifest under test so unrelated httpx clients (gitea-side
fixtures, etc.) don't inflate the count we use for cache-hit
assertions. We marker-match on substrings the manifest carries.
"""
def __init__(
self,
responses: dict[str, tuple[int, str]],
host_markers: tuple[str, ...] = ("rfc-app", "flotilla", "specs.example"),
):
self.responses = responses
self.host_markers = host_markers
self.calls: list[str] = []
def __call__(self, request: httpx.Request) -> httpx.Response:
url = str(request.url)
if any(m in url for m in self.host_markers):
self.calls.append(url)
for key, (status, body) in self.responses.items():
if key in url:
return httpx.Response(status, text=body)
# Default: 404. Lets tests skip declaring "the rest is 404".
return httpx.Response(404, text="not found")
@pytest.fixture
def patched_httpx(monkeypatch):
"""Provide a hook the test can call to install a MockTransport.
Same shape as the docs_sessions fixture `app_with_fake_gitea`
monkeypatches `httpx.AsyncClient` for the gitea side, so we
construct from the unpatched class directly to avoid the
FakeGitea wrapper.
"""
from httpx._client import AsyncClient as RealAsyncClient
def install(handler):
def patched(*args, **kwargs):
kwargs["transport"] = httpx.MockTransport(handler)
return RealAsyncClient(*args, **kwargs)
monkeypatch.setattr("app.docs_specs.httpx.AsyncClient", patched)
return handler
yield install
@pytest.fixture
def app(app_with_fake_gitea):
"""Reset the docs-specs cache so cross-test state can't leak."""
docs_specs.reset_cache()
fastapi_app, _fake = app_with_fake_gitea
return fastapi_app
# ---------------------------------------------------------------------------
# Manifest endpoint
# ---------------------------------------------------------------------------
def test_manifest_default(app, monkeypatch):
"""With `OHM_DOCS_SPECS` unset, the manifest endpoint returns the
framework default (rfc-app + flotilla).
"""
monkeypatch.delenv("OHM_DOCS_SPECS", raising=False)
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
payload = r.json()
assert "specs" in payload
names = [s["name"] for s in payload["specs"]]
assert names == ["rfc-app", "flotilla"]
# The default URLs point at the OHM-canonical gitea raw paths.
assert all("git.wiggleverse.org" in s["url"] for s in payload["specs"])
def test_manifest_overridden(app, monkeypatch):
"""A deployment overriding `OHM_DOCS_SPECS` gets its custom list.
The manifest is parsed per-request from the env var (no startup
binding) so a runtime overlay change is visible without a
restart same shape as the docs_sessions env knobs.
"""
custom = json.dumps(
[
{
"name": "custom-spec",
"title": "Custom Spec",
"url": "https://specs.example.org/CUSTOM.md",
}
]
)
monkeypatch.setenv("OHM_DOCS_SPECS", custom)
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
payload = r.json()
assert payload == {
"specs": [
{
"name": "custom-spec",
"title": "Custom Spec",
"url": "https://specs.example.org/CUSTOM.md",
}
]
}
def test_manifest_malformed_json_falls_back(app, monkeypatch, caplog):
"""A non-JSON value in `OHM_DOCS_SPECS` logs a warning and the
endpoint falls back to the framework default. Startup is
unaffected the deployment continues to render the spec surface
rather than crashing on the typo.
"""
monkeypatch.setenv("OHM_DOCS_SPECS", "{not-json")
with caplog.at_level(logging.WARNING, logger="app.docs_specs"):
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
payload = r.json()
names = [s["name"] for s in payload["specs"]]
assert names == ["rfc-app", "flotilla"]
assert any(
"OHM_DOCS_SPECS is not valid JSON" in rec.message
for rec in caplog.records
), f"expected a logged warning; got {[r.message for r in caplog.records]}"
def test_manifest_non_list_falls_back(app, monkeypatch, caplog):
"""`OHM_DOCS_SPECS` must be a JSON array. A JSON object (or any
non-list value) falls back to the default + logs a warning.
"""
monkeypatch.setenv("OHM_DOCS_SPECS", json.dumps({"name": "not-a-list"}))
with caplog.at_level(logging.WARNING, logger="app.docs_specs"):
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
names = [s["name"] for s in r.json()["specs"]]
assert names == ["rfc-app", "flotilla"]
assert any(
"must be a JSON array" in rec.message for rec in caplog.records
)
def test_manifest_drops_invalid_entry_keeps_valid(app, monkeypatch, caplog):
"""Per-entry validation: an entry with a bad slug or missing field
is dropped; valid entries in the same list are retained.
"""
custom = json.dumps(
[
{"name": "Bad Slug", "title": "Bad", "url": "https://x"}, # uppercase + space
{"name": "..", "title": "Traversal", "url": "https://x"}, # path traversal
{"name": "missing-url", "title": "Missing URL"}, # no url
{
"name": "good-spec",
"title": "Good",
"url": "https://specs.example.org/GOOD.md",
},
]
)
monkeypatch.setenv("OHM_DOCS_SPECS", custom)
with caplog.at_level(logging.WARNING, logger="app.docs_specs"):
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
names = [s["name"] for s in r.json()["specs"]]
assert names == ["good-spec"]
# Three drop warnings (one per bad entry).
drops = [r for r in caplog.records if "failed validation" in r.message]
assert len(drops) == 3
def test_manifest_all_invalid_falls_back(app, monkeypatch, caplog):
"""If every entry is dropped, the framework default applies (the
surface never goes empty due to a bad overlay).
"""
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps([{"name": "BAD"}, {"name": "..", "title": "x", "url": "y"}]),
)
with caplog.at_level(logging.WARNING, logger="app.docs_specs"):
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
assert r.status_code == 200, r.text
names = [s["name"] for s in r.json()["specs"]]
assert names == ["rfc-app", "flotilla"]
assert any(
"yielded no valid entries" in rec.message for rec in caplog.records
)
def test_manifest_drops_duplicate_names(app, monkeypatch, caplog):
"""A duplicate `name` is dropped (the first occurrence wins). The
route layer's `/api/docs/specs/{name}` path lookup is by name, so
duplicates would otherwise be ambiguous.
"""
custom = json.dumps(
[
{"name": "x", "title": "First", "url": "https://specs.example/1"},
{"name": "x", "title": "Second", "url": "https://specs.example/2"},
]
)
monkeypatch.setenv("OHM_DOCS_SPECS", custom)
with caplog.at_level(logging.WARNING, logger="app.docs_specs"):
with TestClient(app) as client:
r = client.get("/api/docs/specs/manifest")
payload = r.json()
assert [s["title"] for s in payload["specs"]] == ["First"]
assert any("duplicate name" in rec.message for rec in caplog.records)
# ---------------------------------------------------------------------------
# Spec endpoint — happy + error paths
# ---------------------------------------------------------------------------
def test_spec_happy_path(app, patched_httpx, monkeypatch):
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "rfc-app",
"title": "rfc-app SPEC",
"url": "https://specs.example.org/rfc-app/SPEC.md",
}
]
),
)
body = "# rfc-app SPEC\n\nSection 1...\n"
patched_httpx(_UpstreamHandler({"rfc-app/SPEC.md": (200, body)}))
with TestClient(app) as client:
r = client.get("/api/docs/specs/rfc-app")
assert r.status_code == 200, r.text
assert "text/markdown" in r.headers["content-type"]
assert r.text == body
def test_spec_upstream_404(app, patched_httpx, monkeypatch):
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "missing-spec",
"title": "Missing",
"url": "https://specs.example.org/missing.md",
}
]
),
)
patched_httpx(_UpstreamHandler({})) # everything 404s
with TestClient(app) as client:
r = client.get("/api/docs/specs/missing-spec")
assert r.status_code == 404, r.text
def test_spec_upstream_5xx_returns_502(app, patched_httpx, monkeypatch):
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "broken-spec",
"title": "Broken",
"url": "https://specs.example.org/broken.md",
}
]
),
)
patched_httpx(_UpstreamHandler({"broken.md": (500, "internal")}))
with TestClient(app) as client:
r = client.get("/api/docs/specs/broken-spec")
assert r.status_code == 502, r.text
body = r.json()
assert body["detail"]["error"] == "specs fetch failed"
def test_spec_unknown_name_returns_404(app, patched_httpx, monkeypatch):
"""A name that doesn't appear in the manifest returns 404 without
touching the network. The handler treats "no such configured spec"
and "upstream 404" as the same outcome both render the same
"spec not found" empty state on the frontend.
"""
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "rfc-app",
"title": "rfc-app",
"url": "https://specs.example.org/x.md",
}
]
),
)
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
r = client.get("/api/docs/specs/does-not-exist")
assert r.status_code == 404, r.text
assert handler.calls == [], "unknown-name lookup must not hit the network"
# ---------------------------------------------------------------------------
# Spec endpoint — slug validation
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"raw_name",
[
"UPPER", # uppercase
"spaces here", # whitespace (post-decoding)
"with~tilde", # tilde
"with.dot", # dot
"with_under", # underscore (not allowed by [a-z0-9-]+)
],
)
def test_spec_rejects_invalid_name(app, patched_httpx, raw_name):
"""Names that don't match `^[a-z0-9-]+$` are rejected with 400 at
the route layer before any network or cache work.
Note: `..` is intentionally not in this list because the URL-
parsing layer collapses `/api/docs/specs/..` to `/api/docs/specs`
before the handler is reached the path-traversal protection is
therefore framework-level (httpx/urllib's path normalizer) rather
than route-layer. The slug-validation guard still rejects any
`..` that *would* reach the handler (e.g. via an env-configured
manifest entry); see `test_manifest_drops_invalid_entry_keeps_valid`
for that path.
"""
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
from urllib.parse import quote
r = client.get(f"/api/docs/specs/{quote(raw_name, safe='')}")
assert r.status_code == 400, r.text
assert handler.calls == [], "rejected name must not hit the network"
def test_spec_rejects_slash_in_name(app, patched_httpx):
"""A literal `/` in the path can't make it through the path
parameter FastAPI routes it as a separate segment. The check
here is that the request never reaches an upstream fetch.
"""
handler = _UpstreamHandler({})
patched_httpx(handler)
with TestClient(app) as client:
# `/api/docs/specs/sub/path` — the second segment makes this
# not match the `/{name}` route at all; FastAPI returns 404.
r = client.get("/api/docs/specs/sub/path")
assert r.status_code == 404, r.text
assert handler.calls == [], "non-matching path must not hit the network"
# ---------------------------------------------------------------------------
# Cache behavior
# ---------------------------------------------------------------------------
def test_spec_cache_hits_within_ttl(app, patched_httpx, monkeypatch):
monkeypatch.setenv("OHM_DOCS_SPECS_CONTENT_TTL_SEC", "300")
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "rfc-app",
"title": "rfc-app",
"url": "https://specs.example.org/rfc-app/SPEC.md",
}
]
),
)
handler = _UpstreamHandler({"rfc-app/SPEC.md": (200, "# body\n")})
patched_httpx(handler)
with TestClient(app) as client:
r1 = client.get("/api/docs/specs/rfc-app")
r2 = client.get("/api/docs/specs/rfc-app")
assert r1.status_code == 200
assert r2.status_code == 200
assert len(handler.calls) == 1, (
f"expected one upstream call, got {handler.calls}"
)
def test_spec_404_is_cached(app, patched_httpx, monkeypatch):
"""Negative caching: a 404 result is cached at the content TTL so
a deployment with a misconfigured spec URL doesn't hammer the
upstream on every navigation.
"""
monkeypatch.setenv("OHM_DOCS_SPECS_CONTENT_TTL_SEC", "300")
monkeypatch.setenv(
"OHM_DOCS_SPECS",
json.dumps(
[
{
"name": "missing-spec",
"title": "Missing",
"url": "https://specs.example.org/missing.md",
}
]
),
)
handler = _UpstreamHandler({}) # everything 404s
patched_httpx(handler)
with TestClient(app) as client:
r1 = client.get("/api/docs/specs/missing-spec")
r2 = client.get("/api/docs/specs/missing-spec")
assert r1.status_code == 404
assert r2.status_code == 404
assert len(handler.calls) == 1, "negative caching should suppress the 2nd call"
+5 -1
View File
@@ -231,13 +231,17 @@ def test_bounce_webhook_refuses_unsigned_when_secret_configured(app_with_fake_gi
# With the right header, the call passes the guard. (No matching
# user exists, so we get {matched: False} — that's the v1 contract.)
# v0.18.0 Slice 5: the response now includes `correlated_id`
# (the outbound_emails row id that matched the bounce's
# `message_id`, if one was supplied). The body didn't pass a
# message_id, so correlated_id is None.
r = client.post(
"/api/webhooks/email-bounce",
json={"email": "stranger@example.com", "kind": "hard"},
headers={"X-Webhook-Secret": "shhh"},
)
assert r.status_code == 200, r.text
assert r.json() == {"ok": True, "matched": False}
assert r.json() == {"ok": True, "matched": False, "correlated_id": None}
def test_bounce_webhook_open_when_secret_unset(app_with_fake_gitea):
+179
View File
@@ -0,0 +1,179 @@
"""Unit tests for `app.email_envelope.build_envelope` (v0.18.0 Slice 1).
These tests don't spin up the FastAPI app or touch the DB — they
exercise the helper directly. The integration tests in
test_otc_vertical / test_admin_create_user_invite_vertical /
test_notifications_vertical exercise the helper's *use* via the
shared `_SENT` buffer (the send path appends the envelope dict
before invoking the helper).
"""
from __future__ import annotations
from email.utils import parsedate_to_datetime
import pytest
from app.email_envelope import build_envelope
def _base_kwargs(**overrides):
base = dict(
to_address="recipient@example.com",
from_address="notifications@ohm.wiggleverse.org",
from_name="OHM",
subject="A test subject",
body_plain="Hello, world.\n",
)
base.update(overrides)
return base
# ---------------------------------------------------------------------------
# Always-present headers
# ---------------------------------------------------------------------------
def test_envelope_sets_from_to_subject():
msg = build_envelope(**_base_kwargs())
assert msg["To"] == "recipient@example.com"
assert msg["Subject"] == "A test subject"
# `From` is the display-form: "OHM <notifications@ohm.wiggleverse.org>".
assert "OHM" in msg["From"]
assert "<notifications@ohm.wiggleverse.org>" in msg["From"]
def test_envelope_sets_date_header_parseable():
msg = build_envelope(**_base_kwargs())
raw = msg["Date"]
assert raw, "Date header must be set"
# parsedate_to_datetime raises ValueError on malformed input.
dt = parsedate_to_datetime(raw)
assert dt is not None
def test_envelope_sets_message_id_with_from_domain_by_default():
msg = build_envelope(**_base_kwargs())
mid = msg["Message-ID"]
assert mid, "Message-ID must be set"
# Shape per RFC 5322 / make_msgid: <random@domain>
assert mid.startswith("<") and mid.endswith(">")
assert "@ohm.wiggleverse.org>" in mid
def test_envelope_message_id_domain_override():
msg = build_envelope(**_base_kwargs(msgid_domain="example.test"))
assert "@example.test>" in msg["Message-ID"]
def test_envelope_message_id_falls_back_to_localhost_if_from_has_no_at():
# Defensive: a malformed from_address shouldn't crash the helper.
msg = build_envelope(**_base_kwargs(from_address="bare-no-at-sign"))
assert "@localhost>" in msg["Message-ID"]
# ---------------------------------------------------------------------------
# Auto-Submitted (RFC 3834)
# ---------------------------------------------------------------------------
def test_envelope_sets_auto_submitted_for_transactional_default():
msg = build_envelope(**_base_kwargs())
assert msg["Auto-Submitted"] == "auto-generated"
def test_envelope_omits_auto_submitted_when_transactional_is_false():
msg = build_envelope(**_base_kwargs(is_transactional=False))
assert msg["Auto-Submitted"] is None
# ---------------------------------------------------------------------------
# Reply-To
# ---------------------------------------------------------------------------
def test_envelope_sets_reply_to_when_provided():
msg = build_envelope(**_base_kwargs(reply_to="ohm@wiggleverse.org"))
assert msg["Reply-To"] == "ohm@wiggleverse.org"
def test_envelope_omits_reply_to_when_absent():
msg = build_envelope(**_base_kwargs())
assert msg["Reply-To"] is None
# ---------------------------------------------------------------------------
# List-Unsubscribe (the headers RFC 8058 / Gmail-Yahoo care about)
# ---------------------------------------------------------------------------
def test_envelope_no_list_unsubscribe_when_neither_given():
"""OTC mail: the recipient explicitly requested the code; no
unsubscribe semantics. The header MUST be absent (presence would
imply OHM has the recipient on a list, which it doesn't)."""
msg = build_envelope(**_base_kwargs())
assert msg["List-Unsubscribe"] is None
assert msg["List-Unsubscribe-Post"] is None
def test_envelope_mailto_only_list_unsubscribe():
"""Admin invite / per-RFC invite: `mailto:` form only, no URL.
The recipient isn't a user yet, so there's no per-user opt-out
URL to flip; the operator handles ad-hoc opt-outs manually."""
msg = build_envelope(**_base_kwargs(
unsubscribe_mailto="ohm@wiggleverse.org?subject=remove",
))
assert msg["List-Unsubscribe"] == "<mailto:ohm@wiggleverse.org?subject=remove>"
# NO List-Unsubscribe-Post when only a mailto is present — the
# one-click semantic requires a URL the MUA can POST to.
assert msg["List-Unsubscribe-Post"] is None
def test_envelope_full_one_click_list_unsubscribe():
"""Watcher notification / bundle: `mailto:` + signed-URL +
`List-Unsubscribe-Post: List-Unsubscribe=One-Click`. Gmail and
Yahoo enforce this for bulk-adjacent mail per RFC 8058."""
msg = build_envelope(**_base_kwargs(
unsubscribe_mailto="ohm@wiggleverse.org?subject=remove",
unsubscribe_url="https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc",
))
lu = msg["List-Unsubscribe"]
assert "<mailto:ohm@wiggleverse.org?subject=remove>" in lu
assert "<https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc>" in lu
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
def test_envelope_url_only_list_unsubscribe_still_sets_post():
msg = build_envelope(**_base_kwargs(
unsubscribe_url="https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc",
))
assert msg["List-Unsubscribe"] == "<https://ohm.wiggleverse.org/api/email/unsubscribe?t=abc>"
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
# ---------------------------------------------------------------------------
# Body shape — plain-only vs multipart/alternative
# ---------------------------------------------------------------------------
def test_envelope_plain_only_body_is_text_plain():
msg = build_envelope(**_base_kwargs())
# No HTML alternative -> single-part text/plain.
assert msg.get_content_type() == "text/plain"
assert msg.get_content().strip() == "Hello, world."
def test_envelope_html_body_is_guarded_not_enabled():
# I3 (security-audit-0026): the HTML/multipart-alternative path is
# intentionally not enabled — passing body_html must fail loudly so
# a future caller can't silently ship unescaped user HTML (the C1
# stored-XSS class in the mail channel). When HTML mail is enabled
# deliberately, this test flips to assert the multipart shape.
with pytest.raises(NotImplementedError):
build_envelope(**_base_kwargs(body_html="<p>Hello, <b>world</b>.</p>"))
def test_envelope_html_none_is_plain_only():
# The guard keys on `is not None`, so the default (None) stays the
# live plain-text path — exercised here to lock the boundary.
msg = build_envelope(**_base_kwargs(body_html=None))
assert msg.get_content_type() == "text/plain"
@@ -612,3 +612,127 @@ def test_explicit_watch_set_overrides_auto(app_with_fake_gitea):
# the user put them.
assert row["set_by"] == "explicit"
assert row["state"] == "following"
# ---------------------------------------------------------------------------
# v0.18.0 — envelope headers + RFC 8058 one-click POST endpoint
#
# Watcher notifications are bulk-adjacent (a busy RFC can produce
# dozens of structural events); per the proposal, they MUST carry
# `Date`, `Message-ID`, `Auto-Submitted`, full `List-Unsubscribe`
# (mailto + signed URL), AND `List-Unsubscribe-Post:
# List-Unsubscribe=One-Click` per RFC 8058. Gmail and Yahoo
# enforce this for senders at OHM's volume tier.
# ---------------------------------------------------------------------------
def test_notification_envelope_carries_full_one_click_headers(app_with_fake_gitea):
"""A `proposal_merged` event lands a watcher notification email
with the full one-click unsubscribe shape."""
from fastapi.testclient import TestClient
from email.utils import parsedate_to_datetime
from app import db, email as email_mod
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
r = client.post("/api/rfcs/propose", json={"title": "OHM", "slug": "ohm", "pitch": PITCH, "tags": []})
assert r.status_code == 200
email_mod.reset_sent_envelopes()
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner", email="ben@test")
merge_r = client.post(f"/api/proposals/{r.json()['pr_number']}/merge")
assert merge_r.status_code == 200, merge_r.text
envelopes = [e for e in email_mod.sent_envelopes() if e["to"] == "alice@test"]
assert envelopes, "watcher notification did not fire"
msg = envelopes[-1]["message"]
# Always-present headers from the helper.
assert parsedate_to_datetime(msg["Date"]) is not None
assert msg["Message-ID"].startswith("<") and msg["Message-ID"].endswith(">")
assert msg["Auto-Submitted"] == "auto-generated"
# Full one-click unsubscribe.
lu = msg["List-Unsubscribe"]
assert lu is not None
assert "<mailto:" in lu
# URL part carries the signed token per make_unsubscribe_url.
assert "/api/email/unsubscribe?t=" in lu
assert msg["List-Unsubscribe-Post"] == "List-Unsubscribe=One-Click"
def test_email_unsubscribe_post_one_click_flips_category_off(app_with_fake_gitea):
"""RFC 8058: Gmail/Yahoo POST `List-Unsubscribe=One-Click` to the
URL in the List-Unsubscribe header. The endpoint MUST accept POST
+ the same token shape as the GET handler + return 200 + flip the
flag."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
token = email_mod.make_unsubscribe_url(2, "personal-direct").split("t=", 1)[1]
r = client.post(
f"/api/email/unsubscribe?t={token}",
data={"List-Unsubscribe": "One-Click"},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["ok"] is True
assert body["category"] == "personal-direct"
row = db.conn().execute(
"SELECT email_personal_direct FROM users WHERE id = 2"
).fetchone()
assert row["email_personal_direct"] == 0
def test_email_unsubscribe_post_all_sets_global_opt_out(app_with_fake_gitea):
"""The v0.18.0 `all` synthetic category (used by the bundle +
digest paths) MUST set `email_opt_out_all = 1`."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
token = email_mod.make_unsubscribe_url(2, "all").split("t=", 1)[1]
r = client.post(f"/api/email/unsubscribe?t={token}")
assert r.status_code == 200
assert r.json() == {"ok": True, "category": "all"}
row = db.conn().execute(
"SELECT email_opt_out_all FROM users WHERE id = 2"
).fetchone()
assert row["email_opt_out_all"] == 1
def test_email_unsubscribe_get_all_sets_global_opt_out(app_with_fake_gitea):
"""GET handler also accepts the `all` category and lands the
global opt-out (so an MUA that doesn't honor RFC 8058 POST and
just opens the URL in a browser still works)."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
token = email_mod.make_unsubscribe_url(2, "all").split("t=", 1)[1]
r = client.get(f"/api/email/unsubscribe?t={token}")
assert r.status_code == 200
assert "Unsubscribed" in r.text
row = db.conn().execute(
"SELECT email_opt_out_all FROM users WHERE id = 2"
).fetchone()
assert row["email_opt_out_all"] == 1
def test_email_unsubscribe_post_refuses_invalid_token(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post("/api/email/unsubscribe?t=not-a-valid-token")
assert r.status_code == 400
+50
View File
@@ -347,3 +347,53 @@ def test_otc_re_request_invalidates_prior_unused_code(app_with_fake_gitea, monke
# The new code still works.
r = client.post("/auth/otc/verify", json={"email": "alice@example.com", "code": second})
assert r.status_code == 200
# ---------------------------------------------------------------------------
# v0.18.0: envelope headers — Slice 2
#
# OTC mail goes through `build_envelope` and MUST land Date,
# Message-ID, and Auto-Submitted but MUST NOT carry a
# List-Unsubscribe header (the recipient explicitly requested the
# code; advertising a list semantic would be wrong).
# ---------------------------------------------------------------------------
def _last_otc_envelope():
from app import email as email_mod
otc = [e for e in email_mod.sent_envelopes() if e.get("kind") == "otc"]
assert otc, "no OTC envelope in the buffer"
return otc[-1]
def test_otc_envelope_sets_date_messageid_autosubmitted(app_with_fake_gitea):
from fastapi.testclient import TestClient
from email.utils import parsedate_to_datetime
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
_reset_outbound()
client.post("/auth/otc/request", json={"email": "headers@example.com"})
msg = _last_otc_envelope()["message"]
# Date is RFC 5322 parseable.
assert parsedate_to_datetime(msg["Date"]) is not None
# Message-ID is bracketed and carries the From-address @-domain.
mid = msg["Message-ID"]
assert mid.startswith("<") and mid.endswith(">")
# Auto-Submitted prevents auto-responder loops.
assert msg["Auto-Submitted"] == "auto-generated"
def test_otc_envelope_has_no_list_unsubscribe(app_with_fake_gitea):
"""The recipient explicitly typed their email and asked for a
code; the framework MUST NOT advertise a list semantic on this
mail. Per the v0.18.0 proposal's tradeoff discussion."""
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
_reset_outbound()
client.post("/auth/otc/request", json={"email": "headers@example.com"})
msg = _last_otc_envelope()["message"]
assert msg["List-Unsubscribe"] is None
assert msg["List-Unsubscribe-Post"] is None
@@ -0,0 +1,368 @@
"""End-to-end integration tests for the v0.18.0 Slice 4
outbound_emails audit table + admin endpoint.
The release adds:
* `backend/migrations/020_outbound_emails.sql` the audit table.
* `record_outbound()` in `email.py` the write helper every send
path calls before returning, capturing status='sent' / 'failed'
/ 'deferred' (the dev-fallback path when SMTP_HOST is unset).
* `GET /api/admin/outbound-emails` admin-only listing, filterable
by kind / status / to_address.
These tests prove:
* Sending OTC / invite / notification mail writes one row per send
(status='deferred' under tests since SMTP_HOST is unset).
* The Message-ID on the row matches the envelope's Message-ID
header (the seam Slice 5 uses for bounce correlation).
* `kind` is populated per send path.
* `GET /api/admin/outbound-emails` lists rows newest-first,
accepts filters, refuses non-admins.
"""
from __future__ import annotations
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
# ---------------------------------------------------------------------------
# Write-on-send wiring
# ---------------------------------------------------------------------------
def test_otc_send_writes_outbound_row_with_message_id(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
email_mod.reset_sent_envelopes()
r = client.post("/auth/otc/request", json={"email": "newcomer@ex.co"})
assert r.status_code == 200
# Audit row landed.
rows = db.conn().execute(
"SELECT id, to_address, kind, status, message_id, error "
"FROM outbound_emails WHERE to_address = 'newcomer@ex.co'"
).fetchall()
assert len(rows) == 1
row = rows[0]
assert row["kind"] == "otc"
# No SMTP_HOST in tests -> 'deferred', not 'sent'.
assert row["status"] == "deferred"
assert row["error"] is None
# Message-ID matches the envelope's header (the seam Slice 5 uses).
envelopes = [e for e in email_mod.sent_envelopes() if e.get("kind") == "otc"]
assert envelopes
envelope_mid = envelopes[-1]["message"]["Message-ID"]
assert row["message_id"] == envelope_mid
def test_invite_send_writes_outbound_row(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import db
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=500, login="adminQ", role="admin")
sign_in_as(
client, user_id=500, gitea_login="adminQ",
display_name="Admin Q", role="admin",
email="adminq@test",
)
r = client.post(
"/api/admin/users",
json={
"email": "invitee@ex.co",
"first_name": "Inv", "last_name": "Itee",
"role": "contributor", "custom_message": "",
},
)
assert r.status_code == 200, r.text
rows = db.conn().execute(
"SELECT kind, status, message_id FROM outbound_emails "
"WHERE to_address = 'invitee@ex.co'"
).fetchall()
assert len(rows) == 1
assert rows[0]["kind"] == "invite"
assert rows[0]["status"] == "deferred"
assert rows[0]["message_id"] is not None
def test_notification_send_writes_outbound_row_with_notification_id(app_with_fake_gitea):
"""Watcher notifications carry a `notification_id` FK so the
admin can join through to the notifications table to see what
triggered the send."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
from test_notifications_vertical import PITCH
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(
client, user_id=2, gitea_login="alice",
display_name="Alice", role="contributor", email="alice@test",
)
r = client.post("/api/rfcs/propose", json={
"title": "OHM", "slug": "ohm", "pitch": PITCH, "tags": [],
})
email_mod.reset_sent_envelopes()
# Wipe pre-merge audit rows so the assertion below is unambiguous.
db.conn().execute("DELETE FROM outbound_emails")
sign_in_as(
client, user_id=1, gitea_login="ben",
display_name="Ben", role="owner", email="ben@test",
)
merge_r = client.post(f"/api/proposals/{r.json()['pr_number']}/merge")
assert merge_r.status_code == 200, merge_r.text
rows = db.conn().execute(
"SELECT kind, status, notification_id, message_id "
"FROM outbound_emails WHERE to_address = 'alice@test'"
).fetchall()
assert rows, "no outbound_emails row for alice@test"
# At least one notification kind, with a populated FK.
notif_rows = [r for r in rows if r["kind"] == "notification"]
assert notif_rows
for nr in notif_rows:
assert nr["status"] == "deferred"
assert nr["notification_id"] is not None
assert nr["message_id"] is not None
# ---------------------------------------------------------------------------
# Admin endpoint
# ---------------------------------------------------------------------------
def test_admin_outbound_emails_lists_rows(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
# Generate a few rows.
client.post("/auth/otc/request", json={"email": "one@ex.co"})
provision_user_row(user_id=600, login="adminR", role="admin")
sign_in_as(
client, user_id=600, gitea_login="adminR",
display_name="Admin R", role="admin", email="adminr@test",
)
client.post("/api/admin/users", json={
"email": "two@ex.co", "first_name": "T", "last_name": "Wo",
"role": "contributor", "custom_message": "",
})
r = client.get("/api/admin/outbound-emails")
assert r.status_code == 200, r.text
items = r.json()["items"]
kinds = {it["kind"] for it in items}
assert "otc" in kinds
assert "invite" in kinds
# Newest-first.
ids = [it["id"] for it in items]
assert ids == sorted(ids, reverse=True)
def test_admin_outbound_emails_filters_by_kind(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
client.post("/auth/otc/request", json={"email": "filter1@ex.co"})
provision_user_row(user_id=601, login="adminS", role="admin")
sign_in_as(
client, user_id=601, gitea_login="adminS",
display_name="Admin S", role="admin", email="admins@test",
)
client.post("/api/admin/users", json={
"email": "filter2@ex.co", "first_name": "F", "last_name": "Two",
"role": "contributor", "custom_message": "",
})
r = client.get("/api/admin/outbound-emails?kind=otc")
assert r.status_code == 200
items = r.json()["items"]
assert items
assert all(it["kind"] == "otc" for it in items)
def test_admin_outbound_emails_filters_by_to_address(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
client.post("/auth/otc/request", json={"email": "TARGET@ex.co"})
client.post("/auth/otc/request", json={"email": "other@ex.co"})
provision_user_row(user_id=602, login="adminT", role="admin")
sign_in_as(
client, user_id=602, gitea_login="adminT",
display_name="Admin T", role="admin", email="admint@test",
)
# to_address filter is case-insensitive.
r = client.get("/api/admin/outbound-emails?to_address=target@ex.co")
assert r.status_code == 200
items = r.json()["items"]
assert items
assert all(it["to_address"].lower() == "target@ex.co" for it in items)
def test_admin_outbound_emails_refuses_non_admin(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=700, login="contribU", role="contributor")
sign_in_as(
client, user_id=700, gitea_login="contribU",
display_name="Contrib U", role="contributor",
)
r = client.get("/api/admin/outbound-emails")
assert r.status_code == 403
# ---------------------------------------------------------------------------
# v0.18.0 Slice 5: bounce correlation
# ---------------------------------------------------------------------------
def test_bounce_with_message_id_marks_outbound_row_bounced(app_with_fake_gitea):
"""When the bounce body includes the original `message_id`, the
framework looks it up in outbound_emails and stamps
status='bounced' on the matching row. The hard-bounce ->
global-opt-out logic still fires."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=800, login="bouncey", role="contributor")
db.conn().execute("UPDATE users SET email = 'bouncey@ex.co' WHERE id = 800")
# Send something to bouncey to land an outbound_emails row.
email_mod.reset_sent_envelopes()
client.post("/auth/otc/request", json={"email": "bouncey@ex.co"})
row = db.conn().execute(
"SELECT id, message_id, status FROM outbound_emails "
"WHERE to_address = 'bouncey@ex.co'"
).fetchone()
assert row is not None
original_id = row["id"]
message_id = row["message_id"]
assert row["status"] == "deferred" # pre-bounce baseline
# Bounce comes in carrying that message_id.
r = client.post(
"/api/webhooks/email-bounce",
json={
"email": "bouncey@ex.co",
"kind": "hard",
"message_id": message_id,
},
)
assert r.status_code == 200
body = r.json()
assert body["matched"] is True
assert body["correlated_id"] == original_id
# Audit row stamped.
post = db.conn().execute(
"SELECT status, error FROM outbound_emails WHERE id = ?",
(original_id,),
).fetchone()
assert post["status"] == "bounced"
assert "bounce (hard)" in (post["error"] or "")
# Hard-bounce global opt-out still fires.
urow = db.conn().execute(
"SELECT email_opt_out_all FROM users WHERE id = 800"
).fetchone()
assert urow["email_opt_out_all"] == 1
def test_bounce_with_unknown_message_id_does_not_crash(app_with_fake_gitea):
"""A message_id the framework doesn't recognize logs but does
NOT 5xx bounce providers replay old bounces, and the
framework can't refuse just because the row was pruned."""
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post(
"/api/webhooks/email-bounce",
json={
"email": "nobody@ex.co",
"kind": "hard",
"message_id": "<not-in-our-db@ex.co>",
},
)
assert r.status_code == 200
assert r.json()["correlated_id"] is None
def test_bounce_without_message_id_still_flips_opt_out(app_with_fake_gitea):
"""Backward compat: providers that don't surface Message-ID
still get the legacy v1 behavior match by email + flip the
global opt-out."""
from fastapi.testclient import TestClient
from app import db
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=801, login="legacybounce", role="contributor")
db.conn().execute("UPDATE users SET email = 'legacy@ex.co' WHERE id = 801")
r = client.post(
"/api/webhooks/email-bounce",
json={"email": "legacy@ex.co", "kind": "hard"},
)
assert r.status_code == 200
body = r.json()
assert body["matched"] is True
assert body["correlated_id"] is None
urow = db.conn().execute(
"SELECT email_opt_out_all FROM users WHERE id = 801"
).fetchone()
assert urow["email_opt_out_all"] == 1
def test_bounced_rows_show_in_admin_endpoint(app_with_fake_gitea):
"""The admin endpoint surfaces bounced rows alongside the rest;
filtering by `status=bounced` isolates them."""
from fastapi.testclient import TestClient
from app import db, email as email_mod
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=802, login="adminB", role="admin")
sign_in_as(
client, user_id=802, gitea_login="adminB",
display_name="Admin B", role="admin", email="adminb@test",
)
email_mod.reset_sent_envelopes()
client.post("/auth/otc/request", json={"email": "willbounce@ex.co"})
row = db.conn().execute(
"SELECT message_id FROM outbound_emails WHERE to_address = 'willbounce@ex.co'"
).fetchone()
client.post(
"/api/webhooks/email-bounce",
json={"email": "willbounce@ex.co", "kind": "hard", "message_id": row["message_id"]},
)
r = client.get("/api/admin/outbound-emails?status=bounced")
assert r.status_code == 200
items = r.json()["items"]
assert items
assert all(it["status"] == "bounced" for it in items)
assert any(it["to_address"] == "willbounce@ex.co" for it in items)
+17 -1
View File
@@ -438,8 +438,24 @@ def tmp_env(monkeypatch):
"SECRET_KEY": "test-secret-key-for-cookies",
"DATABASE_PATH": str(db_path),
"OWNER_GITEA_LOGIN": "ben",
"GITEA_WEBHOOK_SECRET": "",
# v0.18.0: `GITEA_WEBHOOK_SECRET` is now mandatory at startup
# per the email + webhook hygiene proposal. Tests bind a fake
# value so the framework boots; tests that want to exercise
# the dev-bypass path monkeypatch `RFC_APP_INSECURE_WEBHOOKS=1`.
"GITEA_WEBHOOK_SECRET": "test-webhook-secret-for-signature-verification",
"ENABLED_MODELS": "claude",
# v0.27.0 (audit 0026 M4): the session cookie now defaults to
# Secure. The TestClient talks plain http://testserver, so a
# Secure cookie is never sent back and every authenticated flow
# would fail. Tests opt out explicitly, exactly as a dev box on
# plain http does.
"SESSION_COOKIE_SECURE": "false",
# v0.27.0 (audit 0026 M5): the bounce webhook fails closed (503)
# when its secret is unset. Tests exercise the legacy behavioral
# path via the documented dev opt-in, mirroring the
# RFC_APP_INSECURE_WEBHOOKS bypass above. Tests that assert the
# fail-closed default delenv this key themselves.
"RFC_APP_INSECURE_BOUNCE_WEBHOOK": "1",
}
for k, v in env.items():
monkeypatch.setenv(k, v)
@@ -0,0 +1,161 @@
"""End-to-end vertical for roadmap #26 (rfc-app v0.22.0): the optional
"What will you be using this for?" capture on the two propose surfaces.
Reuses the FakeGitea + session helpers from test_propose_vertical.py and
the active-RFC seed from test_rfc_view_vertical.py. Proves:
(a) propose-RFC persists and returns `proposed_use_case` when supplied,
and the value survives onto the merged super-draft's RFC view;
(b) propose-RFC accepts a NULL / omitted use case ("left blank");
(c) propose-PR persists and returns `proposed_use_case` when supplied,
and accepts a NULL / omitted one.
"""
from __future__ import annotations
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
from test_pr_flow_vertical import _cut_branch_and_accept_change
from test_rfc_view_vertical import SEED_BODY, seed_active_rfc
# ---------------------------------------------------------------------------
# propose-RFC
# ---------------------------------------------------------------------------
def test_propose_rfc_persists_and_returns_use_case(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
provision_user_row(user_id=1, login="ben", role="owner")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
r = client.post("/api/rfcs/propose", json={
"title": "Open Human Model",
"slug": "open-human-model",
"pitch": "A shared definition of what we mean by *human*.",
"tags": ["identity"],
"proposed_use_case": "Wiring OHM into the OpenXML consent surface.",
})
assert r.status_code == 200, r.text
pr_number = r.json()["pr_number"]
# The pending-idea list carries the use case.
items = client.get("/api/proposals").json()["items"]
assert items[0]["proposed_use_case"] == "Wiring OHM into the OpenXML consent surface."
# The pending-idea detail view carries it too.
proposal = client.get(f"/api/proposals/{pr_number}").json()
assert proposal["proposed_use_case"] == "Wiring OHM into the OpenXML consent surface."
# Merge as owner; the use case survives onto the RFC view (looked
# up by slug from the canonical side table, since the idea PR
# closes on merge).
sign_in_as(client, user_id=1, gitea_login="ben", display_name="Ben", role="owner", email="ben@test")
r = client.post(f"/api/proposals/{pr_number}/merge")
assert r.status_code == 200, r.text
view = client.get("/api/rfcs/open-human-model").json()
assert view["proposed_use_case"] == "Wiring OHM into the OpenXML consent surface."
def test_propose_rfc_use_case_optional(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=3, login="carol", role="contributor")
sign_in_as(client, user_id=3, gitea_login="carol", display_name="Carol", role="contributor")
# Omitted entirely.
r = client.post("/api/rfcs/propose", json={
"title": "No Use Case", "slug": "no-use-case", "pitch": "p", "tags": [],
})
assert r.status_code == 200, r.text
pr_a = r.json()["pr_number"]
# Explicit null.
r = client.post("/api/rfcs/propose", json={
"title": "Null Use Case", "slug": "null-use-case", "pitch": "p",
"tags": [], "proposed_use_case": None,
})
assert r.status_code == 200, r.text
pr_b = r.json()["pr_number"]
# Blank/whitespace — treated as "left blank", no row written.
r = client.post("/api/rfcs/propose", json={
"title": "Blank Use Case", "slug": "blank-use-case", "pitch": "p",
"tags": [], "proposed_use_case": " ",
})
assert r.status_code == 200, r.text
pr_c = r.json()["pr_number"]
for pr in (pr_a, pr_b, pr_c):
assert client.get(f"/api/proposals/{pr}").json()["proposed_use_case"] is None
# ---------------------------------------------------------------------------
# propose-PR (against an active RFC)
# ---------------------------------------------------------------------------
def test_propose_pr_persists_and_returns_use_case(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY)
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
branch, _ = _cut_branch_and_accept_change(
client, fake, slug="ohm",
original="Open Human Model is a framework for representing humans.",
proposed="Open Human Model is a framework for representing humans across systems.",
)
r = client.post(
f"/api/rfcs/ohm/branches/{branch}/open-pr",
json={
"title": "Tighten the opening",
"description": "Scope to systems.",
"proposed_use_case": "Building a cross-system consent registry.",
},
)
assert r.status_code == 200, r.text
pr_number = r.json()["pr_number"]
pr = client.get(f"/api/rfcs/ohm/prs/{pr_number}").json()
assert pr["proposed_use_case"] == "Building a cross-system consent registry."
def test_propose_pr_use_case_optional(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY)
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
branch, _ = _cut_branch_and_accept_change(
client, fake, slug="ohm",
original="It defines consent, trait, and agency in compatible terms.",
proposed="It defines consent, trait, harm, and agency in compatible terms.",
)
# No proposed_use_case key at all.
r = client.post(
f"/api/rfcs/ohm/branches/{branch}/open-pr",
json={"title": "Add harm", "description": "Name harm explicitly."},
)
assert r.status_code == 200, r.text
pr_number = r.json()["pr_number"]
pr = client.get(f"/api/rfcs/ohm/prs/{pr_number}").json()
assert pr["proposed_use_case"] is None
+235
View File
@@ -0,0 +1,235 @@
"""Roadmap #28 Part 1 — auto-link RFC references in PR text + comments.
Two layers:
* Unit tests over the pure scanner (`rfc_links.segment_text` /
`_keys_for` / `LinkIndex`) the matching rules and their
false-positive guards, no DB.
* End-to-end tests that the PR description, PR review comments, and
PR-less discussion comments all surface `*_segments` enriched against
the live accepted-RFC corpus, with self-references suppressed.
Reuses the FakeGitea + session helpers from test_propose_vertical.py and
the active-RFC seed from test_rfc_view_vertical.py.
"""
from __future__ import annotations
from app import rfc_links
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
grant_rfc_collaborator,
provision_user_row,
sign_in_as,
tmp_env,
)
from test_rfc_view_vertical import SEED_BODY, seed_active_rfc
from test_pr_flow_vertical import _cut_branch_and_accept_change
# ---------------------------------------------------------------------------
# Unit — the pure scanner
# ---------------------------------------------------------------------------
def _idx(*terms):
"""Build a LinkIndex from raw (key, slug, title) tuples (keys lower)."""
return rfc_links.LinkIndex(list(terms))
def test_empty_text_is_single_empty_segment():
assert rfc_links.segment_text("", []) == [{"type": "text", "text": ""}]
assert rfc_links.segment_text(None, []) == [{"type": "text", "text": ""}]
def test_no_terms_returns_plain_text():
out = rfc_links.segment_text("hello world", [])
assert out == [{"type": "text", "text": "hello world"}]
def test_multiword_title_links_and_preserves_casing():
idx = _idx(("open human model", "open-human-model", "Open Human Model"))
out = idx.segment("See the Open Human Model for details.")
assert out == [
{"type": "text", "text": "See the "},
{"type": "rfc", "slug": "open-human-model", "label": "Open Human Model",
"title": "Open Human Model"},
{"type": "text", "text": " for details."},
]
def test_match_is_case_insensitive():
idx = _idx(("open human model", "open-human-model", "Open Human Model"))
out = idx.segment("see the OPEN HUMAN MODEL")
assert out[-1] == {"type": "rfc", "slug": "open-human-model",
"label": "OPEN HUMAN MODEL", "title": "Open Human Model"}
def test_word_boundary_prevents_substring_match():
# "harm" must not match inside "charming" / "harmless".
idx = _idx(("rfc-0001", "open-human-model", "Open Human Model"))
out = idx.segment("a charming rfc-00012 not real")
# rfc-0001 is a prefix of rfc-00012 but the trailing '2' is a word char,
# so no match — the whole string stays plain text.
assert out == [{"type": "text", "text": "a charming rfc-00012 not real"}]
def test_rfc_id_token_links():
idx = _idx(("rfc-0001", "open-human-model", "Open Human Model"))
out = idx.segment("as established in RFC-0001.")
assert out[1] == {"type": "rfc", "slug": "open-human-model",
"label": "RFC-0001", "title": "Open Human Model"}
def test_longest_match_wins():
# A bare "Open" term and the full title both present; the full title
# (longer) must win at the position.
idx = _idx(
("open", "open", "Open"),
("open human model", "open-human-model", "Open Human Model"),
)
out = idx.segment("the Open Human Model")
assert out[-1]["slug"] == "open-human-model"
assert out[-1]["label"] == "Open Human Model"
def test_keys_for_gating():
keys = lambda **kw: set(rfc_links._keys_for(**kw))
# rfc_id always contributes.
assert "rfc-0001" in keys(slug="x", title="X", rfc_id="RFC-0001")
# multi-word title contributes; single common word does NOT.
assert "open human model" in keys(slug="ohm", title="Open Human Model", rfc_id=None)
assert keys(slug="human", title="Human", rfc_id=None) == set()
# hyphenated slug contributes; single-token slug does NOT.
assert "open-human-model" in keys(slug="open-human-model", title="X", rfc_id=None)
assert "ohm" not in keys(slug="ohm", title="OHM", rfc_id=None)
# ---------------------------------------------------------------------------
# End-to-end — enrichment surfaces on the read paths
# ---------------------------------------------------------------------------
def _open_pr_on(client, fake, *, host_slug: str, description: str):
"""Seed branch + accepted change on host_slug and open a PR. Returns
the pr_number."""
# `original` must exist verbatim in SEED_BODY or the accept is "stale".
branch, _ = _cut_branch_and_accept_change(
client, fake, slug=host_slug,
original="It defines consent, trait, and agency in compatible terms.",
proposed="It defines consent, trait, harm, and agency in compatible terms.",
)
r = client.post(
f"/api/rfcs/{host_slug}/branches/{branch}/open-pr",
json={"title": "A change", "description": description},
)
assert r.status_code == 200, r.text
return r.json()["pr_number"]
def _rfc_segments(segments):
return [s for s in segments if s["type"] == "rfc"]
def test_pr_description_autolinks_other_rfc(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
# Two accepted RFCs: a host for the PR + a referenceable target.
seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY)
seed_active_rfc(fake, slug="open-human-model", title="Open Human Model", body=SEED_BODY)
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
pr_number = _open_pr_on(
client, fake, host_slug="ohm",
description="This builds on the Open Human Model definition.",
)
r = client.get(f"/api/rfcs/ohm/prs/{pr_number}")
assert r.status_code == 200, r.text
pr = r.json()
links = _rfc_segments(pr["description_segments"])
assert len(links) == 1
assert links[0]["slug"] == "open-human-model"
assert links[0]["label"] == "Open Human Model"
# The plain text is still present for non-segment callers (the bot
# appends a §6.5 On-behalf-of trailer, so this is a containment check).
assert "This builds on the Open Human Model definition." in pr["description"]
def test_pr_review_comment_autolinked(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY)
seed_active_rfc(fake, slug="open-human-model", title="Open Human Model", body=SEED_BODY)
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
pr_number = _open_pr_on(client, fake, host_slug="ohm", description="plain.")
r = client.post(
f"/api/rfcs/ohm/prs/{pr_number}/review",
json={"text": "See Open Human Model and RFC-0001.", "anchor_payload": {}},
)
assert r.status_code == 200, r.text
pr = client.get(f"/api/rfcs/ohm/prs/{pr_number}").json()
all_msgs = [m for msgs in pr["messages_by_thread"].values() for m in msgs]
review_msgs = [m for m in all_msgs if "Open Human Model" in (m["text"] or "")]
assert review_msgs, "review comment not found in payload"
links = _rfc_segments(review_msgs[0]["text_segments"])
# Both "Open Human Model" (title) and "RFC-0001" (id) point to the
# one referenceable RFC.
assert {s["slug"] for s in links} == {"open-human-model"}
assert {s["label"] for s in links} == {"Open Human Model", "RFC-0001"}
def test_discussion_comment_autolinked(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
seed_active_rfc(fake, slug="ohm", title="OHM", body=SEED_BODY)
seed_active_rfc(fake, slug="open-human-model", title="Open Human Model", body=SEED_BODY)
# alice is the seeded owner of ohm (owners=["alice"]); grant the
# per-RFC collaborator row explicitly so the #12 discuss gate passes.
grant_rfc_collaborator(user_id=2, rfc_slug="ohm", role_in_rfc="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
r = client.post(
"/api/rfcs/ohm/discussion/threads",
json={"message": "Compare with the Open Human Model."},
)
assert r.status_code == 200, r.text
thread_id = r.json()["thread_id"]
r = client.get(f"/api/rfcs/ohm/discussion/threads/{thread_id}/messages")
assert r.status_code == 200, r.text
msgs = r.json()["messages"]
assert msgs and "text_segments" in msgs[0]
links = _rfc_segments(msgs[0]["text_segments"])
assert len(links) == 1
assert links[0]["slug"] == "open-human-model"
def test_self_reference_not_linked(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
# The host RFC has a multi-word title, so absent exclude_slug it
# WOULD self-link. exclude_slug must suppress it.
seed_active_rfc(fake, slug="open-human-model", title="Open Human Model", body=SEED_BODY)
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
pr_number = _open_pr_on(
client, fake, host_slug="open-human-model",
description="Refines the Open Human Model definition.",
)
pr = client.get(f"/api/rfcs/open-human-model/prs/{pr_number}").json()
assert _rfc_segments(pr["description_segments"]) == []
@@ -0,0 +1,140 @@
"""End-to-end integration tests for the v0.23.0 sign-in state-resume
vertical (§6.2, roadmap item #29).
New behavior: each authenticated user's last-viewed route + a small bag
of light view state is tracked server-side, so the next sign-in can land
them back where they left off rather than on the empty-state home view.
The tests below prove:
* `PUT /api/me/last-state` requires auth an anonymous client gets
401, and nothing is stored.
* An authenticated PUT upserts the route, and the stored route is
read back for that user off `GET /api/auth/me` (`last_route`).
* A second PUT overwrites (upsert, one row per user) the latest
route wins.
* `last_route_state` round-trips as decoded JSON on `/api/auth/me`.
* Per-user isolation: user A's stored route is not visible to user B.
* `resume_enabled = 0` disables resume: the PUT no-ops (does not
rewrite the stored route) and `/api/auth/me` hands back a null
`last_route` even though a stored row exists.
The fakes from `test_propose_vertical` give us a working app harness +
the `sign_in_as` / `provision_user_row` seams.
"""
from __future__ import annotations
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
def test_put_last_state_requires_auth(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import db
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
# Anonymous — no session cookie set.
r = client.put("/api/me/last-state", json={"route": "/rfc/open-human-model"})
assert r.status_code == 401
# Nothing landed in the table.
row = db.conn().execute("SELECT COUNT(*) AS n FROM user_session_state").fetchone()
assert row["n"] == 0
def test_put_last_state_upserts_and_reads_back(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
# First POST stores a route + light state.
r = client.put(
"/api/me/last-state",
json={"route": "/rfc/open-human-model", "state": {"tab": "discussion", "scroll": 420}},
)
assert r.status_code == 200
assert r.json()["stored"] is True
# /api/auth/me hands the route + decoded state back.
me = client.get("/api/auth/me").json()
assert me["authenticated"] is True
assert me["user"]["resume_enabled"] is True
assert me["user"]["last_route"] == "/rfc/open-human-model"
assert me["user"]["last_route_state"] == {"tab": "discussion", "scroll": 420}
# A later POST overwrites — one row per user, latest wins.
r = client.put("/api/me/last-state", json={"route": "/proposals/7"})
assert r.status_code == 200
me = client.get("/api/auth/me").json()
assert me["user"]["last_route"] == "/proposals/7"
# state was omitted on the second POST → cleared to null.
assert me["user"]["last_route_state"] is None
def test_last_state_is_per_user(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
provision_user_row(user_id=3, login="bob", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
client.put("/api/me/last-state", json={"route": "/rfc/alice-route"})
# Switch to bob — he has no stored route yet.
sign_in_as(client, user_id=3, gitea_login="bob", display_name="Bob", role="contributor", email="bob@test")
me = client.get("/api/auth/me").json()
assert me["user"]["last_route"] is None
client.put("/api/me/last-state", json={"route": "/rfc/bob-route"})
me = client.get("/api/auth/me").json()
assert me["user"]["last_route"] == "/rfc/bob-route"
# Back to alice — her route is untouched by bob's write.
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
me = client.get("/api/auth/me").json()
assert me["user"]["last_route"] == "/rfc/alice-route"
def test_resume_disabled_no_ops_put_and_hides_route(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import db
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor", email="alice@test")
# Seed a stored row, then flip resume_enabled off directly (the
# profile-settings toggle UI to do this from the client is a
# follow-up; the column + behavior ship now).
client.put("/api/me/last-state", json={"route": "/rfc/before-disable"})
db.conn().execute(
"UPDATE user_session_state SET resume_enabled = 0 WHERE user_id = ?",
(2,),
)
# /api/auth/me reports resume off and hands back a null route
# even though a stored row exists.
me = client.get("/api/auth/me").json()
assert me["user"]["resume_enabled"] is False
assert me["user"]["last_route"] is None
# A PUT while disabled no-ops: stored=False and the stored route
# is NOT rewritten.
r = client.put("/api/me/last-state", json={"route": "/rfc/after-disable"})
assert r.status_code == 200
assert r.json()["stored"] is False
row = db.conn().execute(
"SELECT last_route FROM user_session_state WHERE user_id = ?", (2,)
).fetchone()
assert row["last_route"] == "/rfc/before-disable"
+264
View File
@@ -0,0 +1,264 @@
"""Vertical + unit coverage for roadmap #27 (rfc-app v0.24.0): Claude
Haiku tag suggestions on the propose-RFC modal.
Reuses the FakeGitea + session helpers from test_propose_vertical.py.
The Anthropic call is never made for real tests monkeypatch the
`tag_suggest.haiku_provider` seam with a stub provider whose `send`
returns canned text, so the HTTP contract is exercised without a key.
Proves:
(a) the endpoint is contributor-gated (anon 401);
(b) a contributor gets suggestions, filtered to the corpus tag
universe, with invented tags dropped;
(c) no Anthropic key bound empty list, not an error;
(d) an empty corpus empty list (model is never even called);
(e) the per-user rate limit surfaces as a 429;
plus unit coverage of the universe gather, the reply parser's tolerance,
and the suggest() orchestration short-circuits.
"""
from __future__ import annotations
import json
import pytest
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
provision_user_row,
sign_in_as,
tmp_env,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
class StubProvider:
"""A BaseProvider stand-in whose send() returns a fixed string (or
raises, to exercise the graceful-failure path)."""
def __init__(self, reply: str = "[]", *, raises: bool = False):
self.reply = reply
self.raises = raises
self.calls: list[tuple[str, list]] = []
def send(self, system, history):
self.calls.append((system, history))
if self.raises:
raise RuntimeError("boom")
return self.reply
def _seed_tags(slug: str, title: str, tags: list[str], state: str = "active") -> None:
from app import db
db.conn().execute(
"INSERT OR REPLACE INTO cached_rfcs (slug, title, state, tags_json) VALUES (?, ?, ?, ?)",
(slug, title, state, json.dumps(tags)),
)
def _use_provider(monkeypatch, provider) -> None:
monkeypatch.setattr("app.tag_suggest.haiku_provider", lambda config: provider)
@pytest.fixture(autouse=True)
def _reset_rate_limits():
from app import tag_suggest
tag_suggest.reset_rate_limits()
yield
tag_suggest.reset_rate_limits()
# ---------------------------------------------------------------------------
# Endpoint (vertical)
# ---------------------------------------------------------------------------
def test_anonymous_cannot_suggest(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
r = client.post("/api/rfcs/suggest-tags", json={"title": "X"})
assert r.status_code == 401
def test_contributor_gets_filtered_suggestions(app_with_fake_gitea, monkeypatch):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
_seed_tags("ohm", "OHM", ["identity", "schema", "consent"])
_seed_tags("other", "Other", ["identity", "governance"])
# Model returns two real tags (one lowercased to test canonical
# mapping is exact-set anyway), plus one invented tag that MUST
# be dropped.
reply = json.dumps([
{"tag": "identity", "confidence": 0.9},
{"tag": "consent", "confidence": 0.7},
{"tag": "totally-invented", "confidence": 0.99},
])
stub = StubProvider(reply=reply)
_use_provider(monkeypatch, stub)
r = client.post("/api/rfcs/suggest-tags", json={
"title": "Consent and identity",
"pitch": "We need a shared definition of consent tied to identity.",
})
assert r.status_code == 200, r.text
tags = [s["tag"] for s in r.json()["suggestions"]]
assert tags == ["identity", "consent"]
# the model was actually invoked
assert len(stub.calls) == 1
# the universe (deduped distinct tags) was handed to the model
user_msg = stub.calls[0][1][0]["content"]
assert "identity" in user_msg and "governance" in user_msg
def test_no_api_key_returns_empty(app_with_fake_gitea, monkeypatch):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
_seed_tags("ohm", "OHM", ["identity"])
# No key bound (test env has no ANTHROPIC_API_KEY) → provider None.
# (Explicitly assert the seam returns None given the test config.)
from app import tag_suggest
assert tag_suggest.haiku_provider(app.state.config) is None
r = client.post("/api/rfcs/suggest-tags", json={"title": "X", "pitch": "y"})
assert r.status_code == 200, r.text
assert r.json()["suggestions"] == []
def test_empty_corpus_returns_empty_without_calling_model(app_with_fake_gitea, monkeypatch):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
stub = StubProvider(reply=json.dumps([{"tag": "x", "confidence": 1}]))
_use_provider(monkeypatch, stub)
# No cached_rfcs rows → empty universe → suggest() short-circuits.
r = client.post("/api/rfcs/suggest-tags", json={"title": "X", "pitch": "y"})
assert r.status_code == 200, r.text
assert r.json()["suggestions"] == []
assert stub.calls == [] # model never invoked on an empty universe
def test_rate_limit_surfaces_429(app_with_fake_gitea, monkeypatch):
from fastapi.testclient import TestClient
monkeypatch.setenv("TAG_SUGGEST_RATE_MAX", "2")
monkeypatch.setenv("TAG_SUGGEST_RATE_WINDOW_SECONDS", "60")
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
provision_user_row(user_id=2, login="alice", role="contributor")
sign_in_as(client, user_id=2, gitea_login="alice", display_name="Alice", role="contributor")
_seed_tags("ohm", "OHM", ["identity"])
_use_provider(monkeypatch, StubProvider(reply="[]"))
body = {"title": "X", "pitch": "y"}
assert client.post("/api/rfcs/suggest-tags", json=body).status_code == 200
assert client.post("/api/rfcs/suggest-tags", json=body).status_code == 200
# Third call inside the window trips the limit.
assert client.post("/api/rfcs/suggest-tags", json=body).status_code == 429
# ---------------------------------------------------------------------------
# Units
# ---------------------------------------------------------------------------
def test_gather_tag_universe_dedupes_and_ranks(app_with_fake_gitea):
from fastapi.testclient import TestClient
from app import tag_suggest
app, _fake = app_with_fake_gitea
# The db is initialized in the app's lifespan; enter the client
# context so cached_rfcs exists before we seed it directly.
with TestClient(app):
_seed_tags("a", "A", ["identity", "schema"])
_seed_tags("b", "B", ["identity", " schema ", "consent", ""]) # whitespace + empty
_seed_tags("c", "C", ["identity"])
universe = tag_suggest.gather_tag_universe()
# identity (3) > schema (2) > consent (1); whitespace trimmed/merged,
# empties dropped.
assert universe == ["identity", "schema", "consent"]
def test_parse_reply_tolerates_junk():
from app import tag_suggest
universe = ["identity", "schema", "consent"]
# Prose around the JSON, an invented tag, a bare string, a dup, a
# missing confidence, and a garbage confidence.
text = (
"Sure! Here are the tags:\n"
'[{"tag": "identity", "confidence": 0.9}, '
'{"tag": "invented", "confidence": 1}, '
'"schema", '
'{"tag": "identity", "confidence": 0.5}, '
'{"tag": "consent"}, '
'{"tag": "consent", "confidence": "high"}]\n'
"Hope that helps!"
)
out = tag_suggest.parse_reply(text, universe, max_suggestions=6)
tags = [s["tag"] for s in out]
assert tags == ["identity", "schema", "consent"] # invented dropped, deduped
by_tag = {s["tag"]: s["confidence"] for s in out}
assert by_tag["identity"] == 0.9
assert by_tag["schema"] == 0.5 # bare string defaults to 0.5
assert by_tag["consent"] == 0.5 # missing/garbage confidence → 0.5
def test_parse_reply_empty_on_unparseable():
from app import tag_suggest
assert tag_suggest.parse_reply("no json here", ["a"], 6) == []
assert tag_suggest.parse_reply("", ["a"], 6) == []
assert tag_suggest.parse_reply("[]", ["a"], 6) == []
def test_parse_reply_respects_max():
from app import tag_suggest
universe = ["a", "b", "c", "d", "e"]
text = json.dumps([{"tag": t, "confidence": 0.5} for t in universe])
out = tag_suggest.parse_reply(text, universe, max_suggestions=3)
assert [s["tag"] for s in out] == ["a", "b", "c"]
def test_suggest_short_circuits_empty_draft():
from app import tag_suggest
stub = StubProvider(reply=json.dumps([{"tag": "a", "confidence": 1}]))
draft = tag_suggest.Draft(title=" ", pitch="", use_case="")
assert tag_suggest.suggest(stub, draft, ["a"]) == []
assert stub.calls == [] # never called for an empty draft
def test_suggest_returns_empty_on_provider_failure():
from app import tag_suggest
stub = StubProvider(raises=True)
draft = tag_suggest.Draft(title="Real title", pitch="a reason")
assert tag_suggest.suggest(stub, draft, ["identity"]) == []
+38 -7
View File
@@ -55,13 +55,19 @@ def _outbound_otc_envelopes(to_address: str | None = None) -> list[dict]:
def _patch_siteverify(monkeypatch, *, success: bool, error_codes: list[str] | None = None):
"""Replace `httpx.post` inside `app.turnstile` with a stub that
"""Replace `turnstile._siteverify_post` with an async stub that
returns the requested success shape. The stub does not touch the
real CloudFlare endpoint and never sees a real secret.
I4 (security-audit-0026): the siteverify call is now awaited on an
`httpx.AsyncClient`, isolated behind the `_siteverify_post` seam.
Patching that narrow function (rather than the shared
`httpx.AsyncClient`, which gitea/docs also construct) keeps app boot
intact.
"""
captured = {}
def fake_post(url, *, data=None, timeout=None, **kwargs):
async def fake_post(url, data):
captured["url"] = url
captured["data"] = data
body = {"success": bool(success)}
@@ -70,7 +76,7 @@ def _patch_siteverify(monkeypatch, *, success: bool, error_codes: list[str] | No
return SimpleNamespace(json=lambda: body)
from app import turnstile as turnstile_mod
monkeypatch.setattr(turnstile_mod.httpx, "post", fake_post)
monkeypatch.setattr(turnstile_mod, "_siteverify_post", fake_post)
return captured
@@ -170,14 +176,15 @@ def test_otc_request_admits_when_secret_unset_and_not_required(app_with_fake_git
monkeypatch.delenv("CLOUDFLARE_TURNSTILE_SECRET", raising=False)
monkeypatch.delenv("TURNSTILE_REQUIRED", raising=False)
# The httpx.post inside turnstile must not be called in this path —
# patch it to a sentinel that explodes if it ever runs.
# The siteverify call inside turnstile must not be made in this path —
# patch the seam to a sentinel that explodes if it ever runs
# (I4: the seam is now `_siteverify_post`, not module-level httpx.post).
from app import turnstile as turnstile_mod
def must_not_be_called(*a, **kw):
async def must_not_be_called(*a, **kw):
raise AssertionError("siteverify should not run when no secret is configured")
monkeypatch.setattr(turnstile_mod.httpx, "post", must_not_be_called)
monkeypatch.setattr(turnstile_mod, "_siteverify_post", must_not_be_called)
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
@@ -218,3 +225,27 @@ def test_otc_request_refuses_when_required_but_secret_unset(app_with_fake_gitea,
)
assert r.status_code == 500, r.text
assert _outbound_otc_envelopes("alice@example.com") == []
# ---------------------------------------------------------------------------
# I4 (security-audit-0026): verify_token is a coroutine — calling it returns
# an awaitable, not a VerifyOutcome. Locks the async contract so a revert to
# the synchronous event-loop-blocking shape fails here, not just in the
# integration paths.
# ---------------------------------------------------------------------------
def test_verify_token_is_async_and_soft_skips_without_secret(monkeypatch):
import asyncio
from app import turnstile as turnstile_mod
monkeypatch.delenv("CLOUDFLARE_TURNSTILE_SECRET", raising=False)
monkeypatch.delenv("TURNSTILE_REQUIRED", raising=False)
coro = turnstile_mod.verify_token("any-token")
assert asyncio.iscoroutine(coro), "verify_token must be a coroutine (I4)"
outcome = asyncio.run(coro)
# No secret + not required → the gate stays open without any network call.
assert outcome.ok is True
assert outcome.reason == "skipped"
+205
View File
@@ -0,0 +1,205 @@
"""End-to-end integration tests for the Gitea webhook receiver
(v0.18.0 Slice 3 webhook tightening per the email + webhook
hygiene proposal).
The release changes the receiver from "verifies the signature only
when a secret is configured; silently accepts unsigned POSTs
otherwise" to "requires the secret unless `RFC_APP_INSECURE_WEBHOOKS=1`
is set as an explicit dev-bypass." The startup-time check lives in
`config.load_config()`; the request-time check lives in
`webhooks.receive`.
These tests prove:
* The framework refuses to start when `GITEA_WEBHOOK_SECRET` is
empty and the dev-bypass is not set.
* The dev-bypass (`RFC_APP_INSECURE_WEBHOOKS=1`) lets the
framework boot with an empty secret AND lets webhook POSTs
land without signature verification (a loud-warning log line
surfaces, but the request is accepted).
* Default path (secret bound): a POST with a valid signature
lands; a POST with an invalid signature gets 401; a POST with
no signature gets 401.
* Unknown-repo POSTs surface in the log (the "stale Gitea hook"
case the proposal targets).
"""
from __future__ import annotations
import hashlib
import hmac
import json
import logging
import pytest
from test_propose_vertical import ( # noqa: F401
FakeGitea,
app_with_fake_gitea,
tmp_env,
)
# ---------------------------------------------------------------------------
# Startup-time secret check (config.load_config)
# ---------------------------------------------------------------------------
def test_config_refuses_to_load_with_empty_secret_and_no_bypass(monkeypatch, tmp_path):
"""The framework MUST refuse to start when `GITEA_WEBHOOK_SECRET`
is empty unless `RFC_APP_INSECURE_WEBHOOKS=1` is set. This is
the v0.18.0 startup-loud-failure shape silent acceptance was
the bug."""
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "")
monkeypatch.delenv("RFC_APP_INSECURE_WEBHOOKS", raising=False)
from app.config import load_config
with pytest.raises(RuntimeError, match="GITEA_WEBHOOK_SECRET"):
load_config()
def test_config_loads_with_empty_secret_when_bypass_is_set(monkeypatch, tmp_path):
"""The explicit `RFC_APP_INSECURE_WEBHOOKS=1` opt-in lets the
framework boot with an empty webhook secret. This is the
local-dev escape hatch."""
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "")
monkeypatch.setenv("RFC_APP_INSECURE_WEBHOOKS", "1")
from app.config import load_config
cfg = load_config() # MUST NOT raise
assert cfg.webhook_secret == ""
def test_config_loads_with_secret_set(monkeypatch, tmp_path):
"""Sanity: the happy path (secret bound, bypass not set) loads
cleanly."""
monkeypatch.setenv("GITEA_URL", "http://gitea.test")
monkeypatch.setenv("GITEA_BOT_USER", "rfc-bot")
monkeypatch.setenv("GITEA_BOT_TOKEN", "bot-token")
monkeypatch.setenv("GITEA_ORG", "wiggleverse")
monkeypatch.setenv("OAUTH_CLIENT_ID", "cid")
monkeypatch.setenv("OAUTH_CLIENT_SECRET", "csec")
monkeypatch.setenv("SECRET_KEY", "test-secret-key")
monkeypatch.setenv("DATABASE_PATH", str(tmp_path / "test.db"))
monkeypatch.setenv("GITEA_WEBHOOK_SECRET", "my-real-secret")
monkeypatch.delenv("RFC_APP_INSECURE_WEBHOOKS", raising=False)
from app.config import load_config
cfg = load_config()
assert cfg.webhook_secret == "my-real-secret"
# ---------------------------------------------------------------------------
# Request-time signature verification (webhooks.receive)
#
# The default `app_with_fake_gitea` fixture binds
# `GITEA_WEBHOOK_SECRET=test-webhook-secret-for-signature-verification`,
# so these tests exercise the production path.
# ---------------------------------------------------------------------------
_SECRET = "test-webhook-secret-for-signature-verification"
def _sign(body: bytes) -> str:
return hmac.new(_SECRET.encode("utf-8"), body, hashlib.sha256).hexdigest()
def test_webhook_post_with_valid_signature_accepted(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
sig = _sign(body)
r = client.post(
"/api/webhooks/gitea",
content=body,
headers={
"X-Gitea-Event": "push",
"X-Gitea-Signature": sig,
"Content-Type": "application/json",
},
)
assert r.status_code == 200, r.text
def test_webhook_post_with_invalid_signature_refused_401(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
r = client.post(
"/api/webhooks/gitea",
content=body,
headers={
"X-Gitea-Event": "push",
"X-Gitea-Signature": "0" * 64, # wrong signature
"Content-Type": "application/json",
},
)
assert r.status_code == 401
def test_webhook_post_with_missing_signature_refused_401(app_with_fake_gitea):
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
body = json.dumps({"repository": {"full_name": "wiggleverse/meta"}}).encode()
r = client.post(
"/api/webhooks/gitea",
content=body,
headers={
"X-Gitea-Event": "push",
"Content-Type": "application/json",
},
)
assert r.status_code == 401
# ---------------------------------------------------------------------------
# Unknown-repo logging (the "stale hook on a fork" surface)
# ---------------------------------------------------------------------------
def test_webhook_unknown_repo_logs_at_info(app_with_fake_gitea, caplog):
"""Per the proposal: a hook on a fork or a stale Gitea binding
used to silently 200-OK. v0.18.0 surfaces it as an INFO log."""
from fastapi.testclient import TestClient
app, _fake = app_with_fake_gitea
with TestClient(app) as client:
body = json.dumps({"repository": {"full_name": "someone-else/unrelated"}}).encode()
sig = _sign(body)
with caplog.at_level(logging.INFO, logger="app.webhooks"):
r = client.post(
"/api/webhooks/gitea",
content=body,
headers={
"X-Gitea-Event": "push",
"X-Gitea-Signature": sig,
"Content-Type": "application/json",
},
)
assert r.status_code == 200 # the handler still 200s; surface is the log line
assert any(
"unknown repo" in rec.message and "someone-else/unrelated" in rec.message
for rec in caplog.records
), f"expected unknown-repo log line; got: {[r.message for r in caplog.records]}"
+50
View File
@@ -18,6 +18,56 @@ server {
listen [::]:80;
server_name ohm.wiggleverse.org;
# v0.25.0 security hardening (audit 0026 M2/L8)
#
# NOTE: certbot promotes THIS server block to the HTTPS listener
# (`listen 443 ssl`) and adds a separate port-80 → 443 redirect
# block (see the install comment above). These response headers
# therefore ride into the HTTPS server block on the VM. They use
# `add_header ... always` so they also apply to nginx-generated
# error responses (4xx/5xx), not just 200s.
#
# `server_tokens off` (L8) — suppress the nginx version in the
# Server header and on error pages so we don't advertise the
# build to scanners.
server_tokens off;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Frame-Options "DENY" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# Content-Security-Policy (M2). Tuned to what the SPA actually loads:
# - default-src 'self': everything not called out below is same-origin.
# - script-src 'self' + challenges.cloudflare.com: the only external
# <script> tag the app injects is the CloudFlare Turnstile widget
# (frontend/src/components/TurnstileWidget.jsx). Amplitude and
# mermaid are BUNDLED (dynamic `import()` from node_modules, served
# from 'self'), so they need no extra script origin — *.amplitude.com
# is listed defensively in case a future SDK build script-injects.
# script-src DELIBERATELY OMITS 'unsafe-inline' — no inline <script>
# is used, so we keep XSS-via-inline-script blocked.
# - style-src 'unsafe-inline' IS REQUIRED by the current build: the
# JSX uses inline `style={...}` attributes throughout and mermaid
# injects <style> blocks at render time. Removing it would break
# layout; tightening this is a future build-side change (nonce/hash).
# - img-src 'self' data: https: — markdown/RFC bodies may embed remote
# images and data: URIs; svg/mermaid output uses data: too.
# - font-src 'self' data: — bundled fonts plus data: webfonts.
# - connect-src 'self' + *.amplitude.com + challenges.cloudflare.com:
# the app's API/auth/SSE are same-origin (nginx proxy); Amplitude
# Analytics + Session Replay (shipped at sampleRate 1) POST to
# *.amplitude.com; Turnstile verifies via challenges.cloudflare.com.
# - worker-src 'self' blob: — Amplitude Session Replay spins up a
# Web Worker from a blob: URL for capture/compression; without
# blob: here session replay breaks for every consenting user.
# - frame-src challenges.cloudflare.com — the Turnstile challenge
# renders in an iframe from that origin.
# - frame-ancestors 'none' — clickjacking defense, pairs with
# X-Frame-Options DENY for older agents.
# - base-uri 'self'; object-src 'none' — lock down <base>/<object>.
add_header Content-Security-Policy "default-src 'self'; script-src 'self' https://challenges.cloudflare.com https://*.amplitude.com; style-src 'self' 'unsafe-inline'; img-src 'self' data: https:; font-src 'self' data:; connect-src 'self' https://*.amplitude.com https://challenges.cloudflare.com; worker-src 'self' blob:; frame-src https://challenges.cloudflare.com; frame-ancestors 'none'; base-uri 'self'; object-src 'none'" always;
# Static SPA assets live in the Vite build output. The systemd unit
# runs as user `rfc-app`; make sure nginx (usually `www-data`) can
# read this path. Either group-add www-data into rfc-app's group, or
+27
View File
@@ -42,5 +42,32 @@ ProtectHome=true
PrivateTmp=true
ReadWritePaths=/opt/rfc-app/backend/data
# v0.25.0 security hardening (audit 0026 L4) — defense-in-depth.
# The service binds 127.0.0.1:8000 and runs plain CPython
# (FastAPI/uvicorn + sqlite + bcrypt + httpx), so it needs no
# capabilities and no exotic syscalls.
CapabilityBoundingSet=
AmbientCapabilities=
PrivateDevices=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
RestrictNamespaces=true
LockPersonality=true
# MemoryDenyWriteExecute=true blocks W^X memory — safe for stock
# CPython (no JIT) and the pure-Python/C-extension stack here, but
# would break a JIT or a C-ext that mmaps W+X. Watch the first
# restart's journal for a crash; if uvicorn fails to come up,
# comment this one line out and reload.
MemoryDenyWriteExecute=true
RestrictRealtime=true
RestrictSUIDSGID=true
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
SystemCallArchitectures=native
UMask=0077
[Install]
WantedBy=multi-user.target
+3 -2
View File
@@ -1,12 +1,12 @@
{
"name": "rfc-app-frontend",
"version": "0.15.0",
"version": "0.24.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "rfc-app-frontend",
"version": "0.15.0",
"version": "0.24.0",
"dependencies": {
"@amplitude/unified": "^1.1.9",
"@codemirror/commands": "^6.10.3",
@@ -18,6 +18,7 @@
"@tiptap/pm": "^3.5.0",
"@tiptap/react": "^3.5.0",
"@tiptap/starter-kit": "^3.5.0",
"dompurify": "^3.2.4",
"marked": "^18.0.4",
"mermaid": "^11.15.0",
"react": "^19.2.6",
+2 -1
View File
@@ -1,7 +1,7 @@
{
"name": "rfc-app-frontend",
"private": true,
"version": "0.17.0",
"version": "0.28.0",
"type": "module",
"scripts": {
"dev": "vite",
@@ -19,6 +19,7 @@
"@tiptap/pm": "^3.5.0",
"@tiptap/react": "^3.5.0",
"@tiptap/starter-kit": "^3.5.0",
"dompurify": "^3.2.4",
"marked": "^18.0.4",
"mermaid": "^11.15.0",
"react": "^19.2.6",
+990 -685
View File
File diff suppressed because it is too large Load Diff
+77 -7
View File
@@ -1,7 +1,8 @@
import { useEffect, useRef, useState } from 'react'
import { Routes, Route, Link, useLocation, useNavigate } from 'react-router-dom'
import { Routes, Route, Link, Navigate, useLocation, useNavigate } from 'react-router-dom'
import { getMe, subscribeToNotifications } from './api'
import { anonymize, EVENTS, identify, track } from './lib/analytics'
import { useLastState } from './lib/useLastState'
import Catalog from './components/Catalog.jsx'
import Inbox from './components/Inbox.jsx'
import RFCView from './components/RFCView.jsx'
@@ -12,7 +13,13 @@ import Landing from './components/Landing.jsx'
import Login from './components/Login.jsx'
import BetaPending from './components/BetaPending.jsx'
import Philosophy from './components/Philosophy.jsx'
import Docs from './components/Docs.jsx'
import DocsLayout from './components/DocsLayout.jsx'
import DocsUserGuide from './components/DocsUserGuide.jsx'
import DocsSessionsAbout from './components/DocsSessionsAbout.jsx'
import DocsSessionIndex from './components/DocsSessionIndex.jsx'
import DocsSessionTranscript from './components/DocsSessionTranscript.jsx'
import DocsSpec from './components/DocsSpec.jsx'
import DocsSpecsIndex from './components/DocsSpecsIndex.jsx'
import NotificationSettings from './components/NotificationSettings.jsx'
import Admin from './components/Admin.jsx'
import AcceptInvitation from './components/AcceptInvitation.jsx'
@@ -36,6 +43,12 @@ export default function App() {
// "Privacy & cookies" tab dispatches a `rfc-app:cookie-consent-reopen`
// event that bumps this.
const [consentReopenTick, setConsentReopenTick] = useState(0)
// v0.23.0 / item #29 flips true once the #21-Part-C identify effect
// has fired (or once we've confirmed there's no authenticated user to
// identify). useLastState gates its resume redirect on this so the
// redirect always happens AFTER identify, preserving identify-then-
// track ordering.
const [identifyReady, setIdentifyReady] = useState(false)
const navigate = useNavigate()
const location = useLocation()
// v0.15.0 Page Viewed event taxonomy. We fire on every
@@ -80,6 +93,9 @@ export default function App() {
if (viewer?.first_sign_in_at) props.first_sign_in_at = ['__setOnce__', viewer.first_sign_in_at]
if (viewer?.created_at) props.account_created_at = ['__setOnce__', viewer.created_at]
identify({ user_id: String(uid), properties: props })
// v0.23.0 / item #29 identify has now fired for this sign-in;
// release useLastState's resume redirect (it waits on this).
setIdentifyReady(true)
} else if (uid == null && lastUserIdRef.current != null) {
// Sign-out edge App-level reset is handled separately by the
// sign-out gesture that fires User Signed Out. Clear our local
@@ -88,6 +104,14 @@ export default function App() {
}
}, [me?.authenticated, me?.user?.id, me?.user?.role, me?.user?.permission_state, me?.user?.passcode_set, me?.user?.device_trusted])
// v0.23.0 / item #29 once `me` has resolved, if there's no
// authenticated user there is nothing to identify, so release the
// resume gate immediately (anonymous boots have no resume to do, but
// the hook still needs the gate resolved to be a clean no-op).
useEffect(() => {
if (me != null && !me.authenticated) setIdentifyReady(true)
}, [me])
useEffect(() => {
const handler = () => setConsentReopenTick(t => t + 1)
window.addEventListener('rfc-app:cookie-consent-reopen', handler)
@@ -101,6 +125,18 @@ export default function App() {
.finally(() => setLoading(false))
}, [])
// v0.23.0 / item #29 server-side sign-in state resume. The hook
// debounce-posts the current route for authenticated users and, once
// identify has fired, redirects a fresh sign-in (which hard-lands on
// "/") to the user's stored last route. Anonymous users: no-op.
useLastState({
authenticated: !!me?.authenticated,
pathname: location.pathname,
identifyReady,
lastRoute: me?.authenticated ? me.user?.last_route : null,
navigate,
})
// §15.3 subscribe to the live SSE stream for authenticated viewers
// so the badge counter and the toast surface stay in lockstep with
// the inbox. Tabs that miss an event because they were closed pick
@@ -163,7 +199,7 @@ export default function App() {
wonders why a conversation is public can reach the answer
in two clicks. Anonymous viewers see it too. */}
<Link to="/philosophy" className="header-about" title="Why this exists (§14)">
About
Philosophy
</Link>
<Link to="/docs" className="header-about" title="User guide">
Docs
@@ -182,9 +218,17 @@ export default function App() {
<button
className="inbox-trigger"
onClick={() => setInboxOpen(o => !o)}
title="Notifications inbox (§15.2)"
aria-label="Inbox"
title="Inbox (§15.2)"
>
<span aria-hidden>📮</span>
<svg
width="18" height="18" viewBox="0 0 24 24"
fill="none" stroke="currentColor" strokeWidth="1.75"
strokeLinecap="round" strokeLinejoin="round" aria-hidden
>
<path d="M4 5h16a1 1 0 0 1 1 1v12a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1V6a1 1 0 0 1 1-1Z" />
<path d="m3.5 6.5 8.5 6 8.5-6" />
</svg>
{unreadCount > 0 && (
<span className="badge">{unreadCount > 99 ? '99+' : unreadCount}</span>
)}
@@ -233,7 +277,13 @@ export default function App() {
itself establishes the session on success. */}
<Route path="/invites/claim" element={<InviteClaim />} />
<Route path="/philosophy" element={<PhilosophyWithSidebar viewer={viewer} />} />
<Route path="/docs" element={<DocsWithSidebar viewer={viewer} />} />
{/* v0.19.0 / roadmap item #30 /docs/* is a hub with sub-nav.
The bare /docs path redirects to the user guide; sessions
browser lives at /docs/sessions/*. See DocsLayout.jsx
for the flyout shape and CHANGELOG v0.19.0 for the
upgrade path. */}
<Route path="/docs" element={<Navigate to="/docs/user-guide" replace />} />
<Route path="/docs/*" element={<DocsWithSidebar viewer={viewer} />} />
{/* §14.5 / §14.6: cookie-consent companions to /philosophy.
Available to anonymous and authenticated viewers alike. */}
<Route path="/privacy" element={<PolicyShell><Privacy /></PolicyShell>} />
@@ -303,9 +353,29 @@ function PhilosophyWithSidebar({ viewer }) {
}
function DocsWithSidebar({ viewer }) {
// v0.19.0 / roadmap item #30 the `/docs/*` surface is a flyout
// shell with sub-routes. The shell (sidebar + content area) is the
// DocsLayout outlet host; the sub-routes mount their respective
// pages into the outlet. Bare `/docs/sessions` redirects to the
// sessions about page so deep-linkers and the flyout's "Sessions"
// header both land somewhere coherent.
return (
<main className="chrome-pane">
<Docs authenticated={!!viewer} />
<Routes>
<Route element={<DocsLayout authenticated={!!viewer} />}>
<Route index element={<Navigate to="user-guide" replace />} />
<Route path="user-guide" element={<DocsUserGuide />} />
<Route path="sessions" element={<Navigate to="about" replace />} />
<Route path="sessions/about" element={<DocsSessionsAbout />} />
<Route path="sessions/:nnnn" element={<DocsSessionIndex />} />
<Route path="sessions/:nnnn/:filename" element={<DocsSessionTranscript />} />
{/* v0.20.0 /docs/specs/* surface (framework spec + flotilla spec
at runtime via gitea raw). Bare /docs/specs lands on the
client-side redirect to the first configured spec. */}
<Route path="specs" element={<DocsSpecsIndex />} />
<Route path="specs/:name" element={<DocsSpec />} />
</Route>
</Routes>
</main>
)
}
+135 -4
View File
@@ -25,6 +25,25 @@ export async function getMe() {
return jsonOrThrow(res)
}
// ── v0.23.0: sign-in state resume (§6.2, roadmap item #29) ───────────────
//
// The route-change hook (useLastState) debounce-posts the user's current
// route + a small bag of *light* view state here for authenticated users.
// The next sign-in reads `last_route` off `/api/auth/me` and redirects.
// Privacy: `state` carries ephemeral view state ONLY — never draft-buffer
// contents (see SPEC §6.2). Best-effort: callers ignore failures (an
// offline/401 POST must never disrupt navigation).
export async function putLastState(route, state) {
const body = { route }
if (state != null) body.state = state
const res = await fetch('/api/me/last-state', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
return jsonOrThrow(res)
}
// ── v0.7.0: email + one-time-code sign-in (§6.2) ─────────────────────────
//
// The legacy /auth/login → /auth/callback OAuth flow remains during the
@@ -166,15 +185,47 @@ export async function getProposal(prNumber) {
return jsonOrThrow(await fetch(`/api/proposals/${prNumber}`))
}
export async function proposeRFC({ title, slug, pitch, tags }) {
export async function proposeRFC({ title, slug, pitch, tags, proposedUseCase }) {
const res = await fetch('/api/rfcs/propose', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title, slug, pitch, tags: tags || [] }),
// #26: proposed_use_case is optional; send null when blank so the
// backend treats it as "left blank".
body: JSON.stringify({
title,
slug,
pitch,
tags: tags || [],
proposed_use_case: proposedUseCase || null,
}),
})
return jsonOrThrow(res)
}
// Roadmap #27: Claude Haiku tag suggestions for the propose-RFC modal.
// Returns a (possibly empty) array of { tag, confidence }. Deliberately
// forgiving — any non-OK response (rate limit, transient error, no key
// configured server-side) resolves to [] so the modal just shows nothing
// rather than surfacing an error for what is a best-effort assist.
export async function suggestTags({ title, pitch, useCase }) {
try {
const res = await fetch('/api/rfcs/suggest-tags', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
title: title || '',
pitch: pitch || '',
use_case: useCase || '',
}),
})
if (!res.ok) return []
const data = await res.json()
return Array.isArray(data.suggestions) ? data.suggestions : []
} catch {
return []
}
}
export async function mergeProposal(prNumber) {
const res = await fetch(`/api/proposals/${prNumber}/merge`, { method: 'POST' })
return jsonOrThrow(res)
@@ -492,13 +543,14 @@ export async function draftPRText(slug, branch) {
return jsonOrThrow(res)
}
export async function openPR(slug, branch, { title, description }) {
export async function openPR(slug, branch, { title, description, proposedUseCase }) {
const res = await fetch(
`/api/rfcs/${slug}/branches/${encodeURIComponent(branch)}/open-pr`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title, description }),
// #26: proposed_use_case is optional; null when blank.
body: JSON.stringify({ title, description, proposed_use_case: proposedUseCase || null }),
},
)
return jsonOrThrow(res)
@@ -720,6 +772,85 @@ export async function getDocs() {
return jsonOrThrow(await fetch('/api/docs'))
}
// ---------------------------------------------------------------------------
// v0.19.0 / roadmap item #30 — /api/docs/sessions/* surface
// ---------------------------------------------------------------------------
//
// The framework mediates reads against the public
// `wiggleverse/ohm-session-history` gitea repo so the rendered
// `/docs/sessions/*` surface inherits the same chrome as
// `/docs/user-guide`. Three text-bearing endpoints return markdown
// (Content-Type: text/markdown) and the manifest returns JSON. We
// wrap each into a small helper.
//
// 404 from `getSessionAbout` / `getSessionTranscript` / `getSessionIndex`
// throws an Error with `.status === 404` so the UI can render its own
// empty-state. 502 (gitea unreachable) throws `.status === 502` so
// the UI can offer a retry button.
export async function getSessionsManifest() {
// Manifest 404 is mapped server-side to HTTP 200 + `{}` so this
// helper never throws on the empty-state path.
return jsonOrThrow(await fetch('/api/docs/sessions/manifest'))
}
async function _textOrThrow(res) {
if (!res.ok) {
let detail = ''
try {
const body = await res.json()
detail = body.detail || JSON.stringify(body)
} catch {
detail = await res.text()
}
const error = new Error(detail || `HTTP ${res.status}`)
error.status = res.status
throw error
}
return res.text()
}
export async function getSessionsAbout() {
return _textOrThrow(await fetch('/api/docs/sessions/about'))
}
export async function getSessionTranscript(nnnn, filename) {
return _textOrThrow(await fetch(
`/api/docs/sessions/${encodeURIComponent(nnnn)}/${encodeURIComponent(filename)}`
))
}
export async function getSessionIndex(nnnn) {
return jsonOrThrow(await fetch(
`/api/docs/sessions/${encodeURIComponent(nnnn)}/index`
))
}
// ---------------------------------------------------------------------------
// v0.20.0 — /api/docs/specs/* surface
// ---------------------------------------------------------------------------
//
// Sibling of the docs-sessions helpers above. The framework mediates
// reads against the configured spec URLs (default: rfc-app's own
// SPEC.md + flotilla's SPEC.md on `git.wiggleverse.org`) so the
// `/docs/specs/*` route inherits the same chrome as `/docs/user-guide`
// and `/docs/sessions/*`. The manifest endpoint always returns 200 +
// {specs: [...]} — a malformed `OHM_DOCS_SPECS` env var falls back to
// the framework default at parse time on the backend.
//
// 404 from `getSpec` throws `.status === 404`; 502 throws `.status === 502`,
// matching the docs-sessions helper convention.
export async function getSpecsManifest() {
return jsonOrThrow(await fetch('/api/docs/specs/manifest'))
}
export async function getSpec(name) {
return _textOrThrow(await fetch(
`/api/docs/specs/${encodeURIComponent(name)}`
))
}
// ---------------------------------------------------------------------------
// Slice 7: admin neighborhood (§17 admin/* + user search for the §15.8 mute
// typeahead).
+128
View File
@@ -0,0 +1,128 @@
/* Docs.css docs-surface polish scoped to v0.21.0 / roadmap item #32.
*
* This sheet owns ONLY the classes introduced by item #32 (the
* transcript metadata header and the session-root sibling list). The
* pre-existing docs classes (.docs-article, .docs-empty, .docs-error,
* .docs-source-link, .philosophy-body, .muted) live in App.css and are
* deliberately NOT touched here redefining them would race the #31
* App.css token sweep for the same selectors. Every value below reads
* a token from tokens.css so the new surfaces sit on the same
* spacing/type/color scale as the rest of the docs chrome.
*
* Imported from DocsSessionTranscript.jsx + DocsSessionIndex.jsx (the
* two components that render these elements). CSS custom properties are
* not import-order-sensitive at use time, so the import site doesn't
* matter for correctness.
*/
/* Transcript metadata header
* A compact card above the rendered transcript body: title, the
* started/ended/duration grid, an optional TL;DR, and the external
* "view source" link. */
.docs-transcript-meta {
margin: 0 0 var(--space-9);
padding: var(--space-7);
border: 1px solid var(--color-border);
border-radius: var(--radius-lg);
background: var(--color-surface-sunken);
}
.docs-transcript-meta-title {
margin: 0 0 var(--space-5);
font-size: var(--text-lg);
font-weight: var(--weight-semibold);
line-height: var(--leading-tight);
color: var(--color-text-strong);
font-family: var(--font-mono);
word-break: break-word;
}
.docs-transcript-meta-grid {
margin: 0;
display: grid;
grid-template-columns: max-content 1fr;
gap: var(--space-2) var(--space-7);
align-items: baseline;
}
.docs-transcript-meta-row {
display: contents;
}
.docs-transcript-meta-grid dt {
margin: 0;
font-size: var(--text-xs);
font-weight: var(--weight-semibold);
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--color-text-muted);
}
.docs-transcript-meta-grid dd {
margin: 0;
font-size: var(--text-base);
color: var(--color-text);
}
.docs-transcript-meta-tldr {
margin: var(--space-6) 0 0;
padding-top: var(--space-6);
border-top: 1px solid var(--color-border);
font-size: var(--text-base);
line-height: var(--leading-relaxed);
color: var(--color-text);
}
.docs-transcript-meta-source {
display: inline-block;
margin-top: var(--space-6);
}
/* Session-root sibling-transcript list
* Rendered above the inlined primary transcript when a session has
* more than one transcript (driver `.0` + subagents). The primary is
* marked "(shown below)"; the rest link to their standalone routes. */
.docs-session-siblings {
margin: 0 0 var(--space-9);
padding: var(--space-6) var(--space-7);
border: 1px solid var(--color-border);
border-radius: var(--radius-lg);
background: var(--color-surface-muted);
display: flex;
flex-wrap: wrap;
align-items: baseline;
gap: var(--space-3) var(--space-6);
}
.docs-session-siblings-label {
font-size: var(--text-xs);
font-weight: var(--weight-semibold);
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--color-text-muted);
}
.docs-session-siblings-list {
list-style: none;
margin: 0;
padding: 0;
display: flex;
flex-wrap: wrap;
gap: var(--space-2) var(--space-5);
font-family: var(--font-mono);
font-size: var(--text-base);
}
.docs-session-siblings-list a {
color: var(--color-link);
text-decoration: none;
}
.docs-session-siblings-list a:hover {
color: var(--color-accent-strong);
text-decoration: underline;
}
.docs-session-siblings-current {
color: var(--color-text-muted);
}
-51
View File
@@ -1,51 +0,0 @@
// `/docs` the user-facing guide.
//
// Sibling of Philosophy.jsx: same chrome, same data path, different
// source file. Renders DOCS.md verbatim with light chrome around it.
// Reachable anonymously, same as `/philosophy`, so a visitor can read
// the guide before deciding to sign in.
import { useEffect, useState } from 'react'
import { Link, useNavigate } from 'react-router-dom'
import MarkdownPreview from './MarkdownPreview.jsx'
import { getDocs } from '../api.js'
export default function Docs({ authenticated }) {
const [body, setBody] = useState('')
const [error, setError] = useState(null)
const [loading, setLoading] = useState(true)
const navigate = useNavigate()
useEffect(() => {
let active = true
getDocs()
.then(r => { if (active) setBody(r.body || '') })
.catch(e => { if (active) setError(e.message || String(e)) })
.finally(() => { if (active) setLoading(false) })
return () => { active = false }
}, [])
return (
<div className="philosophy-page">
<header className="philosophy-header">
<button
className="philosophy-back"
onClick={() => (history.length > 1 ? navigate(-1) : navigate('/'))}
>
Back
</button>
<span className="philosophy-title">User guide</span>
{!authenticated && (
<Link className="philosophy-signin" to="/">Home</Link>
)}
</header>
<article className="philosophy-body">
{loading && <p className="muted">Loading</p>}
{error && <p className="error">Could not load the guide: {error}</p>}
{!loading && !error && (
<MarkdownPreview content={body} />
)}
</article>
</div>
)
}
+340
View File
@@ -0,0 +1,340 @@
// DocsLayout.jsx v0.20.0 (was v0.19.0 / roadmap item #30).
//
// Left-side flyout nav + content area for the `/docs/*` route tree:
//
// /docs redirect to /docs/user-guide
// /docs/user-guide DOCS.md (existing v0.14.0 content)
// /docs/specs client-side redirect to first configured spec
// /docs/specs/:name a single framework spec (v0.20.0)
// /docs/sessions redirect to /docs/sessions/about
// /docs/sessions/about README.md from the sessions repo
// /docs/sessions/:nnnn per-session overview (nav-only navigation)
// /docs/sessions/:nnnn/:file per-transcript view
//
// v0.20.0 changes (Session 0018.0):
// - Adds a "Specs" section between User Guide and Sessions, driven
// by `/api/docs/specs/manifest`.
// - Sessions render a nested tree: each session row has the
// session's transcripts nested under it as their own nav rows
// (labeled by `.N` ordinal). The transcript list is fetched per
// session via `/api/docs/sessions/:nnnn/index` (cached server-
// side, so the manifest+index fan-out is cheap on subsequent
// loads). Always-expanded; no collapse toggle (current scale is
// under twenty sessions well under the threshold where lazy
// expansion would pay).
//
// The flyout is a persistent left sidebar on desktop and a slide-out
// drawer on mobile (toggled by the icon button in the docs header).
//
// Amplitude analytics (per SPEC §21):
// - track('Doc Viewed', { section: '...' }) on each sub-route mount;
// the sub-route component owns the fire.
// - flyout buttons + links carry `aria-label` + `data-amp-track-name`
// so autocapture rows are readable rather than ":nth-child(7)".
import { useEffect, useState, useCallback } from 'react'
import { Link, useNavigate, useLocation, Outlet } from 'react-router-dom'
import { getSessionsManifest, getSessionIndex, getSpecsManifest } from '../api.js'
// Extract the `.N` ordinal from a transcript filename:
// "SESSION-0014.1-TRANSCRIPT-...md" "0014.1"
// "SESSION-0013.1.1-TRANSCRIPT-...md" "0013.1.1" (nested subagent)
// Returns the bare filename as fallback if the expected shape isn't
// matched (which shouldn't happen the backend index endpoint
// filters by the same regex).
function transcriptOrdinal(filename) {
const m = /^SESSION-(\d{4}\.\d+(?:\.\d+)*)-TRANSCRIPT/.exec(filename)
return m ? m[1] : filename
}
export default function DocsLayout({ authenticated }) {
const [manifest, setManifest] = useState(null)
const [manifestState, setManifestState] = useState('loading') // loading | ok | error
const [sessionFiles, setSessionFiles] = useState({}) // { nnnn: [filename, ...] }
const [specs, setSpecs] = useState([])
const [specsState, setSpecsState] = useState('loading') // loading | ok | error
const [drawerOpen, setDrawerOpen] = useState(false)
const [reloadTick, setReloadTick] = useState(0)
const navigate = useNavigate()
const location = useLocation()
// Manifest fetch drives the Sessions section.
useEffect(() => {
let active = true
setManifestState('loading')
getSessionsManifest()
.then(payload => {
if (!active) return
setManifest(payload || {})
setManifestState('ok')
})
.catch(() => {
if (!active) return
setManifest({})
setManifestState('error')
})
return () => { active = false }
}, [reloadTick])
// Per-session transcript lists fan out from the manifest. Always-
// expanded means we pre-fetch every session's index alongside the
// manifest, gated on the manifest having loaded successfully. The
// backend's 5-minute content TTL makes the repeat cost negligible.
useEffect(() => {
if (manifestState !== 'ok' || !manifest) return
let active = true
const nnnnList = Object.keys(manifest).sort()
Promise.all(
nnnnList.map(nnnn =>
getSessionIndex(nnnn)
.then(payload => [nnnn, (payload && payload.files) || []])
.catch(() => [nnnn, []])
)
).then(pairs => {
if (!active) return
setSessionFiles(Object.fromEntries(pairs))
})
return () => { active = false }
}, [manifest, manifestState])
// Specs fetch drives the Specs section. Independent of sessions.
useEffect(() => {
let active = true
setSpecsState('loading')
getSpecsManifest()
.then(payload => {
if (!active) return
setSpecs((payload && payload.specs) || [])
setSpecsState('ok')
})
.catch(() => {
if (!active) return
setSpecs([])
setSpecsState('error')
})
return () => { active = false }
}, [reloadTick])
// Close the mobile drawer on every navigation so a click in the nav
// doesn't strand the user on a drawer-open view.
useEffect(() => {
setDrawerOpen(false)
}, [location.pathname])
const retryManifest = useCallback(() => {
setReloadTick(t => t + 1)
}, [])
return (
<div className="docs-layout">
<header className="docs-header">
<button
className="docs-back"
onClick={() => (history.length > 1 ? navigate(-1) : navigate('/'))}
aria-label="Back to previous page"
data-amp-track-name="Docs Back"
>
Back
</button>
<button
className="docs-drawer-toggle"
onClick={() => setDrawerOpen(o => !o)}
aria-label="Toggle docs navigation"
aria-expanded={drawerOpen}
data-amp-track-name="Docs Drawer Toggle"
>
<span aria-hidden></span>
</button>
<span className="docs-title">Docs</span>
{!authenticated && (
<Link
className="docs-signin"
to="/"
aria-label="Home"
data-amp-track-name="Docs Home"
>
Home
</Link>
)}
</header>
<div className={'docs-body' + (drawerOpen ? ' docs-body--drawer-open' : '')}>
<aside className="docs-nav" aria-label="Docs navigation">
<DocsNav
manifest={manifest}
manifestState={manifestState}
sessionFiles={sessionFiles}
specs={specs}
specsState={specsState}
onRetry={retryManifest}
currentPath={location.pathname}
/>
</aside>
<main className="docs-content">
<Outlet />
</main>
</div>
{drawerOpen && (
<button
className="docs-drawer-scrim"
aria-label="Close drawer"
onClick={() => setDrawerOpen(false)}
data-amp-track-name="Docs Drawer Close"
/>
)}
</div>
)
}
function DocsNav({
manifest,
manifestState,
sessionFiles,
specs,
specsState,
onRetry,
currentPath,
}) {
const isActive = (path) => currentPath === path || currentPath.startsWith(path + '/')
const isExactly = (path) => currentPath === path
const sessionKeys = Object.keys(manifest || {}).sort()
return (
<nav className="docs-nav-inner">
<div className="docs-nav-section">
<div className="docs-nav-section-label">Docs</div>
<ul className="docs-nav-list">
<li>
<Link
to="/docs/user-guide"
className={isActive('/docs/user-guide') ? 'active' : ''}
aria-label="User Guide"
data-amp-track-name="Docs Nav User Guide"
>
User Guide
</Link>
</li>
</ul>
</div>
<div className="docs-nav-section">
<div className="docs-nav-section-label">Specs</div>
{specsState === 'loading' && (
<ul className="docs-nav-list docs-nav-skeleton" aria-hidden>
<li><span className="skeleton-row" /></li>
<li><span className="skeleton-row" /></li>
</ul>
)}
{specsState === 'error' && (
<div className="docs-nav-error" role="alert">
<span>Couldn't load specs.</span>
</div>
)}
{specsState === 'ok' && specs.length > 0 && (
<ul className="docs-nav-list">
{specs.map(spec => {
const to = `/docs/specs/${spec.name}`
return (
<li key={spec.name}>
<Link
to={to}
className={isExactly(to) ? 'active' : ''}
aria-label={`Spec: ${spec.title}`}
data-amp-track-name="Docs Nav Spec"
data-amp-track-spec={spec.name}
>
{spec.title}
</Link>
</li>
)
})}
</ul>
)}
</div>
<div className="docs-nav-section">
<div className="docs-nav-section-label">Sessions</div>
<ul className="docs-nav-list">
<li>
<Link
to="/docs/sessions/about"
className={isExactly('/docs/sessions/about') ? 'active' : ''}
aria-label="About sessions"
data-amp-track-name="Docs Nav Sessions About"
>
About
</Link>
</li>
</ul>
{manifestState === 'loading' && (
<ul className="docs-nav-list docs-nav-skeleton" aria-hidden>
<li><span className="skeleton-row" /></li>
<li><span className="skeleton-row" /></li>
<li><span className="skeleton-row" /></li>
</ul>
)}
{manifestState === 'error' && (
<div className="docs-nav-error" role="alert">
<span>Couldn't load session list.</span>
<button
type="button"
onClick={onRetry}
aria-label="Retry session list"
data-amp-track-name="Docs Nav Sessions Retry"
>
Try again
</button>
</div>
)}
{manifestState === 'ok' && sessionKeys.length > 0 && (
<ul className="docs-nav-list docs-nav-list--tree">
{sessionKeys.map(nnnn => {
const entry = manifest[nnnn] || {}
const title = entry.title || ''
const label = title ? `${nnnn}${title}` : nnnn
const to = `/docs/sessions/${nnnn}`
const files = sessionFiles[nnnn] || []
return (
<li key={nnnn}>
<Link
to={to}
className={isExactly(to) ? 'active' : ''}
aria-label={`Session ${nnnn}${title ? ': ' + title : ''}`}
data-amp-track-name="Docs Nav Session"
data-amp-track-session={nnnn}
>
{label}
</Link>
{files.length > 0 && (
<ul className="docs-nav-list docs-nav-list--children">
{files.map(f => {
const tTo = `/docs/sessions/${nnnn}/${f}`
return (
<li key={f}>
<Link
to={tTo}
className={isExactly(tTo) ? 'active' : ''}
aria-label={`Transcript ${transcriptOrdinal(f)}`}
data-amp-track-name="Docs Nav Transcript"
data-amp-track-session={nnnn}
data-amp-track-filename={f}
>
{transcriptOrdinal(f)}
</Link>
</li>
)
})}
</ul>
)}
</li>
)
})}
</ul>
)}
</div>
</nav>
)
}
@@ -0,0 +1,246 @@
// DocsSessionIndex.jsx v0.21.0 (was v0.20.0 / roadmap item #30).
//
// Per-session landing at `/docs/sessions/:nnnn`. v0.20.0 rendered a
// dead-end "N transcript(s) in this session. Select one from the
// navigation." placeholder. v0.21.0 / roadmap item #32 collapses that:
// the session root now renders a transcript INLINE so the URL is never
// an empty stop.
//
// - Exactly one transcript render it inline at the session root.
// - Multiple transcripts render the `.0` driver transcript
// inline (fall back to the first file by
// sort order if there's no `.0`), AND
// list/link the remaining transcripts so
// the siblings are one click away.
//
// The URL stays stable to the session number this is an inline
// render, not a 301/redirect. The per-transcript route
// (`/docs/sessions/:nnnn/:filename`) still exists and is what the
// sibling links and the left-nav transcript rows point at.
//
// The transcript count + filenames come from `/api/docs/sessions/:nnnn/index`
// so the empty-state ("no transcripts yet"), not-found, and error
// paths remain meaningful when the upstream is mid-publish or
// unreachable. The metadata header + body rendering are imported from
// DocsSessionTranscript.jsx so the inline view is byte-identical to the
// standalone per-transcript view.
import { useEffect, useState, useCallback } from 'react'
import { Link, useParams } from 'react-router-dom'
import MarkdownPreview from './MarkdownPreview.jsx'
import {
getSessionsManifest,
getSessionIndex,
getSessionTranscript,
} from '../api.js'
import {
TranscriptMetaHeader,
transcriptOrdinal,
} from './DocsSessionTranscript.jsx'
import { EVENTS, track } from '../lib/analytics'
import './Docs.css'
// Pick the transcript to render inline at the session root: prefer the
// `.0` driver transcript; otherwise the first file by sort order. The
// backend already returns the file list sorted, so `files[0]` is a
// stable fallback.
function pickPrimary(files) {
if (!files || files.length === 0) return null
const driver = files.find(f => /^SESSION-\d{4}\.0-TRANSCRIPT/.test(f))
return driver || files[0]
}
export default function DocsSessionIndex() {
const { nnnn } = useParams()
const [title, setTitle] = useState('')
const [tldr, setTldr] = useState('')
const [files, setFiles] = useState([])
const [status, setStatus] = useState('loading') // loading | ok | notfound | error
const [reloadTick, setReloadTick] = useState(0)
// The inline body for the primary transcript.
const [body, setBody] = useState('')
const [bodyStatus, setBodyStatus] = useState('idle') // idle | loading | ok | notfound | error
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: `sessions/${nnnn}` })
}, [nnnn])
// Title + optional TL;DR from the manifest cheap, cached server-side.
useEffect(() => {
let active = true
getSessionsManifest()
.then(payload => {
if (!active) return
const entry = (payload && payload[nnnn]) || {}
setTitle(entry.title || '')
// `tldr` is an optional manifest field (string). Absent the
// header renders no TL;DR line (graceful degrade).
setTldr(typeof entry.tldr === 'string' ? entry.tldr : '')
})
.catch(() => {
// Title + TL;DR are decorative; the transcript list + body
// fetches below are the load-bearing ones.
})
return () => { active = false }
}, [nnnn])
// File list from the per-session index endpoint.
useEffect(() => {
let active = true
setStatus('loading')
getSessionIndex(nnnn)
.then(payload => {
if (!active) return
setFiles((payload && payload.files) || [])
setStatus('ok')
})
.catch(e => {
if (!active) return
if (e.status === 404) {
setStatus('notfound')
} else {
setStatus('error')
}
})
return () => { active = false }
}, [nnnn, reloadTick])
const primary = status === 'ok' ? pickPrimary(files) : null
// Fetch the primary transcript body once we know which file it is.
useEffect(() => {
if (!primary) {
setBody('')
setBodyStatus('idle')
return
}
let active = true
setBodyStatus('loading')
getSessionTranscript(nnnn, primary)
.then(text => {
if (!active) return
setBody(text || '')
setBodyStatus('ok')
})
.catch(e => {
if (!active) return
setBodyStatus(e.status === 404 ? 'notfound' : 'error')
})
return () => { active = false }
}, [nnnn, primary, reloadTick])
const retry = useCallback(() => setReloadTick(t => t + 1), [])
const header = title ? `${nnnn}${title}` : `Session ${nnnn}`
const siblings = primary ? files.filter(f => f !== primary) : []
return (
<article className="docs-article">
<h1 className="docs-article-title">{header}</h1>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'notfound' && (
<div className="docs-empty">
<p>
No transcripts have been published for this session yet.{' '}
<Link
to="/docs/sessions/about"
aria-label="About sessions"
data-amp-track-name="Docs Session Empty About Link"
>
About sessions
</Link>
</p>
</div>
)}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't reach the session-history repo.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs Session Index Retry"
>
Try again
</button>
</div>
)}
{status === 'ok' && files.length === 0 && (
<div className="docs-empty">
<p>This session has no transcripts published.</p>
</div>
)}
{status === 'ok' && primary && (
<>
{siblings.length > 0 && (
<nav className="docs-session-siblings" aria-label="Transcripts in this session">
<span className="docs-session-siblings-label">Transcripts</span>
<ul className="docs-session-siblings-list">
<li>
<span
className="docs-session-siblings-current"
aria-current="true"
>
{transcriptOrdinal(primary)} (shown below)
</span>
</li>
{siblings.map(f => (
<li key={f}>
<Link
to={`/docs/sessions/${nnnn}/${f}`}
aria-label={`Transcript ${transcriptOrdinal(f)}`}
data-amp-track-name="Docs Session Sibling Transcript"
data-amp-track-session={nnnn}
data-amp-track-filename={f}
>
{transcriptOrdinal(f)}
</Link>
</li>
))}
</ul>
</nav>
)}
{bodyStatus === 'loading' && <p className="muted">Loading transcript</p>}
{bodyStatus === 'notfound' && (
<div className="docs-empty">
<p>This transcript isn't published yet.</p>
</div>
)}
{bodyStatus === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't reach the session-history repo.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs Session Inline Retry"
>
Try again
</button>
</div>
)}
{bodyStatus === 'ok' && (
<>
<TranscriptMetaHeader
nnnn={nnnn}
filename={primary}
title={title}
tldr={tldr}
/>
<div className="philosophy-body">
<MarkdownPreview content={body} />
</div>
</>
)}
</>
)}
</article>
)
}
@@ -0,0 +1,266 @@
// DocsSessionTranscript.jsx v0.21.0 (was v0.19.0 / roadmap item #30).
//
// Per-transcript view at `/docs/sessions/:nnnn/:filename`. Fetches the
// transcript body via the backend mediator and renders it through the
// shared MarkdownPreview.
//
// v0.21.0 / roadmap item #32:
// - A compact metadata header now sits above the rendered body
// (title, started/ended, duration, optional TL;DR, and a
// "View source on git.wiggleverse.org" external link). The parse
// + render helpers (`parseTranscriptMeta`, `TranscriptMetaHeader`,
// `gitSourceUrl`) are exported here so the session-root inline-
// collapse view (DocsSessionIndex.jsx) reuses the exact same
// rendering for the transcript(s) it inlines.
//
// Empty-state contract:
// 404 "This transcript isn't published yet" with a link back to
// the parent session index
// 502 "Couldn't reach the session-history repo" + retry button
import { useEffect, useState, useCallback } from 'react'
import { Link, useParams } from 'react-router-dom'
import MarkdownPreview from './MarkdownPreview.jsx'
import { getSessionTranscript, getSessionsManifest } from '../api.js'
import { EVENTS, track } from '../lib/analytics'
import './Docs.css'
// The canonical published-repo source URL for a transcript file, per
// SESSION-PROTOCOL.md §1's folder layout (one folder per session).
export function gitSourceUrl(nnnn, filename) {
return (
'https://git.wiggleverse.org/wiggleverse/ohm-session-history/src/branch/main/' +
`${encodeURIComponent(nnnn)}/${encodeURIComponent(filename)}`
)
}
// Extract the `.N` ordinal from a transcript filename:
// "SESSION-0014.1-TRANSCRIPT-...md" "0014.1"
// "SESSION-0013.1.1-TRANSCRIPT-...md" "0013.1.1" (nested subagent)
export function transcriptOrdinal(filename) {
const m = /^SESSION-(\d{4}\.\d+(?:\.\d+)*)-TRANSCRIPT/.exec(filename || '')
return m ? m[1] : filename || ''
}
// Parse the `<start>--<end>` ISO segment out of a transcript filename.
// Per the protocol the segment is `YYYY-MM-DDTHH-MM--YYYY-MM-DDTHH-MM`
// (colons replaced by dashes for filesystem portability, minute
// precision, PST implied). Legacy renamed-letter transcripts omit the
// segment entirely; in that case every derived field comes back null
// and the header degrades gracefully.
//
// Returns { ordinal, start: Date|null, end: Date|null, durationMs: number|null }.
export function parseTranscriptMeta(filename) {
const ordinal = transcriptOrdinal(filename)
const m = /-TRANSCRIPT-(\d{4}-\d{2}-\d{2})T(\d{2})-(\d{2})--(\d{4}-\d{2}-\d{2})T(\d{2})-(\d{2})\.md$/.exec(
filename || ''
)
if (!m) {
return { ordinal, start: null, end: null, durationMs: null }
}
const [, sDate, sH, sM, eDate, eH, eM] = m
// Parse as local wall-clock time. The filename carries no timezone
// (PST is implied per the protocol); we render the wall-clock value
// verbatim rather than shifting it, so we build a local Date and read
// it back with the same calendar fields. Duration is a difference of
// two local Dates, so the implied-timezone ambiguity cancels out.
const start = new Date(`${sDate}T${sH}:${sM}:00`)
const end = new Date(`${eDate}T${eH}:${eM}:00`)
const startOk = !Number.isNaN(start.getTime())
const endOk = !Number.isNaN(end.getTime())
const durationMs =
startOk && endOk && end.getTime() >= start.getTime()
? end.getTime() - start.getTime()
: null
return {
ordinal,
start: startOk ? start : null,
end: endOk ? end : null,
durationMs,
}
}
function fmtDateTime(d) {
if (!d) return null
// e.g. "May 28, 2026, 11:11 AM" human-readable, wall-clock.
try {
return d.toLocaleString(undefined, {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: 'numeric',
minute: '2-digit',
})
} catch {
return d.toISOString()
}
}
function fmtDuration(ms) {
if (ms == null || ms <= 0) return null
const totalMin = Math.round(ms / 60000)
const h = Math.floor(totalMin / 60)
const m = totalMin % 60
if (h > 0 && m > 0) return `${h}h ${m}m`
if (h > 0) return `${h}h`
return `${m}m`
}
// The compact metadata block rendered above every transcript body.
// Shared between the standalone transcript route and the session-root
// inline-collapse view. `tldr` is optional absent rendered nothing
// (graceful degrade, per the manifest schema where `tldr` may be unset).
export function TranscriptMetaHeader({ nnnn, filename, title, tldr }) {
const { ordinal, start, end, durationMs } = parseTranscriptMeta(filename)
const started = fmtDateTime(start)
const ended = fmtDateTime(end)
const duration = fmtDuration(durationMs)
const heading = title ? `${ordinal}${title}` : `Session ${ordinal}`
return (
<header className="docs-transcript-meta">
<h2 className="docs-transcript-meta-title">{heading}</h2>
{(started || ended || duration) && (
<dl className="docs-transcript-meta-grid">
{started && (
<div className="docs-transcript-meta-row">
<dt>Started</dt>
<dd>{started}</dd>
</div>
)}
{ended && (
<div className="docs-transcript-meta-row">
<dt>Ended</dt>
<dd>{ended}</dd>
</div>
)}
{duration && (
<div className="docs-transcript-meta-row">
<dt>Duration</dt>
<dd>{duration}</dd>
</div>
)}
</dl>
)}
{tldr && <p className="docs-transcript-meta-tldr">{tldr}</p>}
<a
className="docs-source-link docs-transcript-meta-source"
href={gitSourceUrl(nnnn, filename)}
target="_blank"
rel="noopener noreferrer"
aria-label={`View transcript ${ordinal} source on git.wiggleverse.org`}
data-amp-track-name="Docs Transcript Source Link"
data-amp-track-session={nnnn}
data-amp-track-filename={filename}
>
View source on git.wiggleverse.org
</a>
</header>
)
}
export default function DocsSessionTranscript() {
const { nnnn, filename } = useParams()
const [body, setBody] = useState('')
const [title, setTitle] = useState('')
const [tldr, setTldr] = useState('')
const [status, setStatus] = useState('loading') // loading | ok | notfound | error
const [reloadTick, setReloadTick] = useState(0)
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: `sessions/${nnnn}/${filename}` })
}, [nnnn, filename])
// Title + optional tl;dr from the manifest decorative metadata that
// feeds the header. Failure leaves the header showing the bare NNNN
// and no TL;DR; the body fetch below is the load-bearing one.
useEffect(() => {
let active = true
getSessionsManifest()
.then(payload => {
if (!active) return
const entry = (payload && payload[nnnn]) || {}
setTitle(entry.title || '')
setTldr(typeof entry.tldr === 'string' ? entry.tldr : '')
})
.catch(() => {})
return () => { active = false }
}, [nnnn])
useEffect(() => {
let active = true
setStatus('loading')
getSessionTranscript(nnnn, filename)
.then(text => {
if (!active) return
setBody(text || '')
setStatus('ok')
})
.catch(e => {
if (!active) return
if (e.status === 404) {
setStatus('notfound')
} else {
setStatus('error')
}
})
return () => { active = false }
}, [nnnn, filename, reloadTick])
const retry = useCallback(() => setReloadTick(t => t + 1), [])
return (
<article className="docs-article">
<div className="docs-breadcrumbs">
<Link
to={`/docs/sessions/${nnnn}`}
aria-label={`Back to session ${nnnn} index`}
data-amp-track-name="Docs Transcript Back To Index"
>
Session {nnnn}
</Link>
</div>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'notfound' && (
<div className="docs-empty">
<p>This transcript isn't published yet.</p>
<p>
<Link
to={`/docs/sessions/${nnnn}`}
aria-label={`Back to session ${nnnn}`}
data-amp-track-name="Docs Transcript Back To Session"
>
Back to session {nnnn}
</Link>
</p>
</div>
)}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't reach the session-history repo.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs Transcript Retry"
>
Try again
</button>
</div>
)}
{status === 'ok' && (
<>
<TranscriptMetaHeader
nnnn={nnnn}
filename={filename}
title={title}
tldr={tldr}
/>
<div className="philosophy-body">
<MarkdownPreview content={body} />
</div>
</>
)}
</article>
)
}
@@ -0,0 +1,99 @@
// DocsSessionsAbout.jsx v0.19.0 / roadmap item #30.
//
// Renders the README.md of the public `wiggleverse/ohm-session-history`
// repo at `/docs/sessions/about`. The framework backend mediates the
// gitea fetch (see backend/app/docs_sessions.py); this component
// handles three response paths:
//
// 200 render the markdown via MarkdownPreview
// 404 "About not yet published" empty-state (the upstream README
// doesn't exist yet happens when a deployment hasn't
// restructured its session-history repo yet, expected at
// v0.19.0 deploy time per the CHANGELOG)
// 502 "Couldn't reach the session-history repo" with a retry
// button. The retry just re-invokes the fetch no extra
// backoff because the backend cache already smoothes
// repeated 502s.
import { useEffect, useState, useCallback } from 'react'
import MarkdownPreview from './MarkdownPreview.jsx'
import { getSessionsAbout } from '../api.js'
import { EVENTS, track } from '../lib/analytics'
export default function DocsSessionsAbout() {
const [body, setBody] = useState('')
const [status, setStatus] = useState('loading') // loading | ok | notfound | error
const [reloadTick, setReloadTick] = useState(0)
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: 'sessions/about' })
}, [])
useEffect(() => {
let active = true
setStatus('loading')
getSessionsAbout()
.then(text => {
if (!active) return
setBody(text || '')
setStatus('ok')
})
.catch(e => {
if (!active) return
if (e.status === 404) {
setStatus('notfound')
} else {
setStatus('error')
}
})
return () => { active = false }
}, [reloadTick])
const retry = useCallback(() => setReloadTick(t => t + 1), [])
return (
<article className="docs-article">
<h1 className="docs-article-title">About sessions</h1>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'notfound' && (
<div className="docs-empty">
<p>
The session-history About page isn't published yet. Sessions
are still authored once a few have shipped, this page will
render the canonical introduction.
</p>
<p>
In the meantime, browse the source repo directly at{' '}
<a
href="https://git.wiggleverse.org/wiggleverse/ohm-session-history"
target="_blank"
rel="noopener noreferrer"
aria-label="Open session-history repo on gitea"
data-amp-track-name="Docs Sessions About Repo Link"
>
wiggleverse/ohm-session-history
</a>.
</p>
</div>
)}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't reach the session-history repo.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs Sessions About Retry"
>
Try again
</button>
</div>
)}
{status === 'ok' && (
<div className="philosophy-body">
<MarkdownPreview content={body} />
</div>
)}
</article>
)
}
+134
View File
@@ -0,0 +1,134 @@
// DocsSpec.jsx v0.20.0.
//
// Per-spec view at `/docs/specs/:name`. Fetches a configured spec
// body via the backend mediator (which proxies the gitea raw URL)
// and renders it through the shared MarkdownPreview. The page header
// pulls the spec's `title` from the manifest so the breadcrumb-free
// page still names what you're looking at.
//
// Empty-state contract:
// 404 "This spec isn't published yet / unknown name" with a hint
// to pick a configured spec from the nav.
// 502 "Couldn't reach the spec source" + retry button.
//
// History view is intentionally absent the operator's framing for
// v0.20.0 is "current version only; git is the history surface".
// A small "View on gitea" link beside the title points at the
// upstream source URL the manifest carries so the history gesture
// remains one click away.
import { useEffect, useState, useCallback } from 'react'
import { Link, useParams } from 'react-router-dom'
import MarkdownPreview from './MarkdownPreview.jsx'
import { getSpec, getSpecsManifest } from '../api.js'
import { EVENTS, track } from '../lib/analytics'
export default function DocsSpec() {
const { name } = useParams()
const [title, setTitle] = useState('')
const [sourceUrl, setSourceUrl] = useState('')
const [body, setBody] = useState('')
const [status, setStatus] = useState('loading') // loading | ok | notfound | error
const [reloadTick, setReloadTick] = useState(0)
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: `specs/${name}` })
}, [name])
// Title + upstream URL from the manifest (cheap the manifest is
// derived from an env var on the backend, no network).
useEffect(() => {
let active = true
getSpecsManifest()
.then(payload => {
if (!active) return
const entry = (payload && payload.specs || []).find(s => s.name === name)
setTitle((entry && entry.title) || '')
setSourceUrl((entry && entry.url) || '')
})
.catch(() => {
// Title + source link are decorative; the body fetch below
// is the load-bearing one. A failed manifest fetch just
// leaves the page rendering the bare name.
})
return () => { active = false }
}, [name])
useEffect(() => {
let active = true
setStatus('loading')
getSpec(name)
.then(text => {
if (!active) return
setBody(text || '')
setStatus('ok')
})
.catch(e => {
if (!active) return
if (e.status === 404) {
setStatus('notfound')
} else {
setStatus('error')
}
})
return () => { active = false }
}, [name, reloadTick])
const retry = useCallback(() => setReloadTick(t => t + 1), [])
const header = title || `Spec: ${name}`
return (
<article className="docs-article">
<header className="docs-article-header">
<h1 className="docs-article-title">{header}</h1>
{sourceUrl && (
<a
className="docs-source-link"
href={sourceUrl}
target="_blank"
rel="noopener noreferrer"
aria-label="View spec source on gitea"
data-amp-track-name="Docs Spec Source Link"
data-amp-track-spec={name}
>
View source
</a>
)}
</header>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'notfound' && (
<div className="docs-empty">
<p>This spec isn't available.</p>
<p>
<Link
to="/docs/user-guide"
aria-label="User guide"
data-amp-track-name="Docs Spec Notfound User Guide Link"
>
Back to the user guide
</Link>
</p>
</div>
)}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't reach the spec source.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs Spec Retry"
>
Try again
</button>
</div>
)}
{status === 'ok' && (
<div className="philosophy-body">
<MarkdownPreview content={body} />
</div>
)}
</article>
)
}
@@ -0,0 +1,82 @@
// DocsSpecsIndex.jsx v0.20.0.
//
// Landing route at `/docs/specs`. The operator-stated body shape for
// the parallel `/docs/sessions/:nnnn` page is "no body list pick a
// transcript from the nav", and the same gesture applies here: the
// `/docs/specs` route either redirects to the first configured spec
// (the common case) or renders a "no specs configured" empty state
// (only reachable if a deployment overrides `OHM_DOCS_SPECS` to an
// empty list the framework default has two entries).
//
// The redirect is client-side because the manifest is a single API
// call away; server-side redirect would require either a backend
// route for the bare `/docs/specs` path (out of scope for v0.20.0)
// or a build-time bake of the first spec name (which couples the
// frontend bundle to the deployment overlay, which we don't do).
import { useEffect, useState } from 'react'
import { Navigate, Link } from 'react-router-dom'
import { getSpecsManifest } from '../api.js'
import { EVENTS, track } from '../lib/analytics'
export default function DocsSpecsIndex() {
const [firstName, setFirstName] = useState(null)
// Tri-state: 'loading' (waiting on manifest), 'redirect' (we have a
// name to redirect to render <Navigate>), 'empty' (no specs
// configured), or 'error' (couldn't load the manifest at all).
const [status, setStatus] = useState('loading')
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: 'specs' })
}, [])
useEffect(() => {
let active = true
getSpecsManifest()
.then(payload => {
if (!active) return
const specs = (payload && payload.specs) || []
if (specs.length === 0) {
setStatus('empty')
} else {
setFirstName(specs[0].name)
setStatus('redirect')
}
})
.catch(() => {
if (!active) return
setStatus('error')
})
return () => { active = false }
}, [])
if (status === 'redirect' && firstName) {
return <Navigate to={firstName} replace />
}
return (
<article className="docs-article">
<h1 className="docs-article-title">Specs</h1>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'empty' && (
<div className="docs-empty">
<p>No specs are configured for this deployment.</p>
<p>
<Link
to="/docs/user-guide"
aria-label="User guide"
data-amp-track-name="Docs Specs Empty User Guide Link"
>
Back to the user guide
</Link>
</p>
</div>
)}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't load the spec list.</p>
</div>
)}
</article>
)
}
+81
View File
@@ -0,0 +1,81 @@
// DocsUserGuide.jsx v0.19.0 / roadmap item #30.
//
// Renders DOCS.md at `/docs/user-guide`. Was `/docs` before v0.19.0
// (the v0.14.0 single-route Docs.jsx surface, now superseded). The
// content path is unchanged: backend reads `DOCS.md` from disk and
// serves it at `/api/docs`. The body is rendered via the existing
// `MarkdownPreview` (the same component the `/philosophy` route uses,
// so we don't introduce a second markdown library).
// v0.21.0 / roadmap item #31: the loading + error states are brought
// onto the same `.docs-empty` / `.docs-error` convention every other
// docs surface uses (was a bare `<p className="error">`), with a retry
// button so a transient `/api/docs` failure isn't a dead end.
import { useEffect, useState, useCallback } from 'react'
import { Link } from 'react-router-dom'
import MarkdownPreview from './MarkdownPreview.jsx'
import { getDocs } from '../api.js'
import { EVENTS, track } from '../lib/analytics'
export default function DocsUserGuide() {
const [body, setBody] = useState('')
const [status, setStatus] = useState('loading') // loading | ok | error
const [reloadTick, setReloadTick] = useState(0)
useEffect(() => {
track(EVENTS.DOC_VIEWED, { section: 'user-guide' })
}, [])
useEffect(() => {
let active = true
setStatus('loading')
getDocs()
.then(r => {
if (!active) return
setBody(r.body || '')
setStatus('ok')
})
.catch(() => {
if (!active) return
setStatus('error')
})
return () => { active = false }
}, [reloadTick])
const retry = useCallback(() => setReloadTick(t => t + 1), [])
return (
<article className="docs-article">
<h1 className="docs-article-title">User guide</h1>
{status === 'loading' && <p className="muted">Loading</p>}
{status === 'error' && (
<div className="docs-error" role="alert">
<p>Couldn't load the user guide.</p>
<button
type="button"
onClick={retry}
aria-label="Retry"
data-amp-track-name="Docs User Guide Retry"
>
Try again
</button>
<p>
<Link
to="/docs/sessions/about"
aria-label="About sessions"
data-amp-track-name="Docs User Guide Error About Link"
>
About sessions
</Link>
</p>
</div>
)}
{status === 'ok' && (
<div className="philosophy-body">
<MarkdownPreview content={body} />
</div>
)}
</article>
)
}
+2 -2
View File
@@ -17,7 +17,7 @@
import { useEditor, EditorContent, Extension } from '@tiptap/react'
import StarterKit from '@tiptap/starter-kit'
import { useEffect, useRef, useCallback } from 'react'
import { marked } from 'marked'
import { renderMarkdown } from '../lib/sanitizeHtml'
import { Plugin, PluginKey } from 'prosemirror-state'
import { Decoration, DecorationSet } from 'prosemirror-view'
@@ -122,7 +122,7 @@ export default function Editor({
useEffect(() => {
if (!editor || content == null) return
const html = marked.parse(content)
const html = renderMarkdown(content)
editor.commands.setContent(html, false)
}, [content, editor])
+128
View File
@@ -0,0 +1,128 @@
/* Inbox.css §15.2 inbox panel refinements (roadmap #25, light pass).
*
* The base inbox layout/structure lives in App.css (the §15 / Slice 6
* block). This sheet is a TOKENIZED polish layer on top of it: it does
* NOT re-lay-out the panel, it sharpens the unread/read distinction,
* adds the per-row "mark read" affordance + the unread dot, and gives
* the empty/loading states real copy and spacing.
*
* Cascade note: Inbox.jsx is imported by App.jsx (line 6) BEFORE the
* App.css import (line 30), so under ESM depth-first evaluation this
* sheet is injected FIRST and App.css wins on equal specificity. Any
* rule here that must override an App.css value is therefore written
* one notch more specific (e.g. `.inbox-list .inbox-row.unread`).
* New classes that App.css doesn't define need no such guard.
*/
/* ===== Unread vs. read distinction ===== */
/* A clear left accent bar + warmer tint on unread; read rows sit calm. */
.inbox-list .inbox-row {
position: relative;
border-bottom: 1px solid var(--color-border);
transition: background var(--motion-fast) var(--ease-out);
}
.inbox-list .inbox-row.unread {
background: var(--color-warning-bg-soft, var(--c-warning-bg-soft));
box-shadow: inset 3px 0 0 var(--color-accent);
}
.inbox-list .inbox-row.read .inbox-summary {
color: var(--color-text-muted);
font-weight: var(--weight-normal);
}
.inbox-list .inbox-row.unread .inbox-summary {
color: var(--color-text);
font-weight: var(--weight-medium);
}
/* The dot is a NEW affordance: a filled accent dot for unread, hidden
* (but space-reserved) for read so summaries stay column-aligned. */
.inbox-unread-dot {
flex: 0 0 auto;
width: 8px;
height: 8px;
border-radius: var(--radius-pill);
background: var(--color-accent);
}
.inbox-row.read .inbox-unread-dot {
background: transparent;
}
/* ===== Per-row "mark read" affordance ===== */
/* The row is a flex Link followed by this button; pin the button to the
* right edge, revealed on row hover/focus and always visible on touch. */
.inbox-row {
display: flex;
align-items: center;
}
.inbox-row .inbox-row-link {
flex: 1 1 auto;
min-width: 0;
}
.inbox-row-dismiss {
flex: 0 0 auto;
display: inline-flex;
align-items: center;
justify-content: center;
width: 28px;
height: 28px;
margin-right: var(--space-5);
padding: 0;
color: var(--color-text-subtle);
background: transparent;
border: 1px solid transparent;
border-radius: var(--radius-md);
cursor: pointer;
opacity: 0;
transition:
opacity var(--motion-fast) var(--ease-out),
color var(--motion-fast) var(--ease-out),
background var(--motion-fast) var(--ease-out),
border-color var(--motion-fast) var(--ease-out);
}
.inbox-row:hover .inbox-row-dismiss,
.inbox-row:focus-within .inbox-row-dismiss,
.inbox-row-dismiss:focus-visible {
opacity: 1;
}
.inbox-row-dismiss:hover {
color: var(--color-success-fg);
background: var(--color-success-bg);
border-color: var(--color-success-bg);
}
.inbox-row-dismiss:focus-visible {
outline: 2px solid var(--color-focus-ring);
outline-offset: 1px;
}
/* Coarse pointers (touch) have no hover; keep the affordance discoverable. */
@media (hover: none) {
.inbox-row-dismiss { opacity: 1; }
}
/* ===== Mark-all-read button ===== */
.inbox-mark-all {
margin-left: auto;
}
/* ===== Empty / loading states ===== */
.inbox-state {
padding: var(--space-9) var(--space-7);
text-align: center;
}
.inbox-empty {
padding: var(--space-11) var(--space-7);
text-align: center;
}
.inbox-empty-title {
margin: 0 0 var(--space-3);
font-size: var(--text-md);
font-weight: var(--weight-semibold);
color: var(--color-text-strong);
}
.inbox-empty .muted {
margin: 0;
font-size: var(--text-base);
line-height: var(--leading-normal);
color: var(--color-text-muted);
}
+56 -11
View File
@@ -15,6 +15,7 @@ import {
markNotificationRead,
markNotificationsReadByFilter,
} from '../api.js'
import './Inbox.css'
const CATEGORIES = [
{ value: '', label: 'All categories' },
@@ -56,11 +57,15 @@ export default function Inbox({ onClose, lastChangeTick }) {
return Array.from(seen.entries())
}, [items])
async function markOneRead(item) {
if (item.read_at) return
await markNotificationRead(item.id)
setItems(prev => prev.map(p => p.id === item.id ? { ...p, read_at: new Date().toISOString() } : p))
setUnreadCount(c => Math.max(0, c - 1))
}
async function handleRowClick(item) {
if (!item.read_at) {
await markNotificationRead(item.id)
setItems(prev => prev.map(p => p.id === item.id ? { ...p, read_at: new Date().toISOString() } : p))
}
await markOneRead(item)
}
async function markAllUnderFilter() {
@@ -121,22 +126,36 @@ export default function Inbox({ onClose, lastChangeTick }) {
</label>
<button
className="btn-link"
className="btn-link inbox-mark-all"
onClick={markAllUnderFilter}
disabled={items.every(i => i.read_at)}
title="Mark every notification matching the current filter as read"
>
Mark all read (under filter)
Mark all read
</button>
</div>
<div className="inbox-body">
{loading && <p className="muted">Loading</p>}
{loading && <p className="inbox-state muted">Loading your inbox</p>}
{!loading && items.length === 0 && (
<p className="muted">No notifications match. Try a different filter, or come back later.</p>
<div className="inbox-empty">
<p className="inbox-empty-title">You're all caught up.</p>
<p className="muted">
{filters.unread || filters.rfcSlug || filters.category
? 'Nothing matches the current filters. Clear them to see everything.'
: 'New activity on RFCs you follow will show up here.'}
</p>
</div>
)}
<ul className="inbox-list">
{items.map(item => (
<InboxRow key={item.id} item={item} onClick={handleRowClick} onClose={onClose} />
<InboxRow
key={item.id}
item={item}
onClick={handleRowClick}
onMarkRead={markOneRead}
onClose={onClose}
/>
))}
</ul>
</div>
@@ -145,16 +164,24 @@ export default function Inbox({ onClose, lastChangeTick }) {
)
}
function InboxRow({ item, onClick, onClose }) {
function InboxRow({ item, onClick, onMarkRead, onClose }) {
const unread = !item.read_at
const target = deepLink(item)
const handle = async () => {
await onClick(item)
if (target) onClose?.()
}
const handleMarkRead = async (e) => {
// Don't let the row's Link fire this affordance only marks read,
// it never navigates.
e.preventDefault()
e.stopPropagation()
await onMarkRead(item)
}
return (
<li className={`inbox-row ${unread ? 'unread' : ''}`}>
<li className={`inbox-row ${unread ? 'unread' : 'read'}`}>
<Link to={target || '#'} onClick={handle} className="inbox-row-link">
<span className="inbox-unread-dot" aria-hidden />
<span className={`inbox-cat cat-${item.category || 'unknown'}`}>{item.category || '·'}</span>
<span className="inbox-summary">{item.summary}</span>
{item.bundled_count > 1 && (
@@ -162,6 +189,24 @@ function InboxRow({ item, onClick, onClose }) {
)}
<span className="inbox-when">{formatWhen(item.created_at)}</span>
</Link>
{unread && (
<button
type="button"
className="inbox-row-dismiss"
onClick={handleMarkRead}
aria-label="Mark as read"
title="Mark as read"
>
{/* check glyph — dependency-free inline SVG */}
<svg
width="14" height="14" viewBox="0 0 24 24"
fill="none" stroke="currentColor" strokeWidth="2.25"
strokeLinecap="round" strokeLinejoin="round" aria-hidden
>
<path d="m5 13 4 4 10-11" />
</svg>
</button>
)}
</li>
)
}
+38
View File
@@ -0,0 +1,38 @@
// LinkedText.jsx roadmap #28 Part 1.
//
// Renders a backend-provided list of text/rfc-link segments (see
// backend/app/rfc_links.py). RFC references in PR descriptions and
// comments arrive pre-scanned as structured segments this component
// maps them onto plain text runs and anchor elements. It never renders
// HTML from the server (no dangerouslySetInnerHTML), so the surface is
// XSS-safe regardless of what a comment author typed.
//
// `segments` is the enriched array; `text` is the raw fallback used when
// the field is absent (an older cached response, or a caller that didn't
// pass segments). Either way the visible text is identical only the
// links differ.
export default function LinkedText({ segments, text }) {
if (!Array.isArray(segments) || segments.length === 0) {
return <>{text ?? ''}</>
}
return (
<>
{segments.map((seg, i) => {
if (seg.type === 'rfc') {
return (
<a
key={i}
className="rfc-autolink"
href={`/rfc/${seg.slug}`}
title={seg.title ? `RFC: ${seg.title}` : undefined}
>
{seg.label}
</a>
)
}
return <span key={i}>{seg.text}</span>
})}
</>
)
}
+2 -1
View File
@@ -21,6 +21,7 @@
import { useEffect, useRef, useState, useCallback } from 'react'
import { Marked } from 'marked'
import { sanitizeHtml } from '../lib/sanitizeHtml'
import { decorateAcceptedChanges } from './trackedOverlay.js'
import ChangeTooltip from './ChangeTooltip.jsx'
@@ -98,7 +99,7 @@ export default function MarkdownPreview({
// synchronously with the body itself no flash of un-decorated text.
useEffect(() => {
if (!hostRef.current) return
const html = previewMarked.parse(content || '')
const html = sanitizeHtml(previewMarked.parse(content || ''))
hostRef.current.innerHTML = html
const token = ++renderTokenRef.current
// Reset memo so the new block set re-renders from scratch.
+21 -1
View File
@@ -15,6 +15,8 @@ import { EVENTS, track } from '../lib/analytics'
export default function PRModal({ slug, branch, branchIsPrivate, onClose, onOpened }) {
const [title, setTitle] = useState('')
const [description, setDescription] = useState('')
// #26: optional ground-truth use case for this change.
const [useCase, setUseCase] = useState('')
const [drafting, setDrafting] = useState(true)
const [submitting, setSubmitting] = useState(false)
const [confirmed, setConfirmed] = useState(!branchIsPrivate)
@@ -39,7 +41,11 @@ export default function PRModal({ slug, branch, branchIsPrivate, onClose, onOpen
setSubmitting(true)
setError(null)
try {
const { pr_number } = await openPR(slug, branch, { title: title.trim(), description: description.trim() })
const { pr_number } = await openPR(slug, branch, {
title: title.trim(),
description: description.trim(),
proposedUseCase: useCase.trim() || null,
})
// v0.15.0 analytics: fire on §10.2 PR-open success. slug
// and pr_number are the join keys; title/description stay out.
track(EVENTS.PR_OPENED, { rfc_slug: slug, pr_number })
@@ -104,6 +110,20 @@ export default function PRModal({ slug, branch, branchIsPrivate, onClose, onOpen
what was argued, what shifted, what the arbiters are asked
to consider.
</p>
<label className="modal-label">What will you be using this change for? (optional)</label>
<textarea
className="modal-textarea"
value={useCase}
onChange={e => setUseCase(e.target.value)}
placeholder="The concrete thing this change unlocks for you. Optional."
disabled={drafting || submitting}
rows={3}
maxLength={8000}
/>
<p className="field-help">
#26: the concrete ground-truth use case distinct from "why
it's needed" above. Leave blank if you'd rather not say.
</p>
{error && <p className="field-error">{error}</p>}
</div>
<div className="modal-actions">
+18 -2
View File
@@ -23,6 +23,7 @@ import {
withdrawPR,
} from '../api'
import { EVENTS, track } from '../lib/analytics'
import LinkedText from './LinkedText'
export default function PRView({ viewer }) {
const { slug, prNumber: prNumberParam } = useParams()
@@ -218,8 +219,21 @@ export default function PRView({ viewer }) {
<>
<h1 className="pr-title">{pr.title}</h1>
{pr.description && (
<p className="pr-description">{pr.description}</p>
<p className="pr-description">
<LinkedText segments={pr.description_segments} text={pr.description} />
</p>
)}
{/* #26: the optional ground-truth use case for this change,
captured when the PR was opened. Muted "left blank"
treatment when none was supplied. */}
<div className="pr-use-case" style={{ margin: '6px 0', fontSize: 13 }}>
<span style={{ fontWeight: 700, color: '#888', textTransform: 'uppercase', letterSpacing: '0.05em', fontSize: 11 }}>
Intended use case:
</span>{' '}
{pr.proposed_use_case
? <span style={{ whiteSpace: 'pre-wrap' }}>{pr.proposed_use_case}</span>
: <span style={{ color: '#999', fontStyle: 'italic' }}>left blank</span>}
</div>
{pr.capabilities?.can_edit_text && (
<button className="btn-link" onClick={startHeaderEdit}>Edit title & description</button>
)}
@@ -429,7 +443,9 @@ function PRConversation({ threads, messagesByThread, threadsByKind, seenMsgId })
{isNew && <span className="chat-msg-new-pip" title="New since your last visit"></span>}
</div>
{m.quote && <pre className="chat-msg-quote">{m.quote}</pre>}
<div className="chat-msg-body">{m.text}</div>
<div className="chat-msg-body">
<LinkedText segments={m.text_segments} text={m.text} />
</div>
</li>
)
})}
+10 -2
View File
@@ -10,7 +10,7 @@
import { useEffect, useState } from 'react'
import { useParams, useNavigate } from 'react-router-dom'
import { marked } from 'marked'
import { renderMarkdown } from '../lib/sanitizeHtml'
import { getProposal, mergeProposal, declineProposal, withdrawProposal } from '../api'
export default function ProposalView({ viewer, onChange }) {
@@ -161,8 +161,16 @@ export default function ProposalView({ viewer, onChange }) {
</h3>
<div
className="entry-body"
dangerouslySetInnerHTML={{ __html: marked.parse(data.entry?.body || '') }}
dangerouslySetInnerHTML={{ __html: renderMarkdown(data.entry?.body || '') }}
/>
{/* #26: the optional ground-truth use case the proposer supplied. */}
<h3 style={{ fontSize: 13, fontWeight: 700, color: '#888', textTransform: 'uppercase', letterSpacing: '0.05em', marginTop: 24 }}>
Intended use case
</h3>
{data.proposed_use_case
? <div className="entry-body" dangerouslySetInnerHTML={{ __html: renderMarkdown(data.proposed_use_case) }} />
: <p style={{ color: '#999', fontStyle: 'italic' }}>Left blank by the proposer.</p>}
</article>
)
}
+100 -5
View File
@@ -2,17 +2,24 @@
//
// Title (required) and pitch (required textarea), with a slug field
// that auto-fills from the title via the same deterministic kebab-case
// the backend uses. Tags are chip-input (free-form for slice 1; the
// AI-suggested chips of §9.1 are deferred to Slice 2 when the AI surface
// is wired up).
// the backend uses. Tags are chip-input (free-form), with the §9.1
// Slice 2 AI-suggested chips (roadmap #27) wired in: as the draft fills
// in, the backend asks Claude Haiku for tags drawn from the corpus's
// existing tag set, surfaced as clickable suggestion chips. The assist
// is best-effort it stays silent when unavailable.
//
// The submit button drives the §17 POST /api/rfcs/propose endpoint;
// success navigates the proposer to the pending-idea view per §9.3.
import { useEffect, useState } from 'react'
import { proposeRFC } from '../api'
import { useEffect, useRef, useState } from 'react'
import { proposeRFC, suggestTags } from '../api'
import { EVENTS, track } from '../lib/analytics'
// How long the draft must sit unchanged before we ask for suggestions
// long enough to fire on typing pauses / field-blur, not on every
// keystroke (the backend is also per-user rate-limited as a backstop).
const SUGGEST_DEBOUNCE_MS = 700
function slugify(title) {
return title
.toLowerCase()
@@ -26,21 +33,56 @@ export default function ProposeModal({ viewer, onClose, onSubmitted }) {
const [slug, setSlug] = useState('')
const [slugEdited, setSlugEdited] = useState(false)
const [pitch, setPitch] = useState('')
// #26: optional ground-truth use case, sibling to the required pitch.
const [useCase, setUseCase] = useState('')
const [tagInput, setTagInput] = useState('')
const [tags, setTags] = useState([])
const [submitting, setSubmitting] = useState(false)
const [error, setError] = useState(null)
// #27: Claude Haiku tag suggestions. `suggestions` is the latest
// ranked list from the backend ({ tag, confidence }); we render the
// subset not already chosen. `suggestedOnce` gates the disclosure +
// row so they only appear after the assist has actually run.
const [suggestions, setSuggestions] = useState([])
const [suggestedOnce, setSuggestedOnce] = useState(false)
useEffect(() => {
if (!slugEdited) setSlug(slugify(title))
}, [title, slugEdited])
// #27: debounced tag-suggestion fetch. Fires after the draft sits
// unchanged for SUGGEST_DEBOUNCE_MS, only once there's something to go
// on (a title). A stale-response guard keeps an earlier in-flight
// request from clobbering a newer one.
const suggestSeq = useRef(0)
useEffect(() => {
if (!title.trim()) {
setSuggestions([])
return
}
const handle = setTimeout(async () => {
const seq = ++suggestSeq.current
const result = await suggestTags({ title, pitch, useCase })
if (seq !== suggestSeq.current) return // a newer request superseded us
setSuggestions(Array.isArray(result) ? result : [])
if (result && result.length) setSuggestedOnce(true)
}, SUGGEST_DEBOUNCE_MS)
return () => clearTimeout(handle)
}, [title, pitch, useCase])
// Suggestions the user hasn't already added.
const freshSuggestions = suggestions.filter(s => !tags.includes(s.tag))
function addTag() {
const t = tagInput.trim()
if (t && !tags.includes(t)) setTags([...tags, t])
setTagInput('')
}
function addSuggested(tag) {
if (!tags.includes(tag)) setTags([...tags, tag])
}
async function handleSubmit(e) {
e.preventDefault()
if (!title.trim() || !slug || !pitch.trim()) return
@@ -52,6 +94,7 @@ export default function ProposeModal({ viewer, onClose, onSubmitted }) {
slug,
pitch: pitch.trim(),
tags,
proposedUseCase: useCase.trim() || null,
})
// v0.15.0 analytics: fire on the §9.1 propose-RFC submit.
// Slug is a stable, low-cardinality identifier (kebab-case
@@ -105,6 +148,19 @@ export default function ProposeModal({ viewer, onClose, onSubmitted }) {
required
/>
<label htmlFor="propose-use-case">What will you be using this RFC for? (optional)</label>
<textarea
id="propose-use-case"
value={useCase}
onChange={e => setUseCase(e.target.value)}
placeholder="The concrete thing you intend to build or do with this RFC. Optional, but it helps ground the work."
rows={3}
/>
<p className="field-help">
The concrete ground-truth use case distinct from "why it's
needed" above. Leave blank if you'd rather not say.
</p>
<label htmlFor="propose-tag">Tags (optional)</label>
<div style={{ display: 'flex', gap: 6, alignItems: 'center', marginBottom: 4 }}>
<input
@@ -136,6 +192,45 @@ export default function ProposeModal({ viewer, onClose, onSubmitted }) {
</div>
)}
{/* #27: Claude Haiku suggested tags. Clickable chips that add
to the tag list; nothing auto-applies. The disclosure is
required the draft text is sent to Anthropic to generate
these so it renders whenever the suggestion row does. */}
{(freshSuggestions.length > 0 || (suggestedOnce && suggestions.length > 0)) && (
<div style={{ marginTop: 4, marginBottom: 14 }}>
{freshSuggestions.length > 0 && (
<>
<p className="field-help" style={{ marginTop: 0, marginBottom: 4 }}>
Suggested tags click to add:
</p>
<div>
{freshSuggestions.map(s => (
<button
key={s.tag}
type="button"
className="entry-tag"
onClick={() => addSuggested(s.tag)}
title={`Add "${s.tag}"`}
style={{
display: 'inline-block',
marginRight: 4,
marginBottom: 4,
border: '1px dashed var(--color-border, #ccc)',
background: 'none',
cursor: 'pointer',
}}
>+ {s.tag}</button>
))}
</div>
</>
)}
<p className="field-help" style={{ marginTop: 4, marginBottom: 0, fontStyle: 'italic' }}>
Suggestions are generated by Claude (Anthropic). The text you've
entered above is sent to Anthropic to produce them.
</p>
</div>
)}
{viewer && (
<p className="field-help" style={{ marginTop: 14, marginBottom: 0 }}>
Owner: <strong>{viewer.display_name || viewer.gitea_login}</strong> you'll be the first owner of this super-draft. Additional owners can claim later (§13.1).
@@ -19,6 +19,7 @@ import {
resolveDiscussionThread,
} from '../api'
import { EVENTS, track } from '../lib/analytics'
import LinkedText from './LinkedText'
export default function RFCDiscussionPanel({ slug, viewer }) {
const [threads, setThreads] = useState([])
@@ -279,7 +280,9 @@ function DiscussionMessage({ message }) {
{message.quote && (
<div className="discussion-message-quote">"{message.quote}"</div>
)}
<div className="discussion-message-body">{message.text}</div>
<div className="discussion-message-body">
<LinkedText segments={message.text_segments} text={message.text} />
</div>
</div>
)
}
+13
View File
@@ -669,6 +669,19 @@ export default function RFCView({ viewer }) {
: 'main is read-only — PRs are the only path to change it. Open a branch to propose edits.'}
</div>
)}
{/* #26: the optional ground-truth use case captured at propose
time. Shown on the canonical (main) view; muted "left blank"
treatment when the proposer didn't supply one. */}
{branchParam === 'main' && (
<div className="rfc-use-case" style={{ margin: '8px 0 16px', padding: '10px 14px', borderLeft: '3px solid #e0e0e0', background: '#fafafa' }}>
<div style={{ fontSize: 11, fontWeight: 700, color: '#888', textTransform: 'uppercase', letterSpacing: '0.05em', marginBottom: 4 }}>
Intended use case
</div>
{entry.proposed_use_case
? <div style={{ whiteSpace: 'pre-wrap' }}>{entry.proposed_use_case}</div>
: <span style={{ color: '#999', fontStyle: 'italic' }}>Left blank by the proposer.</span>}
</div>
)}
{inDiscuss && branchParam !== 'main' && (
<div className="discuss-mode-banner">
Discuss mode on <strong>{branchParam}</strong> chat freely;
+3 -4
View File
@@ -1,8 +1,7 @@
:root {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue",
Arial, sans-serif;
color: #1a1a1a;
background: #fafaf8;
font-family: var(--font-sans);
color: var(--color-text);
background: var(--color-bg);
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
+7
View File
@@ -97,6 +97,13 @@ export const EVENTS = Object.freeze({
// v0.17.0 / item #16 — admin-create user + invite email.
USER_INVITED: 'User Invited',
INVITE_CLAIMED: 'Invite Claimed',
// v0.19.0 / item #30 — `/docs/*` flyout + sessions browser. Carries
// `section`: 'user-guide' | 'sessions/about' | 'sessions/<NNNN>' |
// 'sessions/<NNNN>/<filename>' so the dashboard can answer which
// docs surfaces get read most. The transcript-section value includes
// the filename so an aggregator can group by `sessions/<NNNN>` or by
// exact transcript.
DOC_VIEWED: 'Doc Viewed',
})
// Internal state.
+47
View File
@@ -0,0 +1,47 @@
// sanitizeHtml.js — the single chokepoint for turning user-authored
// markdown into DOM-bound HTML.
//
// Security audit 0026 (finding C1, Critical): every `marked.parse(...)`
// result that reaches an `innerHTML` / `dangerouslySetInnerHTML` sink was
// previously written raw. `marked` passes through embedded HTML and
// `javascript:`/event-handler attributes verbatim, so any user-authored
// document (RFC body, proposal body, proposed_use_case, transcript) was a
// stored-XSS vector — a contributor's payload executed in the session of
// whoever viewed it, including an admin/owner during review.
//
// Fix: route EVERY markdown render through `renderMarkdown` (or, for
// already-rendered HTML, `sanitizeHtml`). DOMPurify's defaults already
// strip <script>, on* event handlers, and javascript:/unsafe-data: URIs;
// we add a hook so any link opening a new tab carries rel="noopener
// noreferrer". The html profile keeps the standard markdown tag set plus
// class + data-* attributes (the latter is what MarkdownPreview's mermaid
// placeholder relies on); mermaid renders its SVG into the DOM *after*
// sanitization and is itself locked down with securityLevel:'strict'.
import DOMPurify from 'dompurify'
import { marked } from 'marked'
let _hookInstalled = false
function ensureHook() {
if (_hookInstalled) return
DOMPurify.addHook('afterSanitizeAttributes', (node) => {
if (node.tagName === 'A' && node.getAttribute('target') === '_blank') {
node.setAttribute('rel', 'noopener noreferrer')
}
})
_hookInstalled = true
}
// Sanitize an already-rendered HTML string. Use when the HTML did not come
// from `marked` (rare) or when a caller parses markdown with a bespoke
// `Marked` instance and only needs the sanitize step.
export function sanitizeHtml(html) {
ensureHook()
return DOMPurify.sanitize(html || '', { USE_PROFILES: { html: true } })
}
// Parse markdown with the shared `marked` and sanitize the result. This is
// the drop-in replacement for `marked.parse(src)` at any HTML sink.
export function renderMarkdown(src) {
return sanitizeHtml(marked.parse(src || ''))
}
+91
View File
@@ -0,0 +1,91 @@
// useLastState — v0.23.0 / roadmap item #29: server-side sign-in state
// resume. Two responsibilities, kept in one small hook so App.jsx's
// surface stays minimal:
//
// 1. Debounce-post the authenticated user's current route to
// `PUT /api/me/last-state` on every route change (~1s debounce so
// a fast click-through doesn't spray the endpoint). Anonymous
// users: no-op. Best-effort: a failed POST never disrupts nav.
//
// 2. On the first authenticated load, read the stored `last_route`
// (handed back on `/api/auth/me`) and `navigate()` to it ONCE.
// Stale routes (a withdrawn RFC, a slug the user lost rights to)
// are not special-cased here: navigating lands on whatever that
// route renders today, and the existing routing already falls
// through to the catalog/empty-state for a missing RFC. Keeping
// this dumb is deliberate (see SPEC §6.2 + the #29 scope note).
//
// Ordering with #21 Part C (Amplitude identify): App.jsx fires
// `identify` in its own effect when `me.user.id` first appears. This
// hook's resume redirect is gated on `identifyReady` — App.jsx flips it
// true only after the identify effect has run — so the redirect always
// happens AFTER identify, preserving the identify-then-track ordering
// the roadmap calls out.
import { useEffect, useRef } from 'react'
import { putLastState } from '../api'
// ~1s debounce on the route-change POST. A frontend constant, not a
// server knob — the backend takes whatever lands.
const DEBOUNCE_MS = 1000
// Routes we never want to resume *to* — auth/landing surfaces that
// would be nonsensical or hostile to drop a returning user onto. We
// still record them (cheap, and the user may legitimately be sitting on
// /docs), but the resume redirect skips them and falls through to the
// default landing. Anything not listed resumes normally.
const NON_RESUMABLE_PREFIXES = ['/login', '/welcome', '/invites/', '/invitations/']
function isResumable(route) {
if (!route || typeof route !== 'string') return false
if (route === '/') return false // "/" is already the default landing
return !NON_RESUMABLE_PREFIXES.some(p => route.startsWith(p))
}
/**
* @param {object} opts
* @param {boolean} opts.authenticated whether a viewer is signed in
* @param {string} opts.pathname current location.pathname
* @param {boolean} opts.identifyReady App flips true after identify fires
* @param {string|null} opts.lastRoute stored route from /api/auth/me
* @param {function} opts.navigate react-router navigate()
*/
export function useLastState({ authenticated, pathname, identifyReady, lastRoute, navigate }) {
// ── 1. Debounced route-change POST ────────────────────────────────
const timerRef = useRef(null)
useEffect(() => {
if (!authenticated) return undefined
if (timerRef.current) clearTimeout(timerRef.current)
timerRef.current = setTimeout(() => {
// Best-effort — swallow failures so an offline/401 POST never
// surfaces as a navigation error.
putLastState(pathname).catch(() => {})
}, DEBOUNCE_MS)
return () => {
if (timerRef.current) clearTimeout(timerRef.current)
}
}, [authenticated, pathname])
// ── 2. One-time resume redirect ───────────────────────────────────
// Fires once, after identify is ready, when there's a resumable
// stored route AND the user is currently sitting on the default
// landing ("/"). We only redirect from "/" so we never yank a user
// who deep-linked somewhere specific (or refreshed mid-RFC) back to
// their last route.
const resumedRef = useRef(false)
useEffect(() => {
if (resumedRef.current) return
if (!authenticated || !identifyReady) return
// Only resume when the app booted on the default landing — a hard
// sign-in nav lands on "/", which is exactly the case we want.
if (pathname !== '/') {
resumedRef.current = true // user deep-linked; don't resume later either
return
}
if (isResumable(lastRoute)) {
resumedRef.current = true
navigate(lastRoute, { replace: true })
} else {
resumedRef.current = true // nothing to resume to; keep default landing
}
}, [authenticated, identifyReady, lastRoute, pathname, navigate])
}
+1
View File
@@ -2,6 +2,7 @@ import React from 'react'
import ReactDOM from 'react-dom/client'
import { BrowserRouter } from 'react-router-dom'
import App from './App.jsx'
import './styles/tokens.css'
import './index.css'
ReactDOM.createRoot(document.getElementById('root')).render(
+162
View File
@@ -0,0 +1,162 @@
/* tokens.css the design-token foundation for rfc-app's UI.
*
* Roadmap item #31 (comprehensive UX polish). Before this file the app
* had ~98 distinct hardcoded hex colors, font sizes scattered across 16
* values with no scale, and radii across 13 values classic prototype
* sprawl. This module establishes ONE coherent system; the App.css sweep
* (and component-scoped CSS) reference these custom properties instead of
* literal values, so "what color/size/space is this" has a single answer.
*
* Imported FIRST in main.jsx so :root is defined before any other sheet.
* Custom properties are not cascade-order-sensitive at use time, but
* importing first keeps the dependency obvious.
*
* Conventions for anyone sweeping values to these tokens:
* - Map each literal to the NEAREST semantic token, then fall back to a
* primitive ramp step. Consolidating near-duplicate grays is the point.
* - Never invent a new literal in a component; add a token here instead.
* - Spacing/radii/type use the scales below no off-scale px values.
*/
:root {
/* ===== Color primitives — neutral ramp ===== */
--c-white: #ffffff;
--c-gray-50: #fafafa;
--c-gray-100: #f3f4f6;
--c-gray-150: #f0f0ee; /* the app's warm canvas tint */
--c-gray-200: #e5e7eb;
--c-gray-300: #d1d5db;
--c-gray-400: #9ca3af;
--c-gray-500: #6b7280;
--c-gray-600: #4b5563;
--c-gray-700: #374151;
--c-gray-800: #1f2937;
--c-gray-900: #111111;
--c-ink: #1a1a1a; /* near-black used for the header + body text */
/* ===== Color primitives — accent (indigo/violet) ===== */
--c-accent: #5b5bd6;
--c-accent-strong: #4338ca;
--c-violet: #7c3aed;
/* ===== Color primitives — status ===== */
--c-success-fg: #166534;
--c-success-bg: #dcfce7;
--c-danger-fg: #991b1b;
--c-danger-fg-strong: #b91c1c;
--c-danger-bg: #fef2f2;
--c-danger-border: #fecaca;
--c-warning-fg: #92400e;
--c-warning-accent: #b45309;
--c-warning-bg: #fef3c7;
--c-warning-bg-soft:#fffbeb;
/* ===== Semantic colors ===== */
--color-bg: var(--c-gray-150);
--color-surface: var(--c-white);
--color-surface-sunken: var(--c-gray-50);
--color-surface-muted: var(--c-gray-100);
--color-header-bg: var(--c-ink);
--color-text: var(--c-ink);
--color-text-strong: var(--c-gray-900);
--color-text-muted: var(--c-gray-500);
--color-text-subtle: var(--c-gray-400);
--color-text-inverse: var(--c-white);
--color-border: var(--c-gray-200);
--color-border-strong: var(--c-gray-300);
--color-link: var(--c-accent);
--color-accent: var(--c-accent);
--color-accent-strong: var(--c-accent-strong);
--color-accent-contrast: var(--c-white);
--color-success-fg: var(--c-success-fg);
--color-success-bg: var(--c-success-bg);
--color-danger-fg: var(--c-danger-fg);
--color-danger-bg: var(--c-danger-bg);
--color-warning-fg: var(--c-warning-fg);
--color-warning-bg: var(--c-warning-bg);
/* On the dark header, translucent white is the established pattern. */
--color-on-dark-soft: rgba(255, 255, 255, 0.15);
--color-on-dark-hover: rgba(255, 255, 255, 0.25);
--color-on-dark-muted: #dddddd;
--color-focus-ring: rgba(91, 91, 214, 0.45);
/* ===== Type ===== */
--font-sans: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
Helvetica, Arial, sans-serif;
--font-mono: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas,
monospace;
--text-2xs: 10px;
--text-xs: 11px;
--text-sm: 12px;
--text-base: 13px; /* the app's dominant body size */
--text-md: 14px;
--text-lg: 16px;
--text-xl: 18px;
--text-2xl: 22px;
--text-3xl: 28px;
--leading-tight: 1.25;
--leading-normal: 1.5;
--leading-relaxed: 1.65;
--weight-normal: 400;
--weight-medium: 500;
--weight-semibold: 600;
--weight-bold: 700;
/* ===== Spacing scale (4-based, with the 2/6/10 half-steps the app
* already leans on heavily) ===== */
--space-0: 0;
--space-1: 2px;
--space-2: 4px;
--space-3: 6px;
--space-4: 8px;
--space-5: 10px;
--space-6: 12px;
--space-7: 16px;
--space-8: 20px;
--space-9: 24px;
--space-10: 32px;
--space-11: 48px;
--space-12: 64px;
/* ===== Radius ===== */
--radius-xs: 2px;
--radius-sm: 4px;
--radius-md: 6px;
--radius-lg: 8px;
--radius-xl: 12px;
--radius-pill: 999px;
/* ===== Elevation ===== */
--shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.06);
--shadow-md: 0 2px 8px rgba(0, 0, 0, 0.08);
--shadow-lg: 0 8px 24px rgba(0, 0, 0, 0.12);
/* ===== Motion ===== */
--motion-fast: 120ms;
--motion-base: 150ms;
--motion-slow: 200ms;
--ease-out: cubic-bezier(0.16, 1, 0.3, 1);
--ease-in-out: cubic-bezier(0.4, 0, 0.2, 1);
/* ===== Layout ===== */
--header-height: 48px;
}
/* Honor reduced-motion globally any transition/animation that reads
* these duration tokens collapses to instant. */
@media (prefers-reduced-motion: reduce) {
:root {
--motion-fast: 0ms;
--motion-base: 0ms;
--motion-slow: 0ms;
}
}